This is the package you would file tickets against/submit config patches for if you want the work on a minimal kernel to go upstream:
Ideally it’s a matter of figuring out what modules our virtual hardware configurations use and let them know to include it in their next builds. For example shared folders aren’t even recognized because they don;t even consider this usecase for a kernel running in a public cloud so your contributions would be about making this kernel friendly towards desktop users.
VirtualBox modules aren’t even part of it AFAICT, but there’s no reason why they can’t accept it.
On the otherhand the kernel/system bootup is a real speed demon.