Page MenuHomeFreeBSD

Update virtualbox to 6.1.18 and repocopy version 5 to -legacy ports.
Needs ReviewPublic

Authored by madpilot on Mon, Feb 22, 5:58 PM.

Details

Reviewers
lwhsu
jkim
Group Reviewers
vbox
Summary

This comes from pr 234878 [1] on bugzilla, many people contributed patches there to create this update, my work has mainly been with cleaning up the patches and integrating in present ports tree, so credit is due to the various contributors there (jkim, Mario Lobo, lwhsu contributed patches, many other testing and suggestions)

I added patches to make additions 6.1.18 work. The module gave me a bcmp undefined symbol error, the fix I used comes from a post on the mailin lists [2]

Patch also includes a tentative entry for UPDATING.

Some notes:

  • At present this compiles only on amd64, and while it produces a package on 11 and 12 it fails to work there.
  • The patch includes repocopies of the version 5 ports as legacy versions. This will ensure some path for users who can't run the latest hardware/software. BTW these legacy ports could be updated to 6.0 later, AFAIK 6.0 has the same requirements as 5.x.
  • I regenerated most of the patches to have them fresh

[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234878

[2] https://lists.freebsd.org/pipermail/freebsd-virtualization/2020-May/008448.html

Test Plan

I'm running this on head without any problems. It actually works smoother than old version, especially for graphic output.

I've tested it in poudriere myself, and plan to also test non default options better (performed only some quick test there)

User reports are still mixed, but it should work fine on head and 13.

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 37340
Build 34229: arc lint + arc unit

Event Timeline

Some personal thoughts:

Ideally this should be committed before the next quarterly for two reason, obviously to get in quarterly, and also because I'd rather commit this work I did in subversion without need to rework it through git [1].

At the same time, unless it can be made to work on 12.2 at least, it makes little sense to commit this before 13.0 is released, so the ideal window for committing this amounts to a few days at the end of March at most.

[1] this is no critique of git, in fact this activity would have been easier with git, but having done all the patching in subversion moving it to git would be extra work

I was able to build, install, run, create/run VMs with this patch on 12.2-RELEASE-p3 host. Should the ignore 12 lines be removed?

emulators/virtualbox-ose-kmod/Makefile
27

What's the exact reason for ignoring 12 here?

With this patch and a 12.2-RELEASE-p3 amd64 host and poudriere I'm able to build, install, create, and run a freebsd 12 amd64 and Windows 10 64 VM a 12.2-RELEASE-p3 host.

madpilot marked an inline comment as done.

I was able to build, install, run, create/run VMs with this patch on 12.2-RELEASE-p3 host. Should the ignore 12 lines be removed?

If it works yes. Tomorrow I'll test myself, but you report looks promising.

emulators/virtualbox-ose-kmod/Makefile
27

This is here because people reported it failing on 12 and I have been unable to test it until now.

I am reinstalling an old laptop with FreeBSD 12.2 to test just this. I plan to test it tomorrow myself.

You report is good news though. I'll do my own testing anyway, just in case, I'll be quite happy to remove this line if at all possible.

Further note:

6.1.18 additions fail to compile on i386 with a strange gmake "missing separator" error. I've been unable to fix it. Any help appreciated!

I was able to build, install, run, create/run VMs with this patch on 12.2-RELEASE-p3 host. Should the ignore 12 lines be removed?

If it works yes. Tomorrow I'll test myself, but you report looks promising.

I see where there might be an issue with 12.2-RELEASE-p3 and clang.

I was building with DEBUG and it works. Rebuild with default options (DEBUG off) you get the same error reported here https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234878#c134 while trying to start a VM.
It seems to either be an issue with the code or the way clang (10?) is building it?

So it's possible DEBUG (for the being) might be required for 12.2?

I was able to build, install, run, create/run VMs with this patch on 12.2-RELEASE-p3 host. Should the ignore 12 lines be removed?

If it works yes. Tomorrow I'll test myself, but you report looks promising.

I see where there might be an issue with 12.2-RELEASE-p3 and clang.

I was building with DEBUG and it works. Rebuild with default options (DEBUG off) you get the same error reported here https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234878#c134 while trying to start a VM.
It seems to either be an issue with the code or the way clang (10?) is building it?

Interesting finding. Ideally we should understand why this happens. Maybe forcing another compiler for 12.2 would make it work?

Also, what are you compiling with DEBUG? Only the kmod is enough or is it required for all of virtualbox-ose?

So it's possible DEBUG (for the being) might be required for 12.2?

Could be an option, but I'd like to try forcing clang 11 or some other compiler before going down that way.

Maybe negligible for review purposes:

  • the application icons that appear, temporarily, whilst a guest starts are generic X.org icons.

Interesting finding. Ideally we should understand why this happens. Maybe forcing another compiler for 12.2 would make it work?

Yeah, finding the root cause would be best however if needed forcing gcc would be next.

Also, what are you compiling with DEBUG? Only the kmod is enough or is it required for all of virtualbox-ose?

DEBUG was only enabled on emulators/virtualbox-ose

Interesting finding. Ideally we should understand why this happens. Maybe forcing another compiler for 12.2 would make it work?

Yeah, finding the root cause would be best however if needed forcing gcc would be next.

Also, what are you compiling with DEBUG? Only the kmod is enough or is it required for all of virtualbox-ose?

DEBUG was only enabled on emulators/virtualbox-ose

I tested now virtualbox 6 modified to force clang 11 from ports on 12.2 and it works fine. I think this fix is cleaner than forcing DEBUG.

I'm almost sure this would work for 11.x too, but it's difficult to test because I'd have to build a full set of ports using SSL from ports, due to the QT/OpenSSL issue. Maybe I'll test that too, anyway I'll remove the restriction on 11 along with the one on 12.

This removes the need to wait for 13.0 to be released before committing.

Updated patch set coming shortly.

Maybe negligible for review purposes:

  • the application icons that appear, temporarily, whilst a guest starts are generic X.org icons.

Good catch, although I suspect this is due to the way virtualbox creates its windows and not much can be done. Anyway if it's only limited to aesthetics a fix can also be added at a later time.

Improved patch:

  • Make virtualbox-ose-legacy depend on virtualbox-ose-kmod-legacy.
  • On FreeBSD <13 force using clang 11 from ports to build. Thiss makes it work correctly there.
  • Change wording in UPDATING accordingly and add note about inability to move saved states accross the update.

I tested now virtualbox 6 modified to force clang 11 from ports on 12.2 and it works fine. I think this fix is cleaner than forcing DEBUG.

I'm almost sure this would work for 11.x too, but it's difficult to test because I'd have to build a full set of ports using SSL from ports, due to the QT/OpenSSL issue. Maybe I'll test that too, anyway I'll remove the restriction on 11 along with the one on 12.

The issue I could see with using llvm11 is that building on a fresh env you need gcc10 (kbuild), llvm10 (mesalibs), and if added llvm11 to build virtualbox-ose. That's three very large ports. If you just used gcc (assuming you can reuse gcc10) it's already there.

Is it any different you force virtualbox-ose to use lang/llvm10 vs base's? If no different then using gcc10 (or whatever kbuild uses) would require less to build.

The issue I could see with using llvm11 is that building on a fresh env you need gcc10 (kbuild), llvm10 (mesalibs), and if added llvm11 to build virtualbox-ose. That's three very large ports. If you just used gcc (assuming you can reuse gcc10) it's already there.

Is it any different you force virtualbox-ose to use lang/llvm10 vs base's? If no different then using gcc10 (or whatever kbuild uses) would require less to build.

Yes, you make a compelling point. Honestly I did not think about it.

I need to spin a new round of tests before replying. Should not take long though, since the dependencies should all be present in the poudriere jail I used.

EDIT: At the same time I'm checking the "CHOSEN_COMPILER_TYPE" variable to know what to do...Having that on clang and then forcing gcc looks strange. Not sure if there is a way to make USES=compiler choose gcc himself but only depending on OSVERSION

I coerced it in using gcc, but it's failing with a bunch of errors like this:

-- SUPDrvSem.o ---
gcc10: error: unrecognized command-line option '-fformat-extensions'; did you mean '-fno-ms-extensions'?
*** [SUPDrvSem.o] Error code 1

First, I cannot find any documentation for the flag -fformat-extensions, I'd rather know what it does before removing it, maybe it should be replaced by some gcc specific one. Anyone has any clue?

Anyway I have no idea how to remove it, since grepping the whole virtualbox sources only returns a match in Config.kmk, but in a comment, so, right now, I have no idea where that flag is growing from. I'll keep looking.

EDIT:

https://stackoverflow.com/questions/48927926/cc-error-unknown-argument-fformat-extensions

Interesting enough, this flag used to be an extension from our own gcc "fork", unsupported in newer gcc. But I could try forcing gcc only on main virtualbox port. Could work anyway.

Forcing gcc 10 gives errors both in kmod and in main port. I'd say it's a non starter.

Main port error with gcc 10:

kBuild: Compiling iPxeBaseBin - /wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/src/VBox/Devices/PC/ipxe/src/arch/i386/core/basemem_packet.c
gcc10: error: unrecognized command-line option '-mstack-alignment=2'
kmk: *** [/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/out/freebsd.amd64/release/obj/iPxeBaseBin/src/arch/i386/core/basemem_packet.o] Error 1
The failing command:
@gcc10 -m32 -c -O2 -fno-pie -fcommon -ffreestanding -ffunction-sections -march=i386 -fomit-frame-pointer -fstrength-reduce -falign-jumps=1 -falign-loops=1 -Os -falign-functions=1 -mstack-alignment=2 -mregparm=3 -mrtd -freg-struct-return -fshort-wchar -Ui386 -Ulinux -include compiler.h -Wall -W -Wformat-nonliteral -fno-stack-protector -fno-dwarf2-cfi-asm -Wno-address -I/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/src/VBox/Devices/PC/ipxe/src -I/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/src/VBox/Devices/PC/ipxe/src/include -I/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/src/VBox/Devices/PC/ipxe/src/arch/x86/include -I/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/src/VBox/Devices/PC/ipxe/src/arch/i386/include -I/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/src/VBox/Devices/PC/ipxe/src/arch/i386/include/pcbios -I/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/include -I/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/out/freebsd.amd64/release -DVBOX -DVBOX_OSE -DVBOX_WITH_64_BITS_GUESTS -DRT_OS_FREEBSD -D__FREEBSD__ -DRT_ARCH_AMD64 -D__AMD64__ -DVBOX_WITH_DEBUGGER -DVBOX_WITH_HARDENING -DRTPATH_APP_PRIVATE=\"/usr/local/share/virtualbox-ose\" -DRTPATH_APP_PRIVATE_ARCH=\"/usr/local/lib/virtualbox\" -DRTPATH_SHARED_LIBS=\"/usr/local/lib/virtualbox\" -DRTPATH_APP_DOCS=\"/usr/local/share/doc/virtualbox-ose\" -DARCH=i386 -DPLATFORM=pcbios -DVERSION_MAJOR=1 -DVERSION_MINOR=0 -DVERSION_PATCH=0 -DVERSION=\"1.0.0+\" -D_build_id=0xaffeaffe -DOBJECT=basemem_packet -Wp,-MD,/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/out/freebsd.amd64/release/obj/iPxeBaseBin/src/arch/i386/core/basemem_packet.o.dep -Wp,-MT,/wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/out/freebsd.amd64/release/obj/iPxeBaseBin/src/arch/i386/core/basemem_packet.o -Wp,-MP -o /wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/out/freebsd.amd64/release/obj/iPxeBaseBin/src/arch/i386/core/basemem_packet.o /wrkdirs/usr/ports/emulators/virtualbox-ose-nox11/work/VirtualBox-6.1.18/src/VBox/Devices/PC/ipxe/src/arch/i386/core/basemem_packet.c

I've made a test on 12.2 using base compiler and it fails to start VMs as expected.

For me forcing clang 11 is the only solution working across the board.

I've also got a negative report forcing clang11, and one positive using base clang (also on 12.2).

At this point it's difficult for me to tell what should be done, due to the contrasting reports.

My gut feeling is that going with forcing claang11(the present patch set here) is the safest option at present.

We need to make it work and get it into the tree. Things can be improved later anyway.

Updating the patch to improve some details.

First of all the kmod can be built using base clang on 12.2.

Also cleaned up the code to choose compiler in the main port a little.

I'm still forcing clang11 on 12.2 since this seems to work fine in my tests.

I am not a member of the vbox group, so I still need approval to commit this. I'm here asking for such approval. Obviously I will do my best to address any change request if it's in my ability.