Page MenuHomeFreeBSD

emulators/wine-devel: add WOW64
Needs RevisionPublic

Authored by dbn on Mar 17 2018, 2:11 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, Dec 21, 4:27 PM
Unknown Object (File)
Sat, Dec 21, 4:26 PM
Unknown Object (File)
Thu, Dec 12, 11:52 PM
Unknown Object (File)
Sun, Dec 8, 2:23 AM
Unknown Object (File)
Sat, Nov 30, 5:20 PM
Unknown Object (File)
Fri, Nov 29, 7:59 AM
Unknown Object (File)
Thu, Nov 28, 9:46 AM
Unknown Object (File)
Nov 25 2024, 2:20 PM

Details

Reviewers
mat
gerald
Summary

ChangeLog:

  • Add option WOW64 to wine-devel
  • Script the creation of valid pkg-plist for i386, amd64 and amd64+wow64
  • Remove i386-wine-devel (now superseded by wine-devel)
  • Remove all references to STAGING

Approach:
Since FreeBSD does not support cross compiling ports, and given the wow64 build requirements of wine, the following approach was required. Note that the wine-devel port (when using the WOW64 option) operates in a "meta" mode and calls itself in different states to create a wow64 build:

  • Create a i386 chroot (in /tmp/i386, to allow mounting of directories without overlapping paths)
  • In the i386 chroot compile a clean win32 wine-devel build (the dependencies are compiled in the configure step)
  • In the amd64 environment compile a clean win64 wine-devel build
  • In the i386 chroot compile a combined win32 wine-devel build referencing the win64 build and the clean win32 (this depends on the previous two steps)
  • Install the combined win32, and then the clean win64, wine-devel builds (this order is important)
NOTE: this requires root to build (NEED_ROOT) due to the need to use a chroot. The chroot is required due to the need for building the win64 before the combined win32 wine. Thus, one cannot use a pre-compiled approach for the i386 portion.

Despite the downside of needing to use NEED_ROOT (a DEV_ERROR) this does have many benefits:

  • Users can now easily compile wow64 themselves (with custom patches and options across the entire ports tree)
  • The binaries no longer come from a 3rd party (i.e. increased trust of the results).

How to update the port:

  • Update the version per normal
  • Copy the latest snapshot of 12-Current to LOCAL/${USER}/base12.yyyyMMdd.txz
  • Update OSREL12 value to reflect latest snapshot date in Makefile.wow64
  • make makesum
  • Compile on amd64 with WOW64 option
  • make plist
  • Check pkg-plist and add (WINE32|WINE64|MPG123|OPENAL|X11)_FILE entries as needed
Test Plan

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Lint Skipped
Unit
Tests Skipped

Event Timeline

dbn edited the test plan for this revision. (Show Details)
  • actually delete i386-wine-devel
  • pat portlint
gerald requested changes to this revision.Mar 17 2018, 8:10 PM

Hi David,

Wine Staging is back (at least for now), so please revert the changes/simplifications related to that.
(I had left that code and provisions in awaiting upstream development, which has now materialized.)

All changes to pkg-plist that merely improve sorting are fine; please go ahead and commit those now,
i.e., independently of us working on the other changes.

Can we minimize changes to wine-devel/Makefile? For example, do we need all the "OPTION"_FILES
variables there, or could they to into the specific Makefile?

Finally, given the complexity of this and your intimate knowledge, do you think we can keep this as
slave port instead of making it an option of wine-devel? (I see value in building such a package by
default anyway like in the past (or wine-staging earlier), so mostly a question of where to "host"
the specific bits?

This revision now requires changes to proceed.Mar 17 2018, 8:10 PM

The changes mentioned above will be updated once I have fixed the building issue with poudriere (something about detecting amd64 in the chroot).

Hi David,

Wine Staging is back (at least for now), so please revert the changes/simplifications related to that.
(I had left that code and provisions in awaiting upstream development, which has now materialized.)

I was more pessimistic than you and assumed it will not be back for quite a while. I have restored the required parts.

All changes to pkg-plist that merely improve sorting are fine; please go ahead and commit those now,
i.e., independently of us working on the other changes.

I also noticed some other issues:

  • STAGING is missing description
  • MASTER_SITES is missing the DEFAULT tag (error produced if DEVELOPER=yes is set)

I can commit all these fixes for you?

Can we minimize changes to wine-devel/Makefile? For example, do we need all the "OPTION"_FILES
variables there, or could they to into the specific Makefile?

I only left them in Makefile so that they are obvious, I've moved them into Makefile.wow64 (just above the plist target) and added staging support

Finally, given the complexity of this and your intimate knowledge, do you think we can keep this as
slave port instead of making it an option of wine-devel? (I see value in building such a package by
default anyway like in the past (or wine-staging earlier), so mostly a question of where to "host"
the specific bits?

I would prefer to have this in the wine-devel port and have the option on by default (at a future point in time). This way we offer the user the most transparent and simple experience: install one of 3 ports (wine, wine-devel or wine-staging) depending on stability needs. No need for the user to make the choice between 6 ports.

I think the only need to have a separate port is if WOW64 interferes with another port - and I don't think it will. For example, pipelight needs the staging port, and thus having a separate port makes sense (or, maybe better to have a separate flavor?).

dbn retitled this revision from emulators/wine-devel: add WOW65 to emulators/wine-devel: add WOW64.
dbn edited the test plan for this revision. (Show Details)

Finally I got WoW64 building nicely in poudriere.

Upon reflection, I think I should remove the plist target in Makefile.wow64. As long as one has a valid pkg-plist for 32bit and 64bit Wine then WoW64 works :-)

Changes:

  • fix integration with STAGING
  • hande missing /etc/make.conf
  • fix build in poudriere environment
  • get a shell in the inner make environment
  • restore STAGING, move files to wow64
  • add back STAGING description
  • add default group to MASTER_SITES

oooh this is awesome!

Some notes about trying to build it (without poudriere, directly on a regular 12-current machine):

  • Why not use binary packages in the chroot? I added copying /etc/resolv.conf in chroot and running pkg install in pre-configure and it works (well, the pkg stub interactively asks y/N install pkg, but there's pkg-static or whatever.)
  • The .i386 plist isn't cleared when trying to restage, it's easy to end up with duplicate plist entries if the staging was interrupted. (e.g. my first staging failed because patch didn't add binbounce and I forgot to copy it manually.)
  • Related: restage doesn't restage the chrooted builds.
  • My package did not include some lib32 libs: png, gnutls, freetype, and the dependencies of gnutls. But it did include all of the other wine dependencies. (Possibly related to my usage of binary packages or the restaging fails — did not happen when I tried to apply the previous version of this patch onto the 3.4 port a couple days ago.)

Also, 64-bit wine was broken with clang/lld:

wine: created the configuration directory '/home/greg/.wine'
0009:err:process:start_wineboot failed to start wineboot, err 1359
0009:fixme:actctx:parse_depend_manifests Could not find dependent assembly L"Microsoft.Windows.Common-Controls" (6.0.0.0)
0009:err:winecfg:WinMain failed to restart 64-bit L"C:\\windows\\system32\\winecfg.exe", err 1359
0009:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
0009:err:winediag:nodrv_CreateWindow The explorer process failed to start.

LLD_UNSAFE and USE_GCC fixed it. Haven't yet tested with only one of them.

In D14721#312782, @greg_unrelenting.technology wrote:

oooh this is awesome!

Thanks :-P

Some notes about trying to build it (without poudriere, directly on a regular 12-current machine):

  • Why not use binary packages in the chroot? I added copying /etc/resolv.conf in chroot and running pkg install in pre-configure and it works (well, the pkg stub interactively asks y/N install pkg, but there's pkg-static or whatever.)

That is a viable option (and maybe an option), however it does break the build philosophy for ports. Namely, an Internet connection can only be used in the fetch phase. Poudriere actively blocks Internet connections during the configure/build phases. Obviously, a user can easily override this if they want.

Another reason is compiling the i386 bits with the options and patches a user has selected.

  • The .i386 plist isn't cleared when trying to restage, it's easy to end up with duplicate plist entries if the staging was interrupted. (e.g. my first staging failed because patch didn't add binbounce and I forgot to copy it manually.)
  • Related: restage doesn't restage the chrooted builds.

This is caused by the chrooted build (it only stages wine32-combo) tracking its staging with a separate cookie. If this was removed then both the .i386 plist and the 32bits will be restaged properly.

  • My package did not include some lib32 libs: png, gnutls, freetype, and the dependencies of gnutls. But it did include all of the other wine dependencies. (Possibly related to my usage of binary packages or the restaging fails — did not happen when I tried to apply the previous version of this patch onto the 3.4 port a couple days ago.)

I'll investigate this, we are using the same logic as the i386-wine ports to include the 32-bit libraries. But, maybe I broke something in the cleanup.

Also, 64-bit wine was broken with clang/lld:

wine: created the configuration directory '/home/greg/.wine'
0009:err:process:start_wineboot failed to start wineboot, err 1359
0009:fixme:actctx:parse_depend_manifests Could not find dependent assembly L"Microsoft.Windows.Common-Controls" (6.0.0.0)
0009:err:winecfg:WinMain failed to restart 64-bit L"C:\\windows\\system32\\winecfg.exe", err 1359
0009:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
0009:err:winediag:nodrv_CreateWindow The explorer process failed to start.

LLD_UNSAFE and USE_GCC fixed it. Haven't yet tested with only one of them.

I think this is unrelated to this patch. Can you please try compiling without WOW64 and confirm? (I run 11.1 at home so it isn't easy for me to test these things.)

Please go ahead and apply the changes to emulators/wine-devel/pkg-plist (with the understanding that they change the sorting of a few entries only).

Similarly for emulators/wine itself, to keep the two in sync wrt. that, even if that was not part of your patchset. :-)

Approved-by: Gerald (maintainer), and no need to bump PORTREVISION.

gerald requested changes to this revision.Apr 2 2018, 2:24 PM

Please minimize non-functional changes (such as shifting contents in Makefile).

(One of those was the moving of the workaround for a clang bug; I'll take that out shortly, please make sure to yank it from your patchset.)

And avoid formatting changes such as the one for ​check-wine-devel-vs-wine-staging in wine-devel/Makefile.

I suggest that, at least initially, not to enable WOW by default. What about keeping a child part similar
to how wine-staging used to work in the past?

The restoration of STAGING_DESC is a bug fix, please go ahead and apply that at any time (but independently of the bigger patch set).

This revision now requires changes to proceed.Apr 2 2018, 2:24 PM

And some questions:

(*) ​.if !defined(ARCH)
​ARCH!= uname -p

Why would ARCH ever not be defined? Presumably some tricky circumstances? Can you please either remove that bit or add a comment describing what's going on?

(*) I do not get complaints about :DEFAULT, and things work just fine without this change, so let's omit this?

(*) Shall we use 32-bit and 64-bit (generally)?

(*) Why did you add the amd64 clause to "​.if ${ARCH} == amd64 && ${CHOSEN_COMPILER_TYPE} == clang && ${COMPILER_VERSION} < 38" ? Is it strictly necessary? If so, why (and can you please document in that case)?

(*) Similarly, ​post-extract-WOW64-off: fix-shebang? Why not keep post-extract as it was?

(*) Why "​.if ${PORT_OPTIONS:MWOW64} || defined(WINE_WOW64)", that is, both conditions? Is there a way to only use one?

(*) Why the ${WINE_METAMODE} now in ​pre-build${WINE_METAMODE}?

The primary reason I am asking all these questions is that all this appears to make the port more complex and minimizing complexity unless really necessary has generally proven a good idea.

Could you use devel/arcanist, or at least generate a diff with full context like it does, with svn diff -x -U9999 or git diff -U9999.

emulators/wine-devel/Makefile
8–9

No need to add :DEFAULT, it is the default.

112–114

This is a big no.
Put back the .include, and never manually fill in a variable generated by the framework.

123–124
OPTIONS_DEFINE_amd64= WOW64
OPTIONS_DEFAULT_amd64= WOW64

Side note.

All ports MUST build as a regular user.

We spent a lot of time making sure they all do, DO NOT add new ports that require being root, they will get removed at one point in the very near future.

In D14721#348739, @mat wrote:

Side note.

All ports MUST build as a regular user.

We spent a lot of time making sure they all do, DO NOT add new ports that require being root, they will get removed at one point in the very near future.

Unfortunately this port requires root due to the use of chroot(8). To explain why I want this change in ports (as opposed to the status quo) and why chroot(8) is required:

  1. Currently the i386-wine ports repackage binary ports compiled by myself and hosted using FreeBSD's public_distfiles infrastructure
  2. The problems I have with point (1) are:
    1. The means of a user producing their own i386-wine port (with, say, their own patches) are non-obvious and cumbersome.
    2. There is a security risk that I could maliciously (or unintentionally) upload a compromised binary (which, by being in binary form, a lot harder to detect/review)
    3. There is an implicit dependency on me to keep the i386-wine ports updated - something I have neglected for months.
    4. Creating WOW64 (Windows [32-bit] on Windows 64[-bit]) is not easy under this approach
  3. Thus, based on point (2) I feel this change adds value by:
    1. Automating the update of the i386-wine ports
    2. Making it easy for users to build i386-wine ports themselves (with local patches, if needed)
    3. Allows WoW64 to be shipped
  4. In order to build either i386-wine or WoW64, all or some of Wine needs to be built targeting i386. FreeBSD base (and ports) does not support cross building, and thus chroot(8) is required.

I feel the benefits of (3) warrant an exception for Wine, unless an alternative could be found for using chroot(8). But, I cannot think of anything that does not require changes to the port infrastructure (for it to supply the functionality requiring root).

@mat: your views on this are very welcome.

Wow, this would be super useful, as more and more win64 apps are released. Is it possible to have a patchset generated against a more recent version of the port tree/wine?

I feel a bit rude registering just to make a drive-by comment… Anyway, here are 3 semi-related points:

  1. Wine wiki is quite misleading with regards to the chroot wow64 build. The only difference between "clean" wine32 build and "combined" wine32 build (--with-wine64=../wine64-build --with-wine-tools=../wine32-tools) is that the latter skips building 32-bit wineserver and symlinks wine loader into the wine64 build's directory. Saving time by skipping parts of the build isn't very useful when you already have a full wine32 build and the symlink is just a convenience. There is literally nothing else going on, and thus "combined" build is strictly redundant and can be safely omitted.
  1. If the "combined" build doesn't build wow64 then what does? Turns out there aren't any wow64-specific build flags. It is implemented as an ability of the 64-bit wineserver to talk to the applications using 32-bit ntdll.dll.so (kernel32.dll.so, user32.dll.so, etc) through wineserver's RPC protocol and it is always enabled on both sides. In fact, for the last nine months I'm using wow64 wine on FreeBSD by building i386-wine-devel and wine-devel ports without any modifications, installing the former, unpacking the latter to /opt and adding its bin directory to $PATH. Then I just have to start the 64-bit wineserver (e.g., /opt/wine-staging-3.11/usr/local/bin/wineserver --foreground --persistent) and that's it.
  1. Was there ever any discussion of introducing lib32- packages in Arch Linux style? Basically, one additional slave port per wine dependency with -m32 flag, /usr/local/lib32 lib install path and a customized plist. This is obviously a fair amount of work but it doesn't require chroot or infrastructure improvements.
In D14721#348739, @mat wrote:

Side note.

All ports MUST build as a regular user.

We spent a lot of time making sure they all do, DO NOT add new ports that require being root, they will get removed at one point in the very near future.

A 32-bit Wine requiring root to build from source would not be any worse than the current situation that it cannot be built at all without manually setting up an i386 jail or separate build machine, and would be an improvement for some users who find it acceptable to build ports locally as root. However, the need to have packages buildable by a non-root user is not addressed.

  1. Was there ever any discussion of introducing lib32- packages in Arch Linux style? Basically, one additional slave port per wine dependency with -m32 flag, /usr/local/lib32 lib install path and a customized plist. This is obviously a fair amount of work but it doesn't require chroot or infrastructure improvements.

I propose a different possibility for consideration: Build Wine's 32-bit dependencies by invoking a sub-make which has had ARCH, PREFIX, LOCALBASE, WRKDIRPREFIX, and other needed variables and environment overridden to force the compilation of i386 libs and exes despite the build system being an amd64+lib32 installation. I named by PREFIX /usr/local/i386.

I have attempted this approach, and I have found that it is working. I can run successfully a non-free i386 Windows exe using only locally-built i386 packages, where no chroot or jail was involved. I have not yet tested WoW64 as I have been delayed by the problems with wine64 on -CURRENT which have been referenced earlier.

One detail of note is that the i386 dependency packages are installed with i386- prepended to the package names. However, there is no need to create a new slave port for each dependency. The only changes needed to existing ports are corrections to failures to respect LOCALBASE, so I consider these to be ports bugs which are unrelated to (merely exposed by) this i386 experiment. Only one new port is involved; I have named it i386-wine-src to distinguish it from the existing. It is a slave to emulators/wine.

Getting this to work involves some ugly bits, for which I would create a new review if I hear any interest in this approach.

I propose a different possibility for consideration: Build Wine's 32-bit dependencies by invoking a sub-make which has had ARCH, PREFIX, LOCALBASE, WRKDIRPREFIX, and other needed variables and environment overridden to force the compilation of i386 libs and exes despite the build system being an amd64+lib32 installation. I named by PREFIX /usr/local/i386.

Hm, so building the "Arch style lib32" packages on-the-fly? That's interesting, but I'm not sure how that would work with official packages. Will poudriere add them to the built package set?

In D14721#357408, @greg_unrelenting.technology wrote:

Hm, so building the "Arch style lib32" packages on-the-fly? That's interesting, but I'm not sure how that would work with official packages. Will poudriere add them to the built package set?

Perhaps FLAVORS can serve as a precedent in this case? I am realizing that to resolve the need for i386-on-amd64 packages properly would require much cooperation, making this a large undertaking, as it was for the adoption of FLAVORS.

In the interest of improving Wine experience on FreeBSD for all, it is probably better to avoid such potentially controversial changes and to keep to the model of distributing all i386 Wine libs as a single package, despite the many drawbacks.

David's original proposal might be improved to comply with the "no root" requirement by replacing the chroot dependence with a "fake chroot", i.e. intercept file paths in user-space rather than in kernel-space. The "fake chroot" has none of the security implications of a real chroot, but nonetheless may serve a similar purpose within a package building process, provided that all executables involved are dynamically linked.

I am realizing that to resolve the need for i386-on-amd64 packages properly would require much cooperation, making this a large undertaking, as it was for the adoption of FLAVORS.
In the interest of improving Wine experience on FreeBSD for all, it is probably better to avoid such potentially controversial changes and to keep to the model of distributing all i386 Wine libs as a single package, despite the many drawbacks.

That entirely depends on your definition of the word 'proper'. If you are thinking about Debian style multilib packages, then yes, that would be an overkill.

David's original proposal might be improved to comply with the "no root" requirement by replacing the chroot dependence with a "fake chroot", i.e. intercept file paths in user-space rather than in kernel-space. The "fake chroot" has none of the security implications of a real chroot, but nonetheless may serve a similar purpose within a package building process, provided that all executables involved are dynamically linked.

The 'real' chroot utility has user and group arguments. The path of the least resistance would be writing a suid wrapper handling fs mounts and an actual chroot invocation with immediate switch to a regular user. Then it can placed in it's own port. Would that be acceptable?

Thank you all for the useful ideas. I think the first price approach is to create 'lib32-' companion packages for the 32-bit libraries we need. To this end I have created a patch that adds this functionality: https://reviews.freebsd.org/D16830. Luckily the approach is minimally invasive and often involves:

  1. Adding lib32 to USES
  2. Sometimes replacing uses of ${PREFIX}/lib} with ${LIBDIR}, however lib32 already handles CFLAGS, CPPFLAGS, and CONFIGURE_ARGS (this can be extended as needed)
  3. Sometimes adding USE_LDCONFIG=yes

If anyone has some time to convent and test the other library dependencies of Wine that will be a great help :-).

Cool!!

But:

  1. unmount does not work on errors - and I lost mine: /usr/ports, /var/db/ports, distfiles. DOH!
  1. /etc/resolv.conf needed in chroot to able to download.
  1. mine /etc/make.conf contain -mretpoline compiler flags, this dont work in chroot.
  1. ccache not used in chroot :(
  1. staging download broken if wow enabled

This fixes for 2 and 3:

if [ -f /etc/make.conf ]; then
	${dp_CP} /etc/make.conf ${dp_CHROOT_DIR}/etc/
	sed -i.orig -e 's|-mretpoline||g' ${dp_CHROOT_DIR}/etc/make.conf
fi
${dp_CP} /etc/resolv.conf ${dp_CHROOT_DIR}/etc/

This fix 5:

DISTFILES+=	${EXTRACT_ONLY} ${BASE_FILE}:${OSREL:C/\..+//}

(missed plus)

Got:

Wine cannot find the FreeType font library.  To enable Wine to
use TrueType fonts please install a version of FreeType greater than
or equal to 2.0.5.
http://www.freetype.org

000d:err:wincodecs:PngEncoder_CreateInstance Failed writing PNG because unable to find libpng16.so.16
000d:fixme:ole:CoCreateInstanceEx no instance created for interface {00000103-a8f2-4877-ba0a-fd2b6645fb94} of class {27949969-876a-41d7-9447-568f6a35a4dc}, hres is 0x80004005
000d:err:menubuilder:convert_to_native_icon error 0x80004005 creating bitmap encoder

looks like it need x32 libs from ports.

http://www.netlab.linkpc.net/download/tmp/wine-devel_wow.patch
I update patch:

  • now it for 3.15
  • use ccache to build
  • nullfs mount as ro for all things that do not required write
  • add copy of /etc/resolv.conf to chroot
  • add copy of /etc/make.conf to chroot
  • fix staging download if wow enabled

Not done:

  1. At least MPG123 does not work with WoW:
===>   Registering installation for wine-devel-3.15,1
pkg-static: Unable to access file /tmp/ports/usr/ports/emulators/wine-devel/work/stage/usr/local/lib32/wine/fakedlls/l3codeca.acm:No such file or directory
pkg-static: Unable to access file /tmp/ports/usr/ports/emulators/wine-devel/work/stage/usr/local/lib32/wine/l3codeca.acm.so:No such file or directory
*** Error code 74
  1. Shared libraries does not copy with wine32 staff from chroot or does not install - dont dig this:
Wine cannot find the FreeType font library.  To enable Wine to
use TrueType fonts please install a version of FreeType greater than
or equal to 2.0.5.
http://www.freetype.org

000d:err:wincodecs:PngEncoder_CreateInstance Failed writing PNG because unable to find libpng16.so.16
000d:fixme:ole:CoCreateInstanceEx no instance created for interface {00000103-a8f2-4877-ba0a-fd2b6645fb94} of class {27949969-876a-41d7-9447-568f6a35a4dc}, hres is 0x80004005
000d:err:menubuilder:convert_to_native_icon error 0x80004005 creating bitmap encoder

@rozhuk.im-gmail.com You know the chroot approach is basically unmergable, right? Unless D16830: USES=lib32: add support for lib32- companion ports also fails review for some reason, it will not be used.

@rozhuk.im-gmail.com You know the chroot approach is basically unmergable, right?

chroot call can be allowed to non root user, I can write simple MAC module that do this.
Or just use mac_biba / mac_lomac.

chroot call can be allowed to non root user, I can write simple MAC module that do this.
Or just use mac_biba / mac_lomac.

That's better. However, I still prefer lib32- approach. Mostly because it doesn't waste my time deploying chroot and compiling the same Wine components multiple times. (I also wouldn't mind 32-bit version of mesa-demos package, that would be very handy when I'm trying to diagnose 3d issues for other users on the FreeBSD forum or elsewhere. And that requires establishing some kind of packaging convention unless we want to stuff every potentially useful binary into wine-devel.)

That's better. However, I still prefer lib32- approach. Mostly because it doesn't waste my time deploying chroot and compiling the same Wine components multiple times. (I also wouldn't mind 32-bit version of mesa-demos package, that would be very handy when I'm trying to diagnose 3d issues for other users on the FreeBSD forum or elsewhere. And that requires establishing some kind of packaging convention unless we want to stuff every potentially useful binary into wine-devel.)

#include <sys/param.h>
#include <sys/kernel.h>
#include <sys/module.h>
#include <sys/priv.h>
#include <sys/proc.h>
#include <sys/systm.h>
#include <sys/socket.h>
#include <sys/socketvar.h>
#include <sys/sysctl.h>

#include <security/mac/mac_policy.h>

SYSCTL_DECL(_security_mac);

static SYSCTL_NODE(_security_mac, OID_AUTO, allow_chroot, CTLFLAG_RW, 0,
    "TrustedBSD mac_allow_chroot policy controls");

static int	allow_chroot_enabled = 1;
SYSCTL_INT(_security_mac_allow_chroot, OID_AUTO, enabled, CTLFLAG_RW,
    &allow_chroot_enabled, 0, "Enforce allow_chroot policy");


static int
allow_chroot_priv_grant(struct ucred *cred, int priv)
{
	if (!allow_chroot_enabled)
		return (EPERM);

	switch (priv) {
	case PRIV_VFS_CHROOT:
		return (0);
	}

	return (EPERM);
}

static struct mac_policy_ops allow_chroot_ops =
{
	.mpo_priv_grant = allow_chroot_priv_grant
};

MAC_POLICY_SET(&allow_chroot_ops, mac_allow_chroot,
    "TrustedBSD MAC/allow_chroot", MPC_LOADTIME_FLAG_UNLOADOK, NULL);

mac_allow_chroot works fine, but I got:

===>  Installing for pkg-1.10.5_2
===>  Checking if pkg already installed
pkg-static: /var/db/pkg wrong user or group ownership (expected 0/0 versus actual 1001/0)
*** Error code 74

and su inside does not work because it have no suid bit, that can set only root.

I like lib32 idea.
I dont like current wine-devel build time with wow option, even with ccache enabled.
But lets move with small steps - wine is outdated and lib32 - is different task.

As a reminder to new subscribers, just to make sure we are on the same page: this review is dead. Period.