Page MenuHomeFreeBSD

Add ALLOW_OSVERSION_MISMATCH flag to bsd.ports.mk
Needs ReviewPublic

Authored by grembo on May 13 2021, 5:01 AM.

Details

Reviewers
gjb
bdrewery
rgrimes
Group Reviewers
portmgr
Summary

Sometimes one wants to build a port in a jail that doesn't match the version of the host OS.

Test Plan

Build in a jail that doesn't match the hosts os version. See the failure. Try again, setting the environment variable.

Could be done nicer, error message could be nicer, could also add a "now you're on your own" warning, like other overrides show.

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 39160
Build 36049: arc lint + arc unit

Event Timeline

grembo created this revision.
This revision is now accepted and ready to land.May 13 2021, 5:26 AM

I personally prefer danfe's suggestion on the knob name, but yes, I think this works.

b/Mk/bsd.port.mk
1205 ↗(On Diff #89108)

This error message does not match the test that is performed immediately before it. I note in Glens posted error it appears that ${_OSRELEASE} is null??? It may be worth while to add ${_OSVERSION_MAJOR} != ${_OSRELEASE:R} to the end of the error message to see exactly what values did get compared.

grembo edited the test plan for this revision. (Show Details)

Updating diff using arcanist

This revision now requires review to proceed.May 13 2021, 11:44 AM

Updated the diff using arcanist, as using raw diff upload the way I did won't get us far.

NOTE: I created this to show an example of a solution of the problem raised on the mailing list. I wont push this forward myself though as long as there is no feedback from someone from @portmgr, be it acceptance, suggesting changes, or implementing a different solution (in which case I would simply abandon the review), or simply asking be to abandon the review, even though they don't feel like implementing anything else to the same effect.

This isn't simply about host not matching the jail. It's more nuanced. The jail may not match the jail. If people don't understand that I am strongly against anything like this going in. But what do I care if you shoot yourselves in the foot?

This isn't simply about host not matching the jail. It's more nuanced. The jail may not match the jail.

Does this change anything about the patch though (which in the end implements a force flag, which isn't nuanced at all)? Would you prefer a different name of the flag that emphasizes that you think this error shouldn't be ignored?

If people don't understand that I am strongly against anything like this going in. But what do I care if you shoot yourselves in the foot?

That's the very nature of providing force/override/ignore warning flags, people using them are on their own. This was raised by a couple of experienced engineers, who have their workflows disrupted by the current state of affairs.

As long as the default behavior doesn't change and therefore things stay safe for normal ports tree users, I don't see any negative consequences. People will shoot themselves in the foot anyway by patching the file manually, so having a flag will at least make sure they don't start using diverging local copies of bsd.port.mk.

I have no personal interest in this beyond trying to direct our energy towards more productive discussions, this doesn't seem like something worth fighting over and being subjected to a lot of negativity in the process.

I sympathize with having to field support requests of people shooting themselves in the foot. That said, there can be valid reasons to for example try a newer kernel w/o updated userland to see whether a regression was fixed. In that instance I would like (have to, really) build a new drm-kmod-current package which this check prevents me from doing.

Adding an ALLOW_OSVERSION_MISMATCH seems like a good workaround for that, no?

Just to repeat an earlier comment and asking portmgr to comment - then this review can progress or be abandoned and either way there will be a documented decision.

NOTE: I created this to show an example of a solution of the problem raised on the mailing list. I wont push this forward myself though as long as there is no feedback from someone from @portmgr, be it acceptance, suggesting changes, or implementing a different solution (in which case I would simply abandon the review), or simply asking be to abandon the review, even though they don't feel like implementing anything else to the same effect.

This isn't simply about host not matching the jail. It's more nuanced. The jail may not match the jail.

Does this change anything about the patch though (which in the end implements a force flag, which isn't nuanced at all)? Would you prefer a different name of the flag that emphasizes that you think this error shouldn't be ignored?

If people don't understand that I am strongly against anything like this going in. But what do I care if you shoot yourselves in the foot?

That's the very nature of providing force/override/ignore warning flags, people using them are on their own. This was raised by a couple of experienced engineers, who have their workflows disrupted by the current state of affairs.

My perspective is that we have re@ trying to use it without understanding the problem. It should not be used for any official builds.

As long as the default behavior doesn't change and therefore things stay safe for normal ports tree users, I don't see any negative consequences. People will shoot themselves in the foot anyway by patching the file manually, so having a flag will at least make sure they don't start using diverging local copies of bsd.port.mk.

I have no personal interest in this beyond trying to direct our energy towards more productive discussions, this doesn't seem like something worth fighting over and being subjected to a lot of negativity in the process.

The negativity is because the problem has not been understood since the check went in which has built up frustration over years, and now there is screaming to remove or allow disabling still without understanding the problem which is causing me to get frustrated. Misunderstandings breed frustration. Now everyone is frustrated. Let's slow down and try to understand.

Hopefully this clears up the problem:

  • When building a port it is possible for it to run uname to determine its target system (edit: and bake in different target runtime behavior). If that is coming back not matching the userland of the chroot/jail, how is that _ever ok_? 3rd party ports are far more likely to care about uname than __FreeBSD_version or any other FreeBSD-specific version lookup.
  • OSVERSION can probably always be determined automatically. A user does not need to mess with it so long as /usr/include/sys/param.h has a __FreeBSD_version that is expected for the target system.
  • OSVERSION ultimately determines *Ports framework behavior* and which patches are applied. It needs to be correct for the target system.
  • The only thing that someone trying to build in a chroot must do is ensure UNAME_r is correct.
    • Explain to me why it is easier for them to set ALLOW_OSVERSION_MISMATCH rather than modifying UNAME_r once every 2-3 years or making their chroot setup script do it auto based on git branch or something?
  • UNAME_r could probably be automatically determined from OSVERSION. That is a more sane proposal rather than ignoring the problem but I haven't thought deeply about it yet.

Please, anyone, explain to me why my points are wrong.

Please, anyone, explain to me what I am missing.

This isn't simply about host not matching the jail. It's more nuanced. The jail may not match the jail.

Does this change anything about the patch though (which in the end implements a force flag, which isn't nuanced at all)? Would you prefer a different name of the flag that emphasizes that you think this error shouldn't be ignored?

If people don't understand that I am strongly against anything like this going in. But what do I care if you shoot yourselves in the foot?

That's the very nature of providing force/override/ignore warning flags, people using them are on their own. This was raised by a couple of experienced engineers, who have their workflows disrupted by the current state of affairs.

My perspective is that we have re@ trying to use it without understanding the problem. It should not be used for any official builds.

As long as the default behavior doesn't change and therefore things stay safe for normal ports tree users, I don't see any negative consequences. People will shoot themselves in the foot anyway by patching the file manually, so having a flag will at least make sure they don't start using diverging local copies of bsd.port.mk.

I have no personal interest in this beyond trying to direct our energy towards more productive discussions, this doesn't seem like something worth fighting over and being subjected to a lot of negativity in the process.

The negativity is because the problem has not been understood since the check went in which has built up frustration over years, and now there is screaming to remove or allow disabling still without understanding the problem which is causing me to get frustrated. Misunderstandings breed frustration. Now everyone is frustrated. Let's slow down and try to understand.

Hopefully this clears up the problem:

  • When building a port it is possible for it to run uname to determine its target system (edit: and bake in different target runtime behavior). If that is coming back not matching the userland of the chroot/jail, how is that _ever ok_? 3rd party ports are far more likely to care about uname than __FreeBSD_version or any other FreeBSD-specific version lookup.
  • OSVERSION can probably always be determined automatically. A user does not need to mess with it so long as /usr/include/sys/param.h has a __FreeBSD_version that is expected for the target system.
  • OSVERSION ultimately determines *Ports framework behavior* and which patches are applied. It needs to be correct for the target system.
  • The only thing that someone trying to build in a chroot must do is ensure UNAME_r is correct.
    • Explain to me why it is easier for them to set ALLOW_OSVERSION_MISMATCH rather than modifying UNAME_r once every 2-3 years or making their chroot setup script do it auto based on git branch or something?
  • UNAME_r could probably be automatically determined from OSVERSION. That is a more sane proposal rather than ignoring the problem but I haven't thought deeply about it yet.

Please, anyone, explain to me why my points are wrong.

Please, anyone, explain to me what I am missing.

A build failure should not be the default case when building within a chroot(8) that does not match the running kernel. To quote Doug Barton, "tools, not policy."

If I have a build host running 14.0-CURRENT and a jail running 13.0-STABLE, my expectation is that I should be able to build *anything* for 13.0, world, kernel, and ports, without a hard failure due to a silly constraint enforced in the tree.

If I know what I am doing, I should be able to shoot myself in the foot and deal with the consequences. This is UNIX. As I said before:

"UNIX was not designed to stop you from doing stupid things, because
that would also stop you from doing clever things." -Doug Gwyn

Also note, official release builds do not happen on mismatched userlands/kernels. They are built on machines running the same major OSVERSION/UNAME_r as the target release.

@bdrewery But don't you think that

  1. A feature request from an important consumer of the ports collection should be considered;
  2. There is a large number of ports where none of this (uname, osversion, etc.) matters anyway, at least in a development environment;
  3. It's better to be able to clearly express "I'm willing to allow a mismatch" vs. overriding an environment variable and pretend there is none?

@gjb @bdrewery Can you imagine to agree on some compromise?

@bdrewery But don't you think that

  1. A feature request from an important consumer of the ports collection should be considered;
  2. There is a large number of ports where none of this (uname, osversion, etc.) matters anyway, at least in a development environment;
  3. It's better to be able to clearly express "I'm willing to allow a mismatch" vs. overriding an environment variable and pretend there is none?

@gjb @bdrewery Can you imagine to agree on some compromise?

I just want to be able to build weekly snapshots without adding hacks around other hacks to trick the bsd.ports.mk into thinking "everything is fine" by overriding OSVERSION and UNAME_r.

But, for now, I've solved that problem in a different way - we are no longer having stable/11 snapshots, as I upgraded the builders accordingly.

So, "policy" has overruled "tools" for the moment, at the expense of losing snapshots for a supported stable branch.