Remove MK_AUTO_OBJ from env passed to PORTS_MODULES

[jah]

Summary

Remove MK_AUTO_OBJ from the build environment provided to each port in PORTS_MODULES.
This fixes a failure to resolve object file paths seen when buildkernel (which sets MK_AUTO_OBJ=yes) and installkernel (which sets MK_AUTO_OBJ=no) are run as separate steps. r329232 partially fixed this scenario by removing MAKEOBJDIR, but it seems AUTO_OBJ settings also need to be on the same page for these two steps.

[jah]

Add dep on share/mk contents

[bdrewery]

Is this fixing something? It's adding more complexity.

[jah]

In D14143#296955, @bdrewery wrote:

Is this fixing something? It's adding more complexity.

Killing MK_AUTO_OBJ/MKOBJDIR fixes the installkernel failure described at https://lists.freebsd.org/pipermail/freebsd-current/2018-January/068319.html
I wouldn't be surprised if there's a cleaner way to do that though.

The staging part is more of a conceptual thing. We're already setting OSVERSION and SRC_BASE to build the port as though it's being built against the updated OS. It seems like we should also be building it against the system makefiles as they would be installed on the updated OS. Just setting SYSDIR might not be enough; sys/conf/kmod.mk still pulls bsd.init.mk and bsd.compiler.mk from the system makefile path, not to mention any non-kmod components of the port. It seems reasonably likely that mismatches between those makefiles and e.g. the environment created by buildkernel/installkernel could cause problems on something like a major version upgrade. That said, I did not see any problems when testing an 11->12 upgrade either before or after this change.

[jah]

In D14143#296975, @jah wrote:

In D14143#296955, @bdrewery wrote:

Is this fixing something? It's adding more complexity.

Killing MK_AUTO_OBJ/MKOBJDIR fixes the installkernel failure described at https://lists.freebsd.org/pipermail/freebsd-current/2018-January/068319.html
I wouldn't be surprised if there's a cleaner way to do that though.

The staging part is more of a conceptual thing. We're already setting OSVERSION and SRC_BASE to build the port as though it's being built against the updated OS. It seems like we should also be building it against the system makefiles as they would be installed on the updated OS. Just setting SYSDIR might not be enough; sys/conf/kmod.mk still pulls bsd.init.mk and bsd.compiler.mk from the system makefile path, not to mention any non-kmod components of the port. It seems reasonably likely that mismatches between those makefiles and e.g. the environment created by buildkernel/installkernel could cause problems on something like a major version upgrade. That said, I did not see any problems when testing an 11->12 upgrade either before or after this change.

ping...any thoughts on this? I'm not set on doing things this way by any means, if there's something different you'd prefer to do here.

[bdrewery]

Which port inspired the need for D13053 due to the LDADD/LIBADD problem?
I cannot recreate the issue being reported on current@. I'm still not fond of staging file like this as it is a ~4th way of system files being read. I'd rather go the other direction and always install everything.
Anyway, since I cannot recreate the newly reported issue I want to look at the port you ran into a problem with and see if I can rework D13053.

[jah]

In D14143#299618, @bdrewery wrote:

Which port inspired the need for D13053 due to the LDADD/LIBADD problem?
I cannot recreate the issue being reported on current@. I'm still not fond of staging file like this as it is a ~4th way of system files being read. I'd rather go the other direction and always install everything.
Anyway, since I cannot recreate the newly reported issue I want to look at the port you ran into a problem with and see if I can rework D13053.

That would be multimedia/cx88 (which surprisingly, people still use:). I created that port a long time ago, before PORTS_MODULES was even a thing; besides the kmods there's also a userspace utility in there, and that's the part that fails in 11.0+ due to src.libnames.mk.

But like I said in the review for D13053, maybe the real right answer is to just split ports like that up, so the kmods are in a separate port by themselves?
I'm still the maintainer for cx88, so I can make that happen :)

I did all this because I thought maybe I could make PORTS_MODULES more flexible without adding too much complexity, and staging is just an extension of that. But if that's just going to create a maintenance headache that's worse than any problem it might fix, let's not do it that way.

[jah]

In D14143#299639, @jah wrote:

In D14143#299618, @bdrewery wrote:

Which port inspired the need for D13053 due to the LDADD/LIBADD problem?
I cannot recreate the issue being reported on current@. I'm still not fond of staging file like this as it is a ~4th way of system files being read. I'd rather go the other direction and always install everything.
Anyway, since I cannot recreate the newly reported issue I want to look at the port you ran into a problem with and see if I can rework D13053.

That would be multimedia/cx88 (which surprisingly, people still use:). I created that port a long time ago, before PORTS_MODULES was even a thing; besides the kmods there's also a userspace utility in there, and that's the part that fails in 11.0+ due to src.libnames.mk.

But like I said in the review for D13053, maybe the real right answer is to just split ports like that up, so the kmods are in a separate port by themselves?
I'm still the maintainer for cx88, so I can make that happen :)

I did all this because I thought maybe I could make PORTS_MODULES more flexible without adding too much complexity, and staging is just an extension of that. But if that's just going to create a maintenance headache that's worse than any problem it might fix, let's not do it that way.

Also: to recreate the issue reported on current@, you have to run buildkernel and installkernel as separate steps instead of just 'make kernel'. Doing that I was able to repro the problem with both multimedia/cx88 and x11/nvidia-driver.

[bdrewery]

I fixed the problem differently in r329232

[jah]

In D14143#300791, @bdrewery wrote:

I fixed the problem differently in r329232

I don't think just getting rid of MAKEOBJDIR is enough.
With PORTS_MODULES=x11/nvidia-driver in an otherwise empty src.conf, and an empty src.env.conf, I get this during installkernel when I try to build HEAD using 'make buildkernel; make installkernel':

> src (install)

> src/nvidia (install)

install -T release -o root -g wheel -m 555 nvidia.ko /usr/obj/usr/src/amd64.amd64/sys/GENERIC/usr/ports/x11/nvidia-driver/work/stage/boot/modules/
install: nvidia.ko: No such file or directory

• Error code 71

buildkernel sets MK_AUTO_OBJ=yes while installkernel sets MK_AUTO_OBJ=no.
It looks like this produces some disagreement in where the build output goes. That's why I also removed MK_AUTO_OBJ in this review.
I think you have to do that or something like it to get everything on the same page; adding MK_AUTO_OBJ=yes to PORTSMODULESENV would also work but seems riskier and less correct.

[jah]

Get rid of staging, remove MK_AUTO_OBJ

[jah]

In D14143#301044, @jah wrote:

In D14143#300791, @bdrewery wrote:

I fixed the problem differently in r329232

I don't think just getting rid of MAKEOBJDIR is enough.
With PORTS_MODULES=x11/nvidia-driver in an otherwise empty src.conf, and an empty src.env.conf, I get this during installkernel when I try to build HEAD using 'make buildkernel; make installkernel':

> src (install)

> src/nvidia (install)

install -T release -o root -g wheel -m 555 nvidia.ko /usr/obj/usr/src/amd64.amd64/sys/GENERIC/usr/ports/x11/nvidia-driver/work/stage/boot/modules/
install: nvidia.ko: No such file or directory

• Error code 71

buildkernel sets MK_AUTO_OBJ=yes while installkernel sets MK_AUTO_OBJ=no.
It looks like this produces some disagreement in where the build output goes. That's why I also removed MK_AUTO_OBJ in this review.
I think you have to do that or something like it to get everything on the same page; adding MK_AUTO_OBJ=yes to PORTSMODULESENV would also work but seems riskier and less correct.

Do you want to fix this or should I?

[jah]

In D14143#304010, @jah wrote:

In D14143#301044, @jah wrote:

In D14143#300791, @bdrewery wrote:

I fixed the problem differently in r329232

I don't think just getting rid of MAKEOBJDIR is enough.
With PORTS_MODULES=x11/nvidia-driver in an otherwise empty src.conf, and an empty src.env.conf, I get this during installkernel when I try to build HEAD using 'make buildkernel; make installkernel':

> src (install)

> src/nvidia (install)

install -T release -o root -g wheel -m 555 nvidia.ko /usr/obj/usr/src/amd64.amd64/sys/GENERIC/usr/ports/x11/nvidia-driver/work/stage/boot/modules/
install: nvidia.ko: No such file or directory

• Error code 71

buildkernel sets MK_AUTO_OBJ=yes while installkernel sets MK_AUTO_OBJ=no.
It looks like this produces some disagreement in where the build output goes. That's why I also removed MK_AUTO_OBJ in this review.
I think you have to do that or something like it to get everything on the same page; adding MK_AUTO_OBJ=yes to PORTSMODULESENV would also work but seems riskier and less correct.

Do you want to fix this or should I?

ping