Page MenuHomeFreeBSD

Add SUBDIR_SORTED, a generic mechanism for ordering subdirectories and apply option to *bin/Makefile
AbandonedPublic

Authored by ngie on Jul 17 2017, 6:28 PM.
Tags
None
Referenced Files
Unknown Object (File)
Dec 23 2023, 2:49 AM
Unknown Object (File)
Aug 20 2023, 1:33 AM
Unknown Object (File)
Aug 14 2023, 3:46 PM
Unknown Object (File)
Jul 22 2023, 8:00 PM
Unknown Object (File)
Jun 3 2023, 1:32 AM
Unknown Object (File)
May 14 2023, 6:07 AM
Subscribers

Details

Reviewers
bdrewery
Summary

Add SUBDIR_SORTED, a generic mechanism for ordering subdirectories

This restores the sorting removed in r320701, broken by the SUBDIR.yes
work, and applies it to SUBDIR + SUBDIR.yes, providing equivalent
functionality to the sorting before SUBDIR.yes was added.

The uniquifying piece (:u) doesn't seem like it's really useful in my
opinion, so I didn't readd that feature.

Documentation in bsd.README pending.

MFC after: 2 months

Test Plan
  • Built the change on my local VM and installed it without issue.
  • Ran "make tinderbox" with the patch.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 10534
Build 10942: arc lint + arc unit

Event Timeline

Summary has SUBDIR_ORDERED code has SUBDIR_SORTED

Would SUBDIR_UNSORTED make more sense? Of course SUBDIR_PARALLEL likely should wither and die for a similar reason...

ngie retitled this revision from Add SUBDIR_ORDERED, a generic mechanism for ordering subdirectories and apply option to *bin/Makefile to Add SUBDIR_SORTED, a generic mechanism for ordering subdirectories and apply option to *bin/Makefile.Jul 17 2017, 6:36 PM
ngie edited the summary of this revision. (Show Details)

Summary has SUBDIR_ORDERED code has SUBDIR_SORTED

Thanks -- I fixed that.

In D11628#240879, @imp wrote:

Would SUBDIR_UNSORTED make more sense? Of course SUBDIR_PARALLEL likely should wither and die for a similar reason...

I don't think SUBDIR_UNSORTED would make sense to be honest. That sounds like an opt-out feature that might break downstream consumers of bsd.subdir.mk. I wanted to make this opt-in so projects that used bsd.subdir.mk and didn't use SUBDIR_PARALLEL wouldn't break after this change. FreeBSD has several Makefiles in gnu/ for instance at some point in time that were also affected by this, IIRC, so I expect downstream consumers to be more likely to have Makefiles which might be affected by the converse behavior if it was globally enabled and available as an opt-out feature.

In D11628#240890, @ngie wrote:
In D11628#240879, @imp wrote:

Would SUBDIR_UNSORTED make more sense? Of course SUBDIR_PARALLEL likely should wither and die for a similar reason...

I don't think SUBDIR_UNSORTED would make sense to be honest. That sounds like an opt-out feature that might break downstream consumers of bsd.subdir.mk. I wanted to make this opt-in so projects that used bsd.subdir.mk and didn't use SUBDIR_PARALLEL wouldn't break after this change. FreeBSD has several Makefiles in gnu/ for instance at some point in time that were also affected by this, IIRC, so I expect downstream consumers to be more likely to have Makefiles which might be affected by the converse behavior if it was globally enabled and available as an opt-out feature.

Right, but an opt-out feature breaks things now rather than later when we make it default. I expect this to be used always, except in very rare cases. As such, I think we should be doing it always unless there's a good reason not to. It was the old default behavior, so it's unlikely there will be anything it breaks...

In D11628#240967, @imp wrote:

Right, but an opt-out feature breaks things now rather than later when we make it default. I expect this to be used always, except in very rare cases. As such, I think we should be doing it always unless there's a good reason not to. It was the old default behavior, so it's unlikely there will be anything it breaks...

Again, I'm not so much worried about FreeBSD makefiles as I am about Isilon makefiles, Netflix makefiles, etc. If enabling the feature breaks something that's worked for ages (and worked prior to/during SUBDIR_PARALLEL/SUBDIR.yes), people will rightfully complain.

I would rather make the behavior explicit by default, instead of implicit in this case to avoid unintended consequences in downstream projects -- that is what this CR proposes.

Also, these issues would probably be non-deterministic (the -j1 vs -j4 mess we had with SUBDIR_PARALLEL) -- I would rather not force FreeBSD or downstream consumers to deal with nonsensical non-determinism.

I need to think about this CR some more. I just realized that it doesn't handle .WAIT barriers properly -- so while it makes sense in the *bin/Makefile case, its usefulness is reduced.

Yeah.. Given that .WAIT isn't being handled that it's another knob that would be added to each conforming Makefile for a nice-to-have that only makes sense in either -j1 or -V (in which case one can use -VSUBDIR:O) -- I don't see the point in this item.

In D11628#241309, @ngie wrote:

Again, I'm not so much worried about FreeBSD makefiles as I am about Isilon makefiles, Netflix makefiles, etc. If enabling the feature breaks something that's worked for ages (and worked prior to/during SUBDIR_PARALLEL/SUBDIR.yes), people will rightfully complain.

Would it be useful for Isilon's makefiles if bsd.subdir.mk did '.-include "local.subdir.mk" '
to allow you to frob SUBDIR as you wish?
I generally find the local.*.mk a simple means of customizing without seriously affecting the generic logic.

In D11628#241667, @sjg wrote:
In D11628#241309, @ngie wrote:

Again, I'm not so much worried about FreeBSD makefiles as I am about Isilon makefiles, Netflix makefiles, etc. If enabling the feature breaks something that's worked for ages (and worked prior to/during SUBDIR_PARALLEL/SUBDIR.yes), people will rightfully complain.

Would it be useful for Isilon's makefiles if bsd.subdir.mk did '.-include "local.subdir.mk" '
to allow you to frob SUBDIR as you wish?
I generally find the local.*.mk a simple means of customizing without seriously affecting the generic logic.

I'm not just worried about Isilon -- I'm also thinking about IronPort, Netflix, downstream open source projects of FreeBSD, etc.