Fix -jN META_MODE bootstrap for usr.bin/netstat

[lifanov_mail.lifanov.com]

Summary

add a local dependency to Makefile.depend, which is the smallest
change that fixes -jN build of usr.bin/netstat in META_MODE.

Test Plan

make -C usr.bin/netstat -j8 -WITH_META_MODE=y

[bdrewery]

It's not enough. I was just preparing a tree-wide update.

My build generated this:

~/svn/clean # svn diff usr.bin/netstat

Index: usr.bin/netstat/Makefile.depend

• usr.bin/netstat/Makefile.depend (revision 287844)

+++ usr.bin/netstat/Makefile.depend (working copy)
@@ -21,4 +21,14 @@

.if ${DEP_RELDIR} == ${_DEP_RELDIR}

1. local dependencies - needed for -jN in clean tree

+main.o: nl_defs.h
+main.po: nl_defs.h
+mroute.o: nl_defs.h
+mroute.po: nl_defs.h
+netisr.o: nl_defs.h
+netisr.po: nl_defs.h
+nl_symbols.o: nl_symbols.c
+nl_symbols.po: nl_symbols.c
+route.o: nl_defs.h
+route.po: nl_defs.h
.endif

[lifanov_mail.lifanov.com]

That's correct. This is what my build generates also.
The main.o <- nl_defs.h is the minimal change to get dirdeps to generate the rest of them.

[lifanov_mail.lifanov.com]

Maybe I'm not sure what the goal of Makefile.depend files in the tree is.
I understood that they are there to bootstrap the auto-generated Makefile.depend files
from a build with local changes/options in it?

[bdrewery]

In D3683#75966, @lifanov_mail.lifanov.com wrote:

Maybe I'm not sure what the goal of Makefile.depend files in the tree is.
I understood that they are there to bootstrap the auto-generated Makefile.depend files
from a build with local changes/options in it?

Think of DIRDEPS in Makefile.depend like an auto-generated ports DEPENDS list. It will build all dependencies "first".

The file: dep targets are to support -j builds in a clean tree. Without them -j0 would work fine as the dependencies are typically found otherwise, but -jN would not build them correctly.

The Makefile.depend are highly option dependent. It's a flaw in the design for FreeBSD. Committers have to be careful to not commit non-default-option-dependent changes of Makefile.depend. Most people who build will have changes to these files which is fine. It arguably would be better to just do this like ports and have a manual dependency list to avoid the list being fickle depending on options.

In theory committers would keep these files up-to-date with all dependencies.

You're right that your fix is sufficient to get it working. The rest will be auto added. However the next time someone does a build and commits the file the rest that I mentioned will be added. So I could commit your main.o change, but I'll be adding the rest later today anyhow! We have many missing -j dependencies in the tree. I have a commit almost ready with about 32 Makefile.depend updated.

[bdrewery]

Also as for this specific change, I suspect that DPSRCS is not being utilized by META_MODE properly as nl_defs.h is in the DPSRCS list. I haven't checked this theory yet. (Unsure if -j0 on this actually works or not, I would expect it to since DPSRCS has information that could be used)

[lifanov_mail.lifanov.com]

Thanks for the explanation! By the way, this does not work with -j0 without adding this dependency either.

[lifanov_mail.lifanov.com]

The issue has been addressed.

[sjg]

Firstly - way cool to see folk playing with this ;-)

Most Makefile.depend files start with a comment that they are auto generated.
They *should* be self correcting.

That doesn't mean that (until everyone is using them) they won't get out of date occasionally.
If they get updated though and your tree is up to date they *should* be correct.

Q. are you using meta2deps.sh or meta2deps.py ?

[sjg]

In D3683#75968, @bdrewery wrote:

Also as for this specific change, I suspect that DPSRCS is not being utilized by META_MODE properly as nl_defs.h is in the DPSRCS list. I haven't checked this theory yet. (Unsure if -j0 on this actually works or not, I would expect it to since DPSRCS has information that could be used)

DPSRCS is something used by bsd.dep.mk - which isn't used in meta mode since we avoid the whole notion of separate 'make depend' step.
The local dependencies that should be captured in Makefile.depend replace the need for such things.

[lifanov_mail.lifanov.com]

Cool, thanks! Actually, how do you install the built product?
There is a populated stage dir but no install targets.

[bdrewery]

In D3683#75969, @lifanov_mail.lifanov.com wrote:

Thanks for the explanation! By the way, this does not work with -j0 without adding this dependency either.

Great. It would be good to investigate if we can make it work by bootstrapping the DPSRCS list

In D3683#76141, @sjg wrote:

In D3683#75968, @bdrewery wrote:

Also as for this specific change, I suspect that DPSRCS is not being utilized by META_MODE properly as nl_defs.h is in the DPSRCS list. I haven't checked this theory yet. (Unsure if -j0 on this actually works or not, I would expect it to since DPSRCS has information that could be used)

DPSRCS is something used by bsd.dep.mk - which isn't used in meta mode since we avoid the whole notion of separate 'make depend' step.
The local dependencies that should be captured in Makefile.depend replace the need for such things.

Yes I know. I was suggesting there may be a way to utilize it to avoid clean build failures.

[lifanov_mail.lifanov.com]

Actually, how are you generating these?
How did you come up with r287905, for example?

I come up with quite a bigger list. My method was:

1. empty OBJDIR
2. run the-lot with no customizations (no src.conf, GENERIC, etc.)
3. fixup anything that doesn't build by building it individually with -j1
4. repeat 2-3 until the-lot succeeds
5. repeat 1-2 until there is no difference in Makefile.depend files

[bdrewery]

In D3683#76242, @lifanov_mail.lifanov.com wrote:

Actually, how are you generating these?
How did you come up with r287905, for example?

It was a partial commit of some missing local dependencies.

I come up with quite a bigger list. My method was:

1. empty OBJDIR
2. run the-lot with no customizations (no src.conf, GENERIC, etc.)
3. fixup anything that doesn't build by building it individually with -j1
4. repeat 2-3 until the-lot succeeds
5. repeat 1-2 until there is no difference in Makefile.depend files

Yes, it will take several builds to get everything right. I've been working through various bugs
and have not yet gotten to a full 2 builds to resolve all of the Makefile.depend missing entries
but am getting there.

The reason it takes several builds is that the committted Makefile.depend files are missing much
of the build host tools so they use your /usr/bin version during the first build. Once the-lot is finished
though and you do another build it will then find these tools (such as lex, and yacc) in the stagedir
and use those, thus updating Makefile.depend with these yacc.host and lex.host versions. We need
to get the fully correct Makefile.depend files updated to stop them changing so constantly.

I have not gotten to the compiler yet to see how it impacts the DIRDEPS as well.

[bdrewery]

Also see this section for ensuring the various host tools are built first: http://crufty.net/sjg/docs/freebsd-meta-mode.htm#getting-started

[lifanov_mail.lifanov.com]

There is nothing more to do here.