In D21172#460271, @jbeich wrote:

@jmd or @zeising, can you update devel/libclc to a more recent LLVM snapshot? lang/clover (P294) compiles fine against LLVM 9 API while linking errors maybe unrelated.

I trust that it runs fine in poudriere and so on. :)

In D20992#460070, @jhb wrote:

So with this change people will rebuild the module on current without having to reinstall the port, so I think this will effectively do the recommended change, but even more strongly on current. I think that any pre-built kernel modules are always going to have to be targeted at a kind of lowest-common-denominator. We aim to do this already for standalone kernel module builds (so cd /sys/modules/foo; make is the same problem space).

If you really want to customize kernel modules to the running kernel then I think the next step you might consider is to build modules during the post-install step of the package being installed. However, for people running releases, I think shipping a pre-built module that will work with GENERIC is sufficient. In that sense I'm pretty happy with where the current LOCAL_MODULES scheme ends up.

Minor nits, I noticed them, but feel free to change or leave them as-is.

lgtm (I only glanced at the patch, and haven't tested it, though).

In D20992#458955, @jhb wrote:

In D20992#458720, @zeising wrote:

In D20992#455803, @jhb wrote:

@zeising: With the related changes (see the stack, and you've commented on the drm-current-kmod one), a normal GENERIC kernel on head defines INVARIANTS and trips over this. If you apply the other two changes and install the updated package and try to build GENERIC you will hit this.

I'm not sure I understand this comment.
The kmod is built using the kmod facility in ports, and bsd.kmod.mk. I don't know if INVARIANTS and other options gets set by that, but it seems they don't. If the build when building out of tree (building the port) vs. building in tree is different because of this (and other options), this would mean we get different drivers depending on how they are built, and this feels like it's going to become a support nightmare. Perhaps options such as these should be stripped before building ports modules together with source, to avoid this issue.

It's more like getting reliably working modules on HEAD instead of random crashes and panics due to a mismatch in the kernel and module environment. Standalone modules only work with non-changing kernels (e.g. a user staying on a release) or with modules that use stable, well-supported interfaces like (the graphics drivers do not as they regularly use VM internals). Plus, running the graphics drivers with debugging enabled on HEAD such as INVARIANTS is more likely to find bugs more quickly and more nicely. virtualbox is in a similar situation where it needs to be paired with the running kernel instead of mixing old and new versions of kernel interfaces.

In D20990#458950, @jhb wrote:

In D20990#458719, @zeising wrote:

In D20990#456269, @johalun wrote:

Having to specify each source and header file seems like a huge maintenance burden. For every update in Linux version files get added, removed and renamed. Some sort of recursive directory install would be nice...

This is just how ports work, the package list (pkg-plist) must contain all files that are to be installed, otherwise pkg doesn't know how to create a package from the files in the staging area.

There are some ways to do automatic plist general, but my understanding is that portmgr@ generally frowns on it as they want to preserve the ability to grep for a file in all the pkg-plist files. *shrug* FWIW, I generated the list of files with 'find' fairly easily. 'make check-plist' is also handy to let you know if you have it wrong.

In D20992#455803, @jhb wrote:

@zeising: With the related changes (see the stack, and you've commented on the drm-current-kmod one), a normal GENERIC kernel on head defines INVARIANTS and trips over this. If you apply the other two changes and install the updated package and try to build GENERIC you will hit this.

In D20990#456269, @johalun wrote:

Having to specify each source and header file seems like a huge maintenance burden. For every update in Linux version files get added, removed and renamed. Some sort of recursive directory install would be nice...

Thank you for working on this!

I don't think the drm kmods are built with INVARIANTS, unless it's somehow passed into the build environment when building out of tree kmods. In general, are there other options added when building the DRM kmods (or other kmods) as part of the kernel build?

In D20724#455153, @antoine wrote:

This breaks ports that test something and set USE_XORG between bsd.port.pre.mk and bsd.port.post.mk.

Also, I have the feeling that there is a .include "${USESDIR}/*.mk" abuse in xorg-cat.mk, in my opinion this should be pushed to the USES line in individual ports.

Updates based on comments from @mat

Update patch, add documentation for USES=xorg-cat as well.

Submitted for exp-run in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238988 .

Update based on latest comments. Ready for exp-run now.

Patch is updated based on last round of comments. The only outstanding issue is the handling of libtool and autoreconf. Currently, the way it's handled is by moving the check for libtool_ARGS into the POSTMKINCLUDED stage of autoreconf.mk. Based on the discussion, perhaps that is the least bad idea?

Updated based on comments.

The PR in connection with this review has been updated with a patch to update pixman to 0.38.4. This review can be closed/abandoned.

In D20724#449462, @tijl wrote:

In D20724#449276, @zeising wrote:

I am not entirely sure what you mean here. Ports fetched directly from gitlab (or github) will require autoreconf, since when fetching arbitrary revisions, no pre-generated autotools scripts are provided.

Some projects keep pre-generated scripts in their repo.

In D20724#449198, @mat wrote:

In D20724#449167, @zeising wrote:

In D20724#449059, @mat wrote:

I think you are trying to put really too much magic in xorg-cat.

I am merely replicating the functionality that was in bsd.xorg.mk. All of this is in the old bsd.xorg.mk, and there it works. bsd.xorg.mk is only pulled in from bsd.ports.mk if USE_XORG or XORG_CAT is defined, perhaps it is better left this way, rather than converted to USES?

Save for the generic framework files, nothing should be left as a bsd.*.mk file, everything should be moved to a USES. The move is better if it is done by someone who actually understands what the file does.

In D20724#449170, @tijl wrote:

I found two ports with USES=autoreconf and XORG_CAT: x11-drivers/xf86-input-egalax and x11-drivers/xf86-video-intel. The former isn't maintained by x11@ and is fetched from github. If such non-X.org ports are allowed to use XORG_CAT then they may be fetched from gitlab and not need autoreconf for instance, so you cannot include autoreconf.mk. If XORG_CAT is only meant for X.org ports and use by other ports is unsupported then you can keep the libtool.mk and autoreconf.mk includes (as in 9e63b55 but without " || ${USES:Mautoreconf}") and for special cases like x11-drivers/xf86-video-intel you can add "USES= autoreconf libtool" to the port Makefile instead of just USES=autoreconf.

In D20724#449059, @mat wrote:

I think you are trying to put really too much magic in xorg-cat.

In D20724#449034, @tijl wrote:

I think the problem is you are including autoreconf.mk before setting libtool_ARGS. Try moving "libtool_ARGS?= #empty" to right before each libtool.mk include (there are two) so it's only defined if libtool is used. Then include autoreconf.mk if "USE_GITLAB && BUILDSYS == autotools" right after ". endif # ${_XORG_CAT} == <category>".

Update patch again.
Move things around in Uses/autoreconf.mk to check for libtool things later. Uses/autoreconf.mk checks if libtool is being used, by checking libtool_ARGS, and if it's set, add a dependency on libtoolize. Move this check later, in the POSTMKINCLUDED phase, to avoid issues where we're hitting autoreconf before setting libtool_ARGS, and then not bringing in libtoolize.

Updated diff based on comments from @mat

In D20732#448839, @mat wrote:

In D20732#448816, @zeising wrote:

In D20732#448815, @mat wrote:

Could you also add documentation for USES=xorg-cat?

I purposefully omitted it, since it is supposed to be internal to xorg ports, not used by other ports, similar to qt-dist.mk.

Mmmm, I do not understand, you mean I'll have to document it myself when the other review hits the tree? All USES have to be documented, otherwise, the only one who knows how it works is you, so, the bus factor is 1, and it is very bad.

In D20732#448815, @mat wrote:

Could you also add documentation for USES=xorg-cat?

Updated based on comments from @tobik .

In D20724#448218, @kwm wrote:

This porters-handbook chapter needs to be updated/rewritten, maybe there are other bits elsewhere in the porters-handbook that also need attention.
https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/using-x11.html

Thank you very much for the review @tobik !
I think I've fixed everything, I'm just going to do a local smoke test, and then I'll update the patch in phabricator as well.

Some notes about the patch: Uses/xorg.mk and possibly Uses/xorg-cat.mk will be copied from bsd.xorg.mk using svn cp or svn mv as needed. Current patch is pulled from our git repo with rename detection turned off when generating the patch.