bsd.sites.mk: Support tag names with USE_GITLAB

[zirias]

Summary

Since quite some time, it's possible to download generated tarballs from
gitlab for a tag name without knowing the corresponsing git hash. This
is preferable because you have a single source of truth about the
software version.

Support this by replacing GL_COMMIT with GL_TAGNAME which also accepts a
commit hash, similar to USE_GITHUB. Unfortunately, there's a catch, the
DISTFILE must be named exactly: ${GL_PROJECT}-${GL_TAGNAME}.tar.bz2,
otherwise it extracts in a directory that contains the commit hash. So,
we can't add the gitlab account or the revision suffix to make it work.

Therefore, detect whether GL_TAGNAME contains a commit hash, and if it
does, use the old behavior of deriving DISTNAME and DISTFILES.

Of course, globally replacing GL_COMMIT with GL_TAGNAME would be
required to commit this.

PR: 248967

[zirias]

Fix copy&paste error in comments

[zirias]

Fix another error in comments

[zirias]

For demo, add a few example port updates (that won't be part of the final commit):

net/liblinphone: Convert to using tag name by just deleting GL_COMMIT
net/ortp: Keep using commit hash renaming GL_COMMIT to GL_TAGNAME
textproc/linuxdoc-tools: Convert to USE_GITLAB in the first place

[zirias]

Also update affected framework files

[zirias]

Fix ordering of variables in bsd.options.mk

[zirias]

Ok, I guess that's it, finally. Sorry for the noise...

To reiterate, only the changes below Mk/ are actually part of the review, the 3 port updates are just examples for better understanding the change.

The "rollout plan" would be to search/replace GL_COMMIT with GL_TAGNAME and have an exp-run (at least for the fetch phase) for that. Conversion to actual tag names (or, if possible, completely dropping the variable) would be a manual task for the maintainers of ports using gitlab. That's likely unavoidable, because a) older versions of gitlab don't support downloading an archive for a tag and b) you can't be sure tag names exactly reflect the dist version... Instructions would have to be added to CHANGES and possibly the porters' handbook.

[zirias]

As suggested by @diizzy on IRC, also add a DEV_WARNING when MASTER_SITES includes an URL that looks like a gitlab URL (using a tag). With tags support in USE_GITLAB, there shouldn't be any reason left to hardcode MASTER_SITES. Update the commit message accordingly...

[tcberner]

This looks good :)

@antoine can you run a fetch-only-no-build exp-run, or should it be done manually?

[diizzy]

Unrelated changes, (ports) should be split into separate commits.

Mk/Uses/xorg-cat.mk

29

This comment should be removed as we always prefer official release tarballs over generated ones.

[diizzy]

@tcberner
I would recommend make configure too to make sure WRKSRC doesn't get changed and breaks the port

[zirias]

In D37077#842136, @diizzy wrote:

Unrelated changes, (ports) should be split into separate commits.

As stated above, the 3 port updates are just examples to better understand the change. They will be replaced by a global search/replace GL_COMMIT -> GL_TAGNAME (which should be fully backwards compatible) for real commit (and test).

[zirias]

Mk/Uses/xorg-cat.mk

29

This just documents an actual feature of that helper, starting at line 87. I would only remove the comment if the feature was removed as well (and I don't know whether it is needed/used).

[zirias]

Here's now a "real" version, including renaming GL_COMMIT to GL_TAGNAME in all ports.

[diizzy]

No, don't shoehorn this into one commit. This should be in a separate commit and all of this would preferably be committed after 2022-10-30 (maintainer timeout) unless I can get a portmgr approval to commit all pending fixups for ports in tree.
PRs: 267067, 267068, 267069, 267070, 267072, 267089, 267092, 267101, 267118, 267103, 267104, 267076

[zirias]

In D37077#842177, @diizzy wrote:

No, don't shoehorn this into one commit. This should be in a separate commit and all of this would preferably be committed after 2022-10-30 (maintainer timeout) unless I can get a portmgr approval to commit all pending fixups for ports in tree.

It must be committed together no matter what. I personally prefer not to split up commits when they only work correctly together, but *can* do so without problems, of course I do have the "framework-only" commit here, because all the port updates are a moving target. Also unsure whether it's really necessary to wait, cause all it takes is s/GL_COMMIT/GL_TAGNAME/ to be compatible with this change. But OTOH, proper testing will take a while anyways.

PRs: 267067, 267068, 267069, 267070, 267072, 267089, 267092, 267101, 267118, 267103, 267104, 267076

You can drop 267103 from that list, I'll update that port directly without using a commit hash (already tested locally).

[rene]

It probably also makes sense to commit the rest of https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=267076 (i.e. the non-GITLAB changes) on its own.

[zirias]

In D37077#842168, @diizzy wrote:

@tcberner
I would recommend make configure too to make sure WRKSRC doesn't get changed and breaks the port

Not all ports actually do something on configure. The change is supposed to be fully backwards compatible – for a "smoke test", it might be enough to include the patch phase.

[zirias]

Rebase and update to finally get it on the road...

The first commit only contains the changes to the framework, the second commit
is auto-generated by a simple script:

#!/bin/sh
for o in $(portgrep -o1 "GL_COMMIT"); do
        sed -i '' -e 's:GL_COMMIT:GL_TAGNAME:g' $o/Makefile
done

GL_TAGNAME was chosen to be in line with USE_GITHUB, it is designed to be
fully backwards compatible to GL_COMMIT, so ports should still fetch the
exact same distfile. To verify that, a second script was used:

#!/bin/sh
export DISTDIR=/tmp/fetchtest
rm -fr $DISTDIR
for o in $(portgrep -o1 "GL_TAGNAME="); do
        make -C $o do-fetch checksum clean
done

Full results are here: https://people.freebsd.org/~zirias/logs/checkfetch.log

It looks fine:

# grep -c "Checksum [^O]" /tmp/checkfetch.log
0
# grep -c "Checksum OK" /tmp/checkfetch.log
6090

Using actual tag names where appropriate will be left to individual ports'
maintainers.

[zirias]

On suggestion by @tcberner, add a fallback for GL_COMMIT and a DEV_WARNING when it is still used.

[tcberner]