Ports infra: Support for forbidding non-runtime dependencies on specific ports

[olce]

Summary

Create a new NORUNTIME file at ports tree's root which contains a list
of ports that cannot have runtime dependents. Its format is loosely modeled on
the MOVED file. Its first column contains extended regular expressions that are
matched against the origin of dependencies (anchored to the right).

Modify the script Mk/Scripts/do-depends.sh so that, when the dependency type is
RUN_DEPENDS or LIB_DEPENDS, it matches the origin of the current dependency
with the first column of the NORUNTIME file.

While here, enrich all error reporting with a line indicating the precise
dependency causing a problem, which helps when there are lots of them
(previously, only the dependency type would be displayed).

Test Plan

Tested locally with classical 'make' commands. Then an exp run I
guess?

[bapt]

I do like what I see and it can be used in long term for other areas, if no other portmgr step here I will merge this in a week

about the format, why adding a pattern matching? exact match should be enough I think and would simplify the script

[tcberner]

NORUNTIME

11

^ I think this should preferably be the date from whence it is forbidden and not of entry.

[olce]

about the format, why adding a pattern matching? exact match should be enough I think and would simplify the script

Did this because I thought it might be useful. I illustrated it with Python 2, by excluding lang/python2 and lang/python27 with just a single line. I agree that this example is not particularly compelling though.

Do you think there is any chance that we'll want to forbid a run-time dependency on particular flavors of some port only? Then, that might be another use case for regexps (and I'll need to move the test block upper in the script).

I do not have strong opinions about this feature. If you think it's better to remove it as a matter of policy, then of course I'll remove it. In this case, would you want me to remove the temporary precomputed patterns' file also?

NORUNTIME

11

Ok, so you'd like something working like EXPIRATION_DATE in some port's Makefile. Going to update the script to this end.

[tcberner]

NORUNTIME

11

Yes, ideally with an automatic DEPRECATED=runtime depends on foo bar which will be forbidden after ${date}

[olce]

Date is expiration date, regexp support removal, read NORUNTIME only once

Removed support for regexp. Allows some simplifications. The biggest one,
however, is independent, and consists in gathering all runtime dependencies in
the loop, building a regexp that is then used against NORUNTIME once through
awk.

Date is now an expiration date. The dependency is only effectively forbidden
the day after that in NORUNTIME (in UTC). The reason is now printed in the
error message.

[olce]

Fix awk expression to exclude comment lines

[olce]

Set DEPRECATED and EXPIRATION_DATE if appropriate

If DEPRECATED is not set, set both variables with the reason and expiration date for the runtime
dependency expiring first.

If it is, compare the existing EXPIRATION_DATE to the dependency's expiration date, and override the
two variables' values (which can come either from the port itself or some script in Uses, e.g.,
python.mk) only if the latter is earlier than the former (i.e., if the dependency is going to be
forbidden earlier than the expiration date specified elsewhere).

[olce]

So, to recap:

1. NORUNTIME is a file independent from any ports' files listing ports that cannot be runtime

dependencies (to any other ports) after some specific date.

2. The modifications in do-depends.sh ensure that the dependency step will fail when depending on

forbidden ports. So port building will simply hard fail when it's the case.

3. The modifications of bsd.port.mk will set DEPRECATED and EXPIRATION_DATE (as explained above)

according to forbidden runtime dependencies. AFAIK, this is an advisory mechanism only.

Note that 2 and 3 are independent, so either one or both mechanisms can go in.

After discussion with bapt@, I understood that some were considering that adding this feature was
perhaps overkill, and that using the already existing RESTRICTED mechanism instead could solve the
problem of build-only dependencies. Why not, but I think it may also have some drawbacks, such as not
building a package for the restricted dependency anymore.

In any case, I think the main options I could think of are now on the table.

If 2 does not go in, I would like that the very small changes around the introduction of dep_err
be considered, they are useful on their own. I can open a separate review in this case if you agree.

[bapt]

So as we have discussed on irc, while I do like the proprosal here, I wonder if it will be very useful and I am more and more thinking we should just rely on RESTRICTED for that, meaning the packages will not be distributed, and only people who knows how to use the ports can install them. It also means than anything that just build depends on python2 stuff will still be built on the packages and distributed.

[olce]

In D28946#653518, @bapt wrote:

So as we have discussed on irc, while I do like the proprosal here, I wonder if it will be very useful and I am more and more thinking we should just rely on RESTRICTED for that, meaning the packages will not be distributed, and only people who knows how to use the ports can install them. It also means than anything that just build depends on python2 stuff will still be built on the packages and distributed.

Sure, this is additional infrastructure, but arguably small. I understand that, for ports that are supposed to be build-only dependencies, you may want to set RESTRICTED anyway, in order to prevent package distribution, on the ground that these ports are considered outdated for public use.

I think the approaches here have some advantages still:

1. Making the build fail should be better to prevent ports with unwanted dependencies from slipping in the ports tree, because the effect is very visible. With RESTRICTED, packages for dependent ports (including run-time dependent ones) may still be built, no? In which case there could be situations where packages are publicly distributed in the official repository, but cannot be installed directly because some of their dependencies have no packages there. Users will have no clue about what's going on. Surely, this will be noticed, but with the proposal here, this would be *prevented* instead.
2. Hooking on DEPRECATED and EXPIRATION_DATE allows to fit the changes into existing infrastructure, and probable mechanisms around it (maybe there are some equivalent ones for RESTRICTED, but for example not the expiration date part AFAIK). These variables lead to useful package annotations, and meaningful display, e.g., in FreshPorts.
3. Relying exclusively on RESTRICTED may sound astute, but may also blur the real reason of the package being forbidden, which is not a legal one at all. Also, this means that a legal restriction as well as a dependency one cannot co-exist.
4. The list of ports to use at build only will be centralized. I think this is slightly better because restricting a package is a matter of policy, whereas legalese is port-specific and generally checked by maintainers only.

Please let me know where you're heading to, and if you have other ideas (I could propose implementations, if enough time).