After discussing with the team, we're going to go ahead and abandon this revision. While we wanted to see something usable happen in this space, it's just not cost or time effective to spend far more effort fighting politics than the actual implementation took. We will continue using a non-package solution for our products for the future.

Update to the 'base-update' utility:

In D20394#452073, @adamw wrote:

In D20394#451916, @kmoore wrote:

I'll be happy to back out that bit if its a huge concern, but let me first lay out the scenario we're trying to solve, and I'm curious to know if there's some better way to solve this that we've been unaware of for years.

Hi Kris,

I don't personally understand your use-case yet, but that's okay. We can spend some more time brainstorming and implementing an approach together.

For now though, I suggest that you back that part out. Let's keep this review just about building base pkgs, and move that to a separate review. Base pkgs and portstree pkgs shouldn't be dependent on each other. Is that ok?

In D20394#452020, @swills wrote:

In D20394#452005, @kmoore wrote:

In D20394#452000, @swills wrote:

Another issue to consider, -devel in package/port name already means something else in ports (development version), so perhaps a different name for those is needed. Personally, I dislike not having all of a thing when I install it and I don't like getting errors when I go to build something because I have to install headers, so having headers in a separate package isn't something I'm terribly fond of, but if it's really required maybe it is OK.

100% agree, I find it somewhat confusing as well. Here's the possible names I can think of:

"-compiler"
"-buildtools"
"-devtools"
"-development"

I don't really have any strong preference for one vs another, any thoughts on your end which makes the most sense?

And yes, this single package includes the compiler/linker bits, along with headers and other files required to do building on FreeBSD. Single package to enable the build system in its entirety.

-devtools might be OK.

Remove 'os' from SUBDIR+=
Remove 'ports/' port
Rename userland-devel -> userland-devtools

In D20394#452000, @swills wrote:

Another issue to consider, -devel in package/port name already means something else in ports (development version), so perhaps a different name for those is needed. Personally, I dislike not having all of a thing when I install it and I don't like getting errors when I go to build something because I have to install headers, so having headers in a separate package isn't something I'm terribly fond of, but if it's really required maybe it is OK.

In D20394#451999, @swills wrote:

In D20394#451916, @kmoore wrote:

Adam,

I'll be happy to back out that bit if its a huge concern, but let me first lay out the scenario we're trying to solve, and I'm curious to know if there's some better way to solve this that we've been unaware of for years.

First of all, this isn't for developers, this is looking at things with my "end-user" hat on. We've for years had a mantra that you don't mix ports and packages. Packages tend to lag too far behind ports (for understandable reasons, takes time to build them after all).

We have separate branches so we have quarterly "stable" packages, not because of build time. You can switch to latest packages if you want or the quarterly branch of ports if you want and build the packages you need from there.

I'll be happy to back out that bit if its a huge concern, but let me first lay out the scenario we're trying to solve, and I'm curious to know if there's some better way to solve this that we've been unaware of for years.

Updated patch to include 'os/ports' port as well. This is something I've personally wanted in FreeBSD for a LONG time. The ability to 'pkg install ports' and get a ports tree installed which exactly matches the packages I'm running. Over the years its been very frustrating to need to rebuild something from ports, only to find out that the tree I've grabbed from portsnap, git or subversion doesn't match, and my build is now trying to upgrade things I didn't want to upgrade.

Update diff, remove a src-head/ port which was added by mistake from a previous git branch.

Expand the COMMENT sections and pkg-descr files thanks to feedback from AdamW.

This update breaks the userland package down into some smaller sub-packages. This addresses concerns that 1-2 monolithic userland packages are "too big" to download on slow connections, especially in the case of just a single file update. The new compressed package sizes (on HEAD/amd64) are roughly as follows:

In D20394#449200, @adamw wrote:

So, I'm just going to throw this out here. This is just a neat feature of the ports tree that the src framework might choose to leverage.

Many src OPTIONS will necessarily depend upon each other. For instance, enabling ZFS but not CDDL will (afaik) not produce working zfs. Similarly, TELNET without INET will not produce a working telnet. I know that there are a great number of options here, and that this would add some complexity, but it might save some users from shooting themselves in the foot.

We have a syntax that specifies that one option requires another, or that enabling one option should prevent another option from being enabled.

ZFS_IMPLIES=         CDDL
SENDMAIL_IMPLIES=    MAIL MAILWRAPPER INET

Here, if SENDMAIL is enabled, once the options form is submitted, MAIL, MAILWRAPPER, and INET will be automatically enabled.

SENDMAIL_PREVENTS=    RATIONAL_THOUGHT

Here, if SENDMAIL and RATIONAL_THOUGHT are both enabled when the options form is submitted, the build will die with a (configurable) error explaining that those options are incompatible.

Address a few more comments from Adamw

Adam - Updated the patch to reflect your comments. Appreciate the review.

This diff brings in some of the latest changes as it relates to base packages. Specifically:

In D20394#447608, @bapt wrote:

What I think is the matter of how the base system is turned into packages should be a project decision, and yes I would like to see some broader discussion about before adding all of this in the ports tree. Further more it adds lots of maintenance to portmgr to the tree. if you were to compare it with something it would be comparable to the base/ fake category. And I as the one who added it in the ports would gladly remove it if that is what is proposed. With a side note on this case, that it was added to avoid duplication with other binutils and gcc ports in the first place and as always been thought to only concern those 2 things.

So, what I'm hearing from this most recent comment, is that because there's an alternative implementation in src, we shouldn't be allowed to explore and/or use other implementations? Is this the new rule for FreeBSD ports? If so, should we start removing all the alternative implementations of other things from ports as well? We could greatly simplify if we reduced down to a single supported implementation of other things, such as a compiler, sound-system, desktop, browser, SSL, chat program, etc.

We've had a chance internally to have some further discussion around this topic. Before proceeding on any changes to the port distfiles mechanisms we feel it may be necessary to have some conversations about potential drawbacks and chains of trust.

I'm going to have some cycles to work on this in the next few days. What I'm seeing and hearing is that the #1 objection is to the lack of distfiles which is outside the norm of ports. There are some process concerns as well about portmgr "shipping" base bits, which is understandable, however for the project to evolve and grow we're going to have to tackle that in the future anyway.

In D20394#440452, @antoine wrote:

A few questions:

• there are 0 integrity check for os ports, while all non-os ports do check the SHA-256 of their distfiles, is this intended?

Exp-run requested:

Included in: https://reviews.freebsd.org/D20394

Ha, I guess you've not been following the threads about our pkg-base implementation. ;) We're in process of trying to upstream our package base solution that we feel is more viable than 'make packages' for a lot of reasons.

Cleaned up after some feedback from Mat. Using if/else logic to be neater now.

Looks good, thanks Sean!

-Gentle nudge- Would be nice to get this reviewed since it is a vital (haha) part of our package base solution:

Looks pretty good here. I'm fine with a merge once you get an answer on USES=kmod:debug.

I'm going to close this for now. With the incoming changes to package base this may become unnecessary moving forward. We'll maintain the local patch in TrueOS for the time being until we drop it there also.

Just bumped the date on .Dd, thanks @bcr