Page MenuHomeFreeBSD

Subpackages. (deal with it.)
Needs ReviewPublic

Authored by mat on Jul 26 2018, 2:38 PM.

Details

Reviewers
bapt
Group Reviewers
portmgr
Summary

Introduce subpackages to the ports framework - this allows ports to be built once, but have multiple packages created. For example, have a subpackage for docs and examples.

To add subpackages to a port, define:

SUBPACKAGES=  foo bar

and in the plist, prefix files to put in a subpackage with @@foo@@.

For each subpackage, some variables can be set, if needed.

  • DESCR.<subpkg> defaults to the pkg-descr if pkg-descr.<subpkg> does not exist.
  • COMMENT.<subpkg> defaults to ${COMMENT} (subpkg: <subpkg>)
  • All the PKGMESSAGE/PKGINSTALL/... variable have .<subpkg> version, pointing to pkg-message.<subpkg>/pkg-install.<subpkg> files.
  • A few options helpers <opt>_SUBPACKAGES and <opt>_*_DEPENDS.<subpkg> can be used. See lang/perl5-devel example.
  • PLIST_FILES.<subpkg> and PLIST_DIRS.subpkg were added too.

Subpackages can depend on each other, a new variable has been introduced, SELF_DEPENDS, and a special subpackage name has been introduced to refer to the main package. For example:

If the foo subpackage depends on the main subpackage:

SUBPACKAGES=  foo
SELF_DEPENDS.foo= main

If the main package depends on the tools subpackage:

SUBPACKAGE=   tools
SELF_DEPENDS=  tools
Test Plan

EVERYTHING IS FINE.

You will probably need a patched poudriere, patches from either the branch or the pull request

Diff Detail

Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 31581
Build 29177: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
mat updated this revision to Diff 47861.Sep 10 2018, 1:27 PM

rebase.

bapt added a comment.Nov 30 2018, 8:48 AM

I do think that is good enough as a start and that we should go further in an incremental way.
I do like the SELF_DEPENDS thing.
Can we proceed right now?

bapt accepted this revision.Nov 30 2018, 8:48 AM
This revision is now accepted and ready to land.Nov 30 2018, 8:48 AM
brooks added a subscriber: brooks.Nov 30 2018, 5:38 PM

This looks like it will work well with the llvm ports. I'll mostly need to tweak my plist-generator a bit.

lang/perl5-devel/pkg-plist
789

I'm curious why we need both %%GDBM%% and @@gdbm@@.

rene added a subscriber: rene.Dec 2 2018, 8:53 PM
fluffy added a subscriber: fluffy.Dec 4 2018, 2:24 PM
mat marked an inline comment as done.Dec 6 2018, 3:45 PM
In D16457#391200, @bapt wrote:

I do think that is good enough as a start and that we should go further in an incremental way.
I do like the SELF_DEPENDS thing.
Can we proceed right now?

Well, I would like to finish with the php72 port, convert all options to subpackages to make sure I do not need more stuff to handle the few cases I have not done yet.

lang/perl5-devel/pkg-plist
789

%%GDBM%% is there because GDBM support is an option, and @@gdbm@@ because the file gets in a subpackage.

It is still a review because stuff still need to be discussed ;-)

For example, if OPT_SUBPACKAGES is defined, does it mean that files guarded by %%OPT%% automatically get in the subpackage set in there? What if opt_SUBPACKAGES defines more than one subpackage? What if some files from option opt should go in a different subpackage, say the "doc" one?

mat updated this revision to Diff 51771.Dec 9 2018, 10:14 AM

rebase.

This revision now requires review to proceed.Dec 9 2018, 10:14 AM
rene added inline comments.Dec 9 2018, 10:40 AM
lang/perl5-devel/pkg-plist
789

Just some thoughts, feel free to ignore:

  • when defining OPT_SUBPACKAGE (e.g. OPT_gdbm) it would be a nice idea to automatically move the %%GDBM%% files into the gdbm subpackage, so e.g. %%gdbm%%%%ARCH_LIB%%/GDBM_File.pm (so using lowercase option names to combine the subpackage and the regular option names). This way we can reduce some red tape.
  • When GDBM_File.pm should go into multiple/different subpackages: %%GDBM%%@@doc@@%%ARCH_LIB%%/GDBM_File1.pm %%GDBM%%@@gdbm@@%%ARCH_LIB%%/GDBM_File_Explicit.pm %%gdbm%%%%ARCH_LIB%%/GDBM_File_Implicit.pm

This combining option and subpackage names is not foolproof, there are a few ports with lowercase option names (one qt4/kde4-using port IIRC).

mat marked an inline comment as done.Dec 9 2018, 10:54 AM
mat added inline comments.
lang/perl5-devel/pkg-plist
789

I feel the semantics would get overly complicated and confuse people.

Options MUST be uppercase, I think there is a QA check for this. I do not remember finding ports with lowercase options, but in any case, they should be fixed.

rene added inline comments.Dec 9 2018, 11:11 AM
lang/perl5-devel/pkg-plist
789

Yes, probably better to leave the option and subpackage name split.

It was the QA check that notified me of that port ;)

rene added a comment.Dec 17 2018, 6:33 PM

I suggest to wait for 2019Q1, then:

  • update proudriere(-devel) with @mat 's patch
  • update user/developer documentation
  • rebase the patch
  • commit the patch
  • send out some emails to announce subpackages
mat added a comment.Dec 18 2018, 8:46 AM
In D16457#396393, @rene wrote:

I suggest to wait for 2019Q1, then:

  • update proudriere(-devel) with @mat 's patch
  • update user/developer documentation
  • rebase the patch
  • commit the patch
  • send out some emails to announce subpackages

Mmmm, you assume the patch has not changed in the past few months because it is ready, it is most probably not. It has not changed because I do not have had the time to work on it any more.

rene added a comment.Dec 18 2018, 6:28 PM

Ah, I was too optimistic :(

tcberner added inline comments.Feb 3 2019, 9:04 AM
Mk/bsd.port.mk
2632

^ flavors sounds wrong here.

2653

^ I would keep it afterwards and add a DEV_WARNING

2657

How about adding a bsd.subpackages.commentsuffix.mk ala bsd.options.desc.mk:

COMMENTSUFFIX.debug?=           Debug Parts
COMMENTSUFFIX.doc?=             Documentation
COMMENTSUFFIX.l10n?=            Localization Files

And then do something like:

.    if defined(COMMENTSUFFIX.${p})
_COMMENTSUFFIX?=       ${COMMENTSUFFIX.${p}}
.    else
_COMMENTSUFFIX?=       ${p} subpackage
.    endif
COMMENT.${p}?= ${COMMENT} (${_COMMENTSUFFIX})
. endfor
.endif
mat marked an inline comment as done.Feb 4 2019, 2:08 PM
mat added inline comments.
Mk/bsd.port.mk
2632

Ah, yes, too much copy&paste ^^

2653

Mmmm. You mean a DEV_WARNING when the subpackage does not have a specific description file?

2657

First, I think bsd.options.desc.mk should die, preferably in a large bonfire. Because it's all filled in with:

<foo>_DESC= <foo> support

Which is mostly useless to end users, because it does not tell them anything. Of course option foo as something to do with foo being added. But it does not tell the users what foo is, or what adding foo support actually does.

For example, if a port has the ICU option, the user will be shown "Unicode support via ICU", but what does it mean for the port? Does it mean that without it, the port will only accept 7 bit ASCII text input?

So, while your three examples are ok, I feel having this would end up as the options, and not saying much of what's in it. Or it would need to require some sort of approval to add to, unlike options.desc right now.

Also, I feel adding too many helpers defeats the purpose of having everything customizable.

(Side note, your example can be written as two lines:

COMMENTSUFFIX.${p}?=       ${p} subpackage
COMMENT.${p}?= ${COMMENT} (${COMMENTSUFFIX.${p}})

)

mat updated this revision to Diff 53920.Feb 14 2019, 3:02 PM
mat marked an inline comment as done.

rebase & add git subpackages

mat updated this revision to Diff 58893.Jun 22 2019, 9:31 AM

rebase

mat updated this revision to Diff 58894.Jun 22 2019, 11:13 AM

rebase

mat updated this revision to Diff 64226.Nov 12 2019, 11:58 AM

rebase.

mat updated this revision to Diff 64321.Nov 14 2019, 2:40 PM

Small fix in:

  • Strip subpackage markers from the plist. (May not be correct.)

Rebase

mat updated this revision to Diff 64322.Nov 14 2019, 2:47 PM
  • Use sed instead of grep here, grep exists if it does not output
mat updated this revision to Diff 64324.Nov 14 2019, 3:01 PM

Fixup in

  • Start getting php72 to use subpackages.

@mat is this stable enough to start play around with? Because for example Qt has many split ports that could be used as testing ground too (basically all the ports that have the same qt-dist value).

mat added a comment.Nov 25 2019, 6:18 PM

@mat is this stable enough to start play around with? Because for example Qt has many split ports that could be used as testing ground too (basically all the ports that have the same qt-dist value).

Yes, this is definitively a good idea to play around with it. There are still a few things that need fixing/doing, but I doubt the way it works will change.

Note that you need a patch for poudriere to play with subpackages :-)

mat updated this revision to Diff 67393.Jan 28 2020, 11:17 AM

rebase

mat updated this revision to Diff 67410.Jan 28 2020, 1:38 PM
  • Add automatic metapkg support for Python flavors.
mat marked 7 inline comments as done.Jan 28 2020, 1:42 PM
mat updated this revision to Diff 67421.Jan 28 2020, 2:36 PM
  • fixup! Add automatic metapkg support for Python flavors.
  • fixup! Add automatic metapkg support for Python flavors.
mat updated this revision to Diff 67446.Jan 29 2020, 1:33 PM
  • Sanitize environment here, sometime, stuff leaks from qa.sh and breaks
  • fixup! Add automatic metapkg support for Python flavors.
mat updated this revision to Diff 67448.Jan 29 2020, 2:05 PM

Rebase & remove php.

mat updated this revision to Diff 67466.Jan 29 2020, 6:05 PM
  • Ajouter une variable PKGNAME.<subpkg>.
lwhsu added a subscriber: lwhsu.Jan 30 2020, 3:09 AM
mat updated this revision to Diff 67504.Jan 30 2020, 10:52 AM
  • Ajouter une variable PKGNAME.<subpkg>.
  • Add PKGNAMES that has the main package name, and subpackage names.

@mat
Can we add this (from https://wiki.freebsd.org/Ports/FlavorsAndSubPackages)

Sub Packages
Build the port once and create multiple packages. For example, have a sub package for docs and examples.

or some similar description to summary for people who aren't familiar with subpackages.
Thanks,
Nick

0mp added a subscriber: 0mp.Feb 12 2020, 11:50 AM
mat edited the test plan for this revision. (Show Details)May 13 2020, 7:41 AM
mat updated this revision to Diff 72856.Jun 9 2020, 12:03 PM

rebase.

0mp awarded a token.Jun 9 2020, 12:11 PM
woodsb02 edited the summary of this revision. (Show Details)Jun 9 2020, 3:35 PM