Page MenuHomeFreeBSD

mail/*notmuch: take maintainership, update to 0.26 & refactor
ClosedPublic

Authored by seschwar_gmail.com on Jan 16 2018, 10:39 PM.

Details

Summary
  • Take maintainership of mail/notmuch and mail/py-notmuch.
  • Update ports to version to 0.26.
  • Unconditionally install completions according to Porter's Handbook Chapter 6.31. without adding extra dependencies.
  • Turn on the MANPAGES option by default. Building the manual pages is kept as an option to allow for more minimal custom builds.
  • Remove RUBY option as it never did anything. There are no build or install instructions in the Makefile nor %%RUBY%% references in pkg-plist. Building with the RUBY option enabled makes absolutely no difference to the produced package. The only thing ever requiring the Ruby bindings to the Notmuch library is the Vim client, which isn't built either.
  • Turn mail/py-notmuch into a slave port of mail/notmuch.
  • Split off EMACS option into the flavor aware slave port mail/notmuch-emacs.
  • Split off MUTT option into the slave port mail/notmuch-mutt.
  • Perform miscellaneous cleanups.
Test Plan
$ cd /usr/ports/mail/notmuch
$ portlint -AC
WARN: Makefile: for new port, make $FreeBSD$ tag in comment section empty, to make SVN happy.
0 fatal errors and 1 warning found.
$ cd /usr/ports/mail/py-notmuch
$ portlint -AC
WARN: Makefile: for new port, make $FreeBSD$ tag in comment section empty, to make SVN happy.
0 fatal errors and 1 warning found.

The outputs of poudriere testport on FreeBSD:11:i386 and FreeBSD:11:amd64 are clean.

Diff Detail

Lint
No Linters Available
Unit
No Unit Test Coverage
Build Status
Buildable 15048
Build 15146: arc lint + arc unit

Event Timeline

seschwar_gmail.com edited the summary of this revision. (Show Details)Jan 16 2018, 10:45 PM

Looks good, but why optional [bash|zsh]-completion turned to unconditional? And may be convert all to master/slave while here?

mat added a comment.Jan 17 2018, 11:24 AM

From 6.31. Shell Completion Files:

When available, completion files should always be installed. It is not necessary to make an option for it. If an option is used, though, always enable it in OPTIONS_DEFAULT.

While I like having them always installed, having a runtime dependency on bash-completions, and so, bash, is a bad idea. The completions files need to be there so that people who need them have them, but you should never forcefully install the shell they need.

mail/notmuch/Makefile
39

That does not exist.

mat added a comment.Jan 17 2018, 11:36 AM

Also, could you use devel/arcanist, or at least generate a diff with full context like it does, with svn diff -x -U9999 or git diff -U9999.

In D13944#292527, @mat wrote:

While I like having them always installed, having a runtime dependency on bash-completions, and so, bash, is a bad idea.

I agree. That's why I put bash-completion (and therefore bash) in BUILD_DEPENDS. So it shouldn't be a runtime dependency, right?

seschwar_gmail.com marked an inline comment as done.
seschwar_gmail.com added inline comments.
mail/notmuch/Makefile
39

Ha, I assumed the OPTION_VARIABLE pattern applied here too.

seschwar_gmail.com marked 2 inline comments as done.Jan 17 2018, 7:09 PM
In D13944#292530, @mat wrote:

Also, could you use devel/arcanist, or at least generate a diff with full context like it does, with svn diff -x -U9999 or git diff -U9999.

I did read about these things in the past but forgot about them. Thanks for reminding me.

seschwar_gmail.com added a comment.EditedJan 17 2018, 7:28 PM

Looks good, but why optional [bash|zsh]-completion turned to unconditional?

Because I think having two options for just two files without dependencies adds more work during maintenance and build than it creates actual benefit.

FreeBSD's Porter's Handbook agrees with me on that:

When available, completion files should always be installed. It is not necessary to make an option for it. If an option is used, though, always enable it in OPTIONS_DEFAULT.

And may be convert all to master/slave while here?

I've been thinking about that and had asked on #bsdports and #freebsd-ports in the past. Someone suggested rather than having master/slave ports both ports should just have the same maintainer. I wasn't sure about that. I can turn mail/py-notmuch into a slave port of mail/notmuch. Should we do this in this revision here or in another one? Also when Emacs flavors are introduced we might want to place the Emacs files in a slave port too, maybe the mutt, ruby and vim parts as well. That way each component could be built separately and installed with pkg from the official repositories. So how far do you want me to take that master/slave port thing?

Merge DESKTOP into EMACS option

desktop-file-utils' dependencies are already covered by
gmime26, so this only adds a very small dependency.

seschwar_gmail.com edited the summary of this revision. (Show Details)Jan 17 2018, 10:10 PM
seschwar_gmail.com edited the test plan for this revision. (Show Details)

Update list of failing tests to skip

Fix path to GDB for tests

This allows for more tests to pass, provided the sysctl
security.bsd.unprivileged_proc_debug is set to 1.

And may be convert all to master/slave while here?

I've been thinking about that and had asked on #bsdports and #freebsd-ports in the past. Someone suggested rather than having master/slave ports both ports should just have the same maintainer. I wasn't sure about that.

Ask both maintainers to pass maintanership for you as more interested person. Or, at least, for implicit agreement for future updates.

I can turn mail/py-notmuch into a slave port of mail/notmuch. Should we do this in this revision here or in another one? Also when Emacs flavors are introduced we might want to place the Emacs files in a slave port too, maybe the mutt, ruby and vim parts as well. That way each component could be built separately and installed with pkg from the official repositories. So how far do you want me to take that master/slave port thing?

Plan it for next steps: *update*; *split*; *more flavors*;
Requested maintainership will be a nice step before splitting.

mat added a comment.Jan 18 2018, 1:42 PM
In D13944#292527, @mat wrote:

While I like having them always installed, having a runtime dependency on bash-completions, and so, bash, is a bad idea.

I agree. That's why I put bash-completion (and therefore bash) in BUILD_DEPENDS. So it shouldn't be a runtime dependency, right?

I did not see it was only a BUILD_DEPENDS, sorry about that.

Is it required for the completion to be installed ?

Unconditionally install completions according to Porter's Handbook Chapter 6.31.. This adds extra build, but no run dependencies.

Option there exactly because of extra dependency (I dislike extra dependencies), one solution is to install completion manually unconditionally with port infrastructure.

A program without any documentation is worthless. This adds extra build, but no run dependencies.

Actually it requires more than one build - you need python itself for it, and TBH, I assume everyone has internet and everyone can read docs/tutorials online w/o need for python to be built - that was original thought. Although it doesn't matter for pkg install'ation. I think that if man page requires something else to be built it should stay an option.

Rename DOXYGEN option to MAN3 (like Perl or OpenSSL) to make its effect more clear.

"Doxygen" is a name for which ports infrastructure has *_DESC variable, so somehow "default" variable for such situations when we need to build API docs.

I assume you optimize "pkg install" situations, since build deps are not probllem for you, original idea was to optimize ports usage situations.

In D13944#292934, @mat wrote:

Is it required for the completion to be installed ?

There's some auto-detection in notmuch's the build system (see below for an alternative).

Option there exactly because of extra dependency (I dislike extra dependencies),

But it's only a build and not a runtime dependency.

one solution is to install completion manually unconditionally with port infrastructure.

Indeed, that would work.

Actually it requires more than one build - you need python itself for it, and TBH, I assume everyone has internet and everyone can read docs/tutorials online w/o need for python to be built - that was original thought. Although it doesn't matter for pkg install'ation. I think that if man page requires something else to be built it should stay an option.

I did a clean build using Poudriere on FreeBSD:11:i386. Here are the build times per port in seconds. The full build takes 6793 seconds. All the Python ports together required for building the manual pages take 443 seconds, which is just 6% of the total build time. In comparison CMake (required for building indirect dependencies of GMime) by itself takes 1127 seconds (16%). So honestly I think trying to avoid building the dependencies for the manual pages at any cost isn't warranted.

Also I thought FreeBSD cared about documentation. As a user I don't want to open a web browser then search for and open notmuch's online manual pages. Typing man notmuch-search is a lot more convenient that clicking around for several tens of seconds. I'd want the documentation available at my fingertips and not having to Google for everything.

Also using --help on notmuch's subcommands -- like notmuch search --help -- opens the corresponding manual pages. There is no other --help output for them. So missing manual pages really hurt there.

"Doxygen" is a name for which ports infrastructure has *_DESC variable, so somehow "default" variable for such situations when we need to build API docs.

Some ports use DOXYGEN to build HTML documentation, some build man3 stuff. I thought a less ambiguous name would be nice. But we could keep the name just as well.

I assume you optimize "pkg install" situations, since build deps are not probllem for you, original idea was to optimize ports usage situations.

I was thinking of both. I did add some extra dependencies but still tried to not add too much. Your current port while nice for building from source doesn't produce a very useful package for pkg.

Maybe we could satisfy both cases by keeping the options but having them turned on by default. That way the package for the official repository would be built with them, but people building from source could turn them off to get a more minimal result.

Ask both maintainers to pass maintanership for you as more interested person. Or, at least, for implicit agreement for future updates.

As @mp39590_gmail.com, the maintainer of mail/notmuch has joined this discussion I can just ask them directly: @mp39590_gmail.com, do you want to keep maintaining mail/notmuch or do you want me (or someone else like maybe @github_lostpackets.de) take over maintainership? What do you think of turning mail/py-notmuch into a slave port of mail/notmuch and possibly splitting off the Emacs parts into another slave port in the future? As per prior discussion @github_lostpackets.de, the maintainer of mail/py-notmuch seems to agree on turning it into a slave port. @github_lostpackets.de, how do you feel about consolidating maintainership of the notmuch ports?

It's OK from me to pass maintainership.

I am happy to pass the maintainership along (and from my side the current patch looks totally fine (for mail/py-notmuch).

mat added a comment.Jan 24 2018, 1:22 PM

With the emacs flavors coming along, it would probably be a good idea to split the emacs bits out in a slave ports right now.

It's OK from me to pass maintainership.

I am happy to pass the maintainership along (...).

Then I shall take maintainership of both ports.

  • Take maintainership of both ports.
  • Be a bit more conservative with the changes to the port.
    • Keep the MANPAGES option, but turn it on by default.
    • Revert the rename of the DOXYGEN option to MAN3
    • Keep the DESKTOP option separate from EMACS from now. This might get reworked when Emacs flavors land.
  • Perform miscellaneous cleanups.
    • Drop unneeded binutils dependency.
    • Expand pkg-descr.
    • Minor Makefile cleanups.
  • Install Bash completions by default without requiring any extra dependencies.
seschwar_gmail.com retitled this revision from mail/*notmuch: update to 0.26 to mail/*notmuch: take maintainership and update to 0.26.Jan 27 2018, 11:38 AM
seschwar_gmail.com edited the summary of this revision. (Show Details)
In D13944#294549, @mat wrote:

With the emacs flavors coming along, it would probably be a good idea to split the emacs bits out in a slave ports right now.

I can do that, but personally I would prefer to do that in separate followup revision.

  • mail/notmuch: skip fewer tests
  • New port: mail/notmuch-emacs split from mail/notmuch
  • mail/notmuch: prevent poudriere testport from complaining
  • mail/py-notmuch: turn into a slave port of mail/notmuch
  • mail/notmuch-emacs: turn into a slave port of mail/notmuch
  • New port: mail/notmuch-mutt split into slave port off mail/notmuch
  • mail/*notmuch*: miscellaneous cleanups
  • mail/py-notmuch: add option to install documentation
  • mail/notmuch-emacs: add DOCS option to install info file
seschwar_gmail.com retitled this revision from mail/*notmuch: take maintainership and update to 0.26 to mail/*notmuch: take maintainership, update to 0.26 & refactor.Feb 4 2018, 12:30 PM
seschwar_gmail.com edited the summary of this revision. (Show Details)

Since the Emacs flavors have landed in the ports tree I reworked the port. I've split off parts into slave ports. The documentation for that topic is rather limited, so I'm not sure if I did it correctly. Also mail/notmuch-emacs needed a bit of convincing for it to be build separately. Everything seems to work, but there are a few warnings:

$ portlint -AC mail/notmuch
WARN: Makefile: unless this is a master port, COMMENT has to be set by "=", not by "?=".
0 fatal errors and 1 warning found.

Well, it is a master port.

$ portlint -AC mail/notmuch-mutt
FATAL: Makefile: extra item "RUN_DEPENDS" placed in the LICENSE section.
1 fatal error and 0 warnings found.
$ portlint -AC mail/py-notmuch
FATAL: Makefile: extra item "LIB_DEPENDS" placed in the LICENSE section.
1 fatal error and 0 warnings found.

The slave ports don't have a LICENSE section. That information is in the master port. Is this a problem?

$ portlint -AC mail/notmuch-emacs
FATAL: Makefile: the last line of a slave port's Makefile has to be .include "${MASTERDIR}/Makefile"
FATAL: Makefile: extra item "RUN_DEPENDS" placed in the LICENSE section.
2 fatal errors and 0 warnings found.

That adjustment has to be made there since Mk/Uses/emacs.mk sets EMACS=${EMACS_CMD}, but we need EMACS=${EMACS_CMD} --quick for the build.

poudriere testport with USE_PORTLINT=yes shows some additional messages:

make: "/usr/local/poudriere/data/.m/FreeBSD:11:amd64-default/ref/usr/ports/Mk/bsd.port.mk" line 3408: warning: duplicate script for target "/usr/local/poudriere/data/.m/FreeBSD" ignored
make: "/usr/local/poudriere/data/.m/FreeBSD:11:amd64-default/ref/usr/ports/Mk/bsd.licenses.mk" line 742: warning: using previous script for "/usr/local/poudriere/data/.m/FreeBSD" defined here
(...)
FATAL: breaks INDEX (make: "/usr/local/poudriere/data/.m/FreeBSD:11:amd64-default/ref/usr/ports/Mk/bsd.port.mk" line 3408: warning: duplicate script for target "/usr/local/poudriere/data/.m/FreeBSD" ignored make: "/usr/local/poudriere/data/.m/FreeBSD:11:amd64-default/ref/usr/ports/Mk/bsd.licenses.mk" line 742: warning: using previous script for "/usr/local/poudriere/data/.m/FreeBSD" defined here (...)
).

I don't know what this means.

Turns out the FATAL: breaks INDEX message also appears in the output of poudriere testport ports-mgmt/pkg on the latest revision of the ports tree (rP460947). So I guess it's not an issue with my ports.

It turns out the FATAL: breaks INDEX message was a poudriere issue with my jail naming and not at all related to this port.

  • mail/*notmuch*: use MAKE_CMD instead of MAKE or GMAKE
  • mail/*notmuch*: silence commands not installing files
  • mail/py-notmuch: use PORTDOCS instead of DOCS_PORTDOCS
  • mail/py-notmuch: don't install Sphinx' .buildinfo file

Fixup previous update

mail/notmuch-emacs: place variables before targets

@fluffy and @mat, can I get feedback on this so it may get committed eventually?

@fluffy and @mat, can I get feedback on this so it may get committed eventually?

I'll ship it after my poudriere will complete build, if @mat have no objections.

@mat didn't voice any objections. Will you commit this now, @fluffy?

This revision was not accepted when it landed; it landed in state Needs Review.Feb 27 2018, 5:01 AM
This revision was automatically updated to reflect the committed changes.

Thanks a lot, everyone. With this PR#222360 can be closed as well.