Page MenuHomeFreeBSD

mail/notmuch: Update to version 0.25
ClosedPublic

Authored by jrm on Jul 27 2017, 7:32 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Nov 14, 5:06 AM
Unknown Object (File)
Sun, Nov 10, 7:03 AM
Unknown Object (File)
Sat, Nov 9, 5:22 PM
Unknown Object (File)
Sun, Nov 3, 1:05 PM
Unknown Object (File)
Oct 21 2024, 11:19 PM
Unknown Object (File)
Oct 13 2024, 10:55 PM
Unknown Object (File)
Oct 10 2024, 11:09 AM
Unknown Object (File)
Oct 6 2024, 1:53 AM
Subscribers

Details

Summary

Also:

  • Add DESKTOP option to explicitly control installation of the desktop file. Emacs may be installed without desktop-file-utils.
  • Change USE_PERL to USES=perl5.
Test Plan

portlint -C: looks fine
poudriere testport with various options under 10/11 i386/amd64

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Lint Coverage
Unit
No Test Coverage
Build Status
Buildable 10986
Build 11373: arc lint + arc unit

Event Timeline

Can you update NOTMUCH_SKIP_TESTS variable after "make test" passage? - it should be almost empty, if not removed at all.
Bash completion path should stay - prefix/share is for shareable content, as functions.

This revision now requires changes to proceed.Aug 9 2017, 4:18 PM

Can you update NOTMUCH_SKIP_TESTS variable after "make test" passage? - it should be almost empty, if not removed at all.

Could you be more specific? I'm not sure exactly what you are looking for.

Bash completion path should stay - prefix/share is for shareable content, as functions.

This wasn't my change, but upstream's. If you want it to be placed in the old directory we'll have to patch. If I look on my systems, under share/ is where many completions are installed. For example I have completions under /usr/local/share/zsh/5.3.1/functions/Completion/Zsh and /usr/local/share/bash-completion/completions.

% ls -lah /usr/local/share/bash-completion/completions
total 32
drwxr-xr-x  2 root  wheel     7B Aug  4 08:07 .
drwxr-xr-x  3 root  wheel     3B May 30 08:30 ..
-rw-r--r--  1 root  wheel   1.4K Jul 17 12:54 gapplication
-rw-r--r--  1 root  wheel   935B Jul 17 12:54 gdbus
-rw-r--r--  1 root  wheel   1.3K Jul 17 12:54 gresource
-rw-r--r--  1 root  wheel   2.6K Jul 17 12:54 gsettings
-rw-r--r--  1 root  wheel   6.4K Aug  4 07:51 libreoffice.sh
In D11752#247760, @jrm wrote:

Can you update NOTMUCH_SKIP_TESTS variable after "make test" passage? - it should be almost empty, if not removed at all.

Could you be more specific? I'm not sure exactly what you are looking for.

This port has test infrastructure (TEST_* vars), you can run tests with 'make test' in /usr/ports/mail/notmuch - after that notmuch will be built and tests will be run, some tests weren't working in previous versions, but 0.25 contains a patch, which should fix most of the cases, I would start by removing all skipped tests and testing if they're all passed - then add not passed tests to NOTMUCH_SKIP_TESTS var.

In the past tests were not passed because of linuxism inside, one of the test (can't remember exactly which one) didn't pass because of unknown error in sorting.

Can you run 'make test' in ports dir to determine which tests must be skipped now? I have no access to a server, and can't compile anything at all at that moment.

If it is too much work - let the list to be the same.

Bash completion path should stay - prefix/share is for shareable content, as functions.

This wasn't my change, but upstream's. If you want it to be placed in the old directory we'll have to patch. If I look on my systems, under share/ is where many completions are installed. For example I have completions under /usr/local/share/zsh/5.3.1/functions/Completion/Zsh and /usr/local/share/bash-completion/completions.

Then no need to change anything, I think patch will be overengineering in that case.

This revision is now accepted and ready to land.Aug 9 2017, 5:59 PM

I went through the various tests and these ones cause testing to stop and a Makefile error is reported.
address basic count crypto db-features database-modified dump-restore emacs emacs-large-search-buffer from-guessing insert json lib-error new parse-time-string python search search-output sexp smime symbol-hiding tagging thread-order thread-search

Even with all these tests added to NOTMUCH_SKIP_TESTS, make test ends with an error.

Notmuch test suite complete.
494/500 tests passed.
3 broken tests failed as expected.
3 tests failed.
gmake[1]: *** [test/Makefile.local:71: test] Error 1
gmake[1]: Leaving directory '/wrkdirs/usr/ports/mail/notmuch/work/notmuch-0.25'
*** Error code 1

Stop.
make: stopped in /usr/ports/mail/notmuch

This is the full output of make test with the latest diff. Are you OK with this?

INFO: using 2 minute timeout for tests

T000-basic: Testing the test framework itself.
 skip all tests in T000-basic
T010-help-test: Testing online help
 PASS   notmuch --help
 PASS   notmuch help
 PASS   notmuch --version
 PASS   notmuch --help tag
 PASS   notmuch help tag

T020-compact: Testing "notmuch compact"
 PASS   Running compact
 PASS   Compact preserves database
 PASS   Restoring Backup
 PASS   Checking restored backup

T030-config: Testing "notmuch config"
 PASS   Get string value
 PASS   Get list value
 PASS   Set string value
 PASS   Set string value again
 PASS   Set list value
 PASS   Set list value again
 PASS   Remove key
 PASS   Remove non-existent key
 PASS   List all items
 PASS   Top level --config=FILE option
 PASS   Top level --config:FILE option
 PASS   Top level --config<space>FILE option
 PASS   Top level --config=FILE option changed the right file
 PASS   Read config file through a symlink
 PASS   Write config file through a symlink
 PASS   Writing config file through symlink follows symlink

T040-setup: Testing "notmuch setup"
 PASS   Notmuch new without a config suggests notmuch setup
 PASS   Create a new config interactively

T050-new: Testing "notmuch new" in several variations
 skip all tests in T050-new
T060-count: Testing "notmuch count" for messages and threads
 skip all tests in T060-count
T070-insert: Testing "notmuch insert"
 skip all tests in T070-insert
T080-search: Testing "notmuch search" in several variations
 skip all tests in T080-search
T090-search-output: Testing various settings for "notmuch search --output="
 skip all tests in T090-search-output
T095-address: Testing "notmuch address" in several variants
 skip all tests in T095-address
T100-search-by-folder: Testing "notmuch search" by folder: and path: (with variations)
 PASS   Single-world folder: specification (multiple results)
 PASS   Top level folder
 PASS   Two-word path to narrow results to one
 PASS   Folder search with --output=files
 PASS   After removing duplicate instance of matching path
 PASS   Folder search with --output=files part #2
 PASS   After removing duplicate instance of matching path part #2
 PASS   After rename, old path returns nothing
 PASS   After rename, new path returns result
 PASS   folder: search
 PASS   top level folder: search
 PASS   path: search
 PASS   top level path: search
 PASS   recursive path: search

T110-search-position-overlap-bug: Testing that notmuch does not overlap term positions
 PASS   Search for a@b.c matches
 PASS   Search for x@y.z matches
 PASS   Search for a@y.c must not match

T120-search-insufficient-from-quoting: Testing messages with unquoted . in name
 PASS   Search by first name
 PASS   Search by last name:
 PASS   Search by address:
 PASS   Search for all messages:

T130-search-limiting: Testing "notmuch search" --offset and --limit parameters
 PASS   messages: limit does the right thing
 PASS   messages: concatenation of limited searches
 PASS   messages: limit larger than result set
 PASS   messages: limit = 0
 PASS   messages: offset does the right thing
 PASS   messages: offset = 0
 PASS   messages: negative offset
 PASS   messages: negative offset
 PASS   messages: negative offset combined with limit
 PASS   messages: negative offset combined with equal limit
 PASS   messages: negative offset combined with large limit
 PASS   messages: negative offset larger then results
 PASS   threads: limit does the right thing
 PASS   threads: concatenation of limited searches
 PASS   threads: limit larger than result set
 PASS   threads: limit = 0
 PASS   threads: offset does the right thing
 PASS   threads: offset = 0
 PASS   threads: negative offset
 PASS   threads: negative offset
 PASS   threads: negative offset combined with limit
 PASS   threads: negative offset combined with equal limit
 PASS   threads: negative offset combined with large limit
 PASS   threads: negative offset larger then results

T140-excludes: Testing "notmuch search, count and show" with excludes in several variations
 PASS   Search, exclude "deleted" messages from search
 PASS   Search, exclude "deleted" messages from message search
 PASS   Search, exclude "deleted" messages from message search --exclude=false
 PASS   Search, exclude "deleted" messages from message search (non-existent exclude-tag)
 PASS   Search, exclude "deleted" messages from search, overridden
 PASS   Search, exclude "deleted" messages from threads
 PASS   Search, don't exclude "deleted" messages when --exclude=flag specified
 PASS   Search, don't exclude "deleted" messages from search if not configured
 PASS   Search, default exclusion (thread summary)
 PASS   Search, default exclusion (messages)
 PASS   Search, exclude=true (thread summary)
 PASS   Search, exclude=true (messages)
 PASS   Search, exclude=false (thread summary)
 PASS   Search, exclude=false (messages)
 PASS   Search, exclude=flag (thread summary)
 PASS   Search, exclude=flag (messages)
 PASS   Search, exclude=all (thread summary)
 PASS   Search, exclude=all (messages)
 PASS   Search, default exclusion: tag in query (thread summary)
 PASS   Search, default exclusion: tag in query (messages)
 PASS   Search, exclude=true: tag in query (thread summary)
 PASS   Search, exclude=true: tag in query (messages)
 PASS   Search, exclude=false: tag in query (thread summary)
 PASS   Search, exclude=false: tag in query (messages)
 PASS   Search, exclude=flag: tag in query (thread summary)
 PASS   Search, exclude=flag: tag in query (messages)
 PASS   Search, exclude=all: tag in query (thread summary)
 PASS   Search, exclude=all: tag in query (messages)
 PASS   Count, default exclusion (messages)
 PASS   Count, default exclusion (threads)
 PASS   Count, exclude=true (messages)
 PASS   Count, exclude=true (threads)
 PASS   Count, exclude=false (messages)
 PASS   Count, exclude=false (threads)
 PASS   Count, default exclusion: tag in query (messages)
 PASS   Count, default exclusion: tag in query (threads)
 PASS   Count, exclude=true: tag in query (messages)
 PASS   Count, exclude=true: tag in query (threads)
 PASS   Count, exclude=false: tag in query (messages)
 PASS   Count, exclude=false: tag in query (threads)
 PASS   Show, default exclusion
 PASS   Show, default exclusion (entire-thread)
 PASS   Show, exclude=true
 PASS   Show, exclude=true (entire-thread)
 PASS   Show, exclude=false
 PASS   Show, exclude=false (entire-thread)

T150-tagging: Testing "notmuch tag"
 skip all tests in T150-tagging
T160-json: Testing --format=json output
 skip all tests in T160-json
T170-sexp: Testing --format=sexp output
 skip all tests in T170-sexp
T180-text: Testing --format=text output
 PASS   Show message: text
 PASS   Search message: text
 PASS   Show message: text, utf-8
 PASS   Search message: text, utf-8
 PASS   Search message tags: text0
 PASS   Compare text vs. text0 for threads
 PASS   Compare text vs. text0 for messages
 PASS   Compare text vs. text0 for files
 PASS   Compare text vs. text0 for tags

T190-multipart: Testing output of multipart message
 PASS   --format=text --part=0, full message
 PASS   --format=text --part=1, message body
 PASS   --format=text --part=2, multipart/mixed
 PASS   --format=text --part=3, rfc822 part
 PASS   --format=text --part=4, rfc822's multipart
 PASS   --format=text --part=5, rfc822's html part
 PASS   --format=text --part=6, rfc822's text part
 PASS   --format=text --part=7, inline attachement
 PASS   --format=text --part=8, plain text part
 PASS   --format=text --part=9, pgp signature (unverified)
 PASS   --format=text --part=8, no part, expect error
 PASS   --format=json --part=0, full message
 PASS   --format=json --part=1, message body
 PASS   --format=json --part=2, multipart/mixed
 PASS   --format=json --part=3, rfc822 part
 PASS   --format=json --part=4, rfc822's multipart/alternative
 PASS   --format=json --part=5, rfc822's html part
 PASS   --format=json --part=6, rfc822's text part
 PASS   --format=json --part=7, inline attachment
 PASS   --format=json --part=8, plain text part
 PASS   --format=json --part=9, pgp signature (unverified)
 PASS   --format=json --part=10, no part, expect error
 PASS   --format=raw
 PASS   --format=raw --part=0, full message
 BROKEN --format=raw --part=1, message body
        --- T190-multipart.25.multipart_body    2017-08-10 03:03:46.684385000 +0000
        +++ T190-multipart.25.OUTPUT    2017-08-10 03:03:46.685032000 +0000
        @@ -1,3 +1,10 @@
        +From: Carl Worth <cworth@cworth.org>
        +To: cworth@cworth.org
        +Subject: Multipart message
        +Date: Fri, 05 Jan 2001 15:43:57 +0000
        +User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
        +Message-ID: <87liy5ap00.fsf@yoom.home.cworth.org>
        +MIME-Version: 1.0
         Content-Type: multipart/signed; boundary="==-=-=";
                micalg=pgp-sha1; protocol="application/pgp-signature"

 PASS   --format=raw --part=2, multipart/mixed
 PASS   --format=raw --part=3, rfc822 part
 BROKEN --format=raw --part=4, rfc822's multipart
        --- T190-multipart.28.embedded_message_body     2017-08-10 03:03:46.711862000 +0000
        +++ T190-multipart.28.OUTPUT    2017-08-10 03:03:46.712507000 +0000
        @@ -1,3 +1,10 @@
        +From: Carl Worth <cworth@cworth.org>
        +To: cworth@cworth.org
        +Subject: html message
        +Date: Fri, 05 Jan 2001 15:42:57 +0000
        +User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
        +Message-ID: <87liy5ap01.fsf@yoom.home.cworth.org>
        +MIME-Version: 1.0
         Content-Type: multipart/alternative; boundary="==-=-=="

         --==-=-==
 PASS   --format=raw --part=5, rfc822's html part
 PASS   --format=raw --part=6, rfc822's text part
 PASS   --format=raw --part=7, inline attachment
 PASS   --format=raw --part=8, plain text part
 PASS   --format=raw --part=9, pgp signature (unverified)
 PASS   --format=raw --part=10, no part, expect error
 PASS   --format=mbox
 PASS   --format=mbox --part=1, incompatible, expect error
 PASS   'notmuch reply' to a multipart message
 PASS   'notmuch reply' to a multipart message with json format
 PASS   'notmuch show --part' does not corrupt a part with CRLF pair
 PASS   html parts excluded by default
 PASS   html parts included
 PASS   indexes mime-type #1
 PASS   indexes mime-type #2
 PASS   indexes mime-type #3
 PASS   case of Content-Disposition doesn't matter for indexing

T200-thread-naming: Testing naming of threads with changing subject
 PASS   Initial thread name (oldest-first search)
 PASS   Initial thread name (newest-first search)
 PASS   Changed thread name (oldest-first search)
 PASS   Changed thread name (newest-first search)
 PASS   Ignore added reply prefix (Re:)
 PASS   Ignore added reply prefix (Aw:)
 PASS   Ignore added reply prefix (Vs:)
 PASS   Ignore added reply prefix (Sv:)
 PASS   Use empty subjects if necessary.
 PASS   Avoid empty subjects if possible (newest-first).
 PASS   Avoid empty subjects if possible (oldest-first).
 PASS   Test order of messages in "notmuch show"

T205-author-naming: Testing naming of authors with unusual addresses
 PASS   Add author with empty quoted real name

T210-raw: Testing notmuch show --format=raw
 PASS   Attempt to show multiple raw messages
 PASS   Show a raw message
 PASS   Show another raw message

T220-reply: Testing "notmuch reply" in several variations
 PASS   Basic reply
 PASS   Multiple recipients
 PASS   Reply with CC
 PASS   Reply from alternate address
 PASS   Reply from address in named group list
 PASS   Support for Reply-To
 PASS   Un-munging Reply-To
 PASS   Un-munging Reply-To With Exact Match
 PASS   Un-munging Reply-To With Raw addr-spec
 PASS   Message with header of exactly 200 bytes
 PASS   From guessing: Envelope-To
 PASS   From guessing: X-Original-To
 PASS   From guessing: Delivered-To
 PASS   Reply with RFC 2047-encoded headers
 PASS   Reply with RFC 2047-encoded headers (JSON)
 PASS   Reply to a message with multiple Cc headers

T230-reply-to-sender: Testing "notmuch reply --reply-to=sender" in several variations
 PASS   Basic reply-to-sender
 PASS   From Us, Basic reply to message
 PASS   Multiple recipients
 PASS   From Us, Multiple TO recipients
 PASS   Reply with CC
 PASS   From Us, Reply with CC
 PASS   From Us, Reply no TO but with CC
 PASS   Reply from alternate address
 PASS   Support for Reply-To
 PASS   Support for Reply-To with multiple recipients
 PASS   Un-munging Reply-To
 PASS   Message with header of exactly 200 bytes

T240-dump-restore: Testing "notmuch dump" and "notmuch restore"
 skip all tests in T240-dump-restore
T250-uuencode: Testing handling of uuencoded data
 PASS   Ensure content before uu data is indexed
 PASS   Ensure uu data is not indexed
 PASS   Ensure content after uu data is indexed

T260-thread-order: Testing threading when messages received out of order
 skip all tests in T260-thread-order
T270-author-order: Testing author reordering;
 PASS   Adding parent message
 PASS   Adding initial child message
 PASS   Adding second child message
 PASS   Searching when all three messages match
 PASS   Searching when two messages match
 PASS   Searching when only one message matches
 PASS   Searching when only first message matches
 PASS   Adding duplicate author
 PASS   Searching when all four messages match
 PASS   Adding non-monotonic child message
 PASS   Searching non-monotonic messages (oldest-first)
 PASS   Searching non-monotonic messages (newest-first)

T280-from-guessing: Testing From line heuristics (with multiple configured addresses)
 skip all tests in T280-from-guessing
T290-long-id: Testing messages with ridiculously-long message IDs
 PASS   Referencing long ID before adding
 PASS   Adding message with long ID
 PASS   Referencing long ID after adding
 PASS   Ensure all messages were threaded together

T300-encoding: Testing encoding issues
 PASS   Message with text of unknown charset
 PASS   Search for ISO-8859-2 encoded message
 PASS   RFC 2047 encoded word with spaces
 PASS   RFC 2047 encoded words back to back
 PASS   RFC 2047 encoded words without space before or after

T310-emacs: Testing emacs interface
 skip all tests in T310-emacs
T320-emacs-large-search-buffer: Testing Emacs with large search results buffer
 skip all tests in T320-emacs-large-search-buffer
T330-emacs-subject-to-filename: Testing emacs: mail subject to filename
 PASS   no patch sequence number
 PASS   patch sequence number #1
 PASS   patch sequence number #2
 PASS   patch sequence number #3
 PASS   patch sequence number #4
 PASS   patch sequence number #5
 PASS   patch sequence number #6
 PASS   patch sequence number #7
 PASS   filename #1
 PASS   filename #2
 PASS   filename #3
 PASS   filename #4
 PASS   filename #5
 PASS   filename #6
 PASS   filename #7
 PASS   filename #8
 PASS   filename #9
 PASS   patch filename #1
 PASS   patch filename #2
 PASS   patch filename #3
 PASS   patch filename #4

T340-maildir-sync: Testing maildir synchronization
 PASS   Adding 'S' flag to existing filename removes 'unread' tag
 PASS   Adding message with 'S' flag prevents 'unread' tag
 PASS   Adding 'replied' tag adds 'R' flag to filename
 PASS   notmuch show works with renamed file (without notmuch new)
 PASS   notmuch reply works with renamed file (without notmuch new)
 PASS   notmuch new detects no file rename after tag->flag synchronization
 PASS   When read, message moved from new to cur
 PASS   No rename should be detected by notmuch new
 PASS   Adding non-maildir tags does not move message from new to cur
 PASS   Message in cur lacking maildir info gets one on any tag change
 PASS   Message in new with maildir info is moved to cur on any tag change
 PASS   Removing 'S' flag from existing filename adds 'unread' tag
 PASS   Removing info from filename leaves tags unchanged
 PASS   Can remove unread tag from message in non-maildir directory
 PASS   Message in non-maildir directory does not get renamed
 PASS   notmuch dump/restore re-synchronizes maildir tags with flags
 PASS   Adding flags to duplicate message tags the mail
 PASS   Adding duplicate message without flags does not remove tags
 PASS   Tag changes modify flags of multiple files
 PASS   Synchronizing tag changes preserves unsupported maildir flags
 PASS   A file with non-compliant maildir info will not be renamed
 PASS   Files in new/ get default synchronized tags

T350-crypto: Testing PGP/MIME signature verification and decryption
 skip all tests in T350-crypto
T355-smime: Testing S/MIME signature verification and decryption
 skip all tests in T355-smime
T360-symbol-hiding: Testing exception symbol hiding
 skip all tests in T360-symbol-hiding
T370-search-folder-coherence: Testing folder tags removed and added through file renames remain consistent
 PASS   No new messages
 PASS   Single new message
 PASS   Add second folder for same message
 PASS   Multiple files for same message
 PASS   Test matches folder:spam
 PASS   Remove folder:spam copy of email
 PASS   No mails match the folder:spam search

T380-atomicity: Testing atomicity
cat: outcount: No such file or directory
./T380-atomicity.sh: line 79: ((: i < : syntax error: operand expected (error token is "< ")
 PASS   "notmuch new" is idempotent under arbitrary aborts
 FAIL   detected >10 abort points
        test  -gt 10
./test-lib.sh: line 875: test: -gt: unary operator expected

T390-python: Testing python bindings
 skip all tests in T390-python
T395-ruby: Testing ruby bindings
 missing prerequisites: ruby development files
 SKIP   all tests in T395-ruby

T400-hooks: Testing hooks
 PASS   pre-new is run
 PASS   post-new is run
 PASS   post-insert hook is run
 PASS   pre-new is run before post-new
 PASS   pre-new non-zero exit status (hook status)
 PASS   pre-new non-zero exit status (notmuch status)
 PASS   pre-new non-zero exit status aborts new
 PASS   post-new non-zero exit status (hook status)
 PASS   post-new non-zero exit status (notmuch status)
 PASS   post-insert hook does not affect insert status
 PASS   hook without executable permissions
 PASS   hook execution failure

T410-argument-parsing: Testing argument parsing
 PASS   sanity check

T420-emacs-test-functions: Testing emacs test function sanity
 PASS   emacs test function sanity

T430-emacs-address-cleaning: Testing emacs address cleaning
 PASS   notmuch-test-address-clean part 1
 PASS   notmuch-test-address-clean part 2
 PASS   notmuch-test-address-clean part 3

T440-emacs-hello: Testing emacs notmuch-hello view
 PASS   User-defined section with inbox tag
 PASS   User-defined section with empty, hidden entry
 PASS   User-defined section, unread tag filtered out
 PASS   User-defined section, different query for counts
 PASS   Empty custom tags section
 PASS   Empty custom queries section
 PASS   Column alignment for tag/queries with long names

T450-emacs-show: Testing emacs notmuch-show view
 PASS   Hiding Original Message region at beginning of a message
 PASS   Bare subject #1
 PASS   Bare subject #2
 PASS   Bare subject #3
 PASS   don't process cryptographic MIME parts
 PASS   process cryptographic MIME parts
 PASS   process cryptographic MIME parts (w/ notmuch-show-toggle-process-crypto)
 PASS   notmuch-show: don't elide non-matching messages
 PASS   notmuch-show: elide non-matching messages
 PASS   notmuch-show: elide non-matching messages (w/ notmuch-show-toggle-elide-non-matching)
 PASS   notmuch-show: elide non-matching messages (w/ prefix arg to notmuch-show)
 PASS   notmuch-show: disable indentation of thread content (w/ notmuch-show-toggle-thread-indentation)
 PASS   id buttonization
 PASS   Show handles subprocess errors

T455-emacs-charsets: Testing emacs notmuch-show charset handling
 PASS   Text parts are decoded when rendering
 PASS   8bit text parts are decoded when rendering
 PASS   HTML parts are decoded when rendering
 PASS   Text parts are not decoded when saving
 PASS   8bit text parts are not decoded when saving
 PASS   HTML parts are not decoded when saving
 PASS   Binary parts are not decoded when saving
 PASS   Text message are not decoded when viewing
 PASS   8bit text message are not decoded when viewing

T460-emacs-tree: Testing emacs tree view interface
 PASS   Basic notmuch-tree view in emacs
 PASS   Refreshed notmuch-tree view in emacs
 PASS   Tag message in notmuch tree view (display)
 PASS   Tag message in notmuch tree view (database)
 PASS   Untag message in notmuch tree view
 PASS   Untag message in notmuch tree view (database)
 PASS   Tag thread in notmuch tree view
 PASS   Tag message in notmuch tree view (database)
 PASS   Untag thread in notmuch tree view
 PASS   Untag message in notmuch tree view (database)
 PASS   Navigation of notmuch-hello to search results
 PASS   Tree view of a single thread (from search)
 PASS   Tree view of a single thread (from show)
 PASS   Message window of tree view
 PASS   Stash id
 PASS   Move to next matching message
 PASS   Move to next thread
 PASS   Move to previous thread
 PASS   Move to previous previous thread

T470-missing-headers: Testing messages with missing headers
 PASS   Search: text
 PASS   Search: json
 PASS   Show: text
 PASS   Show: json

T480-hex-escaping: Testing hex encoding and decoding
 PASS   round trip
 PASS   punctuation
 PASS   round trip newlines
 PASS   round trip 8bit chars
 PASS   round trip (in-place)
 PASS   punctuation (in-place)
 PASS   round trip newlines (in-place)
 PASS   round trip 8bit chars (in-place)

T490-parse-time-string: Testing date/time parser module
 skip all tests in T490-parse-time-string
T500-search-date: Testing date:since..until queries
 PASS   Absolute date range
 PASS   Absolute date range with 'same' operator
 PASS   Absolute date field
 PASS   Absolute time range with TZ

T510-thread-replies: Testing test of proper handling of in-reply-to and references headers
 PASS   Use References when In-Reply-To is broken
 PASS   Prefer References to In-Reply-To
 PASS   Use In-Reply-To when no References
 PASS   Use last Reference
 PASS   Ignore garbage at the end of References

T520-show: Testing "notmuch show"
 PASS   exit code for show invalid query

T530-upgrade: Testing database upgrade
 missing prerequisites: database-v1.tar.xz - fetch with 'make download-test-databases'
 SKIP   all tests in T530-upgrade

T550-db-features: Testing database version and feature compatibility
 skip all tests in T550-db-features
T560-lib-error: Testing error reporting for library
 skip all tests in T560-lib-error
T570-revision-tracking: Testing database revision tracking
 PASS   notmuch_database_get_revision
 PASS   output of count matches test code
 PASS   modification count increases
 PASS   search succeeds with correct uuid
 PASS   uuid works as global option
 PASS   uuid works as global option II
 PASS   search fails with incorrect uuid
 PASS   show succeeds with correct uuid
 PASS   show fails with incorrect uuid
 PASS   tag succeeds with correct uuid
 PASS   tag fails with incorrect uuid
 PASS   lastmod:0.. matches everything
 PASS   lastmod:1000000.. matches nothing
 PASS   exclude one message using lastmod

T580-thread-search: Testing test of searching by thread-id
 skip all tests in T580-thread-search
T590-libconfig: Testing library config API
 PASS   notmuch_database_{set,get}_config
 PASS   notmuch_database_get_config_list: empty list
 PASS   notmuch_database_get_config_list: all pairs
 PASS   notmuch_database_get_config_list: one prefix
 PASS   dump config
 PASS   restore config

T590-thread-breakage: Testing thread breakage during reindexing
 PASS   There should be no threads initially
 PASS   There should be no ghosts initially
 PASS   One message in: one thread
 PASS   looking for 1 instance of 'apple'
 PASS   looking for 0 instance of 'banana'
 PASS   Expecting 0 ghosts(s)
 PASS   Second message in the same thread: one thread
 PASS   looking for 1 instance of 'apple'
 PASS   looking for 1 instance of 'banana'
 PASS   Expecting 0 ghosts(s)
 PASS   First message removed: still only one thread
 PASS   looking for 0 instance of 'apple'
 PASS   looking for 1 instance of 'banana'
 PASS   should be one ghost after first message removed
 PASS   First message reappears: should return to the same thread
 PASS   looking for 1 instance of 'apple'
 PASS   looking for 1 instance of 'banana'
 PASS   Expecting 0 ghosts(s)
 PASS   Removing second message: still only one thread
 PASS   looking for 1 instance of 'apple'
 PASS   looking for 0 instance of 'banana'
 BROKEN No ghosts should remain after deletion of second message
        --- T590-thread-breakage.22.expected    2017-08-10 03:04:08.298387000 +0000
        +++ T590-thread-breakage.22.output      2017-08-10 03:04:08.298405000 +0000
        @@ -1 +1 @@
        -0
        +1
 PASS   All messages gone: no threads
 PASS   looking for 0 instance of 'apple'
 PASS   looking for 0 instance of 'banana'
 PASS   No ghosts should remain after full thread deletion

T600-named-queries: Testing named queries
 PASS   error adding named query before initializing DB
 PASS   adding named query
 PASS   adding nested named query
 PASS   retrieve named query
 PASS   List all queries
 PASS   dump named queries
 PASS   delete named queries
 PASS   restore named queries
 PASS   search named query
 PASS   search named query with other terms
 PASS   search nested named query

T610-message-property: Testing message property API
 PASS   notmuch_message_{add,get,remove}_property
 PASS   notmuch_message_remove_all_properties
 PASS   notmuch_message_get_properties: empty list
 PASS   notmuch_message_properties: one value
 PASS   notmuch_message_properties: multiple values
 FAIL   notmuch_message_properties: prefix
        --- T610-message-property.6.EXPECTED    2017-08-10 03:04:08.888655000 +0000
        +++ T610-message-property.6.OUTPUT      2017-08-10 03:04:08.889313000 +0000
        @@ -1,8 +1,8 @@
         == stdout ==
        -alice
        +testvalue2
         bob
         testvalue1
        -testvalue2
        +alice
         alice3
         bob3
         testvalue3
 PASS   notmuch_message_properties: modify during iteration
 PASS   dump message properties
 PASS   dump _only_ message properties
 PASS   restore missing message property (single line)
 PASS   restore missing message property (full dump)
 PASS   restore clear extra message property
 PASS   test 'property:' queries: empty
 PASS   test 'property:' queries: single message

T620-lock: Testing locking
 PASS   blocking open

T630-emacs-draft: Testing Emacs Draft Handling
 PASS   Saving a draft indexes it
 PASS   Saving a draft tags previous draft as deleted
 PASS   Saving a signed draft adds header
 PASS   Refusing to save an encrypted draft
 PASS   Resuming a signed draft

T640-database-modified: Testing DatabaseModifiedError handling
 skip all tests in T640-database-modified
T650-regexp-query: Testing regular expression searches
 PASS   empty path:// search
 PASS   empty folder:// search
 PASS   unanchored folder:// specification
 PASS   anchored folder:// search
 PASS   unanchored path:// specification
 PASS   anchored path:// search
 PASS   null from: search
 PASS   null subject: search
 PASS   xapian wildcard search for from:
 PASS   xapian wildcard search for subject:
 PASS   regexp from search, case sensitive
 PASS   empty regexp or query
 PASS   non-empty regexp and query
 PASS   regexp from search, duplicate term search
 PASS   long enough regexp matches only desired senders
 PASS   shorter regexp matches one more sender
 PASS   regexp subject search, non-ASCII
 PASS   regexp subject search, punctuation
 PASS   regexp subject search, no punctuation
 PASS   combine regexp from and subject
 FAIL   regexp error reporting
        --- T650-regexp-query.21.EXPECTED       2017-08-10 03:04:13.092382000 +0000
        +++ T650-regexp-query.21.OUTPUT 2017-08-10 03:04:13.093053000 +0000
        @@ -1,3 +1,3 @@
         notmuch search: A Xapian exception occurred
        -A Xapian exception occurred parsing query: Invalid regular expression
        +A Xapian exception occurred parsing query: brackets ([ ]) not balanced
         Query string was: from:/unbalanced[/
 PASS   empty mid search
 PASS   non-empty mid regex search
 PASS   combine regexp mid and subject
 PASS   unanchored tag search
 PASS   anchored tag search

T660-bad-date: Testing parsing of bad dates
 PASS   Bad dates translate to a date after the Unix epoch

T680-html-indexing: Testing indexing of html parts
 PASS   embedded images should not be indexed
 PASS   ignore > in attribute text
 PASS   non tag text should be indexed

T690-command-line-args: Testing command line arguments
 PASS   bad option to show
 PASS   string option with space
 PASS   string option with =
 PASS   string option with :
 PASS   single keyword option with space
 PASS   single keyword option with =
 PASS   single keyword option with :
 PASS   multiple keyword options with space
 PASS   multiple keyword options with =
 PASS   mixed space and = delimiters
 PASS   mixed space and : delimiters
 PASS   show --entire-thread
 PASS   show --exclude

Notmuch test suite complete.
494/500 tests passed.
3 broken tests failed as expected.
3 tests failed.
gmake[1]: *** [test/Makefile.local:71: test] Error 1
gmake[1]: Leaving directory '/wrkdirs/usr/ports/mail/notmuch/work/notmuch-0.25'
*** Error code 1

Stop.
make: stopped in /usr/ports/mail/notmuch
jrm edited edge metadata.

Fixes for make test

This revision now requires review to proceed.Aug 10 2017, 3:14 AM

Cosmetic change (remove accidentally added space)

Why so many tests are in skip test variable? From your log I can see that it should contain only 3 (because only 3 tests has failed): atomicity message-property database-modified

Maybe my previous message was hidden that explained why? Did you click "Show older changes"?

I don't think *failed* tests should be skipped. I'm also inclined to not skip any of them and leave the broken tests in; that might be more incentive for someone to attempt to fix the broken tests.

Here is the complete make test output with NOTMUCH_SKIP_TESTS="atomicity message-property database-modified".

INFO: using 2 minute timeout for tests

T000-basic: Testing the test framework itself.
 PASS   success is reported like this
 PASS   test runs if prerequisite is satisfied
 PASS   tests clean up after themselves
 PASS   tests clean up even after a failure
 PASS   failure to clean up causes the test to fail
FATAL: ./T000-basic.sh: interrupted by signal -128
gmake[1]: *** [test/Makefile.local:71: test] Error 124
gmake[1]: Leaving directory '/wrkdirs/usr/ports/mail/notmuch/work/notmuch-0.25'
*** Error code 1
In D11752#247995, @jrm wrote:

Maybe my previous message was hidden that explained why? Did you click "Show older changes"?

I don't think *failed* tests should be skipped. I'm also inclined to not skip any of them and leave the broken tests in; that might be more incentive for someone to attempt to fix the broken tests.

All failed tests are skipped because otherwise 'make test' will not be passed, "broken" tests must not be there because they don't influence final result of a 'make test' - this is known broken tests and developers of notmuch fix them. List of broken tests can be added above TEST_VAR as a comment for reference - to track changes in test results between releases, please start comments with "## ".

Regarding "basic" test - it should be added to skip list too at that moment.

As I said above, all of these tests are broken and cause make test to stop with an error.
address basic count crypto db-features database-modified dump-restore emacs emacs-large-search-buffer from-guessing insert json lib-error new parse-time-string python search search-output sexp smime symbol-hiding tagging thread-order thread-search

This revision is now accepted and ready to land.Aug 10 2017, 12:59 PM

please, commit with the long list for test skip.

This revision was automatically updated to reflect the committed changes.