Page MenuHomeFreeBSD

databases/couchdb2: new port via PR#218844
ClosedPublic

Authored by dch on Aug 20 2018, 8:48 PM.

Details

Summary

JSON document database with HTTP API & scalable multi-master sync

Test Plan
  • permits running couchdb 1.7.x and couchdb 2.x concurrently to support migrations
  • improved first-run walk through
  • no explicit dependencies on OTP version in port anymore
  • provides a knob for switching Erlang/OTP releases at build time
  • includes all inline documentation and fauxton for the moment
  • manual testing including user creation and couch verify suite on
  • poudriere bulk -t on 10.4, 11.1, 11.2 on amd64/i386 & 12.0-alpha1 amd64 OK

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
jrm added a comment.Aug 21 2018, 1:45 AM

poudriere testport warnings:

Warning: 'libexec/couchdb2/bin/couchjs' is not stripped consider trying INSTALL_TARGET=install-strip or using ${STRIP_CMD}
Warning: 'libexec/couchdb2/lib/khash-1.0.1/priv/khash.so' is not stripped consider trying INSTALL_TARGET=install-strip or using ${STRIP
_CMD}
Warning: 'libexec/couchdb2/lib/hyper-1.0/priv/hyper_carray.so' is not stripped consider trying INSTALL_TARGET=install-strip or using ${
STRIP_CMD}
Warning: 'libexec/couchdb2/lib/couch-2.2.0/priv/couch_ejson_compare.so' is not stripped consider trying INSTALL_TARGET=install-strip or
 using ${STRIP_CMD}
Warning: 'libexec/couchdb2/lib/couch-2.2.0/priv/couch_icu_driver.so' is not stripped consider trying INSTALL_TARGET=install-strip or us
ing ${STRIP_CMD}
Warning: 'libexec/couchdb2/lib/couch-2.2.0/priv/couchjs' is not stripped consider trying INSTALL_TARGET=install-strip or using ${STRIP_
CMD}
Warning: 'libexec/couchdb2/lib/jiffy-0.14.11/priv/jiffy.so' is not stripped consider trying INSTALL_TARGET=install-strip or using ${STR
IP_CMD}
Warning: 'libexec/couchdb2/lib/b64url-1.0.1/priv/b64url.so' is not stripped consider trying INSTALL_TARGET=install-strip or using ${STR
IP_CMD}
Warning: 'libexec/couchdb2/lib/snappy-1.0.5/priv/snappy_nif.so' is not stripped consider trying INSTALL_TARGET=install-strip or using $
{STRIP_CMD}
databases/couchdb2/Makefile
23–24 ↗(On Diff #47000)

USES and USE_x block needs to move up to conform to Order of Variables in Port Makefiles in PH.

26–29 ↗(On Diff #47000)

These belong in the The Rest of the Variables block according to the PH.

81 ↗(On Diff #47000)

-f is already part of ${RM}

dch updated this revision to Diff 47042.Aug 21 2018, 3:32 PM
dch marked 2 inline comments as done.

clean up ports smells - RM, USES_ USE, PHB var order

mat added a subscriber: mat.Aug 21 2018, 6:50 PM
mat added inline comments.
databases/couchdb2/Makefile
16 ↗(On Diff #47042)

There is no rebar in the base system, so remove ${LOCALBASE}/bin/.

43–44 ↗(On Diff #47042)

OPTIONS_DEFAULT should be first.

59 ↗(On Diff #47042)

This should be post-patch.

71 ↗(On Diff #47042)

No need for the / here.

85 ↗(On Diff #47042)

Not sure why this change is needed, you have other occurrences of couchdb2 in the port, this one is no special.

dch marked 6 inline comments as done.Aug 22 2018, 5:25 PM
dch added inline comments.
databases/couchdb2/Makefile
59 ↗(On Diff #47042)

done - but why is this better?

dch updated this revision to Diff 47113.Aug 22 2018, 5:29 PM
dch marked an inline comment as done.
  • polish and reorder Makefile
  • strip .so and binaries
  • remove surplus duplicate files from distribution
dch added a comment.Aug 22 2018, 5:42 PM

poudriere bulk -t OK on 10.4R, 11.1R, 11.2R amd64+i386 and 12.0-alpha amd64

jrm added a comment.Aug 22 2018, 5:43 PM

How are you creating your revisions? On your feature branch are you doing arc diff --create master? I am not able to arc patch to test.

% arc patch D16819
 INFO  Base commit is not in local repository; trying to fetch.
Created and checked out branch arcpatch-D16819.


    This diff is against commit ddb043f91328c0cc3101a1a98ad096fa408c7377, but
    the commit is nowhere in the working copy. Try to apply it against the
    current working copy state? (c7d35e8d90fc640e0c490652112cc5f60022c624)
    [Y/n]

Checking patch databases/couchdb2/pkg-plist...
error: databases/couchdb2/pkg-plist: does not exist in index
Checking patch databases/couchdb2/files/pkg-message.in...
error: databases/couchdb2/files/pkg-message.in: does not exist in index
Checking patch databases/couchdb2/Makefile...
error: databases/couchdb2/Makefile: does not exist in index

 Patch Failed!
Usage Exception: Unable to apply patch!
dch updated this revision to Diff 47116.Aug 22 2018, 5:54 PM
dch edited the test plan for this revision. (Show Details)

seems the last patch is missing the intermediary commits. Here's a full diff via arc diff --preview --base git:upstream/master

jrm added inline comments.Aug 22 2018, 7:09 PM
databases/couchdb2/Makefile
31 ↗(On Diff #47116)

I think this qualifies under Keep related variables close together. For example, if using USE_GITHUB, always put the GH_* variables right after it.

54 ↗(On Diff #47116)

I think this qualifies under Keep related variables close together. For example, if using USE_GITHUB, always put the GH_* variables right after it.

94–96 ↗(On Diff #47116)

They are still not being stripped.
http://pkg.awarnach.mathstat.dal.ca/data/11amd64-default/2018-08-22_15h28m05s/logs/couchdb2-2.2.0.log

Maybe this?
@${FIND} -E ${STAGEDIR} -type f -regex '.+/(couchjs|.so)$$' -exec ${STRIP_CMD} {} \;

97 ↗(On Diff #47116)

@${MKDIR}.... mkdir is an exception that can be silenced here.

dch updated this revision to Diff 47127.Aug 22 2018, 8:30 PM
dch marked 4 inline comments as done.
reorder Makefile and fix stripping properly
dch marked an inline comment as done.Aug 22 2018, 8:31 PM
dch added inline comments.
databases/couchdb2/Makefile
54 ↗(On Diff #47116)

Now I'm confused. Last diff I understood these all needed to be moved down to the bottom. Its awkward in phab to see where you want to move those back to - where would that end up? back below USES? Lets see if the next diff gets it right.

94–96 ↗(On Diff #47116)

aaah I have STRIP= in my poudriere make.conf which is needed for dtrace support in erlang.. after wrapping that in a if .CURDIR.. endif I see these failures too during stage-qa, and my regex needs tweaking too. BTW is there a command to tell if a given binary hasn't been stripped (outside ports framework)?

dch updated this revision to Diff 47130.Aug 22 2018, 8:55 PM

APPDIR was not propagated to patch-configure

mat added inline comments.Aug 22 2018, 9:56 PM
databases/couchdb2/Makefile
34 ↗(On Diff #47130)

shouldn-t docs be enabled by a DOCS option?

94 ↗(On Diff #47130)

Wouldn't -name '*.so' be enough? Are there any other .so files in there that, for some reason should not be stripped?

95 ↗(On Diff #47130)
-exec ${STRIP_CMD} {} +
97 ↗(On Diff #47130)

Extra $ at the end?

59 ↗(On Diff #47042)

It has nothing to do with better, it is semantically wrong. You are patching the file, which should always happen in the patch phase.

The pre-configure target is in the configure phase, when you run commands to prepare the configuration, like re-generating a configure script, or whatever.

jrm added inline comments.Aug 22 2018, 10:44 PM
databases/couchdb2/Makefile
21–24 ↗(On Diff #47130)

My interpretation of Chapter 15 of the PH, and specifically Start this section with defining USES, and then possible USE_x. Keep related variables close together. For example, if using USE_GITHUB, always put the GH_* variables right after it. is

USES=          cpe gmake libtool ncurses python:run shebangfix
CPE_VENDOR=    apache
SHEBANG_FILES= rel/overlay/bin/couchup
USE_RC_SUBR=   couchdb2

but, to me, this is all small potatoes. @mat probably wrote that chapter, so maybe he can clarify.

mat added inline comments.Aug 22 2018, 10:54 PM
databases/couchdb2/Makefile
21–24 ↗(On Diff #47130)

Yeah, does not really matter. You can also add blank lines to let it breath a bit and be more readable.

dch updated this revision to Diff 47145.Aug 23 2018, 1:14 AM

I've been messing around with DOCS option, I'm not at all sure if this is
correct nor working properly when DOCS knob is off vs on, but it does
at least "work" functionally at present via:

http://localhost:5984/_utils/docs/index.html

Couch actually serves up its documentation set directly from:

/usr/local/share/couchdb2/www/docs/*

It's also feasible to softlink that from /usr/local/share/doc/couchdb2

What's the best approach here?

mat added inline comments.Aug 23 2018, 7:13 AM
databases/couchdb2/Makefile
29 ↗(On Diff #47145)

No need for couchdb2 here.

45–46 ↗(On Diff #47145)

Why +=?

50–51 ↗(On Diff #47145)

Why +=?

91 ↗(On Diff #47145)

Never use CP to install files, use one of the INSTALL_* macros.

dch marked 10 inline comments as done.Aug 24 2018, 10:58 AM
dch added inline comments.
databases/couchdb2/Makefile
45–46 ↗(On Diff #47145)

I'd expected that OTP*_RUN_DEPENDS will have the OTP*_ part stripped, and then needs to be appended to the existing RUN_DEPENDS from earlier in the Makefile that includes the py-requests?

91 ↗(On Diff #47145)

It's going to WRKSRC, not STAGEDIR, and gets installed later. I don't think this needs changing.

34 ↗(On Diff #47130)

I need to look a bit more into this, might be tomorrow. I'm not sure whether my current patch is actually doing what I expected it to.

Am I correct that running make DOCS= ... and make DOCS=1 ... is sufficient to test these options? Or is there a better way?

mat added inline comments.Aug 24 2018, 11:14 AM
databases/couchdb2/Makefile
45–46 ↗(On Diff #47145)

I am not sure what you are saying. You use += when you are appending to a variable. Here, the variable does not exist before, so you do not need to append to it.

91 ↗(On Diff #47145)

Right, missed that bit.

34 ↗(On Diff #47130)

I have no idea how that software works, or how to enable the building of the documentation in it, so I could not say how to hook it up in the port.

dch updated this revision to Diff 47228.Aug 24 2018, 11:22 AM
dch marked an inline comment as done.
  • messing around with PORTDOCS
dch updated this revision to Diff 47230.Aug 24 2018, 12:32 PM
dch marked 9 inline comments as done.

Working html DOCS install from perspective of couchdb2 user/admin however
I'm still not getting this to work correctly with PORTDOCS though

mat added inline comments.Aug 24 2018, 1:39 PM
databases/couchdb2/pkg-plist
485–487 ↗(On Diff #47230)

This is wrong, it should be %%PORTDOCS%%.

dch marked an inline comment as done.Aug 24 2018, 5:54 PM
dch added inline comments.
databases/couchdb2/pkg-plist
485–487 ↗(On Diff #47230)

per IRC the magical missing OPTIONS_DEFINE= DOCS was what was missing. grrr.

dch updated this revision to Diff 47248.Aug 24 2018, 5:59 PM
dch marked an inline comment as done.

PORTDOCS now seems to work as desired
poudriere bulk -t is happy on 10.4,11.1,11.2 amd64/i386 & 12.0a1 amd64

dch marked an inline comment as done.Aug 24 2018, 6:08 PM
dch updated this revision to Diff 47251.Aug 24 2018, 6:25 PM

suppress output on stdio when running couchdb2 without syslog

jrm added a comment.Aug 24 2018, 6:29 PM

The layout of the HTML/WWW is confusing. Shouldn't we install DOCS to ${DOCSDIR} and not DATADIR? Should we install ${WRKSRC}/share/www to ${WWWDIR}?

dch updated this revision to Diff 48306.Sep 21 2018, 11:32 AM
databases/couchdb2: fix all the things
  • sort SUB_LIST
  • bump OTP releases
  • use WWWDIR for runtime UI and also optional docs
  • add inet6 http client support 1-liner from upstream
  • placate portlint 2.18.4
dch marked an inline comment as done.Sep 21 2018, 11:36 AM

I think this is ready to land now, I'll do the poudriere thing and report back if anything breaks.

databases/couchdb2/Makefile
21–24 ↗(On Diff #47130)

latest portlint doesn't like this; I've done this instead:

USES=
USE_RC_SUBR=

CPE_VENDOR=
SHEBANG_FILES=
dch marked an inline comment as done.Sep 21 2018, 12:53 PM
dch added inline comments.
databases/couchdb2/Makefile
71 ↗(On Diff #47042)

BTW I've marked this as /done/ as the / is needed here, I assume its the final slash you referred to @mat ?

 make -V WRKDIR
/tmp/usr/ports/databases/couchdb2/work
...
make package
===>  License APACHE20 BSD3CLAUSE ISCL WTFPL OFL11 accepted by the user
===>   couchdb2-2.2.0 depends on file: /usr/local/sbin/pkg - found
=> apache-couchdb-2.2.0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://www.apache.org/dist/couchdb/source/2.2.0/apache-couchdb-2.2.0.tar.gz
apache-couchdb-2.2.0.tar.gz                             16 MB 1472 kBps    11s
===> Fetching all distfiles required by couchdb2-2.2.0 for building
===>  Extracting for couchdb2-2.2.0
=> SHA256 Checksum OK for apache-couchdb-2.2.0.tar.gz.
===>  Patching for couchdb2-2.2.0
===>  Applying FreeBSD patches for couchdb2-2.2.0
sed: /tmp/usr/ports/databases/couchdb2/work/apache-couchdb-2.2.0configure: No such file or directory
*** Error code 1
jrm accepted this revision.Sep 21 2018, 2:52 PM
This revision is now accepted and ready to land.Sep 21 2018, 2:52 PM
This revision was automatically updated to reflect the committed changes.