Page MenuHomeFreeBSD

devel/apr1: support building DBM with BDB 1.85
ClosedPublic

Authored by ngie on Jun 1 2023, 7:11 AM.
Tags
None
Referenced Files
F104079917: D40366.diff
Tue, Dec 3, 6:54 AM
Unknown Object (File)
Sat, Nov 30, 3:13 PM
Unknown Object (File)
Sun, Nov 17, 10:34 PM
Unknown Object (File)
Oct 31 2024, 5:40 PM
Unknown Object (File)
Oct 19 2024, 7:41 PM
Unknown Object (File)
Oct 5 2024, 6:52 AM
Unknown Object (File)
Oct 2 2024, 11:43 PM
Unknown Object (File)
Sep 17 2024, 9:10 PM

Details

Summary

This change modifies the libapr1 configure logic to search for db_open,
etc, in libc instead of in a standalone library.

Pass the appropriate arguments to configure to support this fact.

Add USES+=autoreconf to regenerate configure from configure.in/build_dbm.m4 .

Diff Detail

Repository
rP FreeBSD ports repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 51961
Build 48852: arc lint + arc unit

Event Timeline

ngie requested review of this revision.Jun 1 2023, 7:11 AM

libc/db in base is currently BSD-3-Clause. Advertising clause removed from FreeBSD source tree in 2007.

I'm not sure we need to mark it as dual licensed if using the BSD licensed db 1.85. The BSD license does not make requirements on derivative works like AGPLv3 does, just on distribution of the lib itself (in binary or source form). The base FreeBSD OS already takes care of those requirements since it is the distributor of db 1.85. I suppose clause 3 is a requirement on derived works (to not use the University's or contributors' names for endorsement of derived works). But other parts of libc are BSD licensed, and we typically don't mark every port that uses libc as dual licensed with the BSD license.

devel/apr1/Makefile
39

See comments regarding 3 clause vs 4 clause.

90–91

See comments about dual licensing.

devel/apr1/Makefile
92

Maybe this should be BDB1_CONFIGURE_ON=--with-dbm=db1 instead of adding a configure arg in BDB1_VARS.

ngie marked an inline comment as done.Jun 7 2023, 6:26 PM

@rfyu28uyeg_snkmail.com: thank you for the input about the licensing. You're right -- I'm updating the patch now.

devel/apr1/Makefile
92

This is an apr-util specific configure argument -- it shouldn't be passed to apr's configure script.

Respond to comments from @rfyu28uyeg_snkmail.com:

  • fix licensing (lib/libc/db is BSD 3-clause, not 4-clause).
  • remove unnecessary "dual licensing" logic for the BDB1 option.
ngie marked an inline comment as done.Jun 7 2023, 6:28 PM
ngie marked an inline comment as done.

I think this is good. I'd be inclined to switch BDB to BDB1 for the default. The AGPLv3 licensed BDB should probably not be the default.

See also https://bugs.freebsd.org/261523#c30 - other people are thinking along the same lines (to use bdb from base).

Rebase on recent ports tree

This revision is now accepted and ready to land.Oct 5 2024, 4:12 AM
brnrd requested changes to this revision.Oct 13 2024, 8:44 AM
This comment was removed by brnrd.
This revision now requires changes to proceed.Oct 13 2024, 8:44 AM

How did you verify that Berkeley DB support is working with these changes? httxt2dbm is failing for me.

After applying changes, rebuilding devel/apr1, I have an apr-utils lib that shows bdb 1 support

❯ ls -l $(which apu-1-config)
-rwxr-xr-x  1 root wheel 6255 Oct 13 11:16 /usr/local/bin/apu-1-config
❯ apu-1-config --includes --dbm-libs --link-ld
 -I/usr/local/include/apr-1 -I/usr/include -I/usr/local/include -lc   -L/usr/local/lib -laprutil-1
❯ apu-1-config --db-version
1

Building www/apache24 with this, results in an httxt2dbm library that is nonfunctional

❯ ls -l $(which httxt2dbm)
-rwxr-xr-x  1 root wheel 10264 Oct 13 11:20 /usr/local/bin/httxt2dbm
❯ httxt2dbm
Error: No input file specified.

httxt2dbm -- Program to Create DBM Files for use by RewriteMap
Usage: httxt2dbm [-v] [-f format] -i SOURCE_TXT -o OUTPUT_DBM

Options:
 -v    More verbose output

 -i    Source Text File. If '-', use stdin.

 -o    Output DBM.

 -f    DBM Format.  If not specified, will use the APR Default.
           GDBM for GDBM files (unavailable)
           SDBM for SDBM files (available)
           DB   for berkeley DB files (available)
           NDBM for NDBM files (unavailable)
           default for the default DBM type
❯ echo "key value" | httxt2dbm -f DB -i- -o /tmp/httest.db -v
DBM Format: DB
Input File: -
Error: Cannot open output DBM '/tmp/httest.db': (20019) DSO load failed
❯ echo "key value" | httxt2dbm -f SDBM -i- -o /tmp/httest.sdbm -v
DBM Format: SDBM
Input File: -
DBM File: /tmp/httest.sdbm
    'key' -> 'value'
Conversion Complete.

Added SDBM for validation that I have a working httxt2dmb.

How did you verify that Berkeley DB support is working with these changes? httxt2dbm is failing for me.

I didn't execute any tests at runtime (I really should have as illustrated by your testing below...). I was naively relying on build testing to be enough for this component.

After applying changes, rebuilding devel/apr1, I have an apr-utils lib that shows bdb 1 support

❯ ls -l $(which apu-1-config)
-rwxr-xr-x  1 root wheel 6255 Oct 13 11:16 /usr/local/bin/apu-1-config
❯ apu-1-config --includes --dbm-libs --link-ld
 -I/usr/local/include/apr-1 -I/usr/include -I/usr/local/include -lc   -L/usr/local/lib -laprutil-1
❯ apu-1-config --db-version
1

Building www/apache24 with this, results in an httxt2dbm library that is nonfunctional

❯ ls -l $(which httxt2dbm)
-rwxr-xr-x  1 root wheel 10264 Oct 13 11:20 /usr/local/bin/httxt2dbm
❯ httxt2dbm
Error: No input file specified.

httxt2dbm -- Program to Create DBM Files for use by RewriteMap
Usage: httxt2dbm [-v] [-f format] -i SOURCE_TXT -o OUTPUT_DBM

Options:
 -v    More verbose output

 -i    Source Text File. If '-', use stdin.

 -o    Output DBM.

 -f    DBM Format.  If not specified, will use the APR Default.
           GDBM for GDBM files (unavailable)
           SDBM for SDBM files (available)
           DB   for berkeley DB files (available)
           NDBM for NDBM files (unavailable)
           default for the default DBM type
❯ echo "key value" | httxt2dbm -f DB -i- -o /tmp/httest.db -v
DBM Format: DB
Input File: -
Error: Cannot open output DBM '/tmp/httest.db': (20019) DSO load failed
❯ echo "key value" | httxt2dbm -f SDBM -i- -o /tmp/httest.sdbm -v
DBM Format: SDBM
Input File: -
DBM File: /tmp/httest.sdbm
    'key' -> 'value'
Conversion Complete.

Added SDBM for validation that I have a working httxt2dmb.

Input File: -
26931: write(2,"Input File: -\n",14)             = 14 (0xe)
26931: open("/usr/local/lib/apr-util-1/apr_dbm_db-1.so",O_RDONLY|O_CLOEXEC|O_VERIFY,020040100200) ERR#2 'No such file or directory'
Error: Cannot open output DBM '/tmp/httest.db': (20019) DSO load failed

26931: write(2,"Error: Cannot open output DBM '/"...,73) = 73 (0x49)

Hmmm..

Oh neat, the plist is broken for all but OPTIONS_SET+= BDB.

Install the dbm module for OPTIONS_SET+= BDB1 to unbreak uses of the dbm submodule.

brnrd requested changes to this revision.Oct 19 2024, 7:58 PM

Install the dbm module for OPTIONS_SET+= BDB1 to unbreak uses of the dbm submodule.

Not used phabricator for a good while (last was before svn->git?)... WebUI doesn't want to update this rev.

I think I've cracked it, the plist breakage and all. Check [https://brnrd.eu/bsd/patch-devel_apr1-D40366]
Basically:

  1. --with-dbm hints for configure to speed up bdb detection
  2. Switch to PLIST_SUB with BERKELEYDB for all BDB versions

Database creation using httxt2dbm works with all 3 BDB versions:

❯ file test-*.dbm
test-base.dbm: Berkeley DB 1.85 (Hash, version 2, native byte-order)
test-db18.dbm: Berkeley DB (Hash, version 10, native byte-order)
test-db5.dbm:  Berkeley DB (Hash, version 9, native byte-order)

Would love to switch default to base, but that'll trip up people that have existing databases.

This revision now requires changes to proceed.Oct 19 2024, 7:58 PM

Install the dbm module for OPTIONS_SET+= BDB1 to unbreak uses of the dbm submodule.

Not used phabricator for a good while (last was before svn->git?)... WebUI doesn't want to update this rev.

I think I've cracked it, the plist breakage and all. Check [https://brnrd.eu/bsd/patch-devel_apr1-D40366]
Basically:

  1. --with-dbm hints for configure to speed up bdb detection
  2. Switch to PLIST_SUB with BERKELEYDB for all BDB versions

Database creation using httxt2dbm works with all 3 BDB versions:

❯ file test-*.dbm
test-base.dbm: Berkeley DB 1.85 (Hash, version 2, native byte-order)
test-db18.dbm: Berkeley DB (Hash, version 10, native byte-order)
test-db5.dbm:  Berkeley DB (Hash, version 9, native byte-order)

Would love to switch default to base, but that'll trip up people that have existing databases.

Indeed... *sigh*.

Your patch looks great -- thanks!

This revision was not accepted when it landed; it landed in state Needs Revision.Sun, Nov 10, 11:51 AM
This revision was automatically updated to reflect the committed changes.
freebsd_oldach.net added inline comments.
devel/apr1/Makefile
57

This should stay as SQLITE_USES, not QLITE_USES.

Also the change to make db1 marked as "BSD 3-Clause" (vs the incorrect marking "BSD 4-Clause") got lost in the Oct 3, 2024 rebase.

Thanks for moving this forward, brnrd.