Page MenuHomeFreeBSD
Differential D40599

lang/ocaml: Update to 4.08.1
ClosedPublic
Actions

Authored by madpilot on Jun 18 2023, 2:38 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Mar 2, 7:58 AM
Unknown Object (File)
Wed, Feb 26, 2:18 AM
Unknown Object (File)
Feb 15 2025, 8:44 PM
Unknown Object (File)
Feb 13 2025, 8:06 PM
Unknown Object (File)
Feb 13 2025, 5:11 PM
Unknown Object (File)
Feb 12 2025, 9:30 PM
Unknown Object (File)
Jan 28 2025, 6:43 AM
Unknown Object (File)
Jan 23 2025, 6:38 PM
View All 75 Files
Subscribers
mat

Details

Reviewers
danfe
hrs
eduardo
hannes_mehnert.org
dumbbell
kennedy.greg_gmail.com
acm
mandree
madpilot
andrew
Commits
R11:2b40e4609399: lang/ocaml: Update to 4.08.1
Summary

This version of ocaml is now the minimum requirement for most dependent projects. This will also allow to update most ports depending on ocaml to much newer versions, most of the time the latest version.

So I think this small update will make it much easier to step to much more recent versions of ocaml in the near future.

I had to update some dependent ports to keep them working. Sometimes I was able to use the latest version, some other times I had to choose some older versions working with ocaml 4.08-1.

I tried to keep the patch minimal so further updates and incremental improvements will be possible.

I have not bumped PORTVERSION for ports I did not update the version, or did not touch, but I guess I will need to bump PORTVERSION for all ports depending on ocaml. Will do that on final commit, so patches here can be kept reasonable.

Some specific notes:

devel/ocaml-sexplib project was split upstream, and the part we have in the ports now corresponds to what upstream calls "sexplib0" and I decided to rename the port accordingly. Updated to version 0.16.0

BTW, WHen moving the port I kept the PORTEPOCH value, not sure if this is correct.

unison latest versions have a minimum requirement for ocaml 4.08. The author has very kindly provided patches for the latest few updates to keep compatibility, but will not be able to in the future, this was my main motivation to create this patch.

Test Plan

I've tested compiling all ocaml ports multiple times in poudriere and am now running with this patch applied (I use unison daily).

I've also tested running mldonkey which works fine.

No new failure has been observed in my tests, except for unison 2.32.

I plan to ask for an exp-run to confirm my findings.

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Lint Coverage
Unit
No Test Coverage
Build Status
Buildable 52256
Build 49147: arc lint + arc unit

Event Timeline

madpilot created this revision.Jun 18 2023, 2:38 PM
Herald added a subscriber: mat. · View Herald TranscriptJun 18 2023, 2:38 PM
madpilot requested review of this revision.Jun 18 2023, 2:38 PM
Harbormaster completed remote builds in B52128: Diff 123392.Jun 18 2023, 2:39 PM
madpilot added a comment.Jun 18 2023, 2:45 PM

Bug report/exp-run request: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272067

madpilot edited the summary of this revision. (Show Details)Jun 18 2023, 2:50 PM
danfe accepted this revision.Jun 20 2023, 8:30 AM

This is amazing amount and quality of work, thank you for this effort Guido!

Seeing several of those LIBDIR="${PREFIX}/${OCAML_SITELIBDIR}" now contain ${PREFIX}, do I understand correctly that this is to make LIBDIR coherent with other FOODIR variables which are relative to /? I remember there were some inconsistencies which had to be mapped to our Mk/bsd.ocaml.mk ones, cf. make -V OCAML_SITELIBDIR vs. make -V OCAML_DOCSDIR.

BTW, when moving the port I kept the PORTEPOCH value, not sure if this is correct.

Since the PKGNAME is now different and never existed before, I believe you can safely drop it (in fact, it would be silly not to use this chance to get rid of it).

This revision is now accepted and ready to land.Jun 20 2023, 8:30 AM
madpilot edited the summary of this revision. (Show Details)Jun 20 2023, 9:18 AM
madpilot planned changes to this revision.Jun 20 2023, 9:24 AM
In D40599#924849, @danfe wrote:

This is amazing amount and quality of work, thank you for this effort Guido!

Thank you! I'd really like to kick-start the update of ocaml. Once we get this one in further steps should become easier.

Seeing several of those LIBDIR="${PREFIX}/${OCAML_SITELIBDIR}" now contain ${PREFIX}, do I understand correctly that this is to make LIBDIR coherent with other FOODIR variables which are relative to /? I remember there were some inconsistencies which had to be mapped to our Mk/bsd.ocaml.mk ones, cf. make -V OCAML_SITELIBDIR vs. make -V OCAML_DOCSDIR.

Yes, the new dune version expects an absolute path, so I had to fix that.

I'll take a better look at Mk/bsd.ocaml.mk to see if this can be fixed there.

BTW, when moving the port I kept the PORTEPOCH value, not sure if this is correct.

Since the PKGNAME is now different and never existed before, I believe you can safely drop it (in fact, it would be silly not to use this chance to get rid of it).

Good suggestion, I'll change this too.

The exp-run in bug 272067 uncovered a failure on i386 which I need to fix.

madpilot updated this revision to Diff 123670.Jun 23 2023, 9:27 AM

Update patch to fix build on i386 and other arches.

This new version build system was ignoring LDFLAGS.

Harbormaster completed remote builds in B52256: Diff 123670.Jun 23 2023, 9:27 AM
madpilot added a comment.Jun 23 2023, 9:30 AM
In D40599#924849, @danfe wrote:

This is amazing amount and quality of work, thank you for this effort Guido!

Seeing several of those LIBDIR="${PREFIX}/${OCAML_SITELIBDIR}" now contain ${PREFIX}, do I understand correctly that this is to make LIBDIR coherent with other FOODIR variables which are relative to /? I remember there were some inconsistencies which had to be mapped to our Mk/bsd.ocaml.mk ones, cf. make -V OCAML_SITELIBDIR vs. make -V OCAML_DOCSDIR.

I had a look, but OCAML_SITELIBDIR needs to sty relative to ${PREFIX}, because it's used also in PLIST_SUB. I could add an absolute version of the variable, but it looks overkill and unnecessarily complicates the makefile. So I left everything as is.

BTW, when moving the port I kept the PORTEPOCH value, not sure if this is correct.

Since the PKGNAME is now different and never existed before, I believe you can safely drop it (in fact, it would be silly not to use this chance to get rid of it).

The new patch also resets PORTEPOCH.

madpilot added a comment.Jun 25 2023, 8:23 AM

Exp-run was successfull.

I'll wait for approval or timeout (starting when I first posted this review and exp-run bug) to commit.

BTW the commit will also include a PORTREVISION bump for ports depending on ocaml.

madpilot accepted this revision.Jul 5 2023, 10:24 AM
This revision is now accepted and ready to land.Jul 5 2023, 10:24 AM
madpilot closed this revision.Jul 5 2023, 10:24 AM
madpilot added a commit: R11:2b40e4609399: lang/ocaml: Update to 4.08.1.Oct 9 2023, 8:57 PM
Herald added a reviewer: andrew. · View Herald TranscriptOct 9 2023, 8:57 PM

Revision Contents
Changeset List

PathSize
1 line
devel/
2 lines
coccinelle/
6 lines
6 lines
files/
39 lines
9 lines
ocaml-camlp4/
2 lines
6 lines
files/
11 lines
ocaml-camomile/
5 lines
files/
11 lines
17 lines
17 lines
12 lines
9 lines
37 lines
19 lines
11 lines
11 lines
154 lines
ocaml-cfg/
2 lines
ocaml-cppo/
2 lines
32 lines
ocaml-dune/
18 lines
6 lines
files/
29 lines
12 lines
1556 lines
ocaml-findlib/
2 lines
6 lines
files/
61 lines
10 lines
ocaml-ipaddr/
18 lines
5 lines
files/
33 lines
80 lines
ocaml-lwt/
files/
12 lines
ocaml-parmap/
16 lines
6 lines
files/
61 lines
ocaml-pomap/
2 lines
ocaml-ppx-tools/
14 lines
6 lines
60 lines
ocaml-res/
2 lines
ocaml-result/
2 lines
ocaml-sexplib/
files/
ocaml-sexplib0/
47 lines
3 lines
ocaml-sexplib0/
ocaml-sexplib/
11 lines
graphics/
ocaml-cairo/
4 lines
6 lines
files/
lang/
ocaml/
71 lines
16 lines
files/
171 lines
patch-runtime_amd64.S
patch-asmrun_amd64.S
6 lines
patch-runtime_backtrace.c
patch-byterun_backtrace.c
4 lines
patch-runtime_caml_stack.h
patch-byterun_caml_stack.h
6 lines
patch-runtime_power.S
patch-asmrun_power.S
58 lines
6 lines
patch-testsuite_tools_asmgen__power.S
patch-testsuite_tests_asmcomp_power.S
4 lines
561 lines
net-p2p/
mldonkey-gui/
1 line
mldonkey/
14 lines
5 lines
files/
6 lines
net/
unison/
1 line
files/
unison232/
2 lines
unison240/
2 lines
files/
52 lines
unison248/
2 lines
files/
52 lines
security/
ocaml-ssl/
4 lines
6 lines
2 lines
sks/
files/
1001 lines
x11-toolkits/
ocaml-lablgtk3/
2 lines
ocaml-labltk/
6 lines
6 lines

Diff 123670

View Options

MOVED

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec
View Options

devel/Makefile

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec
View Options

devel/coccinelle/Makefile

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec
View Options

devel/coccinelle/distinfo

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec
View Options

devel/coccinelle/files/patch-bundles_pyml_Makefile

Loading...
View Options

devel/coccinelle/files/patch-bundles_pyml_pyml-current_pyml__arch__freebsd.ml

Loading...
View Options

devel/coccinelle/files/patch-stdcompat-fix

Loading...
View Options

devel/coccinelle/pkg-plist

Loading...
View Options

devel/ocaml-camlp4/Makefile

Loading...
View Options

devel/ocaml-camlp4/distinfo

Loading...
View Options

devel/ocaml-camlp4/files/patch-camlp4_Camlp4_Debug.ml

Loading...
View Options

devel/ocaml-camomile/Makefile

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_internal_byte__labeled__dag.ml

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_public_charEncoding.ml

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_public_charEncoding.mli

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_public_oOChannel.mli

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_public_uCS4.ml

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_public_uCol.ml

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_public_uCol.mli

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_public_uTF8.ml

Loading...
View Options

devel/ocaml-camomile/files/patch-Camomile_toolslib_absCe.ml

Loading...
View Options

devel/ocaml-camomile/pkg-plist

Loading...
View Options

devel/ocaml-cfg/Makefile

Loading...
View Options

devel/ocaml-cppo/Makefile

Loading...
View Options

devel/ocaml-cppo/pkg-plist

Loading...
View Options

devel/ocaml-dune/Makefile

Loading...
View Options

devel/ocaml-dune/distinfo

Loading...
View Options

devel/ocaml-dune/files/patch-Makefile

Loading...
View Options

devel/ocaml-dune/files/setup.ml.in

Loading...
View Options

devel/ocaml-dune/pkg-plist

Loading...
View Options

devel/ocaml-findlib/Makefile

Loading...
View Options

devel/ocaml-findlib/distinfo

Loading...
View Options

devel/ocaml-findlib/files/patch-Makefile

Loading...
View Options

devel/ocaml-findlib/files/patch-src__bytes__Makefile

Loading...
View Options

devel/ocaml-findlib/files/patch-src_findlib-toolbox_Makefile

Loading...
View Options

devel/ocaml-findlib/files/patch-src_findlib_Makefile

Loading...
View Options

devel/ocaml-findlib/pkg-plist

Loading...
View Options

devel/ocaml-ipaddr/Makefile

Loading...
View Options

devel/ocaml-ipaddr/distinfo

Loading...
View Options

devel/ocaml-ipaddr/files/patch-lib_ipaddr.ml

Loading...
View Options

devel/ocaml-ipaddr/pkg-plist

Loading...
View Options

devel/ocaml-lwt/files/patch-src_unix_lwt__bytes.ml

Loading...
View Options

devel/ocaml-parmap/Makefile

Loading...
View Options

devel/ocaml-parmap/distinfo

Loading...
View Options

devel/ocaml-parmap/files/patch-Makefile.in

Loading...
View Options

devel/ocaml-parmap/pkg-plist

Loading...
View Options

devel/ocaml-pomap/Makefile

Loading...
View Options

devel/ocaml-ppx-tools/Makefile

Loading...
View Options

devel/ocaml-ppx-tools/distinfo

Loading...
View Options

devel/ocaml-ppx-tools/pkg-plist

Loading...
View Options

devel/ocaml-res/Makefile

Loading...
View Options

devel/ocaml-result/Makefile

Loading...
View Options

devel/ocaml-sexplib/Makefile

Loading...
View Options

devel/ocaml-sexplib/distinfo

Loading...
View Options

devel/ocaml-sexplib/files/patch-lib_conv.ml

Loading...
View Options

devel/ocaml-sexplib/files/patch-lib_lexer.mll

Loading...
View Options

devel/ocaml-sexplib/files/patch-lib_path.ml

Loading...
View Options

devel/ocaml-sexplib/files/patch-lib_pre__sexp.ml

Loading...
View Options

devel/ocaml-sexplib/files/patch-lib_sexp__intf.ml

Loading...
View Options

devel/ocaml-sexplib/pkg-descr

Loading...
View Options

devel/ocaml-sexplib0/Makefile

Loading...
View Options

devel/ocaml-sexplib0/distinfo

Loading...
View Options

devel/ocaml-sexplib0/pkg-descr

Loading...
View Options

graphics/ocaml-cairo/Makefile

Loading...
View Options

graphics/ocaml-cairo/distinfo

Loading...
View Options

graphics/ocaml-cairo/files/patch-issue-19

Loading...
View Options

lang/ocaml/Makefile

Loading...
View Options

lang/ocaml/distinfo

Loading...
View Options

lang/ocaml/files/patch-675e98a2.c

Loading...
View Options

lang/ocaml/files/patch-asmrun_amd64.S

Loading...
View Options

lang/ocaml/files/patch-asmrun_power.S

Loading...
View Options

lang/ocaml/files/patch-byterun_backtrace.c

Loading...
View Options

lang/ocaml/files/patch-byterun_caml_stack.h

Loading...
View Options

lang/ocaml/files/patch-configure

Loading...
View Options

lang/ocaml/files/patch-runtime_amd64.S

Loading...
View Options

lang/ocaml/files/patch-runtime_backtrace.c

Loading...
View Options

lang/ocaml/files/patch-runtime_caml_stack.h

Loading...
View Options

lang/ocaml/files/patch-runtime_power.S

Loading...
View Options

lang/ocaml/files/patch-runtime_startup__nat.c

Loading...
View Options

lang/ocaml/files/patch-testsuite_tests_asmcomp_power.S

Loading...
View Options

lang/ocaml/files/patch-testsuite_tools_asmgen__power.S

Loading...
View Options

lang/ocaml/pkg-plist

Loading...
View Options

net-p2p/mldonkey-gui/Makefile

Loading...
View Options

net-p2p/mldonkey/Makefile

Loading...
View Options

net-p2p/mldonkey/distinfo

Loading...
View Options

net-p2p/mldonkey/files/patch-src_utils_net_terminal.ml

Loading...
View Options

net/unison/Makefile

Loading...
View Options

net/unison/files/patch-ocaml_4.07-compat

Loading...
View Options

net/unison/files/patch-ocaml_4.07-compat-copyquoterem-01

Loading...
View Options

net/unison/files/patch-ocaml_4.07-compat-copyquoterem-02

Loading...
View Options

net/unison232/Makefile

Loading...
View Options

net/unison240/Makefile

Loading...
View Options

net/unison240/files/patch-ocaml8

Loading...
View Options

net/unison248/Makefile

Loading...
View Options

net/unison248/files/patch-ocaml8

Loading...
View Options

security/ocaml-ssl/Makefile

Loading...
View Options

security/ocaml-ssl/distinfo

Loading...
View Options

security/ocaml-ssl/pkg-plist

Loading...
View Options

security/sks/files/patch-ocaml408

Loading...
View Options

x11-toolkits/ocaml-lablgtk3/Makefile

Loading...
View Options

x11-toolkits/ocaml-labltk/Makefile

Loading...
View Options

x11-toolkits/ocaml-labltk/distinfo

Loading...