Page MenuHomeFreeBSD
Differential D37634

devel/cl-asdf: Update to 3.3.7
ClosedPublic
Actions

Authored by jrm on Dec 8 2022, 5:06 PM.
Tags
None
Referenced Files
F112866966: D37634.id113942.diff
Sun, Mar 23, 7:01 PM
Unknown Object (File)
Sun, Mar 16, 2:37 AM
Unknown Object (File)
Tue, Mar 11, 6:25 AM
Unknown Object (File)
Sun, Mar 9, 8:47 AM
Unknown Object (File)
Sun, Mar 9, 5:36 AM
Unknown Object (File)
Fri, Mar 7, 1:04 AM
Unknown Object (File)
Thu, Mar 6, 9:15 PM
Unknown Object (File)
Wed, Mar 5, 11:23 AM
View All 52 Files
Subscribers
emaste
mat

Details

Reviewers
olgeni
ashish
krion
Commits
R11:0c35c3d61d20: devel/cl-asdf: Update to 3.3.7
Summary

Also:

  • resolve warnings from SBCL about the ASDF port being outdated and subsequently disabled [0] by installing version.lisp-expr
  • re-add the devel/cl-asdf-sbcl port to the tree in a subsequent commit because it is required to avoid filesystem violation errors when building *-sbcl ports with Poudriere.

[0]

WARNING: You are using ASDF version 3.3.1 (probably from (require "asdf") or loaded by quicklisp) and have an older version of ASDF (and older than 2.27 at that) registered at #P"/usr/local/lib/common-lisp/asdf/asdf.asd". Having an ASDF installed and registered is the normal way of configuring ASDF to upgrade itself, and having an old version registered is a configuration error. ASDF will ignore this configured system rather than downgrade itself. In the future, you may want to either: (a) upgrade this configured ASDF to a newer version, (b) install a newer ASDF and register it in front of the former in your configuration, or (c) uninstall or unregister this and any other old version of ASDF from your configuration. Note that the older ASDF might be registered implicitly through configuration inherited from your system installation, in which case you might have to specify :ignore-inherited-configuration in your in your ~/.config/common-lisp/source-registry.conf or other source-registry configuration file, environment variable or lisp parameter. Indeed, a likely offender is an obsolete version of the cl-asdf debian or ubuntu package, that you might want to upgrade (if a recent enough version is available) or else remove altogether (since most implementations ship with a recent asdf); if you lack the system administration rights to upgrade or remove this package, then you might indeed want to either install and register a more recent version, or use :ignore-inherited-configuration to avoid registering the old one. Please consult ASDF documentation and/or experts.

Diff Detail

Repository
rP FreeBSD ports repository
Lint
No Lint Coverage
Unit
No Test Coverage
Build Status
Buildable 48652
Build 45538: arc lint + arc unit

Event Timeline

jrm created this revision.Dec 8 2022, 5:06 PM
Herald added a subscriber: mat. · View Herald TranscriptDec 8 2022, 5:06 PM
jrm requested review of this revision.Dec 8 2022, 5:06 PM
Harbormaster completed remote builds in B48651: Diff 113942.Dec 8 2022, 5:06 PM
jrm updated this revision to Diff 113947.Dec 8 2022, 6:57 PM

Remove some commented code in asdf-init.lisp

Harbormaster completed remote builds in B48652: Diff 113947.Dec 8 2022, 6:57 PM
jrm updated this revision to Diff 113954.Dec 8 2022, 10:29 PM

devel/cl-asdf-sbcl: New port to install ASDF FASL for SBCL

With a few tweaks to devel/cl-asdf and re-adding devel/cl-asdf-sbcl, our common
lisp ports behave as before, unless certain conditions are not met.

Details:

  • lang/sbcl has its own bundled asdf, but it still depends on devel/cl-asdf, presumably because devel-cl-asdf installs ${LOCALBASE}/etc/asdf-init.lisp.
  • If we install (the updated) devel/cl-asdf without also installing (the re-added) devel/cl-asdf-sbcl, ASDF will start trying to write files under ${LOCALBASE}/lib/common-lisp on users' systems.
  • lang/sbcl cannot depend on devel/cl-asdf-sbcl, because this would create a dependency loop.

With this update, users can get into trouble if they install some lisp port
without also installing the assocaited fasl port. If they don't, we're back to
having asdf trying to write things under ${LOCALBASE}/lib/common-lisp.

I will take more time to look deeper into the ASDF code and documentation to see
what the best way is to eiter tell it to stop automatically building the fasl
files, or build and install everything under the user's home directory.

If we did want remove SBCL's dependecy on devel/cl-asdf, perhaps we could
decouple devel/cl-asdf and asdf-init.lisp and create a new port to just install
the configuration.

Harbormaster completed remote builds in B48656: Diff 113954.Dec 8 2022, 10:29 PM
jrm added a reviewer: krion.Dec 8 2022, 10:36 PM
jrm updated this revision to Diff 146837.Nov 22 2024, 3:29 PM

Update to 3.3.7 and re-add devel/cl-asdf-sbcl to the tree

Harbormaster completed remote builds in B60726: Diff 146837.Nov 22 2024, 3:29 PM
jrm retitled this revision from devel/cl-asdf: [WIP] Update to 3.3.6 to devel/cl-asdf: Update to 3.3.7.Nov 22 2024, 3:32 PM
jrm edited the summary of this revision. (Show Details)
jrm updated this revision to Diff 146849.Nov 22 2024, 8:24 PM
jrm edited the summary of this revision. (Show Details)

devel/cl-asdf-sbcl: Prevent the fasl file from being installed under the build/ directory.

Harbormaster completed remote builds in B60735: Diff 146849.Nov 22 2024, 8:24 PM
jrm updated this revision to Diff 147685.Dec 8 2024, 12:01 AM

Split changes into multiple reviews

Harbormaster completed remote builds in B61026: Diff 147685.Dec 8 2024, 12:01 AM
emaste added a subscriber: emaste.Dec 10 2024, 4:10 PM
jrm updated this revision to Diff 147982.Dec 15 2024, 5:52 PM

Handle removal of asdf-init.lisp

Harbormaster completed remote builds in B61171: Diff 147982.Dec 15 2024, 5:52 PM
This revision was not accepted when it landed; it landed in state Needs Review.Dec 20 2024, 2:48 AM
Closed by commit R11:0c35c3d61d20: devel/cl-asdf: Update to 3.3.7 (authored by jrm). · Explain Why
This revision was automatically updated to reflect the committed changes.
jrm added a commit: R11:0c35c3d61d20: devel/cl-asdf: Update to 3.3.7.

Revision Contents
Changeset List

PathSize
devel/
cl-asdf/
32 lines
6 lines
files/
23 lines
78 lines

Diff 113947

View Options

devel/cl-asdf/Makefile

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec
View Options

devel/cl-asdf/distinfo

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec
View Options

devel/cl-asdf/files/asdf-init.lisp

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec
View Options

devel/cl-asdf/pkg-plist

  • This file was added.
The goal of this branch is to simplify the data structures used to describe
session and crypto op parameters.
jhbAuthorUnsubmitted
Done
Inline Actions

I won't commit this file, but it lives in the branch and ended up in the review as a result.

jhb: I won't commit this file, but it lives in the branch and ended up in the review as a result.
- replace linked/list of cryptoini for session init with a structure
- explicit IV length?
- replace linked/list of cryptodesc with members in struct cryptop
- maybe require data to always be laid out as
AAD || IV (if not sideband) || DATA || TAG/MAC
(had to scrap this due to IPsec AH and permit arbitrary IV and
digest offsets)
- generate IV if needed in common code, not drivers
- add flag for hash verify rather than hash compute
- change copyto/copyback to operate on the crp instead of the fields
- permit separate in/out buffers?
- need to ensure crypto_getreq() zeroes the crp (it does)
- change for AEAD:
- previously with AES+HMAC crda covered all of data to auth, now
need to hash AAD region + payload. If IV needs to be hashed,
AAD should cover IV.
- Notes on IVs:
- CRYPTO_F_IV_GENERATE (encrypt only)
- generate full IV and copy to crp_iv_start before encryption
- CRYPTO_F_IV_SEPARATE (both, cannot use with GENERATE)
- IV is in crp_iv, crp_iv_start should be ignored
- neither flag
- full IV is in data stream at crp_iv_start
- AES-GCM now uses CIPHER_MODE_AEAD which ignores most of csp_auth_*
except for csp_auth_mlen
- Eventually retire 3 separate GMAC algs and repurpose NIST_GMAC as a single
MAC?
- Maybe retire driver HIDs with 'struct cryptocap' pointers instead? We
could just use a TAILQ instead of an array
- Separate session operation to rekey instead of implicit rekeying?
Implicit klen is not really great for auth, for example
- XXX: Need to think about how to define GMAC in the new world order. I
think it should be CSP_MODE_DIGEST, but with csp_cipher_ivlen set?
- Yes, GMAC is CSP_MODE_DIGEST with payload area describing data to hash.
It uses csp_ivlen.
- Maybe change crp_cipher_key to not change session key but only the
specific operation?
- Need to figure out what to do about accelerated software vs plain
software
- maybe need "priority" and a "session probe/test" instead of just
"newsession"
- Add a reference count on sessions
- crypto_getop would take a session pointer and set crp_session along
with bumping refcount now
- Kill the UIO usage in cryptodev.c, it's silly (done)
- GCM IV length is kind of busted, perhaps need IV length as part of crp
instead of csp?
- Testing
+ cryptocheck
+ soft
+ ccr
- ipsec
- auth
- hmac-md5
- hmac-sha1
- keyed-md5
- keyed-sha1
- null
- hmac-sha2-256
- hmac-sha2-384
- hmac-sha2-512
- hmac-ripemd160
- aes-xcbc-mac
- tcp-md5
- encrypt
- des-cbc
- 3desc-cbc
- null
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- rijndael-cbc
- aes-ctr
- aes-gcm16
- camellia-cbc
- 3des-hmac-md5
+ soft
- aes-cbc-hmac-sha
+ soft
- aes-cbc
+ soft
- aes-ctr-hmac-sha2-512
+ soft
- aes-ctr
+ soft
- aes-gcm-256
+ soft
- aes-gcm
+ soft
- blowfish
+ soft
- camellia
+ soft
- cast128
+ soft
- des-32iv
+ soft
- des-deriv
+ soft
- hmac-sha1
+ soft
- hmac-sha2-256
+ soft
- hmac-sha2-384
+ soft
- hmac-sha2-512
+ soft
- ipcomp
- doesn't work with stock FreeBSD
- null-ipcomp
- doesn't work with stock FreeBSD
- null
+ soft
- rijndael-hmac-sha2-256
+ soft
- rijndael
+ soft
- tcp-md5
- needs kernel with TCP_SIGNATURE
- Tested with cryptocheck (re-check)
- cryptosoft
- aesni
- blake2
- ccr
- Needs testing:
- ccp
- aesni with sha
- hifn
- kgssapi_krb5
- ubsec
- padlock
- safe
- GELI
- armv8_crypto (aarch64)
- glxsb (i386)
- sec (ppc)
- cesa (armv7)
- cryptocteon (mips64)
- nlmsec (mips64)
- Benchmarks
- aes-gcm setup
- frodo
- kldload ipsec
- ifconfig cc0 inet 172.16.0.1/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/source-aes-gcm-256.conf
- netperf -H 172.16.0.2 -cC
- sam
- kldload ipsec
- ifconfig cc0 inet 172.16.0.2/24 alias
- setkey -f /mnt/jhb/work/git/netperf/IPSEC/Configs/dut-aes-gcm-256.conf
- netserver
- clean svn (r348883)
- no ipsec 44000 18% 25%
- null 5500 28% 28%
- aes-gcm
- cryptosoft 550 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ocf_rework (r348883)
- no ipsec 44000 18% 25%
- null 5800 28% 28%
- aes-gcm
- cryptosoft 580 26% 17%
- aesni 3000 28% 26%
- ccr 4300 45% 45%
- ktls using aes-gcm
- frodo
- ktls.sh
- kldload ktls_ocf
- service nginx start
- sam
- normal ktls client
- clean svn (r354096)
- cryptosoft 2.9 Gbps 76-80% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 46-44% CPU
- ocf_rework (r354096)
- cryptosoft 2.9 Gbps 100% CPU
- aesni 27 Gbps 63-66% CPU
- ccr 30 Gbps 40-44% CPU
- dd /dev/zero to 4g md geli using AES-XTS / HMAC/SHA256
- svn
- software 43.002561 secs (49938506 bytes/sec)
- ccr 17.595058 secs (122050390 bytes/sec)
- ocf_rework
- software 43.544508 secs (49316981 bytes/sec)
- ccr 19.096170 secs (112456250 bytes/sec)
- dd /dev/zero to 4g md geli using AES-XTS
- svn
- software 34.214655 secs (125530048 bytes/sec)
- aesni 10.575568 secs (406121652 bytes/sec)
- ccr 10.530135 secs (407873915 bytes/sec)
- ocf_rework
- software 34.942433 secs (122915518 bytes/sec)
- aesni 10.498557 secs (409100708 bytes/sec)
- ccr 11.955181 secs (359255745 bytes/sec)
- steps to convert a driver
- add probesession
- cri -> csp
- crd -> crp fields
- axe crypto_register
+ rename CRYPTO_BUF_IOV -> CRYPTO_BUF_UIO
+ make key pointers const in csp and crp
(keep drivers from XOR'ing keys with IPAD/OPAD)
+ kill klen in crp (assume they match session)
+ allow NULL keys during setup but require klen to be
set during probesession / newsession, also have to require a key
when processing
+ change klen to bytes?
- sysctl node to list crypto drivers
- update python test to use this
- update places for new sysctl names
+ csp_lookup_hash() returns auth_hash *?
+ csp_lookup_cipher() returns enc_xform *?
+ pull checks into crypto_newsession
+ checks on mlen
+ csp_auth_klen 0 vs !0
+ kill CSP_IV_VALID and update drivers to all bail for flags != 0 to
permit adding new flags in the future
- update crypto.9
+ bus_dmamap_load_crp()
+ hifn
+ sec (ppc)
+ safe
+ cesa (armv7)
+ ubsec