Could this be imported as is, and then when the hash modules are in and exported it could be switched over?

In D41509#1022312, @imp wrote:

I've created a hash module that has some of the sha routines. I don't have big issues connecting those.

I've created a hash module that has some of the sha routines. I don't have big issues connecting those.

Updated with all PR comments, save 1.

src/sys/contrib/dev/acpica/include/actypes.h:235:41: error: redefinition of typedef 'BOOLEAN' is a C11 feature [-Werror,-Wtypedef-redefinition]
typedef unsigned char BOOLEAN;

^

/usr/src/stand/efi/include/efidef.h:33:25: note: previous definition is here
typedef UINT8 BOOLEAN;

This broke the armv7 build...
https://ci.freebsd.org/job/FreeBSD-main-armv7-build/19710/
Please fix ASAP.

I'm not seeing any remaining problems here, seems to LGTM.

Just one minor nit

looks good now. Thanks for the split.

Added @jrtc27 and @mhorne to chime in concerning @imp's question concerning risc-v (and possibly armv7)

In D42459#969394, @imp wrote:

I generally like this change, one or two quibbles to work out.
I'd also split the lua and other stuff into separate commits (the old lua code will work with the new loader.efi, which I like as well).
But I can do the splitting if that's a hassle. This is otherwise fairly clean so I wouldn't mind a small amount of extra work.

Stripped out lua diff to submit separately

I generally like this change, one or two quibbles to work out.
I'd also split the lua and other stuff into separate commits (the old lua code will work with the new loader.efi, which I like as well).
But I can do the splitting if that's a hassle. This is otherwise fairly clean so I wouldn't mind a small amount of extra work.

Address Warner's comment regarding legacy hints and update the man page as suggested by Kyle

I'll take a look at the rest of this later...

Nothing too concerning, mostly some style complaints but as a general rule I'd like to keep luacheck happy for the times it does actually identify legitimate issues.

6. Updated documentation

5. refactored tests to make them easier to add to.

Updated diff with all comments addressed, specifically:

1. hashes.lua removed and moved into crypt.lua with no reexport
2. tests refactored
3. tests for NIST vectors for SHA256/512 added (and verified)
4. added "crypt:" prefix on password

In D41509#947276, @david_crossfamilyweb.com wrote:

Thinking about this, I can make a couple of changes that could make this more palatable:

1. move hashes into crypt and make it non-exported.

Thinking about this, I can make a couple of changes that could make this more palatable:

In D41509#947260, @jhb wrote:

In D41509#945967, @kevans wrote:

Ok, this is fine, but if we're going to go through the effort of exporting C routines to the lua environment I'd STRONGLY advocate just exposing crypt(3). For numerous reasons:

Unfortunately I don't think that's really feasible, particularly not without a build knob to turn it off. Keep in mind that biosloader is heavily constrained (under 512k-ish total) and we're already pushing the envelope on that, enough so that various downstreams do actually have to turn various things off to fit everything they need in loader (this is why I was wondering about the size requirements of moving sha256/512 of the GELI knob).

But that just means we don't support non-plaintext passwords for the BIOS loader, right? It would basically mean that we would not be adding this new feature for that case (which is becoming more and more obsolete) but would be adding the new feature everywhere else. That tradeoff is probably fine. Exporting just crypt(3) also I think narrows your concerns about other people deciding to use SHA hashes in other places.

In D41509#947261, @imp wrote:

btw, lua's math.* aren't included because float/double math isn't included and I thought there's very little there that doesn't use it. If my thoughts are wrong, of course we can reconder.

In D41509#947260, @jhb wrote:

But that just means we don't support non-plaintext passwords for the BIOS loader, right?

In D41509#945967, @kevans wrote:

Ok, this is fine, but if we're going to go through the effort of exporting C routines to the lua environment I'd STRONGLY advocate just exposing crypt(3). For numerous reasons:

Unfortunately I don't think that's really feasible, particularly not without a build knob to turn it off. Keep in mind that biosloader is heavily constrained (under 512k-ish total) and we're already pushing the envelope on that, enough so that various downstreams do actually have to turn various things off to fit everything they need in loader (this is why I was wondering about the size requirements of moving sha256/512 of the GELI knob).

(Thanks for the correction in the earlier comment)

In D41509#945885, @kevans wrote:

In D41509#945841, @david_crossfamilyweb.com wrote:

Uncomfortable how? This is a very straightforward implementation of SHA256/SHA512 modeled after existing LUA SHA2 libraries. I only reimplemented because:

The problem is that now the implementation will exist, and thus it will get used for uses that we aren't anticipating and it becomes crypto code that actually should be audited.

1. existing ones were based on lua <5.4 and relied on the bit32 library which is not in LUA54
2. were vastly overcomplicated and not BSD licensed (ones that worked on every version of LUA and included multiple different sub-implementations optimized for a vast number of possible LUA configurations.)
3. were incapable of being used in this context (they would return hex encoded values, where the crypt(3) algorithms require the raw binary to feed into subsequent stages

Exposing SHA2 from to loader in this way would be complicated that would require either:

1. exposing complex 'context' objects to the underlying scripting language, since crypt(3) builds up the hashes with multiple updates, with those needing to be constructed/initialized, returned, and passed to subsequent calls for update and finalization
   • or -
2. rewriting the crypt(3) routines to concatenate their entire requests together into a single call such that it could be wrapped into a single shaX() that would return the binary result. I guess technically the crypt(3) implementation could unenode the hex to get back the binary.. gross. Regardless this would mean fairly substantial rewrite of the crypt implementation, and constructing kilobyte strings to sling the data around in the interpreter.

Honestly the crypt(3) implementation frightens me much worse, that is a grotesquely overly complicated algorithm.

If we were to export anything to the loader(8) level the right choice would be libcrypt; I looked at doing that originally but figured this would much less invasive to loader and easier to get in since it was entirely in the interpreter level.

I don't know where you get the idea that you'd need crypt(3), to be honest. In my first comment, I noted that we already have sha256/sha512 available in libsa that could be broken out relatively easily (same API that's seen in libmd). You don't even need the complex context you suggested, you can still do closures just fine in C and keep your consumers as-is. I'd even be willing to write said replacement if you'd use it, but I need to check how much size we gain in non-default loader configurations if we move sha256/sha512 out from behind the GELI knob.

But, what exactly is uncomfortable? This is used ONLY for doing the password hash, nothing is being validated/signed against this, there's no network access, it only even comes into play to match a provided password that someone who already had root access had to set.
It is *explicitly* extensively tested (I tested every sequence of 0s from none to 65536 and compared to sha256/sha512) offline (I didn't include those)
It is *implicitly* extensively tested (I included all provided test vectors of crypt for $5$ and $6$), (these are included in the source) and they all passed, which puts the sha libraries through a rather thorough torture test themselves.

Already generally answered above, but to re-iterate: for your current use it's perfectly fine, but as soon as it's made available it becomes a public API that could actually get used for more sensitive applications that we didn't anticipate or want.

In D41509#945936, @kevans wrote:

In D41509#945887, @delphij wrote:

Why do we even have support of loader password other than unlocking the GELI provider in the first place? Or in other words, what was the threat model it's trying to protect the user from?

No idea where it originated, I just ported it from 4th.

I think password.lua should only support entering passphrases for unlocking encrypted providers, and absolutely nothing beyond that. Storing passphrase material in the unencrypted boot storage is calling all kinds of problems and I can't see real security benefits of doing over using e.g. GELI.

IMO we can't easily get rid of these; one is to prevent boot entirely if you don't have the password, the other is to just prevent user from getting to the loader menu. At least for bootlock_password, the most obvious application I think is to prevent a local user from pushing the system into single-user mode assuming they don't have any other access to the box.

In D41509#945887, @delphij wrote:

Why do we even have support of loader password other than unlocking the GELI provider in the first place? Or in other words, what was the threat model it's trying to protect the user from?

I think password.lua should only support entering passphrases for unlocking encrypted providers, and absolutely nothing beyond that. Storing passphrase material in the unencrypted boot storage is calling all kinds of problems and I can't see real security benefits of doing over using e.g. GELI.

These have existed for decades and pretty much every boot loader and every link in the booting process has optional passwords in place as a check against random people interfering with the boot process. BIOS has it, openboot has it, grub has it.

In D41509#945887, @delphij wrote:

Why do we even have support of loader password other than unlocking the GELI provider in the first place? Or in other words, what was the threat model it's trying to protect the user from?

Why do we even have support of loader password other than unlocking the GELI provider in the first place? Or in other words, what was the threat model it's trying to protect the user from?

In D41509#945841, @david_crossfamilyweb.com wrote:

Uncomfortable how? This is a very straightforward implementation of SHA256/SHA512 modeled after existing LUA SHA2 libraries. I only reimplemented because:

Uncomfortable how? This is a very straightforward implementation of SHA256/SHA512 modeled after existing LUA SHA2 libraries. I only reimplemented because:

1. existing ones were based on lua <5.4 and relied on the bit32 library which is not in LUA54
2. were vastly overcomplicated and not BSD licensed (ones that worked on every version of LUA and included multiple different sub-implementations optimized for a vast number of possible LUA configurations.)
3. were incapable of being used in this context (they would return hex encoded values, where the crypt(3) algorithms require the raw binary to feed into subsequent stages

To be frank, stand/lua/hashses.lua makes me a bit uncomfortable... we have sha256/sha512 in libsa if GELI is enabled, I wonder how expensive it would be to re-scope those out to libsa out of the GELI option and break those out to Lua. Less worried about the latter part being too chunky than the former...