libcpufeats and amd64 assembly for sha256

[jmg]

Summary

There are three parts to this patch.

The first part adds a private lib that has a similar interface to cpu features
as the kernel has.

The second part is to complete the move of sha256 from userland to the kernel.

The third part is to add support for optimized assembly to sha256, and to use it when available.

[jmg]

fix libcrypt so it compiles.. This moves most of the cpufeats
work into the Makefile.inc in libmd..

[bapt]

lib/libmd/Makefile.inc

16

Are you sure uudecode is in the buildtools? if not it should be added to

[jmg]

lib/libmd/Makefile.inc

16

What are the rules on buildtools? I don't think uudecode has had any functional changes that would prevent the base system uudecode from working. As a quick grep of src/Makefile* didn't turn up anything, it isn't.

[jmg]

this time have libcpufeats added so it gets included in the diff...

[jmg]

fix README to not using X86_...

[jmg]

I plan to commit this next week if I don't get any reviews. This has been in review for over a month w/o comments.

bapt, I haven't heard back from you on my reply, so I'm going to assume that you're fine w/ uudecode not being in build tools. btw, we use uudecode for other parts of the system, like share/vt/fonts w/o it being part of build tools.

[imp]

Don't like the generated code that's committed to the tree, nor the yasm sources.

sys/crypto/sha2/sha256_sse4.o.uu

2

So why are we committing generated files?

sys/crypto/sha2/sha256_sse4.yasm

2

You should have a rule that fails the build if this is newer than the .o.

But committing yasm code to the tree is a first, and I've seen no discussion about it.

[kib]

In D2651#58647, @imp wrote:

Don't like the generated code that's committed to the tree, nor the yasm sources.

I have exactly the same opinion about this issue. IMO we should provide consistent source which can be self-rebuilt. We do accept .o blobs for obscure hardware, or keep such blobs in the ports. Having the sensitive crypto code in blob, for which the external tool is needed, does not feel right. Also note that there are many offspring x86 assemblers, which are known to be badly maintained and eventually fall into the stagnation.

I noted some options which could be tried, e.g. using intel mode for gas/clang, or whole file convertion to the att syntax.

Whatever the decision is, I dislike the .o for core kernel component very much.

[jmg]

In D2651#58654, @kib wrote:

In D2651#58647, @imp wrote:

Don't like the generated code that's committed to the tree, nor the yasm sources.

I have exactly the same opinion about this issue. IMO we should provide consistent source which can be self-rebuilt. We do accept .o blobs for obscure hardware, or keep such blobs in the ports. Having the sensitive crypto code in blob, for which the external tool is needed, does not feel right. Also note that there are many offspring x86 assemblers, which are known to be badly maintained and eventually fall into the stagnation.

I noted some options which could be tried, e.g. using intel mode for gas/clang, or whole file convertion to the att syntax.

Whatever the decision is, I dislike the .o for core kernel component very much.

I agree that committing yasm source is less than ideal, but the performance speed up it brings is worth it IMO, and yasm appears to be around for a while. If yasm becomes unmaintained, then we can decide if we want to revert this, but the original C source will still be available.

If either of you want to embark upon a project to convert or some how otherwise munge the yasm source to something more meaningful, feel free, but my time limit still stands. As I have talked w/ kib about this before on IRC multiple times, is that I HAVE attempted to get it to compile w/ it, and have spend a few hours working on it w/o any clear solution to the problem short of spending many more hours manually rewriting all the of the assembly along w/ the macros.

[jmg]

sys/crypto/sha2/sha256_sse4.yasm

2

we don't have a rule to fail the build when syscalls.master is newer than init_sysent.c, why here?

[imp]

I must say I'm disappointed by the attitude. Maybe it's an OK position (and maybe not), but the presentation rubs me the wrong way.

sys/crypto/sha2/sha256_sse4.yasm

2

Because this is a weird outlier? syscall.master is well understood. This isn't. you can't use that as an excuse.

[jmg]

Marking uudecode as done as bapt hasn't replied to my question for over a month and there are already existing requirements on uudecode.

[jmg]

add rule to tell the user how to rebuild the .o.uu when it is out
of date...

[jmg]

I have no plans to work on this in the future. Someone else is free to take this over and get it committed.