fix libcrypt so it compiles.. This moves most of the cpufeats
work into the Makefile.inc in libmd..

The nonce change and some of the define changes I mentioned are required to be fixed before I'd approve this change.

we should either include both w/ quotes or both w/ angle braces… I personally would prefer quotes, but that's me, but I won't object to angle braces.

hmm.. maybe we should put this hack into a different header file and just include that one… since all these files are in crypto/aesni, why not a local header (use quotes) that is named gnuhack.h or something (can't think of a good name right now)?

I can't think of anything else to change.

In D2177#22, @glebius wrote:

Robert, tracking it based on IP pair tuples can be achieved only by turning FLOWTABLE (or anything alike) on by default. This doesn't sound like what we want to do now.

In D2177#23, @glebius wrote:

Emeric, who observed collisions with old ip_id++ code, reports that with per-CPU code he doesn't see them:

https://lists.freebsd.org/pipermail/svn-src-head/2015-March/070091.html

I would dare to speculate, that the per-CPU code is more safe than a single atomic counter for all CPUs,
despite the birthday paradox. My point is that in the network stack we are trying to make network flows
affine to certain RX/TX queues in NICs, and at the same we are trying to make networking threads affine
to CPUs. So, what happens (at least we try to achieve that!) that a certain CPU serves a certain set of
IP flows. And making the ID counter not global, we make it overflow less often, making collisions less
probable. Of course, it is all very speculative. :)

I'd prefer if you would make the changes.

Closed by commit rP381886 (authored by @jmg).

As D1956 says, no point in working on this.

No point in working on this per latest request.

A spin lock is RIDICULOUSLY expensive, and where they are in the random(4) code is going to noticeably affect kernel performance. See locking(9).

Note: No comments will be addressed till it is decided the correct path forward. If others would like a single final patch, make the decision, but I won't break it up again after doing the work to make a single patch.

In D1956#39, @jhb wrote:

Just because people do not agree with you does not mean they are not reading your e-mails. They may still disagree. It's hard for me to see why loadavg() really needs crypto-quality RNG. Given the infrequent updates it would be a far lower bandwidth side channel than something like the hyperthreading SSL thing. Your refusal to really consider such comments raised by multiple people doesn't really give warm fuzzies. The fact is that there are a _lot_ of kernel APIs that are not safe from interrupt context. printf() is barely safe and it's the one case that arguably should be. Most code does not run in interrupt context. The callout stuff added some recent uses, but I suspect there are actually very few places that you would need to fix and you can fix them by removing C_DIRECT_EXEC. In fact, here's the entire list for C_DIRECT_EXEC. This does not look hard to audit:

In D1956#36, @jhb wrote:

I am still opposed to splatting spin locks all over the place. We have tools in place to catch violations, and you will just have to fix things like loadavg() appropriately (either by using something simpler or by moving the context it runs in, e.g. removing C_DIRECT).

In D2012#6, @delphij wrote:

Is the code at https://github.com/bitwiseshiftleft/crandom freely redistributable? I didn't find a license there, but it's okay if there is a license allows us to do it.

(I think it's probably a good idea to move cpu_random() and friends to the MD code and create a MI interface for them, as some other platforms may offer similar features in the future).

I assert that people can't well the difference between 1 and 2 and there are cases where picking 1 seems like a good choice, but later we discover that it should have been cryptographicly secure. Always using cryptographicly secure RNG ensures that an entire class of bugs is eliminated from FreeBSD. Even if 90% of the developers could tell the difference, we still should. The new ChaCha example in D2012, it would be comparable to an LCG most (99%) of the time.

All of these comments are exactly why we should make random able to be called in any context. We will continue to find many cases of this, and end up introducing more code than is worth it if we add LCG's to all of these cases.

add comment about confusion on how this new panic shouldn't really be new.

I reverted the yarrow spin lock changes, booted the kernel, and found
the following callchain:
lapic_handle_timer -> timercb -> handleevents -> callout_process ->
softclock_call_cc -> loadav -> arc4rand

Now that the scheduler has been "fixed", I'll try to run w/ normal mutexs and see if I have any issues. If that works, I'll update the patch, and also include an update for fortuna.

fixed compile, and included a test suite for random_uniform...

change random_uniform to eliminate a division/multiplication and
instead only use addition or simple bit operations, this should help
performance on embeded systems that don't have 64bit mul/div units..

update to address some of delphij's comments...

I'll update the patch to address the ones I agree with.

I agree with hselasky.. putting this as a module and kernel option means that we don't bloat the kernel more on embedded systems...

Sorry, I'll accept the _wrap.c change, but would prefer not to change _ghash.c as it's contributed code…

actually, I consider this a bug in clang.. requiring to cast a point to add const is stupid.. I understand requiring to remove it, but adding const should not generate a warning… This just adds const for no benefit and no change..

committed, why I couldn't do this in one step is beyond me… :(

committed

committed

I don't know the IPsec code well enough, but as long as everything has been tested, it looks ok.

remove the aesni parts as that still needs to be reviewed by Mike.