I have committed part to properly break up the change...

changed to using M_ZERO instead... Maintainer should clean this up more.

switch to zero'ing all of memory, as selinfo and other fields aren't
properly initalized..

In D2890#56145, @mjg wrote:

This particular piece of code actually expects a zeroed struct. In particular, you can see that neither mtx_init nor knlist_init initialize the whole struct, leaving e.g.vpi_events and vpi_revents indeterminate. Then vn_pollrecord immediately examines vpi_revents of a newly added object and we are in trouble.

In other words, while it was mtx_init which complained about junk-filled memory, the issue is bigger. initialisation here needs to either be done with M_ZERO or extended to cover missed fields. Uninitialised parts also include something from struct knlist, so it's more than zeroing 2 shorts, but I'm too lazy to check how to do deal with that knlist.

tomorrow..

committed in svn commit: r284630 - head/usr.sbin/bhyve

addressed

address grehan's comments...

I'll look over the code more later.

looks like arc does support git trees too...

In D2842#55489, @emaste wrote:

In D2842#55479, @jmg wrote:

Read it, completely unconvinced that adding braces would have caught it, anyone who missed the strange indentation of two goto statements would have missed what extra braces ment... style(9) is there to help make code easier to read, but does not alleviate the developer from reading and understanding the code...

I think you've missed my point: in this case the braces aren't there to make it more likely someone would catch the extra goto fail in code review, but to make it less likely that a merge process ends up producing it at all.

I would recommend a statement that adds that the preferred/recommended is no extra braces if this must go in...

In D2842#54946, @emaste wrote:

It might have slipped through code review, but it's less likely to have been introduced in the first place if braces are used.

David Wheeler's got a pretty good description: http://www.dwheeler.com/essays/apple-goto-fail.html

needs work... style(9) is a guide, clearly not stricktly enforced, and if code is truely confusing w/o the braces, then add them, but changing style(9) is unneeded... The original example of double goto fail doesn't point out that:
if (case) {

goto fail;

}
goto fail;

looks fine, I haven't tested it though

In D1503#28407, @ae wrote:

Is there any news?

fix README to not using X86_...

this time have libcpufeats added so it gets included in the diff...

fix libcrypt so it compiles.. This moves most of the cpufeats
work into the Makefile.inc in libmd..

The nonce change and some of the define changes I mentioned are required to be fixed before I'd approve this change.

we should either include both w/ quotes or both w/ angle braces… I personally would prefer quotes, but that's me, but I won't object to angle braces.

hmm.. maybe we should put this hack into a different header file and just include that one… since all these files are in crypto/aesni, why not a local header (use quotes) that is named gnuhack.h or something (can't think of a good name right now)?

I can't think of anything else to change.

In D2177#22, @glebius wrote:

Robert, tracking it based on IP pair tuples can be achieved only by turning FLOWTABLE (or anything alike) on by default. This doesn't sound like what we want to do now.

In D2177#23, @glebius wrote:

Emeric, who observed collisions with old ip_id++ code, reports that with per-CPU code he doesn't see them:

https://lists.freebsd.org/pipermail/svn-src-head/2015-March/070091.html

I would dare to speculate, that the per-CPU code is more safe than a single atomic counter for all CPUs,
despite the birthday paradox. My point is that in the network stack we are trying to make network flows
affine to certain RX/TX queues in NICs, and at the same we are trying to make networking threads affine
to CPUs. So, what happens (at least we try to achieve that!) that a certain CPU serves a certain set of
IP flows. And making the ID counter not global, we make it overflow less often, making collisions less
probable. Of course, it is all very speculative. :)

I'd prefer if you would make the changes.

Closed by commit rP381886 (authored by @jmg).

As D1956 says, no point in working on this.

No point in working on this per latest request.

A spin lock is RIDICULOUSLY expensive, and where they are in the random(4) code is going to noticeably affect kernel performance. See locking(9).

Note: No comments will be addressed till it is decided the correct path forward. If others would like a single final patch, make the decision, but I won't break it up again after doing the work to make a single patch.

In D1956#39, @jhb wrote:

Just because people do not agree with you does not mean they are not reading your e-mails. They may still disagree. It's hard for me to see why loadavg() really needs crypto-quality RNG. Given the infrequent updates it would be a far lower bandwidth side channel than something like the hyperthreading SSL thing. Your refusal to really consider such comments raised by multiple people doesn't really give warm fuzzies. The fact is that there are a _lot_ of kernel APIs that are not safe from interrupt context. printf() is barely safe and it's the one case that arguably should be. Most code does not run in interrupt context. The callout stuff added some recent uses, but I suspect there are actually very few places that you would need to fix and you can fix them by removing C_DIRECT_EXEC. In fact, here's the entire list for C_DIRECT_EXEC. This does not look hard to audit:

In D1956#36, @jhb wrote:

I am still opposed to splatting spin locks all over the place. We have tools in place to catch violations, and you will just have to fix things like loadavg() appropriately (either by using something simpler or by moving the context it runs in, e.g. removing C_DIRECT).

In D2012#6, @delphij wrote:

Is the code at https://github.com/bitwiseshiftleft/crandom freely redistributable? I didn't find a license there, but it's okay if there is a license allows us to do it.

(I think it's probably a good idea to move cpu_random() and friends to the MD code and create a MI interface for them, as some other platforms may offer similar features in the future).

I assert that people can't well the difference between 1 and 2 and there are cases where picking 1 seems like a good choice, but later we discover that it should have been cryptographicly secure. Always using cryptographicly secure RNG ensures that an entire class of bugs is eliminated from FreeBSD. Even if 90% of the developers could tell the difference, we still should. The new ChaCha example in D2012, it would be comparable to an LCG most (99%) of the time.

All of these comments are exactly why we should make random able to be called in any context. We will continue to find many cases of this, and end up introducing more code than is worth it if we add LCG's to all of these cases.

add comment about confusion on how this new panic shouldn't really be new.

I reverted the yarrow spin lock changes, booted the kernel, and found
the following callchain:
lapic_handle_timer -> timercb -> handleevents -> callout_process ->
softclock_call_cc -> loadav -> arc4rand

Now that the scheduler has been "fixed", I'll try to run w/ normal mutexs and see if I have any issues. If that works, I'll update the patch, and also include an update for fortuna.

fixed compile, and included a test suite for random_uniform...

change random_uniform to eliminate a division/multiplication and
instead only use addition or simple bit operations, this should help
performance on embeded systems that don't have 64bit mul/div units..

update to address some of delphij's comments...

I'll update the patch to address the ones I agree with.

I agree with hselasky.. putting this as a module and kernel option means that we don't bloat the kernel more on embedded systems...

Sorry, I'll accept the _wrap.c change, but would prefer not to change _ghash.c as it's contributed code…

actually, I consider this a bug in clang.. requiring to cast a point to add const is stupid.. I understand requiring to remove it, but adding const should not generate a warning… This just adds const for no benefit and no change..

committed, why I couldn't do this in one step is beyond me… :(

committed

committed

I don't know the IPsec code well enough, but as long as everything has been tested, it looks ok.

remove the aesni parts as that still needs to be reviewed by Mike.