Looks good to me.

Looks fine to me. Just two small code nits.

In D11025#232116, @kib wrote:

In D11025#232110, @sobomax wrote:

@kib how difficult is it to also implement real NOTE_MONOTONIC, i.e. event that expires in monotonic time from the getgo? I think this would be very useful for real-world applications that care about precise timekeeping. We use CLOCK_MONOTONIC pretty much all across our code here. Yes, it can be added later on, but without it the API is somewhat incomplete IMHO.

In which way would this timer different from the existing timers without NOTE_ABSTIME flag ?

In D11025#228355, @kib wrote:

In D11025#228344, @ed wrote:

One suggestion, though: instead of calling this NOTE_MONOTONIC, would it make sense to use NOTE_ABSTIME instead?

NOTE_MONOTONIC sounds as if it uses CLOCK_MONOTONIC under the hood, even though your implementation still uses CLOCK_REALTIME, right?

Apple uses NOTE_ABSOLUTE, but their API is different from us, and from my understanding, glib maintainers, who asked for this feature, do not like Apple' edge-triggering feature of NOTE_ABSOLUTE.

I will rename the flag to NOTE_ABSTIME on next patch update.

Adjusted a bit as suggested.

Re-phrase a bit based on suggestions.

Reformat in-line example based on input from wblock.

Improve wording as suggested by the rpokala, bcr & bjk.

I've checked both forward and backward compatibility (3-way actually, old mdconfig+new kernel, old kernel + new mdconfig, new mdconfig+new kernel) everything seems to be working properly.

In D10457#216855, @imp wrote:

Love this idea, but the padding issue needs to be resolved.

Collapse if/return/else/return into a single return statement.

Rework previous change to restore ability to shutdown datagram
sockets to be easier to understand and put a comment explaining
what's going on and why.

I think Ed's comparison with other OSes here and in the relevant differential was not entirely correct. What linux does (tested with 4.4.0) when UDP socket is shut down is actually shutting down receiving end, so any threads that are blocked in recv() on that socket return. Still shutdown() system call itself returns ENOTCONN. FreeBSD on the other hand does not do anything for the socket, so that the threads just hang. I am pretty sure there are at least some software out there that relies on that behavior, at least in our case we do. Bumped into this after upgrading to the 11.0.

Looks like necessary functionality is already part of the -atime/-ctime/-mtime.

In D9171#191411, @wblock wrote:

Please also bump .Dd in getsockopt.2. Thanks!

In D9171#189873, @adrian wrote:

hiya,

cool. +1 from me.

can this be used to pass up per-packet timestamps from an underlying hardware device? Right now it looks like it's getting timestamped in the IP stack.

eg, if I craft up a way to say, provide a PHY/MAC timestamp and BSSID from a wifi driver up in the mbufs up to the IP stack, it'd be nice to have a socket option to have that data be attached.

Move so_options before padding. Should not be really necessary, that padding is for lower layers to use I think, not really to protect the structure size. But it does not hurt.

Not necessary it turns out.

@butcher says some improvements might be needed:

BB

This has been integrated already. GC this for good, so it does not stick in the middle of my list. :)

Yes, ok, looks like the world has taken another turn, I'll look into adapting some of the techniques for igb/ix to align interrupts properly.

Thanks @jhb for explanation. Would you mind sharing your ixgbe patches? Put them here as a review request for example or just email to me directly, so that I won't reinvent the wheel. Thanks!

@jhb in case if you are curious, here is the corresponding diff for the igb(4) driver. Not only it solves two issues that I've outlined already (i.e. popping up too many MSIX threads when on small hardware and not distributing threads properly WRT SMTs on high-end system), but also exports binding information into the userland so that we can see which queue handler is bound where and use it in our application to bind its netmap threads accordingly.

In D7001#146824, @jhb wrote:

1. Drivers are supposed to use bus_get_cpus() going forward and not directly inspect SMT vs not. The policy of what ends up in INTR_CPUS shouldn't belong in device drivers like igb(4) directly. It should belong elsewhere, and bus_get_cpus() allows that policy to be defined outside of individual drivers. Today INTR_CPUS excludes SMT threads by default and only includes CPUS "local" to the device in NUMA systems.