This is the blackbox recorder code we discussed in the transport session at the BSDCan Developer Summit.
It allows you to capture events on a TCP connection in a ring buffer. It stores metadata with the event. It optionally stores the TCP header associated with an event (if the event is associated with a packet) and also optionally stores information on the sockets.
It supports setting a log ID on a TCP connection and using this to correlate multiple connections that share a common log ID.
You can program the system to put the connections in different modes. If we are doing a coordinated test with a particular connection, we may tell the system to put it in mode 4 (continuous dump). Or, if we just want to monitor for errors, we can put it in mode 1 (ring buffer) and dump all the ring buffers associated with the connection ID when we receive an error signal for that connection ID. You can set a default mode that will be applied to a particular ratio of incoming connections. You can also manually set a mode using a socket option.
Also, this is a fairly simplistic example, since it only provides the most basic of probes. @rrs has added quite an abundance of probes in his work.
There is a user-space program which we plan to commit as a port. It reads data from the log device and outputs pcapng files. (Write me for a copy of the user-space program.)