It may reduce probability, but it does not fix the problem, plus it makes lseek() to lie, that may or may not be good, depending on situation. I don't think this worth extra activity right now if https://github.com/openzfs/zfs/pull/15571, hopefully fixing the issue, land next days.

@rmacklem Yes, you should open PR in OpenZFS git. We are trying to keep FreeBSD ZFS code in sync with upstream, allowing out of the base builds. But considering you also extend FreeBSD KPI, I guess the process should include first extending the KPI and bumping FreeBSD version, and then opening upstream PR ifdef'ing that version, since the same upstream ZFS sources are expected to work on any FreeBSD version starting from 12.2 and up.

But I am still not getting sense of remaining scsiio_req->DataLength == 0 check.

Was I looking grumpy? I didn't mean to, sorry if so.

This version still may go to sleep with MPI3MR_FLAGS_SHUTDOWN flag set, that is suboptimal.

@jhb Won't the block before drain cause deadlock? At least that is how I read the man page.

This patch looks OK to me, but I generally have difficulties to understand why we need this callout_owner flag. callout_stop() should not be a reason. I am just not sure about semantics of mpi3mr_flush_io().

I guess the idea could be not try to handle events while resetting. I wonder if we could block new events before calling mpi3mr_cleanup_event_taskq(), unless we do it already.

Abandoning this review in favor of @imp ones.

Makes sense, but while there I wonder what is the SBT_1S * 90 timeout? Why not the value in CCB received from CAM? Is it something else?

The lock seems should protect not only msleep(), that is indeed pointless, but also both MPI3MR_FLAGS_SHUTDOWN accesses and also wakeup(&sc->watchdog_chan) in mpi3mr_pci_detach(). Otherwise the thread may not exit until the timeout expiration, that is a waste of time.

I have no clue where are they are coming from, it if not needed -- sure, axe them out.

Mutex is definitely wrong there, and I think I actually hit the panic on it on one of just several reboots, but I wonder if something needs to protect from task addition between the drain and block, if it is important.

Would be nice to change all tags that way, but this was just required the change any way.

I also suspect that most of bus_dmamem_alloc() calls could use BUS_DMA_WAITOK, but I haven't looked on the driver locking yet.

Add BUS_DMA_NOWAIT to bus_dmamap_load() with mpi3mr_memaddr_cb() callback. They by definition must be synchronous.

It looks OK to me, except getsbinuptime() by default gives a millisecond precision, not microsecond. Not a huge deal probably. Should average over time if it happen often.

After more experiments I found that my problem is really caused by false timeout, just 60 seconds may still be not enough. It seems that first 1.5 minutes after hot-plug the SSD is doing some sort of internal initialization and can not process the namespace delete command, postponing it, that causes the timeout. Timeout increase to couple minutes seems to fix the issue. Increase to 60 seconds at least reduces the time frame after hot-plug when the issue is reproducible.

I've found that increased timeout does not really help the CD8 SSDs, so I'll continue looking. Though I still think that coherency with Linux should be good.

I have no objections to commit this directly for the release. But to upstream I would commit a bigger patch.

@cperciva Please open PR on the OpenZFS github first, so that more people could review it. After being merged there it will be merged into FreeBSD.

In D41991#957622, @mjg wrote:

for example there was panic under load when running poudriere, i don't know if that is fixed

@mjg Does any specific bug come to your mind?

In D41627#953185, @dev_submerge.ch wrote:

The patch I'm working on also does the latter. With a reduced buffer size, I'm confident that we can have less latency at a 4ms interval than currently at 2ms. I think that would make 4ms an acceptable default for everybody.

I didn't promise it to be easy, but I think USB transfer size should be as close to sound(4) fragment size, if it can't be equal, as for other sound cards.

As I can see, uaudio_buffer_ms is primarily used to set intr_frames variable. I haven't dug deep, but looks like it controls the size of individual USB transfer, which I suppose affects USB controller interrupt rate, or at very least the rate of its DMA transfers. Frequent DMA transfers may prevent deep sleeps on CPU package-level. Frequent interrupts in addition wake up at least one CPU core. It all burns a lot of power, that may be critical for laptops and embedded systems, while not every application require so low latency. I generally don't like tunables, and I would be happy to remove this one, but I don't like the idea of getting 1KHz of USB interrupts for every sound application. Sound subsystem already has its own concept of buffer sizes and latency control. Can't those values be reused here without additional knobs to only have level of overhead required by specific application?