HomeFreeBSD

Fix deadlock in IO pipeline

Description

Fix deadlock in IO pipeline

In vdev_queue_aggregate() the zio_execute() bypass should not be
called under the vdev queue lock. This can result in a deadlock
as shown in the stack traces below.

Drop the vdev queue lock then walk the parents of the aggregate IO
to determine the list of component IOs to be bypassed. This can
be done safely without holding the io_lock since the new aggregate
IO has not yet been returned and its parents cannot change.

  • THREAD 1 ---

arc_read()

  zio_nowait()
    zio_vdev_io_start()
      vdev_queue_io() <--- mutex_enter(vq->vq_lock)
        vdev_queue_io_to_issue()
          vdev_queue_aggregate()
            zio_execute()
              zio_vdev_io_assess()
                zio_wait_for_children() <- mutex_enter(zio->io_lock)
  • THREAD 2 --- (inverse order)

arc_read()

  zio_change_priority() <- mutex_enter(zio->zio_lock)
    vdev_queue_change_io_priority() <- mutex_enter(vq->vq_lock)

Reviewed-by: Tom Caputi <tcaputi@datto.com>
Reviewed-by: Don Brady <don.brady@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #7307

Details

Provenance
Brian Behlendorf <behlendorf1@llnl.gov>Authored on Mar 16 2018, 11:46 PM
GitHub <noreply@github.com>Committed on Mar 16 2018, 11:46 PM
Parents
rG17dd88352e65: Allow QAT to handle non page-aligned buffers
Branches
Unknown
Tags
Unknown

Event Timeline

GitHub <noreply@github.com> committed rGa76f3d0437e5: Fix deadlock in IO pipeline (authored by Brian Behlendorf <behlendorf1@llnl.gov>).Mar 16 2018, 11:46 PM