HomeFreeBSD

Make nvme(4) driver some more NUMA aware.

Description

Make nvme(4) driver some more NUMA aware.

  • For each queue pair precalculate CPU and domain it is bound to.

If queue pairs are not per-CPU, then use the domain of the device.

  • Allocate most of queue pair memory from the domain it is bound to.
  • Bind callouts to the same CPUs as queue pair to avoid migrations.
  • Do not assign queue pairs to each SMT thread. It just wasted

resources and increased lock congestions.

  • Remove fixed multiplier of CPUs per queue pair, spread them even.

This allows to use more queue pairs in some hardware configurations.

  • If queue pair serves multiple CPUs, bind different NVMe devices to

different CPUs.

MFC after: 1 month
Sponsored by: iXsystems, Inc.

Details

Provenance
mavAuthored on
Parents
rS352629: Pull in r372606 from upstream llvm trunk (by Sanjay Patel):
Branches
Unknown
Tags
Unknown