Page MenuHomeFreeBSD

aacraid: reduce max I/O size to avoid DMA issues
ClosedPublic

Authored by luporl on Jun 17 2021, 12:11 PM.

Details

Summary

Reserve one page for the DMA subsystem, that may need it when the I/O buffer is not page aligned.

Without this change, writes with the maximum allowed size failed, if:

  • physical memory was fragmented, making it necessary to use one DMA segment for each page
  • the buffer to be written was not page aligned, causing the DMA subsystem to need one extra segment

In the scenario above, the DMA subsystem would run out of segments, resulting in a write with no SG segments, that would fail.
(See also D30799, that improves DMA error reporting in aacraid)

Test Plan

Tested on a Talos II machine with an aacraid controller.

Without this change, newfs_msdos /dev/da0p1 had about 50% chance of failing.
This also happened in large writes and when kldxref was run after initial install.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

This is similar to the hacks we've done elsewhere in the system. Does it need to be also limited by MAXPHYS?

sys/dev/aacraid/aacraid_cam.c
1207

This seems unrelated...

sys/dev/aacraid/aacraid_cam.c
1207

In that I'd make it a separate commit.

Address review's comments.

Common filesystem operations, such as read and write, seem to work fine without limiting max I/O size by maxphys, probably because the upper layers never exceed this.
But maybe some other use cases need it, so I've added it.

By the way, it seems that what exposed this issue in aacraid was the bump in MAXPHYS, from 128K to 1M.

luporl marked 2 inline comments as done.
luporl added inline comments.
sys/dev/aacraid/aacraid_cam.c
1207

Moved to D30843.

thanks for the split. these look great!

This revision is now accepted and ready to land.Jun 21 2021, 3:18 PM
This revision was automatically updated to reflect the committed changes.
luporl marked an inline comment as done.