Page MenuHomeFreeBSD

vmm: Skip TMR rebuilds for edge-to-edge IOAPIC writes
Needs ReviewPublic

Authored by hayzam_gmail.com on Wed, Oct 14, 11:48 AM.

Details

Reviewers
markj
wanpengqian_gmail.com
Group Reviewers
bhyve
Summary

IOAPIC redirection-table writes rendezvous all active vCPUs to
rebuild their vlapic trigger-mode registers when fields other than
mask or polarity change. Retargeting an edge-triggered pin incurs
this cost even though it does not contribute to the TMR.

vioapic_update_tmr() rebuilds the TMR from level-triggered entries
only. Skip the rendezvous when the modified entry is edge-triggered
both before and after the write. Check the old and new trigger modes
so that edge-to-level and level-to-edge transitions still rebuild
the TMR. Retain the existing all-vCPU rebuild path, mask/polarity
exclusions and remote-IRR handling.

Reported by: Wanpeng Qian

Test Plan

Tested this patch with a Windows 11 guest using 24 vCPUs
and 24 GiB RAM on a dual-Xeon E5-2680 v3 host.

After ten minutes of settling following each VM start, three
untraced 60-second windows averaged:

  • Stock: 1063.1% aggregate bhyve CPU.
  • Patched: 62.1% aggregate bhyve CPU.

An observed 94.2% reduction in these captures.

Return to stock again showed substantially higher host CPU use
than the patched module.

Separate 10-second traces recorded approximately 342
vm_smp_rendezvous() calls/sec on stock and zero with this patch.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 78212
Build 75095: arc lint + arc unit