vmm: Convert vmm_ops calls to IFUNC

[adam_fenn.io]

Summary

Convert vmm_ops calls to IFUNC.

Test Plan

Sanity-checked by using a 12.2-RELEASE guest and confirming vmm_ops
calls and callpaths via dtrace (example below).

Relevant portions of readelf -a /boot/kernel/vmm.ko:

Relocation section with addend (.rela.text):
r_offset     r_info       r_type              st_value         st_name + r_addend
0000000004c7 03b400000004 R_X86_64_PLT32      0000000000000320 vmmops_vmspace_alloc + fffffffffffffffc
0000000005b1 03aa00000004 R_X86_64_PLT32      00000000000000e0 vmmops_init + fffffffffffffffc
000000000752 03b300000004 R_X86_64_PLT32      00000000000003a0 vmmops_vlapic_init + fffffffffffffffc
000000000920 03b200000004 R_X86_64_PLT32      00000000000003e0 vmmops_vlapic_cleanup + fffffffffffffffc
000000000940 03a600000004 R_X86_64_PLT32      0000000000000160 vmmops_cleanup + fffffffffffffffc
000000000a98 03b500000004 R_X86_64_PLT32      0000000000000360 vmmops_vmspace_free + fffffffffffffffc
000000001abd 03a900000004 R_X86_64_PLT32      00000000000001a0 vmmops_getreg + fffffffffffffffc
000000001b0b 03b100000004 R_X86_64_PLT32      00000000000001e0 vmmops_setreg + fffffffffffffffc
000000001b60 03a800000004 R_X86_64_PLT32      0000000000000220 vmmops_getdesc + fffffffffffffffc
000000001b90 03b000000004 R_X86_64_PLT32      0000000000000260 vmmops_setdesc + fffffffffffffffc
000000002195 03ae00000004 R_X86_64_PLT32      0000000000000120 vmmops_run + fffffffffffffffc
0000000022f7 03b100000004 R_X86_64_PLT32      00000000000001e0 vmmops_setreg + fffffffffffffffc
000000002ffb 03a900000004 R_X86_64_PLT32      00000000000001a0 vmmops_getreg + fffffffffffffffc
000000003379 03a900000004 R_X86_64_PLT32      00000000000001a0 vmmops_getreg + fffffffffffffffc
0000000033a9 03b100000004 R_X86_64_PLT32      00000000000001e0 vmmops_setreg + fffffffffffffffc
000000003491 03b100000004 R_X86_64_PLT32      00000000000001e0 vmmops_setreg + fffffffffffffffc
00000000385d 03a700000004 R_X86_64_PLT32      00000000000002a0 vmmops_getcap + fffffffffffffffc
00000000388d 03af00000004 R_X86_64_PLT32      00000000000002e0 vmmops_setcap + fffffffffffffffc
000000004830 03ad0000000b R_X86_64_32S        00000000000000a0 vmmops_modresume + 0
00000000483b 03ac00000004 R_X86_64_PLT32      0000000000000000 vmmops_modinit + fffffffffffffffc
000000004885 03ab00000004 R_X86_64_PLT32      0000000000000060 vmmops_modcleanup + fffffffffffffffc

Symbol table (.symtab) contains 1378 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
    55: 0000000000000160    57 FUNC    LOCAL  DEFAULT    1 vmmops_cleanup_resolver
    56: 00000000000002a0    57 FUNC    LOCAL  DEFAULT    1 vmmops_getcap_resolver
    57: 0000000000000220    57 FUNC    LOCAL  DEFAULT    1 vmmops_getdesc_resolver
    58: 00000000000001a0    57 FUNC    LOCAL  DEFAULT    1 vmmops_getreg_resolver
    59: 00000000000000e0    57 FUNC    LOCAL  DEFAULT    1 vmmops_init_resolver
    60: 0000000000000060    57 FUNC    LOCAL  DEFAULT    1 vmmops_modcleanup_resolver
    61: 0000000000000000    57 FUNC    LOCAL  DEFAULT    1 vmmops_modinit_resolver
    62: 00000000000000a0    49 FUNC    LOCAL  DEFAULT    1 vmmops_modresume_resolver
    63: 0000000000000040    18 FUNC    LOCAL  DEFAULT    1 vmmops_panic
    64: 0000000000000120    57 FUNC    LOCAL  DEFAULT    1 vmmops_run_resolver
    65: 00000000000002e0    57 FUNC    LOCAL  DEFAULT    1 vmmops_setcap_resolver
    66: 0000000000000260    57 FUNC    LOCAL  DEFAULT    1 vmmops_setdesc_resolver
    67: 00000000000001e0    57 FUNC    LOCAL  DEFAULT    1 vmmops_setreg_resolver
    68: 00000000000003e0    57 FUNC    LOCAL  DEFAULT    1 vmmops_vlapic_cleanup_resolver
    69: 00000000000003a0    57 FUNC    LOCAL  DEFAULT    1 vmmops_vlapic_init_resolver
    70: 0000000000000320    57 FUNC    LOCAL  DEFAULT    1 vmmops_vmspace_alloc_resolver
    71: 0000000000000360    57 FUNC    LOCAL  DEFAULT    1 vmmops_vmspace_free_resolver
   934: 0000000000000160    57 IFUNC   LOCAL  DEFAULT    1 vmmops_cleanup
   935: 00000000000002a0    57 IFUNC   LOCAL  DEFAULT    1 vmmops_getcap
   936: 0000000000000220    57 IFUNC   LOCAL  DEFAULT    1 vmmops_getdesc
   937: 00000000000001a0    57 IFUNC   LOCAL  DEFAULT    1 vmmops_getreg
   938: 00000000000000e0    57 IFUNC   LOCAL  DEFAULT    1 vmmops_init
   939: 0000000000000060    57 IFUNC   LOCAL  DEFAULT    1 vmmops_modcleanup
   940: 0000000000000000    57 IFUNC   LOCAL  DEFAULT    1 vmmops_modinit
   941: 00000000000000a0    49 IFUNC   LOCAL  DEFAULT    1 vmmops_modresume
   942: 0000000000000120    57 IFUNC   LOCAL  DEFAULT    1 vmmops_run
   943: 00000000000002e0    57 IFUNC   LOCAL  DEFAULT    1 vmmops_setcap
   944: 0000000000000260    57 IFUNC   LOCAL  DEFAULT    1 vmmops_setdesc
   945: 00000000000001e0    57 IFUNC   LOCAL  DEFAULT    1 vmmops_setreg
   946: 00000000000003e0    57 IFUNC   LOCAL  DEFAULT    1 vmmops_vlapic_cleanup
   947: 00000000000003a0    57 IFUNC   LOCAL  DEFAULT    1 vmmops_vlapic_init
   948: 0000000000000320    57 IFUNC   LOCAL  DEFAULT    1 vmmops_vmspace_alloc
   949: 0000000000000360    57 IFUNC   LOCAL  DEFAULT    1 vmmops_vmspace_free

Example use of dtrace to confirm a 'vmm_ops' callpath:

$ sudo dtrace -n 'fbt:vmm:vmmops_run:entry {stack(); exit(0);}'
dtrace: description 'fbt:vmm:vmmops_run:entry ' matched 1 probe
dtrace: buffer size lowered to 512k
CPU     ID    FUNCTION:NAME
  1  67121    vmmops_run:entry
              vmm.ko`vm_run+0x269
              vmm.ko`vmmdev_ioctl+0x72f
              kernel`devfs_ioctl+0xcc
              kernel`vn_ioctl+0x131
              kernel`devfs_ioctl_f+0x1e
              kernel`kern_ioctl+0x289
              kernel`sys_ioctl+0x12a
              kernel`amd64_syscall+0x12e
              kernel`0xffffffff81005a4e

$ sudo dtrace -n 'fbt:vmm:vmx_enter_guest:entry {stack(); exit(0);}'
dtrace: description 'fbt:vmm:vmx_enter_guest:entry ' matched 1 probe
dtrace: buffer size lowered to 512k
CPU     ID    FUNCTION:NAME
  2  67356    vmx_enter_guest:entry
              vmm.ko`vmx_run+0xb8c
              vmm.ko`vm_run+0x269
              vmm.ko`vmmdev_ioctl+0x72f
              kernel`devfs_ioctl+0xcc
              kernel`vn_ioctl+0x131
              kernel`devfs_ioctl_f+0x1e
              kernel`kern_ioctl+0x289
              kernel`sys_ioctl+0x12a
              kernel`amd64_syscall+0x12e
              kernel`0xffffffff81005a4e

[grehan]

Tests out fine on a Ryzen system.

[markj]

sys/amd64/vmm/amd/svm.c

2675

Is there any reason this and vmm_ops_intel can't be marked const?

sys/amd64/vmm/intel/vmx.c

4185

It's bugged me in the past that the naming is inconsistent here: vmx_run() vs. svm_vmrun(). Since the op name is now just "run", I think vmx_run() is fine, but could you change the other VMX/SVM function names to match the corresponding op name? That makes it a bit easier to find function definitions with cscope or similar since the use of ifuncs breaks tag lookup.

[adam_fenn.io]

Incorporate review feedback from @markj.

[adam_fenn.io]

Thanks for the review @grehan and @markj! Diff updated as per @markj's feedback.