Page MenuHomeFreeBSD

linuxkpi: honor the prot argument in vmap
AcceptedPublic

Authored by ashafer on Fri, Sep 4, 11:24 PM.
Tags
None
Referenced Files
F171213884: D59436.id185902.diff
Wed, Sep 9, 2:06 PM
F171203409: D59436.id185902.diff
Wed, Sep 9, 12:08 PM
Unknown Object (File)
Tue, Sep 8, 8:15 PM
Unknown Object (File)
Tue, Sep 8, 8:03 PM
Unknown Object (File)
Tue, Sep 8, 8:03 PM
Unknown Object (File)
Tue, Sep 8, 2:36 PM
Unknown Object (File)
Tue, Sep 8, 1:30 PM
Unknown Object (File)
Tue, Sep 8, 10:57 AM

Details

Reviewers
bz
Group Reviewers
linuxkpi
Summary

This adds missing support to the vmap function to respect what the user
requested via the prot argument. This matches what we do for
linuxkpi_vmap_pfn. We create a memattr from the passed in prot and apply
it to all mapped pages. This also updates vunmap to clear any
non-default memattrs that are set before we return the page to the
vm_page allocator.

We hit this in the wild with drm-kmod, specifically when the i915 driver
tries to initialize firmware. The firmware is supposed to be mapped as
write combined but doesn't due to us not honoring prot, which goes on to
fail due to the memory not being visible to the GPU properly.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 76556
Build 73439: arc lint + arc unit

Event Timeline

I found that this was needed for the GT1 hardware unit to come up with GuC firmware enabled on meteorlake

I found that this was needed for the GT1 hardware unit to come up with GuC firmware enabled on meteorlake

Can you tell us in which version and possibly where the code (argument) comes from?
I am trying to understand how this is supposed to work but cannot find many PAGE_* definitions (well PAGE_KERNEL[_IO]) in LinuxKPI.

sys/compat/linuxkpi/common/src/linux_page.c
426

We should probably fix the prot argument type as well into something resembling "unsigned long" or uint64_t, like pgprot_t.
I am not sure if it matters for LinuxKPI given vm_memattr_t is a char, but for "correctness".
Probably a separate commit?

It gets called from drm-kmod, in the case of i915 this seems to usually originate from i915_gem_object_map_page()->vmap() when the caller uses I915_MAP_WC. There's quite a few paths that make it to using this function, some of them will set I915_MAP_(WB|WC) such as from intel_guc_init->intel_uc_fw_init()

It gets called from drm-kmod, in the case of i915 this seems to usually originate from i915_gem_object_map_page()->vmap() when the caller uses I915_MAP_WC. There's quite a few paths that make it to using this function, some of them will set I915_MAP_(WB|WC) such as from intel_guc_init->intel_uc_fw_init()

Let's stay with this example: I915_MAP_WC is an internal value from an enum; how does that then get mapped to PAGE_KERNEL* values (which we do not define). I wonder does it get mapped straight to FreeBSD internal VM_ values?

306     case I915_MAP_WC:
307         pgprot = pgprot_writecombine(PAGE_KERNEL_IO);
308         break;

Here's the bit from i915_gem_object_map_page. We calculate page protections from PAGE_KERNEL_IO and linuxkpi's pgprot_writecombine really just turns that into VM_*. Then i915 passes pgprot into vmap. We do also define PAGE_KERNEL in vmalloc.h and PAGE_KERNEL_IO in page.h

306     case I915_MAP_WC:
307         pgprot = pgprot_writecombine(PAGE_KERNEL_IO);
308         break;

Here's the bit from i915_gem_object_map_page. We calculate page protections from PAGE_KERNEL_IO and linuxkpi's pgprot_writecombine really just turns that into VM_*. Then i915 passes pgprot into vmap. We do also define PAGE_KERNEL in vmalloc.h and PAGE_KERNEL_IO in page.h

Yes, that's what I meant in the first comment; We have PAGE_KERNEL[_IO] but none of all the others.

Neither is used beyond the define.

% grep -r PAGE_KERNEL sys/compat/linuxkpi/
sys/compat/linuxkpi/common/include/linux/vmalloc.h:#define      PAGE_KERNEL     0x0000
sys/compat/linuxkpi/common/include/linux/page.h:#define PAGE_KERNEL_IO  0x0000
%

And pgprot2cachemode() then maps these both to VM_MEMATTR_DEFAULT.

But in your sample I see that pgprot_writecombine(); and that and others map things to native VM_MEMATTR_ values in LinuxKPI directly.

Thanks a lot for the explanations and code samples! Really helped to understand how this is supposed to come together.

bz added inline comments.
sys/compat/linuxkpi/common/src/linux_page.c
531

Are we always doing full size pages here?
Or may we be possible missing a last partial page due to round-down?
Hmm given the previous pmap_qremove() this is hopefully fine.

This revision is now accepted and ready to land.Mon, Sep 7, 7:39 PM

I take it, this code fixes the problem reported here: https://github.com/freebsd/drm-kmod/pull/496 ? If it does, thank you very much for writing it! I'll test this code later.

Not 100% confident, but that drm-kmod issue seems similar enough I think there's a decent chance this solves it. More testing would definitely be helpful

I tested your patch with vanilla drm-kmod today, and it didn't solve the problem that my github PR's code had solved for me. But thanks anyway for writing it.