Is this something cheri could use too?

In CheriBSD we have a somewhat different mechanism where we embed the tags in struct swblk due to a hesitance to add more allocation to the swap path. That's not great for memory usage since it's quite a lot of space and it should be fairly compressible in most cases. I'd expect MTE to be before more compressible given it's often going to be runs of the same value. I think we could adapt this code to add CHERI tags, likely in parallel, though maybe with a shared scan loop.

In D55957#1314750, @brooks wrote:

In CheriBSD we have a somewhat different mechanism where we embed the tags in struct swblk due to a hesitance to add more allocation to the swap path. That's not great for memory usage since it's quite a lot of space and it should be fairly compressible in most cases. I'd expect MTE to be before more compressible given it's often going to be runs of the same value. I think we could adapt this code to add CHERI tags, likely in parallel, though maybe with a shared scan loop.

struct swblk is a fixed size, so compressing tags there won't affect memory usage. I put the tags in a separate allocation as I expect the common case is no MTE in the immediate future, so the extra space will be wasted. In CheriBSD it's different as it's more likely there will be tags in a given page.

I didn't want to compress the tags yet as it's more likely the code would be incorrect as it will likely need to allocate a number of smaller fixed-sized regions to store them in. I don't have any numbers on the size of these regions as we still need userspace support, e.g. ASAN with MTE.

i object against this code structure on principle.

• swp_pager_meta_archXXX should go into the arm64/arm64/somewhere.c. All of them should become arch hooks that do not add any #ifdefs to the sys/vm sources.
• If you absolutely need the extension of the struct swblk, it should be done in a way similar to how struct vm_page gets it MD members.

That said, I believe there was some idea circulating about adding hooks to the page lifetime. The motivation for it was e.g. support for demand-paging and page fault events from IOMMUs. Did you considered defining the KPI for hooks on page insertion and removal from the objects, that would allow you to do this stuff without such pollution of the VM code?

In D55957#1329686, @kib wrote:

i object against this code structure on principle.

• swp_pager_meta_archXXX should go into the arm64/arm64/somewhere.c. All of them should become arch hooks that do not add any #ifdefs to the sys/vm sources.
• If you absolutely need the extension of the struct swblk, it should be done in a way similar to how struct vm_page gets it MD members.

The space required to store tags is quite large, and we do not want to impose that on all platforms, so this would need to be done with some indirection.

That said, I believe there was some idea circulating about adding hooks to the page lifetime. The motivation for it was e.g. support for demand-paging and page fault events from IOMMUs. Did you considered defining the KPI for hooks on page insertion and removal from the objects, that would allow you to do this stuff without such pollution of the VM code?

Do you mean that the tag metadata should be associated with the VM object, rather than the swap block?

In D55957#1330132, @markj wrote:

In D55957#1329686, @kib wrote:

i object against this code structure on principle.

• swp_pager_meta_archXXX should go into the arm64/arm64/somewhere.c. All of them should become arch hooks that do not add any #ifdefs to the sys/vm sources.
• If you absolutely need the extension of the struct swblk, it should be done in a way similar to how struct vm_page gets it MD members.

The space required to store tags is quite large, and we do not want to impose that on all platforms, so this would need to be done with some indirection.

That said, I believe there was some idea circulating about adding hooks to the page lifetime. The motivation for it was e.g. support for demand-paging and page fault events from IOMMUs. Did you considered defining the KPI for hooks on page insertion and removal from the objects, that would allow you to do this stuff without such pollution of the VM code?

Do you mean that the tag metadata should be associated with the VM object, rather than the swap block?

Might be, I did not tried to design the hooks yet.
Lets try to discuss the draft at D58018.