Page MenuHomeFreeBSD

bitset: expand bit index type to `long`
ClosedPublic

Authored by scottph on Aug 26 2020, 6:42 AM.

Details

Summary

An upcoming patch to use the bitset macros for tracking vm page
dump information could conceivably need more than INT_MAX bits.
Expand the bit type to long so that the extra range is available
on 64-bit platforms where it would most likely be needed.

CPUSET_COUNT and DOMAINSET_COUNT are also modified to remain of
type int.

MFC after: 1 week
Sponsored by: Ampere Computing, Inc.

Diff Detail

Repository
rS FreeBSD src repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

If dump bitmap becomes so large, may be it is better to get rid of it at all ?
You can use vm_page aflags to track dumpable state.

This revision is now accepted and ready to land.Aug 27 2020, 2:20 AM
In D26190#582299, @kib wrote:

If dump bitmap becomes so large, may be it is better to get rid of it at all ?
You can use vm_page aflags to track dumpable state.

I like this idea. Even the plain flags field should be safe, and has more spare bits.

In D26190#582299, @kib wrote:

If dump bitmap becomes so large, may be it is better to get rid of it at all ?
You can use vm_page aflags to track dumpable state.

It seems that dump_avail includes some physical memory that doesn't have a vm_page, is that right? So we would need to first expand what vm_page covers, then we could use it to replace the bitmap? I can take a crack at it since i'm messing with the minidump format so much already.

In D26190#582299, @kib wrote:

If dump bitmap becomes so large, may be it is better to get rid of it at all ?
You can use vm_page aflags to track dumpable state.

It seems that dump_avail includes some physical memory that doesn't have a vm_page, is that right? So we would need to first expand what vm_page covers, then we could use it to replace the bitmap? I can take a crack at it since i'm messing with the minidump format so much already.

Can we reuse the algorithm from is_dumpable()? That is, if the PA corresponds to a vm_page, use a flag to decide whether the page is to be included in the minidump, otherwise include it so long as the PA is in dump_avail[].

Can we reuse the algorithm from is_dumpable()? That is, if the PA corresponds
to a vm_page, use a flag to decide whether the page is to be included in the
minidump, otherwise include it so long as the PA is in dump_avail[].

For the dump_avail case we would need to track that somewhere, maybe in a much
smaller bitmap that just covers memory that is in dump_avail and doesn't have a
vm_page? Thinking about the minidump binary format too, we could replace dumping
the bitmap with dumping vm_page_array and then libkvm can use vm_page to index
the dumped system memory area, except we would also need some way to know what
memory was dumped that's not in vm_page_array.

Can we reuse the algorithm from is_dumpable()? That is, if the PA corresponds
to a vm_page, use a flag to decide whether the page is to be included in the
minidump, otherwise include it so long as the PA is in dump_avail[].

For the dump_avail case we would need to track that somewhere, maybe in a much
smaller bitmap that just covers memory that is in dump_avail and doesn't have a
vm_page?

I see. Before going down that path I wonder what exactly is contained in that set. One example
is the msgbuf, allocated at the end of getmemsize() on amd64, but this is dumped in a special
segment in the minidump. I'm not sure what else is in there. It may be possible to just add
vm_phys segments to cover the rest, so that everything in dump_avail[] is backed by the page
array.

Thinking about the minidump binary format too, we could replace dumping
the bitmap with dumping vm_page_array and then libkvm can use vm_page to index
the dumped system memory area, except we would also need some way to know what
memory was dumped that's not in vm_page_array.

If possible I'd want to avoid changing the format: it's quite handy to be able to debug
vmcores from new kernels on older systems. If we can use the page array to capture the
entire set of dumped pages, then the bitmap could be synthesized at dump time using a
fixed-size buffer.

This revision was automatically updated to reflect the committed changes.