vm_meter: Fix laundry accounting

[markj]

Summary

Pages in PQ_UNSWAPPABLE should be considered part of the laundry.
Otherwise, on systems with no swap, the total amount of memory visible
to tools like top(1) decreases.

It doesn't seem very useful to have a dedicated counter for unswappable
pages, and updating applications accordingly would be painful, so just
lump them in with laundry for now.

[bnovkov]

LGTM, it would be good to briefly note this change in the 'Physical Memory Stats' section of the top(1) manpage as well.

[kib]

sys/vm/vm_meter.c

519–521

BTW this changed ABI, does 32bit top still work on 64bit host?

[markj]

sys/vm/vm_meter.c

519–521

No, this is a bug, I missed that the old sysctl returned an int and not a uint64_t.

We eventually need to widen these sysctls to u_long at least, but it should be possible to provide backwards compat by looking at the size of the user-supplied buffer.

[markj]

In D47216#1076712, @bnovkov wrote:

LGTM, it would be good to briefly note this change in the 'Physical Memory Stats' section of the top(1) manpage as well.

IMO the description there is already correct. Unswappable pages are still "queued for laundering", the fact that they live in a separate queue is just an implementation detail.

[markj]

Fix sysctl output type.

[olce]

I noticed this problem something like ~2 years ago but never took the time to investigate nor report it. Thanks for the fix.

[olce]

I fear that leaving unswappable pages in the "Laundry" count could confuse users. To me, "Laundry" has the connotation that you want to clean-up pages (by writing them to persistent media), so having a large amount of memory staying in "Laundry" forever (until completely freed) is disturbing.

I guess that @bnovkov's suggestion to amend top(1)'s page stems from a similar concern.

So, going forward, I tend to think that having a separate "Unswappable" category would be an improvement.

[markj]

In D47216#1079237, @olce wrote:

I fear that leaving unswappable pages in the "Laundry" count could confuse users. To me, "Laundry" has the connotation that you want to clean-up pages (by writing them to persistent media), so having a large amount of memory staying in "Laundry" forever (until completely freed) is disturbing.

That it's disturbing is not a bad thing. It means that the system might benefit from having a swap device.

I guess that @bnovkov's suggestion to amend top(1)'s page stems from a similar concern.

So, going forward, I tend to think that having a separate "Unswappable" category would be an improvement.

There are many utilities that want to gather info about memory usage from sysctls, so adding a new category is quite intrusive. The benefit should be worth the cost, but I don't yet see that it is.

[olce]

In D47216#1079251, @markj wrote:

That it's disturbing is not a bad thing.

Well, something disturbing takes energy, so should be worth the trouble.

It means that the system might benefit from having a swap device.

I agree. But then, maybe the name "Laundry" is not the proper one. "Dirty" seems better in this context (no action suggested now).

There are many utilities that want to gather info about memory usage from sysctls, so adding a new category is quite intrusive. The benefit should be worth the cost, but I don't yet see that it is.

In this precise case, the benefit is not to get questions/bug reports about "Laundry" never shrinking even under memory pressure. I already asked myself this very question some time ago, and I'll bet I'm not going to be the only one.

We should probably take advantage of the next category addition/change to revise the sysctls so that applications can actually enumerate all memory categories covering all allocated memory. E.g., having all these categories under a single sysctl() node would in large part alleviate the need of applications to be impacted by changes in category. That, and perhaps also outputting flags per category if relevant (like: immediately freeable). In other words, I'm suggesting to ponder at some point on a better interface lowering the cost of such changes.

[markj]

In D47216#1079256, @olce wrote:

In D47216#1079251, @markj wrote:

That it's disturbing is not a bad thing.

Well, something disturbing takes energy, so should be worth the trouble.

It means that the system might benefit from having a swap device.

I agree. But then, maybe the name "Laundry" is not the proper one. "Dirty" seems better in this context (no action suggested now).

There are many utilities that want to gather info about memory usage from sysctls, so adding a new category is quite intrusive. The benefit should be worth the cost, but I don't yet see that it is.

In this precise case, the benefit is not to get questions/bug reports about "Laundry" never shrinking even under memory pressure. I already asked myself this very question some time ago, and I'll bet I'm not going to be the only one.

This can be addressed by making the documentation (e.g., for top(1)) more useful.

We should probably take advantage of the next category addition/change to revise the sysctls so that applications can actually enumerate all memory categories covering all allocated memory. E.g., having all these categories under a single sysctl() node would in large part alleviate the need of applications to be impacted by changes in category. That, and perhaps also outputting flags per category if relevant (like: immediately freeable). In other words, I'm suggesting to ponder at some point on a better interface lowering the cost of such changes.

The reason I changed it this way is that prior to PQ_UNSWAPPABLE, pages in the laundry that couldn't be swapped out would just be migrated endlessly between the other queues. An alternate solution would have been to just keep them at the end of the laundry queue. The fact that they ended up in a separate queue is just an implementation detail, and I don't want to codify it by adding top-level counters for pages in that state.

[olce]

In D47216#1079267, @markj wrote:

This can be addressed by making the documentation (e.g., for top(1)) more useful.

Yes, that would be great. That said, having a non-ambiguous (or at least, less ambiguous) name could even have alleviated the need for that documentation ("Dirty" is not perfect either, maybe "Modified"; still not sure). I certainly understand that changing this has a cost that is not worth paying now, but we may want to keep that in mind for the future.

An alternate solution would have been to just keep them at the end of the laundry queue. The fact that they ended up in a separate queue is just an implementation detail, and I don't want to codify it by adding top-level counters for pages in that state.

But which benefits would that alternative have had? At a glance, I'm mostly seeing drawbacks to it.

Irrespective of the implementation (although the particular one chosen certainly may complicate some reporting), I think it would make sense to report on memory that cannot be freed by the VM itself (with possible cooperation of other subsystems indicating that some memory is just cache that can be thrown away).

As for the other categories reported by top(1), they are anyway quite tied to the implementation. As an example, I'm not sure that distinguishing wired memory with unswappable one is useful to users/administrators.

The absence of a definitive answer here is also what prompted my suggestion to have some flexibility in changing what the kernel reports and having an interface for applications to cope easily (ideally) with these changes. I remember having had to patch some KDE memory reporting application almost 20 years ago to cope with changes in the sysctl reporting memory, so this problem is not new. Nor do I think it should stay one going forward.