I think I addressed all of feed backs so far.
I'm wondering if someone can take a look.

Address other review comments.

Fixed style, moved 0 assignment to *before and *after when SWWAPBLK_NONE,
and adjusted if/else statement for *after case.

Split backward and forward search into separate functions for ease of reading.

Thank you for quick response, Mark.

Updated function name in comment.

Changed its prefix to swdevt.

Check NULL pointers explicitly.

Restore KASSERT.

Update and merge D22437.

It looks we better drop swp_pager_meta_ctl() changes from this, D22324, and let D22409 take over the lookup implementation while dropping SWM_POP for simpler diff.
D22409 change is to optimize swap_pager_haspage() with additional searches after successful pindex lookup.

Dropped self-test.

D22280 has been committed.

Correct swblk index and exclude swp_pager_swblk_free_empty() changes.

I don't think this code duplication in swap_pager_copy() is a good idea.
I updated D22256 to avoid the double lookup cost from the first revision.
I'm still testing the 2nd version though.

Fixed spaces/style.

Created swp_pager_meta_find_least private API to find and delete in a single
operation. swap_pager_find_least calls it for look up.

Suspend for now.

I've been using the comparison based optimization for 2 weeks on current
following daily, buildworld/buildkernel daily and haven't seen an issue
with this approach.

I see. I just added zfs/zvol swap space temporary from command line within the last a couple of weeks for other reasons. I see few of too early OOM on few of machines although it is very rare and spontaneous such that it hasn't been easy to find a trigger.

swap_pager_full != 0 was indeed too aggressive. I'm wondering if we can feed in swap space usage such that we can delay OOM electing at low usage and push for OOM at high usage.

That's another idea.
I will check the code and test if that is also an option.

Thanks for follow up, markj.

Use "== SWPG_LOW" comparison for almost full warning.
Drop swp_gauge > SWPG_LOW comparison for high watermark check.

The only reason I created a patch was to proceed removal of sys/dir.h in the base system.
Given the upstream source hasn't been updated for more than a couple of decade.
It looks the code still assume 20 year old standard.

But sysctl describes totally different:

Skipping is about https://svnweb.freebsd.org/base/head/sys/vm/swap_pager.c?revision=352407&view=markup#l2313

destLen -> destlen.

Thanks.

Remove uninteded debug print.

Also print length with PRIu32.

Include zutil.h instead of zlib.h in order to pick up OS_CODE and DEF_MEM_LEVEL.

We have a last item - sys/kern/subr_compressor.c includes sys/zutil.h.
I attempted and saw crc32() API conflict between gsb_crc32.h and zutil.h.
I haven't got a chance this week, though.

Thank you for explanations.
I found and had looked at that git repo while reviewing this change.
However, I didn't have enough guts to look into each of repo history to figure out which one came from where.
That explains why all of ppp use Z_PACKET_FLUSH while really NONE uses Z_SYNC_FLUSH.

The change looks reasonable based on PPP protocol and comparison to existing zlib code.

I will take a closer look.
I've never used nor looked into details of this API.
Please allow me a bit extra time for review.

I haven't looked into FreeBSD extension of zlib yet to see if this change is safe.

I wasn't aware of swap partition being the the first in a slice was the case. All of my swaps have been either one of whole device, whole slice or non-first partition, or NFS files. No wonder I never encountered any issues over years even with this change on my local tree.

I found and have one more minor comment.

I agree with the changes.

Expose functions from compress.c with modified MY_ZCALLC handling.

Enhancce MY_ZCALLOC to expose functions in compress.c.
This also allows to use default zfree.

Given we added -DZ_SOLO to zlib, pass zfree function explicitly.
Expose compressBound() function even with -DZ_SOLO.
compress2() doesn't work with -DZ_SOLO and not worth exposing as is.
Alternative is improve MY_ZCALLOC conditions along with Z_SOLO for compress2 to work as well.

Given we added -DZ_SOLO to zlib, pass zfree function explicitly.

I tested reads and writes on gzipped ZFS between 12.0-RELEASE and 13-CURRENT with this changes.
The test result was good.

It looks ZFS changes, dropping zlib files in zfs, are melded into this one.
I recall needing to add -DGZIP and/or -DGUNZIP option for zfs compression=gzip to properly uncompress gzipped zfs devices with older version; there was gzip header regression...

D21099 removed gzip'ed a.out support. Rebased and adjusted.

Someone mentioned https://svnweb.freebsd.org/base?view=revision&revision=231885 was trigger in one of the mailing list. Anyway, if that's true, it's been broken for 7 years.