Revert "vnode read(2)/write(2): acquire rangelock regardless of do_vn_io_fault()"

[kib]

Summary

Revert "vnode read(2)/write(2): acquire rangelock regardless of do_vn_io_fault()"

This reverts commit 5b353925ff61b9ddb97bb453ba75278b578ed7d9.

The reason is the lesser scalability of the vnode' rangelock comparing
with the vnode lock.

Requested by:   mjg

vnode io: request range-locking when pgcache reads are enabled

PR:     272678

tmpfs: add a knob to enable pgcache read for mount

[mjg]

ufs is the only other pgcache vop user, does it work without rangelocks?

why not just revert the commit at hand and flip the pgcache sysctl to 0 until fixed, maybe add a comment above it describing the problem

[andrew_tao173.riddles.org.uk]

In D41334#941270, @mjg wrote:

ufs is the only other pgcache vop user, does it work without rangelocks?

ufs by default uses the vn_io_fault code path, which uses rangelocks, and does not (normally?) reach the pgcache path.

If you disable vn_io_fault but have pgcache reads enabled, then the bug manifests on ufs as well.

This was all in the original bug report ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=272678 )

[andrew_tao173.riddles.org.uk]

Do we have documentation of the scalability impact? I think that should be made clear.

[mjg]

In D41334#941284, @andrew_tao173.riddles.org.uk wrote:

If you disable vn_io_fault but have pgcache reads enabled, then the bug manifests on ufs as well.

ok, this is what i suspected

Do we have documentation of the scalability impact? I think that should be made clear.

numbers are provided in my response to the commit:

This time around I ran: ./readseek3_processes -t 10 (10 workers
reading from *disjoint* offsets from the same vnode. this in principle
can scale perfectly)

I observed a 90% drop in performance:
before: total:25723459 ops/s
after: total:2455794 ops/s

Going to an unpatched kernel and disabling pgcache reads instead:
disabled: total:6522480 ops/s

or about 2.6x of performance of the current kernel

[kib]

In D41334#941270, @mjg wrote:

ufs is the only other pgcache vop user, does it work without rangelocks?

why not just revert the commit at hand and flip the pgcache sysctl to 0 until fixed, maybe add a comment above it describing the problem

Because I do not want the pgcache read code to rot until I or somebody else work on rangelocks.

In D41334#941284, @andrew_tao173.riddles.org.uk wrote:

In D41334#941270, @mjg wrote:

ufs is the only other pgcache vop user, does it work without rangelocks?

ufs by default uses the vn_io_fault code path, which uses rangelocks, and does not (normally?) reach the pgcache path.

UFS does use pgcache reads.

[mjg]

i'm not going to argue over this

[andrew_tao173.riddles.org.uk]

In D41334#941330, @kib wrote:

ufs by default uses the vn_io_fault code path, which uses rangelocks, and does not (normally?) reach the pgcache path.

UFS does use pgcache reads.

Unless I'm misreading, when the vn_io_fault code path is enabled, then by the time we could end up doing a pgcache read, we already have a rangelock. So I misspoke about it not reaching that code, but it can't be reached unlocked on that path.