Page MenuHomeFreeBSD
Feed Advanced Search

Dec 25 2021

jah retitled D33611: unionfs: simplify writecount management from unionfs: leave writecount management to underlying filesystem to unionfs: simplify writecount management.
Dec 25 2021, 8:28 PM
jah updated the diff for D33611: unionfs: simplify writecount management.

Restore writecount tracking on unionfs vnode, in simplified form

Dec 25 2021, 8:26 PM

Dec 24 2021

jah added a comment to D33611: unionfs: simplify writecount management.
In D33611#759740, @kib wrote:
In D33611#759655, @jah wrote:

Ok, will do. On a related note, I did test nullfs enough to know that it doesn't have the same panic-on-process-termination issue as unionfs. So for nullfs the text ref seems to be managed on the correct vnode even though nullfs also does not implement VOP_[UN]SET_TEXT.
I assume this is because of null_bypass? If so would it be sufficient to simply delete null_add_writecount?

Yes, null_bypass results in text references go onto the lower vnode.

I mostly agree with the note that maintaining writecount on the upper vnode is effectively nop. So I am interested in see if nullfs can work without it, and indeed for nullfs the corresponding change would be to remove the custom bypass.

Dec 24 2021, 3:34 AM

Dec 23 2021

jah added a comment to D33611: unionfs: simplify writecount management.
In D33611#759601, @kib wrote:
In D33611#759589, @jah wrote:
In D33611#759420, @kib wrote:

Did you tried similar change for nullfs?

Basically text ref count needs to exist at least on the vnode which owns the actual v_object.

I haven't tried any nullfs changes, but the thought did occur to me since the unionfs code I changed was obviously copied from nullfs.
Would you like me to change nullfs as well (in a separate commit of course)?

Yes, and in fact I think that nullfs should be done first.

Dec 23 2021, 3:55 AM

Dec 22 2021

jah added a comment to D33611: unionfs: simplify writecount management.
In D33611#759420, @kib wrote:

Did you tried similar change for nullfs?

Basically text ref count needs to exist at least on the vnode which owns the actual v_object.

Dec 22 2021, 8:40 PM

Dec 21 2021

jah added a comment to D33611: unionfs: simplify writecount management.

@pho I'll post a separate fix for the panic exposed by unionfs7.sh, which just amounts to unionfs_open() not correctly accounting for the vnode being locked shared on the loader path.

Dec 21 2021, 11:59 PM
jah added a comment to D33611: unionfs: simplify writecount management.

From what I can tell, the main reason that managing v_writecount on the unionfs vnode might be useful would be in handling vflush(WRITECLOSE).
But WRITECLOSE only seems to be used by a few filesystems in support of read-only remounts, and unionfs isn't one of them. It's entirely possible I've missed something here, but if I can get away with it this seems like a useful simplification.

Dec 21 2021, 11:57 PM
jah requested review of D33611: unionfs: simplify writecount management.
Dec 21 2021, 11:52 PM

Dec 8 2021

jah committed rGcfc2cfeca195: unionfs: implement VOP_VPUT_PAIR (authored by jah).
unionfs: implement VOP_VPUT_PAIR
Dec 8 2021, 12:13 AM
jah committed rG6d8420d444b5: Remove unnecessary thread argument from unionfs_nodeget() and _noderem() (authored by jah).
Remove unnecessary thread argument from unionfs_nodeget() and _noderem()
Dec 8 2021, 12:13 AM
jah closed D33008: Clean up unionfs thread arguments and implement VOP_VPUT_PAIR.
Dec 8 2021, 12:13 AM

Dec 5 2021

jah added a comment to D33008: Clean up unionfs thread arguments and implement VOP_VPUT_PAIR.

ping

Dec 5 2021, 6:58 PM

Nov 24 2021

jah updated the diff for D33008: Clean up unionfs thread arguments and implement VOP_VPUT_PAIR.

Remove some write-only variables, add locking commentary

Nov 24 2021, 1:13 AM

Nov 18 2021

jah added inline comments to D33008: Clean up unionfs thread arguments and implement VOP_VPUT_PAIR.
Nov 18 2021, 2:02 AM

Nov 16 2021

jah added inline comments to D33008: Clean up unionfs thread arguments and implement VOP_VPUT_PAIR.
Nov 16 2021, 3:40 PM
jah requested review of D33008: Clean up unionfs thread arguments and implement VOP_VPUT_PAIR.
Nov 16 2021, 6:16 AM

Nov 15 2021

jah closed D32986: unionfs: fix potential deadlock in VOP_RMDIR.
Nov 15 2021, 4:01 AM
jah committed rG06f79675b7a0: unionfs: fix potential deadlock in VOP_RMDIR (authored by jah).
unionfs: fix potential deadlock in VOP_RMDIR
Nov 15 2021, 4:01 AM

Nov 14 2021

jah updated the diff for D32986: unionfs: fix potential deadlock in VOP_RMDIR.

style tweak

Nov 14 2021, 10:37 PM
jah updated the diff for D32986: unionfs: fix potential deadlock in VOP_RMDIR.

Check for doomed uvp on re-lock

Nov 14 2021, 7:42 PM
jah added inline comments to D32986: unionfs: fix potential deadlock in VOP_RMDIR.
Nov 14 2021, 4:12 PM
jah requested review of D32986: unionfs: fix potential deadlock in VOP_RMDIR.
Nov 14 2021, 7:33 AM

Nov 6 2021

jah committed rG5f73b3338ee1: unionfs: Improve vnode validation (authored by jah).
unionfs: Improve vnode validation
Nov 6 2021, 2:03 PM
jah committed rGfb273fe70f8b: unionfs: replace zero-length read check with KASSERT (authored by jah).
unionfs: replace zero-length read check with KASSERT
Nov 6 2021, 2:03 PM
jah committed rG3ecefc4a61cc: unionfs: assorted style fixes (authored by jah).
unionfs: assorted style fixes
Nov 6 2021, 2:03 PM
jah committed rG66191a76ace5: unionfs: Improve locking assertions (authored by jah).
unionfs: Improve locking assertions
Nov 6 2021, 2:03 PM
jah committed rG866dd6335a6c: unionfs: various locking fixes (authored by jah).
unionfs: various locking fixes
Nov 6 2021, 2:03 PM
jah closed D32629: unionfs: various locking fixes.
Nov 6 2021, 2:02 PM

Oct 31 2021

jah updated the diff for D32629: unionfs: various locking fixes.

Remove asserts from unionfs_get_cached_vnode(), add locking asserts to unionfs_ins_cached_vnode()

Oct 31 2021, 7:39 PM
jah added inline comments to D32629: unionfs: various locking fixes.
Oct 31 2021, 1:47 PM

Oct 30 2021

jah updated the diff for D32629: unionfs: various locking fixes.

Further style fixes, improve locking assertions and vnode checking

Oct 30 2021, 5:07 AM

Oct 26 2021

jah added inline comments to D32629: unionfs: various locking fixes.
Oct 26 2021, 3:40 PM

Oct 24 2021

jah added a comment to D32629: unionfs: various locking fixes.

phabricator doesn't make it very clear, but the locking and style changes are 2 different commits.

Oct 24 2021, 10:01 PM
jah requested review of D32629: unionfs: various locking fixes.
Oct 24 2021, 9:59 PM
jah committed rGfd8ad2128dcd: unionfs: implement vnode-based cache lookup (authored by jah).
unionfs: implement vnode-based cache lookup
Oct 24 2021, 4:59 PM
jah closed D32533: unionfs: implement vnode-based cache lookup.
Oct 24 2021, 4:59 PM

Oct 21 2021

jah added inline comments to D32533: unionfs: implement vnode-based cache lookup.
Oct 21 2021, 11:27 PM
jah added inline comments to D32533: unionfs: implement vnode-based cache lookup.
Oct 21 2021, 3:44 PM

Oct 20 2021

jah added inline comments to D32533: unionfs: implement vnode-based cache lookup.
Oct 20 2021, 7:16 PM

Oct 17 2021

jah requested review of D32533: unionfs: implement vnode-based cache lookup.
Oct 17 2021, 6:41 AM

Oct 14 2021

jah closed D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations.
Oct 14 2021, 2:20 AM
jah committed rG152c35ee4fda: unionfs: Ensure SAVENAME is set for unionfs vnode operations (authored by jah).
unionfs: Ensure SAVENAME is set for unionfs vnode operations
Oct 14 2021, 2:20 AM

Oct 13 2021

jah added inline comments to D32477: kenv: allow static hints to disable loader hints.
Oct 13 2021, 2:36 PM

Sep 27 2021

jah added a comment to D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations.
In D32148#725684, @pho wrote:

I got this panic while testing with D32148.95756.diff:

panic: vn_lock: error 16 incompatible with flags 0x82400
cpuid = 1
time = 1632729527
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0141a13740
vpanic() at vpanic+0x187/frame 0xfffffe0141a137a0
panic() at panic+0x43/frame 0xfffffe0141a13800
_vn_lock_fallback() at _vn_lock_fallback+0xd6/frame 0xfffffe0141a13860
_vn_lock() at _vn_lock+0x86/frame 0xfffffe0141a138c0
unionfs_nodeget() at unionfs_nodeget+0x719/frame 0xfffffe0141a13960
unionfs_lookup() at unionfs_lookup+0x63b/frame 0xfffffe0141a13ab0
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0x5a/frame 0xfffffe0141a13ad0
vfs_cache_lookup() at vfs_cache_lookup+0xa6/frame 0xfffffe0141a13b20
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x5a/frame 0xfffffe0141a13b40
lookup() at lookup+0x4d1/frame 0xfffffe0141a13be0
namei() at namei+0x379/frame 0xfffffe0141a13c90
kern_frmdirat() at kern_frmdirat+0x15e/frame 0xfffffe0141a13e00
amd64_syscall() at amd64_syscall+0x147/frame 0xfffffe0141a13f30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0141a13f30

https://people.freebsd.org/~pho/stress/log/log0182.txt

Sep 27 2021, 2:04 PM

Sep 26 2021

jah updated the summary of D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations.
Sep 26 2021, 9:16 PM
jah retitled D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations from unionfs: avoid trying to use cn_nameptr for lookup in unionfs_rmdir() to unionfs: Ensure SAVENAME is set for unionfs vnode operations.
Sep 26 2021, 9:16 PM
jah updated the diff for D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations.

Set SAVEBANE in unionfs_lookup() instead

Sep 26 2021, 9:15 PM
jah added a comment to D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations.
In D32148#725527, @mjg wrote:

You should be able to add the flag within the bowels of unionfs.

Sep 26 2021, 6:16 PM
jah added a comment to D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations.

I do have a question here:
Instead of taking this approach, would it be acceptable to simply change kern_frmdirat()/kern_funlinkat() to pass SAVENAME to namei()?
unionfs might of course be the only FS that would make use of the name buffer in those cases, but it seems like a straightforward change.

Sep 26 2021, 3:02 AM
jah requested review of D32148: unionfs: Ensure SAVENAME is set for unionfs vnode operations.
Sep 26 2021, 2:56 AM

Sep 24 2021

jah committed rGf9e28f900353: unionfs: lock newly-created vnodes before calling insmntque() (authored by jah).
unionfs: lock newly-created vnodes before calling insmntque()
Sep 24 2021, 2:19 AM
jah closed D31917: unionfs: lock newly-created vnodes before calling insmntque().
Sep 24 2021, 2:18 AM

Sep 19 2021

jah updated the diff for D31917: unionfs: lock newly-created vnodes before calling insmntque().

Remove unintentionally added line from a different commit

Sep 19 2021, 10:45 PM
jah updated the diff for D31917: unionfs: lock newly-created vnodes before calling insmntque().

Also free the vnode cache hashtable

Sep 19 2021, 10:36 PM

Sep 12 2021

jah requested review of D31917: unionfs: lock newly-created vnodes before calling insmntque().
Sep 12 2021, 5:53 AM

Sep 1 2021

jah committed rGabe95116ba10: unionfs: rework pathname handling (authored by jah).
unionfs: rework pathname handling
Sep 1 2021, 2:52 PM
jah committed rG312d49ef7afc: unionfs: style (authored by jah).
unionfs: style
Sep 1 2021, 2:52 PM
jah closed D31728: unionfs: rework pathname handling.
Sep 1 2021, 2:52 PM

Aug 30 2021

jah added inline comments to D31728: unionfs: rework pathname handling.
Aug 30 2021, 4:27 PM
jah added a comment to D31728: unionfs: rework pathname handling.

Based on my reading of namei(9) and the code, it seems like what I'm doing here (directly passing the pathname buffer) should be OK.
During testing I also instrumented the unionfs code to verify that the lower layers weren't altering the contents of cn_pnbuf. But being new to namei, I also wouldn't be surprised if this approach is problematic for reasons that aren't apparent to me. If so, I'll go back to allocating duplicate buffers for the unionfs lookup.

Aug 30 2021, 3:09 PM
jah requested review of D31728: unionfs: rework pathname handling.
Aug 30 2021, 3:07 PM

Aug 20 2021

jah committed rGa8c732f4e52e: VFS: add retry limit and delay for failed recursive unmounts (authored by jah).
VFS: add retry limit and delay for failed recursive unmounts
Aug 20 2021, 8:19 PM
jah committed rGe81e71b0e9cb: Use interruptible wait for blocking recursive unmounts (authored by jah).
Use interruptible wait for blocking recursive unmounts
Aug 20 2021, 8:19 PM
jah closed D31450: VFS: add retry limit and delay for failed recursive unmounts.
Aug 20 2021, 8:19 PM

Aug 15 2021

jah updated the diff for D31450: VFS: add retry limit and delay for failed recursive unmounts.

Add comment on (lack of) synchronization for counters, clarify sysctl description

Aug 15 2021, 1:08 AM

Aug 13 2021

jah added inline comments to D31450: VFS: add retry limit and delay for failed recursive unmounts.
Aug 13 2021, 8:55 PM

Aug 12 2021

jah added inline comments to D31450: VFS: add retry limit and delay for failed recursive unmounts.
Aug 12 2021, 2:58 PM

Aug 10 2021

jah updated the diff for D31450: VFS: add retry limit and delay for failed recursive unmounts.

Clean up error handling logic, properly release the mount on error in the blocking case

Aug 10 2021, 6:22 AM

Aug 8 2021

jah added inline comments to D31450: VFS: add retry limit and delay for failed recursive unmounts.
Aug 8 2021, 1:40 PM
jah updated the diff for D31450: VFS: add retry limit and delay for failed recursive unmounts.

Split PCATCH into a separate commit

Aug 8 2021, 1:35 PM
jah updated the diff for D31450: VFS: add retry limit and delay for failed recursive unmounts.

Add sysctl node for managing deferred unmount behavior

Aug 8 2021, 2:32 AM
jah added inline comments to D31450: VFS: add retry limit and delay for failed recursive unmounts.
Aug 8 2021, 2:31 AM

Aug 7 2021

jah requested review of D31450: VFS: add retry limit and delay for failed recursive unmounts.
Aug 7 2021, 6:34 AM

Jul 24 2021

jah committed rGc44685732899: Add stress2 test to exercise FFS forcible unmount with stacked nullfs (authored by jah).
Add stress2 test to exercise FFS forcible unmount with stacked nullfs
Jul 24 2021, 8:40 PM
jah committed rG2bc16e8aaf75: VFS: remove MNTK_MARKER (authored by jah).
VFS: remove MNTK_MARKER
Jul 24 2021, 8:40 PM
jah committed rG211ec9b7d6ec: FFS: remove ffs_fsfail_task (authored by jah).
FFS: remove ffs_fsfail_task
Jul 24 2021, 8:40 PM
jah committed rGc746ed724d01: Allow stacked filesystems to be recursively unmounted (authored by jah).
Allow stacked filesystems to be recursively unmounted
Jul 24 2021, 8:40 PM
jah closed D31016: Allow stacked filesystems to be recursively unmounted.
Jul 24 2021, 8:39 PM
jah added a comment to D31016: Allow stacked filesystems to be recursively unmounted.
In D31016#704531, @pho wrote:
In D31016#704507, @kib wrote:
In D31016#704447, @pho wrote:

All done. No problems seen.

20210722 19:00:47 all.sh done, elapsed 2 day(s), 04:40.10

And no nullfs issues observed? I am confused.

Yes, sorry. What I should have stated was that all of the stress2 tests were run, excluding known problem tests.
Specifically the force4.sh test, a nullfs / mdconfig -o force test, was excluded. This test failed before this patch and also fails with this patch.
I have lately tested a separate patch (by you) combined with D31016.92436.diff that fixes the issue seen with the force4.sh test. This combo is still being tested.

Jul 24 2021, 12:55 AM

Jul 19 2021

jah added inline comments to D31016: Allow stacked filesystems to be recursively unmounted.
Jul 19 2021, 7:24 PM
jah updated the diff for D31016: Allow stacked filesystems to be recursively unmounted.

Prefix new fields with mnt_, MNT_TASKQUEUE -> MNT_DEFERRED

Jul 19 2021, 7:11 PM
jah updated the diff for D31016: Allow stacked filesystems to be recursively unmounted.

Split into multiple commits

Jul 19 2021, 5:59 PM
jah updated the diff for D31016: Allow stacked filesystems to be recursively unmounted.

Rebase

Jul 19 2021, 5:52 PM
jah added a comment to D31016: Allow stacked filesystems to be recursively unmounted.

Suggestion on how to run forcible unmount test on UFS.

Jul 19 2021, 2:22 AM
jah updated the diff for D31016: Allow stacked filesystems to be recursively unmounted.

Fix whitespace, reap MNTK_MARKER, remove fsfail_task and add associated test

Jul 19 2021, 2:19 AM

Jul 15 2021

jah added inline comments to D31016: Allow stacked filesystems to be recursively unmounted.
Jul 15 2021, 11:49 PM
jah added a comment to D31016: Allow stacked filesystems to be recursively unmounted.

@jah - are you ready to have Peter Holm test these changes? If so, I will enlist his help.

Jul 15 2021, 11:43 PM
jah added a reviewer for D31016: Allow stacked filesystems to be recursively unmounted: pho.
Jul 15 2021, 11:38 PM

Jul 14 2021

jah updated the diff for D31016: Allow stacked filesystems to be recursively unmounted.

Simplify by requiring taskqueue unmounts to be recursive

Jul 14 2021, 1:08 PM
jah updated the diff for D31016: Allow stacked filesystems to be recursively unmounted.

--Make forced taskqueue unmount operations resilient to spurious failure

A concurrent unmount attempt from another thread (which may fail) or a
concurrent update mount may induce a spurious failure.  Allow the
tasqueue to requeue the unmount request in these cases.
Jul 14 2021, 5:45 AM

Jul 10 2021

jah added inline comments to D31016: Allow stacked filesystems to be recursively unmounted.
Jul 10 2021, 5:19 PM
jah added inline comments to D31016: Allow stacked filesystems to be recursively unmounted.
Jul 10 2021, 4:37 PM

Jul 4 2021

jah added reviewers for D31016: Allow stacked filesystems to be recursively unmounted: kib, markj, mjg, mckusick, chs.
Jul 4 2021, 2:21 AM
jah added a comment to D31016: Allow stacked filesystems to be recursively unmounted.

This is a first draft of the change; I've tested it enough to verify that it works, but I'd like feedback on the basic approach as well as some specific questions I have below.

Jul 4 2021, 2:20 AM
jah requested review of D31016: Allow stacked filesystems to be recursively unmounted.
Jul 4 2021, 2:11 AM

Jun 29 2021

jah closed D30748: unionfs: release parent vnodes in deferred context.
Jun 29 2021, 1:01 PM
jah committed rG372691a7ae18: unionfs: release parent vnodes in deferred context (authored by jah).
unionfs: release parent vnodes in deferred context
Jun 29 2021, 1:01 PM

Jun 18 2021

jah updated the diff for D30748: unionfs: release parent vnodes in deferred context.

Use STAILQ_FOREACH_SAFE to simplify the release loop

Jun 18 2021, 11:36 PM

Jun 16 2021

jah updated the diff for D30748: unionfs: release parent vnodes in deferred context.

Use only CTLFLAG_RD

Jun 16 2021, 1:54 AM
jah added a comment to D30748: unionfs: release parent vnodes in deferred context.
In D30748#691844, @jah wrote:
In D30748#691598, @jah wrote:
In D30748#691212, @mjg wrote:

If memory serves unionfs panics on mount if DEBUG_VFS_LOCKS is enabled, but the patch needs to be tested with it. I don't remember what stands in the way of fixing it, but bare minimum the triggering assert can be commented out for testing.

I didn't know we had DEBUG_VFS_LOCKS. My guess is that unionfs under stress will fail these checks in a variety of ways, but hopefully none of them will be caused by this change. This seems like another useful tool for evaluating unionfs, so I'll give it a shot as soon as I get some time.

Surprisingly, the panic on mount (new vnode not exclusively locked for insmntque()) is the only issue I've found in testing with DEBUG_VFS_LOCKS so far.

Jun 16 2021, 12:50 AM