Page MenuHomeFreeBSD

vfs: allow tail call optimisation in vops in the common case
ClosedPublic

Authored by mjg on Dec 9 2019, 9:14 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Dec 10, 5:57 PM
Unknown Object (File)
Oct 28 2024, 4:25 PM
Unknown Object (File)
Oct 3 2024, 9:37 PM
Unknown Object (File)
Oct 1 2024, 12:48 PM
Unknown Object (File)
Oct 1 2024, 3:48 AM
Unknown Object (File)
Sep 25 2024, 5:47 PM
Unknown Object (File)
Sep 22 2024, 9:09 PM
Unknown Object (File)
Sep 19 2024, 5:37 PM
Subscribers

Details

Summary

Most frequently used vops boil down to checking SDT probes, doing the call and checking again. There is no vop_post/pre in their case. The check after the call prevents tail call optimisation from taking place. Instead, check once upfront. Kernels with debug or vops with non-empty vop_post still don't short circuit.

Generated code:

int
VOP_LOCK1_APV(struct vop_vector *vop, struct vop_lock1_args *a)
{
        int rc;

        VNASSERT(a->a_gen.a_desc == &vop_lock1_desc, a->a_vp,
            ("Wrong a_desc in vop_lock1(%p, %p)", a->a_vp, a));
        VNASSERT(vop != NULL, a->a_vp, ("No vop_lock1(%p, %p)", a->a_vp, a));
        KTR_START4(KTR_VOP, "VOP", "VOP_LOCK1", (uintptr_t)a,
            "vp:0x%jX", (uintptr_t)a->a_vp, "flags:0x%jX", a->a_flags, "file:0x%jX", a->a_file, "line:0x%jX", a->a_line);
        vop_lock_pre(a);
        if (__predict_true(!SDT_PROBES_ENABLED() && vop->vop_lock1 != NULL)) {
                rc = vop->vop_lock1(a);
        } else {
                SDT_PROBE2(vfs, vop, vop_lock1, entry, a->a_vp, a);

                if (vop->vop_lock1 != NULL)
                        rc = vop->vop_lock1(a);
                else
                        rc = vop->vop_bypass(&a->a_gen);
                SDT_PROBE3(vfs, vop, vop_lock1, return, a->a_vp, a, rc);
        }
        if (rc == 0) {
        } else {
        }
        vop_lock_post(a, rc);
        KTR_STOP4(KTR_VOP, "VOP", "VOP_LOCK1", (uintptr_t)a,
            "vp:0x%jX", (uintptr_t)a->a_vp, "flags:0x%jX", a->a_flags, "file:0x%jX", a->a_file, "line:0x%jX", a->a_line);
        return (rc);
}

I verified that at least on amd64 tail call is indeed used.

Test Plan

will ask pho

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Skipped
Unit
Tests Skipped
Build Status
Buildable 28048

Event Timeline

It would be nice to flatten the vop_pre/vop_post stuff. We could define them in a file that is included in the interface generated source so they can be inlined and removed when necessary.

This would also look nicer when we don't have to check for NULL vops.

The vop_*_{pre,post} stuff is per operation and on kernels without debug they typically expand to nothing. I even noted vop->vop_lock1(a); gets tail-called which would not be possible if vop_lock_post had anything to do. The few cases which do anything in production kernels require fixing by hand (as in their internals are iffy).

  • drop spurious newline
This revision is now accepted and ready to land.Dec 13 2019, 4:52 PM