Change Details

Path lookup on illumos uses zfs_fastaccesschk_execute, which in particular tries to elide all the hard work in the common case of checking for VEXEC while it is definitely allowed. It still takes the acl lock, unnecessarily. The FreeBSD variant keeps calling what's effectively a slowpath lookup -- performing ZFS_ENTER, taking the acl lock and iterating acl list (if any). This can be trivially patch to also elide all the work + the lock. Apart from being slower single-threaded, this indunces significant lock contention. Sample wait times from doing an incremental -j 40 build of the Linux kernel: | before | after | 223760259 (sx:zp->z_acl_lock)|2472269 (rw:vm object) | 129273053 (sx:rrl->rr_lock)|2103785 (sleep mutex:vm page) | 2471834 (lockmgr:zfs)|1980574 (lockmgr:zfs) | 2108764 (rw:vm object)|1675503 (sx:rrl->rr_lock) | 1283443 (spin mutex:sleepq chain)|999978 (rw:pmap pv list) | 350059 (rw:pmap pv list)|616252 (sleep mutex:vnode interlock) That is, acl lock contention is eliminated and rrl lock (ZFS_ENTER) goes significantly down.

Path lookup on illumos uses zfs_fastaccesschk_execute, which in particular tries to elide all the hard work in the common case of checking for VEXEC while it is definitely allowed. It still takes the acl lock, unnecessarily. The FreeBSD variant keeps calling what's effectively a slowpath lookup -- performing ZFS_ENTER, taking the acl lock and iterating acl list (if any). This can be trivially patch to also elide all the work + the lock. Apart from being slower single-threaded, this indunces significant lock contention. Sample wait times from doing an incremental make -j 40 bzImage on a kernel running with other patches on top: | before | after | 223760259 (sx:zp->z_acl_lock)|2472269 (rw:vm object) | 129273053 (sx:rrl->rr_lock)|2103785 (sleep mutex:vm page) | 2471834 (lockmgr:zfs)|1980574 (lockmgr:zfs) | 2108764 (rw:vm object)|1675503 (sx:rrl->rr_lock) | 1283443 (spin mutex:sleepq chain)|999978 (rw:pmap pv list) | 350059 (rw:pmap pv list)|616252 (sleep mutex:vnode interlock) That is, acl lock contention is eliminated and rrl lock (ZFS_ENTER) goes significantly down.

Path lookup on illumos uses zfs_fastaccesschk_execute, which in particular tries to elide all the hard work in the common case of checking for VEXEC while it is definitely allowed. It still takes the acl lock, unnecessarily. The FreeBSD variant keeps calling what's effectively a slowpath lookup -- performing ZFS_ENTER, taking the acl lock and iterating acl list (if any). This can be trivially patch to also elide all the work + the lock. Apart from being slower single-threaded, this indunces significant lock contention. Sample wait times from doing an incremental make -j 40 build of the Linux kernel:zImage on a kernel running with other patches on top: | before | after | 223760259 (sx:zp->z_acl_lock)|2472269 (rw:vm object) | 129273053 (sx:rrl->rr_lock)|2103785 (sleep mutex:vm page) | 2471834 (lockmgr:zfs)|1980574 (lockmgr:zfs) | 2108764 (rw:vm object)|1675503 (sx:rrl->rr_lock) | 1283443 (spin mutex:sleepq chain)|999978 (rw:pmap pv list) | 350059 (rw:pmap pv list)|616252 (sleep mutex:vnode interlock) That is, acl lock contention is eliminated and rrl lock (ZFS_ENTER) goes significantly down.