Defer freeing until we drop devmtx

[trasz]

Summary

Defer freeing until we drop devmtx ("cdev").

Before r332974 the old code would sometimes cause a rare lock order reversal against pagequeue, which looked roughly like this:

witness_checkorder()
__mtx_lock-flags()
vm_page_alloc()
uma_small_alloc()
keg_alloc_slab()
keg_fetch-slab()
zone_fetch-slab()
zone_import()
zone_alloc_bucket()
uma_zalloc_arg()
bucket_alloc()
uma_zfree_arg()
free()
devfs_metoo()
devfs_populate_loop()
devfs_populate()
devfs_rioctl()
VOP_IOCTL_APV()
VOP_IOCTL()
vn_ioctl()
fo_ioctl()
kern_ioctl()
sys_ioctl()

Since r332974 the original problem no longer exists, but it still makes sense to move things out of the - often congested - lock.

[markj]

This change isn't needed anymore, with r332974 the page queue locks are leaf locks and I updated the witness table accordingly.

[kib]

Still devmtx should be leaf.

Also it is contended, so less work is done under it, the better.

sys/fs/devfs/devfs_devs.c

500

olddep = cdp->cdp_maxdirent > 0 ? cdp->cdp_dirents : NULL;

[markj]

sys/fs/devfs/devfs_devs.c

502

BTW, we now have malloc_size() for this, though I'm not sure we want to encourage its use for situations like this.

[trasz]

Prettify.

[trasz]

(Tinderboxed.)