Handle O_DIRECTORY on fdescfs files

[kib]

Summary

If opened file with descriptor N references a directory, allow the open("/dev/fd/N", O_DIRECTORY) to succeed. To do it, postpone the check for v_type on fdescfs vnodes to fdesc_open() VOP, where check the type of real vnode instead of fdescfs node.

[markj]

sys/fs/fdescfs/fdesc_vnops.c

426

You could get rid of error1 and several lines of code if error is initialized to 0 and the routine has return (error == 0 ? ENODEV : error).

sys/kern/vfs_vnops.c

405

We have vn_irflag_read(). In particular, the vnode interlock is not held here.

413

Don't we need to handle fdescfs dirs at least here as well?

[kib]

I fixed things you noted, but I am not sure if this patch is needed after O_EMPTY_PATH is added. I prefer to not add quirks for fdescfs if O_EMPTY_PATH is enough, since it both more clean and faster.

Lets wait for Andrew opinion.

sys/kern/vfs_vnops.c

405

VIRF_FDESCFS is stable after the vnode is visible, so I do not really see a need to use atomic_load() there, but ok.

413

I am not sure, and in fact I tend to answer 'no' to your question.

open("/dev/fd/N") is nothing more than dup(2). From this PoV, not returning error when O_DIRECTORY is specified and fd N references directory improves the semantic.

But we de-facto ignore open mode and O_TRUNC when opening fdescfs, so the fast that EISDIR is not returned is IMO fine. [Until somebody asks for this, for real porting need, of course]

[kib]

Use vn_irflag() KPI.
Simplify error tracking logic in fdescfs_open()

[kib]

Rebase