I recommend adding an int flags here (for example, compare Linux's renameat2()). Since no flags are defined yet, any non-zero flags argument should cause the call to return error [EINVAL].

.Fa path_from

This adds some additional complication (boolean) to avoid two refcount operations in the rename path. I can't judge whether it's worth it.

It still does two hash lookups. Consider returning the shmfd via a struct shmfd ** parameter that can be NULL.

This case is clearly possible since permissions to rename/unlink a shared memory object are determined by bits on the object itself and may differ between two objects.

Since shm_dict_lock is not unlocked in the meantime, the temporary removal should not be visible to other threads.

Good idea! I'm under crunch time, so I won't do this right now, but it is absolutely worth a TODO in the code to remember the idea. Marking 'done', having added the TODO. Please let me know if you think this is crucial enough to attack now.

It also avoids some cleanup like mtx_destroy that would invalidate the object's state. So: avoiding the shm_drop is best.

Sounds right. I'll clean up the comment.

Derp; let's just shm_hold before shm_remove to prevent any invalidation. Fixed.

Adding this later is an incompatible ABI and API change, so it should be done soon and definitely before MFC or heavier use.

The period (.) following shm_unlink should be replaced by a comma (,). Alternatively, just delete the period; a comma really isn't needed between the two clauses in this sentence.

s/exceeded/exceeds/

I actually asked for the flags mechanism only, without the actual flags like renameat2(). However, having it will be useful.

Please add something like

if ((flags & ~(SHM_RENAME_NOREPLACE | SHM_RENAME_EXCHANGE)) != 0)
    return (EINVAL);

so new flags can be safely added later.

This comment now applies to both paths.

Are you planning on adding the instrumentation to audit the syscall arguments in a future revision?

Looks like you found the OpenBSM PR :)
https://github.com/openbsm/openbsm/pull/54

Yes; my plan is:

1. Get some agreement on the syscall here on the FreeBSD side (done)
2. Get a commit into OpenBSM so I know exactly what the audit event num is going to be (waiting for it to merge).
3. Make a second review on the FreeBSD side for the audit changes.