Add missing vm_map_lock() in kern_mmap_racct_check

[arichardson]

Summary

GENERIC-KSAN reported a race between vm_map_insert and
kern_mmap_racct_check:

(gdb) l *0xffffffff816d883b
0xffffffff816d883b is in vm_map_insert (/local/scratch/alr48/cheri/freebsd/sys/vm/vm_map.c:1847).
1842		/*
1843		 * Insert the new entry into the list
1844		 */
1845		vm_map_entry_link(map, new_entry);
1846		if ((new_entry->eflags & MAP_ENTRY_GUARD) == 0)
1847			map->size += new_entry->end - new_entry->start;
1848
1849		/*
1850		 * Try to coalesce the new entry with both the previous and next
1851		 * entries in the list.  Previously, we only attempted to coalesce
(gdb) l *0xffffffff816ebad9
0xffffffff816ebad9 is in kern_mmap_racct_check (/local/scratch/alr48/cheri/freebsd/sys/vm/vm_mmap.c:1526).
1521	kern_mmap_racct_check(struct thread *td, vm_map_t map, vm_size_t size)
1522	{
1523		int error;
1524
1525		RACCT_PROC_LOCK(td->td_proc);
1526		if (map->size + size > lim_cur(td, RLIMIT_VMEM)) {
1527			RACCT_PROC_UNLOCK(td->td_proc);
1528			return (ENOMEM);
1529		}
1530		if (racct_set(td->td_proc, RACCT_VMEM, map->size + size)) {

I am not particularly familiar with the locking in the VM system but it
seems to me that map should be locked when accessing the size field.

Reported By: GENERIC-KCSAN

Test Plan

Race warning no longer printed.

[markj]

I suspect that this race is harmless since the size is effectively constant after initialization. The flags access is also racy but that seems harmless too. I'm not sure how to better express this to KCSAN - would making them atomic loads be sufficient?

sys/vm/vm_mmap.c

1526

This is the wrong order, the vm_map lock is a (sleepable) sx lock whereas the proc lock is a mutex. The sleepable lock must be acquired first. Also I think vm_map_lock_read() could be used here.

[kib]

map->size is not constant, but I think that it is innocent race.

If ever fixing it, just locking the map is not enough, since you might get positive result, then unlock, and then other thread maps something that invalidates the check.
kern_mmap_racct_check() would need to be called after the vm_map lock is taken for map modification.

But I suggest to not bothering.