kqueue: Allocate marker knotes on the stack

[markj]

Summary

The scan marker was originally stack-allocated. In commit
1c0f9af5b5224, it became heap-allocated since the marker is visible to
other threads and a scanning thread's stack may be swapped out. Now
that kernel stacks can no longer be swapped out, we avoid these heap
allocations.

[kib]

I do not object strong, but knote is 160 bytes.

[markj]

In D58402#1339721, @kib wrote:

I do not object strong, but knote is 160 bytes.

This is less than 1% of the kernel stack size on amd64, and the markers will always be close to the base of the stack. I do not think it is likely to be a problem.

[kib]

I am thinking about reducing the amd64 default kstack size to 4->3, or even 2.

Since we got both user fpu save area and pcb out of kstack, it essentially freed one page.

[markj]

In D58402#1339756, @kib wrote:

I am thinking about reducing the amd64 default kstack size to 4->3, or even 2.

Since we got both user fpu save area and pcb out of kstack, it essentially freed one page.

I have not tried to profile this in a long time, it would be good to see how much stack ZFS uses. The following one-liner samples kernel stack usage and generates a histogram:

# sudo dtrace -n 'profile-997hz /arg0 != 0 && (curthread->td_flags & 0x20) == 0/{@ = quantize(((uintptr_t)curthread->td_kstack + 4096 * curthread->td_kstack_pages) - regs[R_RSP]);}'
...
load: 37.36  cmd: dtrace 31931 [uwait] 174.35r 0.07u 0.11s 0% 34272k
           value  ------------- Distribution ------------- count    
              64 |                                         0        
             128 |                                         11333    
             256 |@                                        66519    
             512 |@                                        87238    
            1024 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@   4869084  
            2048 |@                                        65635    
            4096 |                                         504      
            8192 |                                         0

This is mostly just running local builds on a ZFS system. It would be nice to test this with classic kernel stack hogs, NFS, IPSec, more complex ZFS workloads.

(We could also stand to reduce the number of kernel threads entirely; the taskqueue interface makes it too easy to start new threads unnecessarily. @jhb and I talked recently about introducing a central thread pool that can be used to service all work queues.)

[kib]

My reading of the graph you posted is that we never exceed 2 pages. So reducing the stack to 3 pages should be fine for this workload, while reducing to two is probably not going to work. Am I correct?

I think it is too much for one person to try different ZFS loads. I would either post a request for help to the list asking people to try to run with reduced KSTACK_PAGES. Or just commit the reduction and watch for the result. I think it is acceptable for HEAD.

[markj]

In D58402#1340318, @kib wrote:

My reading of the graph you posted is that we never exceed 2 pages. So reducing the stack to 3 pages should be fine for this workload, while reducing to two is probably not going to work. Am I correct?

Right, the usage is typically below 2 pages. This is sampling-based though, it's possible it missed some rare usage of >2 pages.

I think it is too much for one person to try different ZFS loads. I would either post a request for help to the list asking people to try to run with reduced KSTACK_PAGES. Or just commit the reduction and watch for the result. I think it is acceptable for HEAD.

I think it would not be nice to commit usage a change outright. Certainly others should help test. In a GENERIC kernel I cannot even load zfs.ko with kstack_pages=2:

Fatal double fault                                            
rip 0xffffffff8520b217 rsp 0xfffffe00d8081c80 rbp 0xfffffe00d80821f0
rax 0x4 rdx 0x2000 rbx 0xfffff80026a93000                           
rcx 0x1000 rsi 0xffffffff85598900 rdi 0xfffffe014fb6e000
r8 0 r9 0x10 r10 0                                            
r11 0xffffffff85598900 r12 0x10 r13 0xfffff80026a93000
r14 0x8 r15 0x2000 rflags 0x10206                                    
cs 0x20 ss 0x28 ds 0x3b es 0x3b fs 0x13 gs 0x1b
fsbase 0x3f54e1822340 gsbase 0xffffffff82813000 kgsbase 0     
cpuid = 3; apic id = 03                
panic: double fault                                                          
cpuid = 3                                                                                                                                                      
time = 1784847793                                                      
KDB: stack backtrace:                                         
db_trace_self_wrapper() at db_trace_self_wrapper+0x36/frame 0xfffffe001c495cc0
vpanic() at vpanic+0x149/frame 0xfffffe001c495df0             
panic() at panic+0x43/frame 0xfffffe001c495e50              
dblfault_handler() at dblfault_handler+0x1f7/frame 0xfffffe001c495f20                                                                                          
Xdblfault() at Xdblfault+0xd7/frame 0xfffffe001c495f20        
--- trap 0x17, rip = 0xffffffff8520b217, rsp = 0xfffffe00d8081c80, rbp = 0xfffffe00d80821f0 ---
blake3_compress_subtree_wide() at blake3_compress_subtree_wide+0x37/frame 0xfffffe00d80821f0
blake3_compress_subtree_wide() at blake3_compress_subtree_wide+0xd9/frame 0xfffffe00d8082780
blake3_compress_subtree_wide() at blake3_compress_subtree_wide+0xd9/frame 0xfffffe00d8082d10                                                                   
blake3_compress_subtree_wide() at blake3_compress_subtree_wide+0xd9/frame 0xfffffe00d80832a0                                                                                                                                                                                                                                  
Blake3_Update() at Blake3_Update+0xbdc/frame 0xfffffe00d8083760
blake3_incremental() at blake3_incremental+0x15/frame 0xfffffe00d8083770
abd_iterate_func() at abd_iterate_func+0x11b/frame 0xfffffe00d8083810                                                                                          
abd_checksum_blake3_native() at abd_checksum_blake3_native+0x69/frame 0xfffffe00d8083850
chksum_run() at chksum_run+0xd1/frame 0xfffffe00d8083910                                                                                                       
chksum_benchit() at chksum_benchit+0x161/frame 0xfffffe00d8083940                                                                                              
chksum_benchmark() at chksum_benchmark+0x348/frame 0xfffffe00d80839a0                                                                                          
chksum_init() at chksum_init+0x12/frame 0xfffffe00d80839c0                                                                                                     
spa_init() at spa_init+0x10b/frame 0xfffffe00d80839e0                                                                                                          
zfs_kmod_init() at zfs_kmod_init+0x1f/frame 0xfffffe00d8083a00                                                                                                                                                                                                                                                                
zfs_modevent() at zfs_modevent+0x34/frame 0xfffffe00d8083a20                   
module_register_init() at module_register_init+0xb0/frame 0xfffffe00d8083a50
linker_load_module() at linker_load_module+0xcf7/frame 0xfffffe00d8083d60                                                                                      
kern_kldload() at kern_kldload+0x172/frame 0xfffffe00d8083db0  
sys_kldload() at sys_kldload+0x67/frame 0xfffffe00d8083df0
amd64_syscall() at amd64_syscall+0x17c/frame 0xfffffe00d8083f30                                                                                                
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00d8083f30                                                                                                                                                                                                                                                    
--- syscall (304, FreeBSD ELF64, kldload), rip = 0x1811acdb836a, rsp = 0x1811aa315658, rbp = 0x1811aa315bd0 ---

I tried with 3 pages and got a huge number of test failures, and the VM became unresponsive. I don't know why. Maybe GET_STACK_USAGE() is returning false in some paths where it did not before, and it's causing problems. I will try to debug further.

[markj]

In D58402#1340410, @markj wrote:

I tried with 3 pages and got a huge number of test failures, and the VM became unresponsive. I don't know why. Maybe GET_STACK_USAGE() is returning false in some paths where it did not before, and it's causing problems. I will try to debug further.

I still don't know what's going on, but if I boot a VM with kstack_pages=3 and run tests in /usr/tests/libexec/tftpd, they all fail.

[kib]

I tried on my crash box which is netbooted.

With kstack_pages == 2, it paniced right in the first (?) access to the nfs root, due to VFS->nfs->network->recursion.

With kstack_pages == 3, the machine booted, and I see

Results file id is usr_tests_libexec_tftpd.20260724-024024-115803
Results saved to /root/.kyua/store/results.usr_tests_libexec_tftpd.20260724-024024-115803.db

66/66 passed (0 broken, 0 failed, 0 skipped)
root@r-freeb44:~ # sysctl kern.kstack_pages
kern.kstack_pages: 3

[markj]

In D58402#1340417, @markj wrote:

In D58402#1340410, @markj wrote:

I tried with 3 pages and got a huge number of test failures, and the VM became unresponsive. I don't know why. Maybe GET_STACK_USAGE() is returning false in some paths where it did not before, and it's causing problems. I will try to debug further.

I still don't know what's going on, but if I boot a VM with kstack_pages=3 and run tests in /usr/tests/libexec/tftpd, they all fail.

Sorry, this was user error (I accidentally overrode some other tunables when I set kern.kstack_pages, specifically net.inet.ip.fw.default_to_accept=1).

I tried running the test suite again with kstack_pages=3 and it seems ok. I can try the openzfs test suite too, but it takes a long time to run.

[kib]

In D58402#1340563, @markj wrote:

In D58402#1340417, @markj wrote:

In D58402#1340410, @markj wrote:

I tried with 3 pages and got a huge number of test failures, and the VM became unresponsive. I don't know why. Maybe GET_STACK_USAGE() is returning false in some paths where it did not before, and it's causing problems. I will try to debug further.

I still don't know what's going on, but if I boot a VM with kstack_pages=3 and run tests in /usr/tests/libexec/tftpd, they all fail.

Sorry, this was user error (I accidentally overrode some other tunables when I set kern.kstack_pages, specifically net.inet.ip.fw.default_to_accept=1).

I tried running the test suite again with kstack_pages=3 and it seems ok. I can try the openzfs test suite too, but it takes a long time to run.

If you run that tests, then perhaps you should go ahead and commit the change to kstack_pages on amd64.