Page MenuHomeFreeBSD

Bump initial rtld memory pool size
ClosedPublic

Authored by trasz on Oct 30 2017, 9:06 AM.

Details

Summary

Bump initial rtld memory pool size from 32kB to 128kB. This removes
one or more calls to mmap(2) on binary startup.

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

This is silly, same as pre-allocating all address space with the mappings just in case the application would allocate them.

What is the binary which requires more than 32k memory for initialization ? Why did you expanded it only to 128k, and not to 100M ?
Dynamic memory should be allocated dynamically.

It is still being allocated dynamically, it's just the current preallocation size is not enough even for true(1). See the munmap/mmap pair below:

--- przed       2017-10-27 07:11:27.264576000 +0100
+++ po  2017-10-27 07:11:51.661299000 +0100
@@ -1,4 +1,4 @@
-mmap(0x0,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34366169088 (0x800622000)
+mmap(0x0,131072,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34366169088 (0x800622000)
 issetugid()                                     = 0 (0x0)
 openat(AT_FDCWD,"/etc/libmap.conf",O_RDONLY|O_CLOEXEC,00) = 3 (0x3)
 fstat(3,{ mode=-rw-r--r-- ,inode=1373288,size=102,blksize=32768 }) = 0 (0x0)
@@ -12,14 +12,12 @@ pread(3,"/lib:/usr/lib:/usr/lib/compat:/u"...,200,0x80
 close(3)                                        = 0 (0x0)
 openat(AT_FDCWD,"/lib/libc.so.7",O_RDONLY|O_CLOEXEC|O_VERIFY,00) = 3 (0x3)
 fstat(3,{ mode=-r--r--r-- ,inode=1617735,size=1992400,blksize=32768 }) = 0 (0x0)
-mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 34366201856 (0x80062a000)
+mmap(0x0,4096,PROT_READ,MAP_PRIVATE|MAP_PREFAULT_READ,3,0x0) = 34366300160 (0x800642000)
 mmap(0x0,6299648,PROT_NONE,MAP_GUARD,-1,0x0)    = 34368274432 (0x800824000)
 mmap(0x800824000,1871872,PROT_READ|PROT_EXEC,MAP_PRIVATE|MAP_FIXED|MAP_NOCORE|MAP_PREFAULT_READ,3,0x0) = 34368274432 (0x800824000)
 mmap(0x800bed000,61440,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_PREFAULT_READ,3,0x1c9000) = 34372243456 (0x800bed000)
 mmap(0x800bfc000,2269184,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_FIXED|MAP_ANON,-1,0x0) = 34372304896 (0x800bfc000)
 close(3)                                        = 0 (0x0)
-munmap(0x800629000,4096)                        = 0 (0x0)
-mmap(0x0,102400,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34366205952 (0x80062b000)
 sigprocmask(SIG_BLOCK,{ SIGHUP|SIGINT|SIGQUIT|SIGKILL|SIGPIPE|SIGALRM|SIGTERM|SIGURG|SIGSTOP|SIGTSTP|SIGCONT|SIGCHLD|SIGTTIN|SIGTTOU|SIGIO|SIGXCPU|SIGXFSZ|SIGVTALRM|SIGPROF|SIGWINCH|SIGINFO|SIGUSR1|SIGUSR2 },{ }) = 0 (0x0)
 sigprocmask(SIG_SETMASK,{ },0x0)                = 0 (0x0)
 readlink("/etc/malloc.conf",0x7fffffffd4a0,1024) ERR#2 'No such file or directory'
@@ -28,7 +26,7 @@ __sysctl(0x7fffffffd450,0x2,0x7fffffffd44c,0x7fffffffd
 mmap(0x0,2097152,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34374574080 (0x800e26000)
 cap_getmode({ 0 })                              = 0 (0x0)
 open("/dev/hpet0",O_RDONLY,00)                  = 3 (0x3)
-mmap(0x0,4096,PROT_READ,MAP_SHARED,3,0x0)       = 34366197760 (0x800629000)
+mmap(0x0,4096,PROT_READ,MAP_SHARED,3,0x0)       = 34366304256 (0x800643000)
 close(3)                                        = 0 (0x0)
 mmap(0x0,32768,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34366308352 (0x800644000)
 mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) = 34376671232 (0x801026000)

It is still being allocated dynamically, it's just the current preallocation size is not enough even for true(1). See the munmap/mmap pair below:

Does it still eat same amount of memory on 32bit arch, e.g. i386 ?

Also, I am curious what does require so much memory. I suspect that it is TLS, and if I am right, then you tweak the rtld sizing to some parameters external to it, which generally should not be done.
Try to profile memory usage.

No idea if it eats the same amount on 32 bit archs. Why is this important? Even if we wasted this 128kB of virtual address space, it doesn't hurt in any way, right?

As for "things external" - optimization always needs to happen against real world conditions. The 32kB constant no longer matches real world usage, even though it probably did 20 years ago, when it was initially committed. The whole point of preallocation is to avoid few unnecessary syscalls, and as it is now, it simply doesn't work - the additional syscalls are required even for the trivial /usr/bin/true binary.

I've tested it on i386, and yes, 32k is not enough there as well.

This revision was automatically updated to reflect the committed changes.