Page MenuHomeFreeBSD

vm: Fix anonymous memory clustering under ASLR
ClosedPublic

Authored by alc on Jun 23 2023, 11:36 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sat, May 4, 12:26 AM
Unknown Object (File)
Apr 8 2024, 12:15 AM
Unknown Object (File)
Apr 7 2024, 11:59 PM
Unknown Object (File)
Feb 27 2024, 10:18 AM
Unknown Object (File)
Jan 29 2024, 6:36 PM
Unknown Object (File)
Dec 20 2023, 6:35 AM
Unknown Object (File)
Dec 12 2023, 6:59 PM
Unknown Object (File)
Dec 10 2023, 8:09 PM
Subscribers

Details

Summary

By default, our ASLR implementation is supposed to cluster anonymous memory allocations, unless the application's mmap(..., MAP_ANON, ...) call includes a non-zero address hint. Unfortunately, clustering never occurred because kern_mmap() always replaced the given address hint when it was zero. So, the ASLR implementation believed that a non-zero hint had been provided and randomized the mapping's location in the address space. Essentially, I'm pushing down the point at which we convert a hint of zero to a minimum address into vm_map_find_min().

See D39845 for the discussion leading to this revision.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

alc requested review of this revision.Jun 23 2023, 11:36 PM
alc created this revision.
This revision is now accepted and ready to land.Jun 24 2023, 7:38 AM

This change reduces the number of L1 DTLB misses on a Ryzen 5900X during a "make buildkernel" by 14.3%, in large part, because it makes their TLB coalescing feature more effective.

before_aslr_fix

# p/ls_l1_d_tlb_miss.all p/ls_l1_d_tlb_miss.tlb_reload_coalesced_page_miss p/ls_l1_d_tlb_miss.tlb_reload_coalesced_page_hit 
               400896822                                           1387284                                        113431540 
              2031783142                                           6707304                                        633933941 
              3669389530                                          12793336                                       1169481135 
              5327979928                                          18803657                                       1712516190 
              6849225965                                          24171045                                       2189539304 
              8383011553                                          29957442                                       2710545649 
             10018998951                                          35693930                                       3220180354 
             11758817968                                          41969381                                       3798131739 
             13368193796                                          47930175                                       4338865120 
             14927320935                                          53394009                                       4844402586 
             16586636700                                          59238298                                       5424373920 
             18216216253                                          64985077                                       5961366204 
             19833688388                                          70527608                                       6470023696 
             21551085782                                          75958293                                       6979398002 
             23248324097                                          81225729                                       7535539140 
             24976781314                                          85977703                                       8134809166 
             26742046851                                          90704212                                       8722892878 
             28532784215                                          96080508                                       9357090749 
             30317659219                                         101114147                                       9946477234 
             32117770509                                         106555268                                      10543633280 
             33882496774                                         111460955                                      11123162866 
             34453804009                                         113657730                                      11326797278 
             34665157305                                         115116968                                      11425886081 
             34665233869                                         115117920                                      11425932136

after_aslr_fix

# p/ls_l1_d_tlb_miss.all p/ls_l1_d_tlb_miss.tlb_reload_coalesced_page_miss p/ls_l1_d_tlb_miss.tlb_reload_coalesced_page_hit 
                40711929                                            419503                                          7604079 
              1328062736                                           5961272                                        705058719 
              2759717647                                          11550686                                       1505805883 
              4191612880                                          17923180                                       2306316404 
              5602396993                                          24567696                                       3062793029 
              7073041137                                          30800597                                       3892885050 
              8029198693                                          35002429                                       4397677298 
              9448043987                                          40825452                                       5181840777 
             10955283607                                          46988747                                       6049910797 
             12359634178                                          52908380                                       6839913456 
             13765624449                                          58689118                                       7628439186 
             15164756156                                          64437826                                       8399635667 
             16445318496                                          69813455                                       9110015964 
             17922395478                                          75706779                                       9914067264 
             19374919576                                          80951293                                      10642699561 
             20903851357                                          85677443                                      11492207936 
             22417888546                                          89656957                                      12370875894 
             23972558664                                          93992378                                      13267666673 
             25483838991                                          98561192                                      14141177026 
             27092829876                                         103536512                                      15065794170 
             28617221147                                         108044990                                      15922922816 
             29516317827                                         110690692                                      16437928128 
             29533061076                                         112201977                                      16445199131 
             29703155574                                         113527884                                      16586779115
This revision was automatically updated to reflect the committed changes.