I do not understand this change. Why do we need it?

Esp. what if p_offset is not zero, why do we care if it is far enough to cover a superpage?

Why is this a reasonable check? It MAXPAGESIZES > VM_NRESERVLEVEL, the formula selects the max pagesizes index for which there are reservations. But what if e.g MAXPAGESIZES == VM_NRESERVLEVEL on some arch, why fall back to smallest page alignment?

Again, I do not understand the condition. shm largepages should not depend on the reservation levels the kernel provides. It is purely about pmap knowing how to construct PTE for given page size.

For instance, on arm64 which can stop page table walk at any level, there is nothing except a need for some bit of pmap code that prevents us to implement largepages utilizing PTEs at higher levels of page tables. Same for amd64 if it ever start providing for 512G PTEs (at level 4).

Consider, for example, libc.so, which has a file size less than 2MB. However, the guard size exceeds 2MB because of libc.so's bss section. rtld_round_page(segs[0]->p_filesz) >= pagesizes[1] (64KB) will be true. If that were the only requirement (aside from npagesizes > 1), then we would pointlessly 2MB align the guard even though the file size is too small to later allocate a 2MB reservation for caching the file contents. (In fact, we saw this happening before the new requirement was introduced.) In contrast, libcrypto.so meets both the old and new requirements, and as shown in the summary, we allocate a 2MB reservation and map parts of the R/O data and code with 64KB pages.

P.S. As part of the next big patch that introduces real 2-level reservations, we will add another condition that requests pagesizes[1] (64KB) alignment for files that are less than pagesizes[2] but greater than pagesizes[1]. So, libc.so has 64KB reservations allocated and 64KB page mappings.

MAXPAGESIZES always includes pagesizes[0], but VM_NRESERVLEVEL does not. So, they should never be equal in any sane configuration. Specifically, that would imply that we have a reservation size that is larger than the largest page size. That said, we could eliminate MAXPAGESIZES > VM_NRESERVLEVEL, and instead of pagesizes[VM_NRESERVLEVEL] write pagesizes[MIN(MAXPAGESIZES - 1, VM_NRESERVLEVEL]).

At first, I found this code puzzling too. In the end, we simply sought to maintain the current behavior: MAP_ALIGNED_SUPER relies on pmap_align_superpage(), which only implements alignment for page sizes that match reservation sizes, which previously was only 2MB on amd64 and arm64 (or 32MB on arm64 if the base page size is 16KB). In other words, on neither amd64 nor arm64, does pmap_align_superpage() implement alignment for 1GB pages, which are supported by shm_create_largepage() on both architectures. I inferred that this is why the current code rejects the use of MAP_ALIGNED_SUPER if shm_lp_psind is not 1, i.e., what used to be 2MB on arm64. With this change, we still reject the use of MAP_ALIGNED_SUPER in conjunction with 1GB pages, but not 64KB or 2MB.

P.S. If no explicit alignment directive, e.g., MAP_ALIGNED_SUPER, is passed to shm_mmap_large(), then it automatically requests alignment based on shm_lp_psind.

Let me add that I expect p_offset for nsegs to be non-zero.

I still do not understand it. First, you ignore the bss-like segments, which might be larger than p_filesz. Second, I suspect that what you want there to compare against the pagesizes[XXX} is total mapping size, but then it is just mapsize?

Yes please. For me, the last formula looks reasonable.

I see, thank you for the explanation.

In fact, excluding bss is exactly what I am aiming to do. As I described above, using libc.so as an example, the current code (without this change) will pointlessly 2MB align the start of libc.so. Using mapsize in this change, would do the same, because mapsize includes the bss size.

Without this change, here is an example of what was happening:

604     0x7bbc00400000     0x7bbc00487000 r--   95  392  70  28 CN--- vn /lib/libc.so.7
604     0x7bbc00487000     0x7bbc00496000 ---    0    0   0   0 CN--- gd 
604     0x7bbc00496000     0x7bbc005ca000 r-x  268  392  70  28 CN--- vn /lib/libc.so.7
604     0x7bbc005ca000     0x7bbc005d9000 ---    0    0   0   0 CN--- gd 
604     0x7bbc005d9000     0x7bbc005e3000 r--   10    0   5   0 CN--- vn /lib/libc.so.7
604     0x7bbc005e3000     0x7bbc005f2000 ---    0    0   0   0 CN--- gd 
604     0x7bbc005f2000     0x7bbc005f9000 rw-    7    0   1   0 C---- vn /lib/libc.so.7
604     0x7bbc005f9000     0x7bbc0081b000 rw-   18    0   1   0 C---- sw

The 2MB alignment here accomplishes nothing, except to waste address space and page table memory, because the underlying VM objects are all less than 2MB in size. The decision to 2MB align here should really only be based on file size, not the overall mapsize.

Moreover, the 2MB alignment here doesn't help in terms of creating more superpages under the bss either.

Let me be more precise in making my last point: In general, 2MB alignment for the overall mapping, i.e., the guard, does not guarantee that the bss will optimally aligned for potential 2MB superpages. The outcome with respect to potential 2MB superpages for the bss is going to be a matter of happenstance. For a given bss start and size, a non-aligned 2MB overall mapping might be lucky, and an aligned 2MB overall mapping unlucky. In other words, we can't really achieve a deterministically better result for the bss based on 2MB alignment for the overall mapping.

@kib Could you please explain the reasoning behind these values? I'm wondering if that reasoning leads to a smaller value for 64KB pages, rather than duplicating the value for 2MB pages.

I still do not understand the formula. For instance, p_offset is the file offset, and not VA offset, of the first byte of the segment. This is why I talked about mapsize.

Could you, please, add a comment explaining the purpose of the check? Then, perhaps we can work something out from that.

The values were chosen to provide a reasonably compact layout for mappings, while still providing the number of random bits in the addresses that satisfied some ASLR verification checks.

I do not remember what the programs to measure the entropy from the mapping addresses were, but I am almost sure that it was emaste@ who run them.

The objective is simple: To not 2MB align the start of libc.so's mappings, because doing so is pointless. In general, 2MB alignment for libraries whose code and initialized data segments are together less than 2MB will not deterministically yield the possibility of more superpage mappings. It is simply a waste of address space. On the other hand, I do want the start of libcrypto.so's mappings to be 2MB aligned, because as shown in the summary with this change we will now get some 64KB page mappings on both the code and read-only data. Both libc.so and libcrypto.so have mapsizes greater than 2MB. However, only libcrypto.so comes from a file that is greater than 2MB in size, and thus is eligible to have a 2MB reservation allocated to it. That is why I am examining the file offset and size for the last segment.

Previously, we would not have 2MB aligned either libc.so or libcrypto.so because the size of the initial read-only data segment was not greater than 2MB. Now, pagesizes[1] is only 64KB, not 2MB. So, without the new condition based on the file, we would pass MAP_ALIGNED_SUPER for both, and since they both have mapsizes greater than 2MB, both would be 2MB aligned, libc.so pointlessly so.

That said, this exact change to rtld is transitional. When we have the next big patch that introduces "real" 64KB reservations to vm_reserv.c, we do want libc.so to be aligned, but only 64KB aligned. I'm testing that version of the rtld change together with the rest of this patch. If it that testing doesn't yield any inexplicable results, I will introduce it here. Maybe it will be easier to understand.

After thinking this over some more, I'm going to argue for simply having a _Static_assert that MAXPAGESIZES is greater than VM_NRESERVLEVEL.

I'm going to add a comment here.

@emaste What were these ASLR verification tests?

I'd hesitate to call them verification tests :)

The original tool of interest here was paxtest, ported to FreeBSD in https://github.com/opntr/paxtest-freebsd. It provides a reported number of bits of entropy, although it's going to be inaccurate if the distribution is not uniform.

There's also https://personales.upv.es/iripoll/aslr_main.html from the folks behind ASLR-NG but AFAIK the code is not available.

I suggest handling it differently then. Consider calculating a segment with the largest p_filesz when we loop over the segments. before creating the placeholder mapping. After the max p_filesz is known, code can correctly detect whether it would benefit from superpage promotions.