Add experimental 16k PAGE_SIZE support on arm64

[andrew]

Summary

This is considered experimental, with no guarantee of backwards
compatibility, Userspace built with a 4k PAGE_SIZE assumption will
likely try to pass in a too small size when working with APIs that
take a multiple of a page, e.g. mmap. Userspace built with a 16k
PAGE_SIZE assumption is more likely to run on a kernel providing
4k pages, however the best option is to not use a hard coded value
and ask for it when needed.

This is ready for testing, however some drivers may fail as they
make assumptions that PAGE_SIZE is 4k. Many issues have been found,
however more work is needed to test more hardware.

Test Plan

Known issues:

• Modules loaded from loader.efi may be misaligned
• ufs known to be broken
• nvme is broken
• network drivers broken

[imp]

I'm readying an nvme set of patches.
And I know there's at least one bug in the buffer cache with 16k pages that I think we'll have a review for soon.

[markj]

sys/arm64/arm64/locore.S

43

444

474

570

Why this change?

sys/arm64/arm64/pmap.c

1553

2491

The array indices are wrong.

I also wonder why this can't be a memcmp() with pc_freemask? memcmp() expands to a compiler builtin these days. Then we don't have to define PC_FREE0, ..., PC_FREE9.

sys/arm64/include/pmap.h

136

Maybe have a field uint64_t pv_pad[_NPAD] and define _NPAD to be 0 and 1 for the 4KB and 16KB cases, respectively.

[andrew]

sys/arm64/arm64/locore.S

570

L2_SIZE is too large for an add instruction with 16k pages. It needs to fit in a 12 bit field with an optional left shift of 12 which is too small.

I need to also fix the similar instruction above in the SOCDEV_PA case.

sys/arm64/arm64/pmap.c

2491

I think for 64k PAGE_SIZE it should be memcmp as I calculated _NPCM to be 43 (and _NPCM to be 2714).

[andrew]

• Fixes based on feedback from markj
• Don't use L2_SIZE in an add instruction in the SOCDEV_PA case

[andrew]

sys/arm64/arm64/pmap.c

2491

I tried with memcmp((pc)->pc_map, pc_freemask, sizeof(pc_freemask)) == 0 which clang expands to a call to memcmp (even if I use __builtin_memcmp). In the above case it generates 11 64-bit loads.

[kib]

Did you tried to do something like formatting UFS with the fragment size less than 4k (e.g. block size 8k) and trying to mount it?

[markj]

Seems ok to me overall, just a few small notes.

sys/arm64/arm64/locore.S

477

"... aligned address. However, when the page size is larger than 4k, L2 blocks are too large to map the kernel with such an alignment."

would be clearer to me. At least, there is a typo (will -> with).

sys/arm64/arm64/pmap.c

1049

This can be a while loop.

2491

Why is an out-of-line memcmp() call undesirable?

Anyway, I think we should at least stop defining PC_FREE0..9. Just have

#define PC_FREEN 0xfffffffffffffffful
#define PC_FREEL 0x0000001ffffffffful

and compare pc->pc_map[0..9] == PC_FREEN && pc->pc_map[10] == PC_FREEL.

[andrew]

In D34793#789157, @kib wrote:

Did you tried to do something like formatting UFS with the fragment size less than 4k (e.g. block size 8k) and trying to mount it?

There are patche to fix ufs. Hopefully they will be sent for review soon. With those patches UFS works with this 16k pages.

[imp]

In D34793#789157, @kib wrote:

Did you tried to do something like formatting UFS with the fragment size less than 4k (e.g. block size 8k) and trying to mount it?

That's known to fail. @chs has some fixes to the buffer cache he's polishing. Oh, wait, his fixes are for 64k ufs block size with 16k pages failing...

[andrew]

Update based on feedback from @markj

[andrew]

Fix the indentation of pv_pad and rename it pc_pad to be consistant with the other fields

[gallatin]

I was just wondering if you were planning to commit this, or what was happening with it?

[andrew]

In D34793#803162, @gallatin wrote:

I was just wondering if you were planning to commit this, or what was happening with it?

I was waiting for the UFS fixes to land.

[chs]

I was waiting for the UFS fixes to land.

Sorry about that, I hadn't realized that you were waiting for me. All of the fixes that I have for UFS are in now.