Page MenuHomeFreeBSD
Differential D34793

Add experimental 16k PAGE_SIZE support on arm64
ClosedPublic
Actions

Authored by andrew on Apr 6 2022, 1:56 PM.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Mar 17, 2:14 AM
Unknown Object (File)
Tue, Mar 17, 2:09 AM
Unknown Object (File)
Tue, Mar 10, 3:14 PM
Unknown Object (File)
Tue, Mar 10, 2:41 PM
Unknown Object (File)
Sun, Mar 8, 3:03 AM
Unknown Object (File)
Sun, Mar 8, 1:51 AM
Unknown Object (File)
Feb 27 2026, 8:04 PM
Unknown Object (File)
Feb 26 2026, 4:47 PM
View All Files
Subscribers
chs
emaste

Details

Reviewers
alc
kib
markj
gallatin
imp
manu
Group Reviewers
arm64
Commits
rG36f1526a598c: Add experimental 16k page support on arm64
Summary

This is considered experimental, with no guarantee of backwards
compatibility, Userspace built with a 4k PAGE_SIZE assumption will
likely try to pass in a too small size when working with APIs that
take a multiple of a page, e.g. mmap. Userspace built with a 16k
PAGE_SIZE assumption is more likely to run on a kernel providing
4k pages, however the best option is to not use a hard coded value
and ask for it when needed.

This is ready for testing, however some drivers may fail as they
make assumptions that PAGE_SIZE is 4k. Many issues have been found,
however more work is needed to test more hardware.

Test Plan

Known issues:

  • Modules loaded from loader.efi may be misaligned
  • ufs known to be broken
  • nvme is broken
  • network drivers broken

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 45018
Build 41906: arc lint + arc unit

Event Timeline

andrew created this revision.Apr 6 2022, 1:56 PM
Herald added a reviewer: manu. · View Herald TranscriptApr 6 2022, 1:56 PM
Herald added a subscriber: emaste. · View Herald Transcript
andrew requested review of this revision.Apr 6 2022, 1:56 PM
Harbormaster completed remote builds in B45018: Diff 104658.Apr 6 2022, 1:56 PM
andrew edited the test plan for this revision. (Show Details)Apr 6 2022, 1:59 PM
imp added a comment.Apr 6 2022, 2:54 PM

I'm readying an nvme set of patches.
And I know there's at least one bug in the buffer cache with 16k pages that I think we'll have a review for soon.

markj added inline comments.Apr 6 2022, 6:05 PM
sys/arm64/arm64/locore.S
43
433
463
555

Why this change?

sys/arm64/arm64/pmap.c
1553
2494

The array indices are wrong.

I also wonder why this can't be a memcmp() with pc_freemask? memcmp() expands to a compiler builtin these days. Then we don't have to define PC_FREE0, ..., PC_FREE9.

sys/arm64/include/pmap.h
134

Maybe have a field uint64_t pv_pad[_NPAD] and define _NPAD to be 0 and 1 for the 4KB and 16KB cases, respectively.

andrew added inline comments.Apr 6 2022, 7:02 PM
sys/arm64/arm64/locore.S
555

L2_SIZE is too large for an add instruction with 16k pages. It needs to fit in a 12 bit field with an optional left shift of 12 which is too small.

I need to also fix the similar instruction above in the SOCDEV_PA case.

sys/arm64/arm64/pmap.c
2494

I think for 64k PAGE_SIZE it should be memcmp as I calculated _NPCM to be 43 (and _NPCM to be 2714).

andrew updated this revision to Diff 104708.Apr 7 2022, 9:20 AM
andrew marked 5 inline comments as done.
  • Fixes based on feedback from markj
  • Don't use L2_SIZE in an add instruction in the SOCDEV_PA case
Harbormaster completed remote builds in B45044: Diff 104708.Apr 7 2022, 9:20 AM
andrew added inline comments.Apr 7 2022, 11:03 AM
sys/arm64/arm64/pmap.c
2494

I tried with memcmp((pc)->pc_map, pc_freemask, sizeof(pc_freemask)) == 0 which clang expands to a call to memcmp (even if I use __builtin_memcmp). In the above case it generates 11 64-bit loads.

kib added a comment.Apr 7 2022, 1:50 PM

Did you tried to do something like formatting UFS with the fragment size less than 4k (e.g. block size 8k) and trying to mount it?

markj added a comment.Apr 7 2022, 2:32 PM

Seems ok to me overall, just a few small notes.

sys/arm64/arm64/locore.S
466

"... aligned address. However, when the page size is larger than 4k, L2 blocks are too large to map the kernel with such an alignment."

would be clearer to me. At least, there is a typo (will -> with).

sys/arm64/arm64/pmap.c
1049

This can be a while loop.

2494

Why is an out-of-line memcmp() call undesirable?

Anyway, I think we should at least stop defining PC_FREE0..9. Just have

#define PC_FREEN 0xfffffffffffffffful
#define PC_FREEL 0x0000001ffffffffful

and compare pc->pc_map[0..9] == PC_FREEN && pc->pc_map[10] == PC_FREEL.

andrew added a comment.Apr 7 2022, 2:44 PM
In D34793#789157, @kib wrote:

Did you tried to do something like formatting UFS with the fragment size less than 4k (e.g. block size 8k) and trying to mount it?

There are patche to fix ufs. Hopefully they will be sent for review soon. With those patches UFS works with this 16k pages.

imp added a subscriber: chs.Apr 7 2022, 3:02 PM
In D34793#789157, @kib wrote:

Did you tried to do something like formatting UFS with the fragment size less than 4k (e.g. block size 8k) and trying to mount it?

That's known to fail. @chs has some fixes to the buffer cache he's polishing. Oh, wait, his fixes are for 64k ufs block size with 16k pages failing...

andrew updated this revision to Diff 105160.Apr 19 2022, 12:42 PM
andrew marked 2 inline comments as done.

Update based on feedback from @markj

Harbormaster completed remote builds in B45222: Diff 105160.Apr 19 2022, 12:42 PM
andrew updated this revision to Diff 105161.Apr 19 2022, 12:48 PM
andrew marked 3 inline comments as done.

Fix the indentation of pv_pad and rename it pc_pad to be consistant with the other fields

Harbormaster completed remote builds in B45223: Diff 105161.Apr 19 2022, 12:48 PM
markj accepted this revision.Apr 21 2022, 5:50 PM
This revision is now accepted and ready to land.Apr 21 2022, 5:50 PM
gallatin added a comment.Jun 6 2022, 9:51 PM

I was just wondering if you were planning to commit this, or what was happening with it?

andrew added a comment.Jun 17 2022, 4:09 PM
In D34793#803162, @gallatin wrote:

I was just wondering if you were planning to commit this, or what was happening with it?

I was waiting for the UFS fixes to land.

chs added a comment.Jun 23 2022, 2:22 AM

I was waiting for the UFS fixes to land.

Sorry about that, I hadn't realized that you were waiting for me. All of the fixes that I have for UFS are in now.

Closed by commit rG36f1526a598c: Add experimental 16k page support on arm64 (authored by andrew). · Explain WhyJul 19 2022, 9:58 AM
This revision was automatically updated to reflect the committed changes.
andrew added a commit: rG36f1526a598c: Add experimental 16k page support on arm64.

Revision Contents
Changeset List

PathSize
sys/
arm64/
arm64/
129 lines
6 lines
150 lines
include/
11 lines
42 lines

Diff 104658

View Options

sys/arm64/arm64/locore.S

Loading...
View Options

sys/arm64/arm64/minidump_machdep.c

Loading...
View Options

sys/arm64/arm64/pmap.c

Loading...
View Options

sys/arm64/include/pmap.h

Loading...
View Options

sys/arm64/include/pte.h

Loading...