Page MenuHomeFreeBSD

[PPC64] Implement CAS
ClosedPublic

Authored by luporl on Jul 1 2019, 5:57 PM.
Tags
None
Referenced Files
Unknown Object (File)
Sun, Mar 10, 5:10 AM
Unknown Object (File)
Jan 27 2024, 5:08 AM
Unknown Object (File)
Jan 6 2024, 12:19 AM
Unknown Object (File)
Jan 6 2024, 12:19 AM
Unknown Object (File)
Jan 6 2024, 12:18 AM
Unknown Object (File)
Jan 6 2024, 12:18 AM
Unknown Object (File)
Jan 6 2024, 12:18 AM
Unknown Object (File)
Jan 6 2024, 12:18 AM
Subscribers

Details

Summary

Guest PPC OSs running under a hypervisor may communicate the features they support, in order for the hypervisor to expose a virtualized machine in the way the client (guest OS) expects (see LoPAPR 1.1 - B.6.2.3).

This is done by calling the "/ibm,client-architecture-support" (CAS) method, informing supported features in option vectors.
Until now, FreeBSD wasn't using CAS, but instead relied on hypervisor/QEMU's defaults.

The problem is that, without CAS, it is very inconvenient to run POWER9 VMs on a POWER9 host running with radix enabled.
This happens because, in this case, the QEMU default is to present the guest OS a dual MMU (HPT/RPT), instead of presenting a regular HPT MMU, as FreeBSD expects, resulting in an early panic. The known workarounds required either changing the host to disable radix or passing a flag to QEMU to run in a POWER8 compatible mode.

With CAS, FreeBSD is now able to communicate that it wants an HPT MMU, independent of the host setup, which now makes FreeBSD work on POWER9/pseries, with KVM enabled and without hugepages (support added in a previous commit).

As CAS is invoked through OpenFirmware's call-method interface, it needs to be performed early, when OpenFirmware is still operational. Besides, now that FDT is the default way to inspect the device tree on PPC, OFW call-method feature will be unavailable by default, when control is passed to the kernel. Because of this, the call to CAS is being performed at the loader, instead of at the kernel.

Finally, to avoid regressions with old platforms, this change uses CAS only on POWER8/POWER9.

Diff Detail

Lint
Lint Passed
Unit
No Test Coverage
Build Status
Buildable 25159
Build 23853: arc lint + arc unit

Event Timeline

stand/powerpc/ofw/cas.c
199

Would this prevent loader from being used on powernv? I know currently we don't boot with loader on POWER8 and POWER9, but that may change in the future.

stand/powerpc/ofw/cas.c
199

Yes, it would.
What if I add a check for PSL_HV bit and skip CAS if it is set? Kernel code seems to use this check to skip code that is specific to virtualized environments.

stand/powerpc/ofw/cas.c
199

That should work, I think.

Skip CAS if PSL_HV bit is set on MSR

This avoids limiting loader to pseries systems only

  • Fix inverted tab/newline in asm
luporl added inline comments.
stand/powerpc/ofw/cas.c
199

HV bit test added. I've confirmed it works on pseries.

luporl marked an inline comment as done.
  • [PPC64] Implement CAS
  • Skip CAS if PSL_HV bit is set on MSR

I've posted a previous diff over an already patched file, that temporarily screwed the patch/history.

It is fixed now, sorry.

stand/powerpc/ofw/Makefile
42

do we need debugging enabled unconditionally?

stand/powerpc/ofw/cas.c
182

Can you use machine/psl.h and machine/cpufunc.h? Then you can use 'uint64_t hv = mfmsrd() & PSL_SF; return (hv != 0);'

luporl added inline comments.
stand/powerpc/ofw/Makefile
42

Yes, we do. Other parts of stand already do this.

The loader is stripped before being installed, so that all debug info is removed.

But without this, loader.full was inconsistent during debugging: some parts of it had debugging symbols, while others had not.

stand/powerpc/ofw/cas.c
182

Ok, I'll give it a try and see if works.

luporl added inline comments.
stand/powerpc/ofw/cas.c
182

It doesn't work, as doing so raises a couple of issues.

The first issue is that stand is 32-bit, so __powerpc64__ is not defined and would need to be defined manually to get the right contents from psl.h and cpufunc.h.

The second issue is that the needed contents are protected by an "#ifdef _KERNEL", needing to define it manually to make it work.

The third issue, that I haven't investigate deeper, is that register_t ends up being defined as a 32-bit integer, which truncates the value returned by mfmsr(), resulting in wrong code that will test a bit that will always be 0.

So in summary it seems that, at least currently, it is not possible to reuse cpufunc.h and psl.h in stand.

Thanks for checking. Looks good. I assume you tested on bare metal, too?

This revision is now accepted and ready to land.Jul 31 2019, 6:48 PM

Thanks for checking. Looks good. I assume you tested on bare metal, too?

Actually, I've tested on VMs only, as this changes loader only, that is not used on POWER8 and above.
Do you want to test this on a G5? Or is there a way to test loader under PetitBoot?

Thanks for checking. Looks good. I assume you tested on bare metal, too?

Actually, I've tested on VMs only, as this changes loader only, that is not used on POWER8 and above.
Do you want to test this on a G5? Or is there a way to test loader under PetitBoot?

You're right. Nevermind. Good to go.

This revision was automatically updated to reflect the committed changes.