Page MenuHomeFreeBSD

riscv: Set MACHINE_ARCH define based on TARGET_ARCH
ClosedPublic

Authored by kp on Feb 18 2020, 12:58 PM.
Tags
None
Referenced Files
Unknown Object (File)
Mar 9 2024, 4:35 PM
Unknown Object (File)
Dec 20 2023, 7:19 AM
Unknown Object (File)
Sep 11 2023, 6:58 AM
Unknown Object (File)
Aug 19 2023, 12:54 PM
Unknown Object (File)
Aug 8 2023, 3:10 AM
Unknown Object (File)
Aug 8 2023, 3:09 AM
Unknown Object (File)
Aug 8 2023, 3:09 AM
Unknown Object (File)
Aug 8 2023, 3:07 AM
Subscribers

Details

Summary

MACHINE_ARCH sets the hw.machine_arch sysctl, which bsd.cpu.mk uses to
configure the target ABI for ports.
For riscv64sf builds (i.e. soft-float) that needs to be riscv64sf, but
the sysctl didn't reflect that. It is static.

Set the define from the riscv makefile so that we correctly reflect our
actual build (i.e. riscv64 or riscv64sf), depending on what TARGET_ARCH
we were built with.

Sponsored by: Axiado

Diff Detail

Repository
rS FreeBSD src repository - subversion
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

Looks good to me.
Thank you!

This revision is now accepted and ready to land.Feb 20 2020, 3:00 AM

The previous version was an incomple fix.
It did ensure that the hw.machine_arch was correct, but bmake uses the
MACHINE_ARCH define if it exists. As it includes sys/params.h this was the
case. In other words: we need to ensure that MACHINE_ARCH is set correctly for
userspace builds as well.

Happily we *can* rely on the __riscv_float_abi_soft define in userspace.

This revision now requires review to proceed.Feb 20 2020, 8:50 PM

I found the following excerpt in bsd.cpu.mk:

.if ${MACHINE_CPUARCH} == "riscv"
.if ${MACHINE_ARCH:Mriscv*sf}
CFLAGS += -march=rv64imac -mabi=lp64
.else
CFLAGS += -march=rv64imafdc -mabi=lp64d
.endif
.endif

So we could instead check for the presence of hardware floating extensions using the following check in param.h:
#if defined(__riscv_flen) && __riscv_flen != 0

I think this would eliminate the need for the define in Makefile.riscv

I found the following excerpt in bsd.cpu.mk:

.if ${MACHINE_CPUARCH} == "riscv"
.if ${MACHINE_ARCH:Mriscv*sf}
CFLAGS += -march=rv64imac -mabi=lp64
.else
CFLAGS += -march=rv64imafdc -mabi=lp64d
.endif
.endif

So we could instead check for the presence of hardware floating extensions using the following check in param.h:
#if defined(__riscv_flen) && __riscv_flen != 0

https://github.com/riscv/riscv-toolchain-conventions defines __riscv_float_abi_soft as well, so I don't know if that helps us.
It's also not explicit about what the value will set to if neither F nor D extensions are present (although yes, I'd expect it to be unset or 0).

I think this would eliminate the need for the define in Makefile.riscv

I don't think so. We build the kernel without floating point support, so it'd always think it was a soft float version.

In D23741#522430, @kp wrote:

I found the following excerpt in bsd.cpu.mk:

.if ${MACHINE_CPUARCH} == "riscv"
.if ${MACHINE_ARCH:Mriscv*sf}
CFLAGS += -march=rv64imac -mabi=lp64
.else
CFLAGS += -march=rv64imafdc -mabi=lp64d
.endif
.endif

So we could instead check for the presence of hardware floating extensions using the following check in param.h:
#if defined(__riscv_flen) && __riscv_flen != 0

https://github.com/riscv/riscv-toolchain-conventions defines __riscv_float_abi_soft as well, so I don't know if that helps us.
It's also not explicit about what the value will set to if neither F nor D extensions are present (although yes, I'd expect it to be unset or 0).

I've had it explained to me by the LLVM folks that soft-float ABI != lack of hardware floating point. They also said that in general, we can rely on __riscv_flen to detect the presence of the "f" or "d" extensions (see this comment). I agree that the document you linked is under-specified for the case where these extensions are disabled. You're right that at the moment it doesn't matter which one we use for the check, __riscv_soft_float_abi or __riscv_flen.

I think this would eliminate the need for the define in Makefile.riscv

I don't think so. We build the kernel without floating point support, so it'd always think it was a soft float version.

You're right about this still being required, since the kernel gets its CFLAGS from kern.mk, not bsd.cpu.mk.

I see that the riscv64 and riscv64sf kernels are built with identical march and mabi flags, which I think is a separate problem. Ideally, the riscv64 target would be built with -march=rv64imafdc -mabi=lp64 so that floating point instructions can be used within the kernel (e.g. during context switch), but the soft-float ABI is still used generally. To ensure riscv64sf can be run on cores that actually lack hardware floating point support, it should be compiled with -march=rv64imac -mabi=lp64, and the FPE kernel option should be removed/disabled. If this is done, we could rely on checking __riscv_flen alone to determine MACHINE_ARCH for both kernel and userland.

I'm happy with your fix as-is, but we could revisit it if/when the above is addressed.

This revision is now accepted and ready to land.Feb 21 2020, 7:51 PM
This revision was automatically updated to reflect the committed changes.
head/sys/conf/Makefile.riscv
49

This is bogus and must be removed

I suspect that we just need to set CFLAGS right in Makefile.riscv rather than this hack.

head/sys/conf/Makefile.riscv
49

Or rather, if it must be here, use MACHINE_ARCH. TARGET_ARCH isn't defined outside of Makefile.inc1.

There's certainly still something missing. When I build with make TARGET_ARCH=riscv64sf it builds with MACHINE_ARCH = riscv64, and that breaks ports builds (because we set the wrong flags).

I've tried this:

diff --git a/sys/conf/Makefile.riscv b/sys/conf/Makefile.riscv
index a29390f2e85..183fa6dd448 100644
--- a/sys/conf/Makefile.riscv
+++ b/sys/conf/Makefile.riscv
@@ -46,8 +46,6 @@ SYSTEM_LD= @${LD} -N -m ${LD_EMULATION} -Bdynamic -T ${LDSCRIPT} ${_LDFLAGS} \
 CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls
 .endif

-CFLAGS += -DMACHINE_ARCH=\"${TARGET_ARCH}\"
-
 # hack because genassym.c includes sys/bus.h which includes these.
 genassym.o: bus_if.h device_if.h

diff --git a/sys/conf/kern.mk b/sys/conf/kern.mk
index 0737f420321..40d9f24c345 100644
--- a/sys/conf/kern.mk
+++ b/sys/conf/kern.mk
@@ -306,4 +306,5 @@ LD_EMULATION_powerpc= elf32ppc_fbsd
 LD_EMULATION_powerpcspe= elf32ppc_fbsd
 LD_EMULATION_powerpc64= elf64ppc_fbsd
 LD_EMULATION_riscv64= elf64lriscv
+LD_EMULATION_riscv64sf= elf64lriscv
 LD_EMULATION=${LD_EMULATION_${MACHINE_ARCH}}
diff --git a/sys/riscv/conf/DEFAULTS b/sys/riscv/conf/DEFAULTS
index 94c6bf283c4..c06a3af22f4 100644
--- a/sys/riscv/conf/DEFAULTS
+++ b/sys/riscv/conf/DEFAULTS
@@ -3,7 +3,7 @@
 #
 # $FreeBSD$

-machine                riscv   riscv64
+machine                riscv   riscv64sf

 # Pseudo devices.
 device         mem             # Memory and kernel memory devices
diff --git a/sys/riscv/include/param.h b/sys/riscv/include/param.h
index a017ef20f6f..94e18c58041 100644
--- a/sys/riscv/include/param.h
+++ b/sys/riscv/include/param.h
@@ -46,12 +46,8 @@
 #define        MACHINE         "riscv"
 #endif
 #ifndef MACHINE_ARCH
-#ifdef __riscv_float_abi_soft
-#define        MACHINE_ARCH    "riscv64sf"
-#else
 #define        MACHINE_ARCH    "riscv64"
 #endif
-#endif

 #ifdef SMP
 #ifndef MAXCPU

While the LD_EMULATION_riscv64 at least lets it build (without it it doesn't), sysctl hw.machine_arch is still riscv64.

I'm not sure what you mean by this:

I suspect that we just need to set CFLAGS right in Makefile.riscv rather than this hack.

What do we need to set in CFLAGS?

In D23741#523122, @kp wrote:

There's certainly still something missing. When I build with make TARGET_ARCH=riscv64sf it builds with MACHINE_ARCH = riscv64, and that breaks ports builds (because we set the wrong flags).

During the actual build, MACHINE_ARCH is risc64sf for a TARGET_ARCH of riscv64sf.

I've tried this:

diff --git a/sys/conf/Makefile.riscv b/sys/conf/Makefile.riscv
index a29390f2e85..183fa6dd448 100644
--- a/sys/conf/Makefile.riscv
+++ b/sys/conf/Makefile.riscv
@@ -46,8 +46,6 @@ SYSTEM_LD= @${LD} -N -m ${LD_EMULATION} -Bdynamic -T ${LDSCRIPT} ${_LDFLAGS} \
 CFLAGS += -fno-omit-frame-pointer -fno-optimize-sibling-calls
 .endif

-CFLAGS += -DMACHINE_ARCH=\"${TARGET_ARCH}\"
-

This is still needed, but with MACHINE_ARCH instead of TARGET_ARCH. I was mistaken you needed to remove it completely. In the future, I'd like to see that, but currently something is misaligned with the ABI and what's tested for in param.h. Once that's fixed, then this can be removed.

  1. hack because genassym.c includes sys/bus.h which includes these. genassym.o: bus_if.h device_if.h

diff --git a/sys/conf/kern.mk b/sys/conf/kern.mk
index 0737f420321..40d9f24c345 100644

  • a/sys/conf/kern.mk

+++ b/sys/conf/kern.mk
@@ -306,4 +306,5 @@ LD_EMULATION_powerpc= elf32ppc_fbsd
LD_EMULATION_powerpcspe= elf32ppc_fbsd
LD_EMULATION_powerpc64= elf64ppc_fbsd
LD_EMULATION_riscv64= elf64lriscv
+LD_EMULATION_riscv64sf= elf64lriscv

This is a good idea.

LD_EMULATION=${LD_EMULATION_${MACHINE_ARCH}}
diff --git a/sys/riscv/conf/DEFAULTS b/sys/riscv/conf/DEFAULTS
index 94c6bf283c4..c06a3af22f4 100644

  • a/sys/riscv/conf/DEFAULTS

+++ b/sys/riscv/conf/DEFAULTS
@@ -3,7 +3,7 @@

  1. $FreeBSD$

-machine riscv riscv64
+machine riscv riscv64sf

This will cause the generated Makefile to have a MACHINE_ARCH=riscv64sf as it's second line. Once upon a time, we talked about being able to override machine, but I'm not sure how far we got with that. Ideally, you'd have a GENERICSF that is just

include GENERIC
machine riscv riscv64sf

and then that kernel would be the correct MACHINE_ARCH...

  • a/sys/riscv/include/param.h

+++ b/sys/riscv/include/param.h
@@ -46,12 +46,8 @@
#define MACHINE "riscv"
#endif
#ifndef MACHINE_ARCH
-#ifdef __riscv_float_abi_soft
-#define MACHINE_ARCH "riscv64sf"
-#else
#define MACHINE_ARCH "riscv64"
#endif
-#endif

This will cause it to always be riscv64. This change is almost certainly wrong

#ifdef SMP
#ifndef MAXCPU

While the LD_EMULATION_riscv64 at least lets it build (without it it doesn't), sysctl hw.machine_arch is still riscv64.

Yea, by the time you get around to setting an ld emulation, it's too late to affect the kernel .o's that have been built already.

I'm not sure what you mean by this:

I suspect that we just need to set CFLAGS right in Makefile.riscv rather than this hack.

What do we need to set in CFLAGS?

I mean you need to include the right -march or other flag in CFLAGS so that the kernel is built with the same riscv64sf ABI as userland.

With a hack to config, I have the following GENERICSF

include GENERIC
machine riscv riscv64sf

and get a kern_mib.o that had riscv64sf in it (before the change, it was absent). Which should get you want you need in the short term.

I've committed the config hacks and created GENERICSF that compiles as well as GENERIC
https://reviews.freebsd.org/D23812
and has the right machine_arch, at least in the kern_kib.o file. I don't have the means to test it other than that and gcc8 is generating errors

ld: warning: Non-zero addend in R_RISCV_PCREL_LO12 relocation to accf_http.kld:(soishttpconnected) is ignored
ld: error: R_RISCV_PCREL_LO12 relocation points to accf_http.kld:(soishttpconnected) without an associated R_RISCV_PCREL_HI20 relocation
ld: warning: Non-zero addend in R_RISCV_PCREL_LO12 relocation to accf_http.kld:(soparsehttpvers) is ignored
ld: error: R_RISCV_PCREL_LO12 relocation points to accf_http.kld:(soparsehttpvers) without an associated R_RISCV_PCREL_HI20 relocation
*** [accf_http.ko.full] Error code 1