Page MenuHomeFreeBSD

sdt: Use the "cc" operand modifier for the address of probes for GCC 15+
ClosedPublic

Authored by jhb on Jan 30 2026, 3:06 PM.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Mar 5, 1:54 PM
Unknown Object (File)
Tue, Mar 3, 12:28 AM
Unknown Object (File)
Mon, Mar 2, 4:22 AM
Unknown Object (File)
Wed, Feb 25, 3:00 PM
Unknown Object (File)
Tue, Feb 24, 8:14 AM
Unknown Object (File)
Tue, Feb 24, 2:49 AM
Unknown Object (File)
Sun, Feb 22, 6:50 PM
Unknown Object (File)
Sun, Feb 22, 6:13 AM
Subscribers

Details

Summary

This is required for GCC on RISC-V. The GCC 15 docs claim that "cc" is
similar to "c" except that it "tries harder".

NB: I have not yet found a way to make the DTrace probes compile on
RISC-V with older versions of GCC.

Diff Detail

Repository
rG FreeBSD src repository
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

jhb requested review of this revision.Jan 30 2026, 3:06 PM

GCC documents operand modifiers here: https://gcc.gnu.org/onlinedocs/gcc-15.2.0/gcc/Extended-Asm.html#Generic-Operand-Modifiers

Clang doesn't document these things, but Jess went dumpster diving in the source code and believes that clang only supports single-character modifiers. I still need to do a clang universe build and perhaps test the amd64 kernel build as well with this in place.

Oh dear, GCC added "cc" only in GCC 15. GCC 14 is unable to compile SDT probes with GCC 14 on RISC-V. :(

I tried using a constraint of "m" and an operand modifier of "a" which I think is more what we want anyway (if you remove the & in the operand), but that gives me ICEs in both GCC 14 and GCC 15. This whole thing seems a bit gross. I realize we want the real symbol here and don't want it to resolve to 'foo@plt' or the like, but this whole thing feels a bit clunky. I guess we can't just do _SDT_ASM_WORD _SDT_PROBE_NAME(...) directly? (Presumably on CHERI _SDT_WORD here is a .chericap and would really need the symbol name and not some goofy "make the compiler generate a constant address" thing anyway?

In D54964#1256808, @jhb wrote:

I tried using a constraint of "m" and an operand modifier of "a" which I think is more what we want anyway (if you remove the & in the operand), but that gives me ICEs in both GCC 14 and GCC 15. This whole thing seems a bit gross. I realize we want the real symbol here and don't want it to resolve to 'foo@plt' or the like, but this whole thing feels a bit clunky. I guess we can't just do _SDT_ASM_WORD _SDT_PROBE_NAME(...) directly?

Sadly no. This doesn't work for function-scoped static variables, since there the variable name is not equal to that of the emitted symbol. And the DTRACE_PROBE* macros defined in sdt.h (and I think only used by ZFS) need this to work.

(Presumably on CHERI _SDT_WORD here is a .chericap and would really need the symbol name and not some goofy "make the compiler generate a constant address" thing anyway?

It expands to .chericap "%c0" with the operand defined as "i" (__unbounded_addressof(_SDT_PROBE_NAME(...)).

In D54964#1256808, @jhb wrote:

I tried using a constraint of "m" and an operand modifier of "a" which I think is more what we want anyway (if you remove the & in the operand), but that gives me ICEs in both GCC 14 and GCC 15. This whole thing seems a bit gross. I realize we want the real symbol here and don't want it to resolve to 'foo@plt' or the like, but this whole thing feels a bit clunky. I guess we can't just do _SDT_ASM_WORD _SDT_PROBE_NAME(...) directly?

Sadly no. This doesn't work for function-scoped static variables, since there the variable name is not equal to that of the emitted symbol. And the DTRACE_PROBE* macros defined in sdt.h (and I think only used by ZFS) need this to work.

(Presumably on CHERI _SDT_WORD here is a .chericap and would really need the symbol name and not some goofy "make the compiler generate a constant address" thing anyway?

It expands to .chericap "%c0" with the operand defined as "i" (__unbounded_addressof(_SDT_PROBE_NAME(...)).

Humm, what what does the %c0 expand to? You can't really do .chericap 0x12345 and have it be meaningful at all. It has to be a symbol so that the assembler emits a static relocation that the linker can use to figure out the right bounds and permissions so it can construct a suitable relocation (either a .caprelocs / R_MORELLO_RELATIVE for a non-preemptible symbol, or a R_CHERI_CAPABILITY).

jhb retitled this revision from sdt: Use the "cc" operand modifier for the address of probes for GCC to sdt: Use the "cc" operand modifier for the address of probes for GCC 15+.Jan 31 2026, 5:24 PM
jhb edited the summary of this revision. (Show Details)
In D54964#1256922, @jhb wrote:
In D54964#1256808, @jhb wrote:

I tried using a constraint of "m" and an operand modifier of "a" which I think is more what we want anyway (if you remove the & in the operand), but that gives me ICEs in both GCC 14 and GCC 15. This whole thing seems a bit gross. I realize we want the real symbol here and don't want it to resolve to 'foo@plt' or the like, but this whole thing feels a bit clunky. I guess we can't just do _SDT_ASM_WORD _SDT_PROBE_NAME(...) directly?

Sadly no. This doesn't work for function-scoped static variables, since there the variable name is not equal to that of the emitted symbol. And the DTRACE_PROBE* macros defined in sdt.h (and I think only used by ZFS) need this to work.

(Presumably on CHERI _SDT_WORD here is a .chericap and would really need the symbol name and not some goofy "make the compiler generate a constant address" thing anyway?

It expands to .chericap "%c0" with the operand defined as "i" (__unbounded_addressof(_SDT_PROBE_NAME(...)).

Humm, what what does the %c0 expand to? You can't really do .chericap 0x12345 and have it be meaningful at all. It has to be a symbol so that the assembler emits a static relocation that the linker can use to figure out the right bounds and permissions so it can construct a suitable relocation (either a .caprelocs / R_MORELLO_RELATIVE for a non-preemptible symbol, or a R_CHERI_CAPABILITY).

It gives you the symbol

In D54964#1256922, @jhb wrote:
In D54964#1256808, @jhb wrote:

I tried using a constraint of "m" and an operand modifier of "a" which I think is more what we want anyway (if you remove the & in the operand), but that gives me ICEs in both GCC 14 and GCC 15. This whole thing seems a bit gross. I realize we want the real symbol here and don't want it to resolve to 'foo@plt' or the like, but this whole thing feels a bit clunky. I guess we can't just do _SDT_ASM_WORD _SDT_PROBE_NAME(...) directly?

Sadly no. This doesn't work for function-scoped static variables, since there the variable name is not equal to that of the emitted symbol. And the DTRACE_PROBE* macros defined in sdt.h (and I think only used by ZFS) need this to work.

(Presumably on CHERI _SDT_WORD here is a .chericap and would really need the symbol name and not some goofy "make the compiler generate a constant address" thing anyway?

It expands to .chericap "%c0" with the operand defined as "i" (__unbounded_addressof(_SDT_PROBE_NAME(...)).

Humm, what what does the %c0 expand to? You can't really do .chericap 0x12345 and have it be meaningful at all. It has to be a symbol so that the assembler emits a static relocation that the linker can use to figure out the right bounds and permissions so it can construct a suitable relocation (either a .caprelocs / R_MORELLO_RELATIVE for a non-preemptible symbol, or a R_CHERI_CAPABILITY).

It gives you the symbol

But I guess it can come up with the "right" symbol for function-scope variables? Blech. I guess we will just have to say that RISC-V kernels can't be built with GCC older than 15. :(

This revision is now accepted and ready to land.Jan 31 2026, 7:18 PM

I retested SDT with gcc14 on amd64. There is a problem (not related to this patch): the linker decides that the start_set_sdt_providers_set and stop_set_sdt_providers_set symbols are unreferenced and removes them. This means that linker_file_lookup_set() returns nothing for those sets, which causes further problems.

Adding a dummy SYSINIT that does a SET_FOREACH over the linker set fixes the problem, but that's a kludge. Alternately I think I can add directives to all linker scripts to preserve these symbols, but that's pretty ugly too (I have to do it for all kmods too). Is there a better way?

IIRC GNU as and LLVM IAS disagree on what .weak and .glob[a]l mean. LLVM IAS doesn't let you mix them, but that's not really sensible, since one is binding and the other is visibility. We use .weak for linker sets but not .glob[a]l (I think used to use both) because one version of LLVM introduced that error, whilst I think GNU as wants what we used to have where we put both. I should probably raise it with upstream LLVM to point out that what they did doesn't make sense.

IIRC GNU as and LLVM IAS disagree on what .weak and .glob[a]l mean. LLVM IAS doesn't let you mix them, but that's not really sensible, since one is binding and the other is visibility. We use .weak for linker sets but not .glob[a]l (I think used to use both) because one version of LLVM introduced that error, whilst I think GNU as wants what we used to have where we put both. I should probably raise it with upstream LLVM to point out that what they did doesn't make sense.

Ah, this is commit 32231805fbe2b9438c2de50c229b43c016207a08. Indeed, reverting that fixes the problem for me, but of course I can't compile with LLVM after.

IIRC GNU as and LLVM IAS disagree on what .weak and .glob[a]l mean. LLVM IAS doesn't let you mix them, but that's not really sensible, since one is binding and the other is visibility. We use .weak for linker sets but not .glob[a]l (I think used to use both) because one version of LLVM introduced that error, whilst I think GNU as wants what we used to have where we put both. I should probably raise it with upstream LLVM to point out that what they did doesn't make sense.

Ah, this is commit 32231805fbe2b9438c2de50c229b43c016207a08. Indeed, reverting that fixes the problem for me, but of course I can't compile with LLVM after.

We can always use more #ifdef to add it back for only GCC for now, and if LLVM is ever fixed also enable it for sufficiently new versions of LLVM.