ddb: Add basic CTF support [1/2]

[bnovkov]

Summary

This patch makes the kernel's .SUNW_ctf section loadable on amd64 using linker scripts and objcopy.
It is the first of two patches that enable ddb to use CTF data.

This patch adds the following:

• The ability to include the kernel's .SUNW_ctf section in a loadable ELF segment
• A new 'SUNW_LOADABLE' makeoptions variable that controls this feature
• Kernel support to initialize relevant kernel CTF structures if a loadable section is present
• A new 'linker_kernel_ctf_get_ddb' method to fetch loaded CTF data

Retrieving the kernel's CTF data from ddb requires a different approach as ddb
cannot use the existing linker_kernel_ctf_get linker method since it performs dynamic memory allocation and IO.
Since the CTF data must available without IO, it must be integrated into the kernel's image and loaded during boot.
Unfortunately, ctfmerge always assumes that the .SUNW_ctf section is not included in any loadable segments and generates
a non-SHF_ALLOC'd section, making it impossible to load it during boot.

I've tinkered with ctfmerge to try and "append" a new loadable segment, but this did not yield any results and
generally felt inappropriate as this is not something ctfmerge should implement. Instead, I've opted for a series
of objcopy operations and an additional kernel linking to achieve this.
This is done post-compilation after the kernel has been linked and after ctfmerge is run.
The CTF data gets dumped from the kernel into a placeholder file and the kernel gets linked once again.
During this second linking, the linker script places the CTF data from the placeholder file into a loadable segment.
The .SUNW_ctf section's data and properties are preserved and compatible with existing kernel CTF interfaces but are now accessible from ddb.

[markj]

Thank you for working on this. I will be slow to review anything until the end of next week, sorry in advance.

The changes to kern.post.mk seem like they'll be fragile, and I somewhat dislike them. I think I agree that ctfmerge shouldn't mess with the program header. I wonder if there's a better approach. We could eagerly pre-load CTF info before a panic, during boot, once filesystems are available. Though, in that case we cannot use CTF for panics that occur before CTF info is loaded, i.e., early boot panics. Though, with your approach this is also true for panics prior to SI_SUB_KLD, so maybe it's not a fatal problem?

I seem to remember that @jhb had some opinions on the existing CTF loader, maybe he has a suggestion here.

sys/conf/kern.post.mk

208

How does this get set? Is it a kernel config option? Or is one expected to pass it on the command line, as in make -DSUNW_LOADABLE buildkernel?

215

It's rather ugly that this has to be repeated. Ditto for the kernel link. I don't have a better suggestion offhand though.

226

With this approach, do we need to do any extra work for kernel modules?

sys/kern/kern_ctf.c

303 ↗

(On Diff #114628)

sys/kern/kern_linker.c

787 ↗

(On Diff #114628)

Extra newline.

[bnovkov]

Address @markj 's comments.

[bnovkov]

In D37898#862056, @markj wrote:

Thank you for working on this. I will be slow to review anything until the end of next week, sorry in advance.

The changes to kern.post.mk seem like they'll be fragile, and I somewhat dislike them. I think I agree that ctfmerge shouldn't mess with the program header. I wonder if there's a better approach.

While I agree that these changes feel a bit hackish and fragile, I think that they are way less fragile than tampering with an already linked ELF file.
The closest thing to "appending a new segment" that I could find was GNU objcopy's --update-section flag [1] and the 'add' interface in the LIEF reverse engineering framework [2].
Maybe we could extend objcopy with an "--append-segment" flag? That feels more appropriate, but I'm not sure if its possible.

We could eagerly pre-load CTF info before a panic, during boot, once filesystems are available. Though, in that case we cannot use CTF for panics that occur before CTF info is loaded, i.e., early boot panics.

Of course, this is also a viable option.

Though, with your approach this is also true for panics prior to SI_SUB_KLD, so maybe it's not a fatal problem?

This approach is not set in stone, I merely chose it to remain compatible with the existing CTF interfaces.
Since the CTF data gets loaded during boot and is available through a linker symbol, we could easily initialize ddb's CTF data in ddb_init() - the only reason I haven't done this is because I couldn't think of a non-hacky way of populating the linker_kernel_file CTF data afterward.

[1] https://sourceware.org/binutils/docs/binutils/objcopy.html
[2] https://lief-project.github.io/doc/latest/api/python/elf.html

sys/conf/kern.post.mk

208

It is set through a kernel config option -

makeoptions     SUNW_LOADABLE=1

226

We would need a bit of extra work, but this mostly boils down to slight ldscript modifications and adjusting the existing linker methods used for loading modules to load the section and unpack the data. This also wouldn't require any additional linker script symbols as we can load the section header string table.

[bnovkov]

I've changed to use the loader for registering kernel CTF data instead of the previous double-linking approach. The idea is that the user can opt-in to loading raw kernel CTF data using loader.conf as follows:

ddb_kctf_load="YES"
ddb_kctf_name="/boot/kernel/kctf.raw"
ddb_kctf_type="ddb_kctf"

The data gets picked up by ddb during early boot.
The changes to the build files make sure that the merged CTF data is uncompressed and dumped into the kctf.raw file. I am not sure if these changes follow proper procedure for installing non-module files, but I would be glad to rework them if necessary.