This change allows the veriexec binary to (optionally) load its CA store from a verified tarball.

[sebastien.bini_stormshield.eu]

Summary

This development is motivated by the need to bring more flexibility in the build system: veriexec no longer needs to embed its CA store in its source code.

The CA store can then be stored in a separate tarball, which must contain two files:

• "trust.pem": the list of trusted certificates ;
• "forbidden.pem": the list of revoked certificates, which may be empty.

The tarball must be verified by mac_veriexec, or else will not be opened. The tarball is further extracted in memory (and not on the filesystem) so the extracted CAs cannot be tampered with.
The loaded certificates are added to the list of embedded certificates. Then the call to ve_trust_init will load all certificates (embedded and from the tarball) into its crypto engine.

A new option 'a' is introduced to specify the path of the tarball. The default path is "/etc/veriexec/anchors.txz".
If the option is not set and that the default tarball does not exist, then veriexec continues normally and will attempt to load its CA store from its source code.
If the option is set or if the tarball exists then any error while reading the tarball will cause the program to stop.

[wma]

Any comments?

[sjg]

The basic premise here is incorrect. There is a circular dependency.
veriexec cannot rely on O_VERIFY since veriexec is responsible for seeding mac_veriexec to enable O_VERIFY.
You would need to verify a detached signature of the archive - but then where do you get the trust anchors for that...

lib/libsecureboot/Makefile.inc

84

Don't repeat -I. and the define should probably depend on a source of forbidden anchors being provided.

160

You could add something like:

.if ${FA_PEM_LIST:[#]} > 0
echo '#define FORBIDDEN_ANCHORS_STR fa_PEM'
.endif

?

lib/libsecureboot/local.trust.mk

128

Or put the CFLAGS+= -DFORBIDDEN_ANCHORS_STR=fa_PEM here

sbin/veriexec/veriexec.c

152

Sorry this won't fly.
veriexec cannot rely on O_VERIFY since veriexec is responsible for seeding mac_veriexec to enable O_VERIFY.
You would need to do this more like the loader does.

[sjg]

If the point here is to be able to *add* trust anchors or revoke them from an archive that was verified via the built in trust anchors, then that could work.
But the log message implies that is not your goal.
As stated I don't think it can be secure.

[sebastien.bini_stormshield.eu]

In D33926#773586, @sjg wrote:

The basic premise here is incorrect. There is a circular dependency.
veriexec cannot rely on O_VERIFY since veriexec is responsible for seeding mac_veriexec to enable O_VERIFY.
You would need to verify a detached signature of the archive - but then where do you get the trust anchors for that...

Thank you for your feedback. In our code base, we circumvent the circular dependency you mentioned:

• The bootloader embeds its own CA store in its source code and verifies a first manifest file.
• The hash of this manifest is passed to the kernel via kenv.
• When the kernel mounts root, it verifies the hash of the manifest, and if correct, feeds the contents of the manifest to mac_veriexec.

Hence, once the system is started, mac_veriexec is initialized with a small set of files that are verified. Of course, in our case, the CA tarball ("/etc/veriexec/anchors.txz") is protected by the manifest that is verified by the loader, so it is known by mac_veriexec. Hence the O_VERIFY call will succeed.

[sebastien.bini_stormshield.eu]

sbin/veriexec/veriexec.c

152

As I explained, there are no circular dependencies. The aim of this review is to externalize the CA store of veriexec, so we are not forced to embed them in its source code.

[sjg]

In D33926#775518, @sebastien.bini_stormshield.eu wrote:

In D33926#773586, @sjg wrote:

The basic premise here is incorrect. There is a circular dependency.
veriexec cannot rely on O_VERIFY since veriexec is responsible for seeding mac_veriexec to enable O_VERIFY.
You would need to verify a detached signature of the archive - but then where do you get the trust anchors for that...

Thank you for your feedback. In our code base, we circumvent the circular dependency you mentioned:

• The bootloader embeds its own CA store in its source code and verifies a first manifest file.
• The hash of this manifest is passed to the kernel via kenv.
• When the kernel mounts root, it verifies the hash of the manifest, and if correct, feeds the contents of the manifest to mac_veriexec.

Hence, once the system is started, mac_veriexec is initialized with a small set of files that are verified. Of course, in our case, the CA tarball ("/etc/veriexec/anchors.txz") is protected by the manifest that is verified by the loader, so it is known by mac_veriexec. Hence the O_VERIFY call will succeed.

Thanks for clarifying. The problem is; how does the kernel know/trust that the loader really verified anything ? rather than simply a loader.conf putting a hash into kenv?
It is one thing for the loader to verify the kernel before loading it (we also verify the kernel's rootfs) but the kernel cannot really verify the loader - or trust anything in kenv.

[sjg]

Also I'm curious; if you are ok embedding trust anchors in the loader, what is the problem with embedding them in veriexec?

[sjg]

lib/libsecureboot/vets.c

40

This should be bounded by #ifdef FORBIDDEN_ANCHORS_STR

sbin/veriexec/veriexec.c

58

If you really want to do something like this, it should be controlled and bounded by a knob, so that those who are more paranoid do not have to worry about it.

In fact it might be better to put into a separate translation unit.
When we do code reviews for certifications, it is much easier to convince them that code is not active if they can see it isn't compiled.

[sebastien.bini_stormshield.eu]

In D33926#775631, @sjg wrote:

Also I'm curious; if you are ok embedding trust anchors in the loader, what is the problem with embedding them in veriexec?

Legacy build system basically :s

It's much more convenient for us to separate the program compilation from its cryptographic configuration. This way the program can be compiled once and be used with various (trusted) CA stores.

[sebastien.bini_stormshield.eu]

In D33926#775630, @sjg wrote:

Thanks for clarifying. The problem is; how does the kernel know/trust that the loader really verified anything ? rather than simply a loader.conf putting a hash into kenv?
It is one thing for the loader to verify the kernel before loading it (we also verify the kernel's rootfs) but the kernel cannot really verify the loader - or trust anything in kenv.

SecureBoot verifies the loader so we know it has not be tampered with. I wouldn't know how one could guarantee veriexec to work if the loader is not protected by SecureBoot, as you are right, the kernel cannot verify the loader.
Our loader is further patched so to authorize only a whitelist of kenv in loader.conf and the likes, so the manifest hash cannot be passed this way.

[sebastien.bini_stormshield.eu]

sbin/veriexec/veriexec.c

58

Fair point, we will rework the patch then.

[sjg]

In D33926#775929, @sebastien.bini_stormshield.eu wrote:

In D33926#775630, @sjg wrote:

Thanks for clarifying. The problem is; how does the kernel know/trust that the loader really verified anything ? rather than simply a loader.conf putting a hash into kenv?
It is one thing for the loader to verify the kernel before loading it (we also verify the kernel's rootfs) but the kernel cannot really verify the loader - or trust anything in kenv.

SecureBoot verifies the loader so we know it has not be tampered with. I wouldn't know how one could guarantee veriexec to work if the loader is not protected by SecureBoot, as you are right, the kernel cannot verify the loader.
Our loader is further patched so to authorize only a whitelist of kenv in loader.conf and the likes, so the manifest hash cannot be passed this way.

At the end of the day secure-boot is a house of cards.
Each stage can only verify the next, not any prior stage, it should therefor minimize trust in all earlier stages.

[sjg]

In D33926#775928, @sebastien.bini_stormshield.eu wrote:

In D33926#775631, @sjg wrote:

Also I'm curious; if you are ok embedding trust anchors in the loader, what is the problem with embedding them in veriexec?

Legacy build system basically :s

It's much more convenient for us to separate the program compilation from its cryptographic configuration. This way the program can be compiled once and be used with various (trusted) CA stores.

When I originally designed the trust model for veriexec's signed manifest, I end up with embedded trust anchors because I could not come up with an alternative that could not be compromised. Of course as an embedded vendor, the set of trust anchors required is quite small and virtually static, so it is an easy choice.
What about a pre-loaded kernel module to hold your initial trust anchor(s)? need only be a sysctl ? not bullet proof, but a bit harder to spoof than kenv?

[sebastien.bini_stormshield.eu]

In D33926#776097, @sjg wrote:

In D33926#775928, @sebastien.bini_stormshield.eu wrote:

In D33926#775631, @sjg wrote:

Also I'm curious; if you are ok embedding trust anchors in the loader, what is the problem with embedding them in veriexec?

Legacy build system basically :s

It's much more convenient for us to separate the program compilation from its cryptographic configuration. This way the program can be compiled once and be used with various (trusted) CA stores.

When I originally designed the trust model for veriexec's signed manifest, I end up with embedded trust anchors because I could not come up with an alternative that could not be compromised. Of course as an embedded vendor, the set of trust anchors required is quite small and virtually static, so it is an easy choice.
What about a pre-loaded kernel module to hold your initial trust anchor(s)? need only be a sysctl ? not bullet proof, but a bit harder to spoof than kenv?

The pre-loaded kernel module may be a solid alternative. But how "unsafe" could it be? Assuming hat the kernel module is verified by mac_veriexec, and that writes to the kernel memory are reserved to processes that are tagged with "trusted" wrt veriexec? Am I missing something?

Also, the kenv approach is the one implemented in FreeBSD vanilla. I honestly don't know how it could work without it. As the kernel doesn't know how to verify a manifest file, the loader does the job and writes the manifest hash to a kenv, right before execv-ing the kernel. The kernel reads this kenv only once, attackers can modify it later with no effect on the kernel.

[sjg]

[I keep forgetting you cannot reply to the emails from this tool ]

The pre-loaded kernel module may be a solid alternative. But how "unsafe" could it be? Assuming hat the kernel module is verified by mac_veriexec, and that writes to the kernel memory are reserved to processes that are tagged with "trusted" wrt veriexec? Am I missing something?

At least with the kernel module I think you have a shot at confirming it was loaded by the loader? eg if it appears in kldstat before any module you know is loaded by rc.d
whereas anyone with root privs can set a variable in kenv at any time.

I have the loader limit what it will accept in an unverified .conf file etc, but again once booted we cannot necessarily prove the loader we trust was used.
Unless you have a TPM and are doing measured boot - far more complicated.
I don't attempt to make the loader talk to a TPM, but I do make it compute a pseudo PCR value - that it puts in kenv ;-)

kenv | grep pcr
loader.ve.pcr="308ee542cd904a2914ca3d63d556d2426fb42d62daf7cb8bf5ab0078b3f4b5e2"

Also, the kenv approach is the one implemented in FreeBSD vanilla. I honestly don't know how it could work without it. As the kernel doesn't know how to verify a manifest file, the loader does the job and writes the manifest hash to a kenv, right before execv-ing the kernel. The kernel reads this kenv only once, attackers can modify it later with no effect on the kernel.

Actually, what does the kernel do with the manifest? or the hash of the manifest?

[sebastien.bini_stormshield.eu]

Thank you for your input, I believe we will think it over.

In D33926#776527, @sjg wrote:

Also, the kenv approach is the one implemented in FreeBSD vanilla. I honestly don't know how it could work without it. As the kernel doesn't know how to verify a manifest file, the loader does the job and writes the manifest hash to a kenv, right before execv-ing the kernel. The kernel reads this kenv only once, attackers can modify it later with no effect on the kernel.

Actually, what does the kernel do with the manifest? or the hash of the manifest?

Once root is mounted, mac_veriexec verifies the manifest hash, and parses the manifest. Each file listed in the manifest gets resolved into a vnode and is added to mac_veriexec with its corresponding hash and flags ; the vnode can then later be verified when accessed. When the system is ready, mac_veriexec is already in "loaded active enforced" state. Without this mac_veriexec would not know of any file, so no program would be allowed to run, not even veriexec(8).

[sjg]

Actually, what does the kernel do with the manifest? or the hash of the manifest?

Once root is mounted, mac_veriexec verifies the manifest hash, and parses the manifest. Each file listed in the manifest gets resolved into a vnode and is added to mac_veriexec with its corresponding hash and flags ; the vnode can then later be verified when accessed. When the system is ready, mac_veriexec is already in "loaded active enforced" state. Without this mac_veriexec would not know of any file, so no program would be allowed to run, not even veriexec(8).

Currently mac_veriexec does not know anything about manifest parsing - that is the job of sbin/veriexec

I think it would be less disruption to the existing model for the loader to provide a hash of the manifest - that gets fed to mac_veriexec in the same manner that veriexec does.
Then set the state to loaded,active and then veriexec can use O_VERIFY to open that manifest and feed its content to mac_veriexec

OR you could just do all that via a kernel module loaded by loader, that effectively makes calls to initialize mac_veriexec - that would minimize the impact to existing usage, since anyone who doesn't want this need not load that module.

[sjg]

[not sure if editing a comment worked, so repeating it here]

I think it would be less disruption to the existing model for the loader to provide a hash of the manifest - that gets fed to mac_veriexec in the same manner that veriexec does.
Then set the state to loaded,active and then veriexec can use O_VERIFY to open that manifest and feed its content to mac_veriexec, then load more manifests and set state to enforced

OR you could just do all that via a preloaded kernel module that effectively makes calls to initialize mac_veriexec - that would minimize the impact to existing usage, since anyone who doesn't want this need not load that module.

[kd]

In D33926#776749, @sjg wrote:

Actually, what does the kernel do with the manifest? or the hash of the manifest?

Once root is mounted, mac_veriexec verifies the manifest hash, and parses the manifest. Each file listed in the manifest gets resolved into a vnode and is added to mac_veriexec with its corresponding hash and flags ; the vnode can then later be verified when accessed. When the system is ready, mac_veriexec is already in "loaded active enforced" state. Without this mac_veriexec would not know of any file, so no program would be allowed to run, not even veriexec(8).

Currently mac_veriexec does not know anything about manifest parsing - that is the job of sbin/veriexec

I think it would be less disruption to the existing model for the loader to provide a hash of the manifest - that gets fed to mac_veriexec in the same manner that veriexec does.
Then set the state to loaded,active and then veriexec can use O_VERIFY to open that manifest and feed its content to mac_veriexec

OR you could just do all that via a kernel module loaded by loader, that effectively makes calls to initialize mac_veriexec - that would minimize the impact to existing usage, since anyone who doesn't want this need not load that module.

I've implemented mainfest parsing in mac_veriexec_parser a while ago.
As it was said earlier the manifests hash is provided by the loader through a kenv.
The module registers itself for the mountroot event, so that it's run just before init is executed and after the rootfs is mounted.

How about something like this.
Veriexec(4) could be expanded with ioctl(s) to provide trust anchors to veriexec(8).
The certificates themselves will be stored in mac_veriexec(9). The module will have to be extended with some new get/set API for that.
They would be loaded from separate kernel modules. Apart from having them hardcoded we could do something more fancy, e.g. load the from firmware using UEFI protocol.
Any thoughts on that?

[sjg]

[Firstly I'd much rather have this discussion over email - which is more suited to it]

I think it would be less disruption to the existing model for the loader to provide a hash of the manifest - that gets fed to mac_veriexec in the same manner that veriexec does.
Then set the state to loaded,active and then veriexec can use O_VERIFY to open that manifest and feed its content to mac_veriexec

OR you could just do all that via a kernel module loaded by loader, that effectively makes calls to initialize mac_veriexec - that would minimize the impact to existing usage, since anyone who doesn't want this need not load that module.

I've implemented mainfest parsing in mac_veriexec_parser a while ago.
As it was said earlier the manifests hash is provided by the loader through a kenv.
The module registers itself for the mountroot event, so that it's run just before init is executed and after the rootfs is mounted.

How about something like this.
Veriexec(4) could be expanded with ioctl(s) to provide trust anchors to veriexec(8).
The certificates themselves will be stored in mac_veriexec(9). The module will have to be extended with some new get/set API for that.
They would be loaded from separate kernel modules. Apart from having them hardcoded we could do something more fancy, e.g. load the from firmware using UEFI protocol.
Any thoughts on that?

I'm not sure that mac_veriexec needs to be touched to do these things, a separate trust_anchor module (or family of such modules) might be better than adding complexity to mac_veriexec. eg. we have a separate implementation of veriexec(8) which allows for customer supplied trust anchors - but the usage is limited to later in boot - only the built in trust anchors can be used for initializing the system and verifying the core os bits, this all works wonderfully for us and we want to keep it that way, but is obviously not idea for something enabling in a FreeBSD release, so alternate solutions to the trust anchor storage are needed - but they should not compromise the existing functionality.