Verified execution (veriexec) as a MAC module.

[stevek]

Summary

MAC/veriexec implements a verified execution environment using the MAC
framework.

The code is organized into a few distinct pieces:

1. The meta-data store (in veriexec_metadata.c) which maps a file system identifier, file identifier, and generation key tuple to veriexec meta-data record.

2. Fingerprint management (in veriexec_fingerprint.c) which deals with calculating the cryptographic hash for a file and verifying it. It also manages the loadable fingerprint modules.

3. MAC policy implementation (in mac_veriexec.c) which implements the following MAC methods:
   • mpo_init
     • Initializes the veriexec state, meta-data store, fingerprint modules, and registers mount and unmount EVENTHANDLERs
   • mpo_syscall
     • Implements the following per-policy system calls:
     • MAC_VERIEXEC_CHECK_FD_SYSCALL
       • Check a file descriptor to see if the referenced file has a valid fingerprint.
     • MAC_VERIEXEC_CHECK_PATH_SYSCALL
       • Check a path to see if the referenced file has a valid fingerprint.
   • mpo_kld_check_load
     • Check if loading a kld is allowed. This checks if the referenced vnode has a valid fingerprint.
   • mpo_mount_destroy_label
     • Clears the veriexec slot data in a mount point label.
   • mpo_mount_init_label
     • Initializes the veriexec slot data in a mount point label.
     • The file system identifier is saved in the veriexec slot data.
   • mpo_priv_check
     • Check if a process is allowed to write to /dev/kmem and /dev/mem devices.
     • If a process is flagged as trusted, it is allowed to write.
   • mpo_proc_check_debug
     • Check if a process is allowed to be debugged. If a process is not flagged with VERIEXEC_NOTRACE, then debugging is allowed.
   • mpo_vnode_check_exec
     • Check is an exectuable is allowed to run. If veriexec is not enforcing or the executable has a valid fingerprint, then it is allowed to run. NOTE: veriexec will complain about mismatched fingerprints if it is active, regardless of the state of the enforcement.
   • mpo_vnode_check_open
     • Check is a file is allowed to be opened. If verification was not requested, veriexec is not enforcing, or the file has a valid fingerprint, then veriexec will allow the file to be opened.
   • mpo_vnode_copy_label
     • Copies the veriexec slot data from one label to another.
   • mpo_vnode_destroy_label
     • Clears the veriexec slot data in a vnode label.
   • mpo_vnode_init_label
     • Initializes the veriexec slot data in a vnode label.
     • The fingerprint status for the file is stored in the veriexec slot data.
       
       Some sysctls, under security.mac.veriexec, for setting debug level, fetching the current state in a human-readable form, and dumping the fingerprint database are implemented in this source file.
       
       The MAC policy implementation source file also contains some utility functions.

4. A set of fingerprint modules for the following cryptographic hash algorithms:
   • RIPEMD-160
   • SHA1
   • SHA2-256
   • SHA2-384
   • SHA2-512

5. Loadable module builds for MAC/veriexec and fingerprint modules.

WARNING: Using veriexec with NFS (or other network-based) file systems is not recommended as one cannot guarantee the integrity of the files served, nor the uniqueness of file system identifiers which are used as key in the meta-data store.

Depends on D2902

Test Plan

Various versions of this code (with some differences) has been in use for a few years and has gone through in-house testing.

[stevek]

Remove unnecessary copyright in the mac_veriexec module Makefile.

[stevek]

Remove KLD_ON_SECURELEVEL, as it is not necessary.

[• ian]

Note the comments I've added here are a couple simple things I noticed when skimming the code to get familiar with it. I.e., this was not an in-depth review with any understanding of the code.

sys/security/mac_veriexec/mac_veriexec.c

348 ↗

(On Diff #22308)

I don't see anything nearby checking securelevel/

564 ↗

(On Diff #22308)

= 0

before committing?

sys/security/mac_veriexec/veriexec_metadata.c

388 ↗

(On Diff #22308)

M_TEMP should be M_VERIEXEC?

661 ↗

(On Diff #22308)

M_VERIEXEC

[sjg]

sys/security/mac_veriexec/mac_veriexec.c

348 ↗

(On Diff #22308)

Yes, that comment is probably a carryover from original implementation.
In mac_veriexec we do not touch securelevel

[jtl]

It seems like this could use a man page to describe the mechanism. There are some subtleties that are not immediately obvious, such as the way that shared libraries are protected. In addition, the O_VERIFY flag should probably be documented in the open() man page with a pointer to the verified exec man page.

Also, I didn't see in any of the three reviews the mechanism for automatically loading manifests at boot (and enabling/disabling the protection at boot). I also didn't see the Makefile hooks to automatically create manifests. Is it your intention to include that glue with these reviews? Or, is that left as an exercise for the implementer, given the variety of custom packaging systems?

[jtl]

I think one of the few weaknesses I see is the way the hash result is cached.

Have you considered mitigations, such as:

• not caching hash results for remote volumes (e.g. NFS)?
• invalidating the cached results if someone opens the underlying block device using dd?

Overall, I am in favor of this set of patches going in...

[stevek]

In D8554#289476, @jtl wrote:

It seems like this could use a man page to describe the mechanism. There are some subtleties that are not immediately obvious, such as the way that shared libraries are protected. In addition, the O_VERIFY flag should probably be documented in the open() man page with a pointer to the verified exec man page.

Yes, that was the intention when I first committed the O_VERIFY flag and is mentioned in D2902

Also, I didn't see in any of the three reviews the mechanism for automatically loading manifests at boot (and enabling/disabling the protection at boot). I also didn't see the Makefile hooks to automatically create manifests. Is it your intention to include that glue with these reviews? Or, is that left as an exercise for the implementer, given the variety of custom packaging systems?

No, it is not intended to include that glue with these reviews.

Simon and I had some discussions with Robert some time back about this and some other things. What we use at Juniper would not necessarily be directly applicable, given the need for signing keys and managing those.
With the work that Simon has been doing on the verified loader, it's probably time to start the discussion anew and decide what to do about generating, loading, and managing fingerprint manifests.

[stevek]

In D8554#289494, @jtl wrote:

I think one of the few weaknesses I see is the way the hash result is cached.

Have you considered mitigations, such as:

• not caching hash results for remote volumes (e.g. NFS)?

Yes, but even then, there's potentially inconsistencies around determine what is a "remote volume" and being able to identify such.
Note even with NFS, there isn't necessarily a consistent file identifier and mapping the fingerprint to a file becomes problematic.

• invalidating the cached results if someone opens the underlying block device using dd?

Overall, I am in favor of this set of patches going in...

Note I have some updates that I have been working on to handle the meta-data store better in SMP environments. So there will be updates to these reviews, hopefully in the near future, time permitting.

[jtl]

After addressing @ian's commands, I think this should be committed "as is", unless someone raises serious objections in the next week or so.

This has years of "soak time". I've also personally stared at this code for many hours over many years.

I think this does need a man page. And, we'll need to build some infrastructure to make it easy to create and load manifests in the base system. But, that can come later. In some ways, having the code committed will make it easier for others (outside Juniper) to assist with those pieces that are not specific to Juniper's needs/use cases.