jail: Add meta parameter

[igoro]

Summary

This is any string associated with a jail. It can be set upon jail creation
or added/modified later:

    # jail -cm ... meta="tag1=value1 tag2=value2"

Its value is not inherited from a parent jail. A jail can read its own
meta via security.jail.meta sysctl.

The maximum meta size per jail is controlled with a global
security.jail.meta_maxbufsize sysctl. Decreasing it does not alter the
existing meta information.

MFC after:      3 months
Sponsored by:   SkunkWerks GmbH

[igoro]

Currently, there is no accessible way to attach metadata to jails. This is commonly used elsewhere, for example in Kubernetes, to allow non-unique properties to enrich load balancers, schedulers, and volume provisioners to make more informed decisions.

The initial motivation was to be able to have an external load balancer to update its pool of endpoints based on jail metadata - for example, as a web application is updated from v1 to v2, the load balancer would query the “www” tags, and be able to grow its pool until there are 3 new v2 jails, and then remove the older v1 versions.

While there are already other mechanisms to make metadata available inside the jail, these rely on shared memory, or files, and are not directly tied to the jail’s lifecycle.

The Object Specific Data approach is already used with jails, to provide the zfs-specific dataset names, and jailed on/off properties. This implementation follows the same approach allowing generic metadata, accessible from outside, inside, and via the existing jail and flua tools.

From the technical perspective, there are the following open topics:

• It seems that _security_jail sysctl node ref exposure could be extracted as a separate patch if confirmed.
• There is a question of whether the kernel should limit the set of allowed characters within the meta buffer, or if it's better to keep such policies outside of the kernel business.
• The current state of the patch allows reading meta by any user within a jail. Do we want to disable it by default and add something like allow.read_meta or allow.metadata parameter to control it per jail?

[crest_freebsd_rlwinm.de]

I can see the appeal of this to jail managers, but having just a single meta sysctl per jail implies that it has to be "owned" by a single writer. This requires everything wanting to make use of this new feature to be tightly coupled to the writer. A flat list of key=value pairs (instead of a single value) would allow multiple users of this feature per jail e.g. multiple helper commands preparing just one aspect of a jail e.g. dynamic devfs ruleset loading, network setup/teardown, storage provisioning.

[crest_freebsd_rlwinm.de]

If I understand the code correctly security.jail.meta_maxbufsize is the upper limit of the amount of additional kernel memory the jail can tie down with this feature. If a jail is allowed to create sub-jails should each of them be able to allocate metadata up to the global limit or should each of them get its own limit and the allocation be counted (recursively) against the parent limits?

[kevans]

Just tossing in a general +1, I haven't taken any time to review this at the moment; I've thought about a similar notion in the past (jail tags) to be able to do some role-based querying of jails.