jail(8): cleanup jail when not persisting

[rew]

Summary

When a jail is not persisting and exits with success, the jail
is not cleaned up properly.

This was causing a couple problems:

1. If the jail environment was configured with any mount.* parameters, those mounts wouldn't be unmounted upon jail exit.

2. The jail would continue to persist, despite having nopersist set.

To fix this, set JF_STOP and requeue the jail. If there are any jails
that depend on the exiting jail, those will be shut down first.

PR: 222951

[jamie]

This presupposes that a jail that isn't marked to persist isn't going to stick around for some other reason. A jail could be started for example to start a long-running daemon, or to be a parent of other jails. Automatically going away when its task is done is a feature of such a jail.

For example:
jail -cv name=foo children.max=1 command=jail -cv name=bar persist

foo.bar is persistent, so foo will stick around as well, even though it isn't explicitly marked persistent. Now if might generally be better style to then mark it persistent, but that means that foo wouldn't be able to clean itself up when its job is done but must depend on somehow signalling the base system or being left as an empty shell. I think there's a use for such medium-term jails apart from my contrived example.

If there were a way to differentiate such useful jails from the ones that do a single quick task and exit, and also require some cleanup such as devfs and IP aliases, it would make sense to fully stop a jail that finished on its own. But I can't think of a good automated test. We'd want to know that a jail has indeed finished running, i.e. it no longer exists, but then we'd need some kind of timeout before we (arbitrarily) decide that this jail will be sticking around.

Also, back in the old days of jail(2), when persistence wasn't a thing, a jail's lifetime was always decided by the life of its processes. And they were generally created to run /etc/rc which would then exit successfully, but with some daemon processes listening on whatever ports. I suspect that many jail.conf setups are a version of these, and would get broken by this change.

[zlei]

What if usr.sbin/jail process exit before a non-persistent jail exit ?

[rew]

In D32934#743584, @jamie wrote:

This presupposes that a jail that isn't marked to persist isn't going to stick around for some other reason. A jail could be started for example to start a long-running daemon, or to be a parent of other jails. Automatically going away when its task is done is a feature of such a jail.

Got it.

For example:
jail -cv name=foo children.max=1 command=jail -cv name=bar persist

foo.bar is persistent, so foo will stick around as well, even though it isn't explicitly marked persistent. Now if might generally be better style to then mark it persistent, but that means that foo wouldn't be able to clean itself up when its job is done but must depend on somehow signalling the base system or being left as an empty shell.

The example illustrates that point very well - appreciate that.

If there were a way to differentiate such useful jails from the ones that do a single quick task and exit, and also require some cleanup such as devfs and IP aliases, it would make sense to fully stop a jail that finished on its own.

Interesting, yea - after digesting jails for another day, it doesn't seem like there's a clear-cut way to make that differentiation. I'm not suggesting this, but you'd almost want a daemon that gets notified when a jail dies which would then cleanup devfs and IP aliases - not sure if that would even make sense though.

Also, back in the old days of jail(2), when persistence wasn't a thing, a jail's lifetime was always decided by the life of its processes.

The old days...2009 by the looks of it (your commit): Introduce the extensible jail framework.

Thanks for the feedback. I'll give this problem some more thought.