This is a lot of change to digest in a single differential. I don't want to recommend "using LLMs" for reviewing -- it's a slippery slope for all involved as LLMs can be confident bullshitters/idiots.

Is there any way this work could please be broken down into a diff stack?

In D56835#1320727, @ngie wrote:

Please add [explicit? implicit?] versioning information to the schema and UCL files -- otherwise you risk problems when upgrading/downgrading schema/unit files.

for versionning I was thinking like in pkg to never break backward compatibility, by always adding new keys and never mutate existing one, this also solves forward compatibility because rcd will not validate a unit made in the new format.

In D56835#1320563, @bapt wrote:

In D56835#1320536, @otis wrote:

And, when invoked as /sbin/rcd on a live system, I'm getting:

1700  -  Is    0:00.02 |-- rcd -v
2599  -  Z     0:00.00 | |-- <defunct>
2600  -  Z     0:00.00 | |-- <defunct>
2601  -  Z     0:00.00 | |-- <defunct>
2602  -  Z     0:00.00 | |-- <defunct>
2603  -  Z     0:00.00 | |-- <defunct>
3180  -  Z     0:00.00 | |-- <defunct>
3182  -  Z     0:00.00 | |-- <defunct>
3185  -  Z     0:00.00 | |-- <defunct>
3228  -  I     0:00.00 | |-- rcd-exec: reaper: syslogd (pid 3378) (rcd-exec)
3371  -  SCs   0:00.00 | | |-- /usr/sbin/syslogd -s
3376  -  I     0:00.00 | | |-- syslogd: syslogd.casper (syslogd)
3378  -  Is    0:00.00 | | `-- syslogd: system.net (syslogd)
3359  -  Z     0:00.00 | |-- <defunct>
3361  -  Z     0:00.00 | |-- <defunct>
3362  -  Z     0:00.00 | |-- <defunct>
3364  -  Z     0:00.00 | |-- <defunct>
3365  -  Z     0:00.00 | |-- <defunct>
3368  -  Z     0:00.00 | |-- <defunct>
3536  -  I     0:00.00 | `-- rcd-exec: reaper: cron (pid 3715) (rcd-exec)
3715  -  Ss    0:00.00 |   `-- /usr/sbin/cron -s

too many defuct processes.

fixed

Still zombies:

1627  -  Z      0:00.00 | |-- <defunct>
1631  -  Z      0:00.00 | |-- <defunct>
1634  -  Z      0:00.00 | |-- <defunct>
1835  -  Z      0:00.00 | |-- <defunct>
1838  -  Z      0:00.00 | |-- <defunct>
1840  -  Z      0:00.00 | |-- <defunct>

otis@b14:~ % sudo procstat 1627
  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL          COMM
 1627    19    19    19     0   1 root     -         FreeBSD ELF64 logger
otis@b14:~ % sudo procstat 1631
  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL          COMM
 1631    19    19    19     0   1 root     -         FreeBSD ELF64 logger
otis@b14:~ % sudo procstat 1634
  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL          COMM
 1634    19  1634  1634     0   1 root     -         FreeBSD ELF64 logger
otis@b14:~ % sudo procstat 1835
  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL          COMM
 1835    19    19    19     0   1 root     -         FreeBSD ELF64 savecore
otis@b14:~ % sudo procstat 1838
  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL          COMM
 1838    19    19    19     0   1 root     -         FreeBSD ELF64 savecore
otis@b14:~ % sudo procstat 1840
  PID  PPID  PGID   SID  TSID THR LOGIN    WCHAN     EMUL          COMM
 1840    19    19    19     0   1 root     -         FreeBSD ELF64 savecore

In D56835#1311164, @imp wrote:

I'd like to echo the user requirement. I have not current need for it at $WORK, but can see a need in the future. Having the ability for users to tie into the startup, running as themselves, much like we allow users to add jobs to cron, at, etc would be a useful, but not gating, feature.

I believe that user units is actually an orthogonal feature of the system service manager. If we are to implement user units, IMO, it should be a cross-platform project that consumes pristine systemd unit descriptions and hooks into the OS via PAM. It doesn't even has to be a part of base, because only uses of user units I stumbled upon so far were coming from ports.

I'm working on a systemd-logind replacement, which will fill the user units hole nicely, but there is a ton of work to do.

In D56835#1320939, @arrowd wrote:

In D56835#1311164, @imp wrote:

I'd like to echo the user requirement. I have not current need for it at $WORK, but can see a need in the future. Having the ability for users to tie into the startup, running as themselves, much like we allow users to add jobs to cron, at, etc would be a useful, but not gating, feature.

I believe that user units is actually an orthogonal feature of the system service manager. If we are to implement user units, IMO, it should be a cross-platform project that consumes pristine systemd unit descriptions and hooks into the OS via PAM. It doesn't even has to be a part of base, because only uses of user units I stumbled upon so far were coming from ports.

I'm working on a systemd-logind replacement, which will fill the user units hole nicely, but there is a ton of work to do.

we are interested in 2 kind of things here, user units which are tight to "sessions" covert your use case and I don't plan into adding them (they fall into the logindish thing you are working on). but there are plenty of other non desktop user units which might be useful and those I am planning on working on after rcd hits the tree and yes there is a tiny overlap between those 2.

In D56835#1320957, @bapt wrote:

reap hard; this time I hope it should be fine for your case @otis

no, it did not.

excerpt:

# /sbin/rcd -v
rcd starting
rcd: daemonized (pid 22)
WARNING: control bindat: Read-only file system
WARNING: control socket init failed
started dumpon (pid 45, procdesc fd 5)
started sysctl (pid 46, procdesc fd 6)
started localpkg (pid 47, procdesc fd 7)
started zfs (pid 48, procdesc fd 8)
localpkg exited (status 256, pid 47)
sysctl exited (status 0, pid 46)
started hostid (pid 153, procdesc fd 6)
dumpon exited (status 0, pid 45)
started disks (pid 159, procdesc fd 5)
hostid exited (status 0, pid 153)
disks exited (status 0, pid 159)
started swap (pid 207, procdesc fd 5)
swap exited (status 0, pid 207)
started fsck (pid 232, procdesc fd 5)
fsck exited (status 0, pid 232)
started root (pid 256, procdesc fd 5)
root exited (status 0, pid 256)
started mdconfig (pid 283, procdesc fd 5)
started serial (pid 284, procdesc fd 6)
serial exited (status 0, pid 284)
zfs exited (status 0, pid 48)
mdconfig exited (status 256, pid 283)
started mountcritlocal (pid 312, procdesc fd 5)
mountcritlocal exited (status 0, pid 312)
started kldxref (pid 338, procdesc fd 5)
started var_run (pid 339, procdesc fd 6)
started var (pid 340, procdesc fd 7)
started tmp (pid 341, procdesc fd 8)
var_run exited (status 0, pid 339)
var exited (status 0, pid 340)

the sources are positively current. I can provide an access to that system, to help debugging.

In D56835#1320998, @bapt wrote:

more work on zombies
show resources for legacy scripts

This works for me. thanks! No more zombies.

Will it help sshd to start without being indefinitely blocked by e.g. mysqld? That would be enough of a reason for me personally to switch to this thing immediately.

In D56835#1321082, @vsasjason_gmail.com wrote:

Will it help sshd to start without being indefinitely blocked by e.g. mysqld? That would be enough of a reason for me personally to switch to this thing immediately.

Can you elaborate on this?

In D56835#1321115, @michaelo wrote:

In D56835#1321082, @vsasjason_gmail.com wrote:

Will it help sshd to start without being indefinitely blocked by e.g. mysqld? That would be enough of a reason for me personally to switch to this thing immediately.

Can you elaborate on this?

Sure:

$ rcorder /etc/rc.d/* /usr/local/etc/rc.d/* | egrep -n "sshd|usr"
29:/usr/local/etc/rc.d/microcode_update
108:/usr/local/etc/rc.d/git_daemon
114:/usr/local/etc/rc.d/xrdp
115:/usr/local/etc/rc.d/transmission
116:/usr/local/etc/rc.d/syncthing-relaysrv
117:/usr/local/etc/rc.d/syncthing-relaypoolsrv
118:/usr/local/etc/rc.d/syncthing-discosrv
119:/usr/local/etc/rc.d/syncthing
120:/usr/local/etc/rc.d/smartd
121:/usr/local/etc/rc.d/poudriered
122:/usr/local/etc/rc.d/miniupnpc
123:/usr/local/etc/rc.d/dbus
131:/etc/rc.d/sshd
133:/usr/local/etc/rc.d/lightdm
<CUT>

So whatever <131 potentially hangs, I've got no ssh into machine to fix/kill it as rc is sequential and depends are uncontrollable (because every port may install rc script and put it wherever maintainer decided). I, personally, suffered of this issue before; and the issue is probably as old as rc itself.

In D56835#1321082, @vsasjason_gmail.com wrote:

Will it help sshd to start without being indefinitely blocked by e.g. mysqld? That would be enough of a reason for me personally to switch to this thing immediately.

yes