busybox/examples/var_service/README_distro_proposal.txt
<<
>>
Prefs
   1        A distro which already uses runit
   2
   3I installed Void Linux, in order to see what do they have.
   4Xfce desktop looks fairly okay, network is up.
   5ps tells me they did put X, dbus, NM and udev into runsvdir-supervised tree:
   6
   7    1 ?        00:00:01 runit
   8  623 ?        00:00:00   runsvdir
   9  629 ?        00:00:00     runsv
  10  650 tty1     00:00:00       agetty
  11  630 ?        00:00:00     runsv
  12  644 ?        00:00:09       NetworkManager
  13 1737 ?        00:00:00         dhclient
  14  631 ?        00:00:00     runsv
  15  639 tty4     00:00:00       agetty
  16  632 ?        00:00:00     runsv
  17  640 ?        00:00:00       sshd
  18 1804 ?        00:00:00         sshd
  19 1809 pts/3    00:00:00           sh
  20 1818 pts/3    00:00:00             ps
  21  633 ?        00:00:00     runsv
  22  637 tty5     00:00:00       agetty
  23  634 ?        00:00:00     runsv
  24  796 ?        00:00:00       dhclient
  25  635 ?        00:00:00     runsv
  26  649 ?        00:00:00       uuidd
  27  636 ?        00:00:00     runsv
  28  647 ?        00:00:00       acpid
  29  638 ?        00:00:00     runsv
  30  652 ?        00:00:00       console-kit-dae
  31  641 ?        00:00:00     runsv
  32  651 tty6     00:00:00       agetty
  33  642 ?        00:00:00     runsv
  34  660 tty2     00:00:00       agetty
  35  643 ?        00:00:00     runsv
  36  657 ?        00:00:02       dbus-daemon
  37  645 ?        00:00:00     runsv
  38  658 ?        00:00:00       cgmanager
  39  648 ?        00:00:00     runsv
  40  656 tty3     00:00:00       agetty
  41  653 ?        00:00:00     runsv
  42  655 ?        00:00:00       lxdm-binary
  43  698 tty7     00:00:14         Xorg
  44  729 ?        00:00:00         lxdm-session
  45  956 ?        00:00:00           sh
  46  982 ?        00:00:00             xfce4-session
  47 1006 ?        00:00:04               nm-applet
  48  654 ?        00:00:00     runsv
  49  659 ?        00:00:00       udevd
  50
  51Here is a link to Void Linux's wiki:
  52
  53    https://wiki.voidlinux.eu/Runit
  54
  55Void Linux packages install their services as subdirectories of /etc/rc,
  56such as /etc/sv/sshd, with a script file, "run", and a link
  57"supervise" -> /run/runit/supervise.sshd
  58
  59For sshd, "run" contains:
  60
  61    #!/bin/sh
  62    ssh-keygen -A >/dev/null 2>&1 # generate host keys if they don't exist
  63    [ -r conf ] && . ./conf
  64    exec /usr/bin/sshd -D $OPTS
  65
  66That's it from the POV of the packager.
  67
  68This is pretty minimalistic, and yet, it is already distro-specific:
  69the link to /run/runit/* is conceptually wrong, it requires packagers
  70to know that /etc/rc should not be mutable and thus they need to use
  71a different location in filesystem for supervise/ directory.
  72
  73I think a good thing would be to require just one file: the "run" script.
  74The rest should be handled by distro tooling, not by packager.
  75
  76A similar issue is arising with logging. It would be ideal if packagers
  77would not need to know how a particular distro manages logs.
  78Whatever their daemons print to stdout/stderr, should be automagically logged
  79in a way distro prefers.
  80
  81* * * * * * * *
  82
  83        Proposed "standard" on how distros should use runit
  84
  85The original idea of services-as-directories belongs to D.J.Bernstein (djb),
  86and his project to implement it is daemontools: https://cr.yp.to/daemontools.html
  87
  88There are several reimplementations of daemontools:
  89- runit: by Gerrit Pape, http://smarden.org/runit/
  90  (busybox has it included)
  91- s6: by Laurent Bercot, http://skarnet.org/software/s6/
  92
  93It is not required that a specific clone should be used. Let evolution work.
  94
  95
  96        Terminology
  97
  98daemon: any long running background program. Common examples are sshd, getty,
  99ntpd, dhcp client...
 100
 101service: daemon controlled by a service monitor.
 102
 103service directory: a directory with an executable file (script) named "run"
 104which (usually) execs some daemon, possibly after some preparatory steps.
 105It should start it not as a child or daemonized process, but by exec'ing it
 106(inheriting the same PID and the place in the process tree).
 107
 108service monitor: a tool which watches a set of service directories.
 109In daemontools package, it is called "svscan". In runit, it is called
 110"runsvdir". In s6, it is called "s6-svscan".
 111Service monitor starts a supervisor for each service directory.
 112If it dies, it restarts it. If service directory disappears,
 113service monitor will not be restarted if it dies.
 114runit's service monitor (runsvdir) sends SIGTERM to supervisors
 115whose directories disappeared.
 116
 117supervisor: a tool which monitors one service directory.
 118It runs "run" script as its child. It restarts it if it dies.
 119It can be instructed to start/stop/signal its child.
 120In daemontools package, it is called "supervise". In runit, it is called
 121"runsv". In s6, it is called "s6-supervise".
 122
 123Conceptually, a daemontools clone can be designed such that it does not *have*
 124the supervisor component: service monitor can directly monitor all its daemons
 125(for example, this may be a good idea for memory-constrained systems).
 126However all three existing projects (daemontools/runit/s6) do have a per-service
 127supervisor process.
 128
 129log service: a service which is exclusively tasked with logging
 130the output of another service. It is implemented as log/ subdirectory
 131in a service directory. It has the same structure as "normal"
 132service dirs: it has a "run" script which starts a logging tool.
 133
 134If log service exists, stdout of its "main" service is piped
 135to log service. Stops/restarts of either of them do not sever the pipe
 136between them.
 137
 138If log service exists, daemontools and s6 run a pair of supervisors
 139(one for the daemon, one for the logger); runit runs only one supervisor
 140per service, which is handling both of them (presumably this is done
 141to use fewer processes and thus, fewer resources).
 142
 143
 144        User API
 145
 146"Users" of service monitoring are authors of software which has daemons.
 147They need to package their daemons to be installed as services at package
 148install time. And they need to do this for many distros.
 149The less distros diverge, the easier users' lives are.
 150
 151System-wide service dirs reside in a distro-specific location.
 152The recommended location is /var/service. (However, since it is not
 153a mandatory location, avoid depending on it in your run scripts.
 154Void Linux wanted to have it somewhere in /run/*, and they solved this
 155by making /var/service a symlink).
 156
 157The install location for service dirs is /etc/rc:
 158when e.g. ntpd daemon is installed, it creates the /etc/rc/ntpd
 159directory with (minimally) one executable file (script) named "run"
 160which starts ntpd daemon. It can have other files there.
 161
 162At boot, distro should copy /etc/rc/* to a suitable writable
 163directory (common choice are /var/service, /run/service etc).
 164It should create log/ directories in each subdirectory
 165and create "run" files in them with suitable (for this particular distro)
 166logging tool invocation, unless this directory chose to channel
 167all logging from all daemons through service monitor process
 168and log all of them into one file/database/whatever,
 169in which case log/ directories should not be created.
 170
 171It is allowable for a distro to directly use /etc/rc/ as the only
 172location of its service directories. (For example,
 173/var/service may be a symlink to /etc/rc).
 174However, it poses some problems:
 175
 176(1) Supervision tools will need to write to subdirectories:
 177the control of running daemons is implemented via some files and fifos
 178in automatically created supervise/ subdirectory in each /etc/rc/DIR.
 179
 180(2) Creation of a new service can race with the rescanning of /etc/rc/
 181by service monitor: service monitor may see a directory with only some files
 182present. If it attempts to start the service in this state, all sorts
 183of bad things may happen. This may be worked around by various
 184heuristics in service monitor which give new service a few seconds
 185of "grace time" to be fully populated; but this is not yet
 186implemented in any of three packages.
 187This also may be worked around by creating a .dotdir (a directory
 188whose name starts with a dot), populating it, and then renaming;
 189but packaging tools usually do not have an option to do this
 190automatically - additional install scripting in packages will be needed.
 191
 192Daemons' output file descriptors are handled somewhat awkwardly
 193by various daemontools implementations. For example, for runit tools,
 194daemons' stdout goes to wherever runsvdir's stdout was directed;
 195stderr goes to runsvdir, which in turn "rotates" it on its command line
 196(which is visible in ps output).
 197
 198Hopefully this get changed/standardized; while it is not, the "run" file
 199should start with a
 200
 201    exec 2>&1
 202
 203command, making stderr equivalent to stdout.
 204An especially primitive service which does not want its output to be logged
 205with standard tools can do
 206
 207    exec >LOGFILE 2>&1
 208
 209or even
 210
 211    exec >/dev/null 2>&1
 212
 213To prevent creation of distro-specific log/ directory, a service directory
 214in /etc/rc can contain an empty "log" file.
 215
 216
 217        Controlling daemons
 218
 219The "svc" tool is available for admins and scripts to control services.
 220In particular, often one service needs to control another:
 221e.g. ifplugd can detect that the network cable was just plugged in,
 222and it needs to (re)start DHCP service for this network device.
 223
 224The name of this tool is not standard either, which is an obvious problem.
 225I propose to fix this by implementing a tool with fixed name and API by all
 226daemontools clones. Lets use original daemontools name and API. Thus:
 227
 228The following form must work:
 229
 230        svc -udopchaitkx DIR
 231
 232Options map to up/down/once/STOP/CONT/HUP/ALRM/INT/TERM/KILL/exit
 233commands to the daemon being controlled.
 234
 235The form with one option letter must work. If multiple-option form
 236is supported, there is no guarantee in which order they take effect:
 237svc -it DIR can deliver TERM and INT in any order.
 238
 239If more than one DIR can be specified (which is not a requirement),
 240there is no guarantee in which order commands are sent to them.
 241
 242If DIR has no slash and is not "." or "..", it is assumed to be
 243relative to the system-wide service directory.
 244
 245[Currently, "svc" exists only in daemontools and in busybox.
 246This proposal asks developers of other daemontools implementations
 247to add "svc" command to their projects]
 248
 249The "svok DIR" tool exits 0 if service supervisor is running
 250(with service itself either running or stopped), and nonzero if not.
 251
 252Other tools with different names and APIs may exist; however
 253for portability scripts should use the above tools.
 254
 255Creation of a new service on a running system should be done atomically.
 256To this end, first create and populate a new /etc/rc/DIR.
 257
 258Then "activate" it by running ??????? - this copies (or symlinks,
 259depending on the distro) its files to the "live" service directory,
 260wherever it is located on this distro.
 261
 262Removal of the service should be done as follows:
 263svc -d DIR [DIR/log], then remove the service directory:
 264this makes service monitor SIGTERM per-directory supervisors
 265(if they exist in the implementation).
 266
 267
 268        Implementation details
 269
 270Top-level service monitor program name is not standardized
 271[svscan, runsvdir, s6-svscan ...] - it does not need to be,
 272as far as daemon packagers are concerned.
 273
 274It may run one per-directory supervisor, or two supervisors
 275(one for DIR/ and one for DIR/log/); for memory-constrained systems
 276an implementation is possible which itself controls all services, without
 277intermediate supervisors.
 278[runsvdir runs one "runsv DIR" per DIR, runsv handles DIR/log/ if that exists]
 279[svscan runs a pair of "supervise DIR" and "supervise DIR/log"]
 280
 281Directories are remembered by device+inode numbers, not names. Renaming a directory
 282does not affect the running service (unless it is renamed to a .dotdir).
 283
 284Removal (or .dotdiring) of a directory sends SIGTERM to any running services.
 285
 286Standard output of non-logged services goes to standard output of service monitor.
 287Standard output of logger services goes to standard output of service monitor.
 288Standard error of them always goes to standard error of service monitor.
 289
 290If you want to log standard error of your logged service along with its stdout, use
 291"exec 2>&1" in the beginning of your "run" script.
 292
 293Whether stdout/stderr of service monitor is discarded (>/dev/null)
 294or logged in some way is system-dependent.
 295
 296
 297        Containers
 298
 299[What do containers need?]
 300