1<html><head><title>toybox roadmap</title> 2<!--#include file="header.html" --> 3<title>Toybox Roadmap</title> 4 5<h2>Goals and use cases</h2> 6 7<p>We have several potential use cases for a new set of command line 8utilities, and are using those to determine which commands to implement 9for Toybox's 1.0 release.</p> 10 11<p>The most interesting standards are POSIX-2008 (also known as the Single 12Unix Specification version 4) and the Linux Standard Base (version 4.1). 13The main test harness including toybox in Aboriginal Linux and if that can 14build itself using the result to build Linux From Scratch (version 6.8). 15We also aim to replace Android's Toolbox.</p> 16 17<p>At a secondary level we'd like to meet other use cases. We've analyzed 18the commands provided by similar projects (klibc, sash, sbase, embutils, 19nash, and beastiebox), along with various vendor configurations of busybox, 20and some end user requests.</p> 21 22<p>Finally, we'd like to provide a good replacement for the Bash shell, 23which was the first program Linux ever ran and remains the standard shell 24of Linux no matter what Ubuntu says. This doesn't mean including the full 25set of Bash 4.x functionality, but does involve {various,features} beyond 26posix.</p> 27 28<p>See the <a href=status.html>status page</a> for the combined list 29and progress towards implementing it.</p> 30 31<ul> 32<li><a href=#susv4>POSIX-2008/SUSv4</a></li> 33<li><a href=#sigh>Linux "Standard" Base</a></li> 34<li><a href=#dev_env>Development Environment</a></li> 35<li><a href=#android>Android Toolbox</a></li> 36<li><a href=#tizen>Tizen Core</a></li> 37<li>Miscelaneous: <a href=#klibc>klibc</a>, <a href=#glibc>glibc</a>, 38<a href=#sash>sash</a>, <a href=#sbase>sbase</a>, 39<a href=#uclinux>uclinux</a>...</li> 40</ul> 41 42<hr /> 43<a name="standards"> 44<h2>Use case: standards compliance.</h2> 45 46<h3><a name=susv4 /><a href="#susv4">POSIX-2008/SUSv4</a></h3> 47<p>The best standards are the kind that describe reality, rather than 48attempting to impose a new one. (I.E. a good standard should document, not 49legislate.)</p> 50 51<p>The kind of standards which describe existing reality tend to be approved by 52more than one standards body, such ANSI and ISO both approving C. That's why 53the IEEE POSIX committee's 2008 standard, the Single Unix Specification version 544, and the Open Group Base Specification edition 7 are all the same standard 55from three sources.</p> 56 57<p>The <a href="http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html">"utilities" 58section</a> 59of these standards is devoted to the unix command line, and are the best such 60standard for our purposes. (My earlier work on BusyBox was implemented with 61regard to SUSv3, an earlier version of this standard.)</p> 62 63<h3>Problems with the standard</h3> 64 65<p>Unfortunately, these standards describe a subset of reality, lacking any 66mention of commands such as init, login, or mount required to actually boot a 67system. It provides ipcrm and ipcs, but not ipcmk, so you can use System V IPC 68resources but not create them.</p> 69 70<p>These standards also contain a large number of commands that are 71inappropriate for toybox to implement in its 1.0 release. (Perhaps some of 72these could be reintroduced in later releases, but not now.)</p> 73 74<p>Starting with the full "utilities" list, we first remove generally obsolete 75commands (compess ed ex pr uncompress uccp uustat uux), commands for the 76pre-CVS "SCCS" source control system (admin delta get prs rmdel sact sccs unget 77val what), fortran support (asa fort77), and batch processing support (batch 78qalter qdel qhold qmove qmsg qrerun qrls qselect qsig qstat qsub).</p> 79 80<p>Some commands are for a compiler toolchain (ar c99 cflow ctags cxref gencat 81iconv lex m4 make nm strings strip tsort yacc), which is outside of toybox's 82mandate and should be supplied externally. (Again, some of these may be 83revisited later, but not for toybox 1.0.)</p> 84 85<p>Some commands are part of a command shell, and cannot be implemented as 86separate executables (alias bg cd command fc fg getopts hash jobs kill read 87type ulimit umask unalias wait). These may be revisited as part of a built-in 88toybox shell, but are not exported into $PATH via symlinks. (If you fork a 89child process and have it "cd" then exit, you've accomplished nothing. 90This is not a complete list, a shell also needs exit, if, while, for, case, 91export, set, unset, trap, exec... And for bash compatability, function and 92source.)</p> 93 94<blockquote><b> 95<span id=shell> 96alias bg cd command fc fg getopts hash jobs kill read type ulimit umask 97unalias wait exit if while for case export set unset trap exec function source 98</span> 99</b></blockquote> 100 101<p>A few other commands are judgement calls, providing command-line 102internationalization support (iconv locale localedef), System V inter-process 103communication (ipcrm ipcs), and cross-tty communication from the minicomputer 104days (talk mesg write). The "pax" utility was supplanted by tar, "mailx" is 105a command line email client, and "lp" submits files for printing to... what 106exactly? (cups?) The standard defines crontab but not crond.</p> 107 108<p>Removing all of that leaves the following commands, which toybox should 109implement:</p> 110 111<blockquote><b> 112<span id=posix> 113at awk basename bc cal cat chgrp chmod chown cksum cmp comm cp 114csplit cut date dd df diff dirname du echo env expand expr false file find 115fold fuser getconf grep head id join kill link ln logger logname ls man 116mkdir mkfifo more mv newgrp nice nl nohup od paste patch pathchk printf ps 117pwd renice rm rmdir sed sh sleep sort split stty tabs tail tee test time 118touch tput tr true tty uname unexpand uniq unlink uudecode uuencode vi wc 119who xargs zcat 120</span> 121</b></blockquote> 122 123<h3><a name=sigh /><a href="#sigh">Linux Standard Base</a></h3> 124 125<p>One attempt to supplement POSIX towards an actual usable system was the 126Linux Standard Base. Unfortunately, the quality of this "standard" is 127fairly low.</p> 128 129<p>POSIX allowed its standards process to be compromised 130by leaving things out, thus allowing IBM mainframes and Windows NT to drive 131a truck through the holes and declare themselves compilant. But it means what 132they DID standardize tends to be respected (if sometimes obsolete).</p> 133 134<p>The Linux Standard Base's failure mode is different, they respond to 135pressure by including special-case crap, such as allowing Red Hat to shoehorn 136RPM into the standard even though all sorts of distros (Debian, Slackware, Arch, 137Gentoo) don't use it and probably never will. This means anything in the LSB is 138at best a suggestion: arbitrary portions of this standard are widely 139ignored.</p> 140 141<p>The community perception seems to be that the Linux Standard Base is 142the best standard money can buy, I.E. the Linux Foundation is supported by 143financial donations form large companies and the LSB represents the interests 144of those donors more than technical merit. Debian officially 145<a href=http://lwn.net/Articles/658809>washed its hands of LSB</a> when 5.0 146came out in 2015, and no longer even pretends to support it (which may affect 147Debian derivatives like Ubuntu and Knoppix). Toybox hasn't moved to 5.0 for 148similar reasons.</p> 149 150<p>That said, Posix by itself isn't enough, and this is the next most 151comprehensive standards effort for Linux so far.</p> 152 153<p>The LSB specifies a <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html>list of command line 154utilities</a>:</p> 155 156<blockquote><b> 157ar at awk batch bc chfn chsh col cpio crontab df dmesg du echo egrep 158fgrep file fuser gettext grep groupadd groupdel groupmod groups 159gunzip gzip hostname install install_initd ipcrm ipcs killall lpr ls 160lsb_release m4 md5sum mknod mktemp more mount msgfmt newgrp od passwd 161patch pidof remove_initd renice sed sendmail seq sh shutdown su sync 162tar umount useradd userdel usermod xargs zcat 163</b></blockquote> 164 165<p>Where posix specifies one of those commands, LSB's deltas tend to be 166accomodations for broken tool versions which aren't up to date with the 167standard yet. (See <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/more.html>more</a> and <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/xargs.html>xargs</a> 168for examples.)</p> 169 170<p>Since we've already committed to using our own judgement to skip bits of 171POSIX, and LSB's "judgement" in this regard is purely bug workarounds to declare 172various legacy tool implementations "compliant", this means we're mostly 173interested in the set of tools that aren't specified in posix at all.</p> 174 175<p>Of these, gettext and msgfmt are internationalization, install_initd and 176remove_initd aren't present on ubuntu 10.04, lpr is out of scope, and 177lsb_release is a distro issue (it's a nice command, but the output of 178lsb_release -a is the name and version number of the linux distro you're 179running, which toybox doesn't know).</p> 180 181<p>This leaves:</p> 182 183<blockquote><b> 184<span id=lsb> 185chfn chsh dmesg egrep fgrep groupadd groupdel groupmod groups 186gunzip gzip hostname install killall md5sum 187mknod mktemp mount passwd pidof sendmail seq shutdown 188su sync tar umount useradd userdel usermod zcat 189</span> 190</b></blockquote> 191 192<hr /> 193<a name="dev_env"> 194<h2><a href="#dev_env">Use case: provide a self-hosting development environment</a></h2> 195 196<p>The following commands are enough to build the <a href=http://landley.net/aboriginal/about.html>Aboriginal Linux</a> development 197environment, boot it to a shell prompt, and build <a href=http://www.linuxfromscratch.org/lfs/view/6.8/>Linux From Scratch 6.8</a> under 198it. (Aboriginal Linux <a href=http://landley.net/aboriginal/history.html>currently uses</a> BusyBox for this, thus providing a 199drop-in test environment for toybox. We install both implementations side 200by side, redirecting the symlinks a command at a time until the older 201package is no longer used, and can be removed.)</p> 202 203<p>This use case includes running init scripts and other shell scripts, running 204configure, make, and install in each package, and providing basic command line 205facilities such as a text editor. (It does not include a compiler toolchain or 206C library, those are outside the scope of this project.)</p> 207 208<blockquote><b> 209<span id=development> 210bzcat cat cp dirname echo env patch rmdir sha1sum sleep sort sync 211true uname wc which yes zcat 212awk basename chmod chown cmp cut date dd diff 213egrep expr fdisk find grep gzip head hostname id install ln ls 214mkdir mktemp mv od readlink rm sed sh tail tar touch tr uniq 215wget whoami xargs chgrp comm gunzip less logname split 216tee test time bunzip2 chgrp chroot comm cpio dmesg 217dnsdomainname ftpd ftpget ftpput gunzip ifconfig init less 218logname losetup mdev mount mountpoint nc pgrep pkill 219pwd route split stat switch_root tac umount vi 220resize2fs tune2fs fsck.ext2 genext2fs mke2fs xzcat 221</span> 222</b></blockquote> 223 224<p>Note: Aboriginal Linux installs bash 2.05b as #!/bin/sh and its scripts 225require bash extensions not present in shells such as busybox ash. 226This means that toysh needs to supply several bash extensions _and_ work 227when called under the name "bash".</p> 228 229<p>The <a href=http://landley.net/aboriginal>Aboriginal Linux</a> 230self-bootstrapping build still uses the following busybox commands, 231not yet supplied by toybox:</p> 232 233<blockquote><p> 234awk bunzip2 bzcat dd diff expr fdisk ftpd ftpget 235ftpput gunzip gzip less ping route sh 236sha512sum tar test tr unxz vi wget xzcat zcat 237</p></blockquote> 238 239<p>Many of those are in "pending". The remaining "difficult" 240commands are vi, awk, and sh.</p> 241 242<p>Building Linux From Scratch is not the same as building the 243<a href=https://source.android.com>Android Open Source Project</a>, 244but after toybox 1.0 focus may shift to <a href=http://landley.net/aboriginal/about.html#hairball>modifying the AOSP build</a> 245to reduce dependencies. (It's fairly likely we'll have to add at least 246a read-only git utility so repo can download the build's source code, 247but that's actually <a href=https://www.youtube.com/watch?v=P7n6G2IL6eo>not 248that hard</a>. We'll probably also need our own "make" at some point after 2491.0.)</p> 250 251<hr /> 252<h2><a name=android /><a href="#android">Use case: Replacing Android Toolbox</a></h2> 253 254<p>Android has a policy against GPL in userspace, so even though BusyBox 255predates Android by many years, they couldn't use it. Instead they grabbed 256an old version of ash and implemented their own command line utility set 257called "toolbox". ash was later replaced by 258<a href="https://www.mirbsd.org/mksh.htm">mksh</a>; toolbox is being 259replaced by toybox.</p> 260 261<p>Toolbox doesn't have its own repository, instead it's part of Android's 262<a href=https://android.googlesource.com/platform/system/core>system/core 263git repository</a>.</p> 264 265<h3>Toolbox commands:</h3> 266 267<p>According to <a href=https://android.googlesource.com/platform/system/core/+/master/toolbox/Android.mk> 268system/core/toolbox/Android.mk</a> the toolbox directory builds the 269following commands:</p> 270 271<blockquote><b> 272dd getevent newfs_msdos 273</b></blockquote> 274 275<p>The toolbox makefile also builds the BSD grep right now, because toybox 276grep is missing <code>--color</code>.</p> 277 278<h3>Other Android /system/bin commands</h3> 279 280<p>Other than the toolbox links, the currently interesting 281binaries in /system/bin are:</p> 282 283<ul> 284<li><b>arping</b> - ARP REQUEST tool (iputils)</li> 285<li><b>blkid</b> - identify block devices (e2fsprogs)</li> 286<li><b>e2fsck</b> - fsck for ext2/ext3/ext4 (e2fsprogs)</li> 287<li><b>fsck.f2fs</b> - fsck for f2fs (f2fs-tools)</li> 288<li><b>fsck_msdos</b> - fsck for FAT (BSD)</li> 289<li><b>gzip</b> - compression/decompression tool (zlib)</li> 290<li><b>ip</b> - network routing tool (iproute2)</li> 291<li><b>iptables/ip6tables</b> - IPv4/IPv6 NAT admin (iptables)</li> 292<li><b>iw</b> - wireless device config tool (iw)</li> 293<li><b>logwrapper</b> - redirect stdio to android log (Android)</li> 294<li><b>make_ext4fs</b> - make ext4 fs (Android)</li> 295<li><b>make_f2fs</b> - make f2fs fs (f2fs-tools)</li> 296<li><b>ping/ping6</b> - ICMP ECHO_REQUEST tool (iputils)</li> 297<li><b>reboot</b> - reboot (Android)</li> 298<li><b>resize2fs</b> - resize ext2/ext3/ext4 fs (e2fsprogs)</li> 299<li><b>sh</b> - mksh (BSD)</li> 300<li><b>ss</b> - socket statistics (iproute2)</li> 301<li><b>tc</b> - traffic control (iproute2)</li> 302<li><b>tracepath/tracepath6</b> - trace network path (iputils)</li> 303<li><b>traceroute/traceroute6</b> - trace network route (iputils)</li> 304</ul> 305 306<p>The names in parentheses are the source.</p> 307 308<h3>Analysis</h3> 309 310<p>For reference, combining everything listed above, we get:</p> 311 312<blockquote><b> 313arping blkid e2fsck dd fsck.f2fs fsck_msdos getevent gzip ip iptables 314ip6tables iw logwrapper make_ext4fs make_f2fs newfs_msdos ping ping6 315reboot resize2fs sh ss tc tracepath tracepath6 traceroute traceroute6 316</b></blockquote> 317 318<p>We may eventually implement all of that, but for toybox 1.0 we need to 319focus a bit. For our first pass, let's just replace all the "toolbox" 320commands.</p> 321 322<p>This means toybox should implement (or finish implementing):</p> 323<blockquote><b> 324<span id=toolbox> 325dd getevent grep gzip newfs_msdos 326</span> 327</b></blockquote> 328 329<p>Update: Android.mk is currently building the following toybox files out 330of "pending". These should be a priority for cleanup (ones marked with * 331don't have a symlink, so they're a lot less visible):</p> 332 333<blockquote><b> 334chrt dd expr getfattr* lsof modprobe more setfattr* tar tr traceroute 335</b></blockquote> 336 337<p>Android wishlist:</p> 338 339<blockquote><b> 340mtools genvfatfs mke2fs gene2fs 341</b></blockquote> 342 343<hr /> 344<h2><a name=tizen /><a href="#tizen">Use case: Tizen Core</a></h2> 345 346<p>The Tizen project has expressed a desire to eliminate GPLv3 software 347from its core system, and is installing toybox as 348<a href=https://wiki.tizen.org/wiki/Toybox>part of this process</a>.</p> 349 350<p>They have a fairly long list of new commands they'd like to see in toybox:</p> 351 352<blockquote><b> 353<span id=tizen> 354arch base64 users dir vdir unexpand shred join csplit 355hostid nproc runcon sha224sum sha256sum sha384sum sha512sum sha3sum mkfs.vfat fsck.vfat 356dosfslabel uname stdbuf pinky diff3 sdiff zcmp zdiff zegrep zfgrep zless zmore 357</span> 358</b></blockquote> 359 360<p>In addition, they'd like to use several commands currently in pending:</p> 361 362<blockquote><b> 363<span id=tizen> 364tar diff printf wget rsync fdisk vi less tr test stty fold expr dd 365</span> 366</b></blockquote> 367 368<p>Also, tizen uses a different Linux Security Module called SMACK, so 369many of the SELinux options ala ls -Z need smack alternatives in an 370if/else setup.</p> 371 372<hr /><a name=klibc /> 373<h2>klibc:</h2> 374 375<p>Long ago some kernel developers came up with a project called 376<a href=http://en.wikipedia.org/wiki/Klibc>klibc</a>. 377After a decade of development it still has no web page or HOWTO, 378and nobody's quite sure if the license is BSD or GPL. It inexplicably 379<a href=http://www.infoworld.com/d/data-center/perl-isnt-going-anywhere-better-or-worse-211580>requires perl to build</a>, and seems like an ideal candidate for 380replacement.</p> 381 382<p>In addition to a C library even less capable than bionic (obsoleted by 383musl), klibc builds a random assortment of executables to run init scripts 384with. There's no multiplexer command, these are individual executables:</p> 385 386<blockquote><p><b> 387cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill 388kinit ln losetup ls minips mkdir mkfifo mknodes 389mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume 390run-init sh sha1hash sleep sync true umount uname zcat 391</b></p></blockquote> 392 393<p>To get that list, build klibc according to the instructions (I 394<a href=http://landley.net/notes-2013.html#23-01-2013>looked at</a> version 3952.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install 396linux; make) then <b>echo $(for i in $(find . -type f); do file $i | grep -q 397executable && basename $i; done | grep -v '[.]g$' | sort -u)</b> to find 398executables, then eliminate the *.so files and *.shared duplicates.</p> 399 400<p>Some of those binaries are build-time tools that don't get installed, 401which removes mknodes, mksyntax, sha1hash, and fixdep from the list. 402(And sha1hash is just an unpolished sha1sum anyway.)</p> 403 404<p>The run-init command is more commonly called switch_root, nuke is just 405"rm -rf -- $@", and minips is more commonly called "ps". I'm not doing aliases 406for the oddball names.</p> 407 408<p>Yet more stale forks of dash and gzip sucked in here (see "dubious 409license terms" above), adding nothing to the other projects we've looked at. 410But we still need sh, gunzip, gzip, and zcat to replace this package.</p> 411 412<p>At the time I did the initial analysis toybox already had cat, chroot, dmesg, false, 413kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync, 414true, and uname.</p> 415 416<p>The low hanging fruit is cpio, dd, ps, mv, and pivot_root.</p> 417 418<p>The "kinit" command is another gratuitous rename, it's init running as PID 1. 419The halt, poweroff, and reboot commands work with it.</p> 420 421<p>I've got mount and umount queued up already, fstype and nfsmount go with 422those. (And probably smbmount and p9mount, but this hasn't got one. Those 423are all about querying for login credentials, probably workable into the 424base mount command.)</p> 425 426<p>The ipconfig command here has a built in dhcp client, so it's ifconfig 427and dhcpcd and maybe some other stuff.</p> 428 429<p>The resume command is... weird. It finds a swap partition and reads data 430from it into a /proc file, something the kernel is capable of doing itself. 431(Even though the klibc author 432<a href=http://www.zytor.com/pipermail/klibc/2006-June/001748.html>attempted 433to remove</a> that capability from the kernel, current kernel/power/hibernate.c 434still parses "resume=" on the command line). And yet various distros seem to 435make use of klibc for this. 436Given the history of swsusp/hibernate (and 437<a href=http://lwn.net/Articles/333007>TuxOnIce</a> 438and <a href=http://lwn.net/Articles/242107>kexec jump</a>) I've lost track 439of the current state of the art here. Ah, Documentation/power/userland-swsusp.txt 440has the API docs, and <a href=http://suspend.sf.net>here's a better 441tool</a>...</p> 442 443<p>So the list of things actually in klibc are:</p> 444 445<blockquote><b> 446<span id=klibc_cmd> 447cat chroot dmesg false kill ln losetup ls mkdir mkfifo readlink rm switch_root 448sleep sync true uname 449 450cpio dd ps mv pivot_root 451mount nfsmount fstype umount 452sh gunzip gzip zcat 453kinit halt poweroff reboot 454ipconfig 455resume 456</span> 457</b></blockquote> 458 459<hr /> 460<a name=glibc /> 461<h2>glibc</h2> 462 463<p>Rather a lot of command line utilities come bundled with glibc:</p> 464 465<blockquote><b> 466catchsegv getconf getent iconv iconvconfig ldconfig ldd locale localedef 467mtrace nscd rpcent rpcinfo tzselect zdump zic 468</b></blockquote> 469 470<p>Of those, musl libc only implements ldd.</p> 471 472<p>catchsegv is a rudimentary debugger, probably out of scope for toybox.</p> 473 474<p>iconv has been <a href="#susv4">previously discussed</a>.</p> 475 476<p>iconvconfig is only relevant if iconv is user-configurable; musl uses a 477non-configurable iconv.</p> 478 479<p>getconf is a posix utility which displays several variables from 480unistd.h; it probably belongs in the development toolchain.</p> 481 482<p>getent handles retrieving entries from passwd-style databases 483(in a rather lame way) and is trivially replacable by grep.</p> 484 485<p>locale was discussed under <a href=#susv4>posix</a>. 486localedef compiles locale definitions, which musl currently does not use.</p> 487 488<p>mtrace is a perl script to use the malloc debugging that glibc has built-in; 489this is not relevant for musl, and would necessarily vary with libc. </p> 490 491<p>nscd is a name service caching daemon, which is not yet relevant for musl. 492rpcinfo and rpcent are related to rpc, which musl does not include.</p> 493 494<p>The remaining commands involve glibc's bundled timezone database, 495which seems to be derived from the <a href=http://www.iana.org/time-zones>IANA 496timezone database</a>. Unless we want to maintain our own fork of the 497standards body's database like glibc does, these are of no interest, 498but for completeness:</p> 499 500<p>tzselect outputs a TZ variable correponding to user input. 501The documentation does not indicate how to use it in a script, but it seems 502that Debian may have done so. 503zdump prints current time in each of several timezones, optionally 504outputting a great deal of extra information about each timezone. 505zic converts a description of a timezone to a file in tz format.</p> 506 507<p>None of glibc's bundled commands are currently of interest to toybox.</p> 508 509</b></blockquote> 510 511<hr /> 512<a name=sash /> 513<h2>Stand-Alone Shell</h2> 514 515<p>Wikipedia has <a href=http://en.wikipedia.org/wiki/Stand-alone_shell>a good 516summary of sash</a>, with links. The original Stand-Alone Shell project reached 517a stopping point, and then <a href=http://www.baiti.net/sash>"sash plus 518patches"</a> extended it a bit further. The result is a megabyte executable 519that provides 40 commands.</p> 520 521<p>Sash is a shell with built-in commands. It doesn't have a multiplexer 522command, meaning "sash ls -l" doesn't work (you have to go "sash -c 'ls -l'"). 523</p> 524 525<p>The list of commands can be obtained via building it and doing 526"echo help | ./sash | awk '{print $1}' | sed 's/^-//' | xargs echo", which 527gives us:</p> 528 529<blockquote><b> 530alias aliasall ar cd chattr chgrp chmod chown cmp cp chroot dd echo ed exec 531exit file find grep gunzip gzip help kill losetup losetup ln ls lsattr mkdir 532mknod more mount mv pivot_root printenv prompt pwd quit rm rmdir setenv source 533sum sync tar touch umask umount unalias where 534</b></blockquote> 535 536<p>Plus sh because it's a shell. A dozen or so commands can only sanely be 537implemented as shell builtins (alias aliasall cd exec exit prompt quit setenv 538source umask unalias), where is an alias for which, and at triage time toybox 539already has chgrp, chmod, chown, cmp, cp, chroot, echo, help, kill, losetup, 540ln, ls, mkdir, mknod, printenv, pwd, rm, rmdir, sync, and touch.</p> 541 542<p>This leaves:</p> 543 544<blockquote><b> 545<span id=sash_cmd> 546ar chattr dd ed file find grep gunzip gzip lsattr more mount mv pivot_root 547sh sum tar umount 548</span> 549</b></blockquote> 550 551<p>(For once, this project doesn't include a fork of gzip, instead 552it sucks in -lz from the host.)</p> 553 554<hr /> 555<a name=sbase /> 556<h2>sbase:</h2> 557 558<p>It's <a href=http://git.suckless.org/sbase>on suckless</a> in 559<a href=http://git.suckless.org/ubase>two parts</a>. As of November 2015 it's 560implemented the following (renaming "cron" to "crond" for 561consistency, and yanking "sponge", "mesg", "pagesize", "respawn", and 562"vtallow"):</p> 563 564<blockquote><p> 565<span id=sbase_cmd> 566basename cal cat chgrp chmod chown chroot cksum cmp cols comm cp crond cut date 567dirname du echo env expand expr false find flock fold getconf grep head 568hostname join kill link ln logger logname ls md5sum mkdir mkfifo mktemp mv 569nice nl nohup od paste printenv printf pwd readlink renice rm rmdir sed seq 570setsid sha1sum sha256sum sha512sum sleep sort split strings sync tail 571tar tee test tftp time touch tr true tty uname unexpand uniq unlink uudecode 572uuencode wc which xargs yes 573</span> 574</p></blockquote> 575 576<p>and<p> 577 578<blockquote><p> 579<span id=sbase_cmd> 580chvt clear dd df dmesg eject fallocate free id login mknod mountpoint 581passwd pidof ps stat su truncate unshare uptime watch 582who 583</span> 584</p></blockquote> 585 586<hr /> 587<a name=nash /> 588<h2>nash:</h2> 589 590<p>Red Hat's nash was part of its "mkinitrd" package, replacement for a shell 591and utilities on the boot floppy back in the 1990's (the same general idea 592as BusyBox, developed independently). Red Hat discontinued nash development 593in 2010, replacing it with dracut (which collects together existing packages, 594including busybox).</p> 595 596<p>I couldn't figure out how to beat source code out of 597<a href=http://pkgs.fedoraproject.org/git/mkinitrd>Fedora's current git</a> 598repository. The last release version that used it was Fedora Core 12 599which has <a href=http://archive.fedoraproject.org/pub/archive/fedora/linux/releases/12/Fedora/source/SRPMS/mkinitrd-6.0.93-1.fc12.src.rpm>a source rpm</a> 600that can be unwound with "rpm2cpio mkinitrd.src.rpm | cpio -i -d -H newc 601--no-absolute-filenames" and in there is a mkinitrd-6.0.93.tar.bz2 which 602has the source.</p> 603 604<p>In addition to being a bit like a command shell, the nash man page lists the 605following commands:</p> 606 607<blockquote><p> 608access echo find losetup mkdevices mkdir mknod mkdmnod mkrootdev mount 609pivot_root readlink raidautorun setquiet showlabels sleep switchroot umount 610</p></blockquote> 611 612<p>Oddly, the only occurrence of the string pivot_root in the nash source code 613is in the man page, the command isn't there. (It seems to have been removed 614when the underscoreless switchroot went in.)</p> 615 616<p>A more complete list seems to be the handlers[] array in nash.c:</p> 617 618<blockquote><p> 619access buildEnv cat cond cp daemonize dm echo exec exit find kernelopt 620loadDrivers loadpolicy mkchardevs mkblktab mkblkdevs mkdir mkdmnod mknod 621mkrootdev mount netname network null plymouth hotplug killplug losetup 622ln ls raidautorun readlink resume resolveDevice rmparts setDeviceEnv 623setquiet setuproot showelfinterp showlabels sleep stabilized status switchroot 624umount waitdev 625</p></blockquote> 626 627<p>This list is nuts: "plymouth" is an alias for "null" which is basically 628"true" (which thie above list doesn't have). Things like buildEnv and 629loadDrivers are bespoke Red Hat behavior that might as well be hardwired in 630to nash's main() without being called.</p> 631 632<p>Instead of eliminating items 633from the list with an explanation for each, I'm just going to cherry pick 634a few: the device mapper (dm, raidautorun) is probably interesting, 635hotplug (may be obsolete due to kernel changes that now load firmware 636directly), and another "resume" ala klibc.</p> 637 638<p>But mostly: I don't care about this one. And neither does Red Hat anymore.</p> 639 640<p>Verdict: ignore</p> 641 642<hr /> 643<a name=beastiebox /> 644<h2>Beastiebox</h2> 645 646<p>Back in 2008, the BSD guys vented some busybox-envy 647<a href=http://beastiebox.sourceforge.net>on sourceforge</a>. Then stopped. 648Their repository is still in CVS, hasn't been touched in years, it's a giant 649hairball of existing code sucked together. (The web page says the author 650is aware of crunchgen, but decided to do this by hand anyway. This is not 651a collection of new code, it's a katamari of existing code rolled up in a 652ball.)</p> 653 654<p>Combining the set of commands listed on the web page with the set of 655man pages in the source gives us:</P> 656 657<blockquote><p> 658[ cat chmod cp csh date df disklabel dmesg echo ex fdisk fsck fsck_ffs getty 659halt hostname ifconfig init kill less lesskey ln login ls lv mksh more mount 660mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test 661traceroute umount vi wiconfig 662</p></blockquote> 663 664<p>Apparently lv is the missing link between ed and vi, copyright 1982-1997 (do 665not want), ex is another obsolete vi mode, lesskey is "used to 666specify a set of key bindings to be used with less", and csh is a shell they 667sucked in (even though they have mksh?), [ is an alias for test. Several more bsd-isms that don't have Linux 668equivalents (even in the ubuntu "install this package" search) are 669disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a 670wavelan interface network card driver utility. Subtracting all that and the 671commands toybox already implements at triage time, we get:</p> 672 673<blockquote><p> 674<span id=beastiebox_cmd> 675fdisk fsck getty halt ifconfig init kill less more mount mv ping poweroff 676ps reboot route sed sh stty sysctl tar test traceroute umount vi 677</span> 678</p></blockquote> 679 680<p>Not a hugely interesting list, but eh.</p> 681 682<p>Verdict: ignore</p> 683 684<hr /> 685<a name=BsdBox /> 686<h2>BsdBox</h2> 687 688<p>Somebody decided to do a <a href=https://wiki.freebsd.org/AdrianChadd/BsdBox>multicall binary for freebsd</a>.</p> 689 690<p>They based it on crunchgen, a tool that glues existing programs together 691into an archive and uses the name to execute the right one. It has no 692simplification or code sharing benefits whatsoever, it's basically an 693archiver that produces executables.</p> 694 695<p>That's about where I stopped reading.</p> 696 697<p>Verdict: ignore.</p> 698 699<hr /> 700<a name=slowaris /> 701<h2>OpenSolaris Busybox</h2> 702 703<p>Somebody <a href=http://hub.opensolaris.org/bin/view/Project+busybox/>wrote 704a wiki page</a> saying that Busybox for OpenSolaris would be a good idea.</p> 705 706<p>The corresponding "files" tab is an auto-generated stub. The project never 707even got as far as suggesting commands to include before Oracle discontinued 708OpenSolaris.</p> 709 710<p>Verdict: ignore.</p> 711 712<hr /> 713<a name=uclinux /> 714<h2>uClinux</h2> 715 716<p>Long ago a hardware developer named Jeff Dionne put together a 717nommu Linux distribution, which involved rewriting a lot of command line 718utilities that relied on <a href=http://nommu.org/memory-faq.txt>features 719unavailable on nommu</a> hardware.</p> 720 721<p>In 2003 Jeff moved to Japan and handed 722the project off to people who allowed it to roll to a stop. The website 723turned into a mess of 404 links, the navigation indexes stopped being 724updated over a decade ago, and the project's CVS repository suffered a 725hard drive failure for which there were no backups. The project continued 726to put out "releases" through 2014 (you have to scroll down in the "news" 727section to find them, the "HTTP download" section in the nav bar on the 728left hasn't been updated in over a decade), which were hand-updated tarball 729snapshots mostly consisting of software from the 1990's. For example the 7302014 release still contained ipfwadm, the package which predated ipchains, 731which predated iptables, which is in the process of being replaced by 732nftables.</p> 733 734<p>Nevertheless, people still try to use this because (at least until the 735launch of <a href=http://nommu.org>nommu.org</a>) the project was viewed 736as the place to discuss, develop, and learn about nommu Linux. 737The role of uclinux.org as an educational resource kept people coming 738to it long after it had collapsed as a Linux distro.</p> 739 740<p>Starting around 0.6.0 toybox began to address nommu support with the goal 741of putting uClinux out of its misery.</p> 742 743<p>An analysis of <a href=http://www.uclinux.org/pub/uClinux/dist/uClinux-dist-20140504.tar.bz2>uClinux-dist-20140504</a> found 312 package 744subdirectories under "user".</p> 745 746<h3>Taking out the trash</h3> 747 748<p>A bunch of packages (<b>inotify-tools, input-event-demon, ipsec-tools, netifd, 749keepalived, mobile-broadband-provider-info, nuttp, readline, snort, 750snort-barnyard, socat, sqlite, sysklogd, sysstat, tcl, ubus, uci, udev, 751unionfs, uqmi, usb_modeswitch, usbutils, util-linux</b>) 752are hard to evaluate because 753uclinux has directories for them, but their source isn't actually in the 754uclinux tree. In some of these the makefiles download a git repo during 755the build, so I'm assuming you can build the external package if you really 756care. (Even when I know what these packages do, I'm skipping them 757because uclinux doesn't actually contain them, and any given snapshot 758of the build system will bitrot as external web links change over time.)</p> 759 760<p>Other packages are orphaned, meaning they're not mentioned from any Kconfig 761or Makefiles outside of their directory, so uclinux can't actually build 762them: <b>mbus</b> is an orphaned i2c test program expecting to run in some sort 763of hardwired hardware context, <b>mkeccbin</b> is an orphaned "ECC annotated 764binary file" generator (meaning it's half of a flash writer), 765<b>wsc_upnp</b> is a "Ralink WPS" driver (some sort of stale wifi chip)...</p> 766 767<p>The majority of the remaining packages are probably not of interest to 768toybox due to being so obsolete or special purpose they may not actually be 769of interest to anybody anymore. (This list also includes a lot of 770special-purpose network back-end stuff that's hard for anybody but 771datacenter admins to evaluate the current relevance of.)</p> 772 773<blockquote><b><p> 774arj asterisk boottools bpalogin br2684ctl camserv can4linux cgi_generic 775cgihtml clamav clamsmtp conntrack-tools cramfs crypto-tools cxxtest 776ddns3-client de2ts-cal debug demo diald discard dnsmasq dnsmasq2 777ethattach expat-examples ez-ipupdate fakeidentd 778fconfig ferret flatfs flthdr freeradius freeswan frob-led frox fswcert 779game gettyd gnugk haserl horch 780hostap hping httptunnel ifattach ipchains 781ipfwadm ipmasqadm ipportfw ipredir ipset iso_client 782jamvm jffs-tools jpegview jquery-ui kendin-config kismet klaxon kmod 783l2tpd lcd ledcmd ledcon lha lilo lirc lissa load loattach 784lpr lrpstat lrzsz mail mbus mgetty microwin ModemManager msntp musicbox 785nooom null openswan openvpn palmbot pam_* pcmcia-cs playrt plugdaemon pop3proxy 786potrace qspitest quagga radauth 787ramimage readprofile rdate readprofile routed rrdtool rtc-ds1302 788sendip ser sethdlc setmac setserial sgutool sigs siproxd slattach 789smtpclient snmpd net-snmp snortrules speedtouch squashfs scep sslwrap stp 790stunnel tcpblast tcpdump tcpwrappers threaddemos tinylogin tinyproxy 791tpt tripwire unrar unzoo version vpnled w3cam xl2tpd zebra 792</p></b></blockquote> 793 794<p>This stuff is all over the place: arj, lha, rar, and zoo are DOS archivers, 795ethattach describes itself as just "a network tool", 796mail is a textmode smtp mailer literally described as "Some kind of mail 797proggy" in uclinux's kconfig (as opposed to clamsmtp and smtpclient and 798so on), this gettyd isn't a generic version but specifically a 799hardwired ppp dialin utility, mgetty isn't a generic version but is combined 800with "sendfax", hostap is an intersil prism driver, wlan-ng is also an 801intersil prism dirver, null is a program to intentionally dereference a 802null pointer (in case you needed one), iso_client is a 803"Demo Application for the USB Device Driver", kendin-config is 804"for configuring the Micrel Kendin KS8995M over QSPI", speedtouch configures 805a specific brand of asdl modem, portmap is part of Anfs, 806ferret, linux-igd, and miniupnp are all upnp packages, 807lanbypass "can be used to control the LAN 808bypass switches on the Advantech x86 based hardware platforms", lcd is 809"test of lcddma device driver" (an out-of-tree Coldfire driver apparently 810lost to history, the uclinux linux-2.4.x directory has a config symbol for 811it, but nothing in the code actually _uses_ it...), qspitest is another 812coldfire thing, mii-tool-fec is 813"strictly for the FEC Ethernet driver as implemented (and modified) for 814the uCdimm5272", rtc-ds1302 and rtc-m41t11 are usermode drivers for specific 815clock chips, stunnel is basically "openssl s_client -quiet -connect", 816potrace is a bitmap to vector graphic converter, radauth performs command line 817authentication against a radius server, 818clamav, klaxon, ferret, l7-protocols, and nessus are very old network security 819software (it's got a stale snapshot of nmap too), xl2tpd is a PPP over UDP 820tunnel (rfc 2661), zebra is the package quagga replaced, 821lilo is the x86-only bootloader that predated grub (and recently discontinued 822development), lissa is a "framebuffer graphics demo" from 8231998, the squashfs package here is the out of tree patches for 2.4 kernels 824and such before the filesystem was merged upstream (as opposed to the 825squashfs-new package which is a snapshot of the userspace tool from 2011), 826load is basically "dd file /dev/spi", version is basically "cat /proc/version", 827microwin is a port of the WinCE graphics API to Linux, scep is a 2003 828implementation of an IETF draft abandoned in 2010, tpt depends on 829Andrew Morton's 15 year old unmerged "timepegs" kernel patch using the pentium 830cycle counter, vpnled controls a light that reboots systems (what?), 831w3cam is a video4linux 1.0 client (v4l2 showed up during 2.5 and support for 832the old v4l1 was removed in 2.6.38 back in 2011), busybox ate tinylogin 833over a decade ago, lrpstat is a java network monitor 834from 2001, lrzsz is zmodem/ymodem/zmodem, msntp and stp implement rfc2030 835meaning it overflows in 2036 (the package was last updated in 2000), rdate 836is rfc 868 meaning it also overflows in 2036 (which is why ntp was invented 837a few decades back), reiserfsprogs development stopped abruptly after 838Hans Reiser was convicted of murdering his wife Nina (denying it on the 839stand and then leading them to the body as part of his plea bargain during 840sentencing)... 841</p> 842 843<p>Seriously, there's a lot of crap in there. It's hard to analyze most 844of it far enough to prove it _doesn't_ do anything.</p> 845 846<h3>Non-toybox programs</h3> 847 848<p>The following software may actually still do something intelligible 849(although the package versions tend to be years out of date), but 850it's not a direction toybox has chosen to go in.</p> 851 852<p>There are several programming languages (<b>bash, lua, jamvm, tinytcl, 853perl, python</b>) in there. Maybe someone somewhere wants a 2008 release of a 854java virtual machine tested to work on nommu systems (jamvm), but it's out 855of scope for toybox.</p> 856 857<p>A bunch of benchmark programs: <b>cpu, dhrystone, mathtest, nbench, netperf, 858netpipe, and whetstone</b>.</p> 859 860<p>A bunch of web servers: <b>appWeb, boa, fnord (via tcpserver), goahead, httpd, 861mini_httpd, and thttpd</b>.</p> 862 863<p>A bunch of shells: <b>msh</b> is a clever (I.E. obfuscated) little shell, 864<b>nwsh</b> is "new shell" (that's what it called itself in 1999 anyway), 865<b>sash</b> is another shell with a bunch of builtins (ls, ps, df, cp, date, reboot, 866and shutdown, this roadmap analyzes it <a href="#sash">elsewhere</a>), 867<b>sh</b> is a very old minix shell fork, and <b>tcsh</b> is also a shell.</p> 868 869<p>Also in this category, we have:</p> 870 871<blockquote><b><p> 872dropbear jffs-tools jpegview kexec-tools bind ctorrent 873iperf iproute2 ip-sentinel iptables kexec 874nmap oggplay openssl oprofile p7zip pppd pptp play vplay 875hdparm mp3play at clock 876mtd-utils mysql logrotate brcfg bridge-utils flashw 877ebtables etherwake ethtool expect gdb gdbserver hostapd 878lm_sensors load netflash netstat-nat 879radvd recover rootloader resolveip rp-pppoe 880rsyslog rsyslogd samba smbmount squashfs-new squid ssh strace tip 881uboot-envtools ulogd usbhubctrl vconfig vixie-cron watchdogd 882wireless_tools wpa_supplicant 883</p></b></blockquote> 884 885<p>An awful lot of those are borderline: play and vplay are wav file 886audio players, there's oprofile _and_ readprofile (which just reads kernel 887profiling data from /proc/profile), 888radvd is a "routr advertisement daemon" (ipv6 stateless autoconf), 889ctorrent is a bittorent client, 890lm_sensors is hardware (heat?) monitoring, 891resolveip is dig only less so, 892rp-pppoe is ppp over ethernet, 893ebtables is an ethernet version of iptables (for bridging), 894their dropbear is from 2012, and that ssh version is from 2011 895(which means it's about nine months too _old_ to have the heartbleed bug). 896There's both ulogd and ulogd2 (no idea why), and pppd is version 2.4 but 897there's a ppd-2.3 directory also.</p> 898 899<p>Lots of flash stuff: 900flashw is a flash writer, load is an spi flash loader, netflash writes 901to flash via tftp, 902recover is also a reflash daemon intended to come up when the system can't boot, 903rootloader seems to be another reflash daemon but without dhcp.</p> 904 905<h3>Already in roadmap</h3> 906 907<p>The following packages contain commands already in the toybox roadmap:</p> 908 909<blockquote><b><p> 910agetty cal cksum cron dhcpcd dhcpcd-new dhcpd dhcp-isc dosfstools e2fsprogs 911elvis-tiny levee fdisk fileutils ftp ftpd grep hd hwclock inetd init ntp 912iputils login module-init-tools netcat shutils ntpdate lspci ping procps 913proftpd rsync shadow shutils stty sysutils telnet telnetd tftp tftpd traceroute 914unzip wget mawk net-tools 915</p></b></blockquote> 916 917<p>There are some duplicates in there, levee is a tiny vi implementation 918like elvis-tiny, ntp and ntpdate overlap, etc.</p> 919 920<p>Verdict: We don't really need to do a whole lot special for nommu 921systems, just get the existing toybox roadmap working on nommu and 922we're good. The uClinux project can rest in peace.</p> 923 924<hr /> 925<h2>Requests:</h2> 926 927<p>The following additional commands have been requested (and often submitted) 928by various users. I _really_ need to clean up this section.</p> 929 930<p>Also:</p> 931<blockquote><b> 932<span id=request> 933dig freeramdisk getty halt hexdump hwclock klogd modprobe ping ping6 pivot_root 934poweroff readahead rev sfdisk sudo syslogd taskset telnet telnetd tracepath 935traceroute unzip usleep vconfig zip free login modinfo unshare netcat help w 936ntpd iwconfig iwlist rdate 937dos2unix unix2dos catv clear 938pmap realpath setsid timeout truncate 939mkswap swapon swapoff 940count oneit fstype 941acpi blkid eject pwdx 942sulogin rfkill bootchartd 943arp makedevs sysctl killall5 crond crontab deluser last mkpasswd watch 944blockdev rpm2cpio arping brctl dumpleases fsck 945tcpsvd tftpd 946factor fallocate fsfreeze inotifyd lspci nbd-client partprobe strings 947base64 mix 948reset hexedit nsenter shred 949fsync insmod ionice lsmod lsusb rmmod vmstat xxd top iotop 950lsof ionice compress dhcp dhcpd addgroup delgroup host iconv ip 951ipcrm ipcs netstat openvt 952deallocvt iorenice 953udpsvd adduser 954microcom tunctl chrt getfattr setfattr 955</span> 956</b></blockquote> 957 958<!-- #include "footer.html" --> 959 960