1<html><head><title>toybox news</title>
   2<!--#include file="header.html" -->
   5<li><h2><a href="#capitalize">Do you capitalize toybox?</a></h2></li>
   6<li><h2><a href="#why_toybox">Why toybox? (What was wrong with busybox?)</a></h2></li>
   7<li><h2><a href="#support_horizon">Why a 7 year support horizon?</a></h2></li>
   8<li><h2><a href="#releases">Why time based releases?</a></h2></li>
   9<li><h2><a href="#code">Where do I start understanding the toybox source code?</a></h2></li>
  12<a name="capitalize" />
  13<h2>Q: Do you capitalize toybox?</h2>
  15<p>A: Only at the start of a sentence. The command name is all lower case so
  16it seems silly to capitalize the project name, but not capitalizing the
  17start of sentences is awkward, so... compromise. (It is _not_ "ToyBox".)</p>
  19<a name="why_toybox" />
  20<h2>Q: "Why is there toybox? What was wrong with busybox?"</h2>
  22<p>A: Toybox started back in 2006 when I
  23<a href=>handed off BusyBox maintainership</a>
  24and <a href=>started over from
  25scratch</a> on a new codebase after a
  26<a href=>protracted licensing argument</a> took all the fun out of working on BusyBox.</p>
  28<p>Toybox was just a personal project until it got
  29<a href=>relaunched
  30in November 2011</a> with a new goal to
  31<a href=>make Android
  32self-hosting</a>. This involved me relicensing my own
  33code, which <a href=>made people who had
  34never used or participated in the project loudly angry</a>. The switch came
  35after a lot of thinking <a href=>about
  36licenses</a> and <a href=>the
  37transition to smartphones</a>, which led to a
  38<a href=>2013</a>
  39<a href=>talk</a> laying
  40out a strategy to make Android self-hosting using toybox. This helped
  41<a href=>bring
  42it to Android's attention</a>, and they
  43<a href=>merged it</a> into Android M.</p>
  45<p>The answer to the second question is "licensing". BusyBox predates Android
  46by almost a decade but Android still doesn't ship with it because GPLv3 came
  47out around the same time Android did and caused many people to throw
  48out the GPLv2 baby with the GPLv3 bathwater.
  49Android <a href=>explicitly
  50discourages</a> use of GPL and LGPL licenses in its products, and has gradually
  51reimplemented historical GPL components such as its bluetooth stack under the
  52Apache license. Similarly, Apple froze xcode at the last GPLv2 releases
  53(GCC 4.2.1 with binutils 2.17) for over 5 years while it sponsored the
  54development of new projects (clang/llvm/lld) to replace them,
  55implemented its SMB server from scratch to replace samba,
  56<a href=>and so
  57on</a>. Toybox itself exists because somebody with in a legacy position
  58just wouldn't shut up about GPLv3, otherwise I would probably
  59still happily be maintaining BusyBox. (For more on how I wound
  60up working on busybox in the first place,
  61<a href=>see here</a>.)</p>
  63<h2><a name="support_horizon">Q: Why a 7 year support horizon?</a></h2>
  65<p>A: Our <a href=>longstanding rule of thumb</a> is to try to run and build on
  66hardware and distributions released up to 7 years ago, and feel ok dropping
  67support for stuff older than that. (This is a little longer than Ubuntu's
  68Long Term Support, but not by much.)</p>
  70<p>If a kernel or libc feature is less than 7 years old, I try to have a
  71build-time configure test for it and let the functionality cleanly drop out.
  72I also keep old Ubuntu images around in VMs and perform the occasional
  73defconfig build there to see what breaks. (I'm not perfect about this,
  74but I accept bug reports.)</p>
  76<p>My original theory was "4 to 5 18-month cycles of moore's law should cover
  77the vast majority of the installed base of PC hardware", loosely based on some
  78research I did <a href=>back in 2003</a>
  79and <a href=>updated in 2006</a>
  80which said that low end systems were 2 iterations of moore's
  81law below the high end systems, and that another 2-3 iterations should cover
  82the useful lifetime of most systems no longer being sold but still in use and
  83potentially being upgraded to new software releases.</p>
  85<p>It turns out <a href=>I missed
  86industry changes</a> in the 1990's that stretched the gap
  87from low end to high end from 2 cycles to 4 cycles, and _that_ analysis
  88ignored the switch from PC to smartphone cutting off the R&D air supply of the
  89laptop market.  Meanwhile the Moore's Law s-curve started bending
  90down in 2000 and these days is pretty flat because the drive for faster clock
  91speeds <a href=>stumbled</a>
  92then <a href=>died</a>, and
  93the subsequent drive to go wide maxed out around 4x SMP with ~2 megabyte
  94caches for most applications. These days the switch from exponential to
  95linear growth in hardware capabilities is
  96<a href=>common</a>
  97<a href=>knowledge</a>.</p>
  99<p>But the 7 year rule of thumb stuck around anyway: if a kernel or libc
 100feature is less than 7 years old, I try to have a build-time configure test
 101for it and let the functionality cleanly drop out. I also keep old Ubuntu
 102images around in VMs and perform the occasional defconfig build there to
 103see what breaks.</p>
 105<h2><a name="releases" />Q: Why time based releases?</h2>
 106<p>A: Toybox targets quarterly releases (a similar schedule to the Linux
 107kernel) because Martin Michlmayr's
 108<a href=>talk</a> on the
 109subject was convincing.</p>
 111<p>Releases provide synchronization points where the developers certify
 112"it worked for me". Each release is a known version with predictable behavior,
 113and right or wrong at least everyone should be seeing
 114similar results where you might be able to google an unexpected outcome.
 115Releases focus end-user testing on specific versions
 116where issues can be reproduced, diagnosed, and fixed.
 117Releases also force the developers to do periodic tidying, packaging,
 118documentation review, finish up partially implemented features languishing
 119in their private trees, and give regular checkpoints to measure progress.</p>
 121<p>Over time feature sets change, data formats change, control knobs change...
 122For example toybox's switch from "ls -q" to "ls -b" as the default output
 123format wasn't exactly a bug, it was a design improvement... but the
 124difference is academic if the change breaks somebody's script.
 125Releases give you the option to schedule upgrades later, and not to rock
 126the boat just now: just use a known working release version.</p>
 128<p>The counter-argument is that "continuous integration"
 129can be made robust with sufficient automated testing. But like the
 130<a href=>waterfall method</a>, this places insufficent
 131emphasis on end-user feedback and learning from real world experience.
 132Developer testing is either testing that the code does what the developers
 133expect given expected inputs running in an expected environment, or it's
 134regression testing against bugs previously found in the field. No plan
 135survives contact with the enemy, and technology always breaks once it
 136leaves the lab and encounters real world data and use cases, not just
 137at runtime but in different build environments.</p>
 139<p>The best way to give new users a reasonable first experience is to point
 140them at specific stable versions where development quiesced and
 141extra testing occurred. There will still be teething troubles, but multiple
 142people experiencing the _same_ teething troubles can potentially
 143help each other out.</p>
 145<p>As for why releases on a schedule are better than releases "when it's
 146ready", watch the video.</p>
 148<h2><a name="code" />Q: Where do I start understanding the source code?</h2>
 150<p>A: Toybox is written in C. There are longer writeups of the
 151<a href=design.html>design ideas</a> and a <a href=code.html>code walkthrough</a>,
 152and the <a href=about.html>about page</a> summarizes what we're trying to
 153accomplish, but here's a quick start:</p>
 155<p>Toybox uses the standard three stage configure/make/install
 156<a href=code.html#building>build</a>, in this case "<b>make defconfig;
 157make; make install</b>". Type "<b>make help</b>" to
 158see available make targets.</p>
 160<p><b>The configure stage is copied from the Linux kernel</b> (in the "kconfig"
 161directory), and saves your selections in the file ".config" at the top
 162level. The "defconfig" target selects the
 163maximum sane configuration (enabling all the commands and features that
 164aren't unfinished, only intended as examples, debug code, etc) and is
 165probably what you want. You can use "make menuconfig" to manually select
 166specific commands to include, through an interactive menu (cursor up and
 167down, enter to descend into a sub-menu, space to select an entry, ? to see
 168an entry's help text, esc to exit). The menuconfig help text is the
 169same as the command's --help output.</p>
 171<p><b>The "make" stage creates a toybox binary</b> (which is stripped, look in
 172generated/unstripped for the debug versions), and "install" adds a bunch of
 173symlinks to toybox under the various command names. Toybox determines which
 174command to run based on the filename, or you can use the "toybox" name in which case the first
 175argument is the command to run (ala "toybox ls -l"). <b>You can also build
 176individual commands as standalone executables</b>, ala "make sed cat ls".</p>
 178<p><b>The main() function is in main.c at the top level</b>,
 179along with setup plumbing and selecting which command to run this time.
 180The function toybox_main() implements the "toybox" multiplexer command.</p>
 182<p><b>The individual command implementations are under "toys"</b>, and are grouped
 183into categories (mostly based on which standard they come from, posix, lsb,
 184android...) The "pending" directory contains unfinished commands, and the
 185"examples" directory contains examples. Commands in those two directories
 186are _not_ selected by defconfig. (These days pending directory is mostly
 187third party submissions that have not yet undergone proper code review.)</p>
 189<p><b>Common infrastructure shared between commands is under "lib"</b>. Most
 190commands call lib/args.c to parse their command line arguments before calling
 191the command's own main() function, which uses the option string in
 192the command's NEWTOY() macro. This is similar to the libc function getopt(),
 193but more powerful, and is documented at the top of lib/args.c.</p>
 195<p>Most of the actual <b>build/install infrastructure is shell scripts under
 196"scripts"</b>. <b>These populate the "generated" directory</b> with headers
 197created from other files, which are <a href=code.html#generated>described</a>
 198in the code walkthrough. All the
 199build's temporary files live under generated, including the .o files built
 200from the .c files (in generated/obj). The "make clean" target deletes that
 201directory. ("make distclean" also deletes your .config and deletes the
 202kconfig binaries that process .config.)</p>
 204<p>Each command's file contains all the information for that command, so
 205<b>adding a command to toybox means adding a single file under "toys"</b>.
 206Usually you <a href=code.html#adding>start a new command</a> by copying an
 207existing command file to a new filename
 208(toys/examples/hello.c, toys/examples/skeleton.c, toys/posix/cat.c,
 209and toys/posix/true.c have all been used for this purpose) and then replacing
 210all instances of its old name with the new name (which should match the
 211new filename), and modifying the help text, argument string, and what the
 212code does. You might have to "make distclean" before you new command
 213shows up in defconfig or menuconfig.</p>
 215<p><b>The toybox test suite lives in the "tests" directory</b>. From the top
 216level you can "make tests" to test everything, or "make test_sed" test a
 217single command's standalone version (which should behave identically)
 218but that's why we test.</p>
 220<!--#include file="footer.html" -->