toybox/www/news.html
<<
>>
Prefs
   1<html><head><title>toybox news</title>
   2<!--#include file="header.html" -->
   3
   4<p>Toybox combines the most common Linux command line utilities together
   5into a single BSD-licensed executable that's simple, small, fast,
   6reasonably standards-compliant, and powerful enough to turn Android into
   7a development environment. See the links on the left for details.</p>
   8
   9<h2>News</h2>
  10
  11<a name="08-01-2023" /><a href="#08-01-2023"><hr><h2><b>January 8, 2023</b></h2></a>
  12<blockquote>
  13<p>"Why," Arthur said, "is there a sofa in that field?"</br>
  14"I told you!" shouted Ford, leaping to his feet. "Eddies in the space-time
  15continuum!"</br>
  16"And this is his sofa, is it?"</p>
  17<p>- The Hitchhiker's Guide to the Galaxy</p>
  18</p></blockquote>
  19
  20<p><a href=downloads/toybox-0.8.9.tar.gz>Toybox 0.8.9</a>
  21(<a href=https://github.com/landley/toybox/releases/tag/0.8.9>git commit</a>)
  22is out, with prebuilt <a href=downloads/binaries/0.8.9>static binaries</a> and
  23<a href=downloads/binaries/mkroot/0.8.9>mkroot images</a>
  24bootable under QEMU (built using a <a href=https://github.com/landley/linux/tree/mkroot-6.1>lightly patched</a> linux-6.1).</p>
  25
  26<p>The new <b>nbd-server</b> command interoperates with nbd-client to serve
  27network block devices (using the v1 protocol), and Moritz Weber contributed a read-only <b>git</b>
  28implementation to pending.</p>
  29
  30<p><u>Features</u>: New <b>grep</b> fast path for fixed or simple patterns that
  31don't need the full regex engine (I.E. "^", ".", "$" but not "*" or "[]")
  32most noticeable when searching for many patterns at once.
  33Improved <b>tar --xform</b> support parsing flags= and trailing s/// scope
  34flags (but using --xform now requires toybox sed in the $PATH).</p>
  35
  36<p>Added <b>sort -C</b>, <b>lsusb -i</b>, <b>netcat -n</b>,
  37<b>swapoff -a -v</b>, <b>httpd -v</b>, <b>nbd-client -b</b>,
  38a <b>uname -p</b> stub to mollify package builds,
  39<b>sleep</b> accepts multiple arguments,
  40<b>sed</b> now parses interlaced -e and -f arguments in order,
  41several new options in <b>readlink</b> and <b>realpath</b> (and better
  42handling of relative paths), and UDP mode in netcat is more useful now:
  43<b>netcat -u -s 127.0.0.1 -p 9876 -l</b> can type at <b>netcat -u 127.0.0.1 9876</b>
  44in both directions. (Figuring out when the other side hangs up is an unsolved
  45problem, but that's UDP for you.)
  46Elliott added <b>ls -N</b>.</p>
  47
  48<p><u>Bugfixes</u>: Fixed off by one error in <b>sort -c</b> output,
  49fixed <b>sed -z</b> and other sed cases where Linux has never obeyed
  50Posix (N at EOF does a default print, l escapes \n),
  51<b>patch --dry-run</b> should no longer create or delete files,
  52<b>gzip</b>/<b>zcat</b> couldn't handle concatenated archives,
  53the ./configure of gmake 4.3 depends on <b>test</b> treating one argument -a as a synonym for -e,
  54autodetectiong compression types in <b>tar</b> should be more reliable now
  55and sanitize weird path corner cases (like .. past /) better,
  56<b>scripts/make.sh</b> (and thus "make toybox") should no longer truncate log
  57files stderr is redirected to,
  58<b>httpd</b> now handles ? and # in URLs,
  59<b>xxd</b> disables columns for -c 0 and groups for -g 0 (so you can get
  60a long interrupted string of hex digits out of it), two fixes to
  61<b>mountpoint</b> (the conversion to same_file() was inappropriate because
  62the logic isn't quite the same, and -q should also quiet "not found" errors),
  63fixed <b>httpd</b> not always displaying index.html files instead of directory
  64contents, fixed <b>comm -</b> not recognizing it as stdin, and multiple fixes to
  65<b>timeout</b> which now kills process group and thus child processes,
  66isn't suspended by SIGTTIN, and recursive commands it calls don't
  67inherit an inappropriate SIGCHLD handler.</p>
  68
  69<p>Yi-Yo Chiang fixed loopback <b>mount</b> (the recent switch to xrunread()
  70kept the newline from losetup's output) and added <b>tail +123</b>
  71(old-style synonym for tail -n +123).
  72Daniel Mentz added scmversion to the <b>modinfo</b> tag list (Android uses
  73it for external modules). Alexander Holler fixed <b>su</b> to not require
  74/etc/shadow when run as root, and nomas2000 reported an inverted test in
  75<b>date</b>'s check for trailing rubbish. Daniel Mentz fixed an off by
  76one in <b>grep -f</b> that discarded the last character of the pattern file's
  77contents. Kelvin Zhang made <b>modinfo</b> better at handling symlinks.
  78Antoni Villalonga fixed <b>fmt</b> on 64 bit big endian systems (like s390x and
  79powerpc). Li Cheng fixed <b>mount</b>'s type detection not autodetecting
  80the need to bind mount files when type "none" was specified in fstab.
  81Tomasz Sterna reported that <b>modprobe</b> resolving dependencies shouldn't
  82feed a NULL pointer to the syscall's options arguments (the kernel goes -EWTF),
  83and Vincent Donnefort made modprobe work when /proc/modules isn't available.</p>
  84
  85<p><u>Library</u>:
  86The TOYFLAG_ARGFAIL() exit value is now the default <b>error_msg()</b> sets,
  87<b>sendfile()</b> now falls back to the read/write loop for any error (not just
  88EINVAL, the kernel sends ENOSYS and EXDEV and who knows what else),
  89<b>xgetrandom()</b> can now return arbitrary amounts of data (looping internally
  90as necessary),
  91<b>xwaitpid()</b> returns status 127 for cases (like bad PID) which don't
  92return a status for the PID,
  93several fixes to <b>xabspath()</b> as part of the readlink/realpath work,
  94new <b>octal_deslash()</b> to remove octal escapes common in kernel strings
  95(ala /proc/mounts),
  96the <b>FLAG_x</b> macros always use a 64 bit type now (so you don't have to
  97worry about toys.optflags &= ~FLAG_x; blanking the top 32 bits),
  98replaced memcmp() with new library function <b>smemcmp()</b> to placate ASAN
  99when returning first difference in known differing arguments of unequal length.
 100Elliott added three more filesystem types to <b>fs_type_name()</b> which is
 101used by <b>stat -f</b> and friends.</p>
 102
 103<p><u>Mkroot</u>:
 104New <b>scripts/test_mkroot.sh</b> runs each target under qemu to confirm
 1051) it boots, 2) the block device works, 3) networking works, and 4) the clock is set
 106reasonably. It runs them in parallel, with an httpd instance on the host to
 107fetch a file from, and a timeout to detect hangs.
 108New <b>scripts/root/overlay</b> package copies $OVERLAY directory into
 109target filesystem, ala <b>scripts/mkroot.sh overlay OVERLAY=$PWD/blah</b>
 110The init script now mounts any /dev/?da on /mnt and when /mnt/init exists
 111sets $HANDOFF to that instead of running a shell prompt, so you can provide
 112automated control images that run code in the emulator automatically. (The
 113emulator process still when whatever $HANDOFF called exits.)</p>
 114
 115<p>Some kernel changes finally made it upstream so we can drop workarounds
 116for them, such as kernel commit
 117<a href=https://github.com/torvalds/linux/commit/f8f0d06438e5>f8f0d06438e5</a>
 118fixing the <a href=https://landley.net/notes-2014.html#13-08-2014>longstanding</a>
 119"allnoconfig has =y" issue so miniconfig doesn't need to add a magic #comment
 120line forcing a symbol OFF anymore. The remaining kernel patches (including
 121several maintained locally but hadn't merged into mkroot) moved to a
 122<a href=https://github.com/landley/linux/tree/mkroot-6.1>seperate repository</a>
 123so mkroot.sh doesn't have to call sed on the build snapshot. (They're are also
 124included in the mkroot binary release directory, and were all submitted to lkml
 125months if not years ago.)</p>
 126
 127<p>The host airlock setup (creating the temporary restricted environment
 128mkroot builds packages in both so they don't pick up strange dependencies from
 129the host, and to prove the build can use the toybox commands)
 130removed a bunch of commands (dd, diff, vi, xzcat, ar, nm) no longer
 131needed by the linux-6.1 build. The kernel patches also remove the need
 132for gcc (it can use the "cc" symlink all modern distros install, and
 133autodetects whether that points to gcc or clang) and bc (which is just
 134<a href=https://landley.net/notes-2013.html#18-05-2013>generally obsolete</a>).
 135Left those two in for one more release, but NEXT time building a vanilla
 136kernel without the <a href=https://landley.net/toybox/downloads/binaries/mkroot/0.8.9/linux-patches/0001-try-generic-compiler-name-cc-before-falling-back-to-.patch>gcc removal patch</a>
 137and <a href=https://landley.net/toybox/downloads/binaries/mkroot/0.8.9/linux-patches/0004-Replace-timeconst.bc-with-mktimeconst.c.patch>bc removal patch</a>
 138may require you to add <b>HOST_EXTRA="gcc bc"</b> to the mkroot command line
 139for the airlock build to add extra symlinks to root/build/airlock.</p>
 140
 141<p>Added a kernel build for powerpc64 big endian,
 142m68k and powerpc now support "run-qemu.sh -hda file.img", and m68k only
 143has one /dev/?db (disabled Apple Desktop Bus in the config).
 144Added basic module support (the kernel build saves modules in modules.cpio.gz
 145so it can extract them when you rebuild the root filesystem but not the kernel)
 146and added <b>scripts/root/tests</b> which (among other things) builds two
 147innocuous modules that depend on each other (fscache and cachefiles) for
 148modprobe/insmod testing that hopefully won't break anything else. (If
 149there was a scripts/root/pending/ tests would be in it: it does not actually
 150run the test suite under mkroot yet, because toysh needs work.)</p>
 151
 152<p><u>Pending</u>: Elliott made <b>strace</b> build on 32-bit x86.
 153James Farrell added <b>diff -f</b>.
 154Alexander Holler fixed two parameter substitution bugs in <b>toysh</b>.
 155Rob fixed assignment suppression, &amp;&amp; and || parsing, math
 156priority, a number of ASAN test failures, untangled and commented the
 157brace expansion logic, and other ongoing shell work.</p>
 158
 159<p>A large rewrite of diff happened but couldn't be checked in unfinished
 160because if people are using stuff out of pending, how do you do a pending for
 161pending? (Meta-pending?)</p>
 162
 163<p><u>Cleanup</u>:
 164<b>cksum</b>, <b>od</b>, <b>file</b>, <b>getopt</b>, <b>gpiod</b>,
 165<b>nbd-client</b>, better output messages for <b>host</b>,
 166and make <b>base64</b> use line buffering.
 167Moved -Wno-string-plus-int into portability.sh and had it only apply to
 168clang. Moved num_cache() out of lib into netstat (its only user).</p>
 169
 170<p>In the test suite plumbing, $SKIPNEXT was replaced by <b>$SKIP</b> holding
 171a count that counts down with each test (so you can easily skip a block of
 172tests). Fixed <b>txpect</b> to actually listen to error code (oops) and to not
 173complain about multiple X. The old dochroot() and mkchroot() shell functions were removed from
 174scripts/runtest.sh because they required root access to use (which mkroot
 175does not). More conversion to use <b>testcmd</b> in individual command.tests
 176(both to remove workarounds for shell builtins, and to be more concise).
 177Remove redundant cleanup from several tests (which run as proper child processes
 178now, and the work directory is deleted and recreated between each, so
 179individual tests need less cleanup).</p>
 180
 181<p><u>Portability</u>: More tests now pass on MacOS and FreeBSD
 182(weird filesystems, root vs wheel group name, dealing with headers that
 183define stdin as a macro and don't define PATH_MAX, the ancient bash in MacOS
 184doesn't understand &amp;&gt;&gt;, won't let normal users use "chmod +s",
 185symlinks don't have all permission bits set, no O_PATH and the Linux definition
 186is used for something else, and so on), at least half of which Elliott did.
 187Not sure we've quite got Mac and BSD
 188doing "gmake test_singlecommand" right yet, but it's a lot closer.
 189Added all the commands to <b>make bsd_defconfig</b> that compile on FreeBSD 13
 190(which is no guarantee they WORK).
 191ASAN=1 should work in more places  now, including <b>make_test_command</b>.
 192The build uses $CC insted of cc in more places, and doesn't apply $ASAN
 193for $HOSTCC (the legacy kconfig/ directory stands no chance).</p>
 194
 195Fedora inexplicably sticks \x09 style hex escapes into the dmesg output and
 196expects them to be parsed.
 197Android is still running the test suite under mksh which doesn't
 198understand bashisms like &lt;(command). LLVM insisted that "ptr - 1" could
 199never be NULL (demonstrably untrue) and needed some (unsigned long) typecasts
 200to force it to actually do the math.
 201Khem Raj aliased timer_settime to timer_settime64 when the first isn't
 202available. Antoni Villalonga debianized some install paths and added
 203a bunch of --longopt synonyms for existing options that package builds
 204use.</p>
 205
 206<p><u>Documentation</u>: The <a href=help.html>help page</a>
 207generation logic (toybox help -av) now filter out invisible aliases and
 208outputs "See command" for visible aliases, plus fixes for broken corner
 209cases like <b>nohup --help</b>.
 210The <a href=https://landley.net/toybox/git>local git repo</a>
 211now has a simple index.html generated by a
 212<a href=https://github.com/landley/toybox/blob/0.8.9/scripts/git-static-index.sh>small shell script</a>,
 213linking to the <b>git format-patch</b> files for
 214each commit, applicable via <b>git am</b>.</p>
 215
 216<p>There was a
 217<a href=https://github.com/landley/toybox/pull/244>bug report</a> that specifying the cross compiler prefix twice
 218includes the prefix twice (ala CROSS_COMPILE=prefix- CC=prefix-cc results in
 219prefix-prefix-cc) which is what happens when you ask it to do that, yes. It's now
 220<a href=faq.html#cross>better documented</a>. (See also
 221<a href=https://lkml.iu.edu/hypermail/linux/kernel/2202.0/01505.html>this kernel patch</a>.)</p>
 222
 223<p>Yi-Yo Chiang's losetup fix spawned an email thread where the
 224android developers <a href=http://lists.landley.net/pipermail/toybox-landley.net/2022-August/029139.html>explained</a> why Android doesn't (and can't)
 225use devtmpfs. Rob forwarded a link to linux weekly news to see if he could get
 226any attention from the kernel guys, but they didn't reply.</p>
 227
 228<a name="12-08-2022" /><a href="#12-08-2022"><hr><h2><b>August 12, 2022</b></h2></a>
 229<blockquote><p>
 230Huge as office blocks, silent as birds. They hung in the air exactly the same
 231way that bricks don't.</p>
 232<p>- The Hitchhiker's Guide to the Galaxy</p>
 233</p></blockquote>
 234
 235<p><a href=downloads/toybox-0.8.8.tar.gz>Toybox 0.8.8</a>
 236(<a href=https://github.com/landley/toybox/releases/tag/0.8.8>git commit</a>)
 237is out, with prebuilt <a href=downloads/binaries/0.8.8>static binaries</a> and
 238<a href=downloads/binaries/mkroot/0.8.8>mkroot images</a>
 239bootable under QEMU (built using vanilla linux-5.19).</p>
 240
 241<p>No new or promoted commands this time. (Time based releases mean NOT
 242holding the release to wait for things like that.)</p>
 243
 244<p>Thanks to Google for sponsoring the project's maintainer to focus on toybox
 245for a while. (They want a hermetic build, I want Android to build under
 246Android. Working on it...)</p>
 247
 248<p><u>Features</u>:
 249The new <b>timeout -i</b> option kills a command for inactivity (output
 250to stdout resets the timer).
 251Basic support for <b>tar --xform</b> (but not yet the special tar-only s///x
 252options).
 253--long options now support unambigous abbrevations by default, so you can
 254go "ls --col" and it'll figure out you mean --color.
 255Added <b>blkid -o</b> understanding "full", "value", and "export" output
 256formats. Added <b>nsenter -C</b> (cgroup namespace) and <b>-a</b> (enable all
 257supported namespaces).
 258Added <b>mount -R</b> and switched the bind mount default to be recursive
 259(it still autodetects --bind and --loop mounts based on the source/target type,
 260all --bind or --rbind do is switch the recursive flag on and off).</p>
 261
 262<p>Elliott implemented <b>tar --null</b>,
 263and taught <b>file</b> to recognize some Linux kernel images, and
 264binaries for Linux's <a href=https://github.com/torvalds/linux/commit/fa96b57c1490>newly merged</a>
 265arch/loongarch (a <a href=https://linleygroup.com/newsletters/newsletter_detail.php?num=4623>mips64 variant</a>
 266from <a href=https://phoronix.com/news/LoongArch-MIPS-Copy-Kernel>china</a>).</p>
 267
 268<p><u>Bugfixes</u>:
 269Several cleanups to taskset, mostly due to reviewing it before making
 270an explainer video, but the review found an inappropriate use of
 271TOYFLAG_STAYROOT that got fixed.
 272Argument parsing for <b>unshare</b> and <b>nsenter</b> should now match
 273debian's (I.E. "unshare -npu" works the same as "unshare -n -p -u" not
 274"unshare --net=pu").
 275<b>kill 1</b> no longer says "unknown pid 1: not permitted" (it
 276says "bad pid" like everywhere else).
 277<b>test -x</b> and friends now test live access, not permission bits.
 278(Root ignores a lot of permissions, chmod u-x blocks you from executing your
 279own file even if g+x and o+x should let you access it, things like selinux can
 280further break the unix permission model, etc).
 281
 282<p>In lib/ a fencepost error in <b>chomp()</b> could read one byte past the start of the string, and
 283<b>xpclose_both()</b> no longer closes the parent's stdin and stdout
 284when they're explicitly passed through to the child.
 285In scripts/runtest.sh, moving tests into subshells broke failure reporting with VERBOSE=all.
 286While "make install" was creating directories when $PREFIX pointed to a
 287nonexistent directory, "make toybox" didn't even though it tried to place the
 288output file there.</p>
 289
 290<p>James Farrell switched the test suite from detecting root via $UID to
 291$(id -u). (It's not a security issue, this just controls which tests we attempt.
 292It's another workaround for missing features in mksh.)
 293Taolaw spotted that httpd could segfault when xabspath() returns NULL
 294(for error conditions such as infinitely looping symlinks
 295or "." not resolving when the current directory was deleted).</p>
 296
 297<p><u>Documentation</u>:
 298We <a href=design.html>recorded</a> the
 299move from C99 to the C11 standard, which was actually made a while ago:
 300the typecast constants we've
 301been using for a while are a 2011 feature not available in the 1999 edition
 302of the C standard. (This came up when we had to move to C11's "_Noreturn" to work around a compiler bug in llvm.
 303Yes __attribute__((__noreturn__)) should be identical, that's the bug part.)</p>
 304
 305<p>The website's nav bar got updated (among other things, linking to
 306our <a href=https://www.youtube.com/channel/UC4VFy3wc1nzq5tUHhiti6fw>youtube
 307channel</a>), and the FAQ entry on cross compiling got
 308expanded.
 309The toys/example commands moved from "make list" to "make list_example"
 310(since they're not enabled in defconfig either).
 311The cross compiling example command line in "make help" now sets
 312LDFLAGS=--static instead of putting that in CFLAGS.</p>
 313
 314<p>The web archive is back, and since the missing messages were
 315never delivered (the mailserver trying to deliver them timed out and bounced
 316them back after a couple days) this means unlike previous failures the web
 317archive isn't out of sync with reality: the hole was the list actually being
 318down until Dreamhost tech support figured out what they broke this time.</p>
 319
 320<p><u>Pending</u>:
 321More work on toysh: case/esac was confused by nested flow control blocks,
 322support for [[ &lt; &gt; =~ ]] tests, implemented the rest of the $((math))
 323operators and support for ((math)) tests and "for ((math;math;math))" loops,
 324and so on. (The next toysh goal is to get the test suite running under mkroot, but
 325other stuff keeps coming up.)
 326James Farrell upgraded diff, adding three line format options and support
 327for diffing FIFOs.
 328Rob did several rounds of dd cleanup, then got distracted by diff cleanup
 329which turned into a complete rewrite of diff.
 330Neither was ready to promote in time for this release though (although the release
 331got delayed almost a week before Rob admitted that).</p>
 332
 333<p><u>Library</u>:
 334The <b>lib/args.c</b> command line parser now has two optional modifiers
 335(" " and ";") for each argument type, providing 4 combinations (both unset,
 336both set, one of each set), and all 4 now have defined semantics and
 337tests/skeleton.test exercises them. (This was motivated by <b>unshare</b>
 338needing a new combination to match the debian behavior.)
 339<b>lib/args.c</b> also added "~" to collate bare longopts (so multiple bare
 340longopts can be synonyms setting the same flag or argument variable).</p>
 341
 342<p>New function <b>xrunread()</b> which launches a child function, writes a
 343string to its stdin, and returns everything the child wrote to stdout as another
 344string (or NULL if it couldn't run or exited with an error), leaving
 345stderr alone (so it goes to the same place the parent's goes).</p>
 346
 347<p>New functions <b>same_file()</b> and <b>same_def_ino()</b> to simplify
 348file metadata comparisons. (If you're wondering why something so simple should
 349have a function encapsulating the logic, this release also has at least 3
 350different bugfix commits for thinkos from switching all the commands over
 351to use them instead of doing the test ourselves. All missing/extra ! or
 352&& vs || level stuff.)</p>
 353
 354<p>The <b>exit_signal()</b> handler now blocks signals so <b>sigatexit()</b> won't
 355re-enter the list when it receives two different killer signals. (Since
 356<b>xexit()</b> capture uses siglongjmp() this should clean itself up
 357when not actually exiting.)</p>
 358
 359<p><u>Plumbing</u>:
 360Several warning suppressions due to things like llvm's fortify finding
 361false positives, gcc's "int and long are identical on 32 bits!" warnings
 362(yes... they are), the MacOS build being crotchety about vfork()... All verified
 363non-issues at present.</p>
 364
 365<p>The build plumbing now puts more effort into resolving dependencies and figuring
 366out when stuff needs to be rebuilt. This includes noticing when the
 367toolchain we're building with changes flags, and outputting a warning
 368when it detects the need to run "make oldconfig" again. (At the moment
 369calling kconfig/ from scripts/ would be a layering violation: I need to
 370rewrite kconfig from scratch before it can do that automatically.).</p>
 371
 372<p>Removed most compile time probes from <b>scripts/genconfig.sh</b>,
 373replaced with things like #ifdef (_NR_syscall) and
 374#if __has_include(&lt;utmpx.h&gt;), or acknowledging when a probe is a bug
 375workaround for a specific libc version and testing for that instead,
 376plus simply deleting some config options due to the 7 year horizon expiring
 377so we can trust it to be there now.</p>
 378
 379<p>Usual batch of musl workarounds for behavior that
 380<a href=https://github.com/landley/toybox/commit/9a5d7a063c8c>doesn't match</a>
 381glibc and bionic or where they've been
 382<a href=https://www.openwall.com/lists/musl/2022/07/27/5>arguing about</a>
 383stuff for years without actually fixing it.
 384Ed Maste of FreeBSD added some FreeBSD checks/cases to
 385lib/portability.[hc], mostly in places we already supported Apple or
 386OpenBSD.
 387The top level "configure" is now re-entrant (shouldn't append stuff
 388to enviornment varibles like $CFLAGS twice when read twice).
 389The MacOS defconfig now includes a lot more commands.</p>
 390
 391<p><u>Test suite</u>:
 392The diff for failing tests is now output after the FAIL line instead
 393of before, "make tests" now runs tar.test by default,
 394a new skeleton.test uses toys/example/skeleton.c to exercise the
 395lib/args.c command line parsing plumbing,
 396scripts/runtest.sh now uses do_pass and do_fail consistently,
 397and Rob keeps accidentally checking in bashisms and Elliott keeps submitting
 398patches to remove them because android runs the tests under mksh and
 399toysh isn't ready yet.</p>
 400
 401<p>Work in progress to make TEST_HOST=1 pass on newer debian, which is a moving
 402target. (The tests didn't change, but the behavior of the gnu utilities
 403is not consistent. Or standardized. Or coherent. Or predictable. Or sane.)
 404Some minor changes to commands like <b>xxd</b> and <b>cmp</b> to
 405make their output look more like what debian's currently producing.</p>
 406
 407<p>Making the tests pass on MacOS is harder: that calls group 0 "wheel"
 408instead of "root" (so things like "tar.test" don't get the same result),
 409stuff like "du" or "ls" often cares what filesystem you're mounted on
 410(and MacOS hasn't even got the same ones _available_), 
 411and we even found a mac kernel bug where extending a sparse
 412file sparsely allocates the previous length as actual zeroed disk space
 413(ala "truncate -s 3m file; truncate -s 5m file; du file" says it's using
 4143 megabytes of disk space). There's probably more but
 415that's where we stopped looking for now: since Rob doesn't have a mac
 416to test on and the github tests don't reliably run when new commits are
 417pushed (skipped when the servers are busy), fixing this hasn't
 418been a priority. Lots of little red X in the github history because of this,
 419though...</p>
 420
 421<p><u>Mkroot</u>:
 422The output file names are now consistent across architectures: each
 423root/$TARGET directory now contains an <b>fs</b> subdirectory with the
 424expanded root filesystem, packaged into <b>initramfs.cpio.gz</b>,
 425a <b>linux-kernel</b> built from <b>linux-fullconfig</b> which is built
 426from <b>linux-miniconfig</b>, and a <b>run-qemu.sh</b> script to launch
 427it under qemu. No more architecture name in the filenames (that's what
 428the directory name is for), and no more "bzImage" or "vmlinux" depending
 429on what filename the kernel build produced.</p>
 430
 431<p>The fs/init script has better chroot support (not warning about /dev/shm and
 432/dev/pts already existing).
 433The airlock build no longer includes ftpd (you can use
 434httpd and wget, scp out of dropbear, mount a network filesystem from the
 435host, or if you really want to still use ftp can match ftpget and ftpput).
 436When mkroot is downloading extra packages (for example in the dropbear build),
 437it falls back to the host wget if it can't fetch with toybox's version.
 438(This is because toybox hasn't got built-in https support yet, it's on the
 439todo list.)
 440Updated the dropbear build to the current version and fixed a compiler
 441path issue.
 442Fixed version skew in the <a href=http://lkml.iu.edu/hypermail/linux/kernel/2110.3/00278.html>workaround</a>
 443so the Linux kernel build doesn't require a third ELF package (but only
 444for x86, not for any other architecture).</p>
 445
 446<a name="23-07-2022" /><a href="#23-07-2022"><hr><h2><b>July 23, 2022</b></h2></a>
 447<p>Sorry the mailing list has been down since the 19th, Dreamhost
 448<a href=https://landley.net/notes-2022.html#22-07-2022>happened
 449again</a>. (First the web archive was empty, then it was down, then it was
 450restored from a backup taken before the "upgrade" but was not updating with
 451new messages, which they denied was an actual problem but now timeout bounce
 452messages from all the attempts to send to it since it broke have started
 453coming back saying a path doesn't exist on their server...)</p>
 454
 455<p>Dreamhost tech support has been informed of the issue a half-dozen times,
 456and did at least replace the "Currently there are no messages" page with a
 457backup from before they broke it on the first day. Unfortunately they have yet
 458to acknowledge anything else is wrong with it since then, and don't seem to
 459have support staff working over the weekend. This used to be an
 460<a href=https://landley.net/dreamhost.txt>annual</a>
 461<a href=https://landley.net/dreamhost2.txt>event</a> for them,
 462but they hadn't done a truly major breakage in
 463<a href=#12-21-2015>a while</a>. I _think_ the 11 month gap that last link
 464mentions is still missing, but the server's down again (or at least timing
 465out) so I can't check.</p>
 466</span>
 467
 468<a name="06-05-2022" /><a href="#06-05-2022"><hr><h2><b>May 6, 2022</b></h2></a>
 469<blockquote><p>
 470The mighty ships tore across the empty wastes of space and
 471finally dived screaming on to the first planet they came across - which
 472happened to be the Earth - where due to a terrible miscalculation of scale
 473the entire battle fleet was accidentally swallowed by a small dog.</p>
 474<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
 475
 476<p><a href=downloads/toybox-0.8.7.tar.gz>Toybox 0.8.7</a>
 477(<a href=https://github.com/landley/toybox/releases/tag/0.8.7>git commit</a>)
 478is out, with prebuilt <a href=downloads/binaries/0.8.7>static binaries</a> and
 479<a href=downloads/binaries/mkroot/0.8.7>mkroot images</a>
 480bootable under QEMU (using vanilla linux-5.17).</p>
 481
 482<p>Rob also <a href=downloads/binaries/toolchains/latest>uploaded</a>
 483the prebuilt binary cross and native compilers used to build those
 484mkroot images (there's a <a href=downloads/binaries/toolchains/latest/README>README</a>
 485in there), and started producing
 486<a href=https://www.youtube.com/channel/UC4VFy3wc1nzq5tUHhiti6fw>tutorial videos</a> explaining the
 487toybox commands, their implementation, toybox's infrastructure...</p>
 488
 489<p>The commands <b>host</b>, <b>wget</b>, <b>openvt</b>, and
 490<b>deallocvt</b> were cleaned up and promoted.
 491Elliott contributed the new commands <b>uclampset</b>, <b>gpiodetect</b>,
 492<b>gpioinfo</b>, <b>gpioiget</b>, <b>gpiofind</b>, and <b>gpioset</b>,
 493and Rob wrote a simple <b>httpd</b> (largely to give wget.test something to
 494talk to).</p>
 495
 496<p>One command was removed: <b>catv</b> didn't really serve a purpose
 497(everybody just uses <b>cat -v</b>).</p>
 498
 499
 500
 501
 502<p><u>Features</u>:
 503<b>top</b> can now move the list with the LEFT/RIGHT cursor keys (changing the
 504sort field is now SHIFT LEFT/RIGHT). Added <b>find -samefile</b>,
 505<b>cmp -n</b> (and wired up skip1 skip2),
 506<b>tar --strip components</b> (and tar --owner and --group can now specify :UID and :GID making tar.test more portable),
 507<b>lsusb</b> and </b>lspci</b> now read /etc/{usb,pci}.ids[.gz] (when
 508available) to provide human readable device descriptions,
 509<b>ifconfig</b> can now rename interfaces.
 510Moritz Weber added <b>wget</b> post support for form data.
 511In toysh, <b>cd</b> got updated ("cd -" jumps to $OLDPWD,
 512and cd can now handle deleted directories).</p>
 513
 514<p><u>Bugfixes</u>:
 515<b>cp --preserve xattr,context</b> work together now, and
 516Dario Freni fixed <b>cp --xattr</b> to avoid copying selinux context.
 517The <b>sort -u</b> option now implies -s, <b>wget -O -</b> now writes to stdout,
 518<b>pwget -B</b> removes more characters other implementations don't emit,
 519</b>time</b>'s realtime output was not properly zero padding the fractional
 520part, and <b>sed</b> now works with multiple w to the same file.
 521Fixed <b>nl -E</b>, <b>realpath -e</b>, <b>find</b> -newerXY and @time.nanosecond parsing,
 522Various <b>sh</b> (it handles ${X: -1:1} properly now,
 523which is not the same as ${X:-1:1} because :- has special meaning).
 524The <b>hello</b> command in the examples directory now wait for a keypress before
 525exiting when run as PID 1 (because calling wait() returned "No child process"
 526immediately.)
 527The multiplexer's <b>toybox command --help</b> once again shows the command's
 528help (not toybox --help).
 529Elliott fixed a strip bug in the MacOS build, taught <b>file</b> not to be confused
 530by ELF BSS sections but to otherwise detect more invalid ELF files,
 531and fixed an <b>xargs</b> bug where stdin was inappropriately O_CLOEXEC.
 532Github user hg42 reported a bug where <b>tar</b> was looping trying to remove ".."
 533at the start of names.</p>
 534
 535<p><u>Library</u>:
 536Taught flags.h to #define its own CLEANUP macros so switching FLAG()
 537contexts requires fewer steps.
 538When args.c is just enforcing max/min argument count but collecting no
 539options, imply "?" so "command -123" isn't an unknown option.
 540Moved lib/help.c into main.c (nothing else in lib/ had to rebuild
 541when the config changes which commands are enabled).
 542The remains of linestack.c were renamed utf8.c, and
 543toys.h now includes sys/syscall.h so commands don't have to.</p>
 544
 545<p>Yank things with only one remaining user: moved get_line()
 546to patch.c, struct str_len to bc.c, and struct ptr_len to ps.c
 547Removed tty_esc() and tty_jump() wrappers.
 548Various cleanups to lib/password.c and lib/tty.c.
 549Removed the PEDANTIC_ARGS config option which just set a macro that was never
 550used, xsignal_all_killers() now takes SIG_DFL as the reset value instead
 551of explicitly checking for NULL,
 552chomp() now removes any number of trailing \r\n, and it's now safe to
 553llist_pop(0) just like dlist_pop(0).
 554LongPing Wei improved support for copy_file_range().</p>
 555
 556<p><u>Build</u>:
 557If you don't set $PREFIX, <b>make install</b> now defaults to $PWD/install
 558(which does not require root access to write to, and you can tar up your
 559pile of symlinks from there).
 560More command sub-options removed (NETCAT_LISTEN and CAT_V).</p>
 561
 562<p>Some of the uglier scripts/make.sh plumbing moved into scripts/portability.sh
 563(which should now be safe to source twice),
 564and moved almost all config varible definitions to "configure" now. The
 565new $GENDIR variable theoretically lets the "generated" directory live
 566somewhere else (not very tested yet).
 567Various tweaks to scripts/make.sh to try to avoid unnecessary rebuilds
 568and library probes when toolchain and config haven't changed.
 569Added -Wno-pointer-sign to $CFLAGS because passing an unsigned char * to a
 570function that expects char * is not a problem.
 571
 572<p>The airlock install in scripts/install.sh had the $PENDING items sorted
 573into "started" and "not started". Add git and bash to $PENDING (used
 574out of $PATH by --version and the powerpc64le build, respectively).</p>
 575
 576<p><u>Cleanup</u>:
 577Removed the lib/linestack.c plumbing (which was never finished and hadn't
 578been touched in years).
 579Merged lspci.c into lsusb.c (so they can share config file reading
 580infrastructure), and merged chvt/deallocvt into openvt.c.
 581Minor cleanups to ping, fsync, ionice, pmap, truncate, timeout, tty,
 582factor, mount.
 583Went through and replaced \033 with \e in strings (since clang supports
 584it and \033 is just awkward). Added LL to constant 0 in the FLAG macros
 585to prevent gcc from warning that 0<<32 might produce 0.
 586Moved llvm's -Wno-string-plus-int into configure instead of probing
 587for it, since gcc no longer dies when asked to suppress an unknown warning.</p>
 588
 589<p><u>mkroot</u>:
 590the ANSI escape to undo QEMU's wordwrap bug is now in both the init script
 591and launch wrapper script (so wordwrap isn't broken while running the
 592emulated system, nor if the emulated system fails to run).
 593If devtmpfs was already mounted on /dev, we still need to create dev/pts.
 594The init script now includes the linux kernel bug workaround to enable ping
 595support. The armv7 target enabled CONFIG_ARM_LPAE in the kernel to work around
 596around <a href=https://bugs.launchpad.net/qemu/+bug/1790975>a QEMU bug</a>
 597that's gone unfixed for 4 years. Enable network support in m68k target.
 598The mcm-buildall.sh script got some minor updates, and the resulting
 599cross and native compiler binaries are now downloadable so you don't have
 600to run the build if you don't want to.</p>
 601
 602<p><u>Docs</u>:
 603"toybox --help" no longer says "see toybox --help".
 604There's an ANSI escape cheat sheet at the start of lib/tty.c.
 605Reworded some command directory READMEs, and tweaked menuconfig display.
 606Converted www/doc/mount.html and www/doc/ext2.html from text to html.
 607Freshened up design.html a little (and fixed some typos pointed out by
 608Peter Ennis).
 609Replaced the git internals video URL in roadmap.html with a different
 610performance of the same talk that hasn't been made private.
 611Added the kconfig-language.txt from Linux-2.6.12 to the kconfig/ directory.</p>
 612
 613<a name="30-11-2021" /><a href="#30-11-2021"><hr><h2><b>November 30, 2021</b></h2></a>
 614<blockquote><p>
 615Busy? We'll I've just got this bulldozer to lie in front of or otherwise it'll
 616knock my house down but otherwise no, not especially, why?</p>
 617<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
 618
 619<p><a href=downloads/toybox-0.8.6.tar.gz>Toybox 0.8.6</a>
 620(<a href=https://github.com/landley/toybox/releases/tag/0.8.6>git commit</a>)
 621is out, with prebuilt <a href=downloads/binaries/0.8.6>static binaries</a> and
 622<a href=downloads/binaries/mkroot/0.8.6>mkroot images</a>
 623bootable under QEMU (using vanilla linux-5.15).</p>
 624
 625<p><u>Make root</u>: The <a href=faq.html#mkroot>system builder</a> got upgraded, with better
 626logging and layout of the output directory, plus <a href=https://github.com/landley/toybox/blob/0.8.6/scripts/mkroot.sh>scripts/mkroot.sh</a>
 627should be easier to read and understand now.
 628(Also added basic support for the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-August/012497.html>qualcomm hexagon</a>,
 629although it just builds a chroot filesystem, you can't boot a hexagon kernel under qemu
 630<a href=https://lists.nongnu.org/archive/html/qemu-devel/2021-07/msg06430.html>quite yet</a>.)</p>
 631</p>
 632
 633<p><u>New commands</u>: Rob added <b>linux32</b>.
 634Dan Brown added <b>sha256sum</b> (and the related 224/384/512 variants).
 635In pending, Elliott contributed <b>strace</b> and
 636Moritz Röhrich contributed <b>hexdump</b>.</p>
 637
 638<p><u>Features</u>:
 639Elliott added <b>date -s</b>, <b>pmap -p</b>, <b>tail -F -s</b>, <b>kill -0</b>,
 640<b>reboot/halt/poweroff -d</b>, taught <b>file</b> to
 641recognize Mach-o universal binaries, made Android's <b>log</b>
 642take input from stdin when it has no arguments, added <b>tail --bytes --lines</b>
 643(the --long synonyms for -c and -n),
 644added <b>i2cdetect -q</b>, taught <b>i2cdump</b> to write XX for
 645read failures, made <b>timeout</b> use the monotonic clock,
 646and modernized the output of <b>uptime</b>.</p>
 647
 648<p>Rob added <b>find -quit -lname -ilname</b> and <b>-d</b> as a synonym for -depth,
 649taught <b>cut -d $'\n'</b>
 650to cut by line and added posix <b>cut -nb</b> (which wraps to the start of -c
 651utf8 characters), added <b>cpio --ignore-devno --renumber-inodes</b>,
 652taught <b>cal</b> to use 3 arguments (day month year), added <b>tar --selinux</b>
 653support and a "had errors" message at the end, made <b>uname -o</b>
 654say "Toybox", and fixed the <b>grep -Fw</b> combination (used by the 64 bit
 655powerpc kernel build).</p>
 656
 657<p>Ella-0 added <b>split -n</b>. Peter Collingbourne added <b>grep -L</b>.</p>
 658
 659<p><u>Outside world</u>: The 0BSD license is <a href=https://lwn.net/Articles/868882/>now part</a> of the
 660<a href=https://standards.iso.org/ittf/PubliclyAvailableStandards/>ISO-5962</a>
 661standard. Of the currently <a href=https://github.com/search?q=license%3A0bsd&type=Repositories&ref=advsearch>38k 0BSD licensed projects on github</a> toybox
 662isn't even the most popular, it's fourth.</p>
 663
 664<p>Elliott asked that toybox's <b>cut -DF</b> feature (a cheaper
 665alternative to <b>awk '{print $3}'</b> be supported by other packages so
 666it might someday make it into posix, so Rob
 667<a href=http://lists.busybox.net/pipermail/busybox/2021-June/088886.html>submitted it to busybox</a>
 668where it was <a href=https://git.busybox.net/busybox/commit/?id=0068ce2fa0e3>eventually merged</a>.</p>
 669
 670<p>Added a local <a href=https://landley.net/toybox/git>git mirror</a> for
 671those reluctant to trust Microsoft Github.</p>
 672
 673<p>In the web nav bar, our <a href=https://isfreenodedeadyet.com/>freenode</a>
 674channel moved to libera.chat
 675(like <a href=https://boingboing.net/2021/05/19/freenode-irc-staff-quit-after-new-owner-seizes-control.html>everyone else's</a>;
 676<a href=https://lwn.net/Articles/856543/>there</a>
 677<a href=https://hackaday.com/2021/05/20/freenode-debacle-prompts-staff-exodus-new-network/>was</a>
 678<a href=https://twitter.com/marcan42/status/1396280769778098185>a</a>
 679<a href=https://twitter.com/mjg59/status/1396237336988446722>certain</a>
 680<a href=https://www.devever.net/~hl/freenode_abuse>amount</a>
 681<a href=https://twitter.com/TwitchiH/status/1395756163527028739>of</a>
 682<a href=https://twitter.com/joepie91/status/1395364883303182339>drama</a>
 683<a href=https://twitter.com/RichFelker/status/1404816846734385164>involved</a>).</p>
 684
 685<p><u>Pending</u>: lots more work on <b>toysh</b>: it has basic $((math)) support
 686now, += assignment, the "declare" command and corresponding variable
 687types (declare -i, declare -u, etc), several bugfixes, greatly
 688improved support for background processes on NOMMU systems, and a bunch
 689more sh.test entries.</p>
 690
 691<p>Elliott fixed <b>modprobe</b>'s parsing of short lines. Rob fixed a
 692division by zero error when <b>vi</b> tries to edit a zero length file.
 693Anqi Dong made sure that syslogd messages sent to sockets end with a
 694newline. Eric Molitor significantly upgrade <b>wget</b> with http 1.1,
 695chunked encoding, and (external library) https (TLS 1.2) support,
 696plus several bugfixes. Michael Hope fixed <b>tftpd</b>'s option parsing.</p>
 697
 698<p><u>Documentation</u>: The roadmap.html and status.html pages
 699got noticeable updates. (status.html is now directly generated by
 700scripts/mkstatus.py, the old status.html wrapper that server-side-included
 701status.gen has been removed).
 702The <a href=design.html>design</a>
 703page provides better links to the LP64 documentation, and the
 704<a href=code.html>code</a> page explains a corner case where the filename
 705and first command name expect to match (and how to work around it if they don't).
 706Some website files weren't in the repository: two jpegs linked from html
 707files, and the www/.htaccess file that triggers the server side includes
 708in the navigation menu is now checked in. Johannes Schmitz fixed a
 709typo in about.html.</p>
 710
 711<p>The <b>shred</b> usage line now includes all options.
 712The <b>logwrapper</b> command now warns when it isn't built standalone.
 713The <b>help</b> command no longer shows a header line about toybox unless
 714it's called with no arguments. (It still shows it for command --help.)
 715The --help option no longer needs to be the first option, and is
 716checked before the "Not root" error message for restricted commands.</p>
 717
 718<p><u>Bugfixes</u>:
 719Elliott made <b>ps</b> exit with status 1 if no processes were shown (for LTP),
 720fixed an off-by-one in <b>dmesg</b> and a bounds check in <b>netstat</b>,
 721fixed <b>pidof -s</b>, and taught <b>rmmod</b> to handle multiple arguments.
 722Denys Vlasenko pointed out that <b>free -t</b> is "total" not "terabytes".
 723Samanta Navarro pointed out a leftover 32 bit limitation in argument groups.
 724Mikhail Kashkarov fixed mkdir's error handling for existing directories
 725without -p.
 726Rob fixed the 32 bit <b>readelf</b> build on Android, fixed a race condition
 727in <b>tar</b> leading
 728to the occasional spurious archiver warning,
 729<a href=https://github.com/landley/toybox/commit/127cff38809e>changed cp</a>
 730to work around a posix violation in the kernel, made xparsedate() handle more
 731whitespace and ignore trailing + or - (a NOP timezone), made <b>su</b> log the
 732right username, switched getuid() checks to geteuid() in <b>df</b> and <b>ping</b>,
 733fixed a place in the test suite where do_test()
 734could get confused by symlinks in the $PATH (calling realpath too early
 735and losing the name of the command being run),
 736made <b>env -i</b> use the old $PATH to run its command,
 737and made <b>grep --color</b> highlight all matches.</p>
 738
 739<p>Worked around a bug in glibc+gcc 9.3 where linking against librt.a
 740(for things like timer_create()) pulls in libgcc_eh.a which uses
 741threads, meaning you can't statically link against librt.a from
 742a non-threaded app. (I hadn't noticed because it works fine with the musl
 743and llvm/bionic toolchains, only fails statically linking on e.g. debian.)
 744The <a href=https://github.com/torvalds/linux/commit/d18760560593>kernel fix</a> for the cpio encrypted filesystem issue worked around in 0.8.5 made
 745it in. Similarly, the
 746FS_IOC_{GET,SET}{VERSION,FLAGS} ioctl family are <a href=https://www.spinics.net/lists/linux-api/msg49101.html>historically broken</a>
 747(taking an unsigned int argument while advertising signed long),
 748which <a href=https://github.com/landley/toybox/commit/dcfac5c983ad>we fixed</a> and reported upstream to a long
 749round of bikeshedding (sadly
 750the result seems to have been the man page maintainer
 751<a href=https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/commit/man2?id=3de87d46840d>giving up</a>
 752on documenting ioctls at all.</p>
 753
 754<p><u>Cleanups</u>:
 755<b>cut</b> switched to FLAG() macros and no longer allows -f with -F,
 756<b>eject</b> uses FLAG() macros and the linux/eject.h header,
 757prettycool@protonmail added tabs to <b>time</b>'s output, and Rob removed
 758the floating point dependency from time. Minor cleanups
 759to lsattr, lspci, nsenter, xxd, file, insmod, modprobe, arp, bootchartd, pmap,
 760comm, lsof, blkid, crond, init, userdel, makedevs, and scripts/make.sh.
 761The "int fd = fd;"
 762initialization hack to shut up gcc's "may be used uninitalized, but isn't" false
 763positive warning has been replaced by a QUIET macro that resolves
 764to nothing on llvm (which doesn't have the gcc bug).
 765Ryan Pritchard reported a bug where <b>alias ls="ls --color"</b> would prevent
 766--help or --version from being the first argument to the command, which was
 767a pain to fix without breaking things like <b>echo -- --version</b> and
 768<b>basename -s --help file--help</b>.
 769(Both toy_singleinit() in main.c _and_ get_optflags() in lib/args.c now
 770check for those.)</p>
 771
 772<p><u>Infrastructure</u>:
 773xabspath() now has a more granular (flag based) control interface.</p>
 774
 775<p><b>scripts/make.sh</b> now uses wait -n (available since bash 4.3 in February 2014)
 776to keep the processors busier during a parallel build.
 777Tests are now running in a subshell, which means functions and local
 778variables defined in one don't pollute others during "make tests".
 779
 780<p>Guo Chuang reported that lsattr won't build on an 8 year old version of
 781<a href=https://github.com/landley/toybox/pull/292#issuecomment-899895909>centos</a>,
 782which is still supported for 2 more years because Red Hat can
 783be paid to have a 10 year support horizon, longer than toybox's 7. Rob
 784fixed it (and merged a similarly ancient compilation failure fix when
 785building pending/ip.c against a 3.10 kernel),
 786but the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-August/012537.html>philosophal argument</a>
 787remains <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-September/012603.html>unresolved</a>.</p>
 788
 789<p>Standalone builds can now override the .config file location with
 790KCONFIG_CONFIG same as the multiplexer build. scripts/single.sh can
 791now cope with a missing .config file (it uses the defconfig values
 792for the CONFIG_TOYBOX_* global symbols).
 793"make install_airlock" no longer needs host versions of sha256sum and sha512sum.</p>
 794
 795<p>In the test suite, <b>VERBOSE=spam</b> now shows expands successful tests
 796like it would for errors.</p>
 797
 798<p><b>scripts/record-commands</b> got updated and logwrapper got renamed <b>logpath</b>.</p>
 799
 800<a name="19-05-2021" /><a href="#19-05-2021"><hr><h2><b>May 19, 2021</b></h2></a>
 801<blockquote><p>It was of course a descendant of these eccentric poets
 802who invented the spurious tales of impending doom which enabled the people of
 803Golgafrincham to rid themselves of the useless third of their population.
 804The other two-thirds stayed firmly at home and lived full, rich and happy lives
 805until they were all suddenly wiped out by a virulent disease contracted from
 806a dirty telephone.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
 807
 808<p><a href=downloads/toybox-0.8.5.tar.gz>Toybox 0.8.5</a>
 809(<a href=https://github.com/landley/toybox/releases/tag/0.8.5>git commit</a>)
 810is out, with prebuilt <a href=downloads/binaries/0.8.5>static binaries</a> and
 811<a href=downloads/binaries/mkroot/0.8.5>mkroot images</a>
 812bootable under QEMU (using vanilla linux-5.12, except that s390 needed another
 813<a href=http://lkml.iu.edu/hypermail/linux/kernel/2105.2/02904.html>perl
 814removal patch</a>).</p>
 815
 816<p>This development cycle had thirty commits to sh.c fixing bugs and adding
 817features to <b>toysh</b>. Shell functions and local variables are now implemented,
 818along with the "set" builtin and trace support, and job control is
 819about halfway in. The shell as a whole is <a href=https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule>maybe 80% done</a>, but still
 820has some obvious gaps like $((math)) and interactive command line
 821history/editing, plus a bunch of testing.</p>
 822
 823<p><u>New commands</u>:
 824Moritz Röhrich added <b>pwgen</b> and <b>base32</b>, and
 825Elliott Hughes added <b>unicode</b>. The
 826<b>readelf</b> command got promoted out of pending, and
 827<b>sha3sum</b> is now switched on in defconfig.</p>
 828
 829<p>Michael Christensen added <b>chsh</b> to pending, which hasn't been promoted
 830yet because it's part of a group of commands (useradd, userdel, groupadd,
 831groupdel) that should all be promoted together at some point
 832(and depend on lib/passwd.c, which could use some more work).</p>
 833
 834<p><u>Upgrades</u>:
 835<b>cpio</b> now supports extracting multiple concatenated archives.
 836<b>file</b> now recognizes android's new XML binary format.
 837<b>devmem</b> now works on nommu, can handle 0x prefxes on the address,
 838and prints
 839hex or decimal output to match the address type.
 840<b>df</b> -a now shows overmounted filesystems.
 841<b>test</b> -k checks the sticky bit.
 842Added -t to <b>install</b>, <b>cp</b>, and <b>mv</b>, and -u to <b>cp</b>
 843and <b>cpio</b>.
 844<b>find</b> -executable checks that _we_ can execute them (u+x could be owned
 845by a different user or have selinux weirdness).</p>
 846
 847<p>Elliott added several features to <b>date</b> (--iso, %:z output format, and
 848--utc as a synonym for -u), added a <b>sed</b> s///x option,
 849added units to <b>ulimit</b> output, and
 850made several UI changes to <b>hexedit</b> including a color
 851mode (which is now the default, press x to toggle between old and new
 852display), masked out the file type in <b>chmod</b> error messages and
 853taught it to handle complex modes (like u+s+s), 
 854hardened <b>file</b> and <b>readelf</b> against invalid input,
 855added a portability.h workaround for macos statvfs for <b>df</b>,
 856enabled line buffering in <b>echo</b> and <b>yes</b>,
 857and significatly sped up <b>tr</b>. Rob similarly optimized <b>seq</b>
 858and <b>count</b>.</p>
 859
 860<p><b>make tests</b> now defaults to VERBOSE=fail (and now has VERBOSE=all to
 861continue after failure, and VERBOSE=quiet to show FAIL: lines without
 862diff -u output).
 863Cleanups to tee, netstat, file, and df. Added "static" annotations in a
 864lot of commands, made more things use FLAG() macros. Ethan Sommer pointed out
 865an unnecessary wrapper function in chgrp.
 866In pending, Elliott did some cleanups and bugfixes to gettty, telnet, telnetd,
 867tftpd, init, modprobe, and dd, and added more features to vi.
 868Rob did some cleanup on modprobe.</p>
 869
 870<p><u>Bugfixes</u>:
 871Yi-Yo Chiang fixed a <b>rm</b> bug where -f could get confused by read
 872only filesystems and produce an error trying to unlink a nonexistent
 873file, pointed out <b>install</b> -D was broken when the target file
 874already exists, pointed out a <b>cpio</b> interaction with certain encrypted
 875filesystems where readlink() wouldn't write as much payload as the header
 876promised (corrupting the archive) and fixed another cpio bug where -t test
 877mode would chown() files, and fixed quoting in mount.test.
 878Elliott fixed a sequencing issue in <b>setsid</b>'s system calls,
 879fixed some <b>sed</b> tests (sed's lie-to-autoconf --version
 880output was preventing toyonly tests from triggering) and
 881fixed <b>tar</b>'s base 256 output for very large files.
 882Nir Lichtman pointed out that <b>rm</b> -p shouldn't try to rmdir / at the end,
 883Luke Frankart pointed out a typo that broke <b>test</b> checking file modes,
 884Firas Khalil Khana fixed a duplicate entry in the roadmap,
 885Ethan Sommer fixed <b>install</b> -Dt,
 886Andy Hu requested that <b>mount</b> handle mixed options like
 887-tnomsdos,smbfs.
 888Denys Nykula reported a <b>chmod</b> regression,
 889asm89 pointed out a <b>netcat</b> bug writing transactions larger than
 8904k,
 891Daniel Santana fixed <b>df</b> combining -P and -k,
 892Robin Hsu fixed <b>grep</b> getting confused by libc setting errno when
 893a library function call succeeded.
 894Puck Meerburg taught the <b>gunzip</b> code not to be confused by FEXTRA entries
 895(as described in RFC 1952) and added a test suite entry for that.</p>
 896
 897<p><b>find</b> handles empty arguments properly now (with test), <b>date</b>
 898-I with no arguments no longer segfaults, <b>grep</b> -f /dev/null no
 899longer adds a "" regex matching everything, -z no longer splits patterns
 900on \n, and -o can print zero length matches,
 901<b>cp</b> -t works with one argument and with --parents.
 902<b>readelf</b> displays properly in 80 columns and doesn't reject .bss
 903off end of file, and <b>sed</b> s///2 now works when there isn't more than one
 904match.
 905<b>tar</b> now removes empty directories where it wants to put a non-directory.
 906<b>microcom</b> sets serial device terminal attributes correctly.</p>
 907
 908<p>The prebuilt x86-64 binaries shouldn't segfault on older x86-64 processors
 909anymore (workaround for gcc bug).
 910Cross compiling was sometimes confusing install.sh because it created
 911a hyphenated "toybox-$TARGET" and then tried to make symlinks to just "toybox".</p>
 912
 913<p><u>Build</u>:
 914CONFIG_TOYBOX_I18N went away: toybox builds always support utf8 now.
 915Ella-0 added OpenBSD support (but not a separate defconfig, try
 916"make freebsd_defconfig").</p>
 917
 918<p><u>Library</u>:
 919utf8towc() is now unsigned instead of wchar_t, matching wctoutf8().
 920xclose() tests if fd is -1 so the caller doesn't have to.
 921txpect (the test suite's built-in "expect", implemented in shell) can
 922now do regex matches, and sh.test uses this.</p>
 923
 924<p>Elliott added the failing regex to the xregcomp() error message, and
 925made multiple portability tweaks (header tweaks to build in more
 926environments, running tests on Raspberry Pi OS and under mksh,
 927fix utf8 locale selection on macos).</p>
 928
 929<p><u>Docs</u>:
 930<b>chmod</b>'s help text explains why o+s doesn't set +t (and has a
 931test for it). The <b>echo</b> help text is more compact.
 932Checked in a better conversion of <a href=https://landley.net/toybox/toycans.png>toycans.png</a>.</p>
 933
 934<a name="24-10-2020" /><a href="#24-10-2020"><hr><h2><b>October 24, 2020</b></h2></a>
 935<blockquote><p>
 936"We are now cruising at a level of two to the power of twenty-five thousand to
 937one against and falling, and we will be restoring normality just as soon as we
 938are sure what is normal anyway."</p>
 939<p>- The Hitchhiker's Guide to the Galaxy</p>
 940</blockquote>
 941
 942<p>After a longer and slightly more
 943<a href=https://github.com/landley/toybox/commit/07a896862ddf>turbulent</a>
 944development cycle than some,
 945<a href=downloads/toybox-0.8.4.tar.gz>Toybox 0.8.4</a>
 946(<a href=https://github.com/landley/toybox/releases/tag/0.8.4>git commit</a>)
 947is out with new commands <b>sha3sum</b> and <b><a href=https://github.com/landley/toybox/commit/6b4c32ae3986>watchdog</a></b>.</p>
 948
 949<p>There are <a href=downloads/binaries/mkroot/0.8.4>prebuilt mkroot binaries</a>
 950now, tiny toybox linux systems for a dozen architectures, all bootable under
 951qemu. (The vmlinux in each tarball is a vanilla linux-5.9 kernel built
 952from the included config file.)</p>
 953
 954<p>The FAQ got <a href=faq.html>noticeably larger</a>, and the README has
 955more links. New command features include the <b>sed -s</b> flag,
 956<b>cpio --no-preserve-owner</b> now affects archive creation,
 957Elliott added <b>tar -I</b> and multi-type
 958<b>find -type a,b,c</b> support, Mark Salyzyn added <b>xargs -P</b> to
 959run parallel jobs, the <b>ps</b> and <b>top</b> commands
 960now autodetect pid length, and <b>top</b> adjusts units for memory display based on
 961system size.</p>
 962
 963<p><u>Toysh and toyroot</u>:
 964lots of new work on <b>toysh</b>: 29 commits to sh.c since last release, adding 1500
 965lines and deleting 700, plus a bunch of sh.tests entries). Since last release
 966we implemented wildcards, case/esac and select, brace expansion sequences
 967(ala {1..10..2} and {a..z}),
 968the remaining variable slice types ${a#y} ${a%y} ${a^y} ${a,y}
 969${a/search/replace}, the "<b>source</b>" shell builtin,
 970the start of job control, and several bugfixes.
 971Plus the standalone "<b>make sh</b>" build understands MAYFORK now. It's
 972still missing features like functions() and $((math)) but it's getting
 973close to usable now.</p>
 974
 975<p><u>Documentation</u>:
 976Some of the README contents moved to the FAQ, and the README's "presentations"
 977section got some new links. Several new <a href=faq.html>FAQ</a> entries explaining things like
 978mkroot ("how do I build a working Linux system
 979with toybox"), cross compiling (how to get/setup the 2 compilers and
 9803 libc we regression test against), and "where does toybox fit into the
 981linux/android ecosystem".
 982The "toybox --help" output now lists the project's web page (<a href=https://github.com/landley/toybox/issues/50>by request</a>).
 983Elliott removed
 984getevent (an android board bringup/hardware debugging tool built by running
 985a python script against kernel headers, not really in scope for toybox),
 986fixed xargs help formatting, and taught the toybox multiplexer's
 987command list output (and "kill -l") to measure the current the terminal width
 988when wordwrapping (previously hardwired to 80 columns).
 989If you're curious, I checked in my <a href=www/release.txt>release procedure
 990checklist</a>, and
 991fixed a stale link in the nav bar on the left ("statistics" changed domains).
 992Firas Khalil Khana fixed some issues in the roadmap, and Rob
 993tweaked the roadmap so status.html is slightly more current.</p>
 994
 995<p><u>Tests</u>:
 996The "make tests" target now fails if any of the tests it ran failed.
 997Lots of TEST_HOST tests got fixed (checking for specific error messages exposes
 998TEST_HOST to version skew, and scripts/test.sh will now skip TEST_HOST
 999commands that aren't installed) and added toyonly annotations as necessary
1000(replacing most uses of SKIP_HOST).
1001Eric Molitor added automated github tests on MacOS and Ubuntu using their
1002"workflows" thing.
1003The clang asan plumbing slows some code down more than 10 times, so some testing timeouts were expanded.
1004
1005<p><u>Pending</u>:
1006Eric Molitor did a bunch of work on route: moved it to sbin, added xsend and
1007xrecv, taught display_routes() and setroute to use the rtnetlink API and
1008do hostname lookups, added support for mss, win, and irtt, merged ipv6 and ipv4
1009codepaths with autodetection of address type, implemented RTA_CACHEINFO
1010support, switched exit paths to perror_exit(), and removed unused code.
1011Ethan Sommer fixed warnings in dhcpd, removed a bunch of unnecessary
1012; after GLOBALS() blocks and an unnecessary return in df.
1013Erik Moqvist fixed dns setting in the dhcp client.
1014Rob did some cleanup on bootchartd, traceroute, getty...
1015Elliott fixed getty to reliably update utmp.
1016Chris Sarra added ipv6 support to wget.
1017Ariadne Conill submitted several small fixes from testing Alpine Linux with
1018toybox, and Antoni Villalonga i Noceras fixed typos in error messages.</p>
1019
1020<p><u>Bugfixes</u>:
1021Elliott avoided sign extension in devmem, and Ethan Sommer switched it from
1022getpagesize() to posix sysconf(_SC_PAGESIZE).
1023Elliott fixed chmod -R
1024ignoring dangling symlinks, fixed stty <a href=https://github.com/landley/toybox/issues/251>misparsing c_iflags</a>, fixed the chattr f2fs test with
1025compression enabled, taught blkid not to show empty tags, taught xparsedate()
1026to read date's default output format, fixed a recent echo -e \0 regression,
1027changed cpio -p parsing to match a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-August/011955.html>bug</a> in the gnu version
1028which an existing script <a href=https://github.com/landley/toybox/commit/fa1af3b085cc>depended on</a>,
1029and switched hwclock back to looking
1030only at /dev/rtc0 for <a href=https://github.com/landley/toybox/commit/70e2232ce61c>reasons</a> involving kernel version skew and vendor bug reports.</p>
1031
1032<p>David Legault pointed out that unescape2() (and thus echo -e) wasn't handling
1033\0 right.
1034Khem Raj reported that mips glibc doesn't have SIGSTKFLT, leading to
1035a build break if we assume glibc behaves consistently, which led to a bunch of
1036macos signal portability cleanups from Elliott.
1037William Djupström fixed tar extracting long file paths, adding hardlinks
1038to an archive, and reported that --exclude wasn't working.
1039Peter McConalogue pointed out that cp/mv -i prompt should
1040default N, and mv should only prompt when stdin is a tty.
1041Patrick Oppenlander made rtcwake and hwclock default to UTC if /dev/adjtime
1042isn't available, suggested watchdog catch SIGINT, and caught an
1043uninitialized offset in blkdiscard which gcc apparently didn't.
1044Ryan Pritchard reported that <b>file</b> wasn't getting gif heights right,
1045and Elliott added gif version output so TEST_HOST provided similar output.
1046Martin Stjernholm fixed <b>cp -P</b> to not follow symlinks and updated
1047the help text to say it's not the default.
1048Antoni Villalonga added more --long asiases to <b>chgrp/chown/rmdir</b>.
1049
1050<p>The symlink indirection code subtly broke xexec() so it would still try to call
1051a builtin when given a path to a command (fixed now).
1052The code to trip \n off xgetline() was using the length of the allocation
1053instead of the length of the read.
1054Netcat no longer leaks sockfd into child processes.
1055Patch "fuzz" support was outputting context lines from the hunk, not the
1056file. Chrisrfq fixed i2cdetect parameter reading.
1057Chris Sarra taught dd not to throw an error trying to truncate device files,
1058added a misshing fflush() to lib/password.c, fixed a parse error in logger
1059preventing local facility logging from working right, taught init
1060to reload inittab when it receives SIGHUP, and fixed a memory leak in tar.</p>
1061
1062<p><u>Library</u>:
1063New xvdaemon() function daemonizing on nommu systems, to wean commands
1064off daemon() which requires TOYBOX_FORK.
1065Teach sendfile_len() to use copy_file_range() when available, with compile
1066time probe. Fix xsignal_all_killers() to install the correct handler.
1067Rob switched dirtree_path() to a non-recursive implementation, and
1068taught dirtree that a top level entry named "" is equivalent to
1069"." but should not show up in dirtree_path(),
1070read_password() now reads from tty device rather than stdin, and handles ctrl-c and ctrl-d, and
1071human_readable() now has HR_NODOT so it can print single digit numbers without going "0.0".
1072Petri Gynther increased the vmstat column sizes so it didn't fit
1073in 80 columns anymore.
1074Elliott added UTC offset support to xparsedate().</p>
1075
1076<p><u>Infrastructure</u>:
1077mkroot now only passes --no-preserve-owner to cpio (so initramfs files
1078belong to root rather than whichever user ran the build) when using toybox's
1079version, because the other one is brain-damaged and errors out not
1080understanding it. (It understands it for extract, but not create. Yes really.)
1081New PENDING variable you can set to enable more than just sh and route out
1082of pending.
1083The "make distclean" no longer deletes root_download (where mkroot's "extra plumbing"
1084file saves source code downloads it tarballs, such as dropbear).</p>
1085
1086<p><u>Cleanup</u>:
1087Cleanup openvt (35 lines added, 78 lines removed)
1088Cleanup blkdiscard so lib/args.c parses the -o and -l numbers, which means
108932 bit systems are limited to 2 gigabyte sizes which is a TODO item systemwide.
1090oneit now uses flag macros and mentions -rn in the help. demo-utf8towc
1091now has main.c call setlocale for it.
1092Some cleanup on stty.</p>
1093
1094<a name="11-05-2020" /><a href="#11-05-2020"><hr><h2><b>May 11, 2020</b></h2></a>
1095<blockquote>
1096<p>Ford: Ah. A
1097<a href=https://www.youtube.com/watch?v=WSsR419HBpQ>towel</a>.
1098Keep this and guard it with your life.</p>
1099<p>Arthur: Huh?</p>
1100<p>Ford: Listen, it's a tough universe. There's all sorts of people and things
1101trying to do you, kill you, rip you off, everything. If you're going to
1102survive out there, you've really got to know where your
1103<a href=towel.jpg>towel</a> is.</p>
1104<p>- The Hitchhiker's Guide to the Galaxy</p>
1105</blockquote>
1106
1107<p>Despite everything <a href=downloads/toybox-0.8.3.tar.gz>Toybox 0.8.3</a>
1108(<a href=https://github.com/landley/toybox/releases/tag/0.8.3>git commit</a>)
1109is finally out, with new commands <b>rtcwake</b> from Elliott Hughes and
1110<b>blkdiscard</b> from Patrick Oppenlander.
1111The big news is "<b>make root</b>" now boots to a shell prompt,
1112with toysh making it all the way through toyroot's init script.
1113(Some people were looking forward to <b>patch --fuzz</b> support too.)</p>
1114
1115<p><u>Toyroot</u>: <b>make root</b> now does what it says on the tin, it
1116builds a bootable toybox-based Linux system using two source
1117packages (toybox and linux). The trivial version is "make root && sudo chroot
1118root/host/fs /init". Here's
1119a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-April/011667.html>post with instructions</a> if you want to know how to build the
1120cross compilers for testing the various architectures. The self-contained
1121<a href=https://github.com/landley/toybox/blob/0.8.3/scripts/mkroot.sh>script
1122that builds a simple bootable Linux system</a> is 250 lines long, and
1123should be easy to read if you want to know how it works.</p>
1124
1125<p>It also has basic module support, meaning arguments that aren't X=Y variable
1126assignments are the names of scripts to run to build more
1127packages at the end of the build. I checked in an <a href=https://github.com/landley/toybox/blob/master/scripts/root/dropbear>example package</a>,
1128and there's generic "download and extract source tarballs"
1129<a href=https://github.com/landley/toybox/blob/master/scripts/root/plumbing>plumbing</a> available to such modules.
1130(The Makefile doesn't know how to pass module names through to the script,
1131so instead of "make root" you have to
1132call the script directly, ala "scripts/mkroot.sh CROSS=sh4 LINUX=~/linux dropbear".)</p>
1133
1134<p>The resulting root filesystem now uses /root as the home
1135dir for UID 0, and creates /dev/fd and /dev/shm in devtmpfs. The build works
1136around a kernel
1137build bug where "make distclean" doesn't work in a "cp -sfR" symlink
1138directory. (The bug meant if you pointed LINUX= at unclean source, it was
1139unhappy, so now it distcleans the source directory before copying it. This
1140modifies said source directory, which is not ideal, but as usual the kernel guys
1141<a href=http://lkml.iu.edu/hypermail/linux/kernel/2002.2/00083.html>ignored
1142the bug report</a>, so a workaround was required.)
1143The CROSS=all build announces each target in the title bar, puts
1144its logs into root/log, and has better trap handling to stop with one
1145ctrl-c.
1146Since last release it uses a more concise config format for the various kernel
1147arch targets (which shrank the script by about 50%), and merged
1148the old (now removed) scripts/cross.sh into the main script so
1149"make CROSS=armv7l LINUX=~/linux" is
1150literally just a call to "scripts/mkroot.sh CROSS=armv7l LINUX=~/linux" now.</p>
1151
1152<p>As for <b>scripts/mcm-buildall.sh</b> building cross compilers,
1153of COURSE gcc 8.3 requires a different configuration to build the same
1154toolchain as previous versions, it's gcc. It now builds a proper
1155nommu libc for sh2eb without a broken fork() call in it that can never
1156be used but prevents compile-time probes from detecting nommu,
1157and checks that the cross compiler completed before trying to build
1158the native compiler.</p>
1159
1160<p>This was all tested with a recent
1161<a href=https://github.com/richfelker/musl-cross-make/commit/5086175f2902>version</a>
1162of musl-cross-make with the top level Makefile
1163edited to use BINUTILS_VER = 2.32 because the newer one
1164<a href=https://www.spinics.net/lists/linux-sh/msg56844.html>breaks the kernel
1165build</a>
1166and LINUX_VER = 4.19.90 because the default musl-cross-make config uses an
1167out-of-tree headers package for some reason (those who forget history are
1168<a href=https://lkml.org/lkml/2006/4/28/194>doomed to repeat it</a>) which
1169breaks m68k and s390x. I won't post binaries of the resulting toolchains
1170because they're GPLv3, but <a href=https://musl.cc/>Thalheim might</a>.
1171(I've also <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-May/011673.html>test built</a>
1172with the Android NDK, but bionic's startup code currently segfaults before
1173calling main() if it can't open /dev/null, and the kernel guys
1174<a href=https://lkml.org/lkml/2017/9/13/651>ignored my patches</a> to
1175make CONFIG_DEVTMPFS_MOUNT work in initramfs.)</p>
1176
1177<p>At the moment toyroot cheats and uses two commands out of pending: toysh
1178is about 80% of the way to being useful but still missing some
1179obvious features like function support, job and terminal control, command
1180history, $((math)), wildcard expansion... plus a lot of bash features like array
1181variables, so it isn't out of pending yet. And route needs to be redone to use the
1182netlink interface that can handle multiple routing tables. For the moment
1183scripts/mkroot.sh adds both of them to defconfig if you haven't already
1184got a .config when you run it. (If you build and can't boot, your .config
1185probably hasn't got CONFIG_SH switched on. Fix it in menuconfig, or
1186rm .config and try gain.)</p>
1187
1188<p><u>New toybox features</u>:
1189Elliott taught <b>patch</b> to understand [FILE [PATCH]] arguments, made
1190<b>cal</b> highlight the current day, added -T to <b>cp</b>/<b>mv</b>,
1191did a lot of work on <b>lsattr</b>/<b>chattr</b> (including adding -p
1192and "chattr ="), added tar --absolute-names, taught <b>id</b> to
1193support numeric lookup and handle unknown groups, made -G show all
1194groups, and removed context= from -Z.
1195Rob added <b>patch -F</b> (fuzz factor) support and <b>help -u</b> (usage only),
1196taught <b>echo -e</b> about bash extensions like \uXXXX unicode escapes,
1197<b>netcat -L</b> no longer automatically includes stderr (new -E
1198option does that),
1199<b>setsid</b> now uses -c (like the man page says) instead of -t, and added
1200-w (wait) and -d (detach from tty). 
1201Andrew Ilijic added <b>ls -w</b>, removed trailing whitespace on output,
1202and added tests for -C and -x.</p>
1203
1204<p><u>Library</u>:
1205another fix to 32 bit option parsing of long long optflag values,
1206remove getdirname() and just use the libc function (which modifies
1207its argument, but we don't have to free a malloc),
1208dlist_terminate() can now be called repeatedly on the same list,
1209new relative_path() function finds path from one directory to another,
1210and readfd() works like readfile() but on an already open fd.
1211Use MPATH macros in mkpathat(). Elliott added macOS versions of
1212dev_minor()/dev_major()/makedev() to portability.c,
1213implemented posix_fallocate() for macOS, removed a
1214leftover uClibc workaround in fallocate that was breaking macOS,
1215and moved the table of ELF architectures from file to lib/lib.c (so
1216readelf can share).
1217Park Ju Hyung pointed out the fast path of fdlength() was commented out,
1218and we switched it to the 64 bit API while we were there (and then Elliott
1219added macOS support).
1220Joeky taught file to recognize 7z archives.</p>
1221
1222<p><u>Pending</u>:
1223The shell got a bunch of work: standalone "make sh" now includes the
1224multiplexer for builtin commands like "exit" and "cd". Added MAYFORK annotation
1225for commands that exist in the $PATH but can also be run within the shell
1226process (and sometimes have different behavior within the shell):
1227currently applied to help, echo, false, kill, printf, pwd, test, time, and
1228true.</p>
1229
1230<p>Elliott added new commands <b>getopt</b> and <b>readelf</b>.
1231Jarno Mäkipää taught wget how to follow http 301 and 302 redirects,
1232and did lots of work on vi. (Elliott also did work on vi.)
1233Gavin Howard fixed a comparison bug in bc.
1234Ethan Sommer fixed numerous small issues (including several build
1235warnings and FLAG() macro conversions).</p>
1236
1237<p><u>Bugfixes</u>:
1238<b>tar</b> extract now deletes files and symlinks where it's making a directory,
1239<b>find -L -type -l</b> now finds dangling symlinks, extra #ifdefs in
1240portability.h prevent old gcc versions from barfing on __has_include(),
1241xgetline() now returns NULL at EOF, tee with no arguments was
1242writing to stdout twice, setsid works on nommu (I.E. vfork) now,
1243<b>netcat -L</b> no longer accumulates zombie processes,
1244<b>sntp</b> now uses adjtimex instead of adjtime (to build on bionic).
1245xcreate_stdio() was checking WARN_ONLY in the wrong field (and tar was
1246passing it in the wrong field, so it worked there).
1247Several people wrestled with the <b>xargs</b> "argument too long" problem.
1248Alessio Balsini fixed memory leaks in loopback_setup() and "wayling"
1249added a missing continue to losetup.
1250Elliott fixed xargs -E, various things in modinfo,
1251added an error check to gzip when using zlib (which copies non-gzip data to
1252the output verbatim for some reason),
1253found an case where dirtree could use
1254uninitialized data and silenced "Invalid argument" warnings (triggered by
1255Android's selinux policies making stat() and readlink() fail),
1256fixed locale support in macOS (both in iconv
1257and in main.c), taught stat to show filesystem time on macOS,
1258fixed a 32 bit truncation in sntp, fixed a memory access one byte outside
1259of its array in patch.c, removed the cpio --trailer option,
1260and widened the pid display fields in ps to
12616 digits. Rich Felker fixed find.c assuming
1262time_t is a long (y2038 issue on 32 bit).
1263Greg Kaiser fixed a thinko in get_block_device_size().
1264Jarno Mäkipää fixed utf8 support in cut -C, and cp --parents.
1265David Legault complained that dir/.* tells rm to delete dir/.. and we'd do it.
1266JakeSFR pointed out a bug in file's identification of broken symlinks.
1267William Haddon fixed cp to treat a directory with a trailing slash
1268teh same as one without.
1269Denys Nykula fixed rm -i not to prompt for an empty "" argument.
1270SebiderSushi reported that chmod g+s wasn't working.
1271The linux kernel doesn't let an O_PATH fd work with fgetxattr(), so
1272Elliott switched <b>ls</b> to use the path-based functions now (which is racy,
1273it means you can stat() one inode and get the xattrs for a different one,
1274but nobody in kernel land seems to use xattrs much so they're not fully
1275supported by the API, and who's crazy enough to use xattrs for security
1276anyway). Derick Pallas pointed out an xclearenv() bug.
1277Atatsulo (or possibly luolongjuna) did a lot of work on the <b>ping</b>
1278command: pointed out min/range/max were out of order,
1279that we shouldn't print a summary unless we received at least one
1280reply packet, and implemented ttl support.</p>
1281
1282<p><u>Build plumbing</u>:
1283The "make root CROSS=all" build announces each target in the title bar, puts
1284its logs into root/log, and has better trap handling to stop with one
1285ctrl-c.</p>
1286
1287<p>Fixed scripts/single.sh to work when PREFIX has no trailing slash,
1288make silentoldconfig no longer feeds "y" to kconfig (which puts menus in a
1289loop) and instead just feeds in newlines to accept whatever the default is.
1290The non-git version number I keep forgetting to update each release moved
1291from main.c to toys.h.
1292Fixed a couple different errors in mkflags.c (one of which caused ls --color
1293to set all the other flags).</p>
1294
1295<p>Elliott added more macos support and tests, and added
1296fallocate, cp, mktemp, and mv to the macOS defconfig.</p>
1297
1298<p><u>Cleanup</u>:
1299Rob did some cleanup on xargs, ls, md5sum, and sort, tidied up main.c a bit,
1300made cp, base64, dmesg, and free use FLAG macros,
1301and switched fallocate to new style global names.
1302Elliott Hughes switched rfkill from the old byte at a time get_line()
1303to libc getline(), made du use FLAG() macros, and mad chattr use
1304standard toybox argument parsing for -v and -p.
1305Several commands (help, cp) had sub-options removed from menuconfig.
1306Merged realpath into readlink.c, and use xabspath() instead of libc realpath().</p>
1307
1308<p><u>Documentation</u>:
1309New roadmap section about <a href=roadmap.html#packages>other packages</a>
1310that toybox can (eventually) replace.</p>
1311
1312<p>Update roadmap to note that posix-2008 moved to a different URL (content
1313at the old URL undergoes random changes), and link to the IETF RFCs.
1314design.html explains more of the history of environment sizes on links
1315and has a #bits anchor tag.</p>
1316
1317<p>Update the <a href=design.html#bits>LP64 section</a> of design.html to
1318fish the documents out of archive.org now that unix.org is gone,
1319and show the actual size table locally.</p>
1320
1321<p>Elliott improved the help of <b>date</b> and <b>chattr</b>, and
1322made the usage: lines in the posix directory more consistent.</p>
1323
1324<p>Shrank the sed help from 150 lines to 90-ish.</p>
1325
1326<p><u>Tests</u>:
1327New "txpect" performs interactive tests, running through a sequence of
1328writes to a command's stdin and reads from stdout and stderr, when failing it
1329reports the first non-matching step. (This for example lets sh.test check
1330the shell line continuation logic prompts with $PS1 and $PS2 appropriately
1331with various unfinished input lines, and that "echo hello; if" doesn't
1332output hello before prompting for the next line of input.) Added
1333VERBOSE=xpect to print out each read and write successfully performed by txpect.</p>
1334
1335<p>runtest.sh only creates an "input" file when the input argument isn't empty,
1336EVAL doesn't supply -- (you have to provide your own if you want that).</p>
1337
1338<p>Rob added basic <b>stat</b>, <b>patch</b>, and <b>tee</b> tests.
1339Jarno added a bunch of <b>vi</b> tests.
1340Elliott fixed tests for <b>ifconfig</b>, <b>lsattr</b>, <b>chattr</b>,
1341and <b>date</b>, added tests for find, id, xargs, and made
1342the id, iconv, env, file, printf and cat tests work on macOS (and skipped
1343the du tests there). Rob cleaned up chmod tests.</p>
1344
1345<p><u>Sheer pedantry</u>:
1346true and false now have usage: lines (which you have to "help false" to see
1347because they (intentionally!) don't support --help.
1348Renamed get_chunk()/dump_chunk() to read_chunk()/write_chunk() in tail.c
1349Elliott fixed some typos.</p>
1350
1351<a name="18-10-2019" /><a href="#18-10-2019"><hr><h2><b>October 18, 2019</b></h2></a>
1352<blockquote>
1353<p>"In those days spirits were brave, the stakes were high, men were real
1354men, women were real women and small furry creatures from Alpha Centauri
1355were real small furry creatures from Alpha Centauri."
1356</p> <p>- The Hitchhiker's Guide to the Galaxy</p>
1357</blockquote>
1358
1359<p><a href=downloads/toybox-0.8.2.tar.gz>Toybox 0.8.2</a>
1360(<a href=https://github.com/landley/toybox/releases/tag/0.8.2>git commit</a>)
1361is finally out (only two months late).</p>
1362
1363<p>The new commands this time are minor variants of existing ones
1364(<b>dnsdomainname</b> is hostname -d, <b>arch</b> is uname -m), but about
13651/3 of toysh got implemented in pending, <a href=https://github.com/landley/mkroot>mkroot</a> got merged as a "make root" target (see Build),
1366and Android's <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-August/010845.html>hermetic build</a>
1367work continued resulting in a lot of fixes.</p>
1368
1369<p>The android-specific <b>getprop</b>, <b>setprop</b>, <b>start</b>, and <b>stop</b>
1370commands were removed from toybox because they grew dependencies on android
1371libraries, and since toybox tries to avoid mandatory external dependencies
1372they moved to another package.</p>
1373
1374<p><u>Build</u>:
1375The new "make root" target builds a simple toybox root filesystem,
1376using scripts/mkroot.sh which is a stripped
1377down version of the external <a href=https://github.com/landley/mkroot>mkroot</a>
1378project merged into toybox. (Alas, at the moment it wants sh and route out
1379of pending, which aren't quite load bearing yet.)</p>
1380
1381<p>The new "scripts/mcm-buildall.sh" convenience script builds cross
1382compilers based on gcc+musl multiple targets (arm, x86, mips, m68k, s390,
1383microblaze...)
1384using <a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a>,
1385and a new scripts/cross.sh to easily cross compile for one or more targets
1386using the resulting compilers. (This works with "make root", "make
1387toybox", and other builds understanding the CROSS_COMPILE environment variable.)</p>
1388
1389<p>Setting the environment variable ASAN=1 enables the
1390llvm address sanitizer. (You may need to
1391export ASAN_SYMBOLIZER_PATH on debian to use it, they add the version number to
1392the filename in /usr/bin which confuses llvm.)</p>
1393
1394<p>Patrick Oppenlander fixed make -j (our make is a wrapper around
1395scripts/make.sh which was already doing a parallel build, but if you called
1396make with -j it wouldn't wait for make.sh to finish and the build broke).</b>
1397
1398<p>Lots of fixes for the MacOS build.</p>
1399
1400<p><u>New options</u>:
1401We upgraded <b>tar</b> to extract some older tarballs, <b>file</b> added -b and -s
1402options and can also recognize older tarballs,
1403some xz archives, .otf fonts, perf/simpleperf data files,
1404android boot images, and dtb files. Upgraded <b>file</b> to show the device type
1405for block and char devices, the target for symlinks, and it says _why_ it can't
1406open a file.</p>
1407
1408<p>Several commands ignore some options rather than erroring on them:
1409<b>patch</b> ignores the -f -g# and --no-backup-if-mismatch options
1410(all of which we were already doing by default anyway), and <b>xargs</b>
1411ignores -P# (which requests parallel execution, maybe add it later but
1412single threading works), and Denys Nykula made <b>wget</b> ignore
1413--no-check-ceritificate and <b>gzip</b> ignore -n (again, we weren't doing
1414either anyway) to satisfy pkgsrc.</p>
1415
1416<p>William Haddon taught <b>diff</b> and <b>patch</b> to support special characters in path
1417names (via quoting) and more timestamp formats.
1418Eric Molitor implemented <b>nl</b> -v (with negative and zero starting values),
1419and fixed a MacOS compatibility issue in the build scripts.
1420Tom Cherry added <b>netcat</b> -U (UNIX domain socket support).
1421Denys Nykula taught <b>wget</b> to call ftpget for ftp:// URLs.
1422Ethan Sommer added ln -t and rmdir --ignore-fail-on-non-empty.
1423Elliott Hughes added <b>grep</b> -R, <b>killall</b> -w, tar --mode,
1424 fixed <b>ls</b> -Z and spacing in -lZ and --full-time,
1425added <b>xargs</b> --max-args and removed xargs -I (which we never actually
1426supported) and tweaked the size calculation to more closely match other
1427versions,
1428added xargs -o and made -p read from /dev/tty, added <b>blkid</b> -s, SEC_TYPE for ext3,
1429msdos, and ntfs, f2fs and ntfs LABEL, fixed vfat/ntfs UUID, and switched
1430the endianness for UUID of msdos/ntfs/vfat, simplified the octal mode
1431display in <b>lsof</b>, added <b>pidof</b> -x (thus making the default behavior _not_
1432be -x).</p>
1433
1434<p>When readdir() succeeds but stat() fails, <b>ls</b> now prints "??? ? ? filename"
1435entries instead of "cannot access" errors. (Still trying to make ls / on
1436android work more gracefully with the SELinux weirdness they added in 9.)</p>
1437
1438<p>Added %C to <b>stat</b> and made it show device type in its default output,
1439<b>cmp</b> now works
1440with 1 argument (implicit - as second argument reading from stdin).
1441<b>sed</b> added exit codes to the q command (q123), added Q, and fixed a
1442bad error message with "!".
1443Removed TAIL_SEEK config option (it just always does that now),
1444<b>find</b> grew several new options (%Z, -true, -false, and -newerXY), <b>ln</b> added -T.
1445The argument to <b>mktmp</b> --tmpdir is now optional.</p>
1446
1447<p><u>Bugfixes</u>:
1448The unshare probe bit-rotted so it was impossible to build <b>unshare</b>
1449and <b>nsenter</b> on some distros.
1450Commit 771e94e2a08 broke toybox's ability to say "unknown command" when
1451you pointed a symlink at it that it that didn't correspond to a command,
1452pointing a symlink at another symlink
1453to make toybox run a command under a name it doesn't recognize
1454(ln -s toybox sh; ln -s sh ash) had an off by one error,
1455<b>ifconfig</b> no longer error_exits if the "read mac address" ioctl fails (Android N
1456and later block that for non-root users to prevent device tracking).
1457When the libcrypto library was disabled, md5sum was building commands it
1458hasn't got built-in support for yet (like sha3sum) which acted like redundant
1459copies of md5sum,
1460<b>tar</b> wasn't correctly reading or writing sparse files with a hole at
1461the end, and tar tzf blah.tar.xz was calling xz when it meant xzcat,
1462<b>xargs</b> would spuriously fail on large memory systems (due to a missing
1463"unsigned" in a typecast),
1464<b>basename</b> can now remove suffixes starting with a dash,
1465<b>cp</b> -r unlinks() an existing symlink before trying to
1466recreate the symlink (otherwise cp -r into an existing directory will
1467try to follow the existing symlink and create another symlink at its
1468destination, failing if a file already exists there, which
1469was confusing gentoo's package manager), <b>timeout</b> now produces the
1470right exit code when the timed command intercepts the exit signal,
1471and <b>hostname</b> -d no longer segfaults on machines with no domain name.</p>
1472
1473<p>Denys Nykula fixed the <b>ftpget</b> freeze (it was sending REST instead
1474of RETR), made <b>mv</b> work with a trailing slash in the source,
1475fixed a <b>find</b> segfault,
1476and made <b>env</b> always exec rather than recursing to a builtin
1477(on the theory #!/usr/bin/env is expected to search the $PATH).</p>
1478
1479<p>Elliott improved <b>killall</b>'s handling of long filenames, made <b>kill</b> work
1480when killall5 is disabled (missing FORCE_FLAGS), fixed error reporting in
1481<b>hostname</b>, fixed several things in <b>xargs</b>,
1482made <b>printf</b> \c and \0 octal escapes work, fixed a <b>find</b> -name corner
1483case (find src/*.c -name file.c) and find dangling symlink behavior,
1484made some <b>rm</b> error messages consistent, fixed \x behavior in echo and
1485printf, fixed linestack.c's buffer length for utf8 U+XXXX escape printing,
1486fixed a race in <b>losetup</b>, fixed <b>grep</b> -F with multiple matches on the same
1487line and an -I failure in the existing grep tests, made <b>readlink</b> notice
1488when it has more than one argument, and made various
1489fixes to <b>modinfo</b>.</p>
1490
1491<p>Ethan Sommer reported that <b>stat</b> of an suid/sticky file output 5 digits of
1492octal stat date instead of 4.
1493Rob fixed an unaligned access in <b>grep</b>. Alessio Balsini removed an incorrect
1494null termination in <b>losetup</b>.
1495Jarno Mäkipää fixed <b>hexedit</b>
1496scrolling up and down on older terminals, and fixed <b>ls</b> segfaulting with a broken
1497(non-UTF8) locale (didn't expect wcrtomb to return -1).
1498Andrew Ilijic made ls put 2 spaces between columns like other implementations
1499(which helps when terminals get confused and apply UTF8 combining characters
1500to the space after a filename).
1501</p>
1502
1503<p><u>Library</u>:
1504Added dlist_lpop() to use a dlist as a stack (it removes the last entry
1505instead of the first), and dlist_pop() now works on a dlist_terminate()d list.
1506DIRTREE_STATLESS returns entries we couldn't stat() (with a zeroed ->st
1507field, and ->again |= 2), but filling in the file type from readdir().
1508Coversions between signal names and numbers now include all the Linux signals,
1509not just posix ones. The functions sig_to_num() and num_to_sig() moved into
1510lib/portability.h, because MacOS has different signals.
1511TOYFLAG_MAYFORK allows commands to run in toysh's process without forking,
1512but also makes them accessable from the toybox multiplexer or standalone.
1513A MAYFORK command has to clean up after itself (even in error paths) and
1514can't discard anything we need to keep (such as closing stdout).</p>
1515
1516<p>Elliott did a big xbind/xconnect cleanup (adding xbindany and xconnectany),
1517taught xregcomp that an empty regex matches the whole line
1518(because FreeBSD, and thus MacOS, doesn't already do that), and
1519replaced several uses of get_line() (which does single byte reads)
1520with getline() (which doesn't).</p>
1521
1522<p><u>Plumbing</u>:
1523"make clean" doesn't produce a screenful of unnecessary output now,
1524rm, ln, cp, kill, and netcat use the FLAG() macros now, makedevs uses the new
1525GLOBALS naming format, timeout now uses xwaitpid(), and we fixed an
1526off by one error in xwaitpid().</p>
1527
1528<p>Elliott removed a workaround for old NDK versions from getconf and
1529moved sort off of get_rawline() (which let us remove it).
1530Rob undid some loop unrolling in md5sum/sha1sum (we have libcrypto if you want
1531an assembly optimized version instead of an understandable version).</p>
1532
1533<p>Android ndk-r20 doesn't define the __ANDROID_NDK__ symbol, so we switched
1534to using clang's __has_include() (which is an undefined macro and thus a NOP
1535resolving to false on gcc).</p>
1536
1537<p><u>Pending</u>:
1538Rob did a lot of work on toysh: continuation support works (it knows when to ask
1539for another line of text and when to run what it's got), as do
1540nested if/else/elif statements, and while and for loops (supporting "for i"
1541and "for i in", but not for ((;;)) yet).
1542It parses pipes and redirections but doesn't perform them yet.
1543The next big todo chunk is environment variable resolution.
1544</p>
1545
1546<p>Gavin Howard fixed some bugs in bc, such as not returning error when raising
1547zero to a negative power.
1548Jarno Mäkipää continues to extend vi, adding yank and push, support for tabs,
1549rewriting delete, changing the drawing code, etc. Elliott also added several
1550fixes and features to vi,
1551fixed host to cope with large DNS responses, switched crontab to
1552use getline(), did work on man, fixed fixed several things in dd,
1553implemented diff --strip-trailing-cr and made diff not need /tmp.
1554Mike Bennett added extended attribute support to ip.c's route change.</p>
1555
1556<p><u>Test suite</u>: The killall and pgrep tests were having occasional
1557spurious failures due to a race condition (the test script would fork()
1558to call sleep and for just a moment there were two instances of the test
1559script, reporting an extra process if killall or pgrep ran then).
1560
1561<p>Elliott Hughes did a lot of work on the test suite: add kill -l tests,
1562make losetup.test notice (and skip) if loopback
1563devices area already in use, blkid.test can TEST_HOST e2fsprogs blkid now,
1564modinfo.test notices (and skips) if there's no /proc/modules or modules
1565directory, ifconfig.test skips if "dummy0 up" fails and disabled the
1566pointtopoint tests entirely (the kernel never supported it?),
1567truncate.test is less
1568confused by SELinux, hostname.test can run some tests as non-root,
1569addressed a race condition in killall.test on slow systems,
1570made chgrp.test, chown.test, and tar.test handle an empty /etc/passwd or /etc/group
1571(we were checking whether they were _there_ but some android devics have
1572empty ones),
1573fixed a couple things in env.test, extended killall.test to explicitly cover
1574long and short names (and not assume the shell is /bin/sh), taught split.test not to rely on bash process substitution,
1575added missing TZ=utc to touch.test,  fixed cp.test to pass for root
1576or with a restrictive umask, and fixed the FAILCOUNT arithmetic to use more portable
1577$(()). Elliott also made some tests more flexible (find.test can handle any
1578error message about symlink loops, etc), and disabled others (such as du)
1579when SELinux is enabled.</p>
1580
1581<p>Rob taught testcmd not to use the shell builtin for TEST_HOST unless there
1582isn't one in the $PATH, work if $PWD has a space in it,
1583and make kill.test stop testing the shell builtin
1584insead of the command, quieted tests that were producing output
1585other than their PASS/FAIL line, and made NOSPACE tests ignore the
1586presence/absence of leading/trailing whitespace as well as differences
1587in the amount of space.</p>
1588
1589<p><u>Documentation</u>:
1590The <a href=roadmap.html>roadmap</a> got freshened up a bit, with updates
1591for android from Elliott.
1592In <a href=design.html>the design page</a> the Features section starts with
15934 new paragraphs about the scope of toybox, and a new License section talks
1594about 0BSD.
1595The "./toybox --help" output now explains how to do install_flat by hand.</p>
1596
1597<p><u>Sheer pedantry</u>:
1598Elliott fixed a double close in loopfiles_lines() so strace doesn't
1599have an EBADF in it, and added a missing "static" in lib/net.c.</p>
1600
1601<a name="21-05-2019" /><a href="#21-05-2019"><hr><h2><b>May 21, 2019</b></h2></a>
1602<blockquote>
1603<p>"I like the cover," he said. "'Don't Panic.' It's the first helpful or intelligible thing anybody's said to me all day."</p> <p>- The Hitchhiker's Guide to the Galaxy</p>
1604</blockquote>
1605
1606<p><a href=downloads/toybox-0.8.1.tar.gz>Toybox 0.8.1</a>
1607(<a href=https://github.com/landley/toybox/releases/tag/0.8.1>git commit</a>)
1608is out.</p>
1609
1610<p>Android is now using toybox as part of a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-November/009824.html>"hermetic" build</a>
1611(meaning Android is building under toybox, which is halfway to Android
1612building under Android). See Android's <a href=https://android.googlesource.com/platform/prebuilts/build-tools/+log>build-tools</a> and <a href=https://android.googlesource.com/platform/build/soong/+log>soong</a> repositories to follow along with development, or check the list for the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-May/010485.html>current status</a>.</p>
1613
1614<p><u>New commands</u>: Isaac Dunham added <b>mcookie</b>, which is used by
1615X11. Elliott Hughes added <b>devmem</b>, which replaces toolbox's "r".</p>
1616
1617<p><u>Promoted</u>: <b>tar</b>, <b>gunzip</b>, and <b>zcat</b> were cleaned
1618up and moved out of pending.
1619Tar is a near-rewrite with support for --owner --group and --sparse,
1620capable of autodetecting "wget blah.tar.bz2 -O - | tar xv", a --restrict option
1621that only allows files to extract into a single subdirectory, and so on.</p>
1622
1623<p><u>Added to pending</u>:
1624Jarno Mäkipää contributed a new <b>vi</b> implementation. Makepost added
1625<b>man</b>.</p>
1626
1627<p><u>Updates</u>: Elliott added <b>find</b> -wholename/-iwholename, -printf,
1628and -context, <b>grep</b> --exclude-dir, and <b>echo</b> -E.
1629Rob added <b>blkid</b> -U and -L, <b>mount</b> now has UUID= support via blkid -U,
1630you can now specify TZ="blah" in the <b>date</b> -d input string (although
1631why the double quotes are required is an open question), and <b>sed</b> now
1632understands ,+N relative ranges. Makepost added <b>rm</b> -v.
1633The human_readable fields in <b>ps</b>/<b>top</b>/<b>iotop</b> now use all
1634available space to show more precision, <b>iotop</b> displays swap bandwidth
1635properly (it was showing a signed field as unsigned), and
1636<b>touch</b> -t and -d autodetect the time format.</p>
1637
1638<p><u>Cleanups</u>: In pending Rob did some bc cleanup,
1639Colin Davidson did some arp cleanup, 
1640Uros Prestor fixed route argument parsing, and
1641in bc Daniel Rosenberg fixed bc_vec_concat()'s allocation and
1642Gavin Howard fixed an overflow bug in bc_num_ulong().</p>
1643
1644<p>Elliott made scan_key() detect cursor keys and such with several more
1645terminal types, made more better about directories and non-tty output,
1646cleaned up inconsistent uses of ARRAY_LEN(), taught file
1647to recognize a c-sky ELF machine, and fixed timezone and daylight
1648savings time issues in date and its tests.
1649Rob simplified netcat and ping using new lib/net.c infrastructure.
1650When run as non-PID 1 (for a chroot) oneit's exit code now indicates
1651which signal it's exiting due to.
1652Hadrian Węgrzynowski #defined constants for mkpathat.
1653Date now uses the current time for unspecified fields (instead of zeroing
1654them), and should set the weekday properly.
1655Sort's -k2.3,4.5 syntax is now documented and behaves more closely to
1656other implementations (silly as that behavior is).
1657Significant speedup to sed s/a/b/g on a megabyte long line (it was
1658O(n^2) and now it's quite snappy).</p>
1659
1660<p>Several cases of warning about stdout failing to write were silenced
1661because things like piping the output to head means stdout is a special
1662case: its output is often discarded and that's fine. And xprintf()
1663and friends only checks stdout for errors so it can exit early ("yes | head"
1664should not spin endlessly) without actually flushing anything.
1665There's an ongoing design debate about line buffering stdin and stdout
1666that'll have to be resolved next dev cycle.</p>
1667
1668<p><u>Bugs</u>:
1669Elliott sent in a lot of fixes for building AOSP with toybox:
1670support for more date formats, sed substitution of empty capture groups
1671and an endless loop with -EPIPE, fixed find -path/-ipath, made loggit()
1672only feed LOG_AUTH to syslog when we have a tty, gave grep a bunch
1673of --longname synonyms for short options, added getfattr --only-values,
1674and timeout --foreground --preserve-status and --signal.</p>
1675
1676<p>Top got a lot of tweaks: it no longer collates thread's CPU usage into the
1677parent thread (in -H mode), hides the cursor in interactive mode, updates the
1678display properly in batch mode, and accepts ENTER
1679as a synonym for SPACE when refreshing the display. The header lines
1680now abbreviate long numbers with units (via human_readable()),
1681more reliably spans the terminal width, and says "Thread" instead
1682of "Task" when it's showing threads. The task/thread count now adds up
1683(understanding more kernel states and treating unknown ones as "sleeping").
1684Top now treats -o "" as an error (previously it displayed an empty screen).</p>
1685
1686<p>William Djupström pointed out login needs to chown() the tty, and made it
1687clear the environment when -p is absent.
1688Vidar Holen fixed inverted find -exec exit status.
1689Rob made chgrp() stop relying on libc's printf("%s", NULL) printing "(null)"
1690because a <a href=https://github.com/varnishcache/varnish-cache/issues/2879>newly added gcc bug</a> turns that into a build break, and
1691fixed an outright embarassing xstrndup() bug.
1692Liwugang fixed a segfault when stat's format string had a trailing %.
1693A missing else in grep meant -x was sometimes ignored.</p>
1694
1695<p><u>Infrastructure</u>:
1696in the build scripts/portability.sh now tests that the compiler it's trying
1697to use exists up front. We no longer link against librt because on glibc that's
1698pulling in libpthread for some unknown reason.</p>
1699
1700<p>TOYFLAG_ARGFAIL() lets commands return custom exit values when option parsing
1701fails (because AOSP build scripts cared), so timeout --unknown returns 125
1702and so on: chroot, diff, cmp, env, grep, nohup, and sort also use it now with
1703various values.</p>
1704
1705<p>New FLAG() macros: FLAG(x) expands to (toys.optflags&FLAG_x) and returns
1706zero if the command line argument's bit isn't set and nonzero if it is
1707(meaning -x was or wasn't encountered on the command line this time by
1708lib/args.c). Lots of commands got converted to use them.</p>
1709
1710<p>New lib/env.c with reset_env(), xsetenv(), and xunsetenv() which frees
1711the old environment variable when it wasn't inherited from exec(). This
1712lets long-running loops reset environment variables without memory leaks.</p>
1713
1714<p>New xnotify() plumbing in lib/portability.c with Linux and Mac versions
1715(which should also work on BSD).</p>
1716
1717<p>New xparsedate() and xvali_date() functions allow date, tar --mtime, and
1718touch to understand the same date formats,
1719xpopen_both() now knows how to use existing filehandles for the child's
1720stdin/stdout, and no longer redirects stderr (which hides errors),
1721new nanomove(), nanodiff(), union socksaddr, xrecvwait(),
1722and human_readable_long() to show more than 3 digits of output.</p>
1723
1724<p>Taught xgetaddrinfo() to return a wildcard address for a NULL host,
1725xconnbind() sets SO_REUSEADDR by default,
1726fileunderdir() returns the abspath to the file now (since we had to work
1727it out anyway), and
1728xabspath() uses O_PATH now (because Android's SELinux rules got more aggressive).</p>
1729
1730<p>Several new xsendfile() variants, including sendfile_len() to copy
1731a specific amount of data and xsendfile_pad() to append zeroes to the
1732output when there isn't enough input.</p>
1733
1734<p>regexec0() is now using libc's REG_STARTEND supported by every libc out
1735there _except_ musl. (It originated in BSD and was adopted by glibc, bionic,
1736and uClibc 15 years ago, and is also supported in macosx. Musl's maintainer
1737<a href=https://www.openwall.com/lists/musl/2013/01/15/26>turned down</a>
1738the feature request as an invalid use case he didn't think they should
1739be trying to do.)</p>
1740
1741<p><u>Docs</u>:
1742the <a href=license.html>license page</a> now has an SPDX identifier, and links to more
1743history/context. The actual LICENSE file had its title line removed
1744so <a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-463746027>github could recognize it</a> as 0BSD. (We're still waiting for
1745"<a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-464380789>a while</a>" to pass before github actually does, though.)</p>
1746
1747<p>A new www/doc/mount.txt file describes how mount works under the covers.</p>
1748
1749<p>Rob consistently misspelled "canonical" and "millisecond", mountpoint
1750has a synopsis now,
1751Kevin van der Kamp fixed a typo in netstat's help text, and Elliott
1752Hughes fixed typos everywhere and made the tense, capitalization, and
1753punctuation of help text more consistent.</p>
1754
1755<p>New paragraph in the code style part of design.html explaining that
1756"char* a, b;" is not how C works, and since you wind up with "char *a, b;"
1757anyway you might as well say that.</p>
1758
1759<p>Given dreamhost's tendency to repeatedly delete the mailing list
1760archive, the nav bar on the left links to a backup web archive now too.</p>
1761
1762<p>The FAQ now says why we do time based releases.</p>
1763
1764<p><u>Testing</u>: Added VERBOSE=nopass to not show passing tests.
1765The test suite now does an rm -rf on testdir between each command so
1766debris files don't accumulate.
1767New skipnot function skips the next test if a command line fails, and
1768toyonly function only runs command when we're testing the toybox version
1769of command. (This isn't the same as TEST_HOST, the host version could be
1770toybox.)
1771The find -newer test calls "sleep .1" to avoid intermittently failing with an
1772identical timestamp.
1773Elliott added the start of a file.test rogues gallery.
1774No longer call lsattr in "make tests" because the behavior differs by
1775filesystem and it needs fixing somehow.</p>
1776
1777<a name="08-02-2019" /><a href="#08-02-2019"><hr><h2><b>February 8, 2019</b></h2></a>
1778<blockquote>
1779<p>"Most readers get as far as the Future Semiconditionally Modified
1780Subinverted Plagal Past Subjunctive Intentional before giving up; and
1781in fact in later editions of the book all pages beyond this point have
1782been left blank to save on printing costs."</p>
1783<p>- The Hitchhiker's Guide to the Galaxy</p>
1784</blockquote>
1785
1786<p><a href=downloads/toybox-0.8.0.tar.gz>Toybox 0.8.0</a>
1787(<a href=https://github.com/landley/toybox/releases/tag/0.8.0>git commit</a>)
1788is out.</p>
1789
1790<p>Toybox now builds on MacOS and FreeBSD, thanks to the efforts of Elliott
1791Hughes and Ed Maste respectively. Use the "make macos_defconfig" and "make
1792freebsd_defconfig" targets to enable the set of commands that compile on
1793each so far.</p>
1794
1795<p><u>New commands</u>: Added an <b>sntp</b> client/server (RFC 4330 Simple Network Time
1796Protocol, a compatible subset of ntp). The <b>test</b> command was rewritten
1797and promoted out of pending.</p>
1798
1799<p><u>New options</u>: <b>grep</b> now has --color, supports embedded NUL bytes in its input,
1800recognizes binary files, and passes all of grep.test (in case you
1801needed the middle numerical field of -bB, etc).
1802Josh Gao added ipv6 and UDP support to <b>netcat</b>.
1803Volodymyr Medvid reported that install -d doesn't honor -o or -g.
1804Elliott Hughes did a lot of work to support hermetic Android/AOSP builds:
1805<b>mkdir</b> accepts both --parent and --parents as synonyms for -p, <b>touch</b> ignores -f,
1806<b>basename</b> added -s to remove a trailing suffix, <b>dirname</b> now supports multiple
1807arguments, <b>cmp</b> accepts --quiet and --silent as synonyms for -s, <b>hostname</b>
1808added -sfd, <b>head</b> added --bytes as a synonym for -c and --lines as a synonym
1809for -n, <b>mktemp</b> added -t and fixed -u, <b>sed</b> added -z and -iEXT to keep backup files,
1810<b>md5sum</b> and sha1sum added --status and --check as synonyms -s and -c,
1811<b>readlink</b> added --canonicalize as a synonym for -f, <b>sort</b> grew -V,
1812<b>patch</b> added -s its synonym --quiet, <b>stat</b> added --format as
1813a synonym for -c, <b>xargs</b> added -p -t -r,
1814Eduardas Meile asked
1815that <b>umount</b> ignore -c. Reverend Homer added a small optimization to file.c,
1816and Elliott taught <b>file</b> to recognize riscv ELF binaries. Peter Collingbourne
1817taught <b>ls</b> -t to use the nanoseconds field.
1818<b>patch</b> has better support for patching a file with a tab in the name.</p>
1819
1820<p><u>Bugfixes</u>:
1821<b>cp</b> --preserve was segfaulting when you didn't specify _what_ to
1822preserve (it now correctly defaults to "mot") and didn't get the permissions
1823right when copying a symlink's contents as a regular file,
1824<b>sort</b> -x didn't work when
1825attached to a key, <b>host</b> didn't allocate a big enough buffer for worst
1826case ipv6 address size, <b>sed</b> needed a ; between b and } when other
1827implementations don't (we're already well past what posix says but a script
1828out in the field broke...),
1829and several fixes to <b>hostname</b> in a container, <b>ps</b>/<b>top</b> were
1830misreading the VIRT and SHM fields.</p>
1831
1832<p><u>Pending</u>: Gavin Howard updated <b>bc</b> and added more tests,
1833Marius Adaskevicius pointed out <b>mdev</b> -s should follow symlinks,
1834Yangchun Fu reported a <b>dhcp</b> checksum bug, <b>modprobe</b> needed errno reset
1835to avoid reporting spurious errors in verbose mode and no longer
1836exits with status 1 if it can't find /etc/modprobe.conf, the <b>more</b> command
1837wasn't always flushing stdout when it exited.</p>
1838
1839<p><u>Build</u>:
1840<b>chrt</b> no longer #warns about the musl sched_get_priority_min() bug, but
1841instead works around it.
1842We were mixing setjmp/siglongjmp (harmless in some contexts but it kind
1843of annoyed FreeBSD), and
1844make install_airlock now adds sha256sum (because the Linux 4.20 build now
1845needs that for the s390x target).</p>
1846
1847<p><u>Coding style</u>:
1848Rob converted the rest of the option GLOBALS() to the new single letter
1849coding style, and the new FLAG(x) macro is a slightly tidier way to say
1850"toys.optflags&FLAG_x".
1851Removed CFG_SORT_BIG (the sort command always
1852has the full functionality now. The general future direction or toybox
1853is to either have a command or not have it; multiple versions of the
1854same command aren't worth the complexity in testing, documentation,
1855or system adminstration).</p>
1856
1857<p><u>Library code</u>:
1858The mkflags plumbing can now support arbitrary punctuation as option
1859names via an escape syntax (because mkfs.vfat specifies an offset with -@),
1860and lib/args.c now only sets FLAGS_NODASH when the first argument
1861didn't have a dash (allowing "ps ax" and "ps -ax" to behave differently).</p>
1862
1863<p>Added an xrename() function,
1864xchdir() has better error reporting, xconnect()/xbind() had their
1865implementations merged, xsendto() moved from ping to lib, xpoll() now
1866measures elapsed time and only waits for the remaining period when restarted,
1867and Eduardas Meile converted several error_exit() to error_exit_raw() (because
1868yocto 2.6 buils with -Werror=format-security by default).</p>
1869
1870<p>There's a scripts/portability.sh now which lets the build use gsed and gmake
1871(when available) in more places to avoid broken host versions on MacOS and FreeBSD.
1872The syntax of the "noreturn" attribute
1873changed slightly, some calls to strnstr() were replaced with strcasestr(),
1874we work around MacOS' lack of features.h, portability.h can now do Apple's
1875endianness macros, and so on.</p>
1876
1877<p>Following the <a href=faq.html#support_horizon>seven year rule</a>
1878Elliott removed support or glibc 2.10, and also
1879reformatted a lot of --help text for consistency (removing trailing
1880periods and such).</p>
1881
1882<p><u>Test suite</u>:
1883The test suite now has its own version of "pending", specifically "make tests"
1884skips test files without the executable bit set (unless you export $TEST_ALL).
1885This eliminates "expected failures" (I.E. files with unresolved todo items)
1886from the global regression test. Additional tests can be added to the global
1887regression test with chmod +x as the todo items they document get resolved.</p>
1888
1889<p>Several places the test suite got fluffed out, including guards to skip
1890root-only tests when run as a normal user. The testcmd function prints the
1891short name instead of the full path to the command.
1892The tests for the test command
1893("test.test") no longer test the shell builtin but correctly test the
1894toybox command. (Now say that ten times fast.)</p>
1895
1896<p>Bash version skew required replacing "continue" with "return" to stop
1897tests early in bash 4.4 (for VERBOSE=fail).</p>
1898
1899
1900<a name="31-10-2018" /><a href="#31-10-2018"><hr><h2><b>October 31, 2018</b></h2></a>
1901<blockquote><p>
1902"In the beginning the Universe was created.
1903This has made a lot of people very angry and been widely regarded as a bad move."</p><p>- The Hitchhiker's Guide to the Galaxy
1904</p>
1905</blockquote>
1906
1907<p><a href=downloads/toybox-0.7.8.tar.gz>Toybox 0.7.8</a>
1908(<a href=https://github.com/landley/toybox/releases/tag/0.7.8>git commit</a>)
1909is out (a month late).</p>
1910
1911<p>Toybox <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-October/009769.html>now builds</a> with the Android NDK (r18 release), almost out of the box.
1912You have to add an "llvm-cc" symlink to "clang", then use CROSS_COMPILE=llvm-
1913with the appropriate $PATH (or absolute path in CROSS_COMPILE) for the
1914target you want to build for. (To run it on a non-android host, you probably
1915want "LDFLAGS=--static" too.)</p>
1916
1917<p><u>New commands</u>: <b>i2cdetect</b>, <b>i2cdump</b>, <b>i2cget</b>, and <b>i2cset</b>
1918courtesy of Elliott/Android. The <b>watch</b> command got a complete (tty-aware)
1919rewrite, and was promoted out of pending. The <b>prlimit</b> command is back
1920(a configuration bug was always disabling it), and <b>ascii</b> now defaults y
1921in defconfig (an oversight).</p>
1922
1923<p><u>New options</u>: 
1924The <b>ifconfig</b> command added -S (short view) giving one line of
1925information per interface, in "name ipv4/mask macaddr [ipv6/mask@type...]"
1926format. <b>grep</b> grew binary file detection, with -I and -a options to
1927ignore them or force treating them as ascii (really utf8).
1928Zach Van Rijn asked for <b>find -empty</b>.
1929In the <b>file</b> command, Elliott improved java
1930.class support and added .wav, .bmp, and android .dex.
1931Elliott also added "getconf -a" and a lot of missing getconf symbols the
1932AOSP build uses, including support for pathconf(3) symbols requiring two
1933arguments. This means getconf's arguments work more like ubuntu's
1934now (one or two arguments, not iterating through the supplied list and
1935processing each like last release), and -l grew section identifiers (so
1936the "getconf -l" output is no longer a directly consumable list of symbols
1937it takes).
1938Command line options specifying durations (like top -d or ping -i) can
1939now use fractions and units, like "1.5" and ".1m". See "toybox --help" for
1940details.
1941Toybox now dereferences one layer of symlinks if it doesn't recognize the
1942name it's called under (so if you "ln -s $(which sleep) blah" and then
1943"./blah 30", it should figure out you mean sleep 30).</p>
1944
1945<p><u>Pending</u>:
1946Gavin Howard contributed a large new <b>bc</b> implementation to pending,
1947which covers everything the kernel build needs (and more) but is going to
1948take a while to review.
1949In <b>diff</b> the android guys added --color and timestamps on the
1950+++/--- lines. In <b>mdev</b> Faustas Azuolas Bagdonas added support
1951for mdev.conf's fourth field. Reverend Homer suggested an error message
1952improvement in <b>wget</b>, Kevin Spiteri did a cleanup pass on test.c,
1953and there was a cleanup pass on the passwd command.</p>
1954
1955<p><u>Bugfixes</u>:
1956Elliott responded to a bug in <b>getconf</b>'s sed-based header generation by
1957replacing it with a more conventional array of #defined symbols, with #ifdefs
1958for missing symbols on the three libc implementations (glibc, bionic, musl)
1959we've tested so far. (This requires portability.h work to add new supported
1960build environments, but at least it works on bionic now.)
1961The <b>file</b> command now fails gracefully when reading pathologically
1962broken ELF files (the tests for integer overflow missed one), and
1963<b>date</b> no longer shows a meaningless errno when it can't parse the
1964date format.
1965Several fixes in <b>ping</b> (-w and -W didn't work, -c kept sending packets
1966while waiting for replies, and when no reply packets were received the summary
1967data was printed twice).
1968Elliott fixed <b>wc</b> column widths (traditional reality doesn't match
1969posix), fixed <b>modprobe</b>'s error handling (reported by Wen Xie), and
1970also fixed the units on blockdev --blkraget (reported by Martijn Coenen).
1971Several small fixes to the <b>stat</b> command's output (help text, quoting
1972style, leading zeroes, switch -f from %t to %T), <b>oneit</b> no longer
1973reboots the system when it's not called as PID 1.
1974Last release's commit letting $STRIP be redefined had a typo.
1975Nick Kralevich added an istty() check before calling TIOCGWINSZ (which
1976triggered Android's strict ioctl filtering on non-filesystem fifos).
1977An unstripped build (where the "strip" command fails for a given
1978CROSS_COMPILE) will now overwrite the (read-only) output file instead of
1979failing a rebuild. The dirtree plumbing no longer prints a "No" on front
1980of the filename in error messages (such as "permission denied").</p>
1981
1982<p><u>Docmentation</u>
1983Elliott updated the roadmap with Android AOSP status, towards turning that
1984into a hermetic build (hopefully with an eventual self-hosting option, work
1985is <a href=b33d37d6f735>already underway</a> on the airlock step).
1986Rob added a little more context for LSB and described the move from Aboriginal
1987Linux to mkroot.
1988The <b>ps</b> help text changed: now "ps -o help" shows the fields and the normal
1989--help is much more manageable. Ifconfig's --help text also got tidied up.
1990Our LICENSE file has the SPDX identifier (0BSD) now, and the note about
1991the kconfig/ subdirectory still having (build only, non-shipping) GPL code
1992in it moved into the kconfig subdirectory.</p>
1993
1994<p><u>Coding style change:</u>
1995GLOBALS() now uses the argument letter as the variable
1996name for variables automatically set by command line arguments via lib/args.c,
1997and no longer requires they be one per line.
1998Removed use of the gcc
1999\e extension in string constants, replacing it with \033.</p>
2000
2001<p><u>Library code:</u>
2002lib/interestingtimes.c was renamed to lib/tty.c.
2003The deflate code from toys/pending/compress.c moved to lib/deflate.c,
2004which gunzip.c now uses when it's not configured to use zlib. (This reduces
2005the bootstrap dependencies for the "airlock" step of a hermetic build.)
2006The compression side remains a todo item.
2007Added "%" to lib/args.c reading seconds (including fractions/units) into a
2008long milliseconds argument. Redid xparsetime() to not need floating point,
2009added xparsemillitime() for milliseconds common case, and
2010parsetime() now treats leading garbage as an error instead of returning zero.
2011The bufgetpwuid()/pufgetgrgid() functons can now handle more than 4k of group
2012data. The set_terminal() function can now set serial speed, and microcom.c
2013uses it. Added xsignal_flags(), and more consistently use xsignal() (which wraps
2014sigaction()) instead of signal().
2015A new xgetrandom() function calls the new getrandom() system call, falling
2016back to /dev/urandom on older kernel/libc (made to actually _work_ by
2017Patrick Oppenlander).</p>
2018
2019<p><u>Build infrastructure:</u>
2020Updated dependency detection so "make distclean defconfig toybox" now works
2021all on the same command line. (This is nice when $CROSS_COMPILE changes).
2022scripts/config2help.c no longer includes parts of lib/ and just copies what
2023it needs into itself, improving build portability.
2024The build now checks the specified cross compiler exists, hopefully providing
2025a more informative error message when it doesn't. Similarly, when .config is
2026missing the error message suggests running make defconfig.</p>
2027
2028<a name="23-06-2018" /><a href="#23-06-2018"><hr><h2><b>June 23, 2018</b></h2></a>
2029<blockquote>
2030<p>There is a theory which states that if ever anyone discovers exactly what
2031the Universe is for and why it is here, it will instantly disappear and be
2032replaced by something even more bizarre and inexplicable. There is another
2033theory which states that this has already happened.</p>
2034<p> - The Hitchhiker's Guide to the Galaxy</p>
2035</blockquote>
2036
2037<p><a href=downloads/toybox-0.7.7.tar.gz>Toybox 0.7.7</a>
2038(<a href=https://github.com/landley/toybox/releases/tag/0.7.7>git commit</a>)
2039is out.</p>
2040
2041<p><u>New commands</u>: Rob Landley added <b>getconf</b>, Elliott Hughes added
2042<b>uuidgen</b>, and <b>ping</b> and <b>fmt</b> were promoted out of pending.</p>
2043
2044<p><u>New options</u>: The <b>cp</b> command added the --parents option,
2045<b>readlink</b> now has -m to show where a missing path would be,
2046<b>netstat</b> got some updates (better nommu support, it can use /etc/services
2047names, more command line options are bounds checked),
2048Ng Zhi An added --getra and --setra (get/set readahead) to <b>blockdev</b>,
2049Elliott Hughes added <b>xxd</b> -o and made <b>top</b> -d understand fractional
2050sections (because the Linux Test Project uses that), and
2051Minghui Liu added <b>watch</b> -b.</p>
2052
2053<p><u>Bugfixes</u>: Several fixes to <b>vmstat</b>: Elliott Hughes made it work on a
205448-core machine
2055and fixed a glitch with -n, Haroon Maqsood pointed out "vmstat 1 1" didn't
2056stop and "vmstat -n" didn't print the first theader line.
2057Minghui Liu fixed a bug where <b>cp</b> -p didn't preserve
2058timestamps, and pointed out the b, c, and d suffixes on numbers weren't
2059working. Ryan Pritchard pointed out that <b>du</b> -d0 should act like du -s.
2060Filip Perich made an RFC2347 OACK compliance fix to <b>tftp</b> in pending.
2061Zach van Rijn fixed a bug in <b>xxd</b> causing incorrect translation of
2062uppercase characters.
2063Elliott fixed several things in <b>top</b> (removed spurious '\r' characters from -b
2064output, removed interactive flicker, made running processes bold), and
2065pushed Rob to make <b>file</b> work better recognizing things on stdin
2066("cat /bin/ls | file -" still won't work but "file - < /bin/ls" should).
2067Rob fixed a bug in <b>netstat</b> on 64 bit big endian systems,
2068and fixed <b>cut</b> -DF
2069(a posix compliance fix broke its ability to act as a decent awk replacment
2070because outputting all delimiterless lines isn't what you want there, -D
2071now disables that too).</p>
2072</u></p>
2073
2074<p><u>Documentation</u>: Rob added a buildroot section to the roadmap with the
2075prequisites that needs to run, and reformatted more of the ps --help text
2076to two columns.</p>
2077
2078<p><u>Library</u>:
2079FLAGS_NODASH is now set in toys.optargs when an optstring starting
2080with & has no dash in its first argument. (This lets "ps -ax" and "ps ax"
2081behave differently.) Factored out xtestfile() into lib/.
2082The comma-separated-list parsing infrastructure moved to lib/commas.c.
2083Added mkpath() for the common case of mkpathat() and #defined MKPATHAT_*
2084constants for the uncommon cases. Elliott added better error checking
2085to xparsetime() and fixed a bug in names_to_pid() (used in <b>pidof</b>
2086and <b>killall</b> and such).
2087Rob inlined the old toys/e2fs.h into pending's mke2fs,
2088and removed a rogue semicolon from the WOULD_EXIT() macro that screwed
2089up if() statement levels. The do_lines() semantics changed adding a
2090callback(0,0) at the end of each file.</p>
2091
2092<p><u>Tests</u>:
2093The test_* commands under toys/example were all renamed demo_* so
2094they don't share a namespace with running a standalone command
2095through the test suite (ala "make test_sed"), then
2096demo_human_readable became demo_number so it can also test atolx()
2097suffixes. The readlink.test was getting confused by a
2098behavior difference between toybox and ubuntu (Ubuntu's ln -sf wouldn't
2099replace a symlink that pointed to itself, toybox's would), now there's
2100an explicit rm so it works on ubuntu too. The test suite plumbing's
2101error message for a test with the wrong number of arguments now
2102correctly identifies which test it's complaining about, and it can now
2103run a test on a command name beginning with "-" (none yet but it
2104comes up in toysh).</p>
2105
2106<p><u>Building</u>: You can now override strip with $STRIP. The set of
2107"make install_airlock" commands got tweaked for the 4.17 kernel (which
2108needs bison and flex because the new kconfig plumbing doesn't have _shipped
2109versions of those generated files like the old kconfig did), and for
2110the new release (remove ping, toybox provides it now).
2111$LDFLAGS now applies to library probing (the android NDK provides
2112dynamic libraries without corresponding --static versions).
2113Elliott took a CFG_TOYBOX_DEBUG check out of an error_exit() to make
2114clang stop complaining about an error that should never happen.</p>
2115
2116<a name="24-02-2018" /><a href="#24-02-2018"><hr><h2><b>February 24, 2018</b></h2></a>
2117<blockquote><p>
2118"Arthur," said Ford.<br />
2119"Hello? Yes?" said Arthur.<br />
2120"Just believe everything I tell you, and it will all be very, very simple."<br />
2121"Ah, well, I'm not sure I believe that."<br />
2122</p>
2123<p> - The Hitchhiker's Guide to the Galaxy</p>
2124</blockquote>
2125
2126<p><a href=downloads/toybox-0.7.6.tar.gz>Toybox 0.7.6</a>
2127(<a href=https://github.com/landley/toybox/releases/tag/0.7.6>git commit</a>)
2128is out, a month and change late.</p>
2129
2130<p>This release adds a <b>crc32</b> command and
2131cleans up and promotes <b>iconv</b>, <b>logger</b>, and <b>setfattr</b>.
2132Pending added <b>stty</b> and <b>fmt</b>, the other half of <b>ping</b>,
2133and got a lot of cleanup on <b>gzip</b>/<b>gunzip</b>/<b>zcat</b>.</p>
2134
2135<p>The <b>cut</b> command got a complete rewrite adding support for multiple
2136ranges, utf8/unicode characters, and new -CDFO options. Use -C to measure
2137columns rather than characters (for double width chars), and "cut -DF 3-5"
2138selects words (like "awk '{print $3 $4 $5}'").</p>
2139
2140<p>This release adds <b>top -m</b> and <b>iotop -H</b> (and fixed top width truncating for narrow terminal windows),
2141added <b>iconv -c</b>, fixed <b>ps -T 1234</b> to show threads belonging to
2142that PID, improved <b>file</b>'s executable identification (properly recognizing
2143both endiannesses and adding sh/frv fdpic, bpf, and new microblaze
2144identifiers), and <b>blkid</b> now only shows LABEL= when it isn't blank.</p>
2145
2146<p>Running ./configure now does "make defconfig", help_exit() now shows the
2147"See %s --help" message on the same line, and several web pages
2148(about, code, roadmap, cleanup) got updates.</p>
2149
2150<p>Elliott and the Android devs added <b>df -i</b>, <b>uptime -p</b>,
2151<b>time -v</b>, and <b>xxd -i -ri</b>,
2152fixed <b>id -G</b> to only output the group IDs, fixed the <b>xargs -0 -n</b>
2153combination, made <b>find -exec +</b> obey ARG_MAX (just like xargs), made
2154<b>killall</b> kill scripts too, fixed an off by one error in <b>pidof</b>
2155that could cause false positive matches in 15 character names, added
2156gzip/gunzip/zcat tests, and taught <b>file</b> to identify more stuff
2157(ogg, TrueType font, LLVM bitcode, PEM certs, and PE executables), and
2158for some reason needed TOYBOX_VENDOR in the version stuff.
2159Android <b>getprop</b>/<b>setprop</b> now allows ro. properties to have
2160arbitrary lengths, <b>losetup</b> can now find loop devices under /dev/block (where android puts
2161them for historical reasons), and Andreas Gampe pointed out a thinko in
2162<b>top</b> that triggered llvm's address sanitizer checks.</p>
2163
2164<p>Gael PORTAY asked for mdev to work when Linux has the
2165block layer configured out, Patrick Oppenlander pointed out a problem
2166with make bloatcheck's build dependencies, and
2167xuphung on github fixed config2help building on MacOS X.</p>
2168
2169<p>Commands with locale support now setlocale(LC_TYPE, "C.UTF-8") for more
2170targeted locale support (enable character parsing with
2171utf8 support, use "C" semantics otherwise). The toys/example/test_*.c
2172commands got renamed demo_*.c.</p>
2173
2174<p>In lib.c the millitime() function returns current unix time in milliseconds
2175(which as a 64 bit number is good for + or - 292 million years from
2176midnight Jan 1 1970), and the old xgetty() function became tty_fd() returning
2177-1 instead of erroring out.</p>
2178
2179<p>The "make help" text moved from the Makefile to scripts/help.txt,
2180ls.c now lists some of its deviations from posix, and work is underway
2181to make ps.c more intelligible (and figure out how to break it up into
2182multiple files).</p>
2183
2184<a name="12-10-2017" /><a href="#12-10-2017"><hr><h2><b>October 12, 2017</b></h2></a>
2185<blockquote><p>Would it save you a lot of time if I just gave up and went
2186mad now?</p>
2187<p> - The Hitchhiker's Guide to the Galaxy</p>
2188</blockquote>
2189
2190<p><a href=downloads/toybox-0.7.5.tar.gz>Toybox 0.7.5</a>
2191(<a href=https://github.com/landley/toybox/releases/tag/0.7.5>git commit</a>)
2192is out.</p>
2193
2194<p>This is another <b>mostly bugfix release</b>. I delayed it a bit trying to get some
2195of the many half-finished projects (dd, ping, lsof, iconv, cut, sysconf,
2196the cp --parents option...) finished/promoted/tested, but decided adding big
2197things at the end of the dev cycle would be too destabilizing.</p>
2198
2199<p>A couple <b>new features</b> slipped in anyway. Ilya Kuzmich added head -c
2200and strings -t (and corrected strings' output in a couple places). Elliott
2201Hughes taught file to recognize Macintosh (Mach-O) binaries.</p>
2202
2203<p><b>NOTE</b>: the chrt command is broken when built against musl-libc
2204because that project's maintainer decided he didn't like the system calls
2205it depends on, so he
2206<a href=https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5>removed
2207them from his libc</a>. The prebuilt binaries are built against musl,
2208so the chrt command there will always say -ENOSYS. (It works fine built against
2209glibc, and presumably bionic or uClibc.)</p>
2210
2211<p><b>Bugfixes</b>: Remounting (mount -o remount,rw) should work properly now,
2212and we fixed another place ps was segfaulting when /proc
2213entries vanished out from under us (a hard to hit race condition
2214mostly noticeable in long-running "top" exiting; error value was treated
2215as a pointer). Elliott Hughes fixed a -Wformat warning in expr.c,
2216and made xargs split the command
2217line at ARG_MAX (which is 1/4 the stack size ulimit, which normal users
2218can arbitrarily increase but execve() will complain if they don't).
2219Josh Gao made netcat -l exit after handling a request, and reported a
2220seq bug (seq "1000000 10000001" output 1e+06, fixed now). Zach Riggle fixed a
2221broken URL in the README. Rob fixed env -0 checking the flag wrong so
2222it didn't work in combination with other flags (typo).</p>
2223
2224<p><b>Plumbing</b>:
2225Replaced mbrtowc() with a new utf8towc() that doesn't have a context
2226struct or care about locale. A bugfix in comma_scan() (wasn't removing an entry at the end
2227of the list) is why remount _sometimes_ worked (depending on argument
2228order), xgetaddrinfo() is now separate from xconnect(),
2229atolx_range() learned the w (word=2) suffix, and b is now (block=512) instead
2230of (byte=1).</p>
2231
2232<p><b>Build</b>: Patrick Oppenlander added a workaround for a bug in config2help.c
2233that resulted in segfaults on newer toolchains. (It was actually a use
2234after free error; the build infrastructure isn't nearly as heavily audited as
2235code that gets installed on the target; oops.) A new GITHASH
2236environment variable can force the build version from
2237the command line, and the build checks for .git in the top directory
2238before asking git what our version is (so it isn't confused by ../../.git
2239in an enclosing directory). Added a build #warning about musl intentionally
2240breaking chrt (as with nommu fork() musl now provides a broken
2241stub function so compile-time probes for its existence think it's there, and
2242you can't run the result to test behavior when cross compiling).</p>
2243
2244<a name="19-06-2017" /><a href="#19-06-2017"><hr><h2><b>June 19, 2017</b></h2></a>
2245<blockquote><p>It is a well-known fact that those people who most want to rule people are, ipso facto, those least suited to do it.
2246To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job.</p>
2247<p>- The Hitchhiker's Guide to the Galaxy</p>
2248</blockquote>
2249
2250<p><a href=downloads/toybox-0.7.4.tar.gz>Toybox 0.7.4</a>
2251(<a href=https://github.com/landley/toybox/releases/tag/0.7.4>git commit</a>)
2252is out. No new commands this time, but
2253<b>chrt</b> and <b>dmesg</b> got promoted out of pending.</p>
2254
2255<p><u>New features</u>:
2256Rob rewrote paste, which should work much better now, and added grep
2257-M and -S to match and skip wildcards respectively (useful with -r).
2258Elliott's updated dmesg has -T and --color. The file
2259command can recognize gzip now, uptime grew -s, date grew %N, env knows - as a first argument
2260means -i (posix!) and grew -0, ls defaults to -b
2261instead of -q now when there's a tty, and ls has a new -ll option (with
2262--full-time as a compatibility synonym) showing nanoseconds and (for some
2263reason) timezone. (Why do individual files have timezones?) Elliott added
2264"uudecode -o -" support. Illya Kuzmich taught head -v and -q. The cpio
2265code no longer adds the "TRAILER!!!" entry by default (initramfs extractor
2266doesn't care) without which you can concatenate cpio archives with "cat".
2267(Use the new --trailer option if you want the legacy behavior.)
2268In pending, fdisk compiles now and tar understands bzip2.</p>
2269
2270<p><u>Build</u>:
2271The "make install_airlock" target now symlinks bc from the host because
2272the kernel <a href=https://landley.net/notes-2013.html#28-03-2013>inexplicably</a>
2273needs that to build. This was motivated by
2274<a href=https://github.com/landley/mkroot>mkroot</a>, which builds under
2275a toybox airlock directory.</p>
2276
2277<p>Lots of work on the test suite, mostly from the Android guys who are now
2278running it under Android. This fixed several existing tests that didn't
2279pass, made more tests run on a toybox-only system, and so on. The test suite
2280infrastructure now has a second testing function,
2281"testcmd", which supplies the command name being tested (bypassing
2282shell builtins).</p>
2283
2284<p>Various android build and config fixes, getting closer to being able
2285to let android someday use scripts/make.sh instead of generated/* snapshots.
2286Also more work into building under android's NDK; not quite there yet
2287but much closer.
2288Use nproc in scripts/make.sh detect available processors (so you can control the SMP level with taskset).
2289Removed the old uClibc compatibility glue, it's been 5 years since their
2290<a href=http://lists.busybox.net/pipermail/buildroot/2016-December/180102.html>last release</a>.</p>
2291
2292<p>The new config option TOYBOX_PEDANTIC_ARGS checks arguments when there
2293are no arguments, so things like "uptime" no longer silently ignore arguments
2294you pass but instead refuse to run.</p>
2295
2296<p><u>Docs</u>:
2297The FAQ now has more than one entry. Commands no longer output the full
2298help text for argument errors but instead just say "See %s --help" with the
2299command name (in addition to the actual error message).
2300Elliott did a big period-ectomy on all the --help text, and
2301we cleaned up some tab/space inconsistency. The
2302non-html help -a output now has separators with the command name.
2303The top/iotop and pkill/pgrep help text now describe a lot more of what
2304the commands can do. Twitter's code of conduct page went down so we
2305mirrored the text locally.</p>
2306
2307<p><u>Bugfixes</u>:
2308Fixed a race condition in ps/top where a process that exited right as we
2309read its data returned a different error value than we were expecting (which
2310was causing long-running top instances to occasionally exit),
2311mount now gives an error if it can't autodetect the filesystem
2312type, ps no longer queries the terminal size when output isn't to a tty
2313(so "ps -A | cat" doesn't vary), date's chkmktime() was replaced with
2314simple range checks for fields (to avoid false positives from things like
2315timezones and daylight savings time), removed %s from date's help (we
2316didn't implement it, we have @seconds[.nanoseconds] instead), fixed
2317zcat's buffer flush logic (which was always failing on files larger
2318than 32k), and factor now detects requests for numbers >64 bits and fails
2319loudly instead of producing incorrect answers.
2320Elliott fixed touch -a/-m (they were backwards), and allowed ':' in
2321setprop's property names. Grep now exits with 2 for errors (so -q can
2322distinguish "didn't find" from "didn't work"), doesn't stop on symlinks
2323that point nowhere (there was an error_exit() that should just be a warning),
2324and provides error messages for files we could open but not read.</p>
2325
2326<p><u>Library</u>:
2327New library functions: strend() complements strstart(), minof()/maxof()
2328are min/max macros that evalute arguments once and autodetect type (why
2329isn't this in libc?), xmmap() checks MAP_FAILED (which is not NULL).</p>
2330
2331<a name="21-02-2017" /><a href="#21-02-2017"><hr><h2><b>February 21, 2017</b></h2></a>
2332<blockquote><p>Only six people in the Galaxy knew that the job of the
2333Galactic President was not to wield power but to attract attention
2334away from it. Zaphod Beeblebrox was amazingly good at his job.</p>
2335<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
2336
2337<p>Despite everything, <a href=downloads/toybox-0.7.3.tar.gz>Toybox 0.7.3</a>
2338(<a href=https://github.com/landley/toybox/releases/tag/0.7.3>git commit</a>)
2339is out. The <u>new commands</u> this time are <b>ftpget</b>, <b>ftpput</b>, <b>microcom</b>, and <b>ascii</b>.<p>
2340
2341<p>We also had two command _demotions_ out of defconfig:
2342<b>hostid</b> got moved to toys/example and
2343switched to "default n" because despite <a href=http://pubs.opengroup.org/onlinepubs/9699919799/functions/gethostid.html>still being in posix</a>
2344the concept of a unique 32 bit number identifying a system is something
2345Linux outgrew about the time Pauline Middelink wrote the first IP
2346Masquerading code. And Elliott did a complete rewrite of <b>dmesg</b> introducing
2347two codepaths that I didn't get a chance to unify and didn't want to
2348hold up the release for, so that's back in pending.</p>
2349
2350<p><u>New features</u>: Rob added units to <b>find</b> -atime and friends
2351(with the legacy -amin alias). Elliott added color and -w to dmesg, fallocate
2352-o, and improved file's ELF parsing. Steve Muckle added -d and finit_module
2353support to modprobe. Rob and Elliott tweaked the
2354ps/top display format a bit more (extending the USER field from 8 to 18 chars
2355and putting + at the end of string fields that got truncated).
2356df -a isn't entirely new, but wasn't documented and needed a bugfix.</p>
2357
2358<p><u>Bugfixes</u>:
2359Last release broke oneit because -c didn't get moved to xopen_stdio() (oops).
2360Rob and Elliott simultaneously spotted ps padding each line to 99999
2361chars when there's no tty (serial console or adb); now it pads to 80 in
2362that case but also switches on -w to avoid field truncation. The "tty"
2363field also sometimes had trailing debris (that's fixed now). And "top" was
2364endlessly redrawing with out tty because receipt of the ANSI size probe
2365results would set SIGWINCH, and handling that sent another ansi probe. (Sigh.)
2366And while we're there, replace "ADDR" with "BIT" in ps -l so there are
2367more than 4 chars left for the "CMD" field on 64 bit systems.</p>
2368
2369<p>Izabera pointed out that split -b and -l can't mix, and suggested seq should
2370multiply to avoid accumulating rounding errors from repeated fractional
2371increments. Wang Xiao Jian fixed a bug in sort -k.
2372Elliott let getprop use the @ character in property names, and
2373Dimitry Ivanov removed the name length limit for system properties.
2374Elliott also improved some error reporting and improved top -H's display
2375of thread names.
2376
2377<p>Josh Gao pointed out that recursive operations on . and .. could be ignored
2378in chmod -R (and the resulting generic fix to dirtree_notdotdot() fixed
2379it in several other places).</p>
2380
2381<p>Justin Cormack caught tar producing a warning to stdout that screwed up
2382"tar c" to stdout.
2383Rob fixed an option parsing bug (where switching off a --longopt in menuconfig
2384confused the parser), and another one where an option excluding itself
2385(ala "abc[-ab][!abc]" with "command -a -b") would segfault.</p>
2386
2387<p>There's some sort of gcc stack over-optimization bug where musl-libc's
2388version of vfork() doesn't get marked with attribute(returns_twice) so
2389stack varabiles in the same function after that get semi-randomly overwritten
2390when the optimizer decides to reclaim the space. So add the attribute
2391to the function the XVFORK() wrapper macro calls. (It's a nommu thing.)</p>
2392
2393<p>Fixed a couple variable size mismatch bugs that were only tested on 64 bit
2394(printf %x 64) or only tested on 32 bit (modprobe), removed some
2395unnecessary casts in stat.</p>
2396
2397<p>Continuing attempts to build under Android NDK brought up that posix
2398defines the global 'stdout' as a macro, which older versions of bionic
2399turned into an array member, but a function was using it as an argument
2400name. (This worked in the AOSP build because it only builds against current
2401bionic, where there's a global 'stdout'.)</p>
2402
2403<p>Several commits argued with clang's warning generation, eventually
2404settling on a variant of __attribute__((__shut_up__)).</p>
2405
2406<p>Android should no longer give spurious error messages
2407when you "ps -A | head" about EPIPE on output. (Older versions of bionic
2408set an error handler on SIGPIPE, but it shouldn't do that now. More recent
2409versions of adb set the SIGPIPE handler to SIGIGN instead of SIGDFL,
2410leading to write returning an error message instead of silently killing
2411the program. So we set it back to the default.)</p>
2412
2413<p><u>Docs</u>:
2414Removed website link to the gmane archive (which didn't survive gmane's
2415change of ownership). The FAQ now answers a _second_ question. (Woo!)
2416Some roadmap updates.</p>
2417
2418<p><u>Build tweaks</u>:
2419Upgraded "make install_airlock" target to only warn about missing
2420commands (unless $PEDANTIC is set) when it sets up the hermetic build
2421path. (The plan is still to implement everything but the toolchain
2422binaries in toybox, but in the meantime we're symlinking other stuff from
2423the $HOST that isn't ready yet. See
2424<a href=https://github.com/landley/mkroot>mkroot</a> for an example using
2425this.)</p>
2426
2427<p>Elliott and Rob continue to poke at building toybox with Android's NDK,
2428but it's a work in progress (<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-December/008767.html>thread</a>). Various changes
2429removing libcutils dependencies and adding an selinux dependency to getprop
2430are fallout from this.</p>
2431
2432<p>Cross-compiling from Macs needs to use "gsed" instead of apple's
2433version, so teach the build to use that name if it exists in the $PATH.
2434If you try to build without running config first, you should get better
2435error reporting now. Added a workaround for Centos' broken "which" command
2436producing output when it _can't_ find a name in the $PATH.</p>
2437
2438<p><u>Library</u>:
2439The new dirtree flag DIRTREE_PROC skips non-numeric entries so things
2440like ps and top can scan /proc more efficiently.</p>
2441
2442<a name="21-10-2016" /><a href="#21-10-2016"><hr><h2><b>October 21, 2016</b></h2></a>
2443<blockquote><p>Probability factor of one to one. We have normality. I repeat,
2444we have normality. Anything you still can't cope with is therefore your
2445own problem.</p><p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
2446
2447<p><a href=downloads/toybox-0.7.2.tar.gz>Toybox 0.7.2</a>
2448(<a href=https://github.com/landley/toybox/releases/tag/0.7.2>git commit</a>)
2449is out.</p>
2450
2451<p>During this development cycle, Elliott Hughes <a href=http://androidbackstage.blogspot.com/2016/07/episode-53-adb-on-adb.html>got interviewed
2452on the ADB podcast</a>
2453and Rob Landley <a href=http://linuxluddites.com/shows/episode-88/>got interviewed on Linux Luddites</a> (<a href=http://linuxluddites.com/shows/episode-11/>again</a>).
2454Both talk about toybox and many other things.
2455The web page also grew a new <a href=faq.html>FAQ page</a>, currently with
2456just the one.</p>
2457
2458<p>New comands added to defconfig are <b>tunctl</b>,
2459<b>log</b>, <b>start</b>, <b>stop</b>, and <b>sendevent</b>.
2460The commands <b>file</b> and <b>netstat</b> got promoted out of pending.
2461Pending added <b>chrt</b>, <b>setfattr</b>, and <b>getfattr</b>, and saw
2462a lot of cleanups to diffstat and dd but not enough to promote them to
2463defconfig yet. A new toys/net directory was added, moving ifconfig, netcat,
2464netstat, rfkill, and tunctl there so far.</p>
2465
2466<p><b>Upgrades</b>: All commands now parse --version when they understand
2467--help, but "true" and "false" should now ignore their arguments entirely.
2468We taught stat to handle "%12x" and "%.12x" printf-style escapes, which
2469apparently other versions do. The ifconfig output now shows the interface's
2470device driver. Added patch -d and --dry-run, wc can now do -cm together,
2471find has a NOP -noleaf so scripts that use that don't break, add -c to md5sum
2472and sha1sum. Elliott taught ps to treat extra aguments as additional -p
2473pids, implemented xxd -s, did a number of upgrades to file (added -HL,
2474support for ar files, improved ELF support to report android API level
2475and stripped/not stripped and it no longer prints a guessed build ID type).
2476Elliott also added optional build-time support for using openssl's
2477assembly-optimized md5sum/sha1sum implementations (leading to a new
2478<a href=design.html>design</a> policy on shared libraries).</p>
2479
2480<p><b>Bugfixes</b>: Too many fixes to "ps" and "touch" to list, from both
2481Elliott and Rob. Rob taught sed to handle s/[[:space:]/]// type sequences
2482properly, switched grep to a better
2483workaround for <a href=https://sourceware.org/bugzilla/show_bug.cgi?id=17829>glibc bug 17829</a>, made sed -i preserve ownership when run as root,
2484made du max out at 2 terabytes instead of 2 gigabytes on 32-bit systems
2485(it was always designed to, but was missing a typecast),
2486fixed the option parsing infrastructure (config options that remove command
2487line options got the placeholders wrong), fix to printf for printing
2488octal digits and handling the (posix-mandated) difference between %b and
2489non-%b octal output, reading from "-" no longer closes stdin when done,
2490netcat -L works with nommu (although it may need more portability work),
2491and you can now "make test_scankey" if you want to. Several commands
2492(stat, makedeves, chgrp, cp, find) handled user name lookup failure badly
2493(stat was segfaulting if you interrogated a file belonging to a nonexistent
2494user, "chown 12345 file" errored out if you didn't have that user
2495in /etc/passwd... now they should all print/accept the number when
2496appropriate). "LC_ALL=C ls -Cs --color" produces the same output
2497as other versions (two spaces padding, -k hardwired on).<p>
2498
2499<p>Kyungsik Lee fixed a bug
2500in cp (readlink() doesn't actually null terminate the string it reads in),
2501Elliott Hughes made pgrep/pkill return success/failure, fixed trailing
2502whitespace in netstat, fixed a SMACK symbol conflict due to linux/xattr.h
2503changing, fixed ls -sh, and added a lot of
2504stuff to the <a href=roadmap.html>roadmap page</a>.
2505Izabera pointed out cmp -l and -s can't be selected at the same time,
2506that timeout was never actually checking -v, that ls should default to -q
2507when output is to a tty, and that "file -" would sometimes try to open "-"
2508instead of stdin.
2509Usischev Yury pointed out a use after free error, and that id shouldn't
2510call exit() directly. Matthias Urhahn pointed out that stat(2) returns
2511hardwired 512-byte units, so stat.c was wrong. David Hedges pointed out
2512that route could only handle 10 character interface names when the kernel
2513can do 15 (it's still in pending for a reason, but fixed). Evgenii
2514Stepanov found and helped diagnose one of the more subtle ps bugs fixed
2515this time around.</p>
2516
2517<p>Calling "make test_blah" no longer causes make to error out if the last
2518test fails (and thus returns a nonzero error code).
2519Building single commands and the multiplexer used to require a "make clean"
2520between them (because they had different config files both of which were older
2521than generated/config.h so it didn't get rebuilt; now it just always
2522rebuilds it).</p>
2523
2524<p>The defconfig build is now slightly less broken on older centos versions
2525(although <a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-September/008664.html>the consensus</a> is that Centos is just generally broken).<p>
2526
2527<p>Several commands were over-using xprintf(), which flushes its output
2528to check for error (something you only need to do maybe once per line,
2529and even then maybe only in loops because xexit() flushes and checks
2530ferror() for you and adjusts the exit code if we wrote stuff to stdout
2531that couldn't be printed). Lots of little flushes are inefficient,
2532so most things can use normal printf(). (Retransmission
2533of short writes is presumably libc's problem since it's buffering the
2534output and all.)</p>
2535
2536<p><b>Library:</b>
2537New library functions readlink0() and readlinkat0() which properly null
2538terminates the symlink value (which the stock libc function inexplicably
2539doesn't).
2540The new do_lines() function interates reading lines from a filehandle
2541and calling a function on each line.
2542New function pollinate() factoring out netcat's poll() loop so things
2543like telnet can use it.
2544New functions getusername() and getgroupname() return a
2545char * given a uid/gid (and return a string representation of the number
2546if the lookup fails), and xgetpwnamid/xgetgrnamid were renamed to
2547xgetuid/xgetgid and now return an integer instead of a struct (also helping
2548handle lookup failures, you can still return the uid/get for "12345").</p>
2549
2550<p>Switched atolx() to use long long internally.
2551Renamed xopen() to xopen_stdio() and made a new xopen() that never returns
2552stdin, stdout, or stderr (duping /dev/null into the filehandles as necessary).
2553New function xopenro() opens a file read only with one less argument, and
2554understands that "-" means stdin. New flag WARN_ONLY tells these functions
2555to just print a warning on failure, and return -1 instead of exiting.
2556Misc new functions like openro() which defaults to the WARN_ONLY behavior
2557and notstdio() which dup()s a filehandle up beyond stdin/out/err backfilling
2558with /dev/null as necessary. The WARN_ONLY flag let us remove the failok
2559argument from loopfiles().</p>
2560
2561<p>New TOYFLAG_NOHELP disables --help processing (which "true" and "false"
2562should not do).</b>
2563
2564<p>The test suite now has NOSPACE=1 to ignore whitespace (using diff -b to
2565check results), which helps TEST_HOST pass the same tests as toybox.
2566Fixes to chattr and date tests. It also has a new variable $C with the
2567absolute path to the command being tested (bypassing shell builtins),
2568and a function testcmd() which is just like testing() except it prepends
2569the command name ($C) to the test command line as well as the test
2570description.</p>
2571
2572<a name="02-06-2016" /><a href="#02-06-2016"><hr><h2><b>June 2, 2016</b></h2></a>
2573<blockquote><p>When the 'Drink' button is pressed it makes an instant but
2574highly detailed examination of the subject's taste buds, a spectroscopic
2575analysis of the subject's metabolism, and then sends tiny experimental
2576signals down the neural pathways to the taste centres of the subject's
2577brain to see what is likely to be well received. However, no-one knows
2578quite why it does this because it then invariably delivers a cupful of
2579liquid that is almost, but not quite, entirely unlike tea.</p>
2580<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
2581
2582<p><a href=downloads/toybox-0.7.1.tar.gz>Toybox 0.7.1</a>
2583(<a href=https://github.com/landley/toybox/releases/tag/0.7.1>git commit</a>)
2584is out. (Yes, I forgot to update the --version string, but I already
2585uploaded the <a href=downloads/binaries/0.7.1>binaries</a>.)</p>
2586
2587<p>The website has https support now, you can "make cat ps ls"
2588to get standalone commands (and "make list list_pending" to see what's
2589available), and a whole lot of bugfixes and new options to existing
2590commands.</p>
2591
2592<h2>New Commands</h2>
2593<p>Rob implemented <b>ulimit</b>. In pending, Elliott Hughes implemented
2594file. and Lipi Lee implemented a simple wget. (Pending also had minor
2595cleanups to more and lsof, but no promotions this time around.)</p>
2596
2597<h2>New Options</h2>
2598<p>Izabera implemented env -u, suggested adding seq -w, made factor
2599use full unsigned 64 bit math even on 32 bit platforms, pointed out base64
2600-w0 should disable wrapping, and sped up wc -c.
2601Elliott Hughes added mount -o relatime, xxd -p -r, and od -w.
2602Sameer Pradhan (or possibly Bilal Qureshi) suggested adding stat -tL -c %m%t%T.
2603Tom Cherry added getprop -Z. Paul Barker added hostname -b and -F.
2604Rob added ls -b, made ls -q work with utf8,
2605made sed -f - read from stdin, and added top -O (like ps -O).</p>
2606
2607<p>Elliott and Rob <b>added Thread support to ps and top</b>,
2608with -o TID, TNAME. We also added -o PCY (android scheduling policy),
2609-o BIT (process is 32 or 64 bit), and -o TNAME now shows the parent
2610command name for threads.</p>
2611
2612<h2>Documentation</h2>
2613
2614<p>Rob added the sed invocations to convert tabs/spaces and back to
2615design.html. Isaac Dunham updated hexedit's help text.
2616Jakob Flierl pointed out a broken URL in the README.</p>
2617
2618<p>Rob also redid the naming scheme of sed's pattern manipulation code to
2619remove the gratuitous references to Roger Zelazny's "Amber" series, since it
2620was confusing people.</p>
2621
2622<h2>Bugfixes</h2>
2623
2624<p>Grep -H and -n should now work properly with -ABC. Andy Chu pointed out an
2625out of bounds access for zero length lines in rev, fixed a buffer overflow
2626in diff -r, and fixed operator precedence in expr (although Rob is rewriting
2627chunks of expr so toysh can use its plumbing for $(( )) ).
2628Patrick Ohly fixed the too-aggressive suid permission dropping logic.
2629Josh Gao fixed a segfault when find -iname got no argument, and
2630made tail -f work right with just one file argument.
2631Tom Marshall cleaned up tar's long filename support and improved
2632the tar tests, and reported another find bug (with -iname -o -iname not
2633tracking copy lifetimes properly) that got fixed.</p>
2634
2635<p>Elliott Hughes fixed wc -c to not trust zero length files to actually be
2636zero length (/proc does that), fixed "mount -o rw,remount /system" on
2637Android, removed trailing spaces on ps -o cmdline, fixed pkill -9 and
2638the corresponding tests, made "insmod -" work, fixed top -b and tail -NUM,
2639pointed out that ps shouldn't trim numeric fields for display size limits,
2640and added some more
2641explicit "sort" calls to make pipelines so build tempfiles are easier to cache.
2642Rob <a href=https://github.com/landley/toybox/commit/32b3587af261>fixed an insane sed thing</a> the perl 5.22 build was doing.
2643Fixed mount -o to properly pass in leftover string data, and
2644<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-March/004790.html>documented how to use toybox to mount nfs</a> (warning: kernel patch
2645to fix some bitrot in the kernel NFS driver's string parsing attached
2646to that message).</p>
2647
2648<p>George Burgess IV corrected some variable types in traceroute.
2649Base64 now wraps == properly. Fixed two bzcat segfaults reported by
2650John Regehr. Andy Chu found a segfault in "sed -e 'c\'" with no trailing
2651line, and implemented mv -n and cp -n. The cyanogenmod guys pointed out that
2652cp -a shouldn't complain if a non-root user can't chown, and we added
2653the output path to cp -r error messages while we were there (before was just
2654the filename).</p>
2655
2656<p>Samuel Holland fixed blkid's handling of vfat labels, and
2657fixed a segfault when basename was passed an empty string and an empty
2658suffix. Davis Mosenkovs fixed touch -t seconds parsing.
2659Rob fixed a bunzip bug reported by John Regehr (the bad CRC
2660error message was printing a NUL argument).</p>
2661
2662<p>Not all of the commands build standalone, but more of them do now;
2663scripts/single.sh can now build a "mv" that isn't actually "cp".
2664The dependencies are more granualr, so "make top; make ps" no longer
2665produces a broken ps that ignores -A (because ps.o wasn't getting rebuilt
2666even though top had the FLAG macros for -A zeroed).</p>
2667
2668<h2>Build</h2>
2669<p>Rob added a <b>warning when building commands out of
2670pending</b>. (The pending directory is full of code that hasn't been
2671properly vetted. Use at your own risk.)</p>
2672
2673<p><b>New build targets let you build individual commands by name</b>, ala
2674"make ls cat ps", and you can run the test suite for each standalone
2675command with "make test_ls" and such.
2676"make list" shows all such standalone commands in defconfig, and
2677"make list_pending" shows unfinished commands from toys/pending
2678("make list list_pending" shows both together). "make clean" now deletes
2679these filenames at the top level, and the corresponding unstripped files
2680live in the directory generated/unstripped.</p>
2681
2682<p>Nicholas Boichat suggested switching make.sh to use $! for process
2683enumeration during parallel builds (which is both more efficient and more
2684portable), and suggested shell builtin replacements for wc/awk/sed so the
2685build loop has fewer forks now.</p>
2686
2687<p>Lots of work on the test suite, much of it due to Andy Chu. It now
2688consistently prints the name of the command being tested at the start of each
2689test (and the common infrastructure does that, not each individual test), and
2690"make tests" actually runs all the available tests now.
2691Seperated pgrep and pkill tests, split lsattr/chattr, added fstype and base64
2692tests. The "tests/files" directory now collects files for tests to
2693use, with blkid, bzcat, and utf8 subdirectories: the $FILES variable
2694gives a path to it, so "$FILES/blkid/ext2.bz2" and so on.
2695The testsuite now has test files with 3 different types of "not utf8 output"
2696sequences that require escaping, plus some combining character torture
2697tests, direction reversals, and so on.</p>
2698
2699<p>Added dependencies on TOYBOX_FORK to various pending commands that need
2700nommu conversion (which should fix the allyesconfig build).</p>
2701
2702<p>Static builds with selinux should work again.</p>
2703
2704<h2>Library</h2>
2705
2706<p>New bufgetgrgid() and bufgetpwuid() functions cache previous lookup info
2707rather than repeatedly traversing /etc/passwd and /etc/group (which is slow).
2708Added xpipe() to lib to catch pipe creation failure.
2709The HELP_ macros generated by config2help.c now use a capital prefix
2710to avoid collicing with help_exit() and such.</p>
2711
2712<p>The dirtree infrastructure got a cleanup pass in preparation for adding
2713infinite recursion depth support (needed by rm -r), updated the
2714<a href=code.html#ib_dirtree>documentation</a> to describe the new
2715semantics (removing dirtree_start() and adding dirtree_flagread()).
2716Now dirtree_recurse() takes the new dirfd as an argument.</p>
2717
2718<p>Split out _xexit() from xexit() and let sigatexit() set multiple
2719callbacks.</p>
2720
2721<p>For years the man pages have said to #include <sys/types.h> to get
2722major/minor/makedev but now that glibc
2723<a href=https://sourceware.org/ml/libc-alpha/2015-11/msg00253.html>has vowed
2724to break existing programs</a> and replace it with another nonstandard header
2725not in posix or lsb, we added our own functions to lib/ to do the transform
2726ourselves (based on what the kernel actually expects).</p>
2727
2728<h2>Portability</h2>
2729
2730<p>Debian unstable started needing an extra header #include for some reason, and
2731although printf("%.*s", INT_MAX, s) worked fine on Ubuntu 12.04 it
2732didn't on 14.04, so added a workaround for that. Typecast a printf because
2733wchar_t isn't a rigidly defined size. RLIMIT_RTTIME was
2734added to the kernel in 2008 but you can't expect uClibc to have noticed yet,
2735nor did it #define MS_RELATIME (added in 2006), or prlimit (2010)...
2736(Given the improvements in musl and bionic, uClibc support may be dropped
2737in a future release.) Given that the xattr functions were added during
2738linux 2.5, we can #include its header unconditionally.</p>
2739
2740<p>Renamed basename_r() to something else to avoid conflicting with freebsd's
2741libc, and both scripts/install.h and scripts/config2help.c no longer include
2742toys.h (to make cross-compiling from systems we don't run on easier).</p>
2743
2744<p>Debian bug 635570 did something unspeakably nonportable, depending on
2745"sed -e 'a\'" (with no next line of the pattern, so an unterminated
2746continuation) to add a newline to the last line of input if and only if
2747that last line of the input didn't have a newline, and to take no other
2748action. This is well into "depending on a bug" territory, but we implemented
2749it because otherwise Debian's install broke. (Of course this behavior
2750is undocumented, non-obvious, and doesn't really make logical sense.)</p>
2751
2752<p>CONFIG_TOYBOX_NORECURSE now disables the stack measuring logic (which
2753was giving some "security" code fits). Also we typecast pointers to (long)
2754before comparing them to avoid spurious compiler "optimizations" that
2755break the code.</p>
2756
2757<a name="02-02-2016" /><a href="#02-02-2016"><hr><h2><b>February 2, 2016</b></h2></a>
2758<blockquote><p>"I checked it very thoroughly," said the computer, "and that
2759quite definitely is the answer. I think the problem, to be quite honest with
2760you, is that you've never actually known what the question is."
2761</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
2762
2763<p><a href=downloads/toybox-0.7.0.tar.gz>Toybox 0.7.0</a>
2764(<a href=https://github.com/landley/toybox/releases/tag/0.7.0>git commit</a>)
2765is out.</p>
2766
2767<p>The new commands in defconfig are <b>iotop</b>, <b>top</b>, <b>pgrep</b>,
2768and <b>pkill</b>
2769(most replacing corresponding versions from pending). Added grep -ABC,
2770swapon -d (discard), mkswap -L (label) and UUID support, and find -delete.
2771Izabera added free -h and unshare -f. Josh Gao implemented tail -f.
2772Jose Bollo submitted cp --preserve=context,attr. Kylie McClain added
2773mktemp -u.</p>
2774
2775<p>In pending there's the start of a vi command, and Sameer Pradhan contributed
2776a new dhcp6. This cycle saw several rounds of route cleanup and a little dhcp
2777cleanup, but neither are complete yet. Lipi Lee did some cleanup to netstat.c
2778and Elliott Hughes removed warnings from traceroute.</p>
2779
2780<p>Lots of updates to ps: several new -o options, -k (--sort) -O and -M,
2781improved compatibility with Android's historical behavior, and
2782extensive internal code cleanup (including the removal of all
2783the magic constants).</p>
2784
2785<h3><b>Website</b></h3>
2786
2787<p>Dreamhost restored the <a href="#12-21-2015">missing 11 months</a>
2788to the mailing list archive, in the process deleting the month after
2789that. Now they've asked if I have mbox files archiving the new
2790gap (between December 20, 2015 to January 21, 2016, and presumably they
2791could also fill in the gap from December 14, 2014 to January 3, 2015 that's
2792been there since the last time they did this),
2793but due to some gmail filtering I've
2794<a href=http://landley.net/notes-2012.html#15-10-2012>never
2795been able to disable</a>, my copy of those files is spread among 3 different
2796mbox files I'd have to sort/filter/collate. (It's on the todo list.)</p>
2797
2798<p>Added a code of conduct to the README (we're
2799<a href=https://engineering.twitter.com/opensource/code-of-conduct>borrowing twitter's</a>) because somebody
2800made it necessary.</p>
2801
2802<h3><b>Bugfixes</b></h3>
2803<p>Fixed another sed bug where any ] right after [ was skipped (not just the
2804first one in the range, so [[] didn't terminate). Fixed sort -f and added test cases.
2805Assume 80 columns in "ls -m | cat", ls -L is no longer backwards,
2806and ls of files with no paths no longer uses an uninitialized (zero) dirfd.
2807Several bugfixes
2808to find (Gilad Arnold fixed -perm, Daniel K. Levy fixed "find . -exec echo {}",
2809and while we're there I fixed find --prune, made "find . -execdir
2810echo {} + -execdir ls {} +" work, and ripped out the environment size
2811measuring code that checked for a 128k limit removed back in linux 2.6.22).
2812Elliott Hughes fixed the date command's parsing of 4 digit
2813years and documented the %s escape, fixed hwclock -u, and pointed out
2814that runcon needs to exec to do its job (not recursively call another
2815command_main() in the same process). Tom Marshall reported that blkid was
2816handling ext2 wrong. Mike Moreton corrected cpio extraction's uid and gid
2817values, and added a --no-preserve-owner option. Fixed the SUID permission
2818dropping logic (which was a bit over-zealous, preventing some commands from
2819running at all).</p>
2820
2821<p>I'm told that debian-testing broke its libc so the nsenter build breaks,
2822but my attempts to install the debian-testing network cd image under
2823qemu keep breaking. Maybe someday they'll fix it enough I can actually
2824reproduce the problem. (Debootstrap under unbuntu builds an ubuntu-flavored
2825chroot in which toybox builds fine.)</p>
2826
2827<h3><b>Documentation</b></h3>
2828<p>Rewrite of the about.html page, tweaks to design.html, and a re-triage of
2829sbase in roadmap.html. Update to mkstatus.py to collate multiple span
2830tags with the same id, resulting in a larger status.html page (which
2831was previously ignoring some commands in the roadmap).</p>
2832
2833<p>Expanded the defconfig/allyesconfig/allnoconfighelp text in "make help"
2834to explain what they're for.</p>
2835
2836<h3><b>infrastructure</b></h3>
2837<ul>
2838<li><p>Expanded toys.optargs to 64 bits so a command can have more than 32 options.</p></li>
2839<li><p>Added NOEXIT() wrapper to turn xwrap() functions into warning versions
2840using the existing longjump(toys.rebound) infrastructure.</p></li>
2841<li><p>Renamed dirtree->data to dirfd and stopped storing symlink length
2842into it (this fixed a bug where following symlinks to directories
2843didn't give a valid directory filehandle, noticeable with ls -Z).</p></li>
2844<li><p>New TAGGED_ARRAY() infrastructure generates index and bitmask macros
2845for arrays of structures starting with a name string.</p></li>
2846<li><p>New lib/linestack.c for utf8 fontmetrics (draw_str() and utf8len()
2847and so on), and for tracking multiple lines of text
2848(vi, less, shell history) that need wordwrapping and scrolling up/down.</p></li>
2849<li><p>Upgrades to lib/interestingtimes.c: scan_key() now has a timeout
2850in milliseconds and recognizes more sequences including ANSI
2851window size probes. New utf8 test files in tests/files/utf8 including
2852sequence reversing, stacked combining chars, and all three types of
2853unprintable sequences (low ascii <32 ala ^X, invalid utf8 sequences ala
2854<AB><CD>, and invalid unicode code points ala U+1234).</p></li>
2855<li><p>More comma handling code in lib.c: comma_args()</p></li>
2856<li><p>Added error_msg_raw() to shut up fortify's endless static checking false
2857positives.</p></li>
2858<li><p>readfileat() can now realloc() in a loop to read long files
2859("zcat | insmod" needed it).</p></li>
2860</ul>
2861
2862<h3><b>Roadmap</b></h3>
2863<p>We're getting close to having a self-hosting development environment
2864using toybox for the command line. The remaining busybox commands in
2865<a href=http://landley.net/aboriginal/about.html>Aboriginal Linux</a> are:</p>
2866
2867<blockquote><p><b>
2868awk bunzip2 bzcat bzip2 dd diff expr fdisk ftpd ftpget ftpput gunzip gzip
2869less ping route sh sha512sum tar test tr unxz vi wget xzcat zcat
2870</b></p></blockquote>
2871
2872<p>And the remaining non-busybox commands in Aboriginal Linux's build/host
2873directory (from the distcc, genext2fs, e2fsprogs, zlib, and squashfs packagesi)
2874are:</p>
2875
2876<blockquote><p><b>
2877mke2fs fsck.ext2 resize2fs distcc genext2fs unsquashfs distccd mksquashfs tune2fs
2878</b></p></blockquote>
2879
2880<p>Squashfs and distcc are probably out of scope for toybox, but mke2fs,
2881fsck.ext2, resize2fs, genext2fs, and tune2fs should all be added to the
2882above "busybox" replacement list.</p>
2883
2884<p>Remind me to include this countdown in future releases. Once they've all
2885been replaced, the next goal is <a href=http://landley.net/aboriginal/about.html#selfhost>building AOSP under itself</a>.</p>
2886
2887<p>See the full <a href=roadmap.html>roadmap</a> and <a href=status.html>status</a>
2888pages for more details.</p>
2889
2890<a name="12-21-2015" /><a href="#12-21-2015"><hr><h2><b>December 21, 2015</b></h2></a>
2891
2892<p>Yes, 11 months have gone missing from the mailing list web archive.</p>
2893
2894<p>Yesterday evening Dreamhost's mailman server went down (timing out trying
2895to connect). I poked them about it, they
2896<a href=https://twitter.com/landley/status/678781271670149121>blamed
2897DNS</a>, I explained that the hang was _after_ the DNS lookup and
2898entered the dig info into the trouble ticket showing the IPs the DNS
2899queries were returning, they reinstalled the server at that IP from what I
2900assume was their most recent backup, and that's how 11 months of messages
2901vanished out of the archive.</p>
2902
2903<p>I've <a href=https://twitter.com/landley/status/679114451975467008>continued to poke them about it</a> but I honestly believe that's the best they
2904can do. <a href=http://landley.net/dreamhost.txt>Last time</a>
2905this sort of thing <a href=http://landley.net/dreamhost2.txt>happened</a>
2906we went back and forth for months, so I added a link to a backup web
2907archive (in the nav bar on the left) that isn't controlled by dreamhost,
2908and thus doesn't gratuitously lose data on a regular basis. (I note
2909the earlier hole in Dreamhost's archive was never fixed either. That
2910was data never getting archived, this is a year's worth of data that
2911was in the archive until yesterday vanishing after the fact.)</p>
2912
2913<p>If you're wondering why the <a href=http://lists.landley.net>top level</a>
2914list page has been "temporarily disabled" for multiple years now... you'd have
2915to ask Dreamhost. I know I have. More than a dozen times.</p>
2916
2917<a name="03-11-2015" /><a href="#03-11-2015"><hr><h2><b>November 3, 2015</b></h2></a>
2918<blockquote><p>"Alright," said Ford. "How would you react if I said that I'm
2919not from Guildford after all, but from a small planet somewhere in the vicinity
2920of Betelgeuse?" Arthur shrugged in a so-so sort of way. "I don't know," he
2921said, taking a pull of beer. "Why - do you think it's the sort of thing you're
2922likely to say?"</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
2923
2924<p><a href=downloads/toybox-0.6.1.tar.gz>Toybox 0.6.1</a>
2925(<a href=https://github.com/landley/toybox/releases/tag/0.6.1>git commit</a>)
2926is out.</p>
2927
2928<p>We have a new <b>ps</b> command with all the -o fields posix wants (although
2929it doesn't accept BSD non-dash option syntax yet), and <b>bunzip2</b> (not just
2930bzcat but the proper extract-in-place command).
2931Sameer Pradhan added <b>hostid</b> and <b>fsync</b>.
2932Elliott Hughes added <b>flock</b>.
2933
2934<p>The people waiting for <b>human readable number support</b> (du -hH, ls -h,
2935and so on) can thank Elliott Hughes for implementing it. (Our output doesn't
2936exactly match others' because we our "binary" mode will say 1.0G instead of
29371024M, which is a bug in the other one we didn't emulate.)</p>
2938
2939<p>The other big news is <b>nommu support</b>, tested on the new
2940<a href=http://nommu.org/jcore>jcore</a> processor but presumaby working
2941on any nommu system. A few commands don't support nommu yet, but those
2942are disabled by dependencies on TOYBOX_FORK in menuconfig when building
2943for nommu. The roadmap now has a large section analyzing the uClinux
2944project (note that <a href=http://nommu.org>nommu.org</a> is slowly replacing
2945<a href=http://uclinux.org>uclinux.org</a> as the standard repository of
2946all knowledge and wisdom about nommu. The old site <a href=#12-02-2012>contains
2947much that is apocryphal</a>, or at least wildly inaccurate, and the new one
2948is trying to improve on that).</p>
2949
2950<p>Both "make change" and scripts/single.sh (for building standalone commands
2951without the multiplexer logic) now use the top level .config
2952for toybox global settings such as Linux Security Blanket Module selection,
2953(so make defconfig before change now).</p>
2954
2955<p>Documentation updates to the <a href=code.html>code</a> and
2956<a href=roadmap.html>roadmap</a> pages.</p>
2957
2958<h3>pending</h3>
2959
2960<p>In the pending directory Sameer Pradhan added tftp,
2961and Elliott Hughes sent lsof. Isaac Dunham upgraded mdev,
2962reboot, init, login, and modprobe, and fixed a distro-specific build break in
2963scripts/mkflags.h. Elliott Hughes and Lipi Lee made netstat -p handle
2964command lines longer than 21 characters, and Elliott fixed netstat -e and
2965some build warnings. Yeongdeok Suh fixed a warning in dhcpd.
2966I started cleanup on pgrep/pkill.</p>
2967
2968<h3>Command updates, bugfixes, and infrastructure</h3>
2969
2970<p>The multiplexer's "command not found" error exit is now 127, so now you can't
2971distinguish between a command not being found in the multiplexer and
2972the multiplexer itself not being found by the shell, because people wanted
2973that for some reason.</p>
2974
2975<p>Elliott Hughes made date reject invalid dates rather
2976than set the clock to something weird (setting the clock 100 years into the
2977future makes most Linux desktops surprisingly unhappy, and ntpdate won't fix it
2978either), fixed several ls -l display issues (user/group field ordering,
2979make user/group/lsmcontext left aligned), did the aforementioned
2980extensive work on human readable number output, fixed ionice's default
2981class, fixed a mv overwrite bug, made df's columns auto-size, added
2982--ppid and -Z to ps, and teamed up with Daniel K. Levy to fix
2983a segfault in find's handling of -newer -group or -user.</p>
2984
2985<p>Hyejin Kim added stat -c %T support. Colin Cross worked
2986on vmstat fixing
2987a header printing bug and calculating the bi and bo columns in the right
2988units. Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra
2989newline and that grep -w '\(x\)\1' didn't work, both now fixed.
2990Alistair Strachan fixed several problems with switch_root. Kylie McClain
2991pointed out env should be able to clear variables via NAME= syntax.
2992Dima Krasner added support for running blkid without a partition (so it shows
2993all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p>
2994
2995<p>Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra
2996newline and that grep -w '\(x\)\1' didn't work, both now fixed.
2997Alistair Strachan fixed several problems with switch_root. Kylie McClain
2998pointed out env should be able to clear variables via NAME= syntax.
2999Dima Krasner added support for running blkid without a partition (so it shows
3000all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p>
3001
3002<p>Two large thinko fixes in oneit: -3 was always enabled (which would
3003eventually block if the child never read the exiting PID numbers from its file
3004descriptor #3 until the pipe filled up), and the signal handlers weren't
3005set up right (for requesting semi-graceful halt/poweroff/reboot).
3006Calling install without a mode is now 0755, and install -g 0 no longer clashes
3007with cp --preserve. Better error message for ls -r on unreadable
3008directories, and ls -Z now uses O_PATH (with the /proc/self/fd/%d
3009workaround for kernel stupidity as necessary).</p>
3010
3011<p>Date now understands @unixtime[.fraction] and uses -D for
3012the set-side format (matching busybox's extension for this). The seq -f
3013string now checks that it's got exactly one %f escape with the correct
3014attributes (and a whole bunch of test cases for it). Fixed a bug
3015in od that screwed up the position indicator on arm and mips.
3016In stat the d/h units moved from %d %D to the default string.
3017And patch can now correctly apply hunks with trailing context to the start of
3018the file.</p>
3019
3020<p>The prompt argument moved out of yesno() (the caller can print the prompt
3021themselves). Replaced toys.exithelp with help_exit(). Added new
3022XVFORK() macro, and xpopen_both() calls /proc/self/exe when passed
3023a NULL argv (see cpio -p for example usage). Replaced toys.recurse
3024with toys.stacktop so the recurse or re-exec decision is now based
3025on bytes of stack space used. Marked a bunch of command-local functions
3026static.</p>
3027
3028<p>New additions to lib/ include strlower(), xconnect(), and the
3029aforementioned help_exit().
3030The testsuite now has some infrastructure tests based on "example"
3031commands such as toys/examples/test_human_readable.c.
3032The login command finally got a long-overdue cleanup (it's one of the
3033commands that predate the "pending" directory but were part of the reason
3034for it). Hexedit had an
3035uninitialized variable (of course gcc didn't spot it, it was too busy
3036warning about "may be used uninitialized but never actually is" variables).</p>
3037
3038<p>Tweaked makefile so
3039"make CROSS_COMPILE=prefix-" (as well as "CROSS_COMPILE=prefix- make",
3040which still works). Toybox is now installed chmod -w so broken installers
3041(like the bunzip2 package's) that try to overwrite existing binaries won't
3042knock out the whole of toybox.
3043GCC 5.2.0 stopped being able to compile Linux 2.6.12's kconfig, but
3044we added a workaround. You can now build uptime without utmpx.h.
3045Alejandro Joya pointed out that enabling smack required smack on the host
3046as well as target when cross compiling, which is now fixed.</p>
3047
3048<p>Note: toybox can autodetect nommu support when building with a uClibc
3049toolchain such as <a href=http://landley.net/aboriginal/downloads/binaries/old/1.4.3/cross-compiler-sh2eb.tar.gz>the one from Aboriginal Linux</a>,
3050but <a href=http://github.com/richfelker/musl-cross-make>with musl-libc</a>
3051you'll have to enable CONFIG_TOYBOX_MUSL_NOMMU_IS_BROKEN to work around the
3052fact they provide a non-functional fork() implementation that always returns
3053-ENOSYS, to prevent you from compile-time probing for nommu support when
3054cross-compiling. Unfortunately "preventing you from probing" seems to be
3055an explicit policy with musl, they also don't provide an "#ifdef __MUSL__"
3056because their library is perfect and you're only ever allowed to work around
3057other people's bugs, not theirs. So we have to use menuconfig to manually
3058enable musl-specific bug workarounds.</p>
3059
3060<a name="23-07-2015" /><a href="#23-07-2015"><hr><h2><b>July 23, 2015</b></h2></a>
3061<p>I recreated the <a href=downloads/toybox-0.6.0.tar.gz>0.6.0 source tarball</a>
3062(new sha1sum 08fb1c23f520c25a15f262a8a95ea5b676a98d54)
3063because I forgot to add --prefix to the git archive command when I updated
3064my release script from mercurial, so the files weren't in an enclosing
3065directory. (Ooops.)</p>
3066
3067<a name="19-07-2015" /><a href="#19-07-2015"><hr><h2><b>July 19, 2015</b></h2></a>
3068<blockquote><p>
3069The reason why it was published in the form of a micro sub meson electronic
3070component is that if it were printed in normal book form, an interstellar
3071hitchhiker would require several inconveniently large buildings to carry it
3072around in.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3073
3074<p><a href=downloads/toybox-0.6.0.tar.gz>Toybox 0.6.0</a>
3075(<a href=https://github.com/landley/toybox/releases/tag/0.6.0>git commit</a>)
3076is out. (Yes, git. See the <a href=#05-04-2015>previous news entry</a>.)</p>
3077
3078<p>Sorry for the unusually long gap between releases. Since last release Ye
3079Olde Project Maintainer traveled to japan twice and had two more "once
3080a century" floods at home. (Probably a coincidence.) Still catching up.</p>
3081
3082<h3><b>CELF/ELC talk and Wikipedia[citation needed] article</b></h3>
3083
3084<p>I gave another State Of The Toybox talk
3085(<a href=https://www.youtube.com/watch?v=04XwAbtPmAg>video</a>
3086<a href=http://landley.net/talks/celf-2015.txt>outline</a>), in which I
3087repeat my <a href=http://landley.net/notes-2013.html#07-11-2013>perennial</a>
3088<a href=https://twitter.com/landley/status/557309224535851009>complaint</a>
3089that Wikipedia[citation needed]
3090<a href=http://en.wikipedia.org/wiki/Toybox>still</a>
3091<a href=https://en.wikipedia.org/wiki/BusyBox#Controversy_over_Toybox>says</a>
3092toybox was relicensed before its hiatus, when relicensing was why
3093the hiatus ended.</p>
3094
3095<p>Since Wikipedia[citation needed] seems unable to do the
3096<a href=#15-11-2011>most</a>
3097<a href=http://landley.net/hg/toybox/log/tip/LICENSE>basic</a>
3098<a href=http://landley.net/notes-2011.html#13-11-2011>research</a> on
3099this point, and has stuck to an incorrect sequence of events for years,
3100I've been gradually escalating my attempts to correct them. Toybox
3101came out of mothballs in November 2011 <b>because</b> it could be
3102relicensed. That's what opened up a new niche busybox wasn't already
3103filling with a 10 year headstart.</p>
3104
3105<a name="asterisk_back" />
3106<p>The article has plenty of smaller issues<a href=#asterisk>*</a>, but
3107given that I gave an entire talk at Ohio LinuxFest in 2013
3108(<a href=http://landley.net/talks/ohio-2013.txt>outline</a>,
3109<a href=https://archive.org/download/OhioLinuxfest2013/24-Rob_Landley-The_Rise_and_Fall_of_Copyleft.mp3>audio</a>) on why I switched away from GPL for
3110my projects, that one bugs me.</p>
3111
3112<h3><b>New stuff this release</b></h3>
3113
3114<p>There's a new android menu in menuconfig, and rather a lot of Linux
3115Security Module support (Smack for Tizen from Xavier Roche and José Bollo,
3116and SELinux for Android from Elliott Hughes; see
3117the Security Blanket menu under global settings in menuconfig) has
3118trickled in, although there's still more to come.</p>
3119
3120<p><b>New commands:</b> Added reset, nproc, ionice, and iorenice.
3121Elliott Hughes contributed xxd, runcon,
3122restorecon, load_policy, getenforce, setenforce, getprop, and setprop.
3123Promoted shred, nsenter, and hwclock.</p>
3124
3125<p>You can once again build catv now the flag infrastructure's been updated to
3126let it coexist with cat -v.
3127And on a long plane flight I wrote
3128hexedit, an interactive hex editor that implements the start of
3129cursor control infrastructure (for eventual use by less and vi and shell
3130command history and so on).</p>
3131
3132<p><b>New options:</b> Added sed -E as a BSD-compatible synonym for -r.
3133Upgraded oneit with -r (restart), -3 (send exiting PID values to child),
3134and signal handling. Added -v option to timeout, -m to mknod, -u to shred,
3135-t to dmesg, and -123 to head and tail. Added implicit "." to grep -r without
3136any files to work on. Hyejin Kim requested prefix support for truncate -s.
3137Greg Hackman added -inum to find.
3138Jan Cybulski added the smack side of ls -Z support. Various patches also
3139added -Z to mkdir, mknod, and mkfifo.
3140Basic cp --preserve support went in, but not yet the xattr/LSM parts.</p>
3141
3142<p>The toybox command now has a --version option,
3143which uses "git describe" if available.</p>
3144
3145<p><b>Build infrastructure:</b>
3146The "make change" target now saves the output of each failed standalone
3147command build in a .bad file, and "make defconfig" is quieter now.</p>
3148
3149<p>Paul Barker submitted a large patch changing command install paths so
3150"toybox can be installed alongside busybox without confusing
3151update-alternatives". (There's some argument over
3152what the right paths should be, and I'm waiting for
3153people to tell me what else needs fixing because I have no idea. I've
3154been symlinking /bin to /usr/bin since 2002
3155<a href=http://landley.net/writing/hackermonthly-issue022-pg33.pdf>for
3156historical reasons</a>.)</p>
3157
3158<p><b>Docs:</b> The repository link now goes to github, with another link
3159to the commit rss feed.</p>
3160
3161<p>Elliott Hughes updated the Android section of the roadmap
3162(and he would know). Redid bits of scripts/mkstatus.py to make updating
3163status.html easier, and the README is larger.</p>
3164
3165<p>More description of option parsing in code.html, which now describes the
3166FLAG_x macros, switching flag macro sets with FOR_newcommand, how
3167configuration zeroes flag macros and using FORCE_FLAGS to suppress the
3168zeroing of options shared between commands. Also added description of ";"
3169to make --longopts take an optional =value part, and more about TOYBOX_DEBUG
3170to check NEWTOY() option strings (otherwise a bad option string makes
3171lib/args.c obviously segfault, but doesn't explain why).</p>
3172
3173<p>Added a "Why 0BSD?" section to license.html when submitting zero clause bsd
3174to SPDX (according to the pending license spreadsheet, it's been approved for
3175SPDX 2.2).</p>
3176
3177<p>The old list of commands needing cleanup but not in pending was
3178removed from toys/pending/README and instead the issues were added
3179as TODO comments in the individual commands.</p>
3180
3181<p><b>Bugfixes:</b>
3182Fixed mount -a segfaulting without -O (reported by Janus Troelsen),
3183and made it try a "become rw" ioctl() on the block device before falling
3184back to mounting read only (because Android expects that).
3185Fixed printf -- and printf ---. Lots of tweaks to ls -l spacing with
3186different options. Make touch -d and -t actually set time when you don't
3187specify nanoseconds.
3188Fixed a subtle bug where recursive calls (toybox commands that run other
3189toybox commands) weren't resetting all their state. (This manifested as
3190a "no }" error from "find | xargs sed", but could cause other problems.)
3191And David Halls reported another sed bug trying to compile libiconv (which
3192left extra \ at the start of lines in a generated shell script, breaking
3193the build). Output an error message for "cat /mnt".</p>
3194
3195<p>Kylie McClain reported that mktemp broke when $TMPDIR was set to an empty
3196string (which is not the same as unset), that install/find didn't support
3197numeric uid/gids, and that sort -z affects both input and output.
3198Isabella Parakiss fixed a printf.c bug.
3199David Halls fixed bugs in install -D and find -exec. Samuel Holland
3200fixed unshare -r. Hyejin Kim fixed makedevs with a count of 1, fold -w
3201range checking, an error path in scripts/mkflags.c, added -i to dhcpd,
3202and stopped su from prompting the root user for the new user's password.
3203Jan Cybulski spotted wrong indentation when combining ls -s and -i with -C and
3204-x. José Bollo fixed stat %G. Sameer Pradhan fixed a bug in mkfifo -Z.</p>
3205
3206<p>Elliott Hughes asked for a default SIGPIPE handler to disable
3207the signal handler bionic's dynamic loader installs (yes really). Still not
3208100% sure what the correct behavior is there. (Posix is
3209(<a href=http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/10915>actively unhelpful</a>, but at least they're taking
3210<a href=http://austingroupbugs.net/view.php?id=789#c1976>years to
3211make up their mind</a>. Elliott also sent patches to fix a typo in
3212useradd.test, add missing arguments to error_exit() calls and clean up
3213printf() format strings, fix an off by one error in human_readable(),
3214fix dmesg -c error reporting, fix a segfault in comma_scan where the option
3215was the last item in optlist (triggered by mount -o ro,remount), fix
3216hwclock -w, made ifconfig print lowercase MAC addresses (it was bothering
3217him), and make terminal_size() read the right environment variable
3218(LINES, not ROWS). And he suggested the test suite notice high command exit
3219values (corresponding to segfault or other signals).</p>
3220
3221<p>People are apparently using toys/pending commands, despite the police tape
3222and flashing lights, so added louder warnings to toys/pending/README.
3223Elliott Hughes fixed various problems with tar, dd, more, and top.
3224Hyejin Kim cleaned up syslogd and dumpleases. Isaac Dunham added hotplug
3225support to mdev. Yeongdeok Suh added RFC-3315 ipv6 support to dhcpd.</p>
3226
3227<p>I rewrote ps.c from scratch (in pending), but it's not ready for real use
3228yet.</p>
3229
3230<p><b>Portability:</b>
3231On the portability front Bernhard Rosenkranzer fixed a problem where the
3232menuconfig code wouldn't compile in C99 mode. (This led to me documenting
3233the craptacular nature of kconfig in a README, and the plan to replace it
3234sometime before 1.0.) Some extra flags to shut up overzealous llvm warnings
3235were added (and have to be probed for because gcc complains about
3236arguments it doesn't recognize even when they switch stuff _off_ using
3237a standard syntax). Don't depend on malloc(0) to return non-null in ls.
3238David Halls fixed some mac/ios portability issues,
3239implying somebody's built at least part of toybox on a mac.</p>
3240
3241<p>Added basename_r() to lib/lib.c because the posix semantics for basename()
3242are stupid but what the gnu guys did to it was appalling.
3243Turns out bionic already had a basename_r(), but posix still doesn't.
3244Fixed it up in portability.h, but this
3245could break more stuff in future. (Correct fix is to lobby posix to add it,
3246which would probably take about 15 years...)</p>
3247
3248<p><b>Infrastructure:</b>
3249The build now checks $LDFLAGS for linker-only flags, and allows the strip
3250command to fail (binflt toolchains provide a strip that doesn't work).
3251Since time.c uses floating point, added TOYBOX_FLOAT dependency in config.</p>
3252
3253<p>There's a lib/lsm.h defining varous inline functions for linux
3254security modules stuff, if (lsm_enabled()) should turn into a compile-time
3255constant 0 and let code drop out when TOYBOX_LSM_NONE selected, but
3256testing against CFG_TOYBOX_LSM_NONE or derived symbols is still useful
3257becuase when it _is_ enabled the probe turns into a system call you
3258don't want to repeat too much.</p>
3259
3260<p>Switched a bunch of commands from signal() to xsignal(). Factored out
3261xgetgrnamid() and xgetpwnamid() into xwrap.c. Make time.c depend on
3262TOYBOX_FLOAT (since it always uses float so shouldn't be available on
3263build targets without even software float). Added readfileat() to lib/lib.c.</p>
3264
3265<p>The dirtree infrastructure now passes in full flags for the old symlink
3266field, and the new DIRTREE_SHUTUP flag disables warnings if a file vanishes
3267out from under you during traverse. New dirtree_start() wrapper to
3268create dirtree root with only two arguments.</p>
3269
3270<p>The not-curses infrastructure introduced by hexedit mostly moved to
3271lib/interestingtimes.c.</p>
3272
3273<a name="asterisk" />
3274<a href="#asterisk_back" />Asterisk:</a> such when
3275Tim contacted me (my blog says a couple days before nov 13, 2011, I.E.
327611/11/11 not some specific day 2 months later) to ask if I wanted to work
3277on a new project he was proposing called
3278<a href=http://www.elinux.org/Busybox_replacement_project>BentoBox</a>
3279(because I used to do busybox, he'd forgotten toybox existed
3280until I brought it up). And don't ask me what "focuses not on compatibility
3281with its GNU counterparts" means when CP_MORE adds 7 non-posix options
3282and toys/other has 84 commands in neither posix nor LSB. I think they're
3283struggling to explain the difference having dismissed "licensing" as being
3284the reason it started up again after a long hiatus? The reason I don't think
3285GNU is special is there are a half-dozen other independent
3286implementations of the same unix command tools out there (AT&amp;T,
3287BSD, Coherent, Minix, plan 9, busybox, toybox, and several more analyzed in
3288the <a href=roadmap.html>roadmap</a>, and that's ignoring the implementations
3289written for DOS or in assembly over the years). But I do care what
3290Linux From Scratch expects, and if it's
3291<a href=http://archive.linuxfromscratch.org/lfs-museum/7.6/LFS-BOOK-7.6-NOCHUNKS.html#ch-tools-gcc-pass1>calling mv -v</a>
3292then I impelement mv -v
3293even if <a href=http://landley.net/toybox/roadmap.html>posix hasn't got
3294it</a>. And I don't know why "gnu counterparts" would describe this when
3295util-linux isn't a gnu package, nor are info-zip, e2fsprogs, kmod, less,
3296procps, shadow, sysklogd, vim, zlib, sudo, dhcpcd...</p>
3297
3298<a name="05-04-2015" /><a href="#05-04-2015"><hr><h2><b>April 5, 2015</b></h2></a>
3299<p>Since <a href=https://android.googlesource.com/platform/external/toybox/>android</a> and
3300<a href=https://git.tizen.org/cgit/platform/upstream/toybox.git>tizen</a>
3301and <a href=https://github.com/kraj/meta-musl/tree/master/recipes-core/toybox>openembedded</a>
3302and <a href=https://packages.gentoo.org/package/sys-apps/toybox>gentoo</a>
3303and so on have all been using Georgi Chorbadzhiyski's git mirror rather
3304than the mercurial repository, I bit the bullet and switched the project's repo
3305<a href=https://github.com/landley/toybox>to git</a>. Georgi's
3306<a href=https://github.com/gfto/toybox>mirror</a> is now pulling from that.</p>
3307
3308<a name="25-02-2015" /><a href="#25-02-2015"><hr><h2><b>February 25, 2015</b></h2></a>
3309<blockquote><p>"A common mistake that people make when trying to design
3310something completely foolproof is to underestimate the ingenuity of
3311complete fools."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3312
3313<p><a href=downloads/toybox-0.5.2.tar.gz>Toybox 0.5.2</a>
3314(<a href=/hg/toybox/shortlog/1702>commit 1702</a>) is out.</p>
3315
3316<p>New promoted commands: sed (finally fixed enough it builds Linux From
3317Scratch), printf (cleaned up and promoted), shred and
3318base64 (the Tizen guys wanted them), getenforce, setenforce, and chcon (android),
3319mix (promoted with fixes from Isaac Dunham), nsenter (from
3320Andy Lutomirski, merged into unshare).</p>
3321
3322<p>Elliott Hughes submited a bunch of patches to support Android (to
3323both toybox and Bionic libc, which he maintains). On toybox's end this
3324involved a lot of fixups to portability.[ch] and fixes to over a dozen
3325commands, plus several new ones. Other portability fixes included working
3326with buildroot's uclibc fork and building for nommu targets.</p>
3327
3328<p>The new "make change" target builds each toybox command as a standalone
3329binary. Rather a lot of commands that didn't build by themselves (mv depending
3330on cp and so on) were hit with a large rock until they built standalone.
3331This involved rewriting bits of option parsing, more elaborate dependency
3332generation, making each command have its own config
3333symbol and main() function (even when it's just a wrapper calling another
3334command's main()), and so on. Also, some commands can't be built standalone
3335at a conceptual level: "help" describes other enabled commands and "sh"
3336has a number of bulitin commands (cd, exit, set) that require the
3337multiplexer infrastructure, so "make change" filters them out.</p>
3338
3339<p>The mailing list's web archive is still screwed up. Dreamhost has
3340been trying to fix it since approximately September. There are
3341<a href=http://www.mail-archive.com/toybox@lists.landley.net/>two</a>
3342<a href=http://news.gmane.org/gmane.linux.toybox>other</a> less broken
3343archives, but neither has quite the same UI as mailman.</p>
3344
3345<h3>Bugfixes and tweaks</h3>
3346
3347<p>Cynt Rynt sent in tests for ifconfig,
3348Robert Thompson taught factor to accept whitespace separated arguments,
3349Hyejin Kim pointed out that some of mktemp's longopts were attached to
3350the wrong short options,
3351Luis Felipe Strano Moraes fixed a wrong free() call in bootchartd in pending.
3352Patches from Ashwini Sharma to make "df /dev/node" work, prevent du from
3353looping endlessly following symlinks, and to make expr.c
3354(in pending) understand == and regex matches. (Speaking of expr, it gets
3355priority groupings wrong but the bug was actually in the posix spec's
3356HTML conversion. They fixed the posix spec upstream for us. Still need
3357to fix the expr code, but it's in pending for a reason...)</p>
3358
3359<p>Some commands grew new option flags, such as cp --remove-destination
3360and touch -h.</p>
3361
3362<p>The parallel build has better error reporting now. When toybox needs to
3363re-exec itself to regain suid root permissions and hasn't got the suid bit,
3364it now gives the right error message ("not root" instead of "no such command").
3365
3366<p>Added a test to "mount" to not mount the same device/directory combination
3367over itself (the OS catches this for block devices, but not for tmpfs).
3368Make blkid distinguish ext3 from ext4. Added catv back into cat (because
3369the Android guys wanted it, and they have historical usage on their side,
3370so...). Handle nanoseconds in touch.</p>
3371
3372<p>Fixed a segfault when CP_MORE was disabled (the resulting option flag list
3373no longer defined -d but still had it in option groups at the end).
3374Workaround for glibc redefining dirname() and basename() to random non-posix
3375semantics because gnu. (They could have created dirname_r() but didn't want
3376to.)</p>
3377
3378<p>Fix an ifconfig test that was preventing assigning an ipv4 address to
3379interface aliases. Several cleanup passes on hwclock but not quite
3380promoted out of pending yet.<p>
3381
3382<p>Fixed a wrong error message in rm (if you had a chmod 000 directory and
3383did rm -r on it without -f, after the prompt it would complain it was a
3384directory, which was not the problem).</p>
3385
3386<p>The gzip compression code now does "store only" output to stdout, for
3387what that's worth.</p>
3388
3389<p>Cleanup mountpoint and expand, and remove them from toys/pending/README
3390(a list of commands that predate the toys/pending directory but needed
3391another pass).</p>
3392
3393<h3>Library and infrastructure:</h3>
3394
3395<p>Reworked the option parsing infrastructure so more commands build
3396standalone (via scripts/single.sh or "make change"). The option flag bit
3397values are no longer packed, it leaves spaces where currently disabled
3398flags go, and you can #define FORCE_FLAGS so disabled flags aren't zeroed.
3399This allows multiple commands to more easily share infrastructure, even if
3400your current flag context is for a disabled command (switched off in config),
3401you can force them to stay on and as long as the flags read the same right
3402to left they'll have the same values.</p>
3403
3404<p>We've started removing use of strncpy() because it's a hugely broken
3405standard C function: the length is the maximum length to _append_, not
3406the size of the destination buffer. It memsets the remaining space it didn't
3407copy ala "memset(dest+strlen(dest), 0, len);" so
3408if you think len is the size of dest you're guaranteed to stomp memory off the
3409end). And if it runs out of space it won't null terminate because reasons.
3410(Meanwhile sprintf("%*s", len, str) is counting wide characters in your current
3411locale, so if you set a locale other than "C" it will also go past your
3412allocated buffer size. Whoever is maintining the C library standards is really
3413bad at strings.)
3414Instead we have xstrncat() which will error_exit() if src+dest+1 doesn't
3415fit in the buffer. (Because randomly truncating input data isn't necessarily
3416an improvement.) And there's always xmprintf().</p>
3417
3418<p>Similarly, strtol() doesn't return an error indicator on overflow,
3419you have to clear and then check errno. So new xstrtol() that cares
3420about overflow.</p>
3421
3422<p>The bionic and musl guys agree faccessat(AT_SYMLINK_NOFOLLOW) is not
3423supported, so stop using it.</p>
3424
3425<p>Fixed toy_exec() to detect when argc is in optargs, so we don't
3426need a separate xexec_optargs().</p>
3427
3428<a name="18-02-2015" /><a href="#18-02-2015"><hr><h2><b>February 18, 2015</b></h2></a>
3429<p>Dreamhost continues to be unable to make mailing list archives work, so
3430here's <a href=http://www.mail-archive.com/toybox@lists.landley.net/>another
3431list archive</a> with a less awkward interface than gmane.</p>
3432
3433<p>(Neither gives you the convenient historical monthly views of mailman,
3434but I still have hopes dreamhost will someday figure out what they're doing
3435wrong. They've only been trying since October. Last month they did a
3436<a href=http://www.dreamhoststatus.com/2015/01/14/discussion-list-hardware-maintenance/>hardware upgrade to fix a software problem</a>, and the stale
3437data loads much faster now, so that's something.)</p>
3438
3439<p>Update (Feb 19): the archive started updating again, by discarding
3440all the pending data. So there are now _two_ giant holes in Dreamhost's
3441web archive, from Dec 15-Jan 3, and then another hole from Jan 16-Feb 18.
3442The relevant messages are in both of the other archives. Here's hoping
3443the chronic archive constipation problem won't happen a sixth time.</p>
3444
3445<a name="30-12-2014" /><a href="#30-12-2014"><hr><h2><b>December 30, 2014</b></h2></a>
3446<p>Due to Dreamhost's <a href=http://landley.net/dreamhost.txt>ongoing</a>
3447<a href=http://landley.net/dreamhost2.txt>inability</a> to make mailman
3448work reliably, I've added a link to a backup web archive at
3449<a href=http://news.gmane.org/gmane.linux.toybox>gmane</a> to the nav bar
3450on the left.</p>
3451
3452<p>You still subscribe to the list through
3453<a href=http://lists.landley.net/listinfo.cgi/toybox-landley.net>the first link</a>.</p>
3454
3455<p>Update (January 27, 2015): they're <a href=https://twitter.com/landley/status/558428839462703104>still working on it</a>.</p>
3456
3457<a name="19-11-2014" /><a href="#19-11-2014"><hr><h2><b>November 19, 2014</b></h2></a>
3458
3459<blockquote><p>"This time it was right, it would work, and no one would have to get nailed to anything."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3460
3461<p><a href=downloads/toybox-0.5.1.tar.bz2>Toybox 0.5.1</a>
3462(<a href=/hg/toybox/shortlog/1566>commit 1566</a>) is out.</p>
3463
3464<p>It's an interim release, mostly bugfixes. There are several new commands,
3465but they're all in pending.</p>
3466
3467<h3>Development</h3>
3468
3469<p>Finally implemented sed, which is still in pending because although
3470it's feature complete according to posix, and even passes the parts of
3471Busybox's sed test suite that aren't explicitly testing for gnu bugs we
3472don't want to copy, it's not yet good enough to build Linux From Scratch.
3473(The ./configure stages use very long sed scripts. 20 commits worth of
3474implementation and debugging, just under 1000 lines of code, and there's
3475still more to do. We're definitely up to some of the "fiddly" commands now.
3476Did you know "echo hello | sed p - -" segfaults gnu sed in Ubuntu 12.04?
3477Yeah...)</p>
3478
3479<p>Talked with the Tizen developers to follow up on their desire to
3480make toybox a part of the base Tizen system, and got a list of commands
3481to add to the roadmap. The tizen todo list is:</p>
3482
3483<blockquote><p>
3484wget, sha256*, gzip, gunzip, bunzip2, rsync, zdiff*,
3485less, ar, arch, base64, csplit, dir, fmt, join, 
3486nproc, shred, shuf, stdbuf, stty, test, tr, unexpand,
3487users, vdir, diff3, sdiff, dosfsck (fsck.vfat), awk, fdisk
3488</p></blockquote>
3489
3490<p>(Most of which was already on the todo list, but it helps prioritize.)</p>
3491
3492<p>Fixed md5sum and sha1sum on big endian systems (reported by James McMechan).
3493Andy Lutomirski fixed unshare's help text and option parsing,
3494and submitted nsenter (a tool to use setns(2)) to pending.
3495Isaac Dunham implemented acpi -ctV options, and spotted the bug that ls -d
3496was inappropraitely following command line symlinks without -H or -L (it
3497should act like ls -l does), and ls -F handles symlinks wrong too.
3498Lukasz Szpakowski sent in two bugfixes to tail.c. Cynt Rynt spotted an
3499unnecessary assignment in lib/password.c.</p>
3500
3501<p>Ashwini Sharma's team was as busy as usual, submitting tr, crontab, and
3502ipcrm, and hwclock to pending, more features to the pending ip.c, and a
3503pile of bugfixes (to chgrp, killall, ifconfig, insmod,
3504losetup, comm, cp, id, xwrap, netcat, modprobe, nohup...) mostly found by
3505static analysis. (These fixes are mostly to seldom-used codepaths like the
3506TOYBOX_FREE config option, but test coverage is always appreciated.) Ashwini
3507also suggested upgrading ln -f to leave the original target alone if link
3508creation fails, and reported that mv -f and -i weren't implemented (now fixed).</p>
3509
3510<p>New config option: TOYBOX_NORECURSE prevents xexec() from making internal
3511function calls (for nommu systems with a finite stack).</p>
3512
3513<p>The "toybox" multiplexer command no longer adds a trailing space to each
3514line of command names, so things like "./toybox | tr ' \n' '|'" to create
3515a grep pattern snippet are easier to do. (Why you'd want to is your business,
3516but the output is tidier now.)</p>
3517
3518<h3>Infrastructure</h3>
3519
3520<p>Isaac Dunham added Android support to portability.h, including compile
3521probes for functions missing from bionic-libc, and annotated the commands that
3522use those functions. We haven't really tested building against bionic,
3523but in theory it's possible now.</p>
3524
3525<p>Running the test suite now color codes the PASS/SKIP/FAIL notifications
3526if output is to a tty. (And in case you missed it last time, VERBOSE=fail
3527to stop at the first failure is really useful.)</p>
3528
3529<p>In loopfiles_rw() use O_CLOEXEC instead of O_RDONLY to request the loop
3530function close filehandles for us. (Otherwise the callback function must
3531close each supplied filehandle itself.)</p>
3532
3533<p>The printf-style escape parsing ("\n" and friends) got factored out into
3534a new unescape() function.</p>
3535
3536<a name="02-10-2014" /><a href="#02-10-2014"><hr><h2><b>October 2, 2014</b></h2></a>
3537<blockquote><p>"There is an art, it says, or rather, a knack to flying.
3538The knack lies in learning how to throw yourself at the ground and miss...
3539Clearly, it is this second part, the missing, which presents the
3540difficulties."</p><p>- The Hitchhiker's Guide to the Galaxy<p></blockquote>
3541
3542<p><a href=downloads/toybox-0.5.0.tar.bz2>Toybox 0.5.0</a>
3543(<a href=/hg/toybox/shortlog/1512>commit 1512</a>) is out.</p>
3544
3545<h3>New commands</h3>
3546
3547<p>The new commands are find, install, factor, and mount. Promoted commands
3548(cleaned up and moved out of "pending") are lspci, inotifyd, and blockdev.</p>
3549
3550<p>cp now implements -HL and -F to force delete of pending files, cpio now
3551ignores -m and implements -p, ls -C now has utf8 support (using wcwidth
3552instead of strlen), and umount got a number of upgrades involving
3553looking things up in /proc/mounts. Other minor cleanups happend to
3554cut, touch, free, and id.</p>
3555
3556<p>In pending: Bradley Controy submitted mix (adjusts OSS sound volume). Ashwini
3557Sharma submitted diff, userdel, blockdev, ipcs, and crond, upgraded
3558fdisk, fsck, and ftpget, and ran a static analyzer on a lot of other code.
3559Partial cleanup was done to useradd, userdel, groupadd, and groupdel.</p>
3560
3561<h3>Build infrastructure</h3>
3562
3563<p><b>Parallel builds</b></p>
3564
3565<p>The build now takes advantage of SMP, autodetecting the number of
3566processors. (Export the environment variable CPUS to pick a specific number.)
3567Other build changes: split out $LDOPTIMIZE because old compilers complain
3568about linker options passed with -c, and the entire "generated" directory now
3569gets deleted by clean (the README that was in there got merged into code.html).</p>
3570
3571<p><b>Standalone builds</b></p>
3572
3573<p>The standalone build infrastructure (scripts/single.sh) got upgraded to
3574build more commands as standalone executables. In make.sh the source file
3575selection uses a regex to find the source files with the NEWTOY/OLDTOY macro
3576for the command. It enables each command's
3577sub-options (so CP has CP_MORE), enables I18N and FLOAT support to build
3578full-featured commands, and includes --help text (at least when
3579the command doesn't use another command's help). The OLDTOY() macro
3580now produces (redundant) function prototypes so you can build an OLDTOY
3581without the NEWTOY</p>
3582
3583<p>It doesn't quite have complete coverage yet, the defconfig entries that
3584aren't building standalone yet are:</p>
3585
3586<blockquote><p>chown, egrep, fgrep, fstype, halt, mv, nc, poweroff, unix2dos,
3587whoami</p></blockquote>
3588
3589<p>The main reason for standalone build failures is NEWTOY() or OLDTOY()
3590entries that don't have their own config symbol. Another problem is entries
3591that depend on another entry in kconfig, usually because common infrastructure
3592is using one command's flags (which the other commands copy): if that command
3593is disabled, the FLAG macros become 0 so dead code elimination can remove the
3594code. It's <a href=http://landley.net/hg/toybox/rev/1503>possible
3595to untangle</a> this, but a bit awkward. (It boils down to conflicting
3596design goals in the two contexts.)</p>
3597
3598<p>Standalone builds are used by the test suite when testing individual
3599commands.
3600
3601<p><b>Snapshot builds</b></p>
3602
3603<p>A new addition to the "generated" directory is generated/build.sh
3604containing a single compiler command line to build toybox in its current
3605configuration. Combined with the generated/*.{h,sh} files from an
3606exisiting build, this may let you build on a new system that hasn't quite
3607got enough OS bits working to run a full configureand make.</p>
3608
3609<h3>Internals</h3>
3610
3611<p>Library code: xcreate/xopen now O_CLOEXEC by default to avoid leaking
3612filehandles to child processes. DIRTREE_COMEAGAIN's second callback is now
3613done with the directory filehandle still open (new dir->again variable added
3614to distinguish first from second callback, and requesting DIRTREE_RECURSE now
3615requires passing in the specific macro value, not just a true/false).
3616Use daemon() out of libc instead of hand-rolled daemonize() in various
3617pending commands. string_to_mode() now passes through type bits so you can
3618use it to more easily modify a file's existing mode.
3619Split xpopen() into xpopen_both(), xopen(), and xrun() depending on whether
3620we want to redirect both, one, or neither of stdin/stdout.</p>
3621
3622<p>Bugfixes: Better error message when TOYBOX_SUID option can't drop priviliges
3623(which happens when you suid something _other_ than root).
3624The old pending version of nbd_client.c wasn't deleted when the
3625command was promoted (and the build would break if both were enabled),
3626toy_exec() sometimes needs to re-exec from $PATH rather than recurse
3627internally (to gain dropped root permissions or limit stack depth),
3628always call setlocale() when I18N is enabled to switch it back _off_ when
3629we run commands that expect sscanf("%n") to return bytes,
3630dirtree() had a memory leak in an error path, patch.c had some bugs in
3631error paths (didn't report problem clearly). Ashwini Sharma spotted an
3632option parsing bug where [-abc] would forget _all_ command line arguments
3633saved in the GLOBALS() block (not just the ones for options being switched
3634off), plus various minor fixes to nbd_client and cpio.
3635Lukasz Szpakowski fixed rm -f on a broken symlink (failed), and killall
3636with no arguments (segfaulted).</p>
3637
3638<p><b>Portability</b></p>
3639
3640<p>A somewhat fiddly fix to rm -rf (which needs to chmod directories to u+rwx
3641to descend into them) which hit a musl bug in faccessat() which the musl
3642maintainer refuses to fix. (He literally wants the man page changed
3643instead, despite other libcs working.) Added an #ifdef __MUSL__ section
3644to portability.h with a workaround, you may need CFLAGS=-D__MUSL__ in your
3645build if your musl build's features.h doesn't #define that. (I may do
3646a different workaround in future, but sometimes you've just got to make
3647it work so you can ship. Also, toybox grep with multiple patterns
3648requires <a href=http://landley.net/hg/aboriginal/rev/1692>a patch
3649to musl's regex engine</a>, which applies to 1.1.4 but not to the current
3650musl source control.)</p>
3651
3652<p>More portability.h fixes for uClibc too. (I don't expect that to ever have
3653another release, so locally patching around posix-2008 violations is silly).</p>
3654
3655<p><b>Change to username filtering</b></p>
3656
3657<p>Posix recommends the username creation logic filter usernames to a small
3658allowed set of characters (which even Red Hat breaks by explicitly allowing
3659"$" at the end), but this prevents UTF-8 usernames. Posix' stated logic
3660is to allow filesystems to create the user's home directory, but Linux
3661filesystems can accept any character but NUL and "/". The only characters
3662we actually _need_ to filter out are ":" (field separator in passwd),
3663newline (line separator in passwd), and "/" (directory separator in
3664filesystem).</p>
3665
3666<h3>Documentation</h3>
3667
3668<p>Web pages updated: cleanup.html documents more cleanup, code.html
3669documents more code, and about.html now capitalizes "toybox" consistently
3670(it's just a word, capitalize at start of sentence).</p>
3671
3672<p>The pending/README file now lists commands that needed review/cleanup
3673before the pending directory was added.</p>
3674
3675<h3>Test Suite</h3>
3676
3677<p>Moved out of scripts/test into top level "tests" directory, and the
3678testing.sh script is now in scripts rather than mixed into the *.test files.</p>
3679
3680<p>Johan Bergström requested VERBOSE=fail to make tests (telling it to
3681stop at the first failure), and spotted a build bug where using gnu
3682sort on the host broke in non-C locales.</p>
3683
3684<p>Divya Kothari submitted tests for chmod, link, tar, bzcat, xzcat, zcat,
3685and hostname. (And more, but that's all that's merged so far.)</p>
3686
3687<a name="07-07-2014" /><a href="#07-07-2014"><hr><h2><b>July 7, 2014</b></h2></a>
3688<blockquote><p>"This planet has - or rather had - a problem, which was this:
3689most of the people living on it were unhappy for pretty much of the time. Many
3690solutions were suggested for this problem, but most of these were largely
3691concerned with the movement of small green pieces of paper, which was odd
3692because on the whole it wasn't the small green pieces of paper that were
3693unhappy."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3694
3695<p><a href=downloads/toybox-0.4.9.tar.bz2>Toybox 0.4.9</a> (<a href=/hg/toybox/shortlog/1385>commit 1385</a>) is out.</p>
3696
3697<p><b>New commands</b> added to pending include:
3698lsattr, chattr, inotifyd, rfkill, sulogin, strings, makedevs,
3699killall5, and tar from Ashwini Sharma, arp from Kyungwan Han,
3700sysctl by Bilal Qureshi, partprobe from Bertold Van den Bergh,
3701host from Rich felker, and I did nbd-client and the first 2/3 of mount.</p>
3702
3703<p>Finished cleanups (commands promoted out of pending):
3704sysctl, rfkill, strings, mkpasswd, makedevs, partprobe, killall5,
3705fallocate, and nbd-client.</p>
3706
3707<p>(Along the way partial cleanups got made to: last, fold, lspci, ps,
3708bootchartd, init, fsck, telnetd, telnet, vconfig, toysh, iconv, useradd,
3709login, host, openvt, deallocvt, getty, tftpd, and modprobe. But there's
3710still more to do on all of those.)</p>
3711
3712<p>This time around the <a href=bin>static binaries</a> are linked against
3713musl instead of uClibc. (That's why there's no sparc version, musl doesn't
3714support that target yet.)</p>
3715
3716<p><b>Documentation:</b></p>
3717
3718<p>The help text parser expects lower case "usage:" lines with
3719a blank line after them, so go through and regularize those. Expand the
3720"coding style" section in the docs and move it to design.html. (Not a show
3721stopper for incoming
3722contributions, just an explanation of some of the things I'll do to them
3723during cleanup.) The help text for the "toybox" command now includes
3724the shell script snippet to install symlinks to the toybox binary.</p>
3725
3726<p>The <a href=cleanup.html>cleanup page</a> now has descriptions for the
3727full ifconfig cleanup series, among others.</p>
3728
3729<p>The new toys/examples directory contains hello.c and skeleton.c. The first is
3730a simple hello world program in toybox style, the second is a much more
3731elaborate example program using showing how to use the command line option
3732parsing and how to provide multiple commands in the same C file.</p>
3733
3734<p><b>Fixes</b>:</p>
3735
3736<p>Fix od bug reported by Samuel Holland ("od -v -b" was appending the default
3737output type even though an output type was specified). Ashwini Sharma reported
3738bugs where readfile() was incorrectly freeing its buffer, and where toy_init()
3739was zeroing the wrong data because the field it was using to measure (rebound)
3740had moved (when I moved it back I added a comment why the field needs to be
3741there), fixed a segfault in the dhcp client, and made a 0 length read at
3742the start of password entry count as EOF. Make the "we are not root" test
3743in the init code show the help text. Posix implies that fflush() can return
3744success even when the stream's error bit is set, so call both fflush() and
3745ferror() from xprintf().</p>
3746
3747<p>Isaac Dunham pointed out that bloatcheck couldn't deal with diff
3748implementations that only implement "unified diff" format, and that some
3749diff implementations can't handle nonseekable input (I.E. reading from
3750a pipe). Bugfix so "help -a" works again. Option parsing on nohup now stops
3751at first nonoption argument. Fix segfault in "which" if PATH wasn't set,
3752which was actually a bug in lib function find_in_path(). Made rm -rf of
3753chmod 000 directories actually remove them.</p>
3754
3755<p>The build now passes the same $CFLAGS to the library probe as the final
3756build, because arch linux is so broken it provides different sets of
3757libraries for static and dynamic linking.</p>
3758
3759<p>It turns out sprintf("%.123s", str) is counting characters, not bytes,
3760so globally enabling locale support opens stack smashing vulnerabilities.
3761So there's a new TOYFLAGS_LOCALE you set in toyflags when you want the
3762setup code to setlocale().</p>
3763
3764<p><b>Upgrades:</b></p>
3765
3766<p>Isaac Dunham extended cpio to archive unreadable empty files, and I taught it
3767to set uid/gid and timestamp when extracting archives. Isaac also
3768added tests for cpio, link, and du, added lspci -i, made the pci database
3769parsing skip # comment lines, merged logname and whoami into id.</p>
3770
3771<p>Daniel Verkamp sped up md5sum about 30% with some loop unrolling, making
3772it actually smaller in the process. I added -b flags to md5sum and sha1sum
3773for "brief" output that's just the hash with no filename. (I'm aware other
3774implementations use that for MSDOS "binary" mode, and don't care.)</p>
3775
3776<p>When building standalone commands (scripts/singleconfig.sh commandname),
3777the build now switches on all the sub-options of the command so we get
3778a standalone version with all the bells and whistles enabled.</p>
3779
3780<p>Add -ds flags to date and document +FORMAT escapes. Add the shell NOP
3781command ":" as an alias for true (for toysh).</p>
3782
3783<p>Add uClibc probe for iconv() and fallocate. (The fact it didn't always
3784build against uClibc is why fallocate wasn't enabled in defconfig before.)</p>
3785
3786<p>The umount command now does an losetup -d on the device by default, so
3787we don't leak loopback devices. Bugfix to losetup so "losetup /dev/loop0
3788filename" actually works again.</p>
3789
3790<p>Divya Kothari sent in test suite entries for ls, ln, rm, mv, printf, dd,
3791and renice. Then a second round for lsattr/chattr, mount, chmod, pgrep/pkill,
3792groupadd, groupdel, and useradd. Several of these uncovered bugs, still
3793working to fix them.</p>
3794
3795<p>There are now free() functions for the predefined llist types and a
3796dlist_terminate() function to break doubly linked lists. The new
3797generic_signal() handler either sets "toys.signal" or writes a byte
3798to toys.signalfd with the signal number if signalfd isn't -1 (which it's
3799initialized to in toy_init).</p>
3800
3801<p>The option parsing logic can now detect when a double fits in a long and
3802use the more precise type for floating point arguments (the FLOAT macro
3803contains the type used). The human_readable() function now just outputs
3804decimal kilo/mega/gigabytes (so when du -u says 5.0G it means 5.0 billion
3805bytes). The build infrastructure now notices duplicate commands (so if you
3806cp toys/pending/command.c toys/other/command.c and forget to delete the
3807first one, the build break is now more informative).</p>
3808
3809<a name="20-04-2014" /><a href="#20-04-2014"><hr><h2><b>April 20, 2014</b></h2></a>
3810<blockquote><p>And to this end they built themselves a stupendous supercomputer
3811which was so amazingly intelligent that even before the data banks
3812had been connected up it had started from "I think therefore I am" and got as
3813far as the existence of rice pudding and income tax before anyone managed to
3814turn it off.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3815
3816<p><a href=downloads/toybox-0.4.8.tar.bz2>Toybox 0.4.8</a> is based on
3817<a href=http://landley.net/hg/toybox/shortlog/1262>commit 1262</a>. And
3818about time too.</p>
3819
3820<p>The big news is that the build no longer needs python to generate help.h,
3821that's now done in C. The help text generation is also collating help text
3822from multiple options, merging command line option blocks and usage: lines.
3823There's even a new <a href=help.html>help web page</a>.</p>
3824
3825<p><b>New commands:</b> Ifconfig, cpio, and su were cleaned up the rest of the
3826way and promoted out of pending. That saga is mostly explained on the
3827<a href=cleanup.html>cleanup page</a>. Vivek Bhagat's freeramdisk,
3828Isaac Dunham's fsfreeze, and Felix Janda's iconv are also new.</p>
3829
3830<p><b>In pending:</b>
3831Ashwini Sharma's team submitted tcpsvd, udpsvd, telnet, telnetd, last, more,
3832groupdel/delgroup, arping, brctl, ftpget, ftpput, printf, reset, and added
3833ipv6 support to traceroute. Kyungwan Han's team submitted modprobe and getty.
3834Vivek Bhagat submitted openvt and deacllocvt. Samuel Holland submitted fold.
3835I wrote a new inflate (zip/zlib/gzip decompression) implementation in
3836compress.c, and still  need to do a corresponding deflate (compression-side)
3837and plug them into gzip and zip and so on. (Right now it does zcat.)</p>
3838
3839<p>Several commands (vmstat, login, du, vconfig, mountpoint, free, chroot,
3840cut, touch, modinfo, expand) predate the "pending" directory, and are thus
3841in other directories but still need cleanup. Of these, vmstat got some
3842work this time (which would be much easier other vmstat implementations
3843documented what their output actually meant).</p>
3844
3845<p><b>Upgrades:</b> Ifconfig grew /prefix netmask support (ala 1.2.3.4/24). Grep now has -zZ to
3846handle null terminated data, cksum grew -H for hex output. Upgraded od so the
3847fields align better when producing multiple output types. Help has -a and -h
3848options (all commands, html output).
3849Bugfix to blkid building for a 32 bit target. The date command can actually
3850set dates now. The O_NOFOLLOW compile time probe didn't work with cross
3851compiling, so it's back to an #ifdef test in portability.h. Nathan McSween
3852sent in a bugfix to od and a portability fix in the common library code.
3853Ashwini Sharma spotted a bug in pidof -o, and added verbose (-v) options
3854to mkdir and ln, and suggested killall should have an -s option and
3855allow -l to take zero arguments. Ashwini Sharma and Felix Janda upgraded
3856tftpd.  Fixed dumpleases still using toynet.h after
3857that was removed. Corrected killall return code and error reporting.
3858Isacc Dunham fixed bugs all over the tree, did cleanup on a bunch of
3859pending commands (getty, ftpget, init, openvt, modprobe...), and clarified
3860find's help text. Tom Sparrow ran three different static analyzers on
3861the code, which resulted in a few cleanups. The peek()/poke() functions
3862now use "volatile" to prevent broken compiler "optimizations" to do with
3863aliasing.</p>
3864
3865<p><b>Build stuff:</b> Each FOR_xxxx macro now has a complementary CLEANUP_xxxx macro, so you
3866can put multiple commands with different command line options in the same
3867.c file, so they can share infrastructure outside of lib. (This let the
3868bunzip logic move out of lib into bzcat.c.) See XXX for example.
3869i
3870<p>The headers #included in toys.h are now grouped by standard, and headers
3871not listed in Posix or LSB were moved to portability.h. The old xregcomp.h
3872was folded into lib.h because it's posix (and supporting oddball uClibc
3873configurations isn't as important as it once was).</p>
3874
3875<p>Regression tested against Ubuntu 8.04 to fix up bit-rot in defconfig
3876build on older systems. (We depend on Posix-2008, but not necessarily
3877the absolute latest build environment.)</p>
3878
3879<p><b>In lib</b>: lib/xwrap.c added xgetpwnam(), xchroot(), and lib/lib.c now has names_to_pid().
3880xsetuid() was replaced with xsetuser() which takes a struct passwd
3881and sets both gid and uid, mkpathat() got factored out into a library command,
3882get_int_value() became atolx_range(), and
3883xmsprintf() is now just xmprintf(). The bunzip2 logic moved from lib into
3884bzcat.c.</p>
3885
3886<p><b>Documentation</b>: new <a href=help.html>help page</a> with the
3887help text for all the defconfig commands, using the new help -ah output.
3888The <a href=code.html>source code walkthrough</a> now says more about
3889#including header files, and how the generated/* directory works. The
3890<a href=design.html>design page</a> has some new paragraphs about trading
3891of different kinds of simplicity, and why comments aren't a substitute for
3892good code. The README no longer trails off into obvious unfinished confusion
3893at the end. Each page on the website should now have its own title.</p>
3894
3895<a name="18-11-2013" /><a href="#18-11-2013"><hr><h2><b>November 18, 2013</b></h2></a>
3896<blockquote><p>"Space," it says, "is big. Really big. You just won't believe how vastly, hugely, mindbogglingly big it is. I mean, you may think it's a long way down the street to the chemist's, but that's just peanuts to space."</p><p>-
3897The Hitchhiker's Guide to the Galaxy</p></blockquote>
3898
3899<p><a href=downloads/toybox-0.4.7.tar.bz2>Toybox 0.4.7</a> is based on
3900<a href=http://landley.net/hg/toybox/shortlog/1122>commit 1122</a>.</p>
3901
3902<p>New commands: Brad Conroy submitted blkid. Elie De Brauwer submitted
3903reboot, halt, and poweroff. Strake's nl got cleaned up and promoted from
3904pending to posix. In addition, the existing chvt and vconfig got some
3905cleanup.</p>
3906
3907<p>That said, I haven't nearly kept up with the flood of new commands going
3908into pending: Ashwini Sharma's team submitted
3909dd, dumpleases, traceroute, top, useradd, groupadd, mkpasswd, tftpd, and
3910an fsck wrapper (with no filesystem drivers yet). Isaac Dunham sent in cpio.</p>
3911
3912<p>Bugfixes: Jeroen van Rijn added a user count to uptime. Elie De Brauwer
3913added -e to watch, removed a memory leak, and fixed a terminal size problem.
3914William Haddon made xargs call its command line once even with blank input
3915(the standard is vague, but builds expect it), and fixed an off by one bug
3916where grep didn't malloc enough space with -E (leading to a segfault).
3917I fixed a glitch in bunzip2 (same one as went into busybox since they're using
3918the code I wrote), in od to fix -t co, -J, and -c options. Add uname -o as a
3919synonym for -s. Build fix to never use $CC without prefixing it with
3920$CROSS_COMPILE (since $HOSTCC could be different). Anca Emanuel spotted
3921a typo in the web page.</p>
3922
3923<p>The compile-time command line option parsing got rewritten (ported from
3924bash to C), which should speed up builds a bit and allow code controlled by
3925--longopts to drop out properly when disabled in the configuration. Terminal
3926querying got refactored. Patch's -x option is now more informative (a
3927debug thing if you're trying to figure out why a patch didn't apply).
3928The "toynet.h" file got folded into toys.h since musl supports it and
3929micromanging uClibc options isn't very interesting anymore. The test suite
3930now uses scripts/single.sh when testing a single command.</p>
3931
3932<a name="17-09-2013" /><a href="#17-09-2013"><hr><h2><b>September 17, 2013</b></h2></a>
3933<blockquote><p>"Think of a number," said the computer, "any number."
3934Arthur told the computer the telephone number of King's Cross railway
3935station passenger inquiries, on the grounds that it must have some function,
3936and this might turn out to be it.</p><p>- The Hitchhiker's Guide to the Galaxy</p>
3937</blockquote>
3938
3939<p><a href=downloads/toybox-0.4.6.tar.bz2>Toybox 0.4.6</a> is based on
3940<a href=http://landley.net/hg/toybox/shortlog/1068>commit 1068</a>.</p>
3941
3942<p>This release adds
3943several new commands: Felix Janda wrote paste and fallocate, Kyungwan Han
3944submitted eject, Strake contributed grep, Ashwini Sharma added pmap (and
3945a testsuite entry for grep), Lukasz Skalski sent pwdx, Isaac Dunham posted
3946acpi, and I did timeout and umount.</p>
3947
3948<p>The ls command now has a --color=auto option (suggested by Rich Felker).
3949The multiplexer now has a --help option so you can say "./toybox --help blah"
3950instead of using the built-in "help" command. (Which is a shell built-in.
3951Try it on your command line, it's like man for shell builtins. But a certain
3952other project has conditioned people to expect --help, so...) I forget who
3953heehooman at gmail is but they pointed out unshare needed PID and UID
3954namespace support.</p>
3955
3956<h3>Pending</h3>
3957
3958<p>A lot of new commands in toys/pending, to the point the next release should
3959probably just focus on cleanup and review of this backlog. We've got klogd,
3960dhcp, dhcpd, watch, route, and ps from
3961Ashwini Sharma (and an fsck wrapper but no fsck.fstype engines yet),
3962syslogd, pgrep, and pkill from Madhur Verma, netstat by Ranjan Kumar,
3963test by Felix Janda, lspci by Isaac Dunham, nl, su, and renice by strake (I.E.
3964M. Farkas-Dyck), and sysvinit by Kyungwan Han.</p>
3965
3966<p>Some cleanup work on existing pending commands that aren't
3967ready to promote yet: I did a few more rounds on ifconfig
3968and Isaac Dunham's did several cleanups to xzcat, Felix Janda cleaned up
3969logger and syslogd...</p>
3970
3971<p>Also some cleanup work on commands that predate the pending directory,
3972but weren't quite polished when they went in, most prominently du,
3973expand, and touch.</p> 
3974
3975<h3>Infrastructure</h3>
3976
3977<p>The new scripts/single.sh builds a standalone command without the
3978multiplexer, although not all commands can be built that way yet (NEWTOY yes,
3979OLDTOY no) and the space savings aren't anything to write home about. (If a
3980command needs the option parsing logic at all, it needs all of it.) If
3981you're curious, you can do:</p>
3982
3983<blockquote><pre>
3984make defconfig
3985make
3986mkdir singles
3987for i in $(./toybox)
3988do
3989  echo $i
3990  PREFIX=singles/ scripts/single.sh $i || break
3991done
3992</pre>
3993<p>(And then wait a long time and watch almost half the builds fail.)</p>
3994</blockquote>
3995
3996<p>There is now libbuf analogous to toybuf, another global 4k buffer this
3997time for use by lib/ code instead of command code.</p>
3998
3999<p>The lib directory got split up a bit, lib/pending.c contains functions
4000not yet used by anything outside of toys/pending/*, and lib/xwrap.c contains
4001functions that wrap other functions and handle failures (via error_exit).
4002This leaves lib/lib.c containing actual new functions.</p>
4003
4004<p>General improvements and bug fixes to argument parsing. The [-abc] exclude
4005logic should now clear arguments slots when disabling options. Bare --longopts
4006should work now and be able to report errors using their name, the new ;
4007option allows optional arguments to longopts only suppliable with = (I.E.
4008--color and --color=auto but not --color auto).</p>
4009
4010<p>I'm gradually weaning the code off of itoa()/utoa() because sprintf
4011does this already. In this case "simple" probably means "let libc do it
4012for us".</p>
4013
4014<p>Rewrote for_each_pid_with_name_in() and renamed it to just names_to_pid().
4015It shouldn't get confused trying to compare absolute and relative paths quite
4016so much anymore.</p>
4017
4018<p>lib/llist.c grew a new dlist_pop() function for removing a doubly
4019linked list entry while maintaining a circular list; tail and patch are
4020using it now.</p>
4021
4022<p>The musl guys suggested a new optimization flag
4023(-fno-asynchronous-unwind-tables)
4024that shaves about 10% off the binary size by removing a C++ism that crept
4025into gcc's idea of C. While I don't normally try to micromanage the compiler,
4026"-fstop-being-stupid" is a thing you have to hit gcc with from time to time.</p>
4027
4028<h3>Bugfixes</h3>
4029
4030<p>Felix Janda and I did a largeish rewrite of tail to
4031finally make it work right (we think). Still need to implement tail -f
4032someday (the tricky bit is making -f follow multiple files at once).
4033Felix also reported a bug in xpidfile.</p>
4034
4035<p>Juhani Haverinen pointed out that
4036python 3 doesn't work with config2help.py, so the detection logic looks
4037for python2 (until I get around to rewriting that in C). Elie De Brauwer
4038then fixed our first attempt at this, and also fixed uname's help string.</p>
4039
4040<p>Ashwini Sharma
4041pointed out the build was making a FLAG_ macro for " " which broke some
4042configurations. (That's a control character, not a command line option.)</p>
4043
4044<p>Jacek Bukarewicz pointed out a bug in chdir permission handling, and
4045a way to make env segfault. Both should be fixed now.</p> 
4046
4047<p>The new function xexec_optargs()
4048replaces calls to xexec(toys.optargs) to avoid freeing and reusing optargs
4049during option parsing screwing stuff up (such as netcat's exec mode).</p>
4050
4051<p>The stat command's %a output was padded with leading zeroes, which
4052didn't match anybody else's behavior and thus made the test suite hiccup
4053between TEST_HOST and testing toybox. (If you go "TEST_HOST=1 scripts/test.sh
4054command" it sanity checks the tests against the host implementation.)</p>
4055
4056<p>Last release, "mkdir sub/sub && chmod 007 sub/sub && rm -rf sub" didn't
4057delete sub and didn't exit with an error either. Neither was correct, rm
4058should now be fixed.</p>
4059
4060<p>
4061<a name="26-07-2013" /><a href="#26-07-2013"><hr><h2><b>July 26, 2013</b></h2></a>
4062<p>Georgi Chorbadzhiyski maintains a <a href=https://github.com/gfto/toybox>git
4063mirror</a> of the repository on github, automatically updated from the
4064mercurial every 6 hours. The mirror is read only, but you can generate patches
4065against it and post them to the list.</p>
4066
4067<a name="02-07-2013" /><a href="#02-07-2013"><hr><h2><b>July 2, 2013</b></h2></a>
4068<blockquote><p>"Time is an illusion. Lunchtime doubly so." "Very deep. You
4069should send that in to the Reader's Digest. They've got a page for people
4070like you."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
4071
4072<p><a href=downloads/toybox-0.4.5.tar.bz2>Toybox 0.4.5</a> is based on
4073<a href=http://landley.net/hg/toybox/shortlog/941>commit 941</a>. It adds
4074uuencode and uudecode from Erich Plondke, and enables Luis Morales' "who" by
4075default. Felix Janda and I cleaned up last year's "stat" submission and
4076enabled it. Ivo van Poorten added "groups".
4077Andre Renaud added "lsusb". I implemented "split", "pivot_root", and "mv".
4078</p>
4079
4080<p>The "help" command is implemented differently now (lib/help.c) and
4081each command can now understand --help (including both "toybox --help"
4082and "toybox --help command" in the multiplexer).</p>
4083
4084<p>The "pending" directory has several commands (find, xzcat, nbd-client,
4085logger, expr) which work but are not enabled by default pending further cleanup.
4086Ifconfig is enabled, but still in pending because it's only 2/3 cleaned up.
4087(It's an awkward halfway state but I'm not holding up the release for it.)</p>
4088
4089<p>I'm <a href=cleanup.html>documenting the cleanups</a> to teach
4090more people to do it, but the writeups aren't caught up yet. The
4091<a href=roadmap.html>roadmap</a> also got updated a bit with further analysis
4092of other projects, and the README and about pages got updated.</p>
4093
4094<p>Fixed _another_ "ls -C" segfault when terminal size can't be detected,
4095condensed the ls help text to fit on one page, implented --color, and taught
4096-l to print the major, minor numbers when showing block/char devices.
4097Argument parsing now handles "--" properly (to end option checking),
4098and the infrastructure can now handle bare --longopts that have no
4099corresponding short option (both were implemented before but didn't work).
4100Fixed an old bug in "patch", chmod grew -f, who grew -a. Isaac Dunham
4101fixed "-" vs "_" handling in modinfo, added a "firmware" output
4102field, added -b and -k support, and taught it that the ".ko" extension means
4103to look for the file at the specified path instead of under /lib. Felix Janda
4104moved file permission display code to lib so ls and
4105stat could share it. Ashwini Sharma spotted a bug in xabspath when the
4106last path component exists but we haven't got permissions to open it
4107(ala readlink -f /dev/sda as a normal user).
4108</p>
4109
4110<p>In the build infrastructure, scripts/findglobals.sh finds leaked global
4111variables. (Leaked means they aren't part of the global union: Other than glibc
4112debris, toybox should define "this", "toy_list", "toybuf", and "toys", and
4113that's it; the rest add memory footprint to every command for the benefit of
4114just one command; use GLOBALS() to stick 'em in the union.) Static linking
4115against libraries other than the host's libc now applies to feature probes
4116for unshare and such. Neuter stupid internationalization support that makes
4117various host "sort" commands put things in an order other than alphabetical
4118(breaking the multiplexer's binary search on command names).
4119
4120<p>You should now be able to build from a source control snapshot on a build
4121system that hasn't got python: if you disable CONFIG_TOYBOX_HELP. (The
4122release tarballs ship generated/help.h, but it's not in source control.
4123Eventually I should rewrite that python script in C.)</p>
4124</p>
4125
4126<p><b>LICENSE TWEAK</b>: After <a href=http://lists.landley.net/pipermail/toybox-landley.net/2013-March/000794.html>discussion</a> on the mailing list the "2 clause
4127BSD" <a href=license.html>license</a> got slightly simplified so the first
4128paragraph now says:</p>
4129
4130<blockquote><p>Permission to use, copy, modify, and/or distribute this
4131software for any purpose with or without fee is hereby granted.</p></blockquote>
4132
4133<p>It used to continue "provided that the above copyright notice and this
4134permission notice appear in all copies", but A) what's the point? B) does "all
4135copies" mean binaries, or just source code, or what? C) lots of projects
4136that consider BSD and GPL compatible have <a href=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/crypto/aes_generic.c>files with
4137both license notices</a> on them (sometimes at <a href=http://git.busybox.net/busybox/tree/shell/ash.c>opposite ends of the file</a> to make the conflict
4138less obvious) because "all copies must include this function" would violate
4139the GPL but "all copies must include this magic text blob" somehow don't?</p>
4140
4141<p>I don't want to have to care about this anymore. The tweaked version is more
4142or less public domain with a liability disclaimer, but we're still calling it
4143BSD (sometimes "0 clause BSD") to avoid explaining.</p>
4144
4145<a name="21-03-2013" /><a href="#21-03-2013"><hr><h2><b>March 21, 2013</b></h2></a>
4146<p>Video of my ELC talk
4147"<a href=http://youtu.be/SGmtP5Lg_t0>Why is Toybox?</a>"
4148is up on youtube. Related materials include the
4149<a href=http://landley.net/talks/celf-2013.txt>talk outline</a> and an
4150<a href=/aboriginal/about.html#selfhost>android self-hosting writeup</a>.</p>
4151
4152<p>[Updated June 4] The following links jump to specific topics in the video. (Sorry about
4153the ads, it's The Linux Foundation.)</p>
4154
4155<ul>
4156<li>0m29s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=0m29s>The smartphone is replacing the PC</a></li>
4157  <ul>
4158  <li>4m22s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=4m22s>Software needed to become self-hosting</a></li>
4159  <li>6m20s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=6m20s>Do we care if android or iphone wins?</a></li>
4160  </ul>
4161<li>9m45s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=9m45s>Android not vanilla: oppose or accept?</a></li>
4162  <ul>
4163  <li>11m30s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=11m30s>Open source can't do User Interfaces</a></li>
4164  </ul>
4165<li>15m09s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=15m09s>Android is not copyleft: oppose or accept?</a></li>
4166<li>18m23s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=18m23s>Security issues</a></li>
4167<li>21m15s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=21m15s>Solutions to the software problems</a></li>
4168  <ul>
4169  <li>22m55s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=22m55s>What toybox needs to be/do</a></li>
4170  <li>28m17s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m17s>What is toybox?</a></li>
4171    <ul>
4172    <li>28m58s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m58s>Why toybox started...</a></li>
4173    <li>37m50s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=37m50s>What does toybox actually implement?</a></li>
4174    </ul>
4175  </ul>
4176</ul>
4177</span>
4178
4179
4180<a name="14-03-2013" /><a href="#14-03-2013"><hr><h2><b>March 14, 2013</b></h2></a>
4181<blockquote><p>"Ford, you're turning into a penguin. Stop it." -
4182</p><p>The Hitchhiker's Guide to the Galaxy</p></blockquote>
4183
4184<p><a href=downloads/toybox-0.4.4.tar.bz2>Toybox 0.4.4</a> is based on
4185<a href=http://landley.net/hg/toybox/shortlog/813>commit 813</a>, adding
4186the "time" and "readahead" commands, plus some bugfixes.</p>
4187
4188<p>The "cp" command now implements the -s symlink option, plus bugfixes
4189getting various corner cases right as used in actual package builds.
4190"id -Gn root" should now print root's groups
4191instead of the current user's. Several build fixes so toybox builds under
4192Ubuntu 8.04 again (which is about as old a build environment as you
4193can expect to find posix-2008 features in).</p>
4194
4195<p>Unfinished commands have generally been moved to "toys/pending".
4196Everything else should "default y" to participate in make defconfig.
4197Several of those pending commands got some basic cleanup so allyesconfig
4198should at least compile (although defconfig is still what's useful).</p>
4199
4200<p>Significant roadmap updates, checking several other multicall binaries
4201(klibc, sash, sbase, s6...) to see what commands they include.</p>
4202
4203<a name="18-01-2013" /><a href="#18-01-2013"><hr><h2><b>January 18, 2013</b></h2></a>
4204<blockquote><p>This must be Thursday. I never could get the hang of Thursdays.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
4205
4206<p><a href=downloads/toybox-0.4.3.tar.bz2>Toybox 0.4.3</a> is based on
4207<a href=http://landley.net/hg/toybox/shortlog/793>commit 793</a>. There
4208are now exactly 100 commands in defconfig (of a little over 220 on the
4209<a href=roadmap.html>todo list</a>).</p>
4210
4211<p>Elie De Brauwer added the rev command, cleaned up tac, implemented the -s
4212and -f flags for seq, added -v and -i to killall (and fixed killall not to
4213kill itself before finishing its pid list), and added to the test suite.
4214Felix Janda added -m to mkdir, pwd -L and -P, and more test suite entries.</p>
4215
4216<p>Rob Landley added the losetup command, and fixed the existing ls, cp, and
4217readlink commands. The segfault in ls
4218happened when it couldn't determine the screen size (last release changed the
4219default to -C and a screen size of 0 made column view unhappy), and cp got an
4220extensive rewrite bringing it up to date with the dirtree changes and fixing
4221a number of things it never did right in the first place. The xabspath()
4222code in the library now handles a symlink after ".." properly (and the test
4223suite checks for it).</p>
4224
4225<p>Infrastructure-wise the code is better about automatically setting the
4226error return code properly. Now error_msg() sets the exit code to 1 if it's
4227still defaulting to 0, and the global exit path does a fflush(NULL) with error
4228bit check rather than trying to be quite so granular about flushing. (That
4229means if we use printf() instead of xprintf() it still exits with the right
4230error code, it just doesn't end the program early on an output error.)
4231Minor bugfix so TOYBOX_DEBUG
4232doesn't always warn about the lack of suid bit when toybox is built with
4233at least one STAYROOT command. Bugfix for the option [grouping] logic
4234(and then further fixes to the error reporting pointed out by Ashwini Sharma).
4235dirtree_handle_callback() now has a prefix like the rest of the dirtree
4236functions. A lot of stuff doing manual path handling was switched to using
4237libc basename() (including, embarassingly, the basename command), which means
4238it now correctly detects "/trailing/slash/" which the previous code didn't.</p>
4239
4240<p>Also, last release included some accidentally checked in debug code that
4241disabled compiler optimization, so the binary size bloated a bit. It's back
4242to -Os by default now.</p>
4243
4244<a name="15-12-2012" /><a href="#15-12-2012"><hr><h2><b>December 15, 2012</b></h2></a>
4245<blockquote><p>"The major difference between a thing that might go wrong and a
4246thing that cannot possibly go wrong is that when a thing that cannot possibly
4247go wrong goes wrong it usually turns out to be impossible to get at or repair."
4248</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
4249
4250<p><a href=downloads/toybox-0.4.2.tar.bz2>Toybox 0.4.2</a> is based on
4251<a href=http://landley.net/hg/toybox/shortlog/749>commit 749</a> and is
4252just a resync. Linux 3.7 came out, meaning it's time to do an Aboriginal
4253Linux release, and that should use a stable version of toybox. So here's
4254a new stable version.</p>
4255
4256<p>The new commands are cut (from Jason Kyungwan Han), touch
4257(from Choubey Ji), expand (from Jonathan Clairembault, and he fixed a
4258bug in login), and rm (from Rob Landley). Felix Janda added UTF-8
4259support infrastructure (for non-ascii character sets) with a config option.
4260Elie De Brauwer added tests for cat and sha1sum, and -so options to pidof.
4261The "ls" command defaults to -C (column view) now, and "readlink" now supports
4262-fenq.</p>
4263
4264<p>Portability work: toybox should now build against the musl C library,
4265and against older glibc versions (circa 2008, much before that and kernel
4266features we depend on start to drop out).</p>
4267
4268<p>The whole codebase got reindented from "one tab" to "two spaces" per
4269level. The option parsing logic now understands [groups] of commands (when more
4270than one in a group is selected it can switch the others off, or error out,
4271or other things). The error_exit() infrastructure can now longjmp back to an
4272earlier point instead of exiting. Each toys/* directory now has a README,
4273the first line of which is the fancy name menuconfig uses for the directory
4274(so no more hardwired directory list in scripts/genconfig.sh).</p>
4275
4276<p>Fixed a filehandle leak in getmountlist().
4277Pass parent pointer to dirtree_add_node() so it can give error messages with
4278full path. The yesno() function now always reads from stdin and writes to
4279stderr (we can retry tty checking complexity once we've got commands needing
4280it).</p>
4281
4282<p>The open group broke their website so the
4283<a href=http://opengroup.org/onlinepubs/9699919799>old links</a> to POSIX 2008
4284now <a href=http://pubs.opengroup.org/onlinepubs/9699919799>need to start with
4285pubs</a>. Some of the links in the tree have been updated, others haven't while
4286I wait to see if their webmaster notices and fixes it.</p>
4287
4288<p>(I note that the current rm implementation is not technically posix compliant
4289because the standard requires infinite recursion depth and the current
4290implementation uses one filehandle per level. I can add a config option
4291to do it Posix's way, which is more brittle and needs extra security checks,
4292but am waiting for somebody to complain first. The default "ulimit -n" is 1024
4293filehandles, so drilling down over 1000 nested subdirectories).</p>
4294
4295<a name="13-11-2012" /><a href="#13-11-2012"><hr><h2><b>November 13, 2012</b></h2></a>
4296<blockquote><p>"Rule Six: The winning team shall be the first team that wins."
4297</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
4298
4299<p><a href=downloads/toybox-0.4.1.tar.bz2>Toybox 0.4.1</a> is based on
4300<a href=http://landley.net/hg/toybox/shortlog/691>commit 691</a>.</p>
4301
4302<p>Elie De Brauwer contributed usleep, Ashwini Kumar contributed du, and
4303Kyungwan Han contributed vconfig. Other new commands include switch_root and
4304md5sum, and the remaining shell wrappers are now proper commands (dos2unix,
4305unix2dos).</p>
4306
4307<p>The patch command now supports -l, and gethostname is now enabled by
4308default. The df command follows symlinks to get the actual device name.
4309Felix Janda added -m support to wc (for utf8).</p>
4310
4311<p>On the infrastructure side, the commands have now been grouped into
4312"posix", "lsb", and "other" subdirectories (for things required by Posix-2008,
4313the Linux Standard Base 4.1, and commands in neither). This affects menuconfig
4314and the actual source layout (toys/cp.c is now toys/posix/cp.c, and so on).
4315An android directory is planned (see the updated
4316<a href=roadmap.html#android>android roadmap analysis</a>).</p>
4317
4318<p>The FLAG_ macros for command option parsing and TT alias for the command's
4319global block are now automatically generated, commands should
4320#define FOR_commandname before #including <toys.h> to get the macros for that
4321command.</p>
4322
4323<p>An upgrade to the build infrastructure now allows commands with _ and -
4324in them, such as switch_root.</p>
4325
4326<p>Bugfixes: Avery Pennarun spotted a case where ls showed uid twice instead of
4327uid and gid, and that nice was using the wrong range of numbers.
4328The ls command also recursed inappropriately last time (not quite
4329properly converted for the dirtree changes last release), and now it's
4330fixed. Roy Tam pointed out a glitch in sh, and fixed df's percentage
4331calculation to match the POSIX spec. The kernel build didn't like our mktemp
4332and it does now. The wc command wasn't quite posix compliant (trailing spaces
4333break stuff). The ls command recursed inappropriately last time (not quite
4334properly converted for the dirtree changes last release), and now it's
4335fixed. The catv command wasn't displaying byte 255 correctly. Some lib
4336fixes (thinko in xpidfile). Fixed uname -m when running a 32 bit x86 binary
4337on an x86-64 host (it lies and says the system is i686, i586, or i486 depending
4338on what the toolchain that built the binary supported. This makes builds in
4339a 32 bit chroot on a 64 bit kernel break less.) The df command was checking
4340partitions in the wrong order (displaying undermounts instead of overmounts:
4341this used to work but some library code changed out from under it and it
4342wasn't updated to match until now). Felix Janda filled out the test suite
4343some more. The patch file creation logic got tweaked several times to
4344successfully apply more patches. Support for older (pre 2.10) glibc
4345versions was added to portability.h.</p>
4346
4347<p>Miscelaneous cleanups all around (mknod, sha1sum, logname), including a
4348rewrite of taskset to be less dependent on libc getting the headers right. All
4349the command headers should now point to the current relevant standards
4350document, where applicable.</p>
4351
4352<p>This news page had old news entries from before the relaunch moved into
4353a separate <a href=oldnews.html>oldnews</a> page.</p>
4354
4355<p>I forgot to create <a href=bin>static binaries</a> last time, but they're
4356back now.</p>
4357</span>
4358
4359<a name="23-07-2012" /><a href="#23-07-2012"><hr><h2><b>July 23, 2012</b></h2></a>
4360<blockquote><p>"Ford", Arthur said. "There's an infinite number of monkeys
4361out here who want to talk to us about this script for Hamlet they've worked
4362out."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
4363
4364<p><a href=downloads/toybox-0.4.0.tar.bz2>Toybox 0.4.0</a> is based on
4365<a href=http://landley.net/hg/toybox/shortlog/640>commit 640</a>.</p>
4366
4367<p>The new <a href=status.html>status page</a> is calculated from
4368the roadmap info, and should be easier to keep up to date in future.</p>
4369
4370<p>Andre Renaud contributed od and modinfo. Elie De Brauwer contributed
4371taskset, bugfixes to cmp and tail, and tests for sort and tail. Kyungwan Han
4372contributed passwd. Gaurang Shastri contributed w. Ashwini Sharma spotted a
4373case where dirtree was adding extra slashes to a path.</p>
4374
4375<p>I rewrote od, cleaned up comm, documented the
4376<a href=code.html#lib_llist>llist</a> and
4377<a href=code.html#lib_dirtree>dirtree</a> infrastructure, added an -r option
4378to date (and fixed a bug where -u wouldn't override /etc/localtime),
4379fixed bugs in chmod +stw, fixed ls to show suid bits properly when the
4380corresponding executable bit wasn't set, and worked around a longstanding
4381glibc bug where static linking prevents stdout from automatically flushing
4382pending output on exit.</p>
4383
4384<a name="25-06-2012" /><a href="#25-06-2012"><hr><h2><b>June 25, 2012</b></h2></a>
4385<blockquote><p>"For a moment, nothing happened. Then, after a second or so, nothing continued to happen."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
4386
4387<p><a href=downloads/toybox-0.3.1.tar.bz2>Toybox 0.3.1</a> is based on commit
4388<a href=http://landley.net/hg/toybox/shortlog/607>commit 607</a>. It's
4389mostly a bugfix release for ls -l (which was unhappy on targets other than
4390x86-64), plus a new "date" from Andre Renaud and rewritten chgrp/chown which
4391now support the full set of posix flags, plus a little work on the test
4392suite and some more header tweaks towards eventual compatability with the
4393musl libc.</p>
4394
4395<p>The todo list runneth over, but "release early, release often", so here
4396it is. The roadmap and documentation are a bit behind, and I've got ~40
4397pending submissions to review. I need to catch up...</p>
4398</span>
4399
4400<a name="12-06-2012" /><a href="#12-06-2012"><hr><h2><b>June 12, 2012</b></h2></a>
4401<blockquote><p>"For instance, on the planet Earth, man had always assumed that
4402he was more intelligent than dolphins because he had achieved so much - the
4403wheel, New York, wars and so on - whilst all the dolphins had ever done was
4404muck about in the water having a good time. But conversely, the dolphins had
4405always believed that they were far more intelligent than man - for precisely
4406the same reasons."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
4407
4408<p>It's well past time for <a href=downloads/toybox-0.3.0.tar.bz2>toybox 0.3.0</a>,
4409so here it is, based
4410on <a href=http://landley.net/hg/toybox/shortlog/595>commit 595</a>, and the
4411statically linked <a href=downloads/binaries>prebuilt binaries</a> should
4412actually be statically linked this time (thanks Ashwini Sharma for spotting
4413that).</p>
4414
4415<p>It's hard to figure out where to cut a release, because development
4416doesn't stop. "Long before now" is the obviuos answer, of course.
4417The project's maintainer also moved house during this development cycle, which
4418threw things off for a bit (so many boxes). Releases should hopefully be a bit
4419more frequent from here on.</p>
4420
4421<p>The big things Rob worked on this time were the new dirtree (directory
4422tree traversal) infrastructure, and a complete rewrite of ls using that
4423which should now implement all 26 posix options.</p>
4424
4425<p>Georgi Chorbadzhiyski added printenv, whoami, mkdir, mkfifo, chmod, chown,
4426chgrp, and uniq. He also added fraction and extension support to sleep (so if
4427you need a quarter-second sleep, it can do that now), and fixed a build bug
4428on slackware.</p>
4429
4430<p>Daniel Walter contributed a string to mode_t parser (in use by chmod and
4431mkdir -m).  Ilya Kuzmich contributed comm. Elie De Brauwer added mountpoint,
4432vmstat, logname, login, and mktemp. Kevin Chase did some portability cleanups.
4433Pere Orga fixed some documentation.</p>
4434
4435<p>The "tac" and "clear" commands are now normal commands instead of shell
4436wrappers, and the header #includes have been cleaned up a bit to remove
4437deprecated functions and attempt to increase compatability with the bionic and
4438musl C libraries, "tail" should now use lseek() for large files, and "id" got
4439some cleanups and bugfixes.</p>
4440
4441<p>The new TOYBOX_FLOAT configuration option selects whether or not
4442to include floating point support (for embedded targets where that's
4443problematic).</p>
4444
4445<p>Several random bugfixes: unshare() might actually build portably now,
4446yes 'n' | cp -i should no longer bypass stdin and prompt via the tty, the
4447SUID support no longer drops permissions going through the toybox
4448multiplexer command, and a bugfix to xargs -0 means it should no longer
4449segfault. (I have a pending bug report about xargs not doing the full
4450posix whitespace handling that -0 obsoleted, but I'll deal with that next
4451release.)</p>
4452
4453<p>The build infrastructure is now automatically generating FLAG_ macros
4454for the options, but currently with the wrong names. Some more macro glue
4455is necessary, which I haven't quite figured out how to do yet.</p>
4456
4457<p>A defconfig toybox at the start of the $PATH has successfully built
4458Linux From Scratch (in my Aboriginal Linux project). The commands that
4459'default n' in the config are often still broken, cleanup is ongoing.
4460(The new dirtree stuff broke several of them that haven't been converted
4461yet, but if I wait until everything works we won't have a release before
44621.0, so here's a checkpoint.)</p>
4463
4464
4465<a name="03-03-2012" /><a href="#03-03-2012"><hr><h2><b>March 3, 2012</b></h2></a>
4466
4467<blockquote><p>"They went unnoticed at Goonhilly, passed over Cape Canaveral
4468without a blip, and Woomera and Jodrell Bank looked straight through them.
4469Which was a pity, because it was exactly the sort of thing they'd been looking
4470for all these years."</p><p>- The Hitchhiker's Guide to the Galaxy</p>
4471</p></blockquote>
4472
4473<p>Here's <a href=downloads/toybox-0.2.1.tar.bz2>toybox 0.2.1</a> based
4474on <a href=http://landley.net/hg/toybox/shortlog/512>commit 512</a>.  This
4475time around, there are statically linked <a href=downloads/binaries>prebuilt
4476binaries</a> for various embedded targets.</p>
4477
4478<p>It's been a busy few weeks, almost entirely due to new contributors. (I
4479have not quite been keeping up.)</p>
4480
4481<p>Elie De Brauwer contributed free, uptime, swapon, swapoff, lsmod, mknod,
4482insmod, rmmod, and fixed a bug in basename.  Andre Renaud contributed ls, ln,
4483realpath, and hostname. Andres Heck contributed pidof and killall.  Daniel
4484Walter wrote kill and extended id. Timothy Elliott contributed tail and tests
4485for cmp. Frank Bergmann sent a warning fix. Bryce Fricke added -i to cp.
4486Nathan McSween pointed out an optimization. Georgi Chorbadzhiyski fixed
4487cross compiling to work more reliably.</p>
4488
4489<p>(My own contribution this time around was just tightening up other people's
4490code, a build fix to unshare, some random bugfixes, and so on. My only new
4491code this time around was writing a bash replacement for the existing python
4492bloat-o-meter.)</p>
4493
4494<p>Last time (the 0.2.0 release) included the first pass at an id command from
4495Tim Bird, env and basename from Tryn Mirell, cmp and head from Timothy Elliott,
4496more bugfixes from Nathan McSween and Elie De Brauwer, and Luis Felipe Strano
4497Moraes did a first pass at the who command plus other bugfixes and
4498optimizations.</p>
4499
4500<p>(For that release I did xargs, cal, truncate, unlink, nohup, tty, wc, link,
4501dirname, unshare, and various infrastructure tweaks, but it took me 3 months
4502and those guys did their stuff in a week or so.)</p>
4503
4504
4505<a name="12-02-2012" /><a href="#12-02-2012"><hr><h2><b>February 12, 2012</b></h2></a>
4506<blockquote><p>
4507"for though it has many omissions and contains much that is apocryphal, or at
4508least wildly inaccurate, it scores over the older, more pedestrian work in two
4509important respects..."</p>
4510<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
4511
4512<p>Here's the first BSD licensed release,
4513<a href=downloads/toybox-0.2.0.tar.bz2>toybox-0.2.0</a>, more a synchronization
4514point than anything particularly useful.  47 commands in a reasonably
4515ready-to-use state (what "make defconfig" builds), another ten or so partially
4516finished stubs ("make allyesconfig"), and several
4517patches pending on the mailing list I need to review and merge.</p>
4518
4519<p>More to come...</p>
4520
4521<hr>
4522<a name="15-11-2011" /><a href="#15-11-2011"><hr><h2><b>November 15, 2011</b></h2></a>
4523- Back from the dead, Toybox is now under a 2
4524clause BSD license, and aiming to become the default command line
4525implementation of Android systems everywhere.</p>
4526
4527<p>More to come...</p>
4528
4529<hr>
4530
4531<p><a href=oldnews.html>Old news</a> from before the relaunch.</p>
4532
4533<!--#include file="footer.html" -->
4534