toybox/www/news.html
<<
>>
Prefs
   1<html><head><title>toybox news</title>
   2<!--#include file="header.html" -->
   3
   4<p>Toybox combines the most common Linux command line utilities together
   5into a single BSD-licensed executable that's simple, small, fast,
   6reasonably standards-compliant, and powerful enough to turn Android into
   7a development environment. See the links on the left for details.</p>
   8
   9<h2>News</h2>
  10
  11<a name="06-05-2022" /><a href="#06-05-2022"><hr><h2><b>May 6, 2022</b></h2></a>
  12<blockquote><p>
  13The mighty ships tore across the empty wastes of space and
  14finally dived screaming on to the first planet they came across - which
  15happened to be the Earth - where due to a terrible miscalculation of scale
  16the entire battle fleet was accidentally swallowed by a small dog.</p>
  17<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
  18
  19<p><a href=downloads/toybox-0.8.7.tar.gz>Toybox 0.8.7</a>
  20(<a href=https://github.com/landley/toybox/releases/tag/0.8.7>git commit</a>)
  21is out, with prebuilt <a href=downloads/binaries/0.8.7>static binaries</a> and
  22<a href=downloads/binaries/mkroot/0.8.7>mkroot images</a>
  23bootable under QEMU (using vanilla linux-5.17).</p>
  24
  25<p>Rob also <a href=downloads/binaries/toolchains/latest>uploaded</a>
  26the prebuilt binary cross and native compilers used to build those
  27mkroot images (there's a <a href=downloads/binaries/toolchains/latest/README>README</a>
  28in there), and started producing
  29<a href=https://www.youtube.com/channel/UC4VFy3wc1nzq5tUHhiti6fw>tutorial videos</a> explaining the
  30toybox commands, their implementation, toybox's infrastructure...</p>
  31
  32<p>The commands <b>host</b>, <b>wget</b>, <b>openvt</b>, and
  33<b>deallocvt</b> were cleaned up and promoted.
  34Elliott contributed the new commands <b>uclampset</b>, <b>gpiodetect</b>,
  35<b>gpioinfo</b>, <b>gpioiget</b>, <b>gpiofind</b>, and <b>gpioset</b>,
  36and Rob wrote a simple <b>httpd</b> (largely to give wget.test something to
  37talk to).</p>
  38
  39<p>One command was removed: <b>catv</b> didn't really serve a purpose
  40(everybody just uses <b>cat -v</b>).</p>
  41
  42
  43
  44
  45<p><u>Features</u>:
  46<b>top</b> can now move the list with the LEFT/RIGHT cursor keys (changing the
  47sort field is now SHIFT LEFT/RIGHT). Added <b>find -samefile</b>,
  48<b>cmp -n</b> (and wired up skip1 skip2),
  49<b>tar --strip components</b> (and tar --owner and --group can now specify :UID and :GID making tar.test more portable),
  50<b>lsusb</b> and </b>lspci</b> now read /etc/{usb,pci}.ids[.gz] (when
  51available) to provide human readable device descriptions,
  52<b>ifconfig</b> can now rename interfaces.
  53Moritz Weber added <b>wget</b> post support for form data.
  54In toysh, <b>cd</b> got updated ("cd -" jumps to $OLDPWD,
  55and cd can now handle deleted directories).</p>
  56
  57<p><u>Bugfixes</u>:
  58<b>cp --preserve xattr,context</b> work together now, and
  59Dario Freni fixed <b>cp --xattr</b> to avoid copying selinux context.
  60The <b>sort -u</b> option now implies -s, <b>wget -O -</b> now writes to stdout,
  61<b>pwget -B</b> removes more characters other implementations don't emit,
  62</b>time</b>'s realtime output was not properly zero padding the fractional
  63part, and <b>sed</b> now works with multiple w to the same file.
  64Fixed <b>nl -E</b>, <b>realpath -e</b>, <b>find</b> -newerXY and @time.nanosecond parsing,
  65Various <b>sh</b> (it handles ${X: -1:1} properly now,
  66which is not the same as ${X:-1:1} because :- has special meaning).
  67The <b>hello</b> command in the examples directory now wait for a keypress before
  68exiting when run as PID 1 (because calling wait() returned "No child process"
  69immediately.)
  70The multiplexer's <b>toybox command --help</b> once again shows the command's
  71help (not toybox --help).
  72Elliott fixed a strip bug in the MacOS build, taught <b>file</b> not to be confused
  73by ELF BSS sections but to otherwise detect more invalid ELF files,
  74and fixed an <b>xargs</b> bug where stdin was inappropriately O_CLOEXEC.
  75Github user hg42 reported a bug where <b>tar</b> was looping trying to remove ".."
  76at the start of names.</p>
  77
  78<p><u>Library</u>:
  79Taught flags.h to #define its own CLEANUP macros so switching FLAG()
  80contexts requires fewer steps.
  81When args.c is just enforcing max/min argument count but collecting no
  82options, imply "?" so "command -123" isn't an unknown option.
  83Moved lib/help.c into main.c (nothing else in lib/ had to rebuild
  84when the config changes which commands are enabled).
  85The remains of linestack.c were renamed utf8.c, and
  86toys.h now includes sys/syscall.h so commands don't have to.</p>
  87
  88<p>Yank things with only one remaining user: moved get_line()
  89to patch.c, struct str_len to bc.c, and struct ptr_len to ps.c
  90Removed tty_esc() and tty_jump() wrappers.
  91Various cleanups to lib/password.c and lib/tty.c.
  92Removed the PEDANTIC_ARGS config option which just set a macro that was never
  93used, xsignal_all_killers() now takes SIG_DFL as the reset value instead
  94of explicitly checking for NULL,
  95chomp() now removes any number of trailing \r\n, and it's now safe to
  96llist_pop(0) just like dlist_pop(0).
  97LongPing Wei improved support for copy_file_range().</p>
  98
  99<p><u>Build</u>:
 100If you don't set $PREFIX, <b>make install</b> now defaults to $PWD/install
 101(which does not require root access to write to, and you can tar up your
 102pile of symlinks from there).
 103More command sub-options removed (NETCAT_LISTEN and CAT_V).</p>
 104
 105<p>Some of the uglier scripts/make.sh plumbing moved into scripts/portability.sh
 106(which should now be safe to source twice),
 107and moved almost all config varible definitions to "configure" now. The
 108new $GENDIR variable theoretically lets the "generated" directory live
 109somewhere else (not very tested yet).
 110Various tweaks to scripts/make.sh to try to avoid unnecessary rebuilds
 111and library probes when toolchain and config haven't changed.
 112Added -Wno-pointer-sign to $CFLAGS because passing an unsigned char * to a
 113function that expects char * is not a problem.
 114
 115<p>The airlock install in scripts/install.sh had the $PENDING items sorted
 116into "started" and "not started". Add git and bash to $PENDING (used
 117out of $PATH by --version and the powerpc64le build, respectively).</p>
 118
 119<p><u>Cleanup</u>:
 120Removed the lib/linestack.c plumbing (which was never finished and hadn't
 121been touched in years).
 122Merged lspci.c into lsusb.c (so they can share config file reading
 123infrastructure), and merged chvt/deallocvt into openvt.c.
 124Minor cleanups to ping, fsync, ionice, pmap, truncate, timeout, tty,
 125factor, mount.
 126Went through and replaced \033 with \e in strings (since clang supports
 127it and \033 is just awkward). Added LL to constant 0 in the FLAG macros
 128to prevent gcc from warning that 0<<32 might produce 0.
 129Moved llvm's -Wno-string-plus-int into configure instead of probing
 130for it, since gcc no longer dies when asked to suppress an unknown warning.</p>
 131
 132<p><u>mkroot</u>:
 133the ANSI escape to undo QEMU's wordwrap bug is now in both the init script
 134and launch wrapper script (so wordwrap isn't broken while running the
 135emulated system, nor if the emulated system fails to run).
 136If devtmpfs was already mounted on /dev, we still need to create dev/pts.
 137The init script now includes the linux kernel bug workaround to enable ping
 138support. The armv7 target enabled CONFIG_ARM_LPAE in the kernel to work around
 139around <a href=https://bugs.launchpad.net/qemu/+bug/1790975>a QEMU bug</a>
 140that's gone unfixed for 4 years. Enable network support in m68k target.
 141The mcm-buildall.sh script got some minor updates, and the resulting
 142cross and native compiler binaries are now downloadable so you don't have
 143to run the build if you don't want to.</p>
 144
 145<p><u>Docs</u>:
 146"toybox --help" no longer says "see toybox --help".
 147There's an ANSI escape cheat sheet at the start of lib/tty.c.
 148Reworded some command directory READMEs, and tweaked menuconfig display.
 149Converted www/doc/mount.html and www/doc/ext2.html from text to html.
 150Freshened up design.html a little (and fixed some typos pointed out by
 151Peter Ennis).
 152Replaced the git internals video URL in roadmap.html with a different
 153performance of the same talk that hasn't been made private.
 154Added the kconfig-language.txt from Linux-2.6.12 to the kconfig/ directory.</p>
 155
 156<a name="30-11-2021" /><a href="#30-11-2021"><hr><h2><b>November 30, 2021</b></h2></a>
 157<blockquote><p>
 158Busy? We'll I've just got this bulldozer to lie in front of or otherwise it'll
 159knock my house down but otherwise no, not especially, why?</p>
 160<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
 161
 162<p><a href=downloads/toybox-0.8.6.tar.gz>Toybox 0.8.6</a>
 163(<a href=https://github.com/landley/toybox/releases/tag/0.8.6>git commit</a>)
 164is out, with prebuilt <a href=downloads/binaries/0.8.6>static binaries</a> and
 165<a href=downloads/binaries/mkroot/0.8.6>mkroot images</a>
 166bootable under QEMU (using vanilla linux-5.15).</p>
 167
 168<p><u>Make root</u>: The <a href=faq.html#mkroot>system builder</a> got upgraded, with better
 169logging and layout of the output directory, plus <a href=https://github.com/landley/toybox/blob/0.8.6/scripts/mkroot.sh>scripts/mkroot.sh</a>
 170should be easier to read and understand now.
 171(Also added basic support for the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-August/012497.html>qualcomm hexagon</a>,
 172although it just builds a chroot filesystem, you can't boot a hexagon kernel under qemu
 173<a href=https://lists.nongnu.org/archive/html/qemu-devel/2021-07/msg06430.html>quite yet</a>.)</p>
 174</p>
 175
 176<p><u>New commands</u>: Rob added <b>linux32</b>.
 177Dan Brown added <b>sha256sum</b> (and the related 224/384/512 variants).
 178In pending, Elliott contributed <b>strace</b> and
 179Moritz Röhrich contributed <b>hexdump</b>.</p>
 180
 181<p><u>Features</u>:
 182Elliott added <b>date -s</b>, <b>pmap -p</b>, <b>tail -F -s</b>, <b>kill -0</b>,
 183<b>reboot/halt/poweroff -d</b>, taught <b>file</b> to
 184recognize Mach-o universal binaries, made Android's <b>log</b>
 185take input from stdin when it has no arguments, added <b>tail --bytes --lines</b>
 186(the --long synonyms for -c and -n),
 187added <b>i2cdetect -q</b>, taught <b>i2cdump</b> to write XX for
 188read failures, made <b>timeout</b> use the monotonic clock,
 189and modernized the output of <b>uptime</b>.</p>
 190
 191<p>Rob added <b>find -quit -lname -ilname</b> and <b>-d</b> as a synonym for -depth,
 192taught <b>cut -d $'\n'</b>
 193to cut by line and added posix <b>cut -nb</b> (which wraps to the start of -c
 194utf8 characters), added <b>cpio --ignore-devno --renumber-inodes</b>,
 195taught <b>cal</b> to use 3 arguments (day month year), added <b>tar --selinux</b>
 196support and a "had errors" message at the end, made <b>uname -o</b>
 197say "Toybox", and fixed the <b>grep -Fw</b> combination (used by the 64 bit
 198powerpc kernel build).</p>
 199
 200<p>Ella-0 added <b>split -n</b>. Peter Collingbourne added <b>grep -L</b>.</p>
 201
 202<p><u>Outside world</u>: The 0BSD license is <a href=https://lwn.net/Articles/868882/>now part</a> of the
 203<a href=https://standards.iso.org/ittf/PubliclyAvailableStandards/>ISO-5962</a>
 204standard. Of the currently <a href=https://github.com/search?q=license%3A0bsd&type=Repositories&ref=advsearch>38k 0BSD licensed projects on github</a> toybox
 205isn't even the most popular, it's fourth.</p>
 206
 207<p>Elliott asked that toybox's <b>cut -DF</b> feature (a cheaper
 208alternative to <b>awk '{print $3}'</b> be supported by other packages so
 209it might someday make it into posix, so Rob
 210<a href=http://lists.busybox.net/pipermail/busybox/2021-June/088886.html>submitted it to busybox</a>
 211where it was <a href=https://git.busybox.net/busybox/commit/?id=0068ce2fa0e3>eventually merged</a>.</p>
 212
 213<p>Added a local <a href=https://landley.net/toybox/git>git mirror</a> for
 214those reluctant to trust Microsoft Github.</p>
 215
 216<p>In the web nav bar, our <a href=https://isfreenodedeadyet.com/>freenode</a>
 217channel moved to libera.chat
 218(like <a href=https://boingboing.net/2021/05/19/freenode-irc-staff-quit-after-new-owner-seizes-control.html>everyone else's</a>;
 219<a href=https://lwn.net/Articles/856543/>there</a>
 220<a href=https://hackaday.com/2021/05/20/freenode-debacle-prompts-staff-exodus-new-network/>was</a>
 221<a href=https://twitter.com/marcan42/status/1396280769778098185>a</a>
 222<a href=https://twitter.com/mjg59/status/1396237336988446722>certain</a>
 223<a href=https://www.devever.net/~hl/freenode_abuse>amount</a>
 224<a href=https://twitter.com/TwitchiH/status/1395756163527028739>of</a>
 225<a href=https://twitter.com/joepie91/status/1395364883303182339>drama</a>
 226<a href=https://twitter.com/RichFelker/status/1404816846734385164>involved</a>).</p>
 227
 228<p><u>Pending</u>: lots more work on <b>toysh</b>: it has basic $((math)) support
 229now, += assignment, the "declare" command and corresponding variable
 230types (declare -i, declare -u, etc), several bugfixes, greatly
 231improved support for background processes on NOMMU systems, and a bunch
 232more sh.test entries.</p>
 233
 234<p>Elliott fixed <b>modprobe</b>'s parsing of short lines. Rob fixed a
 235division by zero error when <b>vi</b> tries to edit a zero length file.
 236Anqi Dong made sure that syslogd messages sent to sockets end with a
 237newline. Eric Molitor significantly upgrade <b>wget</b> with http 1.1,
 238chunked encoding, and (external library) https (TLS 1.2) support,
 239plus several bugfixes. Michael Hope fixed <b>tftpd</b>'s option parsing.</p>
 240
 241<p><u>Documentation</u>: The roadmap.html and status.html pages
 242got noticeable updates. (status.html is now directly generated by
 243scripts/mkstatus.py, the old status.html wrapper that server-side-included
 244status.gen has been removed).
 245The <a href=design.html>design</a>
 246page provides better links to the LP64 documentation, and the
 247<a href=code.html>code</a> page explains a corner case where the filename
 248and first command name expect to match (and how to work around it if they don't).
 249Some website files weren't in the repository: two jpegs linked from html
 250files, and the www/.htaccess file that triggers the server side includes
 251in the navigation menu is now checked in. Johannes Schmitz fixed a
 252typo in about.html.</p>
 253
 254<p>The <b>shred</b> usage line now includes all options.
 255The <b>logwrapper</b> command now warns when it isn't built standalone.
 256The <b>help</b> command no longer shows a header line about toybox unless
 257it's called with no arguments. (It still shows it for command --help.)
 258The --help option no longer needs to be the first option, and is
 259checked before the "Not root" error message for restricted commands.</p>
 260
 261<p><u>Bugfixes</u>:
 262Elliott made <b>ps</b> exit with status 1 if no processes were shown (for LTP),
 263fixed an off-by-one in <b>dmesg</b> and a bounds check in <b>netstat</b>,
 264fixed <b>pidof -s</b>, and taught <b>rmmod</b> to handle multiple arguments.
 265Denys Vlasenko pointed out that <b>free -t</b> is "total" not "terabytes".
 266Samanta Navarro pointed out a leftover 32 bit limitation in argument groups.
 267Mikhail Kashkarov fixed mkdir's error handling for existing directories
 268without -p.
 269Rob fixed the 32 bit <b>readelf</b> build on Android, fixed a race condition
 270in <b>tar</b> leading
 271to the occasional spurious archiver warning,
 272<a href=https://github.com/landley/toybox/commit/127cff38809e>changed cp</a>
 273to work around a posix violation in the kernel, made xparsedate() handle more
 274whitespace and ignore trailing + or - (a NOP timezone), made <b>su</b> log the
 275right username, switched getuid() checks to geteuid() in <b>df</b> and <b>ping</b>,
 276fixed a place in the test suite where do_test()
 277could get confused by symlinks in the $PATH (calling realpath too early
 278and losing the name of the command being run),
 279made <b>env -i</b> use the old $PATH to run its command,
 280and made <b>grep --color</b> highlight all matches.</p>
 281
 282<p>Worked around a bug in glibc+gcc 9.3 where linking against librt.a
 283(for things like timer_create()) pulls in libgcc_eh.a which uses
 284threads, meaning you can't statically link against librt.a from
 285a non-threaded app. (I hadn't noticed because it works fine with the musl
 286and llvm/bionic toolchains, only fails statically linking on e.g. debian.)
 287The <a href=https://github.com/torvalds/linux/commit/d18760560593>kernel fix</a> for the cpio encrypted filesystem issue worked around in 0.8.5 made
 288it in. Similarly, the
 289FS_IOC_{GET,SET}{VERSION,FLAGS} ioctl family are <a href=https://www.spinics.net/lists/linux-api/msg49101.html>historically broken</a>
 290(taking an unsigned int argument while advertising signed long),
 291which <a href=https://github.com/landley/toybox/commit/dcfac5c983ad>we fixed</a> and reported upstream to a long
 292round of bikeshedding (sadly
 293the result seems to have been the man page maintainer
 294<a href=https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/commit/man2?id=3de87d46840d>giving up</a>
 295on documenting ioctls at all.</p>
 296
 297<p><u>Cleanups</u>:
 298<b>cut</b> switched to FLAG() macros and no longer allows -f with -F,
 299<b>eject</b> uses FLAG() macros and the linux/eject.h header,
 300prettycool@protonmail added tabs to <b>time</b>'s output, and Rob removed
 301the floating point dependency from time. Minor cleanups
 302to lsattr, lspci, nsenter, xxd, file, insmod, modprobe, arp, bootchartd, pmap,
 303comm, lsof, blkid, crond, init, userdel, makedevs, and scripts/make.sh.
 304The "int fd = fd;"
 305initialization hack to shut up gcc's "may be used uninitalized, but isn't" false
 306positive warning has been replaced by a QUIET macro that resolves
 307to nothing on llvm (which doesn't have the gcc bug).
 308Ryan Pritchard reported a bug where <b>alias ls="ls --color"</b> would prevent
 309--help or --version from being the first argument to the command, which was
 310a pain to fix without breaking things like <b>echo -- --version</b> and
 311<b>basename -s --help file--help</b>.
 312(Both toy_singleinit() in main.c _and_ get_optflags() in lib/args.c now
 313check for those.)</p>
 314
 315<p><u>Infrastructure</u>:
 316xabspath() now has a more granular (flag based) control interface.</p>
 317
 318<p><b>scripts/make.sh</b> now uses wait -n (available since bash 4.3 in February 2014)
 319to keep the processors busier during a parallel build.
 320Tests are now running in a subshell, which means functions and local
 321variables defined in one don't pollute others during "make tests".
 322
 323<p>Guo Chuang reported that lsattr won't build on an 8 year old version of
 324<a href=https://github.com/landley/toybox/pull/292#issuecomment-899895909>centos</a>,
 325which is still supported for 2 more years because Red Hat can
 326be paid to have a 10 year support horizon, longer than toybox's 7. Rob
 327fixed it (and merged a similarly ancient compilation failure fix when
 328building pending/ip.c against a 3.10 kernel),
 329but the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-August/012537.html>philosophal argument</a>
 330remains <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-September/012603.html>unresolved</a>.</p>
 331
 332<p>Standalone builds can now override the .config file location with
 333KCONFIG_CONFIG same as the multiplexer build. scripts/single.sh can
 334now cope with a missing .config file (it uses the defconfig values
 335for the CONFIG_TOYBOX_* global symbols).
 336"make install_airlock" no longer needs host versions of sha256sum and sha512sum.</p>
 337
 338<p>In the test suite, <b>VERBOSE=spam</b> now shows expands successful tests
 339like it would for errors.</p>
 340
 341<p><b>scripts/record-commands</b> got updated and logwrapper got renamed <b>logpath</b>.</p>
 342
 343<a name="19-05-2021" /><a href="#19-05-2021"><hr><h2><b>May 19, 2021</b></h2></a>
 344<blockquote><p>It was of course a descendant of these eccentric poets
 345who invented the spurious tales of impending doom which enabled the people of
 346Golgafrincham to rid themselves of the useless third of their population.
 347The other two-thirds stayed firmly at home and lived full, rich and happy lives
 348until they were all suddenly wiped out by a virulent disease contracted from
 349a dirty telephone.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
 350
 351<p><a href=downloads/toybox-0.8.5.tar.gz>Toybox 0.8.5</a>
 352(<a href=https://github.com/landley/toybox/releases/tag/0.8.5>git commit</a>)
 353is out, with prebuilt <a href=downloads/binaries/0.8.5>static binaries</a> and
 354<a href=downloads/binaries/mkroot/0.8.5>mkroot images</a>
 355bootable under QEMU (using vanilla linux-5.12, except that s390 needed another
 356<a href=http://lkml.iu.edu/hypermail/linux/kernel/2105.2/02904.html>perl
 357removal patch</a>).</p>
 358
 359<p>This development cycle had thirty commits to sh.c fixing bugs and adding
 360features to <b>toysh</b>. Shell functions and local variables are now implemented,
 361along with the "set" builtin and trace support, and job control is
 362about halfway in. The shell as a whole is <a href=https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule>maybe 80% done</a>, but still
 363has some obvious gaps like $((math)) and interactive command line
 364history/editing, plus a bunch of testing.</p>
 365
 366<p><u>New commands</u>:
 367Moritz Röhrich added <b>pwgen</b> and <b>base32</b>, and
 368Elliott Hughes added <b>unicode</b>. The
 369<b>readelf</b> command got promoted out of pending, and
 370<b>sha3sum</b> is now switched on in defconfig.</p>
 371
 372<p>Michael Christensen added <b>chsh</b> to pending, which hasn't been promoted
 373yet because it's part of a group of commands (useradd, userdel, groupadd,
 374groupdel) that should all be promoted together at some point
 375(and depend on lib/passwd.c, which could use some more work).</p>
 376
 377<p><u>Upgrades</u>:
 378<b>cpio</b> now supports extracting multiple concatenated archives.
 379<b>file</b> now recognizes android's new XML binary format.
 380<b>devmem</b> now works on nommu, can handle 0x prefxes on the address,
 381and prints
 382hex or decimal output to match the address type.
 383<b>df</b> -a now shows overmounted filesystems.
 384<b>test</b> -k checks the sticky bit.
 385Added -t to <b>install</b>, <b>cp</b>, and <b>mv</b>, and -u to <b>cp</b>
 386and <b>cpio</b>.
 387<b>find</b> -executable checks that _we_ can execute them (u+x could be owned
 388by a different user or have selinux weirdness).</p>
 389
 390<p>Elliott added several features to <b>date</b> (--iso, %:z output format, and
 391--utc as a synonym for -u), added a <b>sed</b> s///x option,
 392added units to <b>ulimit</b> output, and
 393made several UI changes to <b>hexedit</b> including a color
 394mode (which is now the default, press x to toggle between old and new
 395display), masked out the file type in <b>chmod</b> error messages and
 396taught it to handle complex modes (like u+s+s), 
 397hardened <b>file</b> and <b>readelf</b> against invalid input,
 398added a portability.h workaround for macos statvfs for <b>df</b>,
 399enabled line buffering in <b>echo</b> and <b>yes</b>,
 400and significatly sped up <b>tr</b>. Rob similarly optimized <b>seq</b>
 401and <b>count</b>.</p>
 402
 403<p><b>make tests</b> now defaults to VERBOSE=fail (and now has VERBOSE=all to
 404continue after failure, and VERBOSE=quiet to show FAIL: lines without
 405diff -u output).
 406Cleanups to tee, netstat, file, and df. Added "static" annotations in a
 407lot of commands, made more things use FLAG() macros. Ethan Sommer pointed out
 408an unnecessary wrapper function in chgrp.
 409In pending, Elliott did some cleanups and bugfixes to gettty, telnet, telnetd,
 410tftpd, init, modprobe, and dd, and added more features to vi.
 411Rob did some cleanup on modprobe.</p>
 412
 413<p><u>Bugfixes</u>:
 414Yi-Yo Chiang fixed a <b>rm</b> bug where -f could get confused by read
 415only filesystems and produce an error trying to unlink a nonexistent
 416file, pointed out <b>install</b> -D was broken when the target file
 417already exists, pointed out a <b>cpio</b> interaction with certain encrypted
 418filesystems where readlink() wouldn't write as much payload as the header
 419promised (corrupting the archive) and fixed another cpio bug where -t test
 420mode would chown() files, and fixed quoting in mount.test.
 421Elliott fixed a sequencing issue in <b>setsid</b>'s system calls,
 422fixed some <b>sed</b> tests (sed's lie-to-autoconf --version
 423output was preventing toyonly tests from triggering) and
 424fixed <b>tar</b>'s base 256 output for very large files.
 425Nir Lichtman pointed out that <b>rm</b> -p shouldn't try to rmdir / at the end,
 426Luke Frankart pointed out a typo that broke <b>test</b> checking file modes,
 427Firas Khalil Khana fixed a duplicate entry in the roadmap,
 428Ethan Sommer fixed <b>install</b> -Dt,
 429Andy Hu requested that <b>mount</b> handle mixed options like
 430-tnomsdos,smbfs.
 431Denys Nykula reported a <b>chmod</b> regression,
 432asm89 pointed out a <b>netcat</b> bug writing transactions larger than
 4334k,
 434Daniel Santana fixed <b>df</b> combining -P and -k,
 435Robin Hsu fixed <b>grep</b> getting confused by libc setting errno when
 436a library function call succeeded.
 437Puck Meerburg taught the <b>gunzip</b> code not to be confused by FEXTRA entries
 438(as described in RFC 1952) and added a test suite entry for that.</p>
 439
 440<p><b>find</b> handles empty arguments properly now (with test), <b>date</b>
 441-I with no arguments no longer segfaults, <b>grep</b> -f /dev/null no
 442longer adds a "" regex matching everything, -z no longer splits patterns
 443on \n, and -o can print zero length matches,
 444<b>cp</b> -t works with one argument and with --parents.
 445<b>readelf</b> displays properly in 80 columns and doesn't reject .bss
 446off end of file, and <b>sed</b> s///2 now works when there isn't more than one
 447match.
 448<b>tar</b> now removes empty directories where it wants to put a non-directory.
 449<b>microcom</b> sets serial device terminal attributes correctly.</p>
 450
 451<p>The prebuilt x86-64 binaries shouldn't segfault on older x86-64 processors
 452anymore (workaround for gcc bug).
 453Cross compiling was sometimes confusing install.sh because it created
 454a hyphenated "toybox-$TARGET" and then tried to make symlinks to just "toybox".</p>
 455
 456<p><u>Build</u>:
 457CONFIG_TOYBOX_I18N went away: toybox builds always support utf8 now.
 458Ella-0 added OpenBSD support (but not a separate defconfig, try
 459"make freebsd_defconfig").</p>
 460
 461<p><u>Library</u>:
 462utf8towc() is now unsigned instead of wchar_t, matching wctoutf8().
 463xclose() tests if fd is -1 so the caller doesn't have to.
 464txpect (the test suite's built-in "expect", implemented in shell) can
 465now do regex matches, and sh.test uses this.</p>
 466
 467<p>Elliott added the failing regex to the xregcomp() error message, and
 468made multiple portability tweaks (header tweaks to build in more
 469environments, running tests on Raspberry Pi OS and under mksh,
 470fix utf8 locale selection on macos).</p>
 471
 472<p><u>Docs</u>:
 473<b>chmod</b>'s help text explains why o+s doesn't set +t (and has a
 474test for it). The <b>echo</b> help text is more compact.
 475Checked in a better conversion of <a href=https://landley.net/toybox/toycans.png>toycans.png</a>.</p>
 476
 477<a name="24-10-2020" /><a href="#24-10-2020"><hr><h2><b>October 24, 2020</b></h2></a>
 478<blockquote><p>
 479"We are now cruising at a level of two to the power of twenty-five thousand to
 480one against and falling, and we will be restoring normality just as soon as we
 481are sure what is normal anyway."</p>
 482<p>- The Hitchhiker's Guide to the Galaxy</p>
 483</blockquote>
 484
 485<p>After a longer and slightly more
 486<a href=https://github.com/landley/toybox/commit/07a896862ddf>turbulent</a>
 487development cycle than some,
 488<a href=downloads/toybox-0.8.4.tar.gz>Toybox 0.8.4</a>
 489(<a href=https://github.com/landley/toybox/releases/tag/0.8.4>git commit</a>)
 490is out with new commands <b>sha3sum</b> and <b><a href=https://github.com/landley/toybox/commit/6b4c32ae3986>watchdog</a></b>.</p>
 491
 492<p>There are <a href=downloads/binaries/mkroot/0.8.4>prebuilt mkroot binaries</a>
 493now, tiny toybox linux systems for a dozen architectures, all bootable under
 494qemu. (The vmlinux in each tarball is a vanilla linux-5.9 kernel built
 495from the included config file.)</p>
 496
 497<p>The FAQ got <a href=faq.html>noticeably larger</a>, and the README has
 498more links. New command features include the <b>sed -s</b> flag,
 499<b>cpio --no-preserve-owner</b> now affects archive creation,
 500Elliott added <b>tar -I</b> and multi-type
 501<b>find -type a,b,c</b> support, Mark Salyzyn added <b>xargs -P</b> to
 502run parallel jobs, the <b>ps</b> and <b>top</b> commands
 503now autodetect pid length, and <b>top</b> adjusts units for memory display based on
 504system size.</p>
 505
 506<p><u>Toysh and toyroot</u>:
 507lots of new work on <b>toysh</b>: 29 commits to sh.c since last release, adding 1500
 508lines and deleting 700, plus a bunch of sh.tests entries). Since last release
 509we implemented wildcards, case/esac and select, brace expansion sequences
 510(ala {1..10..2} and {a..z}),
 511the remaining variable slice types ${a#y} ${a%y} ${a^y} ${a,y}
 512${a/search/replace}, the "<b>source</b>" shell builtin,
 513the start of job control, and several bugfixes.
 514Plus the standalone "<b>make sh</b>" build understands MAYFORK now. It's
 515still missing features like functions() and $((math)) but it's getting
 516close to usable now.</p>
 517
 518<p><u>Documentation</u>:
 519Some of the README contents moved to the FAQ, and the README's "presentations"
 520section got some new links. Several new <a href=faq.html>FAQ</a> entries explaining things like
 521mkroot ("how do I build a working Linux system
 522with toybox"), cross compiling (how to get/setup the 2 compilers and
 5233 libc we regression test against), and "where does toybox fit into the
 524linux/android ecosystem".
 525The "toybox --help" output now lists the project's web page (<a href=https://github.com/landley/toybox/issues/50>by request</a>).
 526Elliott removed
 527getevent (an android board bringup/hardware debugging tool built by running
 528a python script against kernel headers, not really in scope for toybox),
 529fixed xargs help formatting, and taught the toybox multiplexer's
 530command list output (and "kill -l") to measure the current the terminal width
 531when wordwrapping (previously hardwired to 80 columns).
 532If you're curious, I checked in my <a href=www/release.txt>release procedure
 533checklist</a>, and
 534fixed a stale link in the nav bar on the left ("statistics" changed domains).
 535Firas Khalil Khana fixed some issues in the roadmap, and Rob
 536tweaked the roadmap so status.html is slightly more current.</p>
 537
 538<p><u>Tests</u>:
 539The "make tests" target now fails if any of the tests it ran failed.
 540Lots of TEST_HOST tests got fixed (checking for specific error messages exposes
 541TEST_HOST to version skew, and scripts/test.sh will now skip TEST_HOST
 542commands that aren't installed) and added toyonly annotations as necessary
 543(replacing most uses of SKIP_HOST).
 544Eric Molitor added automated github tests on MacOS and Ubuntu using their
 545"workflows" thing.
 546The clang asan plumbing slows some code down more than 10 times, so some testing timeouts were expanded.
 547
 548<p><u>Pending</u>:
 549Eric Molitor did a bunch of work on route: moved it to sbin, added xsend and
 550xrecv, taught display_routes() and setroute to use the rtnetlink API and
 551do hostname lookups, added support for mss, win, and irtt, merged ipv6 and ipv4
 552codepaths with autodetection of address type, implemented RTA_CACHEINFO
 553support, switched exit paths to perror_exit(), and removed unused code.
 554Ethan Sommer fixed warnings in dhcpd, removed a bunch of unnecessary
 555; after GLOBALS() blocks and an unnecessary return in df.
 556Erik Moqvist fixed dns setting in the dhcp client.
 557Rob did some cleanup on bootchartd, traceroute, getty...
 558Elliott fixed getty to reliably update utmp.
 559Chris Sarra added ipv6 support to wget.
 560Ariadne Conill submitted several small fixes from testing Alpine Linux with
 561toybox, and Antoni Villalonga i Noceras fixed typos in error messages.</p>
 562
 563<p><u>Bugfixes</u>:
 564Elliott avoided sign extension in devmem, and Ethan Sommer switched it from
 565getpagesize() to posix sysconf(_SC_PAGESIZE).
 566Elliott fixed chmod -R
 567ignoring dangling symlinks, fixed stty <a href=https://github.com/landley/toybox/issues/251>misparsing c_iflags</a>, fixed the chattr f2fs test with
 568compression enabled, taught blkid not to show empty tags, taught xparsedate()
 569to read date's default output format, fixed a recent echo -e \0 regression,
 570changed cpio -p parsing to match a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-August/011955.html>bug</a> in the gnu version
 571which an existing script <a href=https://github.com/landley/toybox/commit/fa1af3b085cc>depended on</a>,
 572and switched hwclock back to looking
 573only at /dev/rtc0 for <a href=https://github.com/landley/toybox/commit/70e2232ce61c>reasons</a> involving kernel version skew and vendor bug reports.</p>
 574
 575<p>David Legault pointed out that unescape2() (and thus echo -e) wasn't handling
 576\0 right.
 577Khem Raj reported that mips glibc doesn't have SIGSTKFLT, leading to
 578a build break if we assume glibc behaves consistently, which led to a bunch of
 579macos signal portability cleanups from Elliott.
 580William Djupström fixed tar extracting long file paths, adding hardlinks
 581to an archive, and reported that --exclude wasn't working.
 582Peter McConalogue pointed out that cp/mv -i prompt should
 583default N, and mv should only prompt when stdin is a tty.
 584Patrick Oppenlander made rtcwake and hwclock default to UTC if /dev/adjtime
 585isn't available, suggested watchdog catch SIGINT, and caught an
 586uninitialized offset in blkdiscard which gcc apparently didn't.
 587Ryan Pritchard reported that <b>file</b> wasn't getting gif heights right,
 588and Elliott added gif version output so TEST_HOST provided similar output.
 589Martin Stjernholm fixed <b>cp -P</b> to not follow symlinks and updated
 590the help text to say it's not the default.
 591Antoni Villalonga added more --long asiases to <b>chgrp/chown/rmdir</b>.
 592
 593<p>The symlink indirection code subtly broke xexec() so it would still try to call
 594a builtin when given a path to a command (fixed now).
 595The code to trip \n off xgetline() was using the length of the allocation
 596instead of the length of the read.
 597Netcat no longer leaks sockfd into child processes.
 598Patch "fuzz" support was outputting context lines from the hunk, not the
 599file. Chrisrfq fixed i2cdetect parameter reading.
 600Chris Sarra taught dd not to throw an error trying to truncate device files,
 601added a misshing fflush() to lib/password.c, fixed a parse error in logger
 602preventing local facility logging from working right, taught init
 603to reload inittab when it receives SIGHUP, and fixed a memory leak in tar.</p>
 604
 605<p><u>Library</u>:
 606New xvdaemon() function daemonizing on nommu systems, to wean commands
 607off daemon() which requires TOYBOX_FORK.
 608Teach sendfile_len() to use copy_file_range() when available, with compile
 609time probe. Fix xsignal_all_killers() to install the correct handler.
 610Rob switched dirtree_path() to a non-recursive implementation, and
 611taught dirtree that a top level entry named "" is equivalent to
 612"." but should not show up in dirtree_path(),
 613read_password() now reads from tty device rather than stdin, and handles ctrl-c and ctrl-d, and
 614human_readable() now has HR_NODOT so it can print single digit numbers without going "0.0".
 615Petri Gynther increased the vmstat column sizes so it didn't fit
 616in 80 columns anymore.
 617Elliott added UTC offset support to xparsedate().</p>
 618
 619<p><u>Infrastructure</u>:
 620mkroot now only passes --no-preserve-owner to cpio (so initramfs files
 621belong to root rather than whichever user ran the build) when using toybox's
 622version, because the other one is brain-damaged and errors out not
 623understanding it. (It understands it for extract, but not create. Yes really.)
 624New PENDING variable you can set to enable more than just sh and route out
 625of pending.
 626The "make distclean" no longer deletes root_download (where mkroot's "extra plumbing"
 627file saves source code downloads it tarballs, such as dropbear).</p>
 628
 629<p><u>Cleanup</u>:
 630Cleanup openvt (35 lines added, 78 lines removed)
 631Cleanup blkdiscard so lib/args.c parses the -o and -l numbers, which means
 63232 bit systems are limited to 2 gigabyte sizes which is a TODO item systemwide.
 633oneit now uses flag macros and mentions -rn in the help. demo-utf8towc
 634now has main.c call setlocale for it.
 635Some cleanup on stty.</p>
 636
 637<a name="11-05-2020" /><a href="#11-05-2020"><hr><h2><b>May 11, 2020</b></h2></a>
 638<blockquote>
 639<p>Ford: Ah. A
 640<a href=https://www.youtube.com/watch?v=WSsR419HBpQ>towel</a>.
 641Keep this and guard it with your life.</p>
 642<p>Arthur: Huh?</p>
 643<p>Ford: Listen, it's a tough universe. There's all sorts of people and things
 644trying to do you, kill you, rip you off, everything. If you're going to
 645survive out there, you've really got to know where your
 646<a href=towel.jpg>towel</a> is.</p>
 647<p>- The Hitchhiker's Guide to the Galaxy</p>
 648</blockquote>
 649
 650<p>Despite everything <a href=downloads/toybox-0.8.3.tar.gz>Toybox 0.8.3</a>
 651(<a href=https://github.com/landley/toybox/releases/tag/0.8.3>git commit</a>)
 652is finally out, with new commands <b>rtcwake</b> from Elliott Hughes and
 653<b>blkdiscard</b> from Patrick Oppenlander.
 654The big news is "<b>make root</b>" now boots to a shell prompt,
 655with toysh making it all the way through toyroot's init script.
 656(Some people were looking forward to <b>patch --fuzz</b> support too.)</p>
 657
 658<p><u>Toyroot</u>: <b>make root</b> now does what it says on the tin, it
 659builds a bootable toybox-based Linux system using two source
 660packages (toybox and linux). The trivial version is "make root && sudo chroot
 661root/host/fs /init". Here's
 662a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-April/011667.html>post with instructions</a> if you want to know how to build the
 663cross compilers for testing the various architectures. The self-contained
 664<a href=https://github.com/landley/toybox/blob/0.8.3/scripts/mkroot.sh>script
 665that builds a simple bootable Linux system</a> is 250 lines long, and
 666should be easy to read if you want to know how it works.</p>
 667
 668<p>It also has basic module support, meaning arguments that aren't X=Y variable
 669assignments are the names of scripts to run to build more
 670packages at the end of the build. I checked in an <a href=https://github.com/landley/toybox/blob/master/scripts/root/dropbear>example package</a>,
 671and there's generic "download and extract source tarballs"
 672<a href=https://github.com/landley/toybox/blob/master/scripts/root/plumbing>plumbing</a> available to such modules.
 673(The Makefile doesn't know how to pass module names through to the script,
 674so instead of "make root" you have to
 675call the script directly, ala "scripts/mkroot.sh CROSS=sh4 LINUX=~/linux dropbear".)</p>
 676
 677<p>The resulting root filesystem now uses /root as the home
 678dir for UID 0, and creates /dev/fd and /dev/shm in devtmpfs. The build works
 679around a kernel
 680build bug where "make distclean" doesn't work in a "cp -sfR" symlink
 681directory. (The bug meant if you pointed LINUX= at unclean source, it was
 682unhappy, so now it distcleans the source directory before copying it. This
 683modifies said source directory, which is not ideal, but as usual the kernel guys
 684<a href=http://lkml.iu.edu/hypermail/linux/kernel/2002.2/00083.html>ignored
 685the bug report</a>, so a workaround was required.)
 686The CROSS=all build announces each target in the title bar, puts
 687its logs into root/log, and has better trap handling to stop with one
 688ctrl-c.
 689Since last release it uses a more concise config format for the various kernel
 690arch targets (which shrank the script by about 50%), and merged
 691the old (now removed) scripts/cross.sh into the main script so
 692"make CROSS=armv7l LINUX=~/linux" is
 693literally just a call to "scripts/mkroot.sh CROSS=armv7l LINUX=~/linux" now.</p>
 694
 695<p>As for <b>scripts/mcm-buildall.sh</b> building cross compilers,
 696of COURSE gcc 8.3 requires a different configuration to build the same
 697toolchain as previous versions, it's gcc. It now builds a proper
 698nommu libc for sh2eb without a broken fork() call in it that can never
 699be used but prevents compile-time probes from detecting nommu,
 700and checks that the cross compiler completed before trying to build
 701the native compiler.</p>
 702
 703<p>This was all tested with a recent
 704<a href=https://github.com/richfelker/musl-cross-make/commit/5086175f2902>version</a>
 705of musl-cross-make with the top level Makefile
 706edited to use BINUTILS_VER = 2.32 because the newer one
 707<a href=https://www.spinics.net/lists/linux-sh/msg56844.html>breaks the kernel
 708build</a>
 709and LINUX_VER = 4.19.90 because the default musl-cross-make config uses an
 710out-of-tree headers package for some reason (those who forget history are
 711<a href=https://lkml.org/lkml/2006/4/28/194>doomed to repeat it</a>) which
 712breaks m68k and s390x. I won't post binaries of the resulting toolchains
 713because they're GPLv3, but <a href=https://musl.cc/>Thalheim might</a>.
 714(I've also <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-May/011673.html>test built</a>
 715with the Android NDK, but bionic's startup code currently segfaults before
 716calling main() if it can't open /dev/null, and the kernel guys
 717<a href=https://lkml.org/lkml/2017/9/13/651>ignored my patches</a> to
 718make CONFIG_DEVTMPFS_MOUNT work in initramfs.)</p>
 719
 720<p>At the moment toyroot cheats and uses two commands out of pending: toysh
 721is about 80% of the way to being useful but still missing some
 722obvious features like function support, job and terminal control, command
 723history, $((math)), wildcard expansion... plus a lot of bash features like array
 724variables, so it isn't out of pending yet. And route needs to be redone to use the
 725netlink interface that can handle multiple routing tables. For the moment
 726scripts/mkroot.sh adds both of them to defconfig if you haven't already
 727got a .config when you run it. (If you build and can't boot, your .config
 728probably hasn't got CONFIG_SH switched on. Fix it in menuconfig, or
 729rm .config and try gain.)</p>
 730
 731<p><u>New toybox features</u>:
 732Elliott taught <b>patch</b> to understand [FILE [PATCH]] arguments, made
 733<b>cal</b> highlight the current day, added -T to <b>cp</b>/<b>mv</b>,
 734did a lot of work on <b>lsattr</b>/<b>chattr</b> (including adding -p
 735and "chattr ="), added tar --absolute-names, taught <b>id</b> to
 736support numeric lookup and handle unknown groups, made -G show all
 737groups, and removed context= from -Z.
 738Rob added <b>patch -F</b> (fuzz factor) support and <b>help -u</b> (usage only),
 739taught <b>echo -e</b> about bash extensions like \uXXXX unicode escapes,
 740<b>netcat -L</b> no longer automatically includes stderr (new -E
 741option does that),
 742<b>setsid</b> now uses -c (like the man page says) instead of -t, and added
 743-w (wait) and -d (detach from tty). 
 744Andrew Ilijic added <b>ls -w</b>, removed trailing whitespace on output,
 745and added tests for -C and -x.</p>
 746
 747<p><u>Library</u>:
 748another fix to 32 bit option parsing of long long optflag values,
 749remove getdirname() and just use the libc function (which modifies
 750its argument, but we don't have to free a malloc),
 751dlist_terminate() can now be called repeatedly on the same list,
 752new relative_path() function finds path from one directory to another,
 753and readfd() works like readfile() but on an already open fd.
 754Use MPATH macros in mkpathat(). Elliott added macOS versions of
 755dev_minor()/dev_major()/makedev() to portability.c,
 756implemented posix_fallocate() for macOS, removed a
 757leftover uClibc workaround in fallocate that was breaking macOS,
 758and moved the table of ELF architectures from file to lib/lib.c (so
 759readelf can share).
 760Park Ju Hyung pointed out the fast path of fdlength() was commented out,
 761and we switched it to the 64 bit API while we were there (and then Elliott
 762added macOS support).
 763Joeky taught file to recognize 7z archives.</p>
 764
 765<p><u>Pending</u>:
 766The shell got a bunch of work: standalone "make sh" now includes the
 767multiplexer for builtin commands like "exit" and "cd". Added MAYFORK annotation
 768for commands that exist in the $PATH but can also be run within the shell
 769process (and sometimes have different behavior within the shell):
 770currently applied to help, echo, false, kill, printf, pwd, test, time, and
 771true.</p>
 772
 773<p>Elliott added new commands <b>getopt</b> and <b>readelf</b>.
 774Jarno Mäkipää taught wget how to follow http 301 and 302 redirects,
 775and did lots of work on vi. (Elliott also did work on vi.)
 776Gavin Howard fixed a comparison bug in bc.
 777Ethan Sommer fixed numerous small issues (including several build
 778warnings and FLAG() macro conversions).</p>
 779
 780<p><u>Bugfixes</u>:
 781<b>tar</b> extract now deletes files and symlinks where it's making a directory,
 782<b>find -L -type -l</b> now finds dangling symlinks, extra #ifdefs in
 783portability.h prevent old gcc versions from barfing on __has_include(),
 784xgetline() now returns NULL at EOF, tee with no arguments was
 785writing to stdout twice, setsid works on nommu (I.E. vfork) now,
 786<b>netcat -L</b> no longer accumulates zombie processes,
 787<b>sntp</b> now uses adjtimex instead of adjtime (to build on bionic).
 788xcreate_stdio() was checking WARN_ONLY in the wrong field (and tar was
 789passing it in the wrong field, so it worked there).
 790Several people wrestled with the <b>xargs</b> "argument too long" problem.
 791Alessio Balsini fixed memory leaks in loopback_setup() and "wayling"
 792added a missing continue to losetup.
 793Elliott fixed xargs -E, various things in modinfo,
 794added an error check to gzip when using zlib (which copies non-gzip data to
 795the output verbatim for some reason),
 796found an case where dirtree could use
 797uninitialized data and silenced "Invalid argument" warnings (triggered by
 798Android's selinux policies making stat() and readlink() fail),
 799fixed locale support in macOS (both in iconv
 800and in main.c), taught stat to show filesystem time on macOS,
 801fixed a 32 bit truncation in sntp, fixed a memory access one byte outside
 802of its array in patch.c, removed the cpio --trailer option,
 803and widened the pid display fields in ps to
 8046 digits. Rich Felker fixed find.c assuming
 805time_t is a long (y2038 issue on 32 bit).
 806Greg Kaiser fixed a thinko in get_block_device_size().
 807Jarno Mäkipää fixed utf8 support in cut -C, and cp --parents.
 808David Legault complained that dir/.* tells rm to delete dir/.. and we'd do it.
 809JakeSFR pointed out a bug in file's identification of broken symlinks.
 810William Haddon fixed cp to treat a directory with a trailing slash
 811teh same as one without.
 812Denys Nykula fixed rm -i not to prompt for an empty "" argument.
 813SebiderSushi reported that chmod g+s wasn't working.
 814The linux kernel doesn't let an O_PATH fd work with fgetxattr(), so
 815Elliott switched <b>ls</b> to use the path-based functions now (which is racy,
 816it means you can stat() one inode and get the xattrs for a different one,
 817but nobody in kernel land seems to use xattrs much so they're not fully
 818supported by the API, and who's crazy enough to use xattrs for security
 819anyway). Derick Pallas pointed out an xclearenv() bug.
 820Atatsulo (or possibly luolongjuna) did a lot of work on the <b>ping</b>
 821command: pointed out min/range/max were out of order,
 822that we shouldn't print a summary unless we received at least one
 823reply packet, and implemented ttl support.</p>
 824
 825<p><u>Build plumbing</u>:
 826The "make root CROSS=all" build announces each target in the title bar, puts
 827its logs into root/log, and has better trap handling to stop with one
 828ctrl-c.</p>
 829
 830<p>Fixed scripts/single.sh to work when PREFIX has no trailing slash,
 831make silentoldconfig no longer feeds "y" to kconfig (which puts menus in a
 832loop) and instead just feeds in newlines to accept whatever the default is.
 833The non-git version number I keep forgetting to update each release moved
 834from main.c to toys.h.
 835Fixed a couple different errors in mkflags.c (one of which caused ls --color
 836to set all the other flags).</p>
 837
 838<p>Elliott added more macos support and tests, and added
 839fallocate, cp, mktemp, and mv to the macOS defconfig.</p>
 840
 841<p><u>Cleanup</u>:
 842Rob did some cleanup on xargs, ls, md5sum, and sort, tidied up main.c a bit,
 843made cp, base64, dmesg, and free use FLAG macros,
 844and switched fallocate to new style global names.
 845Elliott Hughes switched rfkill from the old byte at a time get_line()
 846to libc getline(), made du use FLAG() macros, and mad chattr use
 847standard toybox argument parsing for -v and -p.
 848Several commands (help, cp) had sub-options removed from menuconfig.
 849Merged realpath into readlink.c, and use xabspath() instead of libc realpath().</p>
 850
 851<p><u>Documentation</u>:
 852New roadmap section about <a href=roadmap.html#packages>other packages</a>
 853that toybox can (eventually) replace.</p>
 854
 855<p>Update roadmap to note that posix-2008 moved to a different URL (content
 856at the old URL undergoes random changes), and link to the IETF RFCs.
 857design.html explains more of the history of environment sizes on links
 858and has a #bits anchor tag.</p>
 859
 860<p>Update the <a href=design.html#bits>LP64 section</a> of design.html to
 861fish the documents out of archive.org now that unix.org is gone,
 862and show the actual size table locally.</p>
 863
 864<p>Elliott improved the help of <b>date</b> and <b>chattr</b>, and
 865made the usage: lines in the posix directory more consistent.</p>
 866
 867<p>Shrank the sed help from 150 lines to 90-ish.</p>
 868
 869<p><u>Tests</u>:
 870New "txpect" performs interactive tests, running through a sequence of
 871writes to a command's stdin and reads from stdout and stderr, when failing it
 872reports the first non-matching step. (This for example lets sh.test check
 873the shell line continuation logic prompts with $PS1 and $PS2 appropriately
 874with various unfinished input lines, and that "echo hello; if" doesn't
 875output hello before prompting for the next line of input.) Added
 876VERBOSE=xpect to print out each read and write successfully performed by txpect.</p>
 877
 878<p>runtest.sh only creates an "input" file when the input argument isn't empty,
 879EVAL doesn't supply -- (you have to provide your own if you want that).</p>
 880
 881<p>Rob added basic <b>stat</b>, <b>patch</b>, and <b>tee</b> tests.
 882Jarno added a bunch of <b>vi</b> tests.
 883Elliott fixed tests for <b>ifconfig</b>, <b>lsattr</b>, <b>chattr</b>,
 884and <b>date</b>, added tests for find, id, xargs, and made
 885the id, iconv, env, file, printf and cat tests work on macOS (and skipped
 886the du tests there). Rob cleaned up chmod tests.</p>
 887
 888<p><u>Sheer pedantry</u>:
 889true and false now have usage: lines (which you have to "help false" to see
 890because they (intentionally!) don't support --help.
 891Renamed get_chunk()/dump_chunk() to read_chunk()/write_chunk() in tail.c
 892Elliott fixed some typos.</p>
 893
 894<a name="18-10-2019" /><a href="#18-10-2019"><hr><h2><b>October 18, 2019</b></h2></a>
 895<blockquote>
 896<p>"In those days spirits were brave, the stakes were high, men were real
 897men, women were real women and small furry creatures from Alpha Centauri
 898were real small furry creatures from Alpha Centauri."
 899</p> <p>- The Hitchhiker's Guide to the Galaxy</p>
 900</blockquote>
 901
 902<p><a href=downloads/toybox-0.8.2.tar.gz>Toybox 0.8.2</a>
 903(<a href=https://github.com/landley/toybox/releases/tag/0.8.2>git commit</a>)
 904is finally out (only two months late).</p>
 905
 906<p>The new commands this time are minor variants of existing ones
 907(<b>dnsdomainname</b> is hostname -d, <b>arch</b> is uname -m), but about
 9081/3 of toysh got implemented in pending, <a href=https://github.com/landley/mkroot>mkroot</a> got merged as a "make root" target (see Build),
 909and Android's <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-August/010845.html>hermetic build</a>
 910work continued resulting in a lot of fixes.</p>
 911
 912<p>The android-specific <b>getprop</b>, <b>setprop</b>, <b>start</b>, and <b>stop</b>
 913commands were removed from toybox because they grew dependencies on android
 914libraries, and since toybox tries to avoid mandatory external dependencies
 915they moved to another package.</p>
 916
 917<p><u>Build</u>:
 918The new "make root" target builds a simple toybox root filesystem,
 919using scripts/mkroot.sh which is a stripped
 920down version of the external <a href=https://github.com/landley/mkroot>mkroot</a>
 921project merged into toybox. (Alas, at the moment it wants sh and route out
 922of pending, which aren't quite load bearing yet.)</p>
 923
 924<p>The new "scripts/mcm-buildall.sh" convenience script builds cross
 925compilers based on gcc+musl multiple targets (arm, x86, mips, m68k, s390,
 926microblaze...)
 927using <a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a>,
 928and a new scripts/cross.sh to easily cross compile for one or more targets
 929using the resulting compilers. (This works with "make root", "make
 930toybox", and other builds understanding the CROSS_COMPILE environment variable.)</p>
 931
 932<p>Setting the environment variable ASAN=1 enables the
 933llvm address sanitizer. (You may need to
 934export ASAN_SYMBOLIZER_PATH on debian to use it, they add the version number to
 935the filename in /usr/bin which confuses llvm.)</p>
 936
 937<p>Patrick Oppenlander fixed make -j (our make is a wrapper around
 938scripts/make.sh which was already doing a parallel build, but if you called
 939make with -j it wouldn't wait for make.sh to finish and the build broke).</b>
 940
 941<p>Lots of fixes for the MacOS build.</p>
 942
 943<p><u>New options</u>:
 944We upgraded <b>tar</b> to extract some older tarballs, <b>file</b> added -b and -s
 945options and can also recognize older tarballs,
 946some xz archives, .otf fonts, perf/simpleperf data files,
 947android boot images, and dtb files. Upgraded <b>file</b> to show the device type
 948for block and char devices, the target for symlinks, and it says _why_ it can't
 949open a file.</p>
 950
 951<p>Several commands ignore some options rather than erroring on them:
 952<b>patch</b> ignores the -f -g# and --no-backup-if-mismatch options
 953(all of which we were already doing by default anyway), and <b>xargs</b>
 954ignores -P# (which requests parallel execution, maybe add it later but
 955single threading works), and Denys Nykula made <b>wget</b> ignore
 956--no-check-ceritificate and <b>gzip</b> ignore -n (again, we weren't doing
 957either anyway) to satisfy pkgsrc.</p>
 958
 959<p>William Haddon taught <b>diff</b> and <b>patch</b> to support special characters in path
 960names (via quoting) and more timestamp formats.
 961Eric Molitor implemented <b>nl</b> -v (with negative and zero starting values),
 962and fixed a MacOS compatibility issue in the build scripts.
 963Tom Cherry added <b>netcat</b> -U (UNIX domain socket support).
 964Denys Nykula taught <b>wget</b> to call ftpget for ftp:// URLs.
 965Ethan Sommer added ln -t and rmdir --ignore-fail-on-non-empty.
 966Elliott Hughes added <b>grep</b> -R, <b>killall</b> -w, tar --mode,
 967 fixed <b>ls</b> -Z and spacing in -lZ and --full-time,
 968added <b>xargs</b> --max-args and removed xargs -I (which we never actually
 969supported) and tweaked the size calculation to more closely match other
 970versions,
 971added xargs -o and made -p read from /dev/tty, added <b>blkid</b> -s, SEC_TYPE for ext3,
 972msdos, and ntfs, f2fs and ntfs LABEL, fixed vfat/ntfs UUID, and switched
 973the endianness for UUID of msdos/ntfs/vfat, simplified the octal mode
 974display in <b>lsof</b>, added <b>pidof</b> -x (thus making the default behavior _not_
 975be -x).</p>
 976
 977<p>When readdir() succeeds but stat() fails, <b>ls</b> now prints "??? ? ? filename"
 978entries instead of "cannot access" errors. (Still trying to make ls / on
 979android work more gracefully with the SELinux weirdness they added in 9.)</p>
 980
 981<p>Added %C to <b>stat</b> and made it show device type in its default output,
 982<b>cmp</b> now works
 983with 1 argument (implicit - as second argument reading from stdin).
 984<b>sed</b> added exit codes to the q command (q123), added Q, and fixed a
 985bad error message with "!".
 986Removed TAIL_SEEK config option (it just always does that now),
 987<b>find</b> grew several new options (%Z, -true, -false, and -newerXY), <b>ln</b> added -T.
 988The argument to <b>mktmp</b> --tmpdir is now optional.</p>
 989
 990<p><u>Bugfixes</u>:
 991The unshare probe bit-rotted so it was impossible to build <b>unshare</b>
 992and <b>nsenter</b> on some distros.
 993Commit 771e94e2a08 broke toybox's ability to say "unknown command" when
 994you pointed a symlink at it that it that didn't correspond to a command,
 995pointing a symlink at another symlink
 996to make toybox run a command under a name it doesn't recognize
 997(ln -s toybox sh; ln -s sh ash) had an off by one error,
 998<b>ifconfig</b> no longer error_exits if the "read mac address" ioctl fails (Android N
 999and later block that for non-root users to prevent device tracking).
1000When the libcrypto library was disabled, md5sum was building commands it
1001hasn't got built-in support for yet (like sha3sum) which acted like redundant
1002copies of md5sum,
1003<b>tar</b> wasn't correctly reading or writing sparse files with a hole at
1004the end, and tar tzf blah.tar.xz was calling xz when it meant xzcat,
1005<b>xargs</b> would spuriously fail on large memory systems (due to a missing
1006"unsigned" in a typecast),
1007<b>basename</b> can now remove suffixes starting with a dash,
1008<b>cp</b> -r unlinks() an existing symlink before trying to
1009recreate the symlink (otherwise cp -r into an existing directory will
1010try to follow the existing symlink and create another symlink at its
1011destination, failing if a file already exists there, which
1012was confusing gentoo's package manager), <b>timeout</b> now produces the
1013right exit code when the timed command intercepts the exit signal,
1014and <b>hostname</b> -d no longer segfaults on machines with no domain name.</p>
1015
1016<p>Denys Nykula fixed the <b>ftpget</b> freeze (it was sending REST instead
1017of RETR), made <b>mv</b> work with a trailing slash in the source,
1018fixed a <b>find</b> segfault,
1019and made <b>env</b> always exec rather than recursing to a builtin
1020(on the theory #!/usr/bin/env is expected to search the $PATH).</p>
1021
1022<p>Elliott improved <b>killall</b>'s handling of long filenames, made <b>kill</b> work
1023when killall5 is disabled (missing FORCE_FLAGS), fixed error reporting in
1024<b>hostname</b>, fixed several things in <b>xargs</b>,
1025made <b>printf</b> \c and \0 octal escapes work, fixed a <b>find</b> -name corner
1026case (find src/*.c -name file.c) and find dangling symlink behavior,
1027made some <b>rm</b> error messages consistent, fixed \x behavior in echo and
1028printf, fixed linestack.c's buffer length for utf8 U+XXXX escape printing,
1029fixed a race in <b>losetup</b>, fixed <b>grep</b> -F with multiple matches on the same
1030line and an -I failure in the existing grep tests, made <b>readlink</b> notice
1031when it has more than one argument, and made various
1032fixes to <b>modinfo</b>.</p>
1033
1034<p>Ethan Sommer reported that <b>stat</b> of an suid/sticky file output 5 digits of
1035octal stat date instead of 4.
1036Rob fixed an unaligned access in <b>grep</b>. Alessio Balsini removed an incorrect
1037null termination in <b>losetup</b>.
1038Jarno Mäkipää fixed <b>hexedit</b>
1039scrolling up and down on older terminals, and fixed <b>ls</b> segfaulting with a broken
1040(non-UTF8) locale (didn't expect wcrtomb to return -1).
1041Andrew Ilijic made ls put 2 spaces between columns like other implementations
1042(which helps when terminals get confused and apply UTF8 combining characters
1043to the space after a filename).
1044</p>
1045
1046<p><u>Library</u>:
1047Added dlist_lpop() to use a dlist as a stack (it removes the last entry
1048instead of the first), and dlist_pop() now works on a dlist_terminate()d list.
1049DIRTREE_STATLESS returns entries we couldn't stat() (with a zeroed ->st
1050field, and ->again |= 2), but filling in the file type from readdir().
1051Coversions between signal names and numbers now include all the Linux signals,
1052not just posix ones. The functions sig_to_num() and num_to_sig() moved into
1053lib/portability.h, because MacOS has different signals.
1054TOYFLAG_MAYFORK allows commands to run in toysh's process without forking,
1055but also makes them accessable from the toybox multiplexer or standalone.
1056A MAYFORK command has to clean up after itself (even in error paths) and
1057can't discard anything we need to keep (such as closing stdout).</p>
1058
1059<p>Elliott did a big xbind/xconnect cleanup (adding xbindany and xconnectany),
1060taught xregcomp that an empty regex matches the whole line
1061(because FreeBSD, and thus MacOS, doesn't already do that), and
1062replaced several uses of get_line() (which does single byte reads)
1063with getline() (which doesn't).</p>
1064
1065<p><u>Plumbing</u>:
1066"make clean" doesn't produce a screenful of unnecessary output now,
1067rm, ln, cp, kill, and netcat use the FLAG() macros now, makedevs uses the new
1068GLOBALS naming format, timeout now uses xwaitpid(), and we fixed an
1069off by one error in xwaitpid().</p>
1070
1071<p>Elliott removed a workaround for old NDK versions from getconf and
1072moved sort off of get_rawline() (which let us remove it).
1073Rob undid some loop unrolling in md5sum/sha1sum (we have libcrypto if you want
1074an assembly optimized version instead of an understandable version).</p>
1075
1076<p>Android ndk-r20 doesn't define the __ANDROID_NDK__ symbol, so we switched
1077to using clang's __has_include() (which is an undefined macro and thus a NOP
1078resolving to false on gcc).</p>
1079
1080<p><u>Pending</u>:
1081Rob did a lot of work on toysh: continuation support works (it knows when to ask
1082for another line of text and when to run what it's got), as do
1083nested if/else/elif statements, and while and for loops (supporting "for i"
1084and "for i in", but not for ((;;)) yet).
1085It parses pipes and redirections but doesn't perform them yet.
1086The next big todo chunk is environment variable resolution.
1087</p>
1088
1089<p>Gavin Howard fixed some bugs in bc, such as not returning error when raising
1090zero to a negative power.
1091Jarno Mäkipää continues to extend vi, adding yank and push, support for tabs,
1092rewriting delete, changing the drawing code, etc. Elliott also added several
1093fixes and features to vi,
1094fixed host to cope with large DNS responses, switched crontab to
1095use getline(), did work on man, fixed fixed several things in dd,
1096implemented diff --strip-trailing-cr and made diff not need /tmp.
1097Mike Bennett added extended attribute support to ip.c's route change.</p>
1098
1099<p><u>Test suite</u>: The killall and pgrep tests were having occasional
1100spurious failures due to a race condition (the test script would fork()
1101to call sleep and for just a moment there were two instances of the test
1102script, reporting an extra process if killall or pgrep ran then).
1103
1104<p>Elliott Hughes did a lot of work on the test suite: add kill -l tests,
1105make losetup.test notice (and skip) if loopback
1106devices area already in use, blkid.test can TEST_HOST e2fsprogs blkid now,
1107modinfo.test notices (and skips) if there's no /proc/modules or modules
1108directory, ifconfig.test skips if "dummy0 up" fails and disabled the
1109pointtopoint tests entirely (the kernel never supported it?),
1110truncate.test is less
1111confused by SELinux, hostname.test can run some tests as non-root,
1112addressed a race condition in killall.test on slow systems,
1113made chgrp.test, chown.test, and tar.test handle an empty /etc/passwd or /etc/group
1114(we were checking whether they were _there_ but some android devics have
1115empty ones),
1116fixed a couple things in env.test, extended killall.test to explicitly cover
1117long and short names (and not assume the shell is /bin/sh), taught split.test not to rely on bash process substitution,
1118added missing TZ=utc to touch.test,  fixed cp.test to pass for root
1119or with a restrictive umask, and fixed the FAILCOUNT arithmetic to use more portable
1120$(()). Elliott also made some tests more flexible (find.test can handle any
1121error message about symlink loops, etc), and disabled others (such as du)
1122when SELinux is enabled.</p>
1123
1124<p>Rob taught testcmd not to use the shell builtin for TEST_HOST unless there
1125isn't one in the $PATH, work if $PWD has a space in it,
1126and make kill.test stop testing the shell builtin
1127insead of the command, quieted tests that were producing output
1128other than their PASS/FAIL line, and made NOSPACE tests ignore the
1129presence/absence of leading/trailing whitespace as well as differences
1130in the amount of space.</p>
1131
1132<p><u>Documentation</u>:
1133The <a href=roadmap.html>roadmap</a> got freshened up a bit, with updates
1134for android from Elliott.
1135In <a href=design.html>the design page</a> the Features section starts with
11364 new paragraphs about the scope of toybox, and a new License section talks
1137about 0BSD.
1138The "./toybox --help" output now explains how to do install_flat by hand.</p>
1139
1140<p><u>Sheer pedantry</u>:
1141Elliott fixed a double close in loopfiles_lines() so strace doesn't
1142have an EBADF in it, and added a missing "static" in lib/net.c.</p>
1143
1144<a name="21-05-2019" /><a href="#21-05-2019"><hr><h2><b>May 21, 2019</b></h2></a>
1145<blockquote>
1146<p>"I like the cover," he said. "'Don't Panic.' It's the first helpful or intelligible thing anybody's said to me all day."</p> <p>- The Hitchhiker's Guide to the Galaxy</p>
1147</blockquote>
1148
1149<p><a href=downloads/toybox-0.8.1.tar.gz>Toybox 0.8.1</a>
1150(<a href=https://github.com/landley/toybox/releases/tag/0.8.1>git commit</a>)
1151is out.</p>
1152
1153<p>Android is now using toybox as part of a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-November/009824.html>"hermetic" build</a>
1154(meaning Android is building under toybox, which is halfway to Android
1155building under Android). See Android's <a href=https://android.googlesource.com/platform/prebuilts/build-tools/+log>build-tools</a> and <a href=https://android.googlesource.com/platform/build/soong/+log>soong</a> repositories to follow along with development, or check the list for the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-May/010485.html>current status</a>.</p>
1156
1157<p><u>New commands</u>: Isaac Dunham added <b>mcookie</b>, which is used by
1158X11. Elliott Hughes added <b>devmem</b>, which replaces toolbox's "r".</p>
1159
1160<p><u>Promoted</u>: <b>tar</b>, <b>gunzip</b>, and <b>zcat</b> were cleaned
1161up and moved out of pending.
1162Tar is a near-rewrite with support for --owner --group and --sparse,
1163capable of autodetecting "wget blah.tar.bz2 -O - | tar xv", a --restrict option
1164that only allows files to extract into a single subdirectory, and so on.</p>
1165
1166<p><u>Added to pending</u>:
1167Jarno Mäkipää contributed a new <b>vi</b> implementation. Makepost added
1168<b>man</b>.</p>
1169
1170<p><u>Updates</u>: Elliott added <b>find</b> -wholename/-iwholename, -printf,
1171and -context, <b>grep</b> --exclude-dir, and <b>echo</b> -E.
1172Rob added <b>blkid</b> -U and -L, <b>mount</b> now has UUID= support via blkid -U,
1173you can now specify TZ="blah" in the <b>date</b> -d input string (although
1174why the double quotes are required is an open question), and <b>sed</b> now
1175understands ,+N relative ranges. Makepost added <b>rm</b> -v.
1176The human_readable fields in <b>ps</b>/<b>top</b>/<b>iotop</b> now use all
1177available space to show more precision, <b>iotop</b> displays swap bandwidth
1178properly (it was showing a signed field as unsigned), and
1179<b>touch</b> -t and -d autodetect the time format.</p>
1180
1181<p><u>Cleanups</u>: In pending Rob did some bc cleanup,
1182Colin Davidson did some arp cleanup, 
1183Uros Prestor fixed route argument parsing, and
1184in bc Daniel Rosenberg fixed bc_vec_concat()'s allocation and
1185Gavin Howard fixed an overflow bug in bc_num_ulong().</p>
1186
1187<p>Elliott made scan_key() detect cursor keys and such with several more
1188terminal types, made more better about directories and non-tty output,
1189cleaned up inconsistent uses of ARRAY_LEN(), taught file
1190to recognize a c-sky ELF machine, and fixed timezone and daylight
1191savings time issues in date and its tests.
1192Rob simplified netcat and ping using new lib/net.c infrastructure.
1193When run as non-PID 1 (for a chroot) oneit's exit code now indicates
1194which signal it's exiting due to.
1195Hadrian Węgrzynowski #defined constants for mkpathat.
1196Date now uses the current time for unspecified fields (instead of zeroing
1197them), and should set the weekday properly.
1198Sort's -k2.3,4.5 syntax is now documented and behaves more closely to
1199other implementations (silly as that behavior is).
1200Significant speedup to sed s/a/b/g on a megabyte long line (it was
1201O(n^2) and now it's quite snappy).</p>
1202
1203<p>Several cases of warning about stdout failing to write were silenced
1204because things like piping the output to head means stdout is a special
1205case: its output is often discarded and that's fine. And xprintf()
1206and friends only checks stdout for errors so it can exit early ("yes | head"
1207should not spin endlessly) without actually flushing anything.
1208There's an ongoing design debate about line buffering stdin and stdout
1209that'll have to be resolved next dev cycle.</p>
1210
1211<p><u>Bugs</u>:
1212Elliott sent in a lot of fixes for building AOSP with toybox:
1213support for more date formats, sed substitution of empty capture groups
1214and an endless loop with -EPIPE, fixed find -path/-ipath, made loggit()
1215only feed LOG_AUTH to syslog when we have a tty, gave grep a bunch
1216of --longname synonyms for short options, added getfattr --only-values,
1217and timeout --foreground --preserve-status and --signal.</p>
1218
1219<p>Top got a lot of tweaks: it no longer collates thread's CPU usage into the
1220parent thread (in -H mode), hides the cursor in interactive mode, updates the
1221display properly in batch mode, and accepts ENTER
1222as a synonym for SPACE when refreshing the display. The header lines
1223now abbreviate long numbers with units (via human_readable()),
1224more reliably spans the terminal width, and says "Thread" instead
1225of "Task" when it's showing threads. The task/thread count now adds up
1226(understanding more kernel states and treating unknown ones as "sleeping").
1227Top now treats -o "" as an error (previously it displayed an empty screen).</p>
1228
1229<p>William Djupström pointed out login needs to chown() the tty, and made it
1230clear the environment when -p is absent.
1231Vidar Holen fixed inverted find -exec exit status.
1232Rob made chgrp() stop relying on libc's printf("%s", NULL) printing "(null)"
1233because a <a href=https://github.com/varnishcache/varnish-cache/issues/2879>newly added gcc bug</a> turns that into a build break, and
1234fixed an outright embarassing xstrndup() bug.
1235Liwugang fixed a segfault when stat's format string had a trailing %.
1236A missing else in grep meant -x was sometimes ignored.</p>
1237
1238<p><u>Infrastructure</u>:
1239in the build scripts/portability.sh now tests that the compiler it's trying
1240to use exists up front. We no longer link against librt because on glibc that's
1241pulling in libpthread for some unknown reason.</p>
1242
1243<p>TOYFLAG_ARGFAIL() lets commands return custom exit values when option parsing
1244fails (because AOSP build scripts cared), so timeout --unknown returns 125
1245and so on: chroot, diff, cmp, env, grep, nohup, and sort also use it now with
1246various values.</p>
1247
1248<p>New FLAG() macros: FLAG(x) expands to (toys.optflags&FLAG_x) and returns
1249zero if the command line argument's bit isn't set and nonzero if it is
1250(meaning -x was or wasn't encountered on the command line this time by
1251lib/args.c). Lots of commands got converted to use them.</p>
1252
1253<p>New lib/env.c with reset_env(), xsetenv(), and xunsetenv() which frees
1254the old environment variable when it wasn't inherited from exec(). This
1255lets long-running loops reset environment variables without memory leaks.</p>
1256
1257<p>New xnotify() plumbing in lib/portability.c with Linux and Mac versions
1258(which should also work on BSD).</p>
1259
1260<p>New xparsedate() and xvali_date() functions allow date, tar --mtime, and
1261touch to understand the same date formats,
1262xpopen_both() now knows how to use existing filehandles for the child's
1263stdin/stdout, and no longer redirects stderr (which hides errors),
1264new nanomove(), nanodiff(), union socksaddr, xrecvwait(),
1265and human_readable_long() to show more than 3 digits of output.</p>
1266
1267<p>Taught xgetaddrinfo() to return a wildcard address for a NULL host,
1268xconnbind() sets SO_REUSEADDR by default,
1269fileunderdir() returns the abspath to the file now (since we had to work
1270it out anyway), and
1271xabspath() uses O_PATH now (because Android's SELinux rules got more aggressive).</p>
1272
1273<p>Several new xsendfile() variants, including sendfile_len() to copy
1274a specific amount of data and xsendfile_pad() to append zeroes to the
1275output when there isn't enough input.</p>
1276
1277<p>regexec0() is now using libc's REG_STARTEND supported by every libc out
1278there _except_ musl. (It originated in BSD and was adopted by glibc, bionic,
1279and uClibc 15 years ago, and is also supported in macosx. Musl's maintainer
1280<a href=https://www.openwall.com/lists/musl/2013/01/15/26>turned down</a>
1281the feature request as an invalid use case he didn't think they should
1282be trying to do.)</p>
1283
1284<p><u>Docs</u>:
1285the <a href=license.html>license page</a> now has an SPDX identifier, and links to more
1286history/context. The actual LICENSE file had its title line removed
1287so <a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-463746027>github could recognize it</a> as 0BSD. (We're still waiting for
1288"<a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-464380789>a while</a>" to pass before github actually does, though.)</p>
1289
1290<p>A new www/doc/mount.txt file describes how mount works under the covers.</p>
1291
1292<p>Rob consistently misspelled "canonical" and "millisecond", mountpoint
1293has a synopsis now,
1294Kevin van der Kamp fixed a typo in netstat's help text, and Elliott
1295Hughes fixed typos everywhere and made the tense, capitalization, and
1296punctuation of help text more consistent.</p>
1297
1298<p>New paragraph in the code style part of design.html explaining that
1299"char* a, b;" is not how C works, and since you wind up with "char *a, b;"
1300anyway you might as well say that.</p>
1301
1302<p>Given dreamhost's tendency to repeatedly delete the mailing list
1303archive, the nav bar on the left links to a backup web archive now too.</p>
1304
1305<p>The FAQ now says why we do time based releases.</p>
1306
1307<p><u>Testing</u>: Added VERBOSE=nopass to not show passing tests.
1308The test suite now does an rm -rf on testdir between each command so
1309debris files don't accumulate.
1310New skipnot function skips the next test if a command line fails, and
1311toyonly function only runs command when we're testing the toybox version
1312of command. (This isn't the same as TEST_HOST, the host version could be
1313toybox.)
1314The find -newer test calls "sleep .1" to avoid intermittently failing with an
1315identical timestamp.
1316Elliott added the start of a file.test rogues gallery.
1317No longer call lsattr in "make tests" because the behavior differs by
1318filesystem and it needs fixing somehow.</p>
1319
1320<a name="08-02-2019" /><a href="#08-02-2019"><hr><h2><b>February 8, 2019</b></h2></a>
1321<blockquote>
1322<p>"Most readers get as far as the Future Semiconditionally Modified
1323Subinverted Plagal Past Subjunctive Intentional before giving up; and
1324in fact in later editions of the book all pages beyond this point have
1325been left blank to save on printing costs."</p>
1326<p>- The Hitchhiker's Guide to the Galaxy</p>
1327</blockquote>
1328
1329<p><a href=downloads/toybox-0.8.0.tar.gz>Toybox 0.8.0</a>
1330(<a href=https://github.com/landley/toybox/releases/tag/0.8.0>git commit</a>)
1331is out.</p>
1332
1333<p>Toybox now builds on MacOS and FreeBSD, thanks to the efforts of Elliott
1334Hughes and Ed Maste respectively. Use the "make macos_defconfig" and "make
1335freebsd_defconfig" targets to enable the set of commands that compile on
1336each so far.</p>
1337
1338<p><u>New commands</u>: Added an <b>sntp</b> client/server (RFC 4330 Simple Network Time
1339Protocol, a compatible subset of ntp). The <b>test</b> command was rewritten
1340and promoted out of pending.</p>
1341
1342<p><u>New options</u>: <b>grep</b> now has --color, supports embedded NUL bytes in its input,
1343recognizes binary files, and passes all of grep.test (in case you
1344needed the middle numerical field of -bB, etc).
1345Josh Gao added ipv6 and UDP support to <b>netcat</b>.
1346Volodymyr Medvid reported that install -d doesn't honor -o or -g.
1347Elliott Hughes did a lot of work to support hermetic Android/AOSP builds:
1348<b>mkdir</b> accepts both --parent and --parents as synonyms for -p, <b>touch</b> ignores -f,
1349<b>basename</b> added -s to remove a trailing suffix, <b>dirname</b> now supports multiple
1350arguments, <b>cmp</b> accepts --quiet and --silent as synonyms for -s, <b>hostname</b>
1351added -sfd, <b>head</b> added --bytes as a synonym for -c and --lines as a synonym
1352for -n, <b>mktemp</b> added -t and fixed -u, <b>sed</b> added -z and -iEXT to keep backup files,
1353<b>md5sum</b> and sha1sum added --status and --check as synonyms -s and -c,
1354<b>readlink</b> added --canonicalize as a synonym for -f, <b>sort</b> grew -V,
1355<b>patch</b> added -s its synonym --quiet, <b>stat</b> added --format as
1356a synonym for -c, <b>xargs</b> added -p -t -r,
1357Eduardas Meile asked
1358that <b>umount</b> ignore -c. Reverend Homer added a small optimization to file.c,
1359and Elliott taught <b>file</b> to recognize riscv ELF binaries. Peter Collingbourne
1360taught <b>ls</b> -t to use the nanoseconds field.
1361<b>patch</b> has better support for patching a file with a tab in the name.</p>
1362
1363<p><u>Bugfixes</u>:
1364<b>cp</b> --preserve was segfaulting when you didn't specify _what_ to
1365preserve (it now correctly defaults to "mot") and didn't get the permissions
1366right when copying a symlink's contents as a regular file,
1367<b>sort</b> -x didn't work when
1368attached to a key, <b>host</b> didn't allocate a big enough buffer for worst
1369case ipv6 address size, <b>sed</b> needed a ; between b and } when other
1370implementations don't (we're already well past what posix says but a script
1371out in the field broke...),
1372and several fixes to <b>hostname</b> in a container, <b>ps</b>/<b>top</b> were
1373misreading the VIRT and SHM fields.</p>
1374
1375<p><u>Pending</u>: Gavin Howard updated <b>bc</b> and added more tests,
1376Marius Adaskevicius pointed out <b>mdev</b> -s should follow symlinks,
1377Yangchun Fu reported a <b>dhcp</b> checksum bug, <b>modprobe</b> needed errno reset
1378to avoid reporting spurious errors in verbose mode and no longer
1379exits with status 1 if it can't find /etc/modprobe.conf, the <b>more</b> command
1380wasn't always flushing stdout when it exited.</p>
1381
1382<p><u>Build</u>:
1383<b>chrt</b> no longer #warns about the musl sched_get_priority_min() bug, but
1384instead works around it.
1385We were mixing setjmp/siglongjmp (harmless in some contexts but it kind
1386of annoyed FreeBSD), and
1387make install_airlock now adds sha256sum (because the Linux 4.20 build now
1388needs that for the s390x target).</p>
1389
1390<p><u>Coding style</u>:
1391Rob converted the rest of the option GLOBALS() to the new single letter
1392coding style, and the new FLAG(x) macro is a slightly tidier way to say
1393"toys.optflags&FLAG_x".
1394Removed CFG_SORT_BIG (the sort command always
1395has the full functionality now. The general future direction or toybox
1396is to either have a command or not have it; multiple versions of the
1397same command aren't worth the complexity in testing, documentation,
1398or system adminstration).</p>
1399
1400<p><u>Library code</u>:
1401The mkflags plumbing can now support arbitrary punctuation as option
1402names via an escape syntax (because mkfs.vfat specifies an offset with -@),
1403and lib/args.c now only sets FLAGS_NODASH when the first argument
1404didn't have a dash (allowing "ps ax" and "ps -ax" to behave differently).</p>
1405
1406<p>Added an xrename() function,
1407xchdir() has better error reporting, xconnect()/xbind() had their
1408implementations merged, xsendto() moved from ping to lib, xpoll() now
1409measures elapsed time and only waits for the remaining period when restarted,
1410and Eduardas Meile converted several error_exit() to error_exit_raw() (because
1411yocto 2.6 buils with -Werror=format-security by default).</p>
1412
1413<p>There's a scripts/portability.sh now which lets the build use gsed and gmake
1414(when available) in more places to avoid broken host versions on MacOS and FreeBSD.
1415The syntax of the "noreturn" attribute
1416changed slightly, some calls to strnstr() were replaced with strcasestr(),
1417we work around MacOS' lack of features.h, portability.h can now do Apple's
1418endianness macros, and so on.</p>
1419
1420<p>Following the <a href=faq.html#support_horizon>seven year rule</a>
1421Elliott removed support or glibc 2.10, and also
1422reformatted a lot of --help text for consistency (removing trailing
1423periods and such).</p>
1424
1425<p><u>Test suite</u>:
1426The test suite now has its own version of "pending", specifically "make tests"
1427skips test files without the executable bit set (unless you export $TEST_ALL).
1428This eliminates "expected failures" (I.E. files with unresolved todo items)
1429from the global regression test. Additional tests can be added to the global
1430regression test with chmod +x as the todo items they document get resolved.</p>
1431
1432<p>Several places the test suite got fluffed out, including guards to skip
1433root-only tests when run as a normal user. The testcmd function prints the
1434short name instead of the full path to the command.
1435The tests for the test command
1436("test.test") no longer test the shell builtin but correctly test the
1437toybox command. (Now say that ten times fast.)</p>
1438
1439<p>Bash version skew required replacing "continue" with "return" to stop
1440tests early in bash 4.4 (for VERBOSE=fail).</p>
1441
1442
1443<a name="31-10-2018" /><a href="#31-10-2018"><hr><h2><b>October 31, 2018</b></h2></a>
1444<blockquote><p>
1445"In the beginning the Universe was created.
1446This has made a lot of people very angry and been widely regarded as a bad move."</p><p>- The Hitchhiker's Guide to the Galaxy
1447</p>
1448</blockquote>
1449
1450<p><a href=downloads/toybox-0.7.8.tar.gz>Toybox 0.7.8</a>
1451(<a href=https://github.com/landley/toybox/releases/tag/0.7.8>git commit</a>)
1452is out (a month late).</p>
1453
1454<p>Toybox <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-October/009769.html>now builds</a> with the Android NDK (r18 release), almost out of the box.
1455You have to add an "llvm-cc" symlink to "clang", then use CROSS_COMPILE=llvm-
1456with the appropriate $PATH (or absolute path in CROSS_COMPILE) for the
1457target you want to build for. (To run it on a non-android host, you probably
1458want "LDFLAGS=--static" too.)</p>
1459
1460<p><u>New commands</u>: <b>i2cdetect</b>, <b>i2cdump</b>, <b>i2cget</b>, and <b>i2cset</b>
1461courtesy of Elliott/Android. The <b>watch</b> command got a complete (tty-aware)
1462rewrite, and was promoted out of pending. The <b>prlimit</b> command is back
1463(a configuration bug was always disabling it), and <b>ascii</b> now defaults y
1464in defconfig (an oversight).</p>
1465
1466<p><u>New options</u>: 
1467The <b>ifconfig</b> command added -S (short view) giving one line of
1468information per interface, in "name ipv4/mask macaddr [ipv6/mask@type...]"
1469format. <b>grep</b> grew binary file detection, with -I and -a options to
1470ignore them or force treating them as ascii (really utf8).
1471Zach Van Rijn asked for <b>find -empty</b>.
1472In the <b>file</b> command, Elliott improved java
1473.class support and added .wav, .bmp, and android .dex.
1474Elliott also added "getconf -a" and a lot of missing getconf symbols the
1475AOSP build uses, including support for pathconf(3) symbols requiring two
1476arguments. This means getconf's arguments work more like ubuntu's
1477now (one or two arguments, not iterating through the supplied list and
1478processing each like last release), and -l grew section identifiers (so
1479the "getconf -l" output is no longer a directly consumable list of symbols
1480it takes).
1481Command line options specifying durations (like top -d or ping -i) can
1482now use fractions and units, like "1.5" and ".1m". See "toybox --help" for
1483details.
1484Toybox now dereferences one layer of symlinks if it doesn't recognize the
1485name it's called under (so if you "ln -s $(which sleep) blah" and then
1486"./blah 30", it should figure out you mean sleep 30).</p>
1487
1488<p><u>Pending</u>:
1489Gavin Howard contributed a large new <b>bc</b> implementation to pending,
1490which covers everything the kernel build needs (and more) but is going to
1491take a while to review.
1492In <b>diff</b> the android guys added --color and timestamps on the
1493+++/--- lines. In <b>mdev</b> Faustas Azuolas Bagdonas added support
1494for mdev.conf's fourth field. Reverend Homer suggested an error message
1495improvement in <b>wget</b>, Kevin Spiteri did a cleanup pass on test.c,
1496and there was a cleanup pass on the passwd command.</p>
1497
1498<p><u>Bugfixes</u>:
1499Elliott responded to a bug in <b>getconf</b>'s sed-based header generation by
1500replacing it with a more conventional array of #defined symbols, with #ifdefs
1501for missing symbols on the three libc implementations (glibc, bionic, musl)
1502we've tested so far. (This requires portability.h work to add new supported
1503build environments, but at least it works on bionic now.)
1504The <b>file</b> command now fails gracefully when reading pathologically
1505broken ELF files (the tests for integer overflow missed one), and
1506<b>date</b> no longer shows a meaningless errno when it can't parse the
1507date format.
1508Several fixes in <b>ping</b> (-w and -W didn't work, -c kept sending packets
1509while waiting for replies, and when no reply packets were received the summary
1510data was printed twice).
1511Elliott fixed <b>wc</b> column widths (traditional reality doesn't match
1512posix), fixed <b>modprobe</b>'s error handling (reported by Wen Xie), and
1513also fixed the units on blockdev --blkraget (reported by Martijn Coenen).
1514Several small fixes to the <b>stat</b> command's output (help text, quoting
1515style, leading zeroes, switch -f from %t to %T), <b>oneit</b> no longer
1516reboots the system when it's not called as PID 1.
1517Last release's commit letting $STRIP be redefined had a typo.
1518Nick Kralevich added an istty() check before calling TIOCGWINSZ (which
1519triggered Android's strict ioctl filtering on non-filesystem fifos).
1520An unstripped build (where the "strip" command fails for a given
1521CROSS_COMPILE) will now overwrite the (read-only) output file instead of
1522failing a rebuild. The dirtree plumbing no longer prints a "No" on front
1523of the filename in error messages (such as "permission denied").</p>
1524
1525<p><u>Docmentation</u>
1526Elliott updated the roadmap with Android AOSP status, towards turning that
1527into a hermetic build (hopefully with an eventual self-hosting option, work
1528is <a href=b33d37d6f735>already underway</a> on the airlock step).
1529Rob added a little more context for LSB and described the move from Aboriginal
1530Linux to mkroot.
1531The <b>ps</b> help text changed: now "ps -o help" shows the fields and the normal
1532--help is much more manageable. Ifconfig's --help text also got tidied up.
1533Our LICENSE file has the SPDX identifier (0BSD) now, and the note about
1534the kconfig/ subdirectory still having (build only, non-shipping) GPL code
1535in it moved into the kconfig subdirectory.</p>
1536
1537<p><u>Coding style change:</u>
1538GLOBALS() now uses the argument letter as the variable
1539name for variables automatically set by command line arguments via lib/args.c,
1540and no longer requires they be one per line.
1541Removed use of the gcc
1542\e extension in string constants, replacing it with \033.</p>
1543
1544<p><u>Library code:</u>
1545lib/interestingtimes.c was renamed to lib/tty.c.
1546The deflate code from toys/pending/compress.c moved to lib/deflate.c,
1547which gunzip.c now uses when it's not configured to use zlib. (This reduces
1548the bootstrap dependencies for the "airlock" step of a hermetic build.)
1549The compression side remains a todo item.
1550Added "%" to lib/args.c reading seconds (including fractions/units) into a
1551long milliseconds argument. Redid xparsetime() to not need floating point,
1552added xparsemillitime() for milliseconds common case, and
1553parsetime() now treats leading garbage as an error instead of returning zero.
1554The bufgetpwuid()/pufgetgrgid() functons can now handle more than 4k of group
1555data. The set_terminal() function can now set serial speed, and microcom.c
1556uses it. Added xsignal_flags(), and more consistently use xsignal() (which wraps
1557sigaction()) instead of signal().
1558A new xgetrandom() function calls the new getrandom() system call, falling
1559back to /dev/urandom on older kernel/libc (made to actually _work_ by
1560Patrick Oppenlander).</p>
1561
1562<p><u>Build infrastructure:</u>
1563Updated dependency detection so "make distclean defconfig toybox" now works
1564all on the same command line. (This is nice when $CROSS_COMPILE changes).
1565scripts/config2help.c no longer includes parts of lib/ and just copies what
1566it needs into itself, improving build portability.
1567The build now checks the specified cross compiler exists, hopefully providing
1568a more informative error message when it doesn't. Similarly, when .config is
1569missing the error message suggests running make defconfig.</p>
1570
1571<a name="23-06-2018" /><a href="#23-06-2018"><hr><h2><b>June 23, 2018</b></h2></a>
1572<blockquote>
1573<p>There is a theory which states that if ever anyone discovers exactly what
1574the Universe is for and why it is here, it will instantly disappear and be
1575replaced by something even more bizarre and inexplicable. There is another
1576theory which states that this has already happened.</p>
1577<p> - The Hitchhiker's Guide to the Galaxy</p>
1578</blockquote>
1579
1580<p><a href=downloads/toybox-0.7.7.tar.gz>Toybox 0.7.7</a>
1581(<a href=https://github.com/landley/toybox/releases/tag/0.7.7>git commit</a>)
1582is out.</p>
1583
1584<p><u>New commands</u>: Rob Landley added <b>getconf</b>, Elliott Hughes added
1585<b>uuidgen</b>, and <b>ping</b> and <b>fmt</b> were promoted out of pending.</p>
1586
1587<p><u>New options</u>: The <b>cp</b> command added the --parents option,
1588<b>readlink</b> now has -m to show where a missing path would be,
1589<b>netstat</b> got some updates (better nommu support, it can use /etc/services
1590names, more command line options are bounds checked),
1591Ng Zhi An added --getra and --setra (get/set readahead) to <b>blockdev</b>,
1592Elliott Hughes added <b>xxd</b> -o and made <b>top</b> -d understand fractional
1593sections (because the Linux Test Project uses that), and
1594Minghui Liu added <b>watch</b> -b.</p>
1595
1596<p><u>Bugfixes</u>: Several fixes to <b>vmstat</b>: Elliott Hughes made it work on a
159748-core machine
1598and fixed a glitch with -n, Haroon Maqsood pointed out "vmstat 1 1" didn't
1599stop and "vmstat -n" didn't print the first theader line.
1600Minghui Liu fixed a bug where <b>cp</b> -p didn't preserve
1601timestamps, and pointed out the b, c, and d suffixes on numbers weren't
1602working. Ryan Pritchard pointed out that <b>du</b> -d0 should act like du -s.
1603Filip Perich made an RFC2347 OACK compliance fix to <b>tftp</b> in pending.
1604Zach van Rijn fixed a bug in <b>xxd</b> causing incorrect translation of
1605uppercase characters.
1606Elliott fixed several things in <b>top</b> (removed spurious '\r' characters from -b
1607output, removed interactive flicker, made running processes bold), and
1608pushed Rob to make <b>file</b> work better recognizing things on stdin
1609("cat /bin/ls | file -" still won't work but "file - < /bin/ls" should).
1610Rob fixed a bug in <b>netstat</b> on 64 bit big endian systems,
1611and fixed <b>cut</b> -DF
1612(a posix compliance fix broke its ability to act as a decent awk replacment
1613because outputting all delimiterless lines isn't what you want there, -D
1614now disables that too).</p>
1615</u></p>
1616
1617<p><u>Documentation</u>: Rob added a buildroot section to the roadmap with the
1618prequisites that needs to run, and reformatted more of the ps --help text
1619to two columns.</p>
1620
1621<p><u>Library</u>:
1622FLAGS_NODASH is now set in toys.optargs when an optstring starting
1623with & has no dash in its first argument. (This lets "ps -ax" and "ps ax"
1624behave differently.) Factored out xtestfile() into lib/.
1625The comma-separated-list parsing infrastructure moved to lib/commas.c.
1626Added mkpath() for the common case of mkpathat() and #defined MKPATHAT_*
1627constants for the uncommon cases. Elliott added better error checking
1628to xparsetime() and fixed a bug in names_to_pid() (used in <b>pidof</b>
1629and <b>killall</b> and such).
1630Rob inlined the old toys/e2fs.h into pending's mke2fs,
1631and removed a rogue semicolon from the WOULD_EXIT() macro that screwed
1632up if() statement levels. The do_lines() semantics changed adding a
1633callback(0,0) at the end of each file.</p>
1634
1635<p><u>Tests</u>:
1636The test_* commands under toys/example were all renamed demo_* so
1637they don't share a namespace with running a standalone command
1638through the test suite (ala "make test_sed"), then
1639demo_human_readable became demo_number so it can also test atolx()
1640suffixes. The readlink.test was getting confused by a
1641behavior difference between toybox and ubuntu (Ubuntu's ln -sf wouldn't
1642replace a symlink that pointed to itself, toybox's would), now there's
1643an explicit rm so it works on ubuntu too. The test suite plumbing's
1644error message for a test with the wrong number of arguments now
1645correctly identifies which test it's complaining about, and it can now
1646run a test on a command name beginning with "-" (none yet but it
1647comes up in toysh).</p>
1648
1649<p><u>Building</u>: You can now override strip with $STRIP. The set of
1650"make install_airlock" commands got tweaked for the 4.17 kernel (which
1651needs bison and flex because the new kconfig plumbing doesn't have _shipped
1652versions of those generated files like the old kconfig did), and for
1653the new release (remove ping, toybox provides it now).
1654$LDFLAGS now applies to library probing (the android NDK provides
1655dynamic libraries without corresponding --static versions).
1656Elliott took a CFG_TOYBOX_DEBUG check out of an error_exit() to make
1657clang stop complaining about an error that should never happen.</p>
1658
1659<a name="24-02-2018" /><a href="#24-02-2018"><hr><h2><b>February 24, 2018</b></h2></a>
1660<blockquote><p>
1661"Arthur," said Ford.<br />
1662"Hello? Yes?" said Arthur.<br />
1663"Just believe everything I tell you, and it will all be very, very simple."<br />
1664"Ah, well, I'm not sure I believe that."<br />
1665</p>
1666<p> - The Hitchhiker's Guide to the Galaxy</p>
1667</blockquote>
1668
1669<p><a href=downloads/toybox-0.7.6.tar.gz>Toybox 0.7.6</a>
1670(<a href=https://github.com/landley/toybox/releases/tag/0.7.6>git commit</a>)
1671is out, a month and change late.</p>
1672
1673<p>This release adds a <b>crc32</b> command and
1674cleans up and promotes <b>iconv</b>, <b>logger</b>, and <b>setfattr</b>.
1675Pending added <b>stty</b> and <b>fmt</b>, the other half of <b>ping</b>,
1676and got a lot of cleanup on <b>gzip</b>/<b>gunzip</b>/<b>zcat</b>.</p>
1677
1678<p>The <b>cut</b> command got a complete rewrite adding support for multiple
1679ranges, utf8/unicode characters, and new -CDFO options. Use -C to measure
1680columns rather than characters (for double width chars), and "cut -DF 3-5"
1681selects words (like "awk '{print $3 $4 $5}'").</p>
1682
1683<p>This release adds <b>top -m</b> and <b>iotop -H</b> (and fixed top width truncating for narrow terminal windows),
1684added <b>iconv -c</b>, fixed <b>ps -T 1234</b> to show threads belonging to
1685that PID, improved <b>file</b>'s executable identification (properly recognizing
1686both endiannesses and adding sh/frv fdpic, bpf, and new microblaze
1687identifiers), and <b>blkid</b> now only shows LABEL= when it isn't blank.</p>
1688
1689<p>Running ./configure now does "make defconfig", help_exit() now shows the
1690"See %s --help" message on the same line, and several web pages
1691(about, code, roadmap, cleanup) got updates.</p>
1692
1693<p>Elliott and the Android devs added <b>df -i</b>, <b>uptime -p</b>,
1694<b>time -v</b>, and <b>xxd -i -ri</b>,
1695fixed <b>id -G</b> to only output the group IDs, fixed the <b>xargs -0 -n</b>
1696combination, made <b>find -exec +</b> obey ARG_MAX (just like xargs), made
1697<b>killall</b> kill scripts too, fixed an off by one error in <b>pidof</b>
1698that could cause false positive matches in 15 character names, added
1699gzip/gunzip/zcat tests, and taught <b>file</b> to identify more stuff
1700(ogg, TrueType font, LLVM bitcode, PEM certs, and PE executables), and
1701for some reason needed TOYBOX_VENDOR in the version stuff.
1702Android <b>getprop</b>/<b>setprop</b> now allows ro. properties to have
1703arbitrary lengths, <b>losetup</b> can now find loop devices under /dev/block (where android puts
1704them for historical reasons), and Andreas Gampe pointed out a thinko in
1705<b>top</b> that triggered llvm's address sanitizer checks.</p>
1706
1707<p>Gael PORTAY asked for mdev to work when Linux has the
1708block layer configured out, Patrick Oppenlander pointed out a problem
1709with make bloatcheck's build dependencies, and
1710xuphung on github fixed config2help building on MacOS X.</p>
1711
1712<p>Commands with locale support now setlocale(LC_TYPE, "C.UTF-8") for more
1713targeted locale support (enable character parsing with
1714utf8 support, use "C" semantics otherwise). The toys/example/test_*.c
1715commands got renamed demo_*.c.</p>
1716
1717<p>In lib.c the millitime() function returns current unix time in milliseconds
1718(which as a 64 bit number is good for + or - 292 million years from
1719midnight Jan 1 1970), and the old xgetty() function became tty_fd() returning
1720-1 instead of erroring out.</p>
1721
1722<p>The "make help" text moved from the Makefile to scripts/help.txt,
1723ls.c now lists some of its deviations from posix, and work is underway
1724to make ps.c more intelligible (and figure out how to break it up into
1725multiple files).</p>
1726
1727<a name="12-10-2017" /><a href="#12-10-2017"><hr><h2><b>October 12, 2017</b></h2></a>
1728<blockquote><p>Would it save you a lot of time if I just gave up and went
1729mad now?</p>
1730<p> - The Hitchhiker's Guide to the Galaxy</p>
1731</blockquote>
1732
1733<p><a href=downloads/toybox-0.7.5.tar.gz>Toybox 0.7.5</a>
1734(<a href=https://github.com/landley/toybox/releases/tag/0.7.5>git commit</a>)
1735is out.</p>
1736
1737<p>This is another <b>mostly bugfix release</b>. I delayed it a bit trying to get some
1738of the many half-finished projects (dd, ping, lsof, iconv, cut, sysconf,
1739the cp --parents option...) finished/promoted/tested, but decided adding big
1740things at the end of the dev cycle would be too destabilizing.</p>
1741
1742<p>A couple <b>new features</b> slipped in anyway. Ilya Kuzmich added head -c
1743and strings -t (and corrected strings' output in a couple places). Elliott
1744Hughes taught file to recognize Macintosh (Mach-O) binaries.</p>
1745
1746<p><b>NOTE</b>: the chrt command is broken when built against musl-libc
1747because that project's maintainer decided he didn't like the system calls
1748it depends on, so he
1749<a href=https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5>removed
1750them from his libc</a>. The prebuilt binaries are built against musl,
1751so the chrt command there will always say -ENOSYS. (It works fine built against
1752glibc, and presumably bionic or uClibc.)</p>
1753
1754<p><b>Bugfixes</b>: Remounting (mount -o remount,rw) should work properly now,
1755and we fixed another place ps was segfaulting when /proc
1756entries vanished out from under us (a hard to hit race condition
1757mostly noticeable in long-running "top" exiting; error value was treated
1758as a pointer). Elliott Hughes fixed a -Wformat warning in expr.c,
1759and made xargs split the command
1760line at ARG_MAX (which is 1/4 the stack size ulimit, which normal users
1761can arbitrarily increase but execve() will complain if they don't).
1762Josh Gao made netcat -l exit after handling a request, and reported a
1763seq bug (seq "1000000 10000001" output 1e+06, fixed now). Zach Riggle fixed a
1764broken URL in the README. Rob fixed env -0 checking the flag wrong so
1765it didn't work in combination with other flags (typo).</p>
1766
1767<p><b>Plumbing</b>:
1768Replaced mbrtowc() with a new utf8towc() that doesn't have a context
1769struct or care about locale. A bugfix in comma_scan() (wasn't removing an entry at the end
1770of the list) is why remount _sometimes_ worked (depending on argument
1771order), xgetaddrinfo() is now separate from xconnect(),
1772atolx_range() learned the w (word=2) suffix, and b is now (block=512) instead
1773of (byte=1).</p>
1774
1775<p><b>Build</b>: Patrick Oppenlander added a workaround for a bug in config2help.c
1776that resulted in segfaults on newer toolchains. (It was actually a use
1777after free error; the build infrastructure isn't nearly as heavily audited as
1778code that gets installed on the target; oops.) A new GITHASH
1779environment variable can force the build version from
1780the command line, and the build checks for .git in the top directory
1781before asking git what our version is (so it isn't confused by ../../.git
1782in an enclosing directory). Added a build #warning about musl intentionally
1783breaking chrt (as with nommu fork() musl now provides a broken
1784stub function so compile-time probes for its existence think it's there, and
1785you can't run the result to test behavior when cross compiling).</p>
1786
1787<a name="19-06-2017" /><a href="#19-06-2017"><hr><h2><b>June 19, 2017</b></h2></a>
1788<blockquote><p>It is a well-known fact that those people who most want to rule people are, ipso facto, those least suited to do it.
1789To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job.</p>
1790<p>- The Hitchhiker's Guide to the Galaxy</p>
1791</blockquote>
1792
1793<p><a href=downloads/toybox-0.7.4.tar.gz>Toybox 0.7.4</a>
1794(<a href=https://github.com/landley/toybox/releases/tag/0.7.4>git commit</a>)
1795is out. No new commands this time, but
1796<b>chrt</b> and <b>dmesg</b> got promoted out of pending.</p>
1797
1798<p><u>New features</u>:
1799Rob rewrote paste, which should work much better now, and added grep
1800-M and -S to match and skip wildcards respectively (useful with -r).
1801Elliott's updated dmesg has -T and --color. The file
1802command can recognize gzip now, uptime grew -s, date grew %N, env knows - as a first argument
1803means -i (posix!) and grew -0, ls defaults to -b
1804instead of -q now when there's a tty, and ls has a new -ll option (with
1805--full-time as a compatibility synonym) showing nanoseconds and (for some
1806reason) timezone. (Why do individual files have timezones?) Elliott added
1807"uudecode -o -" support. Illya Kuzmich taught head -v and -q. The cpio
1808code no longer adds the "TRAILER!!!" entry by default (initramfs extractor
1809doesn't care) without which you can concatenate cpio archives with "cat".
1810(Use the new --trailer option if you want the legacy behavior.)
1811In pending, fdisk compiles now and tar understands bzip2.</p>
1812
1813<p><u>Build</u>:
1814The "make install_airlock" target now symlinks bc from the host because
1815the kernel <a href=https://landley.net/notes-2013.html#28-03-2013>inexplicably</a>
1816needs that to build. This was motivated by
1817<a href=https://github.com/landley/mkroot>mkroot</a>, which builds under
1818a toybox airlock directory.</p>
1819
1820<p>Lots of work on the test suite, mostly from the Android guys who are now
1821running it under Android. This fixed several existing tests that didn't
1822pass, made more tests run on a toybox-only system, and so on. The test suite
1823infrastructure now has a second testing function,
1824"testcmd", which supplies the command name being tested (bypassing
1825shell builtins).</p>
1826
1827<p>Various android build and config fixes, getting closer to being able
1828to let android someday use scripts/make.sh instead of generated/* snapshots.
1829Also more work into building under android's NDK; not quite there yet
1830but much closer.
1831Use nproc in scripts/make.sh detect available processors (so you can control the SMP level with taskset).
1832Removed the old uClibc compatibility glue, it's been 5 years since their
1833<a href=http://lists.busybox.net/pipermail/buildroot/2016-December/180102.html>last release</a>.</p>
1834
1835<p>The new config option TOYBOX_PEDANTIC_ARGS checks arguments when there
1836are no arguments, so things like "uptime" no longer silently ignore arguments
1837you pass but instead refuse to run.</p>
1838
1839<p><u>Docs</u>:
1840The FAQ now has more than one entry. Commands no longer output the full
1841help text for argument errors but instead just say "See %s --help" with the
1842command name (in addition to the actual error message).
1843Elliott did a big period-ectomy on all the --help text, and
1844we cleaned up some tab/space inconsistency. The
1845non-html help -a output now has separators with the command name.
1846The top/iotop and pkill/pgrep help text now describe a lot more of what
1847the commands can do. Twitter's code of conduct page went down so we
1848mirrored the text locally.</p>
1849
1850<p><u>Bugfixes</u>:
1851Fixed a race condition in ps/top where a process that exited right as we
1852read its data returned a different error value than we were expecting (which
1853was causing long-running top instances to occasionally exit),
1854mount now gives an error if it can't autodetect the filesystem
1855type, ps no longer queries the terminal size when output isn't to a tty
1856(so "ps -A | cat" doesn't vary), date's chkmktime() was replaced with
1857simple range checks for fields (to avoid false positives from things like
1858timezones and daylight savings time), removed %s from date's help (we
1859didn't implement it, we have @seconds[.nanoseconds] instead), fixed
1860zcat's buffer flush logic (which was always failing on files larger
1861than 32k), and factor now detects requests for numbers >64 bits and fails
1862loudly instead of producing incorrect answers.
1863Elliott fixed touch -a/-m (they were backwards), and allowed ':' in
1864setprop's property names. Grep now exits with 2 for errors (so -q can
1865distinguish "didn't find" from "didn't work"), doesn't stop on symlinks
1866that point nowhere (there was an error_exit() that should just be a warning),
1867and provides error messages for files we could open but not read.</p>
1868
1869<p><u>Library</u>:
1870New library functions: strend() complements strstart(), minof()/maxof()
1871are min/max macros that evalute arguments once and autodetect type (why
1872isn't this in libc?), xmmap() checks MAP_FAILED (which is not NULL).</p>
1873
1874<a name="21-02-2017" /><a href="#21-02-2017"><hr><h2><b>February 21, 2017</b></h2></a>
1875<blockquote><p>Only six people in the Galaxy knew that the job of the
1876Galactic President was not to wield power but to attract attention
1877away from it. Zaphod Beeblebrox was amazingly good at his job.</p>
1878<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
1879
1880<p>Despite everything, <a href=downloads/toybox-0.7.3.tar.gz>Toybox 0.7.3</a>
1881(<a href=https://github.com/landley/toybox/releases/tag/0.7.3>git commit</a>)
1882is out. The <u>new commands</u> this time are <b>ftpget</b>, <b>ftpput</b>, <b>microcom</b>, and <b>ascii</b>.<p>
1883
1884<p>We also had two command _demotions_ out of defconfig:
1885<b>hostid</b> got moved to toys/example and
1886switched to "default n" because despite <a href=http://pubs.opengroup.org/onlinepubs/9699919799/functions/gethostid.html>still being in posix</a>
1887the concept of a unique 32 bit number identifying a system is something
1888Linux outgrew about the time Pauline Middelink wrote the first IP
1889Masquerading code. And Elliott did a complete rewrite of <b>dmesg</b> introducing
1890two codepaths that I didn't get a chance to unify and didn't want to
1891hold up the release for, so that's back in pending.</p>
1892
1893<p><u>New features</u>: Rob added units to <b>find</b> -atime and friends
1894(with the legacy -amin alias). Elliott added color and -w to dmesg, fallocate
1895-o, and improved file's ELF parsing. Steve Muckle added -d and finit_module
1896support to modprobe. Rob and Elliott tweaked the
1897ps/top display format a bit more (extending the USER field from 8 to 18 chars
1898and putting + at the end of string fields that got truncated).
1899df -a isn't entirely new, but wasn't documented and needed a bugfix.</p>
1900
1901<p><u>Bugfixes</u>:
1902Last release broke oneit because -c didn't get moved to xopen_stdio() (oops).
1903Rob and Elliott simultaneously spotted ps padding each line to 99999
1904chars when there's no tty (serial console or adb); now it pads to 80 in
1905that case but also switches on -w to avoid field truncation. The "tty"
1906field also sometimes had trailing debris (that's fixed now). And "top" was
1907endlessly redrawing with out tty because receipt of the ANSI size probe
1908results would set SIGWINCH, and handling that sent another ansi probe. (Sigh.)
1909And while we're there, replace "ADDR" with "BIT" in ps -l so there are
1910more than 4 chars left for the "CMD" field on 64 bit systems.</p>
1911
1912<p>Izabera pointed out that split -b and -l can't mix, and suggested seq should
1913multiply to avoid accumulating rounding errors from repeated fractional
1914increments. Wang Xiao Jian fixed a bug in sort -k.
1915Elliott let getprop use the @ character in property names, and
1916Dimitry Ivanov removed the name length limit for system properties.
1917Elliott also improved some error reporting and improved top -H's display
1918of thread names.
1919
1920<p>Josh Gao pointed out that recursive operations on . and .. could be ignored
1921in chmod -R (and the resulting generic fix to dirtree_notdotdot() fixed
1922it in several other places).</p>
1923
1924<p>Justin Cormack caught tar producing a warning to stdout that screwed up
1925"tar c" to stdout.
1926Rob fixed an option parsing bug (where switching off a --longopt in menuconfig
1927confused the parser), and another one where an option excluding itself
1928(ala "abc[-ab][!abc]" with "command -a -b") would segfault.</p>
1929
1930<p>There's some sort of gcc stack over-optimization bug where musl-libc's
1931version of vfork() doesn't get marked with attribute(returns_twice) so
1932stack varabiles in the same function after that get semi-randomly overwritten
1933when the optimizer decides to reclaim the space. So add the attribute
1934to the function the XVFORK() wrapper macro calls. (It's a nommu thing.)</p>
1935
1936<p>Fixed a couple variable size mismatch bugs that were only tested on 64 bit
1937(printf %x 64) or only tested on 32 bit (modprobe), removed some
1938unnecessary casts in stat.</p>
1939
1940<p>Continuing attempts to build under Android NDK brought up that posix
1941defines the global 'stdout' as a macro, which older versions of bionic
1942turned into an array member, but a function was using it as an argument
1943name. (This worked in the AOSP build because it only builds against current
1944bionic, where there's a global 'stdout'.)</p>
1945
1946<p>Several commits argued with clang's warning generation, eventually
1947settling on a variant of __attribute__((__shut_up__)).</p>
1948
1949<p>Android should no longer give spurious error messages
1950when you "ps -A | head" about EPIPE on output. (Older versions of bionic
1951set an error handler on SIGPIPE, but it shouldn't do that now. More recent
1952versions of adb set the SIGPIPE handler to SIGIGN instead of SIGDFL,
1953leading to write returning an error message instead of silently killing
1954the program. So we set it back to the default.)</p>
1955
1956<p><u>Docs</u>:
1957Removed website link to the gmane archive (which didn't survive gmane's
1958change of ownership). The FAQ now answers a _second_ question. (Woo!)
1959Some roadmap updates.</p>
1960
1961<p><u>Build tweaks</u>:
1962Upgraded "make install_airlock" target to only warn about missing
1963commands (unless $PEDANTIC is set) when it sets up the hermetic build
1964path. (The plan is still to implement everything but the toolchain
1965binaries in toybox, but in the meantime we're symlinking other stuff from
1966the $HOST that isn't ready yet. See
1967<a href=https://github.com/landley/mkroot>mkroot</a> for an example using
1968this.)</p>
1969
1970<p>Elliott and Rob continue to poke at building toybox with Android's NDK,
1971but it's a work in progress (<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-December/008767.html>thread</a>). Various changes
1972removing libcutils dependencies and adding an selinux dependency to getprop
1973are fallout from this.</p>
1974
1975<p>Cross-compiling from Macs needs to use "gsed" instead of apple's
1976version, so teach the build to use that name if it exists in the $PATH.
1977If you try to build without running config first, you should get better
1978error reporting now. Added a workaround for Centos' broken "which" command
1979producing output when it _can't_ find a name in the $PATH.</p>
1980
1981<p><u>Library</u>:
1982The new dirtree flag DIRTREE_PROC skips non-numeric entries so things
1983like ps and top can scan /proc more efficiently.</p>
1984
1985<a name="21-10-2016" /><a href="#21-10-2016"><hr><h2><b>October 21, 2016</b></h2></a>
1986<blockquote><p>Probability factor of one to one. We have normality. I repeat,
1987we have normality. Anything you still can't cope with is therefore your
1988own problem.</p><p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
1989
1990<p><a href=downloads/toybox-0.7.2.tar.gz>Toybox 0.7.2</a>
1991(<a href=https://github.com/landley/toybox/releases/tag/0.7.2>git commit</a>)
1992is out.</p>
1993
1994<p>During this development cycle, Elliott Hughes <a href=http://androidbackstage.blogspot.com/2016/07/episode-53-adb-on-adb.html>got interviewed
1995on the ADB podcast</a>
1996and Rob Landley <a href=http://linuxluddites.com/shows/episode-88/>got interviewed on Linux Luddites</a> (<a href=http://linuxluddites.com/shows/episode-11/>again</a>).
1997Both talk about toybox and many other things.
1998The web page also grew a new <a href=faq.html>FAQ page</a>, currently with
1999just the one.</p>
2000
2001<p>New comands added to defconfig are <b>tunctl</b>,
2002<b>log</b>, <b>start</b>, <b>stop</b>, and <b>sendevent</b>.
2003The commands <b>file</b> and <b>netstat</b> got promoted out of pending.
2004Pending added <b>chrt</b>, <b>setfattr</b>, and <b>getfattr</b>, and saw
2005a lot of cleanups to diffstat and dd but not enough to promote them to
2006defconfig yet. A new toys/net directory was added, moving ifconfig, netcat,
2007netstat, rfkill, and tunctl there so far.</p>
2008
2009<p><b>Upgrades</b>: All commands now parse --version when they understand
2010--help, but "true" and "false" should now ignore their arguments entirely.
2011We taught stat to handle "%12x" and "%.12x" printf-style escapes, which
2012apparently other versions do. The ifconfig output now shows the interface's
2013device driver. Added patch -d and --dry-run, wc can now do -cm together,
2014find has a NOP -noleaf so scripts that use that don't break, add -c to md5sum
2015and sha1sum. Elliott taught ps to treat extra aguments as additional -p
2016pids, implemented xxd -s, did a number of upgrades to file (added -HL,
2017support for ar files, improved ELF support to report android API level
2018and stripped/not stripped and it no longer prints a guessed build ID type).
2019Elliott also added optional build-time support for using openssl's
2020assembly-optimized md5sum/sha1sum implementations (leading to a new
2021<a href=design.html>design</a> policy on shared libraries).</p>
2022
2023<p><b>Bugfixes</b>: Too many fixes to "ps" and "touch" to list, from both
2024Elliott and Rob. Rob taught sed to handle s/[[:space:]/]// type sequences
2025properly, switched grep to a better
2026workaround for <a href=https://sourceware.org/bugzilla/show_bug.cgi?id=17829>glibc bug 17829</a>, made sed -i preserve ownership when run as root,
2027made du max out at 2 terabytes instead of 2 gigabytes on 32-bit systems
2028(it was always designed to, but was missing a typecast),
2029fixed the option parsing infrastructure (config options that remove command
2030line options got the placeholders wrong), fix to printf for printing
2031octal digits and handling the (posix-mandated) difference between %b and
2032non-%b octal output, reading from "-" no longer closes stdin when done,
2033netcat -L works with nommu (although it may need more portability work),
2034and you can now "make test_scankey" if you want to. Several commands
2035(stat, makedeves, chgrp, cp, find) handled user name lookup failure badly
2036(stat was segfaulting if you interrogated a file belonging to a nonexistent
2037user, "chown 12345 file" errored out if you didn't have that user
2038in /etc/passwd... now they should all print/accept the number when
2039appropriate). "LC_ALL=C ls -Cs --color" produces the same output
2040as other versions (two spaces padding, -k hardwired on).<p>
2041
2042<p>Kyungsik Lee fixed a bug
2043in cp (readlink() doesn't actually null terminate the string it reads in),
2044Elliott Hughes made pgrep/pkill return success/failure, fixed trailing
2045whitespace in netstat, fixed a SMACK symbol conflict due to linux/xattr.h
2046changing, fixed ls -sh, and added a lot of
2047stuff to the <a href=roadmap.html>roadmap page</a>.
2048Izabera pointed out cmp -l and -s can't be selected at the same time,
2049that timeout was never actually checking -v, that ls should default to -q
2050when output is to a tty, and that "file -" would sometimes try to open "-"
2051instead of stdin.
2052Usischev Yury pointed out a use after free error, and that id shouldn't
2053call exit() directly. Matthias Urhahn pointed out that stat(2) returns
2054hardwired 512-byte units, so stat.c was wrong. David Hedges pointed out
2055that route could only handle 10 character interface names when the kernel
2056can do 15 (it's still in pending for a reason, but fixed). Evgenii
2057Stepanov found and helped diagnose one of the more subtle ps bugs fixed
2058this time around.</p>
2059
2060<p>Calling "make test_blah" no longer causes make to error out if the last
2061test fails (and thus returns a nonzero error code).
2062Building single commands and the multiplexer used to require a "make clean"
2063between them (because they had different config files both of which were older
2064than generated/config.h so it didn't get rebuilt; now it just always
2065rebuilds it).</p>
2066
2067<p>The defconfig build is now slightly less broken on older centos versions
2068(although <a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-September/008664.html>the consensus</a> is that Centos is just generally broken).<p>
2069
2070<p>Several commands were over-using xprintf(), which flushes its output
2071to check for error (something you only need to do maybe once per line,
2072and even then maybe only in loops because xexit() flushes and checks
2073ferror() for you and adjusts the exit code if we wrote stuff to stdout
2074that couldn't be printed). Lots of little flushes are inefficient,
2075so most things can use normal printf(). (Retransmission
2076of short writes is presumably libc's problem since it's buffering the
2077output and all.)</p>
2078
2079<p><b>Library:</b>
2080New library functions readlink0() and readlinkat0() which properly null
2081terminates the symlink value (which the stock libc function inexplicably
2082doesn't).
2083The new do_lines() function interates reading lines from a filehandle
2084and calling a function on each line.
2085New function pollinate() factoring out netcat's poll() loop so things
2086like telnet can use it.
2087New functions getusername() and getgroupname() return a
2088char * given a uid/gid (and return a string representation of the number
2089if the lookup fails), and xgetpwnamid/xgetgrnamid were renamed to
2090xgetuid/xgetgid and now return an integer instead of a struct (also helping
2091handle lookup failures, you can still return the uid/get for "12345").</p>
2092
2093<p>Switched atolx() to use long long internally.
2094Renamed xopen() to xopen_stdio() and made a new xopen() that never returns
2095stdin, stdout, or stderr (duping /dev/null into the filehandles as necessary).
2096New function xopenro() opens a file read only with one less argument, and
2097understands that "-" means stdin. New flag WARN_ONLY tells these functions
2098to just print a warning on failure, and return -1 instead of exiting.
2099Misc new functions like openro() which defaults to the WARN_ONLY behavior
2100and notstdio() which dup()s a filehandle up beyond stdin/out/err backfilling
2101with /dev/null as necessary. The WARN_ONLY flag let us remove the failok
2102argument from loopfiles().</p>
2103
2104<p>New TOYFLAG_NOHELP disables --help processing (which "true" and "false"
2105should not do).</b>
2106
2107<p>The test suite now has NOSPACE=1 to ignore whitespace (using diff -b to
2108check results), which helps TEST_HOST pass the same tests as toybox.
2109Fixes to chattr and date tests. It also has a new variable $C with the
2110absolute path to the command being tested (bypassing shell builtins),
2111and a function testcmd() which is just like testing() except it prepends
2112the command name ($C) to the test command line as well as the test
2113description.</p>
2114
2115<a name="02-06-2016" /><a href="#02-06-2016"><hr><h2><b>June 2, 2016</b></h2></a>
2116<blockquote><p>When the 'Drink' button is pressed it makes an instant but
2117highly detailed examination of the subject's taste buds, a spectroscopic
2118analysis of the subject's metabolism, and then sends tiny experimental
2119signals down the neural pathways to the taste centres of the subject's
2120brain to see what is likely to be well received. However, no-one knows
2121quite why it does this because it then invariably delivers a cupful of
2122liquid that is almost, but not quite, entirely unlike tea.</p>
2123<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
2124
2125<p><a href=downloads/toybox-0.7.1.tar.gz>Toybox 0.7.1</a>
2126(<a href=https://github.com/landley/toybox/releases/tag/0.7.1>git commit</a>)
2127is out. (Yes, I forgot to update the --version string, but I already
2128uploaded the <a href=downloads/binaries/0.7.1>binaries</a>.)</p>
2129
2130<p>The website has https support now, you can "make cat ps ls"
2131to get standalone commands (and "make list list_pending" to see what's
2132available), and a whole lot of bugfixes and new options to existing
2133commands.</p>
2134
2135<h2>New Commands</h2>
2136<p>Rob implemented <b>ulimit</b>. In pending, Elliott Hughes implemented
2137file. and Lipi Lee implemented a simple wget. (Pending also had minor
2138cleanups to more and lsof, but no promotions this time around.)</p>
2139
2140<h2>New Options</h2>
2141<p>Izabera implemented env -u, suggested adding seq -w, made factor
2142use full unsigned 64 bit math even on 32 bit platforms, pointed out base64
2143-w0 should disable wrapping, and sped up wc -c.
2144Elliott Hughes added mount -o relatime, xxd -p -r, and od -w.
2145Sameer Pradhan (or possibly Bilal Qureshi) suggested adding stat -tL -c %m%t%T.
2146Tom Cherry added getprop -Z. Paul Barker added hostname -b and -F.
2147Rob added ls -b, made ls -q work with utf8,
2148made sed -f - read from stdin, and added top -O (like ps -O).</p>
2149
2150<p>Elliott and Rob <b>added Thread support to ps and top</b>,
2151with -o TID, TNAME. We also added -o PCY (android scheduling policy),
2152-o BIT (process is 32 or 64 bit), and -o TNAME now shows the parent
2153command name for threads.</p>
2154
2155<h2>Documentation</h2>
2156
2157<p>Rob added the sed invocations to convert tabs/spaces and back to
2158design.html. Isaac Dunham updated hexedit's help text.
2159Jakob Flierl pointed out a broken URL in the README.</p>
2160
2161<p>Rob also redid the naming scheme of sed's pattern manipulation code to
2162remove the gratuitous references to Roger Zelazny's "Amber" series, since it
2163was confusing people.</p>
2164
2165<h2>Bugfixes</h2>
2166
2167<p>Grep -H and -n should now work properly with -ABC. Andy Chu pointed out an
2168out of bounds access for zero length lines in rev, fixed a buffer overflow
2169in diff -r, and fixed operator precedence in expr (although Rob is rewriting
2170chunks of expr so toysh can use its plumbing for $(( )) ).
2171Patrick Ohly fixed the too-aggressive suid permission dropping logic.
2172Josh Gao fixed a segfault when find -iname got no argument, and
2173made tail -f work right with just one file argument.
2174Tom Marshall cleaned up tar's long filename support and improved
2175the tar tests, and reported another find bug (with -iname -o -iname not
2176tracking copy lifetimes properly) that got fixed.</p>
2177
2178<p>Elliott Hughes fixed wc -c to not trust zero length files to actually be
2179zero length (/proc does that), fixed "mount -o rw,remount /system" on
2180Android, removed trailing spaces on ps -o cmdline, fixed pkill -9 and
2181the corresponding tests, made "insmod -" work, fixed top -b and tail -NUM,
2182pointed out that ps shouldn't trim numeric fields for display size limits,
2183and added some more
2184explicit "sort" calls to make pipelines so build tempfiles are easier to cache.
2185Rob <a href=https://github.com/landley/toybox/commit/32b3587af261>fixed an insane sed thing</a> the perl 5.22 build was doing.
2186Fixed mount -o to properly pass in leftover string data, and
2187<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-March/004790.html>documented how to use toybox to mount nfs</a> (warning: kernel patch
2188to fix some bitrot in the kernel NFS driver's string parsing attached
2189to that message).</p>
2190
2191<p>George Burgess IV corrected some variable types in traceroute.
2192Base64 now wraps == properly. Fixed two bzcat segfaults reported by
2193John Regehr. Andy Chu found a segfault in "sed -e 'c\'" with no trailing
2194line, and implemented mv -n and cp -n. The cyanogenmod guys pointed out that
2195cp -a shouldn't complain if a non-root user can't chown, and we added
2196the output path to cp -r error messages while we were there (before was just
2197the filename).</p>
2198
2199<p>Samuel Holland fixed blkid's handling of vfat labels, and
2200fixed a segfault when basename was passed an empty string and an empty
2201suffix. Davis Mosenkovs fixed touch -t seconds parsing.
2202Rob fixed a bunzip bug reported by John Regehr (the bad CRC
2203error message was printing a NUL argument).</p>
2204
2205<p>Not all of the commands build standalone, but more of them do now;
2206scripts/single.sh can now build a "mv" that isn't actually "cp".
2207The dependencies are more granualr, so "make top; make ps" no longer
2208produces a broken ps that ignores -A (because ps.o wasn't getting rebuilt
2209even though top had the FLAG macros for -A zeroed).</p>
2210
2211<h2>Build</h2>
2212<p>Rob added a <b>warning when building commands out of
2213pending</b>. (The pending directory is full of code that hasn't been
2214properly vetted. Use at your own risk.)</p>
2215
2216<p><b>New build targets let you build individual commands by name</b>, ala
2217"make ls cat ps", and you can run the test suite for each standalone
2218command with "make test_ls" and such.
2219"make list" shows all such standalone commands in defconfig, and
2220"make list_pending" shows unfinished commands from toys/pending
2221("make list list_pending" shows both together). "make clean" now deletes
2222these filenames at the top level, and the corresponding unstripped files
2223live in the directory generated/unstripped.</p>
2224
2225<p>Nicholas Boichat suggested switching make.sh to use $! for process
2226enumeration during parallel builds (which is both more efficient and more
2227portable), and suggested shell builtin replacements for wc/awk/sed so the
2228build loop has fewer forks now.</p>
2229
2230<p>Lots of work on the test suite, much of it due to Andy Chu. It now
2231consistently prints the name of the command being tested at the start of each
2232test (and the common infrastructure does that, not each individual test), and
2233"make tests" actually runs all the available tests now.
2234Seperated pgrep and pkill tests, split lsattr/chattr, added fstype and base64
2235tests. The "tests/files" directory now collects files for tests to
2236use, with blkid, bzcat, and utf8 subdirectories: the $FILES variable
2237gives a path to it, so "$FILES/blkid/ext2.bz2" and so on.
2238The testsuite now has test files with 3 different types of "not utf8 output"
2239sequences that require escaping, plus some combining character torture
2240tests, direction reversals, and so on.</p>
2241
2242<p>Added dependencies on TOYBOX_FORK to various pending commands that need
2243nommu conversion (which should fix the allyesconfig build).</p>
2244
2245<p>Static builds with selinux should work again.</p>
2246
2247<h2>Library</h2>
2248
2249<p>New bufgetgrgid() and bufgetpwuid() functions cache previous lookup info
2250rather than repeatedly traversing /etc/passwd and /etc/group (which is slow).
2251Added xpipe() to lib to catch pipe creation failure.
2252The HELP_ macros generated by config2help.c now use a capital prefix
2253to avoid collicing with help_exit() and such.</p>
2254
2255<p>The dirtree infrastructure got a cleanup pass in preparation for adding
2256infinite recursion depth support (needed by rm -r), updated the
2257<a href=code.html#ib_dirtree>documentation</a> to describe the new
2258semantics (removing dirtree_start() and adding dirtree_flagread()).
2259Now dirtree_recurse() takes the new dirfd as an argument.</p>
2260
2261<p>Split out _xexit() from xexit() and let sigatexit() set multiple
2262callbacks.</p>
2263
2264<p>For years the man pages have said to #include <sys/types.h> to get
2265major/minor/makedev but now that glibc
2266<a href=https://sourceware.org/ml/libc-alpha/2015-11/msg00253.html>has vowed
2267to break existing programs</a> and replace it with another nonstandard header
2268not in posix or lsb, we added our own functions to lib/ to do the transform
2269ourselves (based on what the kernel actually expects).</p>
2270
2271<h2>Portability</h2>
2272
2273<p>Debian unstable started needing an extra header #include for some reason, and
2274although printf("%.*s", INT_MAX, s) worked fine on Ubuntu 12.04 it
2275didn't on 14.04, so added a workaround for that. Typecast a printf because
2276wchar_t isn't a rigidly defined size. RLIMIT_RTTIME was
2277added to the kernel in 2008 but you can't expect uClibc to have noticed yet,
2278nor did it #define MS_RELATIME (added in 2006), or prlimit (2010)...
2279(Given the improvements in musl and bionic, uClibc support may be dropped
2280in a future release.) Given that the xattr functions were added during
2281linux 2.5, we can #include its header unconditionally.</p>
2282
2283<p>Renamed basename_r() to something else to avoid conflicting with freebsd's
2284libc, and both scripts/install.h and scripts/config2help.c no longer include
2285toys.h (to make cross-compiling from systems we don't run on easier).</p>
2286
2287<p>Debian bug 635570 did something unspeakably nonportable, depending on
2288"sed -e 'a\'" (with no next line of the pattern, so an unterminated
2289continuation) to add a newline to the last line of input if and only if
2290that last line of the input didn't have a newline, and to take no other
2291action. This is well into "depending on a bug" territory, but we implemented
2292it because otherwise Debian's install broke. (Of course this behavior
2293is undocumented, non-obvious, and doesn't really make logical sense.)</p>
2294
2295<p>CONFIG_TOYBOX_NORECURSE now disables the stack measuring logic (which
2296was giving some "security" code fits). Also we typecast pointers to (long)
2297before comparing them to avoid spurious compiler "optimizations" that
2298break the code.</p>
2299
2300<a name="02-02-2016" /><a href="#02-02-2016"><hr><h2><b>February 2, 2016</b></h2></a>
2301<blockquote><p>"I checked it very thoroughly," said the computer, "and that
2302quite definitely is the answer. I think the problem, to be quite honest with
2303you, is that you've never actually known what the question is."
2304</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
2305
2306<p><a href=downloads/toybox-0.7.0.tar.gz>Toybox 0.7.0</a>
2307(<a href=https://github.com/landley/toybox/releases/tag/0.7.0>git commit</a>)
2308is out.</p>
2309
2310<p>The new commands in defconfig are <b>iotop</b>, <b>top</b>, <b>pgrep</b>,
2311and <b>pkill</b>
2312(most replacing corresponding versions from pending). Added grep -ABC,
2313swapon -d (discard), mkswap -L (label) and UUID support, and find -delete.
2314Izabera added free -h and unshare -f. Josh Gao implemented tail -f.
2315Jose Bollo submitted cp --preserve=context,attr. Kylie McClain added
2316mktemp -u.</p>
2317
2318<p>In pending there's the start of a vi command, and Sameer Pradhan contributed
2319a new dhcp6. This cycle saw several rounds of route cleanup and a little dhcp
2320cleanup, but neither are complete yet. Lipi Lee did some cleanup to netstat.c
2321and Elliott Hughes removed warnings from traceroute.</p>
2322
2323<p>Lots of updates to ps: several new -o options, -k (--sort) -O and -M,
2324improved compatibility with Android's historical behavior, and
2325extensive internal code cleanup (including the removal of all
2326the magic constants).</p>
2327
2328<h3><b>Website</b></h3>
2329
2330<p>Dreamhost restored the <a href="#12-21-2015">missing 11 months</a>
2331to the mailing list archive, in the process deleting the month after
2332that. Now they've asked if I have mbox files archiving the new
2333gap (between December 20, 2015 to January 21, 2016, and presumably they
2334could also fill in the gap from December 14, 2014 to January 3, 2015 that's
2335been there since the last time they did this),
2336but due to some gmail filtering I've
2337<a href=http://landley.net/notes-2012.html#15-10-2012>never
2338been able to disable</a>, my copy of those files is spread among 3 different
2339mbox files I'd have to sort/filter/collate. (It's on the todo list.)</p>
2340
2341<p>Added a code of conduct to the README (we're
2342<a href=https://engineering.twitter.com/opensource/code-of-conduct>borrowing twitter's</a>) because somebody
2343made it necessary.</p>
2344
2345<h3><b>Bugfixes</b></h3>
2346<p>Fixed another sed bug where any ] right after [ was skipped (not just the
2347first one in the range, so [[] didn't terminate). Fixed sort -f and added test cases.
2348Assume 80 columns in "ls -m | cat", ls -L is no longer backwards,
2349and ls of files with no paths no longer uses an uninitialized (zero) dirfd.
2350Several bugfixes
2351to find (Gilad Arnold fixed -perm, Daniel K. Levy fixed "find . -exec echo {}",
2352and while we're there I fixed find --prune, made "find . -execdir
2353echo {} + -execdir ls {} +" work, and ripped out the environment size
2354measuring code that checked for a 128k limit removed back in linux 2.6.22).
2355Elliott Hughes fixed the date command's parsing of 4 digit
2356years and documented the %s escape, fixed hwclock -u, and pointed out
2357that runcon needs to exec to do its job (not recursively call another
2358command_main() in the same process). Tom Marshall reported that blkid was
2359handling ext2 wrong. Mike Moreton corrected cpio extraction's uid and gid
2360values, and added a --no-preserve-owner option. Fixed the SUID permission
2361dropping logic (which was a bit over-zealous, preventing some commands from
2362running at all).</p>
2363
2364<p>I'm told that debian-testing broke its libc so the nsenter build breaks,
2365but my attempts to install the debian-testing network cd image under
2366qemu keep breaking. Maybe someday they'll fix it enough I can actually
2367reproduce the problem. (Debootstrap under unbuntu builds an ubuntu-flavored
2368chroot in which toybox builds fine.)</p>
2369
2370<h3><b>Documentation</b></h3>
2371<p>Rewrite of the about.html page, tweaks to design.html, and a re-triage of
2372sbase in roadmap.html. Update to mkstatus.py to collate multiple span
2373tags with the same id, resulting in a larger status.html page (which
2374was previously ignoring some commands in the roadmap).</p>
2375
2376<p>Expanded the defconfig/allyesconfig/allnoconfighelp text in "make help"
2377to explain what they're for.</p>
2378
2379<h3><b>infrastructure</b></h3>
2380<ul>
2381<li><p>Expanded toys.optargs to 64 bits so a command can have more than 32 options.</p></li>
2382<li><p>Added NOEXIT() wrapper to turn xwrap() functions into warning versions
2383using the existing longjump(toys.rebound) infrastructure.</p></li>
2384<li><p>Renamed dirtree->data to dirfd and stopped storing symlink length
2385into it (this fixed a bug where following symlinks to directories
2386didn't give a valid directory filehandle, noticeable with ls -Z).</p></li>
2387<li><p>New TAGGED_ARRAY() infrastructure generates index and bitmask macros
2388for arrays of structures starting with a name string.</p></li>
2389<li><p>New lib/linestack.c for utf8 fontmetrics (draw_str() and utf8len()
2390and so on), and for tracking multiple lines of text
2391(vi, less, shell history) that need wordwrapping and scrolling up/down.</p></li>
2392<li><p>Upgrades to lib/interestingtimes.c: scan_key() now has a timeout
2393in milliseconds and recognizes more sequences including ANSI
2394window size probes. New utf8 test files in tests/files/utf8 including
2395sequence reversing, stacked combining chars, and all three types of
2396unprintable sequences (low ascii <32 ala ^X, invalid utf8 sequences ala
2397<AB><CD>, and invalid unicode code points ala U+1234).</p></li>
2398<li><p>More comma handling code in lib.c: comma_args()</p></li>
2399<li><p>Added error_msg_raw() to shut up fortify's endless static checking false
2400positives.</p></li>
2401<li><p>readfileat() can now realloc() in a loop to read long files
2402("zcat | insmod" needed it).</p></li>
2403</ul>
2404
2405<h3><b>Roadmap</b></h3>
2406<p>We're getting close to having a self-hosting development environment
2407using toybox for the command line. The remaining busybox commands in
2408<a href=http://landley.net/aboriginal/about.html>Aboriginal Linux</a> are:</p>
2409
2410<blockquote><p><b>
2411awk bunzip2 bzcat bzip2 dd diff expr fdisk ftpd ftpget ftpput gunzip gzip
2412less ping route sh sha512sum tar test tr unxz vi wget xzcat zcat
2413</b></p></blockquote>
2414
2415<p>And the remaining non-busybox commands in Aboriginal Linux's build/host
2416directory (from the distcc, genext2fs, e2fsprogs, zlib, and squashfs packagesi)
2417are:</p>
2418
2419<blockquote><p><b>
2420mke2fs fsck.ext2 resize2fs distcc genext2fs unsquashfs distccd mksquashfs tune2fs
2421</b></p></blockquote>
2422
2423<p>Squashfs and distcc are probably out of scope for toybox, but mke2fs,
2424fsck.ext2, resize2fs, genext2fs, and tune2fs should all be added to the
2425above "busybox" replacement list.</p>
2426
2427<p>Remind me to include this countdown in future releases. Once they've all
2428been replaced, the next goal is <a href=http://landley.net/aboriginal/about.html#selfhost>building AOSP under itself</a>.</p>
2429
2430<p>See the full <a href=roadmap.html>roadmap</a> and <a href=status.html>status</a>
2431pages for more details.</p>
2432
2433<a name="12-21-2015" /><a href="#12-21-2015"><hr><h2><b>December 21, 2015</b></h2></a>
2434
2435<p>Yes, 11 months have gone missing from the mailing list web archive.</p>
2436
2437<p>Yesterday evening Dreamhost's mailman server went down (timing out trying
2438to connect). I poked them about it, they
2439<a href=https://twitter.com/landley/status/678781271670149121>blamed
2440DNS</a>, I explained that the hang was _after_ the DNS lookup and
2441entered the dig info into the trouble ticket showing the IPs the DNS
2442queries were returning, they reinstalled the server at that IP from what I
2443assume was their most recent backup, and that's how 11 months of messages
2444vanished out of the archive.</p>
2445
2446<p>I've <a href=https://twitter.com/landley/status/679114451975467008>continued to poke them about it</a> but I honestly believe that's the best they
2447can do. <a href=http://landley.net/dreamhost.txt>Last time</a>
2448this sort of thing <a href=http://landley.net/dreamhost2.txt>happened</a>
2449we went back and forth for months, so I added a link to a backup web
2450archive (in the nav bar on the left) that isn't controlled by dreamhost,
2451and thus doesn't gratuitously lose data on a regular basis. (I note
2452the earlier hole in Dreamhost's archive was never fixed either. That
2453was data never getting archived, this is a year's worth of data that
2454was in the archive until yesterday vanishing after the fact.)</p>
2455
2456<p>If you're wondering why the <a href=http://lists.landley.net>top level</a>
2457list page has been "temporarily disabled" for multiple years now... you'd have
2458to ask Dreamhost. I know I have. More than a dozen times.</p>
2459
2460<a name="03-11-2015" /><a href="#03-11-2015"><hr><h2><b>November 3, 2015</b></h2></a>
2461<blockquote><p>"Alright," said Ford. "How would you react if I said that I'm
2462not from Guildford after all, but from a small planet somewhere in the vicinity
2463of Betelgeuse?" Arthur shrugged in a so-so sort of way. "I don't know," he
2464said, taking a pull of beer. "Why - do you think it's the sort of thing you're
2465likely to say?"</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
2466
2467<p><a href=downloads/toybox-0.6.1.tar.gz>Toybox 0.6.1</a>
2468(<a href=https://github.com/landley/toybox/releases/tag/0.6.1>git commit</a>)
2469is out.</p>
2470
2471<p>We have a new <b>ps</b> command with all the -o fields posix wants (although
2472it doesn't accept BSD non-dash option syntax yet), and <b>bunzip2</b> (not just
2473bzcat but the proper extract-in-place command).
2474Sameer Pradhan added <b>hostid</b> and <b>fsync</b>.
2475Elliott Hughes added <b>flock</b>.
2476
2477<p>The people waiting for <b>human readable number support</b> (du -hH, ls -h,
2478and so on) can thank Elliott Hughes for implementing it. (Our output doesn't
2479exactly match others' because we our "binary" mode will say 1.0G instead of
24801024M, which is a bug in the other one we didn't emulate.)</p>
2481
2482<p>The other big news is <b>nommu support</b>, tested on the new
2483<a href=http://nommu.org/jcore>jcore</a> processor but presumaby working
2484on any nommu system. A few commands don't support nommu yet, but those
2485are disabled by dependencies on TOYBOX_FORK in menuconfig when building
2486for nommu. The roadmap now has a large section analyzing the uClinux
2487project (note that <a href=http://nommu.org>nommu.org</a> is slowly replacing
2488<a href=http://uclinux.org>uclinux.org</a> as the standard repository of
2489all knowledge and wisdom about nommu. The old site <a href=#12-02-2012>contains
2490much that is apocryphal</a>, or at least wildly inaccurate, and the new one
2491is trying to improve on that).</p>
2492
2493<p>Both "make change" and scripts/single.sh (for building standalone commands
2494without the multiplexer logic) now use the top level .config
2495for toybox global settings such as Linux Security Blanket Module selection,
2496(so make defconfig before change now).</p>
2497
2498<p>Documentation updates to the <a href=code.html>code</a> and
2499<a href=roadmap.html>roadmap</a> pages.</p>
2500
2501<h3>pending</h3>
2502
2503<p>In the pending directory Sameer Pradhan added tftp,
2504and Elliott Hughes sent lsof. Isaac Dunham upgraded mdev,
2505reboot, init, login, and modprobe, and fixed a distro-specific build break in
2506scripts/mkflags.h. Elliott Hughes and Lipi Lee made netstat -p handle
2507command lines longer than 21 characters, and Elliott fixed netstat -e and
2508some build warnings. Yeongdeok Suh fixed a warning in dhcpd.
2509I started cleanup on pgrep/pkill.</p>
2510
2511<h3>Command updates, bugfixes, and infrastructure</h3>
2512
2513<p>The multiplexer's "command not found" error exit is now 127, so now you can't
2514distinguish between a command not being found in the multiplexer and
2515the multiplexer itself not being found by the shell, because people wanted
2516that for some reason.</p>
2517
2518<p>Elliott Hughes made date reject invalid dates rather
2519than set the clock to something weird (setting the clock 100 years into the
2520future makes most Linux desktops surprisingly unhappy, and ntpdate won't fix it
2521either), fixed several ls -l display issues (user/group field ordering,
2522make user/group/lsmcontext left aligned), did the aforementioned
2523extensive work on human readable number output, fixed ionice's default
2524class, fixed a mv overwrite bug, made df's columns auto-size, added
2525--ppid and -Z to ps, and teamed up with Daniel K. Levy to fix
2526a segfault in find's handling of -newer -group or -user.</p>
2527
2528<p>Hyejin Kim added stat -c %T support. Colin Cross worked
2529on vmstat fixing
2530a header printing bug and calculating the bi and bo columns in the right
2531units. Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra
2532newline and that grep -w '\(x\)\1' didn't work, both now fixed.
2533Alistair Strachan fixed several problems with switch_root. Kylie McClain
2534pointed out env should be able to clear variables via NAME= syntax.
2535Dima Krasner added support for running blkid without a partition (so it shows
2536all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p>
2537
2538<p>Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra
2539newline and that grep -w '\(x\)\1' didn't work, both now fixed.
2540Alistair Strachan fixed several problems with switch_root. Kylie McClain
2541pointed out env should be able to clear variables via NAME= syntax.
2542Dima Krasner added support for running blkid without a partition (so it shows
2543all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p>
2544
2545<p>Two large thinko fixes in oneit: -3 was always enabled (which would
2546eventually block if the child never read the exiting PID numbers from its file
2547descriptor #3 until the pipe filled up), and the signal handlers weren't
2548set up right (for requesting semi-graceful halt/poweroff/reboot).
2549Calling install without a mode is now 0755, and install -g 0 no longer clashes
2550with cp --preserve. Better error message for ls -r on unreadable
2551directories, and ls -Z now uses O_PATH (with the /proc/self/fd/%d
2552workaround for kernel stupidity as necessary).</p>
2553
2554<p>Date now understands @unixtime[.fraction] and uses -D for
2555the set-side format (matching busybox's extension for this). The seq -f
2556string now checks that it's got exactly one %f escape with the correct
2557attributes (and a whole bunch of test cases for it). Fixed a bug
2558in od that screwed up the position indicator on arm and mips.
2559In stat the d/h units moved from %d %D to the default string.
2560And patch can now correctly apply hunks with trailing context to the start of
2561the file.</p>
2562
2563<p>The prompt argument moved out of yesno() (the caller can print the prompt
2564themselves). Replaced toys.exithelp with help_exit(). Added new
2565XVFORK() macro, and xpopen_both() calls /proc/self/exe when passed
2566a NULL argv (see cpio -p for example usage). Replaced toys.recurse
2567with toys.stacktop so the recurse or re-exec decision is now based
2568on bytes of stack space used. Marked a bunch of command-local functions
2569static.</p>
2570
2571<p>New additions to lib/ include strlower(), xconnect(), and the
2572aforementioned help_exit().
2573The testsuite now has some infrastructure tests based on "example"
2574commands such as toys/examples/test_human_readable.c.
2575The login command finally got a long-overdue cleanup (it's one of the
2576commands that predate the "pending" directory but were part of the reason
2577for it). Hexedit had an
2578uninitialized variable (of course gcc didn't spot it, it was too busy
2579warning about "may be used uninitialized but never actually is" variables).</p>
2580
2581<p>Tweaked makefile so
2582"make CROSS_COMPILE=prefix-" (as well as "CROSS_COMPILE=prefix- make",
2583which still works). Toybox is now installed chmod -w so broken installers
2584(like the bunzip2 package's) that try to overwrite existing binaries won't
2585knock out the whole of toybox.
2586GCC 5.2.0 stopped being able to compile Linux 2.6.12's kconfig, but
2587we added a workaround. You can now build uptime without utmpx.h.
2588Alejandro Joya pointed out that enabling smack required smack on the host
2589as well as target when cross compiling, which is now fixed.</p>
2590
2591<p>Note: toybox can autodetect nommu support when building with a uClibc
2592toolchain such as <a href=http://landley.net/aboriginal/downloads/binaries/old/1.4.3/cross-compiler-sh2eb.tar.gz>the one from Aboriginal Linux</a>,
2593but <a href=http://github.com/richfelker/musl-cross-make>with musl-libc</a>
2594you'll have to enable CONFIG_TOYBOX_MUSL_NOMMU_IS_BROKEN to work around the
2595fact they provide a non-functional fork() implementation that always returns
2596-ENOSYS, to prevent you from compile-time probing for nommu support when
2597cross-compiling. Unfortunately "preventing you from probing" seems to be
2598an explicit policy with musl, they also don't provide an "#ifdef __MUSL__"
2599because their library is perfect and you're only ever allowed to work around
2600other people's bugs, not theirs. So we have to use menuconfig to manually
2601enable musl-specific bug workarounds.</p>
2602
2603<a name="23-07-2015" /><a href="#23-07-2015"><hr><h2><b>July 23, 2015</b></h2></a>
2604<p>I recreated the <a href=downloads/toybox-0.6.0.tar.gz>0.6.0 source tarball</a>
2605(new sha1sum 08fb1c23f520c25a15f262a8a95ea5b676a98d54)
2606because I forgot to add --prefix to the git archive command when I updated
2607my release script from mercurial, so the files weren't in an enclosing
2608directory. (Ooops.)</p>
2609
2610<a name="19-07-2015" /><a href="#19-07-2015"><hr><h2><b>July 19, 2015</b></h2></a>
2611<blockquote><p>
2612The reason why it was published in the form of a micro sub meson electronic
2613component is that if it were printed in normal book form, an interstellar
2614hitchhiker would require several inconveniently large buildings to carry it
2615around in.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
2616
2617<p><a href=downloads/toybox-0.6.0.tar.gz>Toybox 0.6.0</a>
2618(<a href=https://github.com/landley/toybox/releases/tag/0.6.0>git commit</a>)
2619is out. (Yes, git. See the <a href=#05-04-2015>previous news entry</a>.)</p>
2620
2621<p>Sorry for the unusually long gap between releases. Since last release Ye
2622Olde Project Maintainer traveled to japan twice and had two more "once
2623a century" floods at home. (Probably a coincidence.) Still catching up.</p>
2624
2625<h3><b>CELF/ELC talk and Wikipedia[citation needed] article</b></h3>
2626
2627<p>I gave another State Of The Toybox talk
2628(<a href=https://www.youtube.com/watch?v=04XwAbtPmAg>video</a>
2629<a href=http://landley.net/talks/celf-2015.txt>outline</a>), in which I
2630repeat my <a href=http://landley.net/notes-2013.html#07-11-2013>perennial</a>
2631<a href=https://twitter.com/landley/status/557309224535851009>complaint</a>
2632that Wikipedia[citation needed]
2633<a href=http://en.wikipedia.org/wiki/Toybox>still</a>
2634<a href=https://en.wikipedia.org/wiki/BusyBox#Controversy_over_Toybox>says</a>
2635toybox was relicensed before its hiatus, when relicensing was why
2636the hiatus ended.</p>
2637
2638<p>Since Wikipedia[citation needed] seems unable to do the
2639<a href=#15-11-2011>most</a>
2640<a href=http://landley.net/hg/toybox/log/tip/LICENSE>basic</a>
2641<a href=http://landley.net/notes-2011.html#13-11-2011>research</a> on
2642this point, and has stuck to an incorrect sequence of events for years,
2643I've been gradually escalating my attempts to correct them. Toybox
2644came out of mothballs in November 2011 <b>because</b> it could be
2645relicensed. That's what opened up a new niche busybox wasn't already
2646filling with a 10 year headstart.</p>
2647
2648<a name="asterisk_back" />
2649<p>The article has plenty of smaller issues<a href=#asterisk>*</a>, but
2650given that I gave an entire talk at Ohio LinuxFest in 2013
2651(<a href=http://landley.net/talks/ohio-2013.txt>outline</a>,
2652<a href=https://archive.org/download/OhioLinuxfest2013/24-Rob_Landley-The_Rise_and_Fall_of_Copyleft.mp3>audio</a>) on why I switched away from GPL for
2653my projects, that one bugs me.</p>
2654
2655<h3><b>New stuff this release</b></h3>
2656
2657<p>There's a new android menu in menuconfig, and rather a lot of Linux
2658Security Module support (Smack for Tizen from Xavier Roche and José Bollo,
2659and SELinux for Android from Elliott Hughes; see
2660the Security Blanket menu under global settings in menuconfig) has
2661trickled in, although there's still more to come.</p>
2662
2663<p><b>New commands:</b> Added reset, nproc, ionice, and iorenice.
2664Elliott Hughes contributed xxd, runcon,
2665restorecon, load_policy, getenforce, setenforce, getprop, and setprop.
2666Promoted shred, nsenter, and hwclock.</p>
2667
2668<p>You can once again build catv now the flag infrastructure's been updated to
2669let it coexist with cat -v.
2670And on a long plane flight I wrote
2671hexedit, an interactive hex editor that implements the start of
2672cursor control infrastructure (for eventual use by less and vi and shell
2673command history and so on).</p>
2674
2675<p><b>New options:</b> Added sed -E as a BSD-compatible synonym for -r.
2676Upgraded oneit with -r (restart), -3 (send exiting PID values to child),
2677and signal handling. Added -v option to timeout, -m to mknod, -u to shred,
2678-t to dmesg, and -123 to head and tail. Added implicit "." to grep -r without
2679any files to work on. Hyejin Kim requested prefix support for truncate -s.
2680Greg Hackman added -inum to find.
2681Jan Cybulski added the smack side of ls -Z support. Various patches also
2682added -Z to mkdir, mknod, and mkfifo.
2683Basic cp --preserve support went in, but not yet the xattr/LSM parts.</p>
2684
2685<p>The toybox command now has a --version option,
2686which uses "git describe" if available.</p>
2687
2688<p><b>Build infrastructure:</b>
2689The "make change" target now saves the output of each failed standalone
2690command build in a .bad file, and "make defconfig" is quieter now.</p>
2691
2692<p>Paul Barker submitted a large patch changing command install paths so
2693"toybox can be installed alongside busybox without confusing
2694update-alternatives". (There's some argument over
2695what the right paths should be, and I'm waiting for
2696people to tell me what else needs fixing because I have no idea. I've
2697been symlinking /bin to /usr/bin since 2002
2698<a href=http://landley.net/writing/hackermonthly-issue022-pg33.pdf>for
2699historical reasons</a>.)</p>
2700
2701<p><b>Docs:</b> The repository link now goes to github, with another link
2702to the commit rss feed.</p>
2703
2704<p>Elliott Hughes updated the Android section of the roadmap
2705(and he would know). Redid bits of scripts/mkstatus.py to make updating
2706status.html easier, and the README is larger.</p>
2707
2708<p>More description of option parsing in code.html, which now describes the
2709FLAG_x macros, switching flag macro sets with FOR_newcommand, how
2710configuration zeroes flag macros and using FORCE_FLAGS to suppress the
2711zeroing of options shared between commands. Also added description of ";"
2712to make --longopts take an optional =value part, and more about TOYBOX_DEBUG
2713to check NEWTOY() option strings (otherwise a bad option string makes
2714lib/args.c obviously segfault, but doesn't explain why).</p>
2715
2716<p>Added a "Why 0BSD?" section to license.html when submitting zero clause bsd
2717to SPDX (according to the pending license spreadsheet, it's been approved for
2718SPDX 2.2).</p>
2719
2720<p>The old list of commands needing cleanup but not in pending was
2721removed from toys/pending/README and instead the issues were added
2722as TODO comments in the individual commands.</p>
2723
2724<p><b>Bugfixes:</b>
2725Fixed mount -a segfaulting without -O (reported by Janus Troelsen),
2726and made it try a "become rw" ioctl() on the block device before falling
2727back to mounting read only (because Android expects that).
2728Fixed printf -- and printf ---. Lots of tweaks to ls -l spacing with
2729different options. Make touch -d and -t actually set time when you don't
2730specify nanoseconds.
2731Fixed a subtle bug where recursive calls (toybox commands that run other
2732toybox commands) weren't resetting all their state. (This manifested as
2733a "no }" error from "find | xargs sed", but could cause other problems.)
2734And David Halls reported another sed bug trying to compile libiconv (which
2735left extra \ at the start of lines in a generated shell script, breaking
2736the build). Output an error message for "cat /mnt".</p>
2737
2738<p>Kylie McClain reported that mktemp broke when $TMPDIR was set to an empty
2739string (which is not the same as unset), that install/find didn't support
2740numeric uid/gids, and that sort -z affects both input and output.
2741Isabella Parakiss fixed a printf.c bug.
2742David Halls fixed bugs in install -D and find -exec. Samuel Holland
2743fixed unshare -r. Hyejin Kim fixed makedevs with a count of 1, fold -w
2744range checking, an error path in scripts/mkflags.c, added -i to dhcpd,
2745and stopped su from prompting the root user for the new user's password.
2746Jan Cybulski spotted wrong indentation when combining ls -s and -i with -C and
2747-x. José Bollo fixed stat %G. Sameer Pradhan fixed a bug in mkfifo -Z.</p>
2748
2749<p>Elliott Hughes asked for a default SIGPIPE handler to disable
2750the signal handler bionic's dynamic loader installs (yes really). Still not
2751100% sure what the correct behavior is there. (Posix is
2752(<a href=http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/10915>actively unhelpful</a>, but at least they're taking
2753<a href=http://austingroupbugs.net/view.php?id=789#c1976>years to
2754make up their mind</a>. Elliott also sent patches to fix a typo in
2755useradd.test, add missing arguments to error_exit() calls and clean up
2756printf() format strings, fix an off by one error in human_readable(),
2757fix dmesg -c error reporting, fix a segfault in comma_scan where the option
2758was the last item in optlist (triggered by mount -o ro,remount), fix
2759hwclock -w, made ifconfig print lowercase MAC addresses (it was bothering
2760him), and make terminal_size() read the right environment variable
2761(LINES, not ROWS). And he suggested the test suite notice high command exit
2762values (corresponding to segfault or other signals).</p>
2763
2764<p>People are apparently using toys/pending commands, despite the police tape
2765and flashing lights, so added louder warnings to toys/pending/README.
2766Elliott Hughes fixed various problems with tar, dd, more, and top.
2767Hyejin Kim cleaned up syslogd and dumpleases. Isaac Dunham added hotplug
2768support to mdev. Yeongdeok Suh added RFC-3315 ipv6 support to dhcpd.</p>
2769
2770<p>I rewrote ps.c from scratch (in pending), but it's not ready for real use
2771yet.</p>
2772
2773<p><b>Portability:</b>
2774On the portability front Bernhard Rosenkranzer fixed a problem where the
2775menuconfig code wouldn't compile in C99 mode. (This led to me documenting
2776the craptacular nature of kconfig in a README, and the plan to replace it
2777sometime before 1.0.) Some extra flags to shut up overzealous llvm warnings
2778were added (and have to be probed for because gcc complains about
2779arguments it doesn't recognize even when they switch stuff _off_ using
2780a standard syntax). Don't depend on malloc(0) to return non-null in ls.
2781David Halls fixed some mac/ios portability issues,
2782implying somebody's built at least part of toybox on a mac.</p>
2783
2784<p>Added basename_r() to lib/lib.c because the posix semantics for basename()
2785are stupid but what the gnu guys did to it was appalling.
2786Turns out bionic already had a basename_r(), but posix still doesn't.
2787Fixed it up in portability.h, but this
2788could break more stuff in future. (Correct fix is to lobby posix to add it,
2789which would probably take about 15 years...)</p>
2790
2791<p><b>Infrastructure:</b>
2792The build now checks $LDFLAGS for linker-only flags, and allows the strip
2793command to fail (binflt toolchains provide a strip that doesn't work).
2794Since time.c uses floating point, added TOYBOX_FLOAT dependency in config.</p>
2795
2796<p>There's a lib/lsm.h defining varous inline functions for linux
2797security modules stuff, if (lsm_enabled()) should turn into a compile-time
2798constant 0 and let code drop out when TOYBOX_LSM_NONE selected, but
2799testing against CFG_TOYBOX_LSM_NONE or derived symbols is still useful
2800becuase when it _is_ enabled the probe turns into a system call you
2801don't want to repeat too much.</p>
2802
2803<p>Switched a bunch of commands from signal() to xsignal(). Factored out
2804xgetgrnamid() and xgetpwnamid() into xwrap.c. Make time.c depend on
2805TOYBOX_FLOAT (since it always uses float so shouldn't be available on
2806build targets without even software float). Added readfileat() to lib/lib.c.</p>
2807
2808<p>The dirtree infrastructure now passes in full flags for the old symlink
2809field, and the new DIRTREE_SHUTUP flag disables warnings if a file vanishes
2810out from under you during traverse. New dirtree_start() wrapper to
2811create dirtree root with only two arguments.</p>
2812
2813<p>The not-curses infrastructure introduced by hexedit mostly moved to
2814lib/interestingtimes.c.</p>
2815
2816<a name="asterisk" />
2817<a href="#asterisk_back" />Asterisk:</a> such when
2818Tim contacted me (my blog says a couple days before nov 13, 2011, I.E.
281911/11/11 not some specific day 2 months later) to ask if I wanted to work
2820on a new project he was proposing called
2821<a href=http://www.elinux.org/Busybox_replacement_project>BentoBox</a>
2822(because I used to do busybox, he'd forgotten toybox existed
2823until I brought it up). And don't ask me what "focuses not on compatibility
2824with its GNU counterparts" means when CP_MORE adds 7 non-posix options
2825and toys/other has 84 commands in neither posix nor LSB. I think they're
2826struggling to explain the difference having dismissed "licensing" as being
2827the reason it started up again after a long hiatus? The reason I don't think
2828GNU is special is there are a half-dozen other independent
2829implementations of the same unix command tools out there (AT&amp;T,
2830BSD, Coherent, Minix, plan 9, busybox, toybox, and several more analyzed in
2831the <a href=roadmap.html>roadmap</a>, and that's ignoring the implementations
2832written for DOS or in assembly over the years). But I do care what
2833Linux From Scratch expects, and if it's
2834<a href=http://archive.linuxfromscratch.org/lfs-museum/7.6/LFS-BOOK-7.6-NOCHUNKS.html#ch-tools-gcc-pass1>calling mv -v</a>
2835then I impelement mv -v
2836even if <a href=http://landley.net/toybox/roadmap.html>posix hasn't got
2837it</a>. And I don't know why "gnu counterparts" would describe this when
2838util-linux isn't a gnu package, nor are info-zip, e2fsprogs, kmod, less,
2839procps, shadow, sysklogd, vim, zlib, sudo, dhcpcd...</p>
2840
2841<a name="05-04-2015" /><a href="#05-04-2015"><hr><h2><b>April 5, 2015</b></h2></a>
2842<p>Since <a href=https://android.googlesource.com/platform/external/toybox/>android</a> and
2843<a href=https://git.tizen.org/cgit/platform/upstream/toybox.git>tizen</a>
2844and <a href=https://github.com/kraj/meta-musl/tree/master/recipes-core/toybox>openembedded</a>
2845and <a href=https://packages.gentoo.org/package/sys-apps/toybox>gentoo</a>
2846and so on have all been using Georgi Chorbadzhiyski's git mirror rather
2847than the mercurial repository, I bit the bullet and switched the project's repo
2848<a href=https://github.com/landley/toybox>to git</a>. Georgi's
2849<a href=https://github.com/gfto/toybox>mirror</a> is now pulling from that.</p>
2850
2851<a name="25-02-2015" /><a href="#25-02-2015"><hr><h2><b>February 25, 2015</b></h2></a>
2852<blockquote><p>"A common mistake that people make when trying to design
2853something completely foolproof is to underestimate the ingenuity of
2854complete fools."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
2855
2856<p><a href=downloads/toybox-0.5.2.tar.gz>Toybox 0.5.2</a>
2857(<a href=/hg/toybox/shortlog/1702>commit 1702</a>) is out.</p>
2858
2859<p>New promoted commands: sed (finally fixed enough it builds Linux From
2860Scratch), printf (cleaned up and promoted), shred and
2861base64 (the Tizen guys wanted them), getenforce, setenforce, and chcon (android),
2862mix (promoted with fixes from Isaac Dunham), nsenter (from
2863Andy Lutomirski, merged into unshare).</p>
2864
2865<p>Elliott Hughes submited a bunch of patches to support Android (to
2866both toybox and Bionic libc, which he maintains). On toybox's end this
2867involved a lot of fixups to portability.[ch] and fixes to over a dozen
2868commands, plus several new ones. Other portability fixes included working
2869with buildroot's uclibc fork and building for nommu targets.</p>
2870
2871<p>The new "make change" target builds each toybox command as a standalone
2872binary. Rather a lot of commands that didn't build by themselves (mv depending
2873on cp and so on) were hit with a large rock until they built standalone.
2874This involved rewriting bits of option parsing, more elaborate dependency
2875generation, making each command have its own config
2876symbol and main() function (even when it's just a wrapper calling another
2877command's main()), and so on. Also, some commands can't be built standalone
2878at a conceptual level: "help" describes other enabled commands and "sh"
2879has a number of bulitin commands (cd, exit, set) that require the
2880multiplexer infrastructure, so "make change" filters them out.</p>
2881
2882<p>The mailing list's web archive is still screwed up. Dreamhost has
2883been trying to fix it since approximately September. There are
2884<a href=http://www.mail-archive.com/toybox@lists.landley.net/>two</a>
2885<a href=http://news.gmane.org/gmane.linux.toybox>other</a> less broken
2886archives, but neither has quite the same UI as mailman.</p>
2887
2888<h3>Bugfixes and tweaks</h3>
2889
2890<p>Cynt Rynt sent in tests for ifconfig,
2891Robert Thompson taught factor to accept whitespace separated arguments,
2892Hyejin Kim pointed out that some of mktemp's longopts were attached to
2893the wrong short options,
2894Luis Felipe Strano Moraes fixed a wrong free() call in bootchartd in pending.
2895Patches from Ashwini Sharma to make "df /dev/node" work, prevent du from
2896looping endlessly following symlinks, and to make expr.c
2897(in pending) understand == and regex matches. (Speaking of expr, it gets
2898priority groupings wrong but the bug was actually in the posix spec's
2899HTML conversion. They fixed the posix spec upstream for us. Still need
2900to fix the expr code, but it's in pending for a reason...)</p>
2901
2902<p>Some commands grew new option flags, such as cp --remove-destination
2903and touch -h.</p>
2904
2905<p>The parallel build has better error reporting now. When toybox needs to
2906re-exec itself to regain suid root permissions and hasn't got the suid bit,
2907it now gives the right error message ("not root" instead of "no such command").
2908
2909<p>Added a test to "mount" to not mount the same device/directory combination
2910over itself (the OS catches this for block devices, but not for tmpfs).
2911Make blkid distinguish ext3 from ext4. Added catv back into cat (because
2912the Android guys wanted it, and they have historical usage on their side,
2913so...). Handle nanoseconds in touch.</p>
2914
2915<p>Fixed a segfault when CP_MORE was disabled (the resulting option flag list
2916no longer defined -d but still had it in option groups at the end).
2917Workaround for glibc redefining dirname() and basename() to random non-posix
2918semantics because gnu. (They could have created dirname_r() but didn't want
2919to.)</p>
2920
2921<p>Fix an ifconfig test that was preventing assigning an ipv4 address to
2922interface aliases. Several cleanup passes on hwclock but not quite
2923promoted out of pending yet.<p>
2924
2925<p>Fixed a wrong error message in rm (if you had a chmod 000 directory and
2926did rm -r on it without -f, after the prompt it would complain it was a
2927directory, which was not the problem).</p>
2928
2929<p>The gzip compression code now does "store only" output to stdout, for
2930what that's worth.</p>
2931
2932<p>Cleanup mountpoint and expand, and remove them from toys/pending/README
2933(a list of commands that predate the toys/pending directory but needed
2934another pass).</p>
2935
2936<h3>Library and infrastructure:</h3>
2937
2938<p>Reworked the option parsing infrastructure so more commands build
2939standalone (via scripts/single.sh or "make change"). The option flag bit
2940values are no longer packed, it leaves spaces where currently disabled
2941flags go, and you can #define FORCE_FLAGS so disabled flags aren't zeroed.
2942This allows multiple commands to more easily share infrastructure, even if
2943your current flag context is for a disabled command (switched off in config),
2944you can force them to stay on and as long as the flags read the same right
2945to left they'll have the same values.</p>
2946
2947<p>We've started removing use of strncpy() because it's a hugely broken
2948standard C function: the length is the maximum length to _append_, not
2949the size of the destination buffer. It memsets the remaining space it didn't
2950copy ala "memset(dest+strlen(dest), 0, len);" so
2951if you think len is the size of dest you're guaranteed to stomp memory off the
2952end). And if it runs out of space it won't null terminate because reasons.
2953(Meanwhile sprintf("%*s", len, str) is counting wide characters in your current
2954locale, so if you set a locale other than "C" it will also go past your
2955allocated buffer size. Whoever is maintining the C library standards is really
2956bad at strings.)
2957Instead we have xstrncat() which will error_exit() if src+dest+1 doesn't
2958fit in the buffer. (Because randomly truncating input data isn't necessarily
2959an improvement.) And there's always xmprintf().</p>
2960
2961<p>Similarly, strtol() doesn't return an error indicator on overflow,
2962you have to clear and then check errno. So new xstrtol() that cares
2963about overflow.</p>
2964
2965<p>The bionic and musl guys agree faccessat(AT_SYMLINK_NOFOLLOW) is not
2966supported, so stop using it.</p>
2967
2968<p>Fixed toy_exec() to detect when argc is in optargs, so we don't
2969need a separate xexec_optargs().</p>
2970
2971<a name="18-02-2015" /><a href="#18-02-2015"><hr><h2><b>February 18, 2015</b></h2></a>
2972<p>Dreamhost continues to be unable to make mailing list archives work, so
2973here's <a href=http://www.mail-archive.com/toybox@lists.landley.net/>another
2974list archive</a> with a less awkward interface than gmane.</p>
2975
2976<p>(Neither gives you the convenient historical monthly views of mailman,
2977but I still have hopes dreamhost will someday figure out what they're doing
2978wrong. They've only been trying since October. Last month they did a
2979<a href=http://www.dreamhoststatus.com/2015/01/14/discussion-list-hardware-maintenance/>hardware upgrade to fix a software problem</a>, and the stale
2980data loads much faster now, so that's something.)</p>
2981
2982<p>Update (Feb 19): the archive started updating again, by discarding
2983all the pending data. So there are now _two_ giant holes in Dreamhost's
2984web archive, from Dec 15-Jan 3, and then another hole from Jan 16-Feb 18.
2985The relevant messages are in both of the other archives. Here's hoping
2986the chronic archive constipation problem won't happen a sixth time.</p>
2987
2988<a name="30-12-2014" /><a href="#30-12-2014"><hr><h2><b>December 30, 2014</b></h2></a>
2989<p>Due to Dreamhost's <a href=http://landley.net/dreamhost.txt>ongoing</a>
2990<a href=http://landley.net/dreamhost2.txt>inability</a> to make mailman
2991work reliably, I've added a link to a backup web archive at
2992<a href=http://news.gmane.org/gmane.linux.toybox>gmane</a> to the nav bar
2993on the left.</p>
2994
2995<p>You still subscribe to the list through
2996<a href=http://lists.landley.net/listinfo.cgi/toybox-landley.net>the first link</a>.</p>
2997
2998<p>Update (January 27, 2015): they're <a href=https://twitter.com/landley/status/558428839462703104>still working on it</a>.</p>
2999
3000<a name="19-11-2014" /><a href="#19-11-2014"><hr><h2><b>November 19, 2014</b></h2></a>
3001
3002<blockquote><p>"This time it was right, it would work, and no one would have to get nailed to anything."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3003
3004<p><a href=downloads/toybox-0.5.1.tar.bz2>Toybox 0.5.1</a>
3005(<a href=/hg/toybox/shortlog/1566>commit 1566</a>) is out.</p>
3006
3007<p>It's an interim release, mostly bugfixes. There are several new commands,
3008but they're all in pending.</p>
3009
3010<h3>Development</h3>
3011
3012<p>Finally implemented sed, which is still in pending because although
3013it's feature complete according to posix, and even passes the parts of
3014Busybox's sed test suite that aren't explicitly testing for gnu bugs we
3015don't want to copy, it's not yet good enough to build Linux From Scratch.
3016(The ./configure stages use very long sed scripts. 20 commits worth of
3017implementation and debugging, just under 1000 lines of code, and there's
3018still more to do. We're definitely up to some of the "fiddly" commands now.
3019Did you know "echo hello | sed p - -" segfaults gnu sed in Ubuntu 12.04?
3020Yeah...)</p>
3021
3022<p>Talked with the Tizen developers to follow up on their desire to
3023make toybox a part of the base Tizen system, and got a list of commands
3024to add to the roadmap. The tizen todo list is:</p>
3025
3026<blockquote><p>
3027wget, sha256*, gzip, gunzip, bunzip2, rsync, zdiff*,
3028less, ar, arch, base64, csplit, dir, fmt, join, 
3029nproc, shred, shuf, stdbuf, stty, test, tr, unexpand,
3030users, vdir, diff3, sdiff, dosfsck (fsck.vfat), awk, fdisk
3031</p></blockquote>
3032
3033<p>(Most of which was already on the todo list, but it helps prioritize.)</p>
3034
3035<p>Fixed md5sum and sha1sum on big endian systems (reported by James McMechan).
3036Andy Lutomirski fixed unshare's help text and option parsing,
3037and submitted nsenter (a tool to use setns(2)) to pending.
3038Isaac Dunham implemented acpi -ctV options, and spotted the bug that ls -d
3039was inappropraitely following command line symlinks without -H or -L (it
3040should act like ls -l does), and ls -F handles symlinks wrong too.
3041Lukasz Szpakowski sent in two bugfixes to tail.c. Cynt Rynt spotted an
3042unnecessary assignment in lib/password.c.</p>
3043
3044<p>Ashwini Sharma's team was as busy as usual, submitting tr, crontab, and
3045ipcrm, and hwclock to pending, more features to the pending ip.c, and a
3046pile of bugfixes (to chgrp, killall, ifconfig, insmod,
3047losetup, comm, cp, id, xwrap, netcat, modprobe, nohup...) mostly found by
3048static analysis. (These fixes are mostly to seldom-used codepaths like the
3049TOYBOX_FREE config option, but test coverage is always appreciated.) Ashwini
3050also suggested upgrading ln -f to leave the original target alone if link
3051creation fails, and reported that mv -f and -i weren't implemented (now fixed).</p>
3052
3053<p>New config option: TOYBOX_NORECURSE prevents xexec() from making internal
3054function calls (for nommu systems with a finite stack).</p>
3055
3056<p>The "toybox" multiplexer command no longer adds a trailing space to each
3057line of command names, so things like "./toybox | tr ' \n' '|'" to create
3058a grep pattern snippet are easier to do. (Why you'd want to is your business,
3059but the output is tidier now.)</p>
3060
3061<h3>Infrastructure</h3>
3062
3063<p>Isaac Dunham added Android support to portability.h, including compile
3064probes for functions missing from bionic-libc, and annotated the commands that
3065use those functions. We haven't really tested building against bionic,
3066but in theory it's possible now.</p>
3067
3068<p>Running the test suite now color codes the PASS/SKIP/FAIL notifications
3069if output is to a tty. (And in case you missed it last time, VERBOSE=fail
3070to stop at the first failure is really useful.)</p>
3071
3072<p>In loopfiles_rw() use O_CLOEXEC instead of O_RDONLY to request the loop
3073function close filehandles for us. (Otherwise the callback function must
3074close each supplied filehandle itself.)</p>
3075
3076<p>The printf-style escape parsing ("\n" and friends) got factored out into
3077a new unescape() function.</p>
3078
3079<a name="02-10-2014" /><a href="#02-10-2014"><hr><h2><b>October 2, 2014</b></h2></a>
3080<blockquote><p>"There is an art, it says, or rather, a knack to flying.
3081The knack lies in learning how to throw yourself at the ground and miss...
3082Clearly, it is this second part, the missing, which presents the
3083difficulties."</p><p>- The Hitchhiker's Guide to the Galaxy<p></blockquote>
3084
3085<p><a href=downloads/toybox-0.5.0.tar.bz2>Toybox 0.5.0</a>
3086(<a href=/hg/toybox/shortlog/1512>commit 1512</a>) is out.</p>
3087
3088<h3>New commands</h3>
3089
3090<p>The new commands are find, install, factor, and mount. Promoted commands
3091(cleaned up and moved out of "pending") are lspci, inotifyd, and blockdev.</p>
3092
3093<p>cp now implements -HL and -F to force delete of pending files, cpio now
3094ignores -m and implements -p, ls -C now has utf8 support (using wcwidth
3095instead of strlen), and umount got a number of upgrades involving
3096looking things up in /proc/mounts. Other minor cleanups happend to
3097cut, touch, free, and id.</p>
3098
3099<p>In pending: Bradley Controy submitted mix (adjusts OSS sound volume). Ashwini
3100Sharma submitted diff, userdel, blockdev, ipcs, and crond, upgraded
3101fdisk, fsck, and ftpget, and ran a static analyzer on a lot of other code.
3102Partial cleanup was done to useradd, userdel, groupadd, and groupdel.</p>
3103
3104<h3>Build infrastructure</h3>
3105
3106<p><b>Parallel builds</b></p>
3107
3108<p>The build now takes advantage of SMP, autodetecting the number of
3109processors. (Export the environment variable CPUS to pick a specific number.)
3110Other build changes: split out $LDOPTIMIZE because old compilers complain
3111about linker options passed with -c, and the entire "generated" directory now
3112gets deleted by clean (the README that was in there got merged into code.html).</p>
3113
3114<p><b>Standalone builds</b></p>
3115
3116<p>The standalone build infrastructure (scripts/single.sh) got upgraded to
3117build more commands as standalone executables. In make.sh the source file
3118selection uses a regex to find the source files with the NEWTOY/OLDTOY macro
3119for the command. It enables each command's
3120sub-options (so CP has CP_MORE), enables I18N and FLOAT support to build
3121full-featured commands, and includes --help text (at least when
3122the command doesn't use another command's help). The OLDTOY() macro
3123now produces (redundant) function prototypes so you can build an OLDTOY
3124without the NEWTOY</p>
3125
3126<p>It doesn't quite have complete coverage yet, the defconfig entries that
3127aren't building standalone yet are:</p>
3128
3129<blockquote><p>chown, egrep, fgrep, fstype, halt, mv, nc, poweroff, unix2dos,
3130whoami</p></blockquote>
3131
3132<p>The main reason for standalone build failures is NEWTOY() or OLDTOY()
3133entries that don't have their own config symbol. Another problem is entries
3134that depend on another entry in kconfig, usually because common infrastructure
3135is using one command's flags (which the other commands copy): if that command
3136is disabled, the FLAG macros become 0 so dead code elimination can remove the
3137code. It's <a href=http://landley.net/hg/toybox/rev/1503>possible
3138to untangle</a> this, but a bit awkward. (It boils down to conflicting
3139design goals in the two contexts.)</p>
3140
3141<p>Standalone builds are used by the test suite when testing individual
3142commands.
3143
3144<p><b>Snapshot builds</b></p>
3145
3146<p>A new addition to the "generated" directory is generated/build.sh
3147containing a single compiler command line to build toybox in its current
3148configuration. Combined with the generated/*.{h,sh} files from an
3149exisiting build, this may let you build on a new system that hasn't quite
3150got enough OS bits working to run a full configureand make.</p>
3151
3152<h3>Internals</h3>
3153
3154<p>Library code: xcreate/xopen now O_CLOEXEC by default to avoid leaking
3155filehandles to child processes. DIRTREE_COMEAGAIN's second callback is now
3156done with the directory filehandle still open (new dir->again variable added
3157to distinguish first from second callback, and requesting DIRTREE_RECURSE now
3158requires passing in the specific macro value, not just a true/false).
3159Use daemon() out of libc instead of hand-rolled daemonize() in various
3160pending commands. string_to_mode() now passes through type bits so you can
3161use it to more easily modify a file's existing mode.
3162Split xpopen() into xpopen_both(), xopen(), and xrun() depending on whether
3163we want to redirect both, one, or neither of stdin/stdout.</p>
3164
3165<p>Bugfixes: Better error message when TOYBOX_SUID option can't drop priviliges
3166(which happens when you suid something _other_ than root).
3167The old pending version of nbd_client.c wasn't deleted when the
3168command was promoted (and the build would break if both were enabled),
3169toy_exec() sometimes needs to re-exec from $PATH rather than recurse
3170internally (to gain dropped root permissions or limit stack depth),
3171always call setlocale() when I18N is enabled to switch it back _off_ when
3172we run commands that expect sscanf("%n") to return bytes,
3173dirtree() had a memory leak in an error path, patch.c had some bugs in
3174error paths (didn't report problem clearly). Ashwini Sharma spotted an
3175option parsing bug where [-abc] would forget _all_ command line arguments
3176saved in the GLOBALS() block (not just the ones for options being switched
3177off), plus various minor fixes to nbd_client and cpio.
3178Lukasz Szpakowski fixed rm -f on a broken symlink (failed), and killall
3179with no arguments (segfaulted).</p>
3180
3181<p><b>Portability</b></p>
3182
3183<p>A somewhat fiddly fix to rm -rf (which needs to chmod directories to u+rwx
3184to descend into them) which hit a musl bug in faccessat() which the musl
3185maintainer refuses to fix. (He literally wants the man page changed
3186instead, despite other libcs working.) Added an #ifdef __MUSL__ section
3187to portability.h with a workaround, you may need CFLAGS=-D__MUSL__ in your
3188build if your musl build's features.h doesn't #define that. (I may do
3189a different workaround in future, but sometimes you've just got to make
3190it work so you can ship. Also, toybox grep with multiple patterns
3191requires <a href=http://landley.net/hg/aboriginal/rev/1692>a patch
3192to musl's regex engine</a>, which applies to 1.1.4 but not to the current
3193musl source control.)</p>
3194
3195<p>More portability.h fixes for uClibc too. (I don't expect that to ever have
3196another release, so locally patching around posix-2008 violations is silly).</p>
3197
3198<p><b>Change to username filtering</b></p>
3199
3200<p>Posix recommends the username creation logic filter usernames to a small
3201allowed set of characters (which even Red Hat breaks by explicitly allowing
3202"$" at the end), but this prevents UTF-8 usernames. Posix' stated logic
3203is to allow filesystems to create the user's home directory, but Linux
3204filesystems can accept any character but NUL and "/". The only characters
3205we actually _need_ to filter out are ":" (field separator in passwd),
3206newline (line separator in passwd), and "/" (directory separator in
3207filesystem).</p>
3208
3209<h3>Documentation</h3>
3210
3211<p>Web pages updated: cleanup.html documents more cleanup, code.html
3212documents more code, and about.html now capitalizes "toybox" consistently
3213(it's just a word, capitalize at start of sentence).</p>
3214
3215<p>The pending/README file now lists commands that needed review/cleanup
3216before the pending directory was added.</p>
3217
3218<h3>Test Suite</h3>
3219
3220<p>Moved out of scripts/test into top level "tests" directory, and the
3221testing.sh script is now in scripts rather than mixed into the *.test files.</p>
3222
3223<p>Johan Bergström requested VERBOSE=fail to make tests (telling it to
3224stop at the first failure), and spotted a build bug where using gnu
3225sort on the host broke in non-C locales.</p>
3226
3227<p>Divya Kothari submitted tests for chmod, link, tar, bzcat, xzcat, zcat,
3228and hostname. (And more, but that's all that's merged so far.)</p>
3229
3230<a name="07-07-2014" /><a href="#07-07-2014"><hr><h2><b>July 7, 2014</b></h2></a>
3231<blockquote><p>"This planet has - or rather had - a problem, which was this:
3232most of the people living on it were unhappy for pretty much of the time. Many
3233solutions were suggested for this problem, but most of these were largely
3234concerned with the movement of small green pieces of paper, which was odd
3235because on the whole it wasn't the small green pieces of paper that were
3236unhappy."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3237
3238<p><a href=downloads/toybox-0.4.9.tar.bz2>Toybox 0.4.9</a> (<a href=/hg/toybox/shortlog/1385>commit 1385</a>) is out.</p>
3239
3240<p><b>New commands</b> added to pending include:
3241lsattr, chattr, inotifyd, rfkill, sulogin, strings, makedevs,
3242killall5, and tar from Ashwini Sharma, arp from Kyungwan Han,
3243sysctl by Bilal Qureshi, partprobe from Bertold Van den Bergh,
3244host from Rich felker, and I did nbd-client and the first 2/3 of mount.</p>
3245
3246<p>Finished cleanups (commands promoted out of pending):
3247sysctl, rfkill, strings, mkpasswd, makedevs, partprobe, killall5,
3248fallocate, and nbd-client.</p>
3249
3250<p>(Along the way partial cleanups got made to: last, fold, lspci, ps,
3251bootchartd, init, fsck, telnetd, telnet, vconfig, toysh, iconv, useradd,
3252login, host, openvt, deallocvt, getty, tftpd, and modprobe. But there's
3253still more to do on all of those.)</p>
3254
3255<p>This time around the <a href=bin>static binaries</a> are linked against
3256musl instead of uClibc. (That's why there's no sparc version, musl doesn't
3257support that target yet.)</p>
3258
3259<p><b>Documentation:</b></p>
3260
3261<p>The help text parser expects lower case "usage:" lines with
3262a blank line after them, so go through and regularize those. Expand the
3263"coding style" section in the docs and move it to design.html. (Not a show
3264stopper for incoming
3265contributions, just an explanation of some of the things I'll do to them
3266during cleanup.) The help text for the "toybox" command now includes
3267the shell script snippet to install symlinks to the toybox binary.</p>
3268
3269<p>The <a href=cleanup.html>cleanup page</a> now has descriptions for the
3270full ifconfig cleanup series, among others.</p>
3271
3272<p>The new toys/examples directory contains hello.c and skeleton.c. The first is
3273a simple hello world program in toybox style, the second is a much more
3274elaborate example program using showing how to use the command line option
3275parsing and how to provide multiple commands in the same C file.</p>
3276
3277<p><b>Fixes</b>:</p>
3278
3279<p>Fix od bug reported by Samuel Holland ("od -v -b" was appending the default
3280output type even though an output type was specified). Ashwini Sharma reported
3281bugs where readfile() was incorrectly freeing its buffer, and where toy_init()
3282was zeroing the wrong data because the field it was using to measure (rebound)
3283had moved (when I moved it back I added a comment why the field needs to be
3284there), fixed a segfault in the dhcp client, and made a 0 length read at
3285the start of password entry count as EOF. Make the "we are not root" test
3286in the init code show the help text. Posix implies that fflush() can return
3287success even when the stream's error bit is set, so call both fflush() and
3288ferror() from xprintf().</p>
3289
3290<p>Isaac Dunham pointed out that bloatcheck couldn't deal with diff
3291implementations that only implement "unified diff" format, and that some
3292diff implementations can't handle nonseekable input (I.E. reading from
3293a pipe). Bugfix so "help -a" works again. Option parsing on nohup now stops
3294at first nonoption argument. Fix segfault in "which" if PATH wasn't set,
3295which was actually a bug in lib function find_in_path(). Made rm -rf of
3296chmod 000 directories actually remove them.</p>
3297
3298<p>The build now passes the same $CFLAGS to the library probe as the final
3299build, because arch linux is so broken it provides different sets of
3300libraries for static and dynamic linking.</p>
3301
3302<p>It turns out sprintf("%.123s", str) is counting characters, not bytes,
3303so globally enabling locale support opens stack smashing vulnerabilities.
3304So there's a new TOYFLAGS_LOCALE you set in toyflags when you want the
3305setup code to setlocale().</p>
3306
3307<p><b>Upgrades:</b></p>
3308
3309<p>Isaac Dunham extended cpio to archive unreadable empty files, and I taught it
3310to set uid/gid and timestamp when extracting archives. Isaac also
3311added tests for cpio, link, and du, added lspci -i, made the pci database
3312parsing skip # comment lines, merged logname and whoami into id.</p>
3313
3314<p>Daniel Verkamp sped up md5sum about 30% with some loop unrolling, making
3315it actually smaller in the process. I added -b flags to md5sum and sha1sum
3316for "brief" output that's just the hash with no filename. (I'm aware other
3317implementations use that for MSDOS "binary" mode, and don't care.)</p>
3318
3319<p>When building standalone commands (scripts/singleconfig.sh commandname),
3320the build now switches on all the sub-options of the command so we get
3321a standalone version with all the bells and whistles enabled.</p>
3322
3323<p>Add -ds flags to date and document +FORMAT escapes. Add the shell NOP
3324command ":" as an alias for true (for toysh).</p>
3325
3326<p>Add uClibc probe for iconv() and fallocate. (The fact it didn't always
3327build against uClibc is why fallocate wasn't enabled in defconfig before.)</p>
3328
3329<p>The umount command now does an losetup -d on the device by default, so
3330we don't leak loopback devices. Bugfix to losetup so "losetup /dev/loop0
3331filename" actually works again.</p>
3332
3333<p>Divya Kothari sent in test suite entries for ls, ln, rm, mv, printf, dd,
3334and renice. Then a second round for lsattr/chattr, mount, chmod, pgrep/pkill,
3335groupadd, groupdel, and useradd. Several of these uncovered bugs, still
3336working to fix them.</p>
3337
3338<p>There are now free() functions for the predefined llist types and a
3339dlist_terminate() function to break doubly linked lists. The new
3340generic_signal() handler either sets "toys.signal" or writes a byte
3341to toys.signalfd with the signal number if signalfd isn't -1 (which it's
3342initialized to in toy_init).</p>
3343
3344<p>The option parsing logic can now detect when a double fits in a long and
3345use the more precise type for floating point arguments (the FLOAT macro
3346contains the type used). The human_readable() function now just outputs
3347decimal kilo/mega/gigabytes (so when du -u says 5.0G it means 5.0 billion
3348bytes). The build infrastructure now notices duplicate commands (so if you
3349cp toys/pending/command.c toys/other/command.c and forget to delete the
3350first one, the build break is now more informative).</p>
3351
3352<a name="20-04-2014" /><a href="#20-04-2014"><hr><h2><b>April 20, 2014</b></h2></a>
3353<blockquote><p>And to this end they built themselves a stupendous supercomputer
3354which was so amazingly intelligent that even before the data banks
3355had been connected up it had started from "I think therefore I am" and got as
3356far as the existence of rice pudding and income tax before anyone managed to
3357turn it off.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3358
3359<p><a href=downloads/toybox-0.4.8.tar.bz2>Toybox 0.4.8</a> is based on
3360<a href=http://landley.net/hg/toybox/shortlog/1262>commit 1262</a>. And
3361about time too.</p>
3362
3363<p>The big news is that the build no longer needs python to generate help.h,
3364that's now done in C. The help text generation is also collating help text
3365from multiple options, merging command line option blocks and usage: lines.
3366There's even a new <a href=help.html>help web page</a>.</p>
3367
3368<p><b>New commands:</b> Ifconfig, cpio, and su were cleaned up the rest of the
3369way and promoted out of pending. That saga is mostly explained on the
3370<a href=cleanup.html>cleanup page</a>. Vivek Bhagat's freeramdisk,
3371Isaac Dunham's fsfreeze, and Felix Janda's iconv are also new.</p>
3372
3373<p><b>In pending:</b>
3374Ashwini Sharma's team submitted tcpsvd, udpsvd, telnet, telnetd, last, more,
3375groupdel/delgroup, arping, brctl, ftpget, ftpput, printf, reset, and added
3376ipv6 support to traceroute. Kyungwan Han's team submitted modprobe and getty.
3377Vivek Bhagat submitted openvt and deacllocvt. Samuel Holland submitted fold.
3378I wrote a new inflate (zip/zlib/gzip decompression) implementation in
3379compress.c, and still  need to do a corresponding deflate (compression-side)
3380and plug them into gzip and zip and so on. (Right now it does zcat.)</p>
3381
3382<p>Several commands (vmstat, login, du, vconfig, mountpoint, free, chroot,
3383cut, touch, modinfo, expand) predate the "pending" directory, and are thus
3384in other directories but still need cleanup. Of these, vmstat got some
3385work this time (which would be much easier other vmstat implementations
3386documented what their output actually meant).</p>
3387
3388<p><b>Upgrades:</b> Ifconfig grew /prefix netmask support (ala 1.2.3.4/24). Grep now has -zZ to
3389handle null terminated data, cksum grew -H for hex output. Upgraded od so the
3390fields align better when producing multiple output types. Help has -a and -h
3391options (all commands, html output).
3392Bugfix to blkid building for a 32 bit target. The date command can actually
3393set dates now. The O_NOFOLLOW compile time probe didn't work with cross
3394compiling, so it's back to an #ifdef test in portability.h. Nathan McSween
3395sent in a bugfix to od and a portability fix in the common library code.
3396Ashwini Sharma spotted a bug in pidof -o, and added verbose (-v) options
3397to mkdir and ln, and suggested killall should have an -s option and
3398allow -l to take zero arguments. Ashwini Sharma and Felix Janda upgraded
3399tftpd.  Fixed dumpleases still using toynet.h after
3400that was removed. Corrected killall return code and error reporting.
3401Isacc Dunham fixed bugs all over the tree, did cleanup on a bunch of
3402pending commands (getty, ftpget, init, openvt, modprobe...), and clarified
3403find's help text. Tom Sparrow ran three different static analyzers on
3404the code, which resulted in a few cleanups. The peek()/poke() functions
3405now use "volatile" to prevent broken compiler "optimizations" to do with
3406aliasing.</p>
3407
3408<p><b>Build stuff:</b> Each FOR_xxxx macro now has a complementary CLEANUP_xxxx macro, so you
3409can put multiple commands with different command line options in the same
3410.c file, so they can share infrastructure outside of lib. (This let the
3411bunzip logic move out of lib into bzcat.c.) See XXX for example.
3412i
3413<p>The headers #included in toys.h are now grouped by standard, and headers
3414not listed in Posix or LSB were moved to portability.h. The old xregcomp.h
3415was folded into lib.h because it's posix (and supporting oddball uClibc
3416configurations isn't as important as it once was).</p>
3417
3418<p>Regression tested against Ubuntu 8.04 to fix up bit-rot in defconfig
3419build on older systems. (We depend on Posix-2008, but not necessarily
3420the absolute latest build environment.)</p>
3421
3422<p><b>In lib</b>: lib/xwrap.c added xgetpwnam(), xchroot(), and lib/lib.c now has names_to_pid().
3423xsetuid() was replaced with xsetuser() which takes a struct passwd
3424and sets both gid and uid, mkpathat() got factored out into a library command,
3425get_int_value() became atolx_range(), and
3426xmsprintf() is now just xmprintf(). The bunzip2 logic moved from lib into
3427bzcat.c.</p>
3428
3429<p><b>Documentation</b>: new <a href=help.html>help page</a> with the
3430help text for all the defconfig commands, using the new help -ah output.
3431The <a href=code.html>source code walkthrough</a> now says more about
3432#including header files, and how the generated/* directory works. The
3433<a href=design.html>design page</a> has some new paragraphs about trading
3434of different kinds of simplicity, and why comments aren't a substitute for
3435good code. The README no longer trails off into obvious unfinished confusion
3436at the end. Each page on the website should now have its own title.</p>
3437
3438<a name="18-11-2013" /><a href="#18-11-2013"><hr><h2><b>November 18, 2013</b></h2></a>
3439<blockquote><p>"Space," it says, "is big. Really big. You just won't believe how vastly, hugely, mindbogglingly big it is. I mean, you may think it's a long way down the street to the chemist's, but that's just peanuts to space."</p><p>-
3440The Hitchhiker's Guide to the Galaxy</p></blockquote>
3441
3442<p><a href=downloads/toybox-0.4.7.tar.bz2>Toybox 0.4.7</a> is based on
3443<a href=http://landley.net/hg/toybox/shortlog/1122>commit 1122</a>.</p>
3444
3445<p>New commands: Brad Conroy submitted blkid. Elie De Brauwer submitted
3446reboot, halt, and poweroff. Strake's nl got cleaned up and promoted from
3447pending to posix. In addition, the existing chvt and vconfig got some
3448cleanup.</p>
3449
3450<p>That said, I haven't nearly kept up with the flood of new commands going
3451into pending: Ashwini Sharma's team submitted
3452dd, dumpleases, traceroute, top, useradd, groupadd, mkpasswd, tftpd, and
3453an fsck wrapper (with no filesystem drivers yet). Isaac Dunham sent in cpio.</p>
3454
3455<p>Bugfixes: Jeroen van Rijn added a user count to uptime. Elie De Brauwer
3456added -e to watch, removed a memory leak, and fixed a terminal size problem.
3457William Haddon made xargs call its command line once even with blank input
3458(the standard is vague, but builds expect it), and fixed an off by one bug
3459where grep didn't malloc enough space with -E (leading to a segfault).
3460I fixed a glitch in bunzip2 (same one as went into busybox since they're using
3461the code I wrote), in od to fix -t co, -J, and -c options. Add uname -o as a
3462synonym for -s. Build fix to never use $CC without prefixing it with
3463$CROSS_COMPILE (since $HOSTCC could be different). Anca Emanuel spotted
3464a typo in the web page.</p>
3465
3466<p>The compile-time command line option parsing got rewritten (ported from
3467bash to C), which should speed up builds a bit and allow code controlled by
3468--longopts to drop out properly when disabled in the configuration. Terminal
3469querying got refactored. Patch's -x option is now more informative (a
3470debug thing if you're trying to figure out why a patch didn't apply).
3471The "toynet.h" file got folded into toys.h since musl supports it and
3472micromanging uClibc options isn't very interesting anymore. The test suite
3473now uses scripts/single.sh when testing a single command.</p>
3474
3475<a name="17-09-2013" /><a href="#17-09-2013"><hr><h2><b>September 17, 2013</b></h2></a>
3476<blockquote><p>"Think of a number," said the computer, "any number."
3477Arthur told the computer the telephone number of King's Cross railway
3478station passenger inquiries, on the grounds that it must have some function,
3479and this might turn out to be it.</p><p>- The Hitchhiker's Guide to the Galaxy</p>
3480</blockquote>
3481
3482<p><a href=downloads/toybox-0.4.6.tar.bz2>Toybox 0.4.6</a> is based on
3483<a href=http://landley.net/hg/toybox/shortlog/1068>commit 1068</a>.</p>
3484
3485<p>This release adds
3486several new commands: Felix Janda wrote paste and fallocate, Kyungwan Han
3487submitted eject, Strake contributed grep, Ashwini Sharma added pmap (and
3488a testsuite entry for grep), Lukasz Skalski sent pwdx, Isaac Dunham posted
3489acpi, and I did timeout and umount.</p>
3490
3491<p>The ls command now has a --color=auto option (suggested by Rich Felker).
3492The multiplexer now has a --help option so you can say "./toybox --help blah"
3493instead of using the built-in "help" command. (Which is a shell built-in.
3494Try it on your command line, it's like man for shell builtins. But a certain
3495other project has conditioned people to expect --help, so...) I forget who
3496heehooman at gmail is but they pointed out unshare needed PID and UID
3497namespace support.</p>
3498
3499<h3>Pending</h3>
3500
3501<p>A lot of new commands in toys/pending, to the point the next release should
3502probably just focus on cleanup and review of this backlog. We've got klogd,
3503dhcp, dhcpd, watch, route, and ps from
3504Ashwini Sharma (and an fsck wrapper but no fsck.fstype engines yet),
3505syslogd, pgrep, and pkill from Madhur Verma, netstat by Ranjan Kumar,
3506test by Felix Janda, lspci by Isaac Dunham, nl, su, and renice by strake (I.E.
3507M. Farkas-Dyck), and sysvinit by Kyungwan Han.</p>
3508
3509<p>Some cleanup work on existing pending commands that aren't
3510ready to promote yet: I did a few more rounds on ifconfig
3511and Isaac Dunham's did several cleanups to xzcat, Felix Janda cleaned up
3512logger and syslogd...</p>
3513
3514<p>Also some cleanup work on commands that predate the pending directory,
3515but weren't quite polished when they went in, most prominently du,
3516expand, and touch.</p> 
3517
3518<h3>Infrastructure</h3>
3519
3520<p>The new scripts/single.sh builds a standalone command without the
3521multiplexer, although not all commands can be built that way yet (NEWTOY yes,
3522OLDTOY no) and the space savings aren't anything to write home about. (If a
3523command needs the option parsing logic at all, it needs all of it.) If
3524you're curious, you can do:</p>
3525
3526<blockquote><pre>
3527make defconfig
3528make
3529mkdir singles
3530for i in $(./toybox)
3531do
3532  echo $i
3533  PREFIX=singles/ scripts/single.sh $i || break
3534done
3535</pre>
3536<p>(And then wait a long time and watch almost half the builds fail.)</p>
3537</blockquote>
3538
3539<p>There is now libbuf analogous to toybuf, another global 4k buffer this
3540time for use by lib/ code instead of command code.</p>
3541
3542<p>The lib directory got split up a bit, lib/pending.c contains functions
3543not yet used by anything outside of toys/pending/*, and lib/xwrap.c contains
3544functions that wrap other functions and handle failures (via error_exit).
3545This leaves lib/lib.c containing actual new functions.</p>
3546
3547<p>General improvements and bug fixes to argument parsing. The [-abc] exclude
3548logic should now clear arguments slots when disabling options. Bare --longopts
3549should work now and be able to report errors using their name, the new ;
3550option allows optional arguments to longopts only suppliable with = (I.E.
3551--color and --color=auto but not --color auto).</p>
3552
3553<p>I'm gradually weaning the code off of itoa()/utoa() because sprintf
3554does this already. In this case "simple" probably means "let libc do it
3555for us".</p>
3556
3557<p>Rewrote for_each_pid_with_name_in() and renamed it to just names_to_pid().
3558It shouldn't get confused trying to compare absolute and relative paths quite
3559so much anymore.</p>
3560
3561<p>lib/llist.c grew a new dlist_pop() function for removing a doubly
3562linked list entry while maintaining a circular list; tail and patch are
3563using it now.</p>
3564
3565<p>The musl guys suggested a new optimization flag
3566(-fno-asynchronous-unwind-tables)
3567that shaves about 10% off the binary size by removing a C++ism that crept
3568into gcc's idea of C. While I don't normally try to micromanage the compiler,
3569"-fstop-being-stupid" is a thing you have to hit gcc with from time to time.</p>
3570
3571<h3>Bugfixes</h3>
3572
3573<p>Felix Janda and I did a largeish rewrite of tail to
3574finally make it work right (we think). Still need to implement tail -f
3575someday (the tricky bit is making -f follow multiple files at once).
3576Felix also reported a bug in xpidfile.</p>
3577
3578<p>Juhani Haverinen pointed out that
3579python 3 doesn't work with config2help.py, so the detection logic looks
3580for python2 (until I get around to rewriting that in C). Elie De Brauwer
3581then fixed our first attempt at this, and also fixed uname's help string.</p>
3582
3583<p>Ashwini Sharma
3584pointed out the build was making a FLAG_ macro for " " which broke some
3585configurations. (That's a control character, not a command line option.)</p>
3586
3587<p>Jacek Bukarewicz pointed out a bug in chdir permission handling, and
3588a way to make env segfault. Both should be fixed now.</p> 
3589
3590<p>The new function xexec_optargs()
3591replaces calls to xexec(toys.optargs) to avoid freeing and reusing optargs
3592during option parsing screwing stuff up (such as netcat's exec mode).</p>
3593
3594<p>The stat command's %a output was padded with leading zeroes, which
3595didn't match anybody else's behavior and thus made the test suite hiccup
3596between TEST_HOST and testing toybox. (If you go "TEST_HOST=1 scripts/test.sh
3597command" it sanity checks the tests against the host implementation.)</p>
3598
3599<p>Last release, "mkdir sub/sub && chmod 007 sub/sub && rm -rf sub" didn't
3600delete sub and didn't exit with an error either. Neither was correct, rm
3601should now be fixed.</p>
3602
3603<p>
3604<a name="26-07-2013" /><a href="#26-07-2013"><hr><h2><b>July 26, 2013</b></h2></a>
3605<p>Georgi Chorbadzhiyski maintains a <a href=https://github.com/gfto/toybox>git
3606mirror</a> of the repository on github, automatically updated from the
3607mercurial every 6 hours. The mirror is read only, but you can generate patches
3608against it and post them to the list.</p>
3609
3610<a name="02-07-2013" /><a href="#02-07-2013"><hr><h2><b>July 2, 2013</b></h2></a>
3611<blockquote><p>"Time is an illusion. Lunchtime doubly so." "Very deep. You
3612should send that in to the Reader's Digest. They've got a page for people
3613like you."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3614
3615<p><a href=downloads/toybox-0.4.5.tar.bz2>Toybox 0.4.5</a> is based on
3616<a href=http://landley.net/hg/toybox/shortlog/941>commit 941</a>. It adds
3617uuencode and uudecode from Erich Plondke, and enables Luis Morales' "who" by
3618default. Felix Janda and I cleaned up last year's "stat" submission and
3619enabled it. Ivo van Poorten added "groups".
3620Andre Renaud added "lsusb". I implemented "split", "pivot_root", and "mv".
3621</p>
3622
3623<p>The "help" command is implemented differently now (lib/help.c) and
3624each command can now understand --help (including both "toybox --help"
3625and "toybox --help command" in the multiplexer).</p>
3626
3627<p>The "pending" directory has several commands (find, xzcat, nbd-client,
3628logger, expr) which work but are not enabled by default pending further cleanup.
3629Ifconfig is enabled, but still in pending because it's only 2/3 cleaned up.
3630(It's an awkward halfway state but I'm not holding up the release for it.)</p>
3631
3632<p>I'm <a href=cleanup.html>documenting the cleanups</a> to teach
3633more people to do it, but the writeups aren't caught up yet. The
3634<a href=roadmap.html>roadmap</a> also got updated a bit with further analysis
3635of other projects, and the README and about pages got updated.</p>
3636
3637<p>Fixed _another_ "ls -C" segfault when terminal size can't be detected,
3638condensed the ls help text to fit on one page, implented --color, and taught
3639-l to print the major, minor numbers when showing block/char devices.
3640Argument parsing now handles "--" properly (to end option checking),
3641and the infrastructure can now handle bare --longopts that have no
3642corresponding short option (both were implemented before but didn't work).
3643Fixed an old bug in "patch", chmod grew -f, who grew -a. Isaac Dunham
3644fixed "-" vs "_" handling in modinfo, added a "firmware" output
3645field, added -b and -k support, and taught it that the ".ko" extension means
3646to look for the file at the specified path instead of under /lib. Felix Janda
3647moved file permission display code to lib so ls and
3648stat could share it. Ashwini Sharma spotted a bug in xabspath when the
3649last path component exists but we haven't got permissions to open it
3650(ala readlink -f /dev/sda as a normal user).
3651</p>
3652
3653<p>In the build infrastructure, scripts/findglobals.sh finds leaked global
3654variables. (Leaked means they aren't part of the global union: Other than glibc
3655debris, toybox should define "this", "toy_list", "toybuf", and "toys", and
3656that's it; the rest add memory footprint to every command for the benefit of
3657just one command; use GLOBALS() to stick 'em in the union.) Static linking
3658against libraries other than the host's libc now applies to feature probes
3659for unshare and such. Neuter stupid internationalization support that makes
3660various host "sort" commands put things in an order other than alphabetical
3661(breaking the multiplexer's binary search on command names).
3662
3663<p>You should now be able to build from a source control snapshot on a build
3664system that hasn't got python: if you disable CONFIG_TOYBOX_HELP. (The
3665release tarballs ship generated/help.h, but it's not in source control.
3666Eventually I should rewrite that python script in C.)</p>
3667</p>
3668
3669<p><b>LICENSE TWEAK</b>: After <a href=http://lists.landley.net/pipermail/toybox-landley.net/2013-March/000794.html>discussion</a> on the mailing list the "2 clause
3670BSD" <a href=license.html>license</a> got slightly simplified so the first
3671paragraph now says:</p>
3672
3673<blockquote><p>Permission to use, copy, modify, and/or distribute this
3674software for any purpose with or without fee is hereby granted.</p></blockquote>
3675
3676<p>It used to continue "provided that the above copyright notice and this
3677permission notice appear in all copies", but A) what's the point? B) does "all
3678copies" mean binaries, or just source code, or what? C) lots of projects
3679that consider BSD and GPL compatible have <a href=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/crypto/aes_generic.c>files with
3680both license notices</a> on them (sometimes at <a href=http://git.busybox.net/busybox/tree/shell/ash.c>opposite ends of the file</a> to make the conflict
3681less obvious) because "all copies must include this function" would violate
3682the GPL but "all copies must include this magic text blob" somehow don't?</p>
3683
3684<p>I don't want to have to care about this anymore. The tweaked version is more
3685or less public domain with a liability disclaimer, but we're still calling it
3686BSD (sometimes "0 clause BSD") to avoid explaining.</p>
3687
3688<a name="21-03-2013" /><a href="#21-03-2013"><hr><h2><b>March 21, 2013</b></h2></a>
3689<p>Video of my ELC talk
3690"<a href=http://youtu.be/SGmtP5Lg_t0>Why is Toybox?</a>"
3691is up on youtube. Related materials include the
3692<a href=http://landley.net/talks/celf-2013.txt>talk outline</a> and an
3693<a href=/aboriginal/about.html#selfhost>android self-hosting writeup</a>.</p>
3694
3695<p>[Updated June 4] The following links jump to specific topics in the video. (Sorry about
3696the ads, it's The Linux Foundation.)</p>
3697
3698<ul>
3699<li>0m29s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=0m29s>The smartphone is replacing the PC</a></li>
3700  <ul>
3701  <li>4m22s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=4m22s>Software needed to become self-hosting</a></li>
3702  <li>6m20s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=6m20s>Do we care if android or iphone wins?</a></li>
3703  </ul>
3704<li>9m45s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=9m45s>Android not vanilla: oppose or accept?</a></li>
3705  <ul>
3706  <li>11m30s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=11m30s>Open source can't do User Interfaces</a></li>
3707  </ul>
3708<li>15m09s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=15m09s>Android is not copyleft: oppose or accept?</a></li>
3709<li>18m23s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=18m23s>Security issues</a></li>
3710<li>21m15s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=21m15s>Solutions to the software problems</a></li>
3711  <ul>
3712  <li>22m55s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=22m55s>What toybox needs to be/do</a></li>
3713  <li>28m17s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m17s>What is toybox?</a></li>
3714    <ul>
3715    <li>28m58s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m58s>Why toybox started...</a></li>
3716    <li>37m50s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=37m50s>What does toybox actually implement?</a></li>
3717    </ul>
3718  </ul>
3719</ul>
3720</span>
3721
3722
3723<a name="14-03-2013" /><a href="#14-03-2013"><hr><h2><b>March 14, 2013</b></h2></a>
3724<blockquote><p>"Ford, you're turning into a penguin. Stop it." -
3725</p><p>The Hitchhiker's Guide to the Galaxy</p></blockquote>
3726
3727<p><a href=downloads/toybox-0.4.4.tar.bz2>Toybox 0.4.4</a> is based on
3728<a href=http://landley.net/hg/toybox/shortlog/813>commit 813</a>, adding
3729the "time" and "readahead" commands, plus some bugfixes.</p>
3730
3731<p>The "cp" command now implements the -s symlink option, plus bugfixes
3732getting various corner cases right as used in actual package builds.
3733"id -Gn root" should now print root's groups
3734instead of the current user's. Several build fixes so toybox builds under
3735Ubuntu 8.04 again (which is about as old a build environment as you
3736can expect to find posix-2008 features in).</p>
3737
3738<p>Unfinished commands have generally been moved to "toys/pending".
3739Everything else should "default y" to participate in make defconfig.
3740Several of those pending commands got some basic cleanup so allyesconfig
3741should at least compile (although defconfig is still what's useful).</p>
3742
3743<p>Significant roadmap updates, checking several other multicall binaries
3744(klibc, sash, sbase, s6...) to see what commands they include.</p>
3745
3746<a name="18-01-2013" /><a href="#18-01-2013"><hr><h2><b>January 18, 2013</b></h2></a>
3747<blockquote><p>This must be Thursday. I never could get the hang of Thursdays.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3748
3749<p><a href=downloads/toybox-0.4.3.tar.bz2>Toybox 0.4.3</a> is based on
3750<a href=http://landley.net/hg/toybox/shortlog/793>commit 793</a>. There
3751are now exactly 100 commands in defconfig (of a little over 220 on the
3752<a href=roadmap.html>todo list</a>).</p>
3753
3754<p>Elie De Brauwer added the rev command, cleaned up tac, implemented the -s
3755and -f flags for seq, added -v and -i to killall (and fixed killall not to
3756kill itself before finishing its pid list), and added to the test suite.
3757Felix Janda added -m to mkdir, pwd -L and -P, and more test suite entries.</p>
3758
3759<p>Rob Landley added the losetup command, and fixed the existing ls, cp, and
3760readlink commands. The segfault in ls
3761happened when it couldn't determine the screen size (last release changed the
3762default to -C and a screen size of 0 made column view unhappy), and cp got an
3763extensive rewrite bringing it up to date with the dirtree changes and fixing
3764a number of things it never did right in the first place. The xabspath()
3765code in the library now handles a symlink after ".." properly (and the test
3766suite checks for it).</p>
3767
3768<p>Infrastructure-wise the code is better about automatically setting the
3769error return code properly. Now error_msg() sets the exit code to 1 if it's
3770still defaulting to 0, and the global exit path does a fflush(NULL) with error
3771bit check rather than trying to be quite so granular about flushing. (That
3772means if we use printf() instead of xprintf() it still exits with the right
3773error code, it just doesn't end the program early on an output error.)
3774Minor bugfix so TOYBOX_DEBUG
3775doesn't always warn about the lack of suid bit when toybox is built with
3776at least one STAYROOT command. Bugfix for the option [grouping] logic
3777(and then further fixes to the error reporting pointed out by Ashwini Sharma).
3778dirtree_handle_callback() now has a prefix like the rest of the dirtree
3779functions. A lot of stuff doing manual path handling was switched to using
3780libc basename() (including, embarassingly, the basename command), which means
3781it now correctly detects "/trailing/slash/" which the previous code didn't.</p>
3782
3783<p>Also, last release included some accidentally checked in debug code that
3784disabled compiler optimization, so the binary size bloated a bit. It's back
3785to -Os by default now.</p>
3786
3787<a name="15-12-2012" /><a href="#15-12-2012"><hr><h2><b>December 15, 2012</b></h2></a>
3788<blockquote><p>"The major difference between a thing that might go wrong and a
3789thing that cannot possibly go wrong is that when a thing that cannot possibly
3790go wrong goes wrong it usually turns out to be impossible to get at or repair."
3791</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3792
3793<p><a href=downloads/toybox-0.4.2.tar.bz2>Toybox 0.4.2</a> is based on
3794<a href=http://landley.net/hg/toybox/shortlog/749>commit 749</a> and is
3795just a resync. Linux 3.7 came out, meaning it's time to do an Aboriginal
3796Linux release, and that should use a stable version of toybox. So here's
3797a new stable version.</p>
3798
3799<p>The new commands are cut (from Jason Kyungwan Han), touch
3800(from Choubey Ji), expand (from Jonathan Clairembault, and he fixed a
3801bug in login), and rm (from Rob Landley). Felix Janda added UTF-8
3802support infrastructure (for non-ascii character sets) with a config option.
3803Elie De Brauwer added tests for cat and sha1sum, and -so options to pidof.
3804The "ls" command defaults to -C (column view) now, and "readlink" now supports
3805-fenq.</p>
3806
3807<p>Portability work: toybox should now build against the musl C library,
3808and against older glibc versions (circa 2008, much before that and kernel
3809features we depend on start to drop out).</p>
3810
3811<p>The whole codebase got reindented from "one tab" to "two spaces" per
3812level. The option parsing logic now understands [groups] of commands (when more
3813than one in a group is selected it can switch the others off, or error out,
3814or other things). The error_exit() infrastructure can now longjmp back to an
3815earlier point instead of exiting. Each toys/* directory now has a README,
3816the first line of which is the fancy name menuconfig uses for the directory
3817(so no more hardwired directory list in scripts/genconfig.sh).</p>
3818
3819<p>Fixed a filehandle leak in getmountlist().
3820Pass parent pointer to dirtree_add_node() so it can give error messages with
3821full path. The yesno() function now always reads from stdin and writes to
3822stderr (we can retry tty checking complexity once we've got commands needing
3823it).</p>
3824
3825<p>The open group broke their website so the
3826<a href=http://opengroup.org/onlinepubs/9699919799>old links</a> to POSIX 2008
3827now <a href=http://pubs.opengroup.org/onlinepubs/9699919799>need to start with
3828pubs</a>. Some of the links in the tree have been updated, others haven't while
3829I wait to see if their webmaster notices and fixes it.</p>
3830
3831<p>(I note that the current rm implementation is not technically posix compliant
3832because the standard requires infinite recursion depth and the current
3833implementation uses one filehandle per level. I can add a config option
3834to do it Posix's way, which is more brittle and needs extra security checks,
3835but am waiting for somebody to complain first. The default "ulimit -n" is 1024
3836filehandles, so drilling down over 1000 nested subdirectories).</p>
3837
3838<a name="13-11-2012" /><a href="#13-11-2012"><hr><h2><b>November 13, 2012</b></h2></a>
3839<blockquote><p>"Rule Six: The winning team shall be the first team that wins."
3840</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3841
3842<p><a href=downloads/toybox-0.4.1.tar.bz2>Toybox 0.4.1</a> is based on
3843<a href=http://landley.net/hg/toybox/shortlog/691>commit 691</a>.</p>
3844
3845<p>Elie De Brauwer contributed usleep, Ashwini Kumar contributed du, and
3846Kyungwan Han contributed vconfig. Other new commands include switch_root and
3847md5sum, and the remaining shell wrappers are now proper commands (dos2unix,
3848unix2dos).</p>
3849
3850<p>The patch command now supports -l, and gethostname is now enabled by
3851default. The df command follows symlinks to get the actual device name.
3852Felix Janda added -m support to wc (for utf8).</p>
3853
3854<p>On the infrastructure side, the commands have now been grouped into
3855"posix", "lsb", and "other" subdirectories (for things required by Posix-2008,
3856the Linux Standard Base 4.1, and commands in neither). This affects menuconfig
3857and the actual source layout (toys/cp.c is now toys/posix/cp.c, and so on).
3858An android directory is planned (see the updated
3859<a href=roadmap.html#android>android roadmap analysis</a>).</p>
3860
3861<p>The FLAG_ macros for command option parsing and TT alias for the command's
3862global block are now automatically generated, commands should
3863#define FOR_commandname before #including <toys.h> to get the macros for that
3864command.</p>
3865
3866<p>An upgrade to the build infrastructure now allows commands with _ and -
3867in them, such as switch_root.</p>
3868
3869<p>Bugfixes: Avery Pennarun spotted a case where ls showed uid twice instead of
3870uid and gid, and that nice was using the wrong range of numbers.
3871The ls command also recursed inappropriately last time (not quite
3872properly converted for the dirtree changes last release), and now it's
3873fixed. Roy Tam pointed out a glitch in sh, and fixed df's percentage
3874calculation to match the POSIX spec. The kernel build didn't like our mktemp
3875and it does now. The wc command wasn't quite posix compliant (trailing spaces
3876break stuff). The ls command recursed inappropriately last time (not quite
3877properly converted for the dirtree changes last release), and now it's
3878fixed. The catv command wasn't displaying byte 255 correctly. Some lib
3879fixes (thinko in xpidfile). Fixed uname -m when running a 32 bit x86 binary
3880on an x86-64 host (it lies and says the system is i686, i586, or i486 depending
3881on what the toolchain that built the binary supported. This makes builds in
3882a 32 bit chroot on a 64 bit kernel break less.) The df command was checking
3883partitions in the wrong order (displaying undermounts instead of overmounts:
3884this used to work but some library code changed out from under it and it
3885wasn't updated to match until now). Felix Janda filled out the test suite
3886some more. The patch file creation logic got tweaked several times to
3887successfully apply more patches. Support for older (pre 2.10) glibc
3888versions was added to portability.h.</p>
3889
3890<p>Miscelaneous cleanups all around (mknod, sha1sum, logname), including a
3891rewrite of taskset to be less dependent on libc getting the headers right. All
3892the command headers should now point to the current relevant standards
3893document, where applicable.</p>
3894
3895<p>This news page had old news entries from before the relaunch moved into
3896a separate <a href=oldnews.html>oldnews</a> page.</p>
3897
3898<p>I forgot to create <a href=bin>static binaries</a> last time, but they're
3899back now.</p>
3900</span>
3901
3902<a name="23-07-2012" /><a href="#23-07-2012"><hr><h2><b>July 23, 2012</b></h2></a>
3903<blockquote><p>"Ford", Arthur said. "There's an infinite number of monkeys
3904out here who want to talk to us about this script for Hamlet they've worked
3905out."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3906
3907<p><a href=downloads/toybox-0.4.0.tar.bz2>Toybox 0.4.0</a> is based on
3908<a href=http://landley.net/hg/toybox/shortlog/640>commit 640</a>.</p>
3909
3910<p>The new <a href=status.html>status page</a> is calculated from
3911the roadmap info, and should be easier to keep up to date in future.</p>
3912
3913<p>Andre Renaud contributed od and modinfo. Elie De Brauwer contributed
3914taskset, bugfixes to cmp and tail, and tests for sort and tail. Kyungwan Han
3915contributed passwd. Gaurang Shastri contributed w. Ashwini Sharma spotted a
3916case where dirtree was adding extra slashes to a path.</p>
3917
3918<p>I rewrote od, cleaned up comm, documented the
3919<a href=code.html#lib_llist>llist</a> and
3920<a href=code.html#lib_dirtree>dirtree</a> infrastructure, added an -r option
3921to date (and fixed a bug where -u wouldn't override /etc/localtime),
3922fixed bugs in chmod +stw, fixed ls to show suid bits properly when the
3923corresponding executable bit wasn't set, and worked around a longstanding
3924glibc bug where static linking prevents stdout from automatically flushing
3925pending output on exit.</p>
3926
3927<a name="25-06-2012" /><a href="#25-06-2012"><hr><h2><b>June 25, 2012</b></h2></a>
3928<blockquote><p>"For a moment, nothing happened. Then, after a second or so, nothing continued to happen."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3929
3930<p><a href=downloads/toybox-0.3.1.tar.bz2>Toybox 0.3.1</a> is based on commit
3931<a href=http://landley.net/hg/toybox/shortlog/607>commit 607</a>. It's
3932mostly a bugfix release for ls -l (which was unhappy on targets other than
3933x86-64), plus a new "date" from Andre Renaud and rewritten chgrp/chown which
3934now support the full set of posix flags, plus a little work on the test
3935suite and some more header tweaks towards eventual compatability with the
3936musl libc.</p>
3937
3938<p>The todo list runneth over, but "release early, release often", so here
3939it is. The roadmap and documentation are a bit behind, and I've got ~40
3940pending submissions to review. I need to catch up...</p>
3941</span>
3942
3943<a name="12-06-2012" /><a href="#12-06-2012"><hr><h2><b>June 12, 2012</b></h2></a>
3944<blockquote><p>"For instance, on the planet Earth, man had always assumed that
3945he was more intelligent than dolphins because he had achieved so much - the
3946wheel, New York, wars and so on - whilst all the dolphins had ever done was
3947muck about in the water having a good time. But conversely, the dolphins had
3948always believed that they were far more intelligent than man - for precisely
3949the same reasons."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote>
3950
3951<p>It's well past time for <a href=downloads/toybox-0.3.0.tar.bz2>toybox 0.3.0</a>,
3952so here it is, based
3953on <a href=http://landley.net/hg/toybox/shortlog/595>commit 595</a>, and the
3954statically linked <a href=downloads/binaries>prebuilt binaries</a> should
3955actually be statically linked this time (thanks Ashwini Sharma for spotting
3956that).</p>
3957
3958<p>It's hard to figure out where to cut a release, because development
3959doesn't stop. "Long before now" is the obviuos answer, of course.
3960The project's maintainer also moved house during this development cycle, which
3961threw things off for a bit (so many boxes). Releases should hopefully be a bit
3962more frequent from here on.</p>
3963
3964<p>The big things Rob worked on this time were the new dirtree (directory
3965tree traversal) infrastructure, and a complete rewrite of ls using that
3966which should now implement all 26 posix options.</p>
3967
3968<p>Georgi Chorbadzhiyski added printenv, whoami, mkdir, mkfifo, chmod, chown,
3969chgrp, and uniq. He also added fraction and extension support to sleep (so if
3970you need a quarter-second sleep, it can do that now), and fixed a build bug
3971on slackware.</p>
3972
3973<p>Daniel Walter contributed a string to mode_t parser (in use by chmod and
3974mkdir -m).  Ilya Kuzmich contributed comm. Elie De Brauwer added mountpoint,
3975vmstat, logname, login, and mktemp. Kevin Chase did some portability cleanups.
3976Pere Orga fixed some documentation.</p>
3977
3978<p>The "tac" and "clear" commands are now normal commands instead of shell
3979wrappers, and the header #includes have been cleaned up a bit to remove
3980deprecated functions and attempt to increase compatability with the bionic and
3981musl C libraries, "tail" should now use lseek() for large files, and "id" got
3982some cleanups and bugfixes.</p>
3983
3984<p>The new TOYBOX_FLOAT configuration option selects whether or not
3985to include floating point support (for embedded targets where that's
3986problematic).</p>
3987
3988<p>Several random bugfixes: unshare() might actually build portably now,
3989yes 'n' | cp -i should no longer bypass stdin and prompt via the tty, the
3990SUID support no longer drops permissions going through the toybox
3991multiplexer command, and a bugfix to xargs -0 means it should no longer
3992segfault. (I have a pending bug report about xargs not doing the full
3993posix whitespace handling that -0 obsoleted, but I'll deal with that next
3994release.)</p>
3995
3996<p>The build infrastructure is now automatically generating FLAG_ macros
3997for the options, but currently with the wrong names. Some more macro glue
3998is necessary, which I haven't quite figured out how to do yet.</p>
3999
4000<p>A defconfig toybox at the start of the $PATH has successfully built
4001Linux From Scratch (in my Aboriginal Linux project). The commands that
4002'default n' in the config are often still broken, cleanup is ongoing.
4003(The new dirtree stuff broke several of them that haven't been converted
4004yet, but if I wait until everything works we won't have a release before
40051.0, so here's a checkpoint.)</p>
4006
4007
4008<a name="03-03-2012" /><a href="#03-03-2012"><hr><h2><b>March 3, 2012</b></h2></a>
4009
4010<blockquote><p>"They went unnoticed at Goonhilly, passed over Cape Canaveral
4011without a blip, and Woomera and Jodrell Bank looked straight through them.
4012Which was a pity, because it was exactly the sort of thing they'd been looking
4013for all these years."</p><p>- The Hitchhiker's Guide to the Galaxy</p>
4014</p></blockquote>
4015
4016<p>Here's <a href=downloads/toybox-0.2.1.tar.bz2>toybox 0.2.1</a> based
4017on <a href=http://landley.net/hg/toybox/shortlog/512>commit 512</a>.  This
4018time around, there are statically linked <a href=downloads/binaries>prebuilt
4019binaries</a> for various embedded targets.</p>
4020
4021<p>It's been a busy few weeks, almost entirely due to new contributors. (I
4022have not quite been keeping up.)</p>
4023
4024<p>Elie De Brauwer contributed free, uptime, swapon, swapoff, lsmod, mknod,
4025insmod, rmmod, and fixed a bug in basename.  Andre Renaud contributed ls, ln,
4026realpath, and hostname. Andres Heck contributed pidof and killall.  Daniel
4027Walter wrote kill and extended id. Timothy Elliott contributed tail and tests
4028for cmp. Frank Bergmann sent a warning fix. Bryce Fricke added -i to cp.
4029Nathan McSween pointed out an optimization. Georgi Chorbadzhiyski fixed
4030cross compiling to work more reliably.</p>
4031
4032<p>(My own contribution this time around was just tightening up other people's
4033code, a build fix to unshare, some random bugfixes, and so on. My only new
4034code this time around was writing a bash replacement for the existing python
4035bloat-o-meter.)</p>
4036
4037<p>Last time (the 0.2.0 release) included the first pass at an id command from
4038Tim Bird, env and basename from Tryn Mirell, cmp and head from Timothy Elliott,
4039more bugfixes from Nathan McSween and Elie De Brauwer, and Luis Felipe Strano
4040Moraes did a first pass at the who command plus other bugfixes and
4041optimizations.</p>
4042
4043<p>(For that release I did xargs, cal, truncate, unlink, nohup, tty, wc, link,
4044dirname, unshare, and various infrastructure tweaks, but it took me 3 months
4045and those guys did their stuff in a week or so.)</p>
4046
4047
4048<a name="12-02-2012" /><a href="#12-02-2012"><hr><h2><b>February 12, 2012</b></h2></a>
4049<blockquote><p>
4050"for though it has many omissions and contains much that is apocryphal, or at
4051least wildly inaccurate, it scores over the older, more pedestrian work in two
4052important respects..."</p>
4053<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote>
4054
4055<p>Here's the first BSD licensed release,
4056<a href=downloads/toybox-0.2.0.tar.bz2>toybox-0.2.0</a>, more a synchronization
4057point than anything particularly useful.  47 commands in a reasonably
4058ready-to-use state (what "make defconfig" builds), another ten or so partially
4059finished stubs ("make allyesconfig"), and several
4060patches pending on the mailing list I need to review and merge.</p>
4061
4062<p>More to come...</p>
4063
4064<hr>
4065<a name="15-11-2011" /><a href="#15-11-2011"><hr><h2><b>November 15, 2011</b></h2></a>
4066- Back from the dead, Toybox is now under a 2
4067clause BSD license, and aiming to become the default command line
4068implementation of Android systems everywhere.</p>
4069
4070<p>More to come...</p>
4071
4072<hr>
4073
4074<p><a href=oldnews.html>Old news</a> from before the relaunch.</p>
4075
4076<!--#include file="footer.html" -->
4077