1<html><head><title>toybox news</title> 2<!--#include file="header.html" --> 3 4<p>Toybox combines the most common Linux command line utilities together 5into a single BSD-licensed executable that's simple, small, fast, 6reasonably standards-compliant, and powerful enough to turn Android into 7a development environment. See the links on the left for details.</p> 8 9<h2>News</h2> 10 11<!-- new entry template 12<a name="DD-MM-YYYY" /><a href="#01-01-2001"><hr><h2><b>Jan 1, 2001</b></h2></a> 13<blockquote> 14<p> 15Don't Panic 16</p> - The Hitchhiker's Guide to the Galaxy</p> 17</blockquote> 18 19<p><a href=downloads/toybox-0.0.1.tar.gz>Toybox 0.0.1</a> 20(<a href=https://github.com/landley/toybox/releases/tag/0.0.1>git commit</a>) 21is out, with prebuilt <a href=downloads/binaries/0.0.1>static binaries</a> and 22<a href=downloads/binaries/mkroot/0.0.1>mkroot images</a> 23bootable under QEMU (built using a <a href=https://github.com/landley/linux/tree/mkroot-0.1>lightly patched</a> linux-0.1).</p> 24 25<p>New commands</p> 26 27<u>Features</u>: 28<u>Bugfixes</u>: 29<u>Library</u>: 30<u>Mkroot</u>: 31<u>Pending</u>: 32<u>Cleanup</u>: 33<u>Portability</u>: 34<u>Documentation</u>: 35<u>Plumbing</u>: 36<u>Test suite</u>: 37<u>Build</u>: 38--> 39 40<a name="18-01-2025" /><a href="#18-01-2025"><hr><h2><b>Jan 18, 2025</b></h2></a> 41<blockquote> 42<p>The regular early morning yell of horror was the sound of Arthur Dent waking 43up and suddenly remembering where he was.</p> 44</p> - The Hitchhiker's Guide to the Galaxy</p> 45</blockquote> 46 47<p><a href=downloads/toybox-0.8.12.tar.gz>Toybox 0.8.12</a> 48(<a href=https://github.com/landley/toybox/releases/tag/0.8.12>git commit</a>) 49is out, with prebuilt <a href=/bin/toybox/0.8.12>static binaries</a> and 50<a href=/bin/mkroot/0.8.12>mkroot images</a> 51bootable under QEMU 9.2.0 (built using a <a href=/bin/mkroot/0.8.12/linux-patches/>lightly patched</a> linux-6.12).</p> 52 53<p>Posix-2024 (SUSv5, issue 8) finally came out, and a few things 54have been tweaked, but as with Posix-2008 (SUSv4, issue 7, released 5516 years ago) it was mostly codifying what the Linux man pages 56already said.</p> 57 58<p>Toybox is now only the fifth most starred repository of the 59<a href=https://github.com/search?q=license%3A0bsd&type=Repositories&ref=advsearch>~60k 0BSD licensed projects on github</a>, which means 60<a href=https://spdx.org/licenses/0BSD.html>Zero Clause BSD</a> has 61outgrown its roots.</p> 62 63<p><u>Features</u>: added <b>netcat -o -O</b> (hex output, the first 64preserving original input groups and the latter continuous), and 65<b>tar</b> now handles another obsolete sparse format you get in old tarballs sometimes (with test). 66Elliott added NPROCESSORS_CONF and NPROCESSORS_ONLN to <b>getconf</b>, 67added -f FILE and --no-sync to <b>devmem</b>, and 68taught <b>tar</b> to call out to zstd. 69Oliver Webb added <b>test -ef -ot -nt</b>, 70Brian Norris added <b>dmesg -W</b>. 71Karthikeyan Ramasubramanian added <b>devmen --no-mmap</b>. 72Kana Steimle added <b>mount LABEL=</b>. 73Firas Khalil Khana made <b>lsusb</b> and <b>lspci</b> also check 74/usr/share/hwdata (since the Linux Foundation killed the Linux Standard 75Base system packagers can't quite agree where stuff should live). 76</p> 77 78<p><u>Bugfixes</u>: 79<b>grep -r</b> now opens files with O_NOBLOCK|O_NOCTTY to avoid getting hung 80up on FIFOs or attaching the console to dev/tty nodes, 81<b>patch -F0</b> disables fuzz support, 82<b>find -size</b> implies -type f, 83<b>host</b> detects truncated replies and doesn't output the length byte at 84the start of TXT replies, 85more complicated relative file permission parsing like <b>chmod g+rX-ws</b> 86is now handled properly, 87<b>tar</b> honors umask unless -p supplied (with tests), 88<b>test</b> -nt and -ot now only check nanoseconds when the seconds match, 89and a weird corner case in <b>ps</b> was because the <b>make.sh</b> plumbing 90wasn't switching FLAG_x to "long long" quite fast enough (when 1<<31 91is negative, using it in nontrivial bitmask checks can go strange).</p> 92 93<p>Elliott fixed a crash when <b>file</b> tried to parse corrupted ELF files, 94fixed <b>cp -i -v</b> verbose notifications (showing source when they should 95show destination, etc), and fixed 96a sendfile_len() bounds check. Ray Gardner found a unicode bug in lib.c's 97strlower() (which is now checked for in find.test). 98Yi-Yo Chiang fixed </b>netcat -f</b> and two bugs in <b>microcom</b>'s 99"paste" command. 100Xiuhong Wang fixed <b>ionice</b>'s output when checking the IO priority of a 101process you don't have permission to access. 102Dima Buzdyk taught <b>lsusb</b> to handle multiple PCIe controllers, including printing 103the controller domain id when there's more than one. 104Peter Collingbourne noticed that <b>ps</b>/<b>top</b> etc were setting the stdout buffer 105twice (which glibc doesn't support). 106Kana Steimle fixed <b>blkid</b> to match util-linux behavior more closely.</p> 107 108<p><u>Library</u>: 109Added <b>cfspeed2bps()</b> and <b>bsp2cfspeed()</b> functions (converting bits per second 110values to/from the cfsetspeed() argument bitmask), and make <b>xsetspeed()</b> 111use them. Moved <b>anystart()</b> and <b>anystr()</b> to lib.c.</p> 112 113<p><u>Mkroot</u>: 114New <b>riscv32</b> and <b>riscv64</b> targets, and <b>or1k</b> switched to the 115-M virt board emulation so it can exit now. The <b>armv5l</b> -M versatilepb 116board no longer micromanages the network card selection (which was working 117around a bug from 2007 long since fixed upstream). 118All <a href=https://landley.net/bin/mkroot/0.8.12>shipped targets</a> fully 119pass <b>mkroot/testroot.sh</b> except <b>or1k</b> 120(no default -hda bus <a href=https://lists.gnu.org/archive/html/qemu-devel/2025-01/msg00996.html>in qemu</a>'s 121board emulation), <b>microblaze</b> (no -hda 122available even with -drive, and the board emulation doesn't know how to exit 123without being killed), and <b>sh4eb</b> (an endianess disagreement between 124the ethernet driver and device emulation, little endian works fine with the 125same config).</p> 126 127<p><b>VMLINUX=</b> no longer needs a path to a zimage file, it looks under 128arch/$KARCH/boot when it's not at the top level. <b>QEMU_M=</b> 129is now a short way to say "qemu-system-$CROSS -M $QEMU_M". 130In <b>mkroot/packages</b>, updated the <b>dropbear</b> and <b>zlib</b> 131versions, and added an initial script to convert the <b>Linux From Scratch</b> 13212.1 sources into a squashfs 133(just a start, several of the shipped patches are in the obsolete diff -c 134"copied context" format because gnu, and I haven't taught patch to apply 135the old format yet. Maye a new LFS release will stop doing that?) 136<b>setupfor</b> now handles tarballs that don't extract to a directory with 137the same name. 138Fixed mkroot running in a container where /dev/tty was never associated 139(where writing to it produces an error). The airlock build is skippable 140so don't rely on it to set mkroot $VERSION.</p> 141 142<p><b>mkroot/record-commands</b> can now be run from an arbitrary directory, 143to more easily log command line usage in other projects' builds. Running 144it with no arguments now sets up a persistent wrapper and outputs 145an "export" line needed to update the $PATH and such to use it. 146(Running a command under it the way chroot or sh -c work still sets up 147the wrapper, runs the command, then tears down the wrapper again leaving 148the log.) Also, make it work on a busybox-based host (where "find -type" 149didn't understand commas).</p> 150 151<p><u>Pending</u>: 152New <b>awk.c</b> from Ray Gadner, which is 4500 lines long and counting 153(46 commits so far). 154Eric Roshan-Eisner fixed several issues in <b>vi</b> and its tests. 155Elliott fixed /etc/issue in <b>getty</b>. 156Daniel Rosenberg added <b>diff --no-dereference</b> and fixed comparisons 157between symlinks and fifos (with test). 158Kana Steimle fixed several things in <b>crond</b>.</p> 159 160<p>Use DRAIN instead of FLUSH in <b>stty</b>, 161Consistently indent the help text blocks (in syslogd.c, dhcp6, diff.c...), 162and the partial mke2fs.c was removed because gcc 14 breaks trying to build it 163(due to a design change in gcc, which used to be able to compile it but 164can't anymore, if I get around to finishing it I can check in one that 165actually performs its intended function but bug reports about code that 166doesn't work = remove it).</p> 167 168<p>More <b>toysh</b> work, most notably a 169<a href=https://github.com/landley/toybox/commit/7a2f81c82d1c>design change</a> 170that allows "trap" to asynchronlysly call functions from an interrupt handler 171(by removing the recursive do_source() function, moving the read/parse/run 172loop to sh_main(), and giving each sh_fcall stack entry a FILE *source 173it can read data from when the parsed sh_pipeline is empty, and then pop 174the sh_fcall on EOF and continue in the calling context.) 175The tests that are currently expected to fail are now 176annotated with $BROKEN (use "BROKEN= make test_sh" to try them), and 177TEST_HOST now passes on bash 5.2. Implement <b>return</b>, move <b>break</b> 178and <b>continue</b> to normal builtin commands (instead of special case 179inline processing in run_lines()), make the xexit() longjmp() back to the shell 180from builtins nest multiple levels deep if necessary (in case builtins call 181builtins, which is less likely under the new design), bump script filehandle 182to high fd and CLOEXEC it (instead of leaking into children). 183Another one line fix had 184<a href=https://github.com/landley/toybox/commit/0b2d5c2bb3f1>several paragraphs</a> 185of explanation, but boils down to "kernels with static initramfs have no 186stdin/stdout/stderr so toysh needs to cope when redirect opens one 187of those as a temporary filehandle". Fixed backslash parsing in $'' so \' 188doesn't end the quoting context.</p> 189 190<p><u>Cleanup</u>: 191Although <b>klogd</b> is cleaned up most of the way to being promotable, 192since it just passes stuff along to syslogd (which isn't) it's not much 193use on its own. Switched most remaining toys.optflags&FLAG_x checks to 194<b>FLAG(x)</b>. Minor cleanup on <b>cp -r</b>. 195Elliott added input buffering to <b>xxd</b>, made find.c use S_ISREG() instead 196of inlining it. Brian Norris added line buffering to dmesg stdout. 197<b>klogd</b> is cleaned up most of the way to being promotable, but 198since it just passes stuff along to syslogd (which isn't) it's not much 199use on its own.</p> 200 201<p><u>Portability</u>: 202Fixed <b>hwclock</b> when settimeofday() syscall isn't available (which 203musl and glibc broke in 204<a href=https://github.com/landley/toybox/commit/da1474b1589a>different ways</a>), 205endianness checking now uses the compiler's built-in <b>__BYTE_ORDER__</b> macro 206instead of needing to #include headers. The nommu-friendly <b>xvdaemon()</b> 207will now chdir("/") so background processes don't pin mount points, and 208find.test now skips a test that triggered a macos bug. 209Elliott dropped ps.c's dependency on android libprocessgroup (it now 210reads /proc/$PID/cgroup directly). Vidar Karlsen had some freebsd updates. 211Kana Steimle worked around closedir(NULL) segfaulting on musl (which doesn't 212consider that a NOP).</p> 213 214<u>Documentation</u>: 215<b>sysctl</b> now accepts -A as an alias for -a but 216<a href=https://github.com/landley/toybox/commit/41e7186b012e>doesn't try to explain it</a>. 217More FAQ entries. and added some links to the end of <a href=license.html>license.html</a>. 218Old entries in this news page no longer drive vi's html syntax highlighting nuts. 219Elliott clarified the relationship between <b>ls -s</b> and --block-size</b>. 220Ivan Mirić fixed some typos in mkroot's README.</p> 221 222<p><u>Plumbing</u>: 223Added <b>TOYFLAG_NOBUF</b>, used by microcom to disable stdio buffering. 224The "lie to autoconf" help pluming moved into main.c as <b>TOYFLAG_AUTOCONF</b> 225because <b>grep</b> needs it too now.</p> 226 227<p><u>Test suite</u>: 228lots of tests updated to pass with TEST_HOST on Devuan Daedalus (I.E. 229Debian Bookworm). Various .1 second timeouts were expanded to give loaded 230servers time to schedule processes. (An unloaded raspberry pi should have no 231trouble, and a swap-thrashing system may not finish even with a 5 second wait 232to flush asynchronous activity, but it's at least a better guess.) 233For some reason, ASAN on Debian's current gcc/glibc turns glibc's <b>crypt()</b> 234function into a null pointer dereference. This seems to be a host toolchain bug, 235and there's work underway to replace the host crypt() with a lib/lib.c 236implementation using the builtin hashes in a future release, but for now 237you have to switch off "mkpasswd" in menuconfig to pass "make tests" on 238Debian. It worked fine on the old debian version, works fine built under 239musl-libc, works fine without ASAN enabled, is not failing IN 240crypt (it's never reaching it, the dynamic linker is turning the call into 241a jump to NULL). The shipped binaries are built against musl where crypt() 242(and thus mkpasswd) still works.</p> 243 244<p><u>Build</u>: 245Setting $TOOLCHAIN for <b>scripts/install.sh</b> (and thus 246<b>make airlock_install</b>) now symlinks the listed additional commands from 247the host $PATH into the new airlock directory. 248Fixed ASAN to mostly work with newer gcc/glibc (from Debian 12) except for 249the crypt() thing. Most files like generated/tags.h are now generated with sed and bash instead 250of building a native C program to emit them. 251Added --start-group and --end-group around the "extra" libraries when static 252linking (because probing them in parallel doesn't preserve the magic order). 253Sadly -f does not reliably shut up gnu/chmod, despite its longopt aliases 254being "--silent" and "--quiet" (the <strike>aristocrats</strike> FSF!) 255so redirect the noise to /dev/null. 256Added "ar" to the airlock install's PENDING list because the dropbear build 257grew a dependency on it. (You'd think that would use the cross compiler's 258prefixed-ar, but no.) 259Squelched more spurious clang warnings 260<a href=https://github.com/landley/toybox/commit/b9cb58b797b7>about nothing</a>.</p> 261 262<a name="08-04-2024" /><a href="#08-04-2024"><hr><h2><b>April 8, 2024</b></h2></a> 263<blockquote> 264<p>Another thing that got forgotten was the fact that against all probability a 265sperm whale had suddenly been called into existence several miles above the 266surface of an alien planet. And since this is not a naturally tenable position 267for a whale, this poor innocent creature had very little time to come to terms 268with its identity as a whale before it then had to come to terms with not being 269a whale any more.</p> 270<p>- The Hitchhiker's Guide to the Galaxy</p> 271</blockquote> 272 273<p><a href=downloads/toybox-0.8.11.tar.gz>Toybox 0.8.11</a> 274(<a href=https://github.com/landley/toybox/releases/tag/0.8.11>git commit</a>) 275is out, with prebuilt <a href=/bin/toybox/0.8.11>static binaries</a> and 276<a href=/bin/mkroot/0.8.11>mkroot images</a> 277bootable under QEMU (built using a <a href=/bin/mkroot/0.8.11/linux-patches>lightly patched</a> linux-6.8).</p> 278 279<p>Highlights include a new <a href=quickstart.html>quickstart</a> 280web page, shorter <a href=/bin>/bin</a> links to prebuilt binaries, 281lots of work on mkroot, and a new "canned" toybox build to reduce 282dependencies.</p> 283 284<p><u>New commands</u>: 285Rob added <b>tsort</b>, and promoted <b>fold</b> and <b>getopt</b>. 286Oliver Webb added <b>ts</b> and <b>csplit</b>, and 287Elliott added <b>memeater</b>.</p> 288 289<p>And one command got temporarily de-promoted: <b>passwd</b> is "default n" 290(disabled in defconfig) this release due to the ongoing lib/passwd.c 291infrastructure rewrite that isn't quite done yet. (See the Library section.)</p> 292 293<u>Features</u>: 294The new "<a href=https://github.com/landley/toybox/commit/d1acc6e88be5>canned</a>" 295toybox build in <b>scripts/prereq/build.sh</b> lets toybox provide its own 296build prerequisites by compiling a minimal toybox against saved headers (ala 297"cc -I scripts/prereq/generated file1.c file2.c..."). This provides 298most of the commands toybox needs to configure and build itself (except make, 299bash, and the compiler toolchain). This may help <a href=https://github.com/landley/toybox/commit/3bbc31c78b41>bootstrap</a> toybox on systems 300that don't provide a modern Linux command line out of the box: install 301toybox-prereq at the start of the $PATH, add 302a .config file, and run <b>scripts/make.sh</b>.</p> 303 304<p>New <b>mv -x</b> option to atomically swap two files, 305<b>env -e FILE</b> executes a different command than argv[0] on the command 306line, the <b>reset</b> command now puts raw terminals back into "cooked" mode, 307negative offsets in <b>sort -k</b> count right to left, 308<b>setsid</b> grabs the tty more forcefully (removing the need for getty) 309and the -cc option will try to steal it, 310<b>mount</b> now shows file= when displaying loopback mounts, 311<b>wc -L</b> shows longest line length, 312<b>tar -h</b> now detects a lot more hardlinks, 313<b>cp</b> now copies xattrs for directories (not just files).</p> 314 315<p>Elliott taught <b>file</b> to recognize wasm binary modules and 316display JPEG width/height, added the LOWER_UP, DORMANT, and ECHO 317flags to <b>ifconfig</b>, added PAC and BTI decoding to <b>readelf</b>, 318cache size querying to <b>sysconf</b> (supported by glibc and bionic but 319not musl-libc), and added a "paste" menu to <b>microcom</b>. Kalesh Singh 320taught <b>readelf</b> to decode a new note type (NT_ANDROID_TYPE_PAD_SEGMENT). 321Christopher Ferris improved <b>readelf</b> section flags handling, 322with test. Oliver Webb suggested <b>count -l</b>. 323Aditya submitted <b>netcat -z</b>.</p> 324 325<u>Bugfixes</u>: 326Fixed <b>printf \0</b> and <b>grep -w</b> (which never quite worked right), 327<b>ls -k</b> now switches off --block-size, 328<b>install -dm 02750</b> works (sticky bit support for -d), 329<b>tar</b> is better at following symlinks and extracting records with broken 330metadata, <b>gzip</b> now knows 331knows how to rename between "tgz/tar" extensions, 332<b>mkpasswd -m sha*</b> salt length can now range from 8-16, not just 16, 333<b>ps</b> can now handle an session ID of 0 (which is what PID 1 starts with 334before the first call to setsid()), 335<b>pwgen</b> can now use ~ in output, 336<b>xargs</b> now stops when a child exits with 255, 337<b>iconv</b> now exits with error if any characters failed to convert with -c, 338things like <b>truncate -s 1e2</b> were saying "too large" and now correctly 339say "not integer" (it's not scientific notation, it's exabyte suffix with 340trailing garbage), 341<b>timeout</b> now takes its exit code from SIGCHLD instead of wait() 342to fix a race condition (SIGCHLD can happen before calling wait and the 343kernel won't deliver status twice, so you can't wait AND have the 344signal handler). The usual round of gcc warnings got tracked down, confirmed 345to be useless, and removed, but despite gcc constantly warning "variable is 346not actually used uninitialized" there were some uninitialized variables in 347<b>dd</b> it DIDN'T warn about: found and fixed.</p> 348 349<p>Elliott added an error message to catch <b>xxd -r</b> receiving -p format 350input without -p, 351fixed <b>readelf -n</b> for x86-64 ibt/shstk notes 352(<a href=https://github.com/landley/toybox/commit/6c23ff0168c1>whatever those 353are</a>), 354and fixed <b>blkid</b> not to check filesystem signatures that would continue 355past the end of the loaded buffer. 356Michael Shavit pointed out that <b>devmem</b> command line number parsing 357wasn't unsigned (although fixing that means it no longer parses the kmg unit 358suffixes). 359Peter Collingbourne fixed <b>inotifyd</b> skipping the first mask character. 360Taisuke Matsushiro fixed a fencepost error preventing <b>stty</b> from 361setting serial baud rate to 57600. 362Askar Safin fixed a typo in mkroot/packages/busybox.</p> 363 364<u>Library</u>: 365The in-progress rewrite of <b>lib/password.c</b> no longer uses shadow.h 366(which lets it build against bionic), 367and a new <b>lib/hash.c</b> where toybox implements its own crypt() using 368the md5/sha1/sha3 etc hash functions we've already got (to work around glibc's 369ongoing deprecation of posix features), but neither are quite finished yet. 370New <b>lib/elf.c</b> with plumbing shared by <b>file</b> and <b>readelf</b>, with 371additional arm and riscv flag decoding, 372the <b>peek()</b> family of functions now return long long instead of 373int64_t so the base types are consistent on 32/64 bit, 374<b>lib/args.c</b> now handles the high bit octal escape sequences in trailing 375[-abc] blocks, so you can have punctuation participate in option groups, 376switching off an option via grouping now sets it back to its default 377value (not to zero), and numeric and date parsing report out of range numeric 378or millisecond values.</p> 379 380<u>Mkroot</u>: 381There's a <b>README</b> in the mkroot directory now. (Microsoft Github was 382picking up the README.root file used by the tar-for-web.sh script 383and showing that instead.) 384Extended <b>run-qemu.sh</b> so it can be run from other directories 385(prepending $(dirname $0)/ to linux-kernel and initramfs.cpio.gz when 386loading them into qemu). 387The initramfs now has <b>/etc/os-release</b> with the toybox $VERSION. 388The big if/else staircase of linux kernel target configurations 389moved into its own <b>get_target_config()</b> shell function, with 390a big comment block right before it explaining what all the variables 391it sets mean, and the generic kconfig symbol list got moved out to 392a <b>GENERIC_KCONF</b> variable assignment right after that function. 393The <b>microconfig</b> format got compressed slightly more 394by use of bash brace expansion, decoded to comma separated values by 395the be2csv function ala KCONF="$(be2csv SPI{,_BITBANG} IOMMU_SUPPORT)" . 396The init script no longer calls <b>oneit</b>, instead doing the work in the 397init script. (Redirect stdin/out/err to the "real" console device on 398the last line of /sys/class/tty/console/active, call setsid, trap SIGCHLD, 399reboot when the shell exits. This (and the setsid tty stealing upgrades) also 400fixed <b>ps</b> not showing any processes in mkroot because stdin/stdout weren't 401properly claimed as the controlling tty. 402The base kernel config now uses <b>CONFIG_PANIC_TIMEOUT=1</b> so 403we don't have to say panic=1 on the kernel command line anymore. 404The baseconfig now automatically adds CONFIG_MODULES and 405CONFIG_MODULES_UNLOAD when any <b>$MODULES</b> are listed. 406Moved the kernel modules archive into the "docs" directory. 407New <b>$NOTOYBOX</b> variable skips the toybox build if set. 408The <b>$DTB</b> files no longer have a path, instead using "find" 409to install them. 410The build now creates the externally loadable cpio.gz file even when 411building initramfs into the kernel, but sticks it in the docs/ subdirectory 412when it's not used.</p> 413 414<p>Several new <b>mcm-buildall.sh</b> targets (or1k, riscv32, riscv64, sh4eb) 415taking advantage of upstream musl-cross-make updating itself to use 416musl-1.2.5. The target list in mcm-buildall.sh moved into a <b>TARGETS=()</b> 417array with a big comment before it explaining the format, and the script 418is better about building host toolchains 419to build other cross compilers with: it tries to autodetect the host 420type (moving the relevant one to the front of the build list), and 421will also build a missing host toolchain when given targets on the 422command line. Added microblaze and or1k kernel configs, neither of which 423know how to exit qemu yet. (If Linux has got a reboot mechanism it knows 424how to poke that qemu is listening for on either virtual board, I haven't 425found it yet.) Also added sh4eb with FDPIC support (see kernel patch, which 426allows qemu-system-sh4eb to run the sh2eb nommu root filesystem), and armv4l 427(the 486 of arm).</p> 428 429<u>Pending</u>: 430Elliott fixed and improved <b>strace</b>, translating more system calls. 431Rob enabled command recursion in <b>toysh</b> (so it can call non-builtins as 432builtins), fixed calling "exec" on NOFORK commands (which MUST run 433in the shell's process context), redid the code to handle trailing 434backslashes gluing lines together, and it no longer leaks script 435filehandles into child processes. 436Oliver Webb noticed that OLDTOY(MAYFORK) aliases didn't become shell 437builtins, added <b>tr -t</b>, in <b>vi</b> added line gotos in ex mode, 438'g', 'v', 'j' commands, and backwards search, cleaned up stuff in 439<b>xzcat</b>, <b>getopt</b>, <b>bc</b>, and removed a bunch of useless autogenerated 440<b>bc</b> tests. Jarno Mäkipää added the list of supported commands to <b>vi</b>'s 441help text. 442Mathieu Anquetin fixed <b>ip link set</b> parsing, and NUL byte handling 443in <b>hexdump</b>. 444General cleanup passes on <b>expr</b> (which now uses 64 bit comparisons), 445<b>tr</b>, and <b>dhcpd</b>.</p> 446 447<u>Cleanup</u>: 448pass over <b>lsattr</b>, <b>getconf</b>, <b>acpi</b>, and <b>blkid</b>. 449Use FLAG() macros in <b>ifconfig</b> and <b>restorecon</b>. Removed TODO block 450from <b>nbd-client</b>. 451More error_msg("%s", str) converted to error_msg_raw(str). 452Trimmed the GLOBALS() sizes (using <b>scripts/probes/GLOBALS</b>): 453removed 1k of global data from <b>cksum</b> and switched a 2k global 454in <b>grep</b> to a malloc. Simplified scripts/probes/bloatcheck and 455mkroot/record-commands.</p> 456 457<u>Portability</u>: 458Fixed <b>truncate -s 8g</b> on 32 bit hosts, 459updated <b>mcm-buildall.sh</b> to autodetect host type and automatically 460select newer package versions, 461and moved the <b>linux32</b> command to its own file because BSD can't 462build it. 463MacOS had a use-after-free with the getpwuid() return value being recycled 464a more aggressively than Linux. 465Elliott fixed diff.test to work with mksh, and posix_fallocate() 466to work with MacOS. 467More commands enabled in <b>make bsd_defconfig</b> and <b>make 468macos_defconfig</b>. 469Added some (void) typecasts to netstat, oneit, and watchdog to try to 470shut up compilers that insist on responding to uncorrectable failures. 471(Such as fgets() reading known /proc data from the kernel, writes to 472the watchdog timer that will reboot the system if they fail, or PID 1 calling 473dup() on stdin to create stdout and stderr after closing the old ones. 474If any of that goes wrong, what are we supposed to do about it?)</p> 475 476<u>Documentation</u>: 477The website now has a <a href=quick.html>quickstart</a> page, 478and the site now defaults to the about page (updated index.html symlink). 479The "current release" is now in the common header displayed by most pages. 480The mkroot dir has a 481<a href=https://github.com/landley/toybox/blob/0.8.11/mkroot/README>README</a>, and its <a href=faq.html#mkroot>faq entry</a> 482has been updated. 483New option to compress help text with <b>gzip</b>, and help text size now shows 484up in <b>make bloatcheck</b>. 485Fix <b>sed --help</b> to show full help, <b>patch -F</b> is now mentioned 486in the help text, tweaks to help text in <b>cut</b>, <b>grep</b>, 487<b>blkdiscard</b>. 488Elliott did a cleanup pass over "usage:" lines and alphabetized more help text 489options. 490Christian Koestlin fixed a typo in the <b>dmesg</b> docs. 491The <a href=release.txt>release 492procedure checklist</a> has been updated, and the README generated 493by <b>mkroot/tar-for-web.sh</b> got updated. 494New faq entry explaining why toybox doesn't have (or need) cttyhack, 495and the architecture list got updated to match mcm-buildall.sh. 496The usual roadmap updates.</p> 497 498<p>New <b>scripts/probes/cmd2dpkg</b> script shows what 499debian package (if any) each toybox command lives in, but only checks 500installed packages (dpkg-query -S "$(which $COMMAND)") and debian breaks 501up source packages strangely, so did not used it to redo the roadmap section 502listing packages (created using Linux From Scratch's list of commands 503installed by each source package).</p> 504 505<u>Plumbing</u>: 506The <b>yes</b> command now uses writev() to go way faster, mostly to 507prove it can. 508The default <b>stdout</b> buffer type switched to block buffering to make 509Elliott happy, which is a rich source of bugs (many of which Elliott 510has already whack-a-moled) and means among other things 511that mixing write() and fwrite() calls now requires manual flushing. 512Meanwhile xflush() became xferror() because it usually 513wasn't flushing anyway, and now that's more noticeable. 514Removed <b>TOYFLAG_LOCALE</b> and just set utf8 locale for all commands. 515The dirtree code now sets <b>DIRTREE_SYMFOLLOW</b> in "again" when we followed 516a symlink to get to this node, callers checking <b>DIRTREE_COMEAGAIN</b> need 517to mask out the bit they're interested in. 518Improved <b>scripts/probes/findglobals</b> output to be more legible.</p> 519 520<u>Test suite</u>: 521Added <b>make test_help test_install</b> targets. (The standalone 522command targets conflict with the help/install make targets, but you 523can still build them via scripts/single.sh and there's no reason not 524to be able to individually test them.) 525New <b>utf8locale</b> shell function attempts to enable a UTF8 locale for 526TEST_HOST, which was used to fix <b>wc.test</b>. 527Colin Cross fixed <b>pidof.test</b> when stdout is a socket. 528Something called "ecryptfs" has a maximum filename length of 143 bytes, 529so our <b>tar.test</b> for long filenames support needs to be skipped there 530because the filesystem can't handle it. Similarly, our <b>truncate.test</b> 531was failing because storing nothing on ecryptfs wasts more space than expected. 532The <b>tar</b> tests now fetch user/group names with "stat" because 533between Linux, Android, FreeBSD, and MacOS, there aren't really consistent 534user and group names for any existing files. Added nbd-client.test 535Thiébaud Weksteen fixed the <b>getfattr</b>/<b>setfattr</b> tests on 536filesystems using selinux. Oliver Webb added <b>sha3sum</b> tests. 537Elliott stabilized another slightly racy <b>ifconfig</b> test (big test 538farm, hits the weird corner cases).</p> 539 540<u>Build</u>: 541Lots of changes to <b>scripts/make.sh</b> and friends to speed up and parallelize 542header generation. The build now figures out which toys/*/*.c files 543to compile by searching for unindented USE() macros wrapping NEW(commandname), 544which among other things fixes enabling the builtins when selecting just 545CONFIG_SH in menuconfig (instead of doing a "make sh" single command build). 546It also probes for optional shared libraries in parallel. 547New <b>scripts/probes</b> directory to collect scripts that aren't actually 548used by the build, with a README in it. 549Updated .gitignore so it doesn't complain about spurious name collisions in 550subdirectories. Elliott shut up a 551<a href=https://github.com/landley/toybox/commit/2c36636377e1>truly sad</a> 552warning where despite sizeof(ptrdiff_t) always being sizeof(long), gcc 553nevertheless warns if you printf("%ld", ptr-ptr) and wants a magic invented 554"%td" type on 32 bit systems (but is just fine on 64 bit). (Or we could add a 555literally NOP typecast to long.) And silenced a spurious gcc 13.2 warning 556in date.c. And Rob shut up <a href=https://github.com/landley/toybox/commit/a17fee778851>more broken gcc warnings</a>. 557Updated .gitignore so it doesn't complain about spurious name collisions in 558subdirectories.</p> 559 560<a name="30-07-2023" /><a href="#30-07-2023"><hr><h2><b>July 30, 2023</b></h2></a> 561<blockquote> 562<p>The way it functioned was very interesting. When the Drink button was 563pressed it made an instant but highly detailed examination of the subject's 564taste buds, a spectroscopic analysis of the subject's metabolism and then 565sent tiny experimental signals down the neural pathways to the taste centers 566of the subject's brain to see what was likely to go down well. However, no 567one knew quite why it did this because it invariably delivered a cupful of 568liquid that was almost, but not quite, entirely unlike tea.</p> 569<p>- The Hitchhiker's Guide to the Galaxy</p> 570</blockquote> 571 572<p><a href=downloads/toybox-0.8.10.tar.gz>Toybox 0.8.10</a> 573(<a href=https://github.com/landley/toybox/releases/tag/0.8.10>git commit</a>) 574is out, with prebuilt <a href=downloads/binaries/0.8.10>static binaries</a> and 575<a href=downloads/binaries/mkroot/0.8.10>mkroot images</a> 576bootable under QEMU (built using a <a href=https://github.com/landley/linux/tree/mkroot-6.4>lightly patched</a> linux-6.4).</p> 577 578<p>New command <b>shuf</b>, and Elliott added <b>i2ctransfer</b>. 579Finished cleaning up <b>dd</b> a promoted it out of pending.</p> 580 581<p>Lots of changes to mkroot, <a href=http://landley.net/talks/mkroot-2023.txt>working</a> 582on updated documentation but the release is already overdue...</p> 583 584<p><u>Features</u>: 585Added a bunch of options to <b>tar</b> (-s --ignore-case --wildcards 586--anchored --wildcards-match-slash --one-file-system), <b>ls --sort</b> 587(which takes a comma separated list of fallback sorts, and new 588sort types to sort by extension, case insensitive, and directory first), 589<b>cpio -R</b> to set user/group of all files and <b>cpio -L</b> to 590follow symlinks, <b>gzip -t</b> to test file integrity, <b>find -readable</b>, 591<b>cut -DFR</b> now has --longopts aliases (as suggested by the coreutils list). 592Taught <b>blkid</b> to recognize iso (cdrom) image files, <b>oneit</b> now 593autodetects console from sysfs with no -c argument, added "next address" 594support to <b>i2cget</b> and enforced bus limit, 595added posix 1x2x3 support to <b>dd</b> (with overflow detection, and 0x 596at the start still means hexadecimal).</p> 597 598<p>Jarno Mäkipää taught <b>i2cget</b> to read without passing a command byte. 599Elliott added <b>xxd -e</b> (little endian), <b>lspci -x</b> (hex dump of 600pci config space), <b>ls --block-size</b>, widened the default 601VSIZE field width in Android's default <b>ps</b> display (they're up to 10 gigabytes of 602ASLR padding for hello world), and increased the <b>vi</b>/<b>top</b> 603output buffer size to reduce flicker redrawing the screen.</p> 604 605<p><u>Bugfixes</u>: 606Taught <b>switch_root</b> to move mountpoints into the new root. 607The <b>head</b> command now uses lseek to restore unused data from seekable 608inputs (tested with <b>read i; echo =$i; head -n 1; done < input</b>). 609In <b>lspci</b> handle PCI bridges and don't show "revision 0". 610Missing /proc entries no longer cause <b>vmstat</b> errors, the output 611fits into 80 columns more often, and adds reclaimable to the cached field. 612Fix </b>help -u -au</b>, and the "see othercommand" generation. 613Adding fuzz support subtly broke <b>patch</b> in a couple places: fixed -R and 614matching EOF. Trying to create a directory that already exists is no longer an 615error for <b>cpio -i</b>, and <b>-d</b> just warns about directory creation 616failure instead of exiting. Adding <b>--wildcards</b> logic to <b>tar</b> 617broke the full string 618matching (it checked after every / but not at the start of the string 619when there weren't any), and <b>tar</b> now waits for the compression program 620to exit and passes on any error result instead of killing it (which could 621truncate output). Fixed more <b>realpath/readlink</b> corner cases, with tests. 622Fix a <b>sed</b> bug when s//&/ (the copy-matched-text operator) was the 623first change made to the string. Fix error message for <b>find</b> -commands 624that need an argument but were at the end of the command line (it said 625"unknown command" instead of "command needs argument"). Fixed <b>killall5</b> 626parsing of command names with spaces in them. Fixed <b>grep -m -A</b> used 627together (-m stops new matches but -A still shows lines after match). 628Fixed <b>mv</b> to --preserve=all when copying between filesystems. 629Moved the EPIPE skip from perror_exit() to verror_msg() so we never 630output a "hey, our output pipe unexpected closed!" error message when 631we're not the last command in a pipeline.</p> 632 633<p>Franke Busse reported that <b>bunzip2</b> was deleting its input file 634on error, and <b>date -s +abc</b> was segfaulting. 635Jooyung Han fixed <b>nsenter</b> using -C and -t together. 636More fixes from Eric Roshan-Eisner's toysh fuzzing, and he also sent a fix 637for an error message in the <b>patch</b> command. 638Elliott fixed stdin handling in <b>log</b> and <b>logger</b> to read a line at 639a time and log each one separately, fixed an off by one bounds check 640in <b>readelf</b> (and factored out the bounds checking logic so it's 641common between sections, and improved the error messages), fixed 642a <b>gpiod</b> crash.</p> 643 644<p><u>Library</u>: 645Implemented DIRTREE_BREADTH to do breadth first processing of directories 646(among other things, allowing entries to be sorted and processed in a 647consistent order). The dirtree plumbing now has macros for the mode bits 648instead of hardwired numbers, 649better adler32 calculation in deflate.c, 650and <b>fs_type_name()</b> can now convert most of the linux/magic.h constants 651into "mount -t TYPE" names.</p> 652 653<p><u>Mkroot</u>: 654Moved mkroot into its own directory: you now run <b>mkroot/mkroot.sh</b> instead 655of scripts/mkroot.sh, and all the mkroot plumbing should be under there now. 656What was scripts/root is now <b>mkroot/packages</b>, and I added a "busybox" 657package to download and build that (useful for testing command 658implementations side by side).</p> 659 660<p>Reorganized the contents of each root/$ARCH build directory so only 661the files necessary to run are at the top level, and everything else 662(kernel configs) is in a docs/ subdirectory. Added linux-microconfig 663with the three CSV lines actually used by the build. 664New <b>mkroot/tar-for-web.sh</b> script that generates tarballs for 665uploading to the web page and adds two README files (one at the top level 666and one to each tarball's docs directory). The generated tarballs don't 667contain the fs/ dir, instead the README explains how to extract 668and recreate the cpio.gz archive.</p> 669 670<p>Taught KCONF=value,value,value... CSV format to understand NAME=VALUE 671entries, and merged most KERNEL_CONFIG lines into KCONF. 672(Note that while it can handle NAME="this that" you need to escape it as 673NAME=\"this\ that\" to work right.) Added the legacy NLS 674dependencies the VFAT driver needs (even though we selected UTF8) to the 675base config. The init script now tests if stdin is open via 676"2>/dev/null <0 || blah" so only does the exec redirect for /dev/console 677when necessary. 678Fixed the .config checker to replace the toybox config when CONFIG_SH=y 679isn't set, so "make defconfig; mkroot/mkroot.sh" should work now. 680Switched initramfs packaging to use more portable <b>cpio -R 0:0</b> 681instead of --no-preserve-owner. Moved the -m 256 before $@ in run-qemu.sh 682so setting memory size on the command line overrides the default. 683The <b>mkroot/record-commands</b> setup script will now use logpath 684out of the $PATH if it exists, rather than having to compile it each time. 685Bo SvangÃ¥rd fixed a typo in the cpio command line that's ignored by toybox but 686breaks with the host tools.</p> 687 688<p><u>Pending</u>: 689Moritz Weber put in more work on <b>git</b> (still very unfinished). 690Eli Lipsitz fixed bugs in init and login. Elliott added risc-v support 691to strace, and fixed :w in vi (it didn't 692takes a filename argument and would crash if you started with no filename) 693and improved error reporting. Jarno Mäkipää taught vi about backspace 694and improved empty file handling. Rob fixed vi -s. Various cleanups to 695<b>tcpsvd</b>.</p> 696 697<p>More toysh work: command line option parsing stops at first nonoption argument, 698all piped segments are now implicitly subshelled so variable 699lifetimes match bash's and loops don't have to exit before proceedinng to the 700next one (ala "while true; do echo hello; done | while read i; do echo $i; done" 701hanging on a full output pipe). Several changes to HERE document processing 702(variable expansions can cause line continuations in a HERE document, etc), 703and better line continuation handling in general, added set -u, 704fixed if/while return codes, wired up the "local" builtin, fixed an escape 705passthrough bug reported by Mingliang Hu, support undelimited redirects 706(ala cat<file with no spaces), fixed ${X::} with no arguments.</p> 707 708<p><u>Cleanup</u>: 709Converted <b>chgrp</b> to FLAG() macros and did some minor cleanup on <b>bootchartd</b>, 710(which needs tests), <b>basename</b>, <b>bzcat</b>, <b>test</b>, 711<b>i2cdetect</b>. Elliott switched <b>flock</b> 712to FLAG() macros, switched nohup to use octal permissions insted of macros, 713widened the default VSIZE field width in Android's default <b>ps</b> display 714(which is now inserting 10 gigabytes of ASLR padding into "hello world"), and increased 715the vi/top output buffer size to reduce flicker redrawing the screen. Multiple cleanups to 716<b>cpio</b>. Added requested --gratuitous-longopt synonyms to <b>ls</b>. 717Consistently capitalize TODO comments in the tree so they're easier to search for. 718Silence yet more perennial "warning: variable is never used uninitialized" gcc false 719positives.</p> 720 721<p><u>Portability</u>: 722utf-8 support still isn't automatic in libc (you have to set an appropriate locale to 723enable it), and locales aren't entirely standardized: try C.UTF-8 (which MacOS hasn't got) 724and fall back to en_US.UTF-8 (which Gentoo hasn't got). 725Removed itoa() from seq.c because cygwin pollutes the namespace with 726DOS debris.</p> 727 728<p>Dan Bryant renamed xrunread()'s "stdin" argument to "to_stdin" to work around 729broken headers that #define stdin as a macro instead of a global variable. 730Elliott enabled copy_file_range() on Android.</p> 731 732<p><u>Documentation</u>: 733Calling <b>command --help</b> and <b>toybox --help command</b> shows 734the toybox URL banner, but "help command" doesn't. 735Reorder the <b>find</b> help text so match filters that take an argument are mostly 736in the left column. Minor help text tweaks in sleep, find, xxd, cpio. 737The README now points to the same binary cross compilers as the FAQ. 738Update <b>sed</b> help text now that -e and -f get processed in command 739line order when used together.</p> 740 741<p><u>Plumbing</u>: 742The FLAG() macros now always return 1 or 0, so we can CONSTANT*FLAG(X) without needing 743a !! in there. The help plumbing now has HELP_FLAGS. 744Each command's option string in NEWTOY() may now use octal escapes with the 745high bit set to use control characters as option flags, with the resulting 746FLAG_X## macro using the 2 digit hex value (without the high bit set). For 747example if you needed "command -:" then ":" is ascii 58 (hex 3A, 748toybox's <b>ascii</b> command is useful for this), +128 is 186, which in octal 749becomes "\272" and tested with if (FLAG(X3A)) in the code.</p> 750 751<p><u>Test suite</u>: 752Each command.test now runs in an empty subdirectory, and working files like 753expected/actual go elsewhere (only "input" is created for you in there, and only 754when you provide input in the current test's 4th argument), so the directory 755listing doesn't contain anything the test didn't put there. (This simplified 756several tests.) 757The realpath test broke on mksh (which doesn't export $PWD by default). 758Elliott fixed the ifconfig tests on android. General tidying and adding 759tests to sed, ls, modinfo. The tar tests broke on github because it 760switched to a filesystem that can't do sparse files with 4k granularity, 761added a check to skip those tests (and also skip them on MacOS, which 762fails for a different reason.) 763When a test's name field was blank (meaning the actual test command line 764should be displayed as the name) the test plumbing was adding the command 765name to the display twice. 766The <b>make tests</b> target now defaults to building toybox with ASAN when 767not testing an existing binary. (You still have to specify ASAN=1 manually 768for individual command tests.)</p> 769 770<p>Fixed <b>demo_utf8towc</b> which had the range check backwards (0x10ffff is 771the biggest _output_, not input). The range check was an attempt to avoid 772spending a long time going through all 4 billion entries, but wasn't testing 773everything it needed to, so now it advances the counter with a bit shift 774based on the number of bytes consumed by the match (avoiding large swaths 775of redundant checks but still theoretically testing all meaningfully different 7764 byte inputs).</p> 777 778<p><u>Build</u>: 779Fix return code of verbose mode in scripts/build.sh so <b>make V=1</b> succeeds. 780Speed up the MacOS build: their gsed is slow to launch so avoid one repeated call in a 781loop, and reimplement the wait $PID codepath because their bash is 17 years old. 782The llvm+bionic toolchain warns about printf(" "+FLAG(m)) unless you add an 783extra ,0 argument, but gcc+glibc warns if you DO add the extra argument... so 784I used the write() system call to avoid the compiler "helping" in a way I 785can't make it STOP. Similarly, now that FLAG(u) is always 1 or 0, 786<b>size = FLAG(u) ? : sizeof(toybuf)</b> 787is either 1 or 4096, which is what I _want_, and the compiler warning about it 788is counterproductive. LLVM needs a new -Wno-invalid-source-encoding flag 789or else it complains about octal escapes in string constants. (When I want 790to put utf-8 in there I can just do so, why would I octal escape it? 791Stop "helping".)</p> 792 793<a name="10-01-2023" /><a href="#10-01-2023"><hr><h2><b>January 10, 2023</b></h2></a> 794<blockquote> 795<p>"Why," Arthur said, "is there a sofa in that field?"</br> 796"I told you!" shouted Ford, leaping to his feet. "Eddies in the space-time 797continuum!"</br> 798"And this is his sofa, is it?"</p> 799<p>- The Hitchhiker's Guide to the Galaxy</p> 800</blockquote> 801 802<p><a href=downloads/toybox-0.8.9.tar.gz>Toybox 0.8.9</a> 803(<a href=https://github.com/landley/toybox/releases/tag/0.8.9>git commit</a>) 804is out, with prebuilt <a href=downloads/binaries/0.8.9>static binaries</a> and 805<a href=downloads/binaries/mkroot/0.8.9>mkroot images</a> 806bootable under QEMU (built using a <a href=https://github.com/landley/linux/tree/mkroot-6.1>lightly patched</a> linux-6.1).</p> 807 808<p>The new <b>nbd-server</b> command interoperates with nbd-client to serve 809network block devices (using the v1 protocol), and Moritz Weber contributed a read-only <b>git</b> 810implementation to pending.</p> 811 812<p><u>Features</u>: New <b>grep</b> fast path for fixed or simple patterns that 813don't need the full regex engine (I.E. "^", ".", "$" but not "*" or "[]") 814most noticeable when searching for many patterns at once. 815Improved <b>tar --xform</b> support parsing flags= and trailing s/// scope 816flags (but using --xform now requires toybox sed in the $PATH).</p> 817 818<p>Added <b>sort -C</b>, <b>lsusb -i</b>, <b>netcat -n</b>, 819<b>swapoff -a -v</b>, <b>httpd -v</b>, <b>nbd-client -b</b>, 820a <b>uname -p</b> stub to mollify package builds, 821<b>sleep</b> accepts multiple arguments, 822<b>sed</b> now parses interlaced -e and -f arguments in order, 823several new options in <b>readlink</b> and <b>realpath</b> (and better 824handling of relative paths), and UDP mode in netcat is more useful now: 825<b>netcat -u -s 127.0.0.1 -p 9876 -l</b> can type at <b>netcat -u 127.0.0.1 9876</b> 826in both directions. (Figuring out when the other side hangs up is an unsolved 827problem, but that's UDP for you.) 828Elliott added <b>ls -N</b>.</p> 829 830<p><u>Bugfixes</u>: Fixed off by one error in <b>sort -c</b> output, 831fixed <b>sed -z</b> and other sed cases where Linux has never obeyed 832Posix (N at EOF does a default print, l escapes \n), 833<b>patch --dry-run</b> should no longer create or delete files, 834<b>gzip</b>/<b>zcat</b> couldn't handle concatenated archives, 835the ./configure of gmake 4.3 depends on <b>test</b> treating one argument -a as a synonym for -e, 836autodetectiong compression types in <b>tar</b> should be more reliable now 837and sanitize weird path corner cases (like .. past /) better, 838<b>scripts/make.sh</b> (and thus "make toybox") should no longer truncate log 839files stderr is redirected to, 840<b>httpd</b> now handles ? and # in URLs, 841<b>xxd</b> disables columns for -c 0 and groups for -g 0 (so you can get 842a long interrupted string of hex digits out of it), two fixes to 843<b>mountpoint</b> (the conversion to same_file() was inappropriate because 844the logic isn't quite the same, and -q should also quiet "not found" errors), 845fixed <b>httpd</b> not always displaying index.html files instead of directory 846contents, fixed <b>comm -</b> not recognizing it as stdin, and multiple fixes to 847<b>timeout</b> which now kills process group and thus child processes, 848isn't suspended by SIGTTIN, and recursive commands it calls don't 849inherit an inappropriate SIGCHLD handler.</p> 850 851<p>Yi-Yo Chiang fixed loopback <b>mount</b> (the recent switch to xrunread() 852kept the newline from losetup's output) and added <b>tail +123</b> 853(old-style synonym for tail -n +123). 854Daniel Mentz added scmversion to the <b>modinfo</b> tag list (Android uses 855it for external modules). Alexander Holler fixed <b>su</b> to not require 856/etc/shadow when run as root, and nomas2000 reported an inverted test in 857<b>date</b>'s check for trailing rubbish. Daniel Mentz fixed an off by 858one in <b>grep -f</b> that discarded the last character of the pattern file's 859contents. Kelvin Zhang made <b>modinfo</b> better at handling symlinks. 860Antoni Villalonga fixed <b>fmt</b> on 64 bit big endian systems (like s390x and 861powerpc). Li Cheng fixed <b>mount</b>'s type detection not autodetecting 862the need to bind mount files when type "none" was specified in fstab. 863Tomasz Sterna reported that <b>modprobe</b> resolving dependencies shouldn't 864feed a NULL pointer to the syscall's options arguments (the kernel goes -EWTF), 865and Vincent Donnefort made modprobe work when /proc/modules isn't available.</p> 866 867<p><u>Library</u>: 868The TOYFLAG_ARGFAIL() exit value is now the default <b>error_msg()</b> sets, 869<b>sendfile()</b> now falls back to the read/write loop for any error (not just 870EINVAL, the kernel sends ENOSYS and EXDEV and who knows what else), 871<b>xgetrandom()</b> can now return arbitrary amounts of data (looping internally 872as necessary), 873<b>xwaitpid()</b> returns status 127 for cases (like bad PID) which don't 874return a status for the PID, 875several fixes to <b>xabspath()</b> as part of the readlink/realpath work, 876new <b>octal_deslash()</b> to remove octal escapes common in kernel strings 877(ala /proc/mounts), 878the <b>FLAG_x</b> macros always use a 64 bit type now (so you don't have to 879worry about toys.optflags &= ~FLAG_x; blanking the top 32 bits), 880replaced memcmp() with new library function <b>smemcmp()</b> to placate ASAN 881when returning first difference in known differing arguments of unequal length. 882Elliott added three more filesystem types to <b>fs_type_name()</b> which is 883used by <b>stat -f</b> and friends.</p> 884 885<p><u>Mkroot</u>: 886New <b>scripts/test_mkroot.sh</b> runs each target under qemu to confirm 8871) it boots, 2) the block device works, 3) networking works, and 4) the clock is set 888reasonably. It runs them in parallel, with an httpd instance on the host to 889fetch a file from, and a timeout to detect hangs. 890New <b>scripts/root/overlay</b> package copies $OVERLAY directory into 891target filesystem, ala <b>scripts/mkroot.sh overlay OVERLAY=$PWD/blah</b> 892The init script now mounts any /dev/?da on /mnt and when /mnt/init exists 893sets $HANDOFF to that instead of running a shell prompt, so you can provide 894automated control images that run code in the emulator automatically. (The 895emulator process still when whatever $HANDOFF called exits.)</p> 896 897<p>Some kernel changes finally made it upstream so we can drop workarounds 898for them, such as kernel commit 899<a href=https://github.com/torvalds/linux/commit/f8f0d06438e5>f8f0d06438e5</a> 900fixing the <a href=https://landley.net/notes-2014.html#13-08-2014>longstanding</a> 901"allnoconfig has =y" issue so miniconfig doesn't need to add a magic #comment 902line forcing a symbol OFF anymore. The remaining kernel patches (including 903several maintained locally but hadn't merged into mkroot) moved to a 904<a href=https://github.com/landley/linux/tree/mkroot-6.1>seperate repository</a> 905so mkroot.sh doesn't have to call sed on the build snapshot. (They're are also 906included in the mkroot binary release directory, and were all submitted to lkml 907months if not years ago.)</p> 908 909<p>The host airlock setup (creating the temporary restricted environment 910mkroot builds packages in both so they don't pick up strange dependencies from 911the host, and to prove the build can use the toybox commands) 912removed a bunch of commands (dd, diff, vi, xzcat, ar, nm) no longer 913needed by the linux-6.1 build. The kernel patches also remove the need 914for gcc (it can use the "cc" symlink all modern distros install, and 915autodetects whether that points to gcc or clang) and bc (which is just 916<a href=https://landley.net/notes-2013.html#18-05-2013>generally obsolete</a>). 917Left those two in for one more release, but NEXT time building a vanilla 918kernel without the <a href=https://landley.net/toybox/downloads/binaries/mkroot/0.8.9/linux-patches/0001-try-generic-compiler-name-cc-before-falling-back-to-.patch>gcc removal patch</a> 919and <a href=https://landley.net/toybox/downloads/binaries/mkroot/0.8.9/linux-patches/0004-Replace-timeconst.bc-with-mktimeconst.c.patch>bc removal patch</a> 920may require you to add <b>HOST_EXTRA="gcc bc"</b> to the mkroot command line 921for the airlock build to add extra symlinks to root/build/airlock.</p> 922 923<p>Added a kernel build for powerpc64 big endian, 924m68k and powerpc now support "run-qemu.sh -hda file.img", and m68k only 925has one /dev/?db (disabled Apple Desktop Bus in the config). 926Added basic module support (the kernel build saves modules in modules.cpio.gz 927so it can extract them when you rebuild the root filesystem but not the kernel) 928and added <b>scripts/root/tests</b> which (among other things) builds two 929innocuous modules that depend on each other (fscache and cachefiles) for 930modprobe/insmod testing that hopefully won't break anything else. (If 931there was a scripts/root/pending/ tests would be in it: it does not actually 932run the test suite under mkroot yet, because toysh needs work.)</p> 933 934<p><u>Pending</u>: Elliott made <b>strace</b> build on 32-bit x86. 935James Farrell added <b>diff -f</b>. 936Alexander Holler fixed two parameter substitution bugs in <b>toysh</b>. 937Rob fixed assignment suppression, && and || parsing, math 938priority, a number of ASAN test failures, untangled and commented the 939brace expansion logic, and other ongoing shell work.</p> 940 941<p>A large rewrite of diff happened but couldn't be checked in unfinished 942because if people are using stuff out of pending, how do you do a pending for 943pending? (Meta-pending?)</p> 944 945<p><u>Cleanup</u>: 946<b>cksum</b>, <b>od</b>, <b>file</b>, <b>getopt</b>, <b>gpiod</b>, 947<b>nbd-client</b>, better output messages for <b>host</b>, 948and make <b>base64</b> use line buffering. 949Moved -Wno-string-plus-int into portability.sh and had it only apply to 950clang. Moved num_cache() out of lib into netstat (its only user).</p> 951 952<p>In the test suite plumbing, $SKIPNEXT was replaced by <b>$SKIP</b> holding 953a count that counts down with each test (so you can easily skip a block of 954tests). Fixed <b>txpect</b> to actually listen to error code (oops) and to not 955complain about multiple X. The old dochroot() and mkchroot() shell functions were removed from 956scripts/runtest.sh because they required root access to use (which mkroot 957does not). More conversion to use <b>testcmd</b> in individual command.tests 958(both to remove workarounds for shell builtins, and to be more concise). 959Remove redundant cleanup from several tests (which run as proper child processes 960now, and the work directory is deleted and recreated between each, so 961individual tests need less cleanup).</p> 962 963<p><u>Portability</u>: More tests now pass on MacOS and FreeBSD 964(weird filesystems, root vs wheel group name, dealing with headers that 965define stdin as a macro and don't define PATH_MAX, the ancient bash in MacOS 966doesn't understand &>>, won't let normal users use "chmod +s", 967symlinks don't have all permission bits set, no O_PATH and the Linux definition 968is used for something else, and so on), at least half of which Elliott did. 969Not sure we've quite got Mac and BSD 970doing "gmake test_singlecommand" right yet, but it's a lot closer. 971Added all the commands to <b>make bsd_defconfig</b> that compile on FreeBSD 13 972(which is no guarantee they WORK). 973ASAN=1 should work in more places now, including <b>make_test_command</b>. 974The build uses $CC insted of cc in more places, and doesn't apply $ASAN 975for $HOSTCC (the legacy kconfig/ directory stands no chance).</p> 976 977Fedora inexplicably sticks \x09 style hex escapes into the dmesg output and 978expects them to be parsed. 979Android is still running the test suite under mksh which doesn't 980understand bashisms like <(command). LLVM insisted that "ptr - 1" could 981never be NULL (demonstrably untrue) and needed some (unsigned long) typecasts 982to force it to actually do the math. 983Khem Raj aliased timer_settime to timer_settime64 when the first isn't 984available. Antoni Villalonga debianized some install paths and added 985a bunch of --longopt synonyms for existing options that package builds 986use.</p> 987 988<p><u>Documentation</u>: The <a href=help.html>help page</a> 989generation logic (toybox help -av) now filter out invisible aliases and 990outputs "See command" for visible aliases, plus fixes for broken corner 991cases like <b>nohup --help</b>. 992The <a href=https://landley.net/toybox/git>local git repo</a> 993now has a simple index.html generated by a 994<a href=https://github.com/landley/toybox/blob/0.8.9/scripts/git-static-index.sh>small shell script</a>, 995linking to the <b>git format-patch</b> files for 996each commit, applicable via <b>git am</b>.</p> 997 998<p>There was a 999<a href=https://github.com/landley/toybox/pull/244>bug report</a> that specifying the cross compiler prefix twice 1000includes the prefix twice (ala CROSS_COMPILE=prefix- CC=prefix-cc results in
1001prefix-prefix-cc) which is what happens when you ask it to do that, yes. It's now 1002<a href=faq.html#cross>better documented</a>. (See also 1003<a href=https://lkml.iu.edu/hypermail/linux/kernel/2202.0/01505.html>this kernel patch</a>.)</p> 1004 1005<p>Yi-Yo Chiang's losetup fix spawned an email thread where the 1006android developers <a href=http://lists.landley.net/pipermail/toybox-landley.net/2022-August/029139.html>explained</a> why Android doesn't (and can't) 1007use devtmpfs. Rob forwarded a link to linux weekly news to see if he could get 1008any attention from the kernel guys, but they didn't reply.</p> 1009 1010<a name="12-08-2022" /><a href="#12-08-2022"><hr><h2><b>August 12, 2022</b></h2></a> 1011<blockquote><p> 1012Huge as office blocks, silent as birds. They hung in the air exactly the same 1013way that bricks don't.</p> 1014<p>- The Hitchhiker's Guide to the Galaxy</p> 1015</p></blockquote> 1016 1017<p><a href=downloads/toybox-0.8.8.tar.gz>Toybox 0.8.8</a> 1018(<a href=https://github.com/landley/toybox/releases/tag/0.8.8>git commit</a>) 1019is out, with prebuilt <a href=downloads/binaries/0.8.8>static binaries</a> and 1020<a href=downloads/binaries/mkroot/0.8.8>mkroot images</a> 1021bootable under QEMU (built using vanilla linux-5.19).</p> 1022 1023<p>No new or promoted commands this time. (Time based releases mean NOT 1024holding the release to wait for things like that.)</p> 1025 1026<p>Thanks to Google for sponsoring the project's maintainer to focus on toybox 1027for a while. (They want a hermetic build, I want Android to build under 1028Android. Working on it...)</p> 1029 1030<p><u>Features</u>: 1031The new <b>timeout -i</b> option kills a command for inactivity (output 1032to stdout resets the timer). 1033Basic support for <b>tar --xform</b> (but not yet the special tar-only s///x 1034options). 1035--long options now support unambigous abbrevations by default, so you can 1036go "ls --col" and it'll figure out you mean --color. 1037Added <b>blkid -o</b> understanding "full", "value", and "export" output 1038formats. Added <b>nsenter -C</b> (cgroup namespace) and <b>-a</b> (enable all 1039supported namespaces). 1040Added <b>mount -R</b> and switched the bind mount default to be recursive 1041(it still autodetects --bind and --loop mounts based on the source/target type, 1042all --bind or --rbind do is switch the recursive flag on and off).</p> 1043 1044<p>Elliott implemented <b>tar --null</b>, 1045and taught <b>file</b> to recognize some Linux kernel images, and 1046binaries for Linux's <a href=https://github.com/torvalds/linux/commit/fa96b57c1490>newly merged</a> 1047arch/loongarch (a <a href=https://linleygroup.com/newsletters/newsletter_detail.php?num=4623>mips64 variant</a> 1048from <a href=https://phoronix.com/news/LoongArch-MIPS-Copy-Kernel>china</a>).</p> 1049 1050<p><u>Bugfixes</u>: 1051Several cleanups to taskset, mostly due to reviewing it before making 1052an explainer video, but the review found an inappropriate use of 1053TOYFLAG_STAYROOT that got fixed. 1054Argument parsing for <b>unshare</b> and <b>nsenter</b> should now match 1055debian's (I.E. "unshare -npu" works the same as "unshare -n -p -u" not 1056"unshare --net=pu"). 1057<b>kill 1</b> no longer says "unknown pid 1: not permitted" (it 1058says "bad pid" like everywhere else). 1059<b>test -x</b> and friends now test live access, not permission bits. 1060(Root ignores a lot of permissions, chmod u-x blocks you from executing your 1061own file even if g+x and o+x should let you access it, things like selinux can 1062further break the unix permission model, etc). 1063 1064<p>In lib/ a fencepost error in <b>chomp()</b> could read one byte past the start of the string, and 1065<b>xpclose_both()</b> no longer closes the parent's stdin and stdout 1066when they're explicitly passed through to the child. 1067In scripts/runtest.sh, moving tests into subshells broke failure reporting with VERBOSE=all. 1068While "make install" was creating directories when $PREFIX pointed to a 1069nonexistent directory, "make toybox" didn't even though it tried to place the 1070output file there.</p> 1071 1072<p>James Farrell switched the test suite from detecting root via $UID to 1073$(id -u). (It's not a security issue, this just controls which tests we attempt. 1074It's another workaround for missing features in mksh.) 1075Taolaw spotted that httpd could segfault when xabspath() returns NULL 1076(for error conditions such as infinitely looping symlinks 1077or "." not resolving when the current directory was deleted).</p> 1078 1079<p><u>Documentation</u>: 1080We <a href=design.html>recorded</a> the 1081move from C99 to the C11 standard, which was actually made a while ago: 1082the typecast constants we've 1083been using for a while are a 2011 feature not available in the 1999 edition 1084of the C standard. (This came up when we had to move to C11's "_Noreturn" to work around a compiler bug in llvm. 1085Yes __attribute__((__noreturn__)) should be identical, that's the bug part.)</p> 1086 1087<p>The website's nav bar got updated (among other things, linking to 1088our <a href=https://www.youtube.com/channel/UC4VFy3wc1nzq5tUHhiti6fw>youtube 1089channel</a>), and the FAQ entry on cross compiling got 1090expanded. 1091The toys/example commands moved from "make list" to "make list_example" 1092(since they're not enabled in defconfig either). 1093The cross compiling example command line in "make help" now sets 1094LDFLAGS=--static instead of putting that in CFLAGS.</p> 1095 1096<p>The web archive is back, and since the missing messages were 1097never delivered (the mailserver trying to deliver them timed out and bounced 1098them back after a couple days) this means unlike previous failures the web 1099archive isn't out of sync with reality: the hole was the list actually being 1100down until Dreamhost tech support figured out what they broke this time.</p> 1101 1102<p><u>Pending</u>: 1103More work on toysh: case/esac was confused by nested flow control blocks, 1104support for [[ < > =~ ]] tests, implemented the rest of the $((math)) 1105operators and support for ((math)) tests and "for ((math;math;math))" loops, 1106and so on. (The next toysh goal is to get the test suite running under mkroot, but 1107other stuff keeps coming up.) 1108James Farrell upgraded diff, adding three line format options and support 1109for diffing FIFOs. 1110Rob did several rounds of dd cleanup, then got distracted by diff cleanup 1111which turned into a complete rewrite of diff. 1112Neither was ready to promote in time for this release though (although the release 1113got delayed almost a week before Rob admitted that).</p> 1114 1115<p><u>Library</u>: 1116The <b>lib/args.c</b> command line parser now has two optional modifiers 1117(" " and ";") for each argument type, providing 4 combinations (both unset, 1118both set, one of each set), and all 4 now have defined semantics and 1119tests/skeleton.test exercises them. (This was motivated by <b>unshare</b> 1120needing a new combination to match the debian behavior.) 1121<b>lib/args.c</b> also added "~" to collate bare longopts (so multiple bare 1122longopts can be synonyms setting the same flag or argument variable).</p> 1123 1124<p>New function <b>xrunread()</b> which launches a child function, writes a 1125string to its stdin, and returns everything the child wrote to stdout as another 1126string (or NULL if it couldn't run or exited with an error), leaving 1127stderr alone (so it goes to the same place the parent's goes).</p> 1128 1129<p>New functions <b>same_file()</b> and <b>same_def_ino()</b> to simplify 1130file metadata comparisons. (If you're wondering why something so simple should 1131have a function encapsulating the logic, this release also has at least 3 1132different bugfix commits for thinkos from switching all the commands over 1133to use them instead of doing the test ourselves. All missing/extra ! or 1134&& vs || level stuff.)</p> 1135 1136<p>The <b>exit_signal()</b> handler now blocks signals so <b>sigatexit()</b> won't 1137re-enter the list when it receives two different killer signals. (Since 1138<b>xexit()</b> capture uses siglongjmp() this should clean itself up 1139when not actually exiting.)</p> 1140 1141<p><u>Plumbing</u>: 1142Several warning suppressions due to things like llvm's fortify finding 1143false positives, gcc's "int and long are identical on 32 bits!" warnings 1144(yes... they are), the MacOS build being crotchety about vfork()... All verified 1145non-issues at present.</p> 1146 1147<p>The build plumbing now puts more effort into resolving dependencies and figuring 1148out when stuff needs to be rebuilt. This includes noticing when the 1149toolchain we're building with changes flags, and outputting a warning 1150when it detects the need to run "make oldconfig" again. (At the moment 1151calling kconfig/ from scripts/ would be a layering violation: I need to 1152rewrite kconfig from scratch before it can do that automatically.).</p> 1153 1154<p>Removed most compile time probes from <b>scripts/genconfig.sh</b>, 1155replaced with things like #ifdef (_NR_syscall) and 1156#if __has_include(<utmpx.h>), or acknowledging when a probe is a bug 1157workaround for a specific libc version and testing for that instead, 1158plus simply deleting some config options due to the 7 year horizon expiring 1159so we can trust it to be there now.</p> 1160 1161<p>Usual batch of musl workarounds for behavior that 1162<a href=https://github.com/landley/toybox/commit/9a5d7a063c8c>doesn't match</a> 1163glibc and bionic or where they've been 1164<a href=https://www.openwall.com/lists/musl/2022/07/27/5>arguing about</a> 1165stuff for years without actually fixing it. 1166Ed Maste of FreeBSD added some FreeBSD checks/cases to 1167lib/portability.[hc], mostly in places we already supported Apple or 1168OpenBSD. 1169The top level "configure" is now re-entrant (shouldn't append stuff 1170to enviornment varibles like $CFLAGS twice when read twice). 1171The MacOS defconfig now includes a lot more commands.</p> 1172 1173<p><u>Test suite</u>: 1174The diff for failing tests is now output after the FAIL line instead 1175of before, "make tests" now runs tar.test by default, 1176a new skeleton.test uses toys/example/skeleton.c to exercise the 1177lib/args.c command line parsing plumbing, 1178scripts/runtest.sh now uses do_pass and do_fail consistently, 1179and Rob keeps accidentally checking in bashisms and Elliott keeps submitting 1180patches to remove them because android runs the tests under mksh and 1181toysh isn't ready yet.</p> 1182 1183<p>Work in progress to make TEST_HOST=1 pass on newer debian, which is a moving 1184target. (The tests didn't change, but the behavior of the gnu utilities 1185is not consistent. Or standardized. Or coherent. Or predictable. Or sane.) 1186Some minor changes to commands like <b>xxd</b> and <b>cmp</b> to 1187make their output look more like what debian's currently producing.</p> 1188 1189<p>Making the tests pass on MacOS is harder: that calls group 0 "wheel" 1190instead of "root" (so things like "tar.test" don't get the same result), 1191stuff like "du" or "ls" often cares what filesystem you're mounted on 1192(and MacOS hasn't even got the same ones _available_), 1193and we even found a mac kernel bug where extending a sparse 1194file sparsely allocates the previous length as actual zeroed disk space 1195(ala "truncate -s 3m file; truncate -s 5m file; du file" says it's using 11963 megabytes of disk space). There's probably more but 1197that's where we stopped looking for now: since Rob doesn't have a mac 1198to test on and the github tests don't reliably run when new commits are 1199pushed (skipped when the servers are busy), fixing this hasn't 1200been a priority. Lots of little red X in the github history because of this, 1201though...</p> 1202 1203<p><u>Mkroot</u>: 1204The output file names are now consistent across architectures: each 1205root/$TARGET directory now contains an <b>fs</b> subdirectory with the 1206expanded root filesystem, packaged into <b>initramfs.cpio.gz</b>, 1207a <b>linux-kernel</b> built from <b>linux-fullconfig</b> which is built 1208from <b>linux-miniconfig</b>, and a <b>run-qemu.sh</b> script to launch 1209it under qemu. No more architecture name in the filenames (that's what 1210the directory name is for), and no more "bzImage" or "vmlinux" depending 1211on what filename the kernel build produced.</p> 1212 1213<p>The fs/init script has better chroot support (not warning about /dev/shm and 1214/dev/pts already existing). 1215The airlock build no longer includes ftpd (you can use 1216httpd and wget, scp out of dropbear, mount a network filesystem from the 1217host, or if you really want to still use ftp can match ftpget and ftpput). 1218When mkroot is downloading extra packages (for example in the dropbear build), 1219it falls back to the host wget if it can't fetch with toybox's version. 1220(This is because toybox hasn't got built-in https support yet, it's on the 1221todo list.) 1222Updated the dropbear build to the current version and fixed a compiler 1223path issue. 1224Fixed version skew in the <a href=http://lkml.iu.edu/hypermail/linux/kernel/2110.3/00278.html>workaround</a> 1225so the Linux kernel build doesn't require a third ELF package (but only 1226for x86, not for any other architecture).</p> 1227 1228<a name="23-07-2022" /><a href="#23-07-2022"><hr><h2><b>July 23, 2022</b></h2></a> 1229<p>Sorry the mailing list has been down since the 19th, Dreamhost 1230<a href=https://landley.net/notes-2022.html#22-07-2022>happened 1231again</a>. (First the web archive was empty, then it was down, then it was 1232restored from a backup taken before the "upgrade" but was not updating with 1233new messages, which they denied was an actual problem but now timeout bounce 1234messages from all the attempts to send to it since it broke have started 1235coming back saying a path doesn't exist on their server...)</p> 1236 1237<p>Dreamhost tech support has been informed of the issue a half-dozen times, 1238and did at least replace the "Currently there are no messages" page with a 1239backup from before they broke it on the first day. Unfortunately they have yet 1240to acknowledge anything else is wrong with it since then, and don't seem to 1241have support staff working over the weekend. This used to be an 1242<a href=https://landley.net/dreamhost.txt>annual</a> 1243<a href=https://landley.net/dreamhost2.txt>event</a> for them, 1244but they hadn't done a truly major breakage in 1245<a href=#12-21-2015>a while</a>. I _think_ the 11 month gap that last link 1246mentions is still missing, but the server's down again (or at least timing 1247out) so I can't check.</p> 1248</span> 1249 1250<a name="06-05-2022" /><a href="#06-05-2022"><hr><h2><b>May 6, 2022</b></h2></a> 1251<blockquote><p> 1252The mighty ships tore across the empty wastes of space and 1253finally dived screaming on to the first planet they came across - which 1254happened to be the Earth - where due to a terrible miscalculation of scale 1255the entire battle fleet was accidentally swallowed by a small dog.</p> 1256<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 1257 1258<p><a href=downloads/toybox-0.8.7.tar.gz>Toybox 0.8.7</a> 1259(<a href=https://github.com/landley/toybox/releases/tag/0.8.7>git commit</a>) 1260is out, with prebuilt <a href=downloads/binaries/0.8.7>static binaries</a> and 1261<a href=downloads/binaries/mkroot/0.8.7>mkroot images</a> 1262bootable under QEMU (using vanilla linux-5.17).</p> 1263 1264<p>Rob also <a href=downloads/binaries/toolchains/latest>uploaded</a> 1265the prebuilt binary cross and native compilers used to build those 1266mkroot images (there's a <a href=downloads/binaries/toolchains/latest/README>README</a> 1267in there), and started producing 1268<a href=https://www.youtube.com/channel/UC4VFy3wc1nzq5tUHhiti6fw>tutorial videos</a> explaining the 1269toybox commands, their implementation, toybox's infrastructure...</p> 1270 1271<p>The commands <b>host</b>, <b>wget</b>, <b>openvt</b>, and 1272<b>deallocvt</b> were cleaned up and promoted. 1273Elliott contributed the new commands <b>uclampset</b>, <b>gpiodetect</b>, 1274<b>gpioinfo</b>, <b>gpioiget</b>, <b>gpiofind</b>, and <b>gpioset</b>, 1275and Rob wrote a simple <b>httpd</b> (largely to give wget.test something to 1276talk to).</p> 1277 1278<p>One command was removed: <b>catv</b> didn't really serve a purpose 1279(everybody just uses <b>cat -v</b>).</p> 1280 1281<p><u>Features</u>: 1282<b>top</b> can now move the list with the LEFT/RIGHT cursor keys (changing the 1283sort field is now SHIFT LEFT/RIGHT). Added <b>find -samefile</b>, 1284<b>cmp -n</b> (and wired up skip1 skip2), 1285<b>tar --strip components</b> (and tar --owner and --group can now specify :UID and :GID making tar.test more portable), 1286<b>lsusb</b> and </b>lspci</b> now read /etc/{usb,pci}.ids[.gz] (when 1287available) to provide human readable device descriptions, 1288<b>ifconfig</b> can now rename interfaces. 1289Moritz Weber added <b>wget</b> post support for form data. 1290In toysh, <b>cd</b> got updated ("cd -" jumps to $OLDPWD, 1291and cd can now handle deleted directories).</p> 1292 1293<p><u>Bugfixes</u>: 1294<b>cp --preserve xattr,context</b> work together now, and 1295Dario Freni fixed <b>cp --xattr</b> to avoid copying selinux context. 1296The <b>sort -u</b> option now implies -s, <b>wget -O -</b> now writes to stdout, 1297<b>pwget -B</b> removes more characters other implementations don't emit, 1298</b>time</b>'s realtime output was not properly zero padding the fractional 1299part, and <b>sed</b> now works with multiple w to the same file. 1300Fixed <b>nl -E</b>, <b>realpath -e</b>, <b>find</b> -newerXY and @time.nanosecond parsing, 1301Various <b>sh</b> (it handles ${X: -1:1} properly now, 1302which is not the same as ${X:-1:1} because :- has special meaning). 1303The <b>hello</b> command in the examples directory now wait for a keypress before 1304exiting when run as PID 1 (because calling wait() returned "No child process" 1305immediately.) 1306The multiplexer's <b>toybox command --help</b> once again shows the command's 1307help (not toybox --help). 1308Elliott fixed a strip bug in the MacOS build, taught <b>file</b> not to be confused 1309by ELF BSS sections but to otherwise detect more invalid ELF files, 1310and fixed an <b>xargs</b> bug where stdin was inappropriately O_CLOEXEC. 1311Github user hg42 reported a bug where <b>tar</b> was looping trying to remove ".." 1312at the start of names.</p> 1313 1314<p><u>Library</u>: 1315Taught flags.h to #define its own CLEANUP macros so switching FLAG() 1316contexts requires fewer steps. 1317When args.c is just enforcing max/min argument count but collecting no 1318options, imply "?" so "command -123" isn't an unknown option. 1319Moved lib/help.c into main.c (nothing else in lib/ had to rebuild 1320when the config changes which commands are enabled). 1321The remains of linestack.c were renamed utf8.c, and 1322toys.h now includes sys/syscall.h so commands don't have to.</p> 1323 1324<p>Yank things with only one remaining user: moved get_line() 1325to patch.c, struct str_len to bc.c, and struct ptr_len to ps.c 1326Removed tty_esc() and tty_jump() wrappers. 1327Various cleanups to lib/password.c and lib/tty.c. 1328Removed the PEDANTIC_ARGS config option which just set a macro that was never 1329used, xsignal_all_killers() now takes SIG_DFL as the reset value instead 1330of explicitly checking for NULL, 1331chomp() now removes any number of trailing \r\n, and it's now safe to 1332llist_pop(0) just like dlist_pop(0). 1333LongPing Wei improved support for copy_file_range().</p> 1334 1335<p><u>Build</u>: 1336If you don't set $PREFIX, <b>make install</b> now defaults to $PWD/install 1337(which does not require root access to write to, and you can tar up your 1338pile of symlinks from there). 1339More command sub-options removed (NETCAT_LISTEN and CAT_V).</p> 1340 1341<p>Some of the uglier scripts/make.sh plumbing moved into scripts/portability.sh 1342(which should now be safe to source twice), 1343and moved almost all config varible definitions to "configure" now. The 1344new $GENDIR variable theoretically lets the "generated" directory live 1345somewhere else (not very tested yet). 1346Various tweaks to scripts/make.sh to try to avoid unnecessary rebuilds 1347and library probes when toolchain and config haven't changed. 1348Added -Wno-pointer-sign to $CFLAGS because passing an unsigned char * to a 1349function that expects char * is not a problem. 1350 1351<p>The airlock install in scripts/install.sh had the $PENDING items sorted 1352into "started" and "not started". Add git and bash to $PENDING (used 1353out of $PATH by --version and the powerpc64le build, respectively).</p> 1354 1355<p><u>Cleanup</u>: 1356Removed the lib/linestack.c plumbing (which was never finished and hadn't 1357been touched in years). 1358Merged lspci.c into lsusb.c (so they can share config file reading 1359infrastructure), and merged chvt/deallocvt into openvt.c. 1360Minor cleanups to ping, fsync, ionice, pmap, truncate, timeout, tty, 1361factor, mount. 1362Went through and replaced \033 with \e in strings (since clang supports 1363it and \033 is just awkward). Added LL to constant 0 in the FLAG macros 1364to prevent gcc from warning that 0<<32 might produce 0. 1365Moved llvm's -Wno-string-plus-int into configure instead of probing 1366for it, since gcc no longer dies when asked to suppress an unknown warning.</p> 1367 1368<p><u>mkroot</u>: 1369the ANSI escape to undo QEMU's wordwrap bug is now in both the init script 1370and launch wrapper script (so wordwrap isn't broken while running the 1371emulated system, nor if the emulated system fails to run). 1372If devtmpfs was already mounted on /dev, we still need to create dev/pts. 1373The init script now includes the linux kernel bug workaround to enable ping 1374support. The armv7 target enabled CONFIG_ARM_LPAE in the kernel to work around 1375around <a href=https://bugs.launchpad.net/qemu/+bug/1790975>a QEMU bug</a> 1376that's gone unfixed for 4 years. Enable network support in m68k target. 1377The mcm-buildall.sh script got some minor updates, and the resulting 1378cross and native compiler binaries are now downloadable so you don't have 1379to run the build if you don't want to.</p> 1380 1381<p><u>Docs</u>: 1382"toybox --help" no longer says "see toybox --help". 1383There's an ANSI escape cheat sheet at the start of lib/tty.c. 1384Reworded some command directory READMEs, and tweaked menuconfig display. 1385Converted www/doc/mount.html and www/doc/ext2.html from text to html. 1386Freshened up design.html a little (and fixed some typos pointed out by 1387Peter Ennis). 1388Replaced the git internals video URL in roadmap.html with a different 1389performance of the same talk that hasn't been made private. 1390Added the kconfig-language.txt from Linux-2.6.12 to the kconfig/ directory.</p> 1391 1392<a name="30-11-2021" /><a href="#30-11-2021"><hr><h2><b>November 30, 2021</b></h2></a> 1393<blockquote><p> 1394Busy? We'll I've just got this bulldozer to lie in front of or otherwise it'll 1395knock my house down but otherwise no, not especially, why?</p> 1396<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 1397 1398<p><a href=downloads/toybox-0.8.6.tar.gz>Toybox 0.8.6</a> 1399(<a href=https://github.com/landley/toybox/releases/tag/0.8.6>git commit</a>) 1400is out, with prebuilt <a href=downloads/binaries/0.8.6>static binaries</a> and 1401<a href=downloads/binaries/mkroot/0.8.6>mkroot images</a> 1402bootable under QEMU (using vanilla linux-5.15).</p> 1403 1404<p><u>Make root</u>: The <a href=faq.html#mkroot>system builder</a> got upgraded, with better 1405logging and layout of the output directory, plus <a href=https://github.com/landley/toybox/blob/0.8.6/scripts/mkroot.sh>scripts/mkroot.sh</a> 1406should be easier to read and understand now. 1407(Also added basic support for the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-August/012497.html>qualcomm hexagon</a>, 1408although it just builds a chroot filesystem, you can't boot a hexagon kernel under qemu 1409<a href=https://lists.nongnu.org/archive/html/qemu-devel/2021-07/msg06430.html>quite yet</a>.)</p> 1410</p> 1411 1412<p><u>New commands</u>: Rob added <b>linux32</b>. 1413Dan Brown added <b>sha256sum</b> (and the related 224/384/512 variants). 1414In pending, Elliott contributed <b>strace</b> and 1415Moritz Röhrich contributed <b>hexdump</b>.</p> 1416 1417<p><u>Features</u>: 1418Elliott added <b>date -s</b>, <b>pmap -p</b>, <b>tail -F -s</b>, <b>kill -0</b>, 1419<b>reboot/halt/poweroff -d</b>, taught <b>file</b> to 1420recognize Mach-o universal binaries, made Android's <b>log</b> 1421take input from stdin when it has no arguments, added <b>tail --bytes --lines</b> 1422(the --long synonyms for -c and -n), 1423added <b>i2cdetect -q</b>, taught <b>i2cdump</b> to write XX for 1424read failures, made <b>timeout</b> use the monotonic clock, 1425and modernized the output of <b>uptime</b>.</p> 1426 1427<p>Rob added <b>find -quit -lname -ilname</b> and <b>-d</b> as a synonym for -depth, 1428taught <b>cut -d $'\n'</b> 1429to cut by line and added posix <b>cut -nb</b> (which wraps to the start of -c 1430utf8 characters), added <b>cpio --ignore-devno --renumber-inodes</b>, 1431taught <b>cal</b> to use 3 arguments (day month year), added <b>tar --selinux</b> 1432support and a "had errors" message at the end, made <b>uname -o</b> 1433say "Toybox", and fixed the <b>grep -Fw</b> combination (used by the 64 bit 1434powerpc kernel build).</p> 1435 1436<p>Ella-0 added <b>split -n</b>. Peter Collingbourne added <b>grep -L</b>.</p> 1437 1438<p><u>Outside world</u>: The 0BSD license is <a href=https://lwn.net/Articles/868882/>now part</a> of the 1439<a href=https://standards.iso.org/ittf/PubliclyAvailableStandards/>ISO-5962</a> 1440standard. Of the currently <a href=https://github.com/search?q=license%3A0bsd&type=Repositories&ref=advsearch>38k 0BSD licensed projects on github</a> toybox 1441isn't even the most popular, it's fourth.</p> 1442 1443<p>Elliott asked that toybox's <b>cut -DF</b> feature (a cheaper 1444alternative to <b>awk '{print $3}'</b> be supported by other packages so 1445it might someday make it into posix, so Rob 1446<a href=http://lists.busybox.net/pipermail/busybox/2021-June/088886.html>submitted it to busybox</a> 1447where it was <a href=https://git.busybox.net/busybox/commit/?id=0068ce2fa0e3>eventually merged</a>.</p> 1448 1449<p>Added a local <a href=https://landley.net/toybox/git>git mirror</a> for 1450those reluctant to trust Microsoft Github.</p> 1451 1452<p>In the web nav bar, our <a href=https://isfreenodedeadyet.com/>freenode</a> 1453channel moved to libera.chat 1454(like <a href=https://boingboing.net/2021/05/19/freenode-irc-staff-quit-after-new-owner-seizes-control.html>everyone else's</a>; 1455<a href=https://lwn.net/Articles/856543/>there</a> 1456<a href=https://hackaday.com/2021/05/20/freenode-debacle-prompts-staff-exodus-new-network/>was</a> 1457<a href=https://twitter.com/marcan42/status/1396280769778098185>a</a> 1458<a href=https://twitter.com/mjg59/status/1396237336988446722>certain</a> 1459<a href=https://www.devever.net/~hl/freenode_abuse>amount</a> 1460<a href=https://twitter.com/TwitchiH/status/1395756163527028739>of</a> 1461<a href=https://twitter.com/joepie91/status/1395364883303182339>drama</a> 1462<a href=https://twitter.com/RichFelker/status/1404816846734385164>involved</a>).</p> 1463 1464<p><u>Pending</u>: lots more work on <b>toysh</b>: it has basic $((math)) support 1465now, += assignment, the "declare" command and corresponding variable 1466types (declare -i, declare -u, etc), several bugfixes, greatly 1467improved support for background processes on NOMMU systems, and a bunch 1468more sh.test entries.</p> 1469 1470<p>Elliott fixed <b>modprobe</b>'s parsing of short lines. Rob fixed a 1471division by zero error when <b>vi</b> tries to edit a zero length file. 1472Anqi Dong made sure that syslogd messages sent to sockets end with a 1473newline. Eric Molitor significantly upgrade <b>wget</b> with http 1.1, 1474chunked encoding, and (external library) https (TLS 1.2) support, 1475plus several bugfixes. Michael Hope fixed <b>tftpd</b>'s option parsing.</p> 1476 1477<p><u>Documentation</u>: The roadmap.html and status.html pages 1478got noticeable updates. (status.html is now directly generated by 1479scripts/mkstatus.py, the old status.html wrapper that server-side-included 1480status.gen has been removed). 1481The <a href=design.html>design</a> 1482page provides better links to the LP64 documentation, and the 1483<a href=code.html>code</a> page explains a corner case where the filename 1484and first command name expect to match (and how to work around it if they don't). 1485Some website files weren't in the repository: two jpegs linked from html 1486files, and the www/.htaccess file that triggers the server side includes 1487in the navigation menu is now checked in. Johannes Schmitz fixed a 1488typo in about.html.</p> 1489 1490<p>The <b>shred</b> usage line now includes all options. 1491The <b>logwrapper</b> command now warns when it isn't built standalone. 1492The <b>help</b> command no longer shows a header line about toybox unless 1493it's called with no arguments. (It still shows it for command --help.) 1494The --help option no longer needs to be the first option, and is 1495checked before the "Not root" error message for restricted commands.</p> 1496 1497<p><u>Bugfixes</u>: 1498Elliott made <b>ps</b> exit with status 1 if no processes were shown (for LTP), 1499fixed an off-by-one in <b>dmesg</b> and a bounds check in <b>netstat</b>, 1500fixed <b>pidof -s</b>, and taught <b>rmmod</b> to handle multiple arguments. 1501Denys Vlasenko pointed out that <b>free -t</b> is "total" not "terabytes". 1502Samanta Navarro pointed out a leftover 32 bit limitation in argument groups. 1503Mikhail Kashkarov fixed mkdir's error handling for existing directories 1504without -p. 1505Rob fixed the 32 bit <b>readelf</b> build on Android, fixed a race condition 1506in <b>tar</b> leading 1507to the occasional spurious archiver warning, 1508<a href=https://github.com/landley/toybox/commit/127cff38809e>changed cp</a> 1509to work around a posix violation in the kernel, made xparsedate() handle more 1510whitespace and ignore trailing + or - (a NOP timezone), made <b>su</b> log the 1511right username, switched getuid() checks to geteuid() in <b>df</b> and <b>ping</b>, 1512fixed a place in the test suite where do_test() 1513could get confused by symlinks in the $PATH (calling realpath too early 1514and losing the name of the command being run), 1515made <b>env -i</b> use the old $PATH to run its command, 1516and made <b>grep --color</b> highlight all matches.</p> 1517 1518<p>Worked around a bug in glibc+gcc 9.3 where linking against librt.a 1519(for things like timer_create()) pulls in libgcc_eh.a which uses 1520threads, meaning you can't statically link against librt.a from 1521a non-threaded app. (I hadn't noticed because it works fine with the musl 1522and llvm/bionic toolchains, only fails statically linking on e.g. debian.) 1523The <a href=https://github.com/torvalds/linux/commit/d18760560593>kernel fix</a> for the cpio encrypted filesystem issue worked around in 0.8.5 made 1524it in. Similarly, the 1525FS_IOC_{GET,SET}{VERSION,FLAGS} ioctl family are <a href=https://www.spinics.net/lists/linux-api/msg49101.html>historically broken</a> 1526(taking an unsigned int argument while advertising signed long), 1527which <a href=https://github.com/landley/toybox/commit/dcfac5c983ad>we fixed</a> and reported upstream to a long 1528round of bikeshedding (sadly 1529the result seems to have been the man page maintainer 1530<a href=https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git/commit/man2?id=3de87d46840d>giving up</a> 1531on documenting ioctls at all.</p> 1532 1533<p><u>Cleanups</u>: 1534<b>cut</b> switched to FLAG() macros and no longer allows -f with -F, 1535<b>eject</b> uses FLAG() macros and the linux/eject.h header, 1536prettycool@protonmail added tabs to <b>time</b>'s output, and Rob removed 1537the floating point dependency from time. Minor cleanups 1538to lsattr, lspci, nsenter, xxd, file, insmod, modprobe, arp, bootchartd, pmap, 1539comm, lsof, blkid, crond, init, userdel, makedevs, and scripts/make.sh. 1540The "int fd = fd;" 1541initialization hack to shut up gcc's "may be used uninitalized, but isn't" false 1542positive warning has been replaced by a QUIET macro that resolves 1543to nothing on llvm (which doesn't have the gcc bug). 1544Ryan Pritchard reported a bug where <b>alias ls="ls --color"</b> would prevent 1545--help or --version from being the first argument to the command, which was 1546a pain to fix without breaking things like <b>echo -- --version</b> and 1547<b>basename -s --help file--help</b>. 1548(Both toy_singleinit() in main.c _and_ get_optflags() in lib/args.c now 1549check for those.)</p> 1550 1551<p><u>Infrastructure</u>: 1552xabspath() now has a more granular (flag based) control interface.</p> 1553 1554<p><b>scripts/make.sh</b> now uses wait -n (available since bash 4.3 in February 2014) 1555to keep the processors busier during a parallel build. 1556Tests are now running in a subshell, which means functions and local 1557variables defined in one don't pollute others during "make tests". 1558 1559<p>Guo Chuang reported that lsattr won't build on an 8 year old version of 1560<a href=https://github.com/landley/toybox/pull/292#issuecomment-899895909>centos</a>, 1561which is still supported for 2 more years because Red Hat can 1562be paid to have a 10 year support horizon, longer than toybox's 7. Rob 1563fixed it (and merged a similarly ancient compilation failure fix when 1564building pending/ip.c against a 3.10 kernel), 1565but the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-August/012537.html>philosophal argument</a> 1566remains <a href=http://lists.landley.net/pipermail/toybox-landley.net/2021-September/012603.html>unresolved</a>.</p> 1567 1568<p>Standalone builds can now override the .config file location with 1569KCONFIG_CONFIG same as the multiplexer build. scripts/single.sh can 1570now cope with a missing .config file (it uses the defconfig values 1571for the CONFIG_TOYBOX_* global symbols). 1572"make install_airlock" no longer needs host versions of sha256sum and sha512sum.</p> 1573 1574<p>In the test suite, <b>VERBOSE=spam</b> now shows expands successful tests 1575like it would for errors.</p> 1576 1577<p><b>scripts/record-commands</b> got updated and logwrapper got renamed <b>logpath</b>.</p> 1578 1579<a name="19-05-2021" /><a href="#19-05-2021"><hr><h2><b>May 19, 2021</b></h2></a> 1580<blockquote><p>It was of course a descendant of these eccentric poets 1581who invented the spurious tales of impending doom which enabled the people of 1582Golgafrincham to rid themselves of the useless third of their population. 1583The other two-thirds stayed firmly at home and lived full, rich and happy lives 1584until they were all suddenly wiped out by a virulent disease contracted from 1585a dirty telephone.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 1586 1587<p><a href=downloads/toybox-0.8.5.tar.gz>Toybox 0.8.5</a> 1588(<a href=https://github.com/landley/toybox/releases/tag/0.8.5>git commit</a>) 1589is out, with prebuilt <a href=downloads/binaries/0.8.5>static binaries</a> and 1590<a href=downloads/binaries/mkroot/0.8.5>mkroot images</a> 1591bootable under QEMU (using vanilla linux-5.12, except that s390 needed another 1592<a href=http://lkml.iu.edu/hypermail/linux/kernel/2105.2/02904.html>perl 1593removal patch</a>).</p> 1594 1595<p>This development cycle had thirty commits to sh.c fixing bugs and adding 1596features to <b>toysh</b>. Shell functions and local variables are now implemented, 1597along with the "set" builtin and trace support, and job control is 1598about halfway in. The shell as a whole is <a href=https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule>maybe 80% done</a>, but still 1599has some obvious gaps like $((math)) and interactive command line 1600history/editing, plus a bunch of testing.</p> 1601 1602<p><u>New commands</u>: 1603Moritz Röhrich added <b>pwgen</b> and <b>base32</b>, and 1604Elliott Hughes added <b>unicode</b>. The 1605<b>readelf</b> command got promoted out of pending, and 1606<b>sha3sum</b> is now switched on in defconfig.</p> 1607 1608<p>Michael Christensen added <b>chsh</b> to pending, which hasn't been promoted 1609yet because it's part of a group of commands (useradd, userdel, groupadd, 1610groupdel) that should all be promoted together at some point 1611(and depend on lib/passwd.c, which could use some more work).</p> 1612 1613<p><u>Upgrades</u>: 1614<b>cpio</b> now supports extracting multiple concatenated archives. 1615<b>file</b> now recognizes android's new XML binary format. 1616<b>devmem</b> now works on nommu, can handle 0x prefxes on the address, 1617and prints 1618hex or decimal output to match the address type. 1619<b>df</b> -a now shows overmounted filesystems. 1620<b>test</b> -k checks the sticky bit. 1621Added -t to <b>install</b>, <b>cp</b>, and <b>mv</b>, and -u to <b>cp</b> 1622and <b>cpio</b>. 1623<b>find</b> -executable checks that _we_ can execute them (u+x could be owned 1624by a different user or have selinux weirdness).</p> 1625 1626<p>Elliott added several features to <b>date</b> (--iso, %:z output format, and 1627--utc as a synonym for -u), added a <b>sed</b> s///x option, 1628added units to <b>ulimit</b> output, and 1629made several UI changes to <b>hexedit</b> including a color 1630mode (which is now the default, press x to toggle between old and new 1631display), masked out the file type in <b>chmod</b> error messages and 1632taught it to handle complex modes (like u+s+s), 1633hardened <b>file</b> and <b>readelf</b> against invalid input, 1634added a portability.h workaround for macos statvfs for <b>df</b>, 1635enabled line buffering in <b>echo</b> and <b>yes</b>, 1636and significatly sped up <b>tr</b>. Rob similarly optimized <b>seq</b> 1637and <b>count</b>.</p> 1638 1639<p><b>make tests</b> now defaults to VERBOSE=fail (and now has VERBOSE=all to 1640continue after failure, and VERBOSE=quiet to show FAIL: lines without 1641diff -u output). 1642Cleanups to tee, netstat, file, and df. Added "static" annotations in a 1643lot of commands, made more things use FLAG() macros. Ethan Sommer pointed out 1644an unnecessary wrapper function in chgrp. 1645In pending, Elliott did some cleanups and bugfixes to gettty, telnet, telnetd, 1646tftpd, init, modprobe, and dd, and added more features to vi. 1647Rob did some cleanup on modprobe.</p> 1648 1649<p><u>Bugfixes</u>: 1650Yi-Yo Chiang fixed a <b>rm</b> bug where -f could get confused by read 1651only filesystems and produce an error trying to unlink a nonexistent 1652file, pointed out <b>install</b> -D was broken when the target file 1653already exists, pointed out a <b>cpio</b> interaction with certain encrypted 1654filesystems where readlink() wouldn't write as much payload as the header 1655promised (corrupting the archive) and fixed another cpio bug where -t test 1656mode would chown() files, and fixed quoting in mount.test. 1657Elliott fixed a sequencing issue in <b>setsid</b>'s system calls, 1658fixed some <b>sed</b> tests (sed's lie-to-autoconf --version 1659output was preventing toyonly tests from triggering) and 1660fixed <b>tar</b>'s base 256 output for very large files. 1661Nir Lichtman pointed out that <b>rm</b> -p shouldn't try to rmdir / at the end, 1662Luke Frankart pointed out a typo that broke <b>test</b> checking file modes, 1663Firas Khalil Khana fixed a duplicate entry in the roadmap, 1664Ethan Sommer fixed <b>install</b> -Dt, 1665Andy Hu requested that <b>mount</b> handle mixed options like 1666-tnomsdos,smbfs. 1667Denys Nykula reported a <b>chmod</b> regression, 1668asm89 pointed out a <b>netcat</b> bug writing transactions larger than 16694k, 1670Daniel Santana fixed <b>df</b> combining -P and -k, 1671Robin Hsu fixed <b>grep</b> getting confused by libc setting errno when 1672a library function call succeeded. 1673Puck Meerburg taught the <b>gunzip</b> code not to be confused by FEXTRA entries 1674(as described in RFC 1952) and added a test suite entry for that.</p> 1675 1676<p><b>find</b> handles empty arguments properly now (with test), <b>date</b> 1677-I with no arguments no longer segfaults, <b>grep</b> -f /dev/null no 1678longer adds a "" regex matching everything, -z no longer splits patterns 1679on \n, and -o can print zero length matches, 1680<b>cp</b> -t works with one argument and with --parents. 1681<b>readelf</b> displays properly in 80 columns and doesn't reject .bss 1682off end of file, and <b>sed</b> s///2 now works when there isn't more than one 1683match. 1684<b>tar</b> now removes empty directories where it wants to put a non-directory. 1685<b>microcom</b> sets serial device terminal attributes correctly.</p> 1686 1687<p>The prebuilt x86-64 binaries shouldn't segfault on older x86-64 processors 1688anymore (workaround for gcc bug). 1689Cross compiling was sometimes confusing install.sh because it created 1690a hyphenated "toybox-$TARGET" and then tried to make symlinks to just "toybox".</p> 1691 1692<p><u>Build</u>: 1693CONFIG_TOYBOX_I18N went away: toybox builds always support utf8 now. 1694Ella-0 added OpenBSD support (but not a separate defconfig, try 1695"make freebsd_defconfig").</p> 1696 1697<p><u>Library</u>: 1698utf8towc() is now unsigned instead of wchar_t, matching wctoutf8(). 1699xclose() tests if fd is -1 so the caller doesn't have to. 1700txpect (the test suite's built-in "expect", implemented in shell) can 1701now do regex matches, and sh.test uses this.</p> 1702 1703<p>Elliott added the failing regex to the xregcomp() error message, and 1704made multiple portability tweaks (header tweaks to build in more 1705environments, running tests on Raspberry Pi OS and under mksh, 1706fix utf8 locale selection on macos).</p> 1707 1708<p><u>Docs</u>: 1709<b>chmod</b>'s help text explains why o+s doesn't set +t (and has a 1710test for it). The <b>echo</b> help text is more compact. 1711Checked in a better conversion of <a href=https://landley.net/toybox/toycans.png>toycans.png</a>.</p> 1712 1713<a name="24-10-2020" /><a href="#24-10-2020"><hr><h2><b>October 24, 2020</b></h2></a> 1714<blockquote><p> 1715"We are now cruising at a level of two to the power of twenty-five thousand to 1716one against and falling, and we will be restoring normality just as soon as we 1717are sure what is normal anyway."</p> 1718<p>- The Hitchhiker's Guide to the Galaxy</p> 1719</blockquote> 1720 1721<p>After a longer and slightly more 1722<a href=https://github.com/landley/toybox/commit/07a896862ddf>turbulent</a> 1723development cycle than some, 1724<a href=downloads/toybox-0.8.4.tar.gz>Toybox 0.8.4</a> 1725(<a href=https://github.com/landley/toybox/releases/tag/0.8.4>git commit</a>) 1726is out with new commands <b>sha3sum</b> and <b><a href=https://github.com/landley/toybox/commit/6b4c32ae3986>watchdog</a></b>.</p> 1727 1728<p>There are <a href=downloads/binaries/mkroot/0.8.4>prebuilt mkroot binaries</a> 1729now, tiny toybox linux systems for a dozen architectures, all bootable under 1730qemu. (The vmlinux in each tarball is a vanilla linux-5.9 kernel built 1731from the included config file.)</p> 1732 1733<p>The FAQ got <a href=faq.html>noticeably larger</a>, and the README has 1734more links. New command features include the <b>sed -s</b> flag, 1735<b>cpio --no-preserve-owner</b> now affects archive creation, 1736Elliott added <b>tar -I</b> and multi-type 1737<b>find -type a,b,c</b> support, Mark Salyzyn added <b>xargs -P</b> to 1738run parallel jobs, the <b>ps</b> and <b>top</b> commands 1739now autodetect pid length, and <b>top</b> adjusts units for memory display based on 1740system size.</p> 1741 1742<p><u>Toysh and toyroot</u>: 1743lots of new work on <b>toysh</b>: 29 commits to sh.c since last release, adding 1500 1744lines and deleting 700, plus a bunch of sh.tests entries). Since last release 1745we implemented wildcards, case/esac and select, brace expansion sequences 1746(ala {1..10..2} and {a..z}), 1747the remaining variable slice types ${a#y} ${a%y} ${a^y} ${a,y} 1748${a/search/replace}, the "<b>source</b>" shell builtin, 1749the start of job control, and several bugfixes. 1750Plus the standalone "<b>make sh</b>" build understands MAYFORK now. It's 1751still missing features like functions() and $((math)) but it's getting 1752close to usable now.</p> 1753 1754<p><u>Documentation</u>: 1755Some of the README contents moved to the FAQ, and the README's "presentations" 1756section got some new links. Several new <a href=faq.html>FAQ</a> entries explaining things like 1757mkroot ("how do I build a working Linux system 1758with toybox"), cross compiling (how to get/setup the 2 compilers and 17593 libc we regression test against), and "where does toybox fit into the 1760linux/android ecosystem". 1761The "toybox --help" output now lists the project's web page (<a href=https://github.com/landley/toybox/issues/50>by request</a>). 1762Elliott removed 1763getevent (an android board bringup/hardware debugging tool built by running 1764a python script against kernel headers, not really in scope for toybox), 1765fixed xargs help formatting, and taught the toybox multiplexer's 1766command list output (and "kill -l") to measure the current the terminal width 1767when wordwrapping (previously hardwired to 80 columns). 1768If you're curious, I checked in my <a href=www/release.txt>release procedure 1769checklist</a>, and 1770fixed a stale link in the nav bar on the left ("statistics" changed domains). 1771Firas Khalil Khana fixed some issues in the roadmap, and Rob 1772tweaked the roadmap so status.html is slightly more current.</p> 1773 1774<p><u>Tests</u>: 1775The "make tests" target now fails if any of the tests it ran failed. 1776Lots of TEST_HOST tests got fixed (checking for specific error messages exposes 1777TEST_HOST to version skew, and scripts/test.sh will now skip TEST_HOST 1778commands that aren't installed) and added toyonly annotations as necessary 1779(replacing most uses of SKIP_HOST). 1780Eric Molitor added automated github tests on MacOS and Ubuntu using their 1781"workflows" thing. 1782The clang asan plumbing slows some code down more than 10 times, so some testing timeouts were expanded. 1783 1784<p><u>Pending</u>: 1785Eric Molitor did a bunch of work on route: moved it to sbin, added xsend and 1786xrecv, taught display_routes() and setroute to use the rtnetlink API and 1787do hostname lookups, added support for mss, win, and irtt, merged ipv6 and ipv4 1788codepaths with autodetection of address type, implemented RTA_CACHEINFO 1789support, switched exit paths to perror_exit(), and removed unused code. 1790Ethan Sommer fixed warnings in dhcpd, removed a bunch of unnecessary 1791; after GLOBALS() blocks and an unnecessary return in df. 1792Erik Moqvist fixed dns setting in the dhcp client. 1793Rob did some cleanup on bootchartd, traceroute, getty... 1794Elliott fixed getty to reliably update utmp. 1795Chris Sarra added ipv6 support to wget. 1796Ariadne Conill submitted several small fixes from testing Alpine Linux with 1797toybox, and Antoni Villalonga i Noceras fixed typos in error messages.</p> 1798 1799<p><u>Bugfixes</u>: 1800Elliott avoided sign extension in devmem, and Ethan Sommer switched it from 1801getpagesize() to posix sysconf(_SC_PAGESIZE). 1802Elliott fixed chmod -R 1803ignoring dangling symlinks, fixed stty <a href=https://github.com/landley/toybox/issues/251>misparsing c_iflags</a>, fixed the chattr f2fs test with 1804compression enabled, taught blkid not to show empty tags, taught xparsedate() 1805to read date's default output format, fixed a recent echo -e \0 regression, 1806changed cpio -p parsing to match a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-August/011955.html>bug</a> in the gnu version 1807which an existing script <a href=https://github.com/landley/toybox/commit/fa1af3b085cc>depended on</a>, 1808and switched hwclock back to looking 1809only at /dev/rtc0 for <a href=https://github.com/landley/toybox/commit/70e2232ce61c>reasons</a> involving kernel version skew and vendor bug reports.</p> 1810 1811<p>David Legault pointed out that unescape2() (and thus echo -e) wasn't handling 1812\0 right. 1813Khem Raj reported that mips glibc doesn't have SIGSTKFLT, leading to 1814a build break if we assume glibc behaves consistently, which led to a bunch of 1815macos signal portability cleanups from Elliott. 1816William Djupström fixed tar extracting long file paths, adding hardlinks 1817to an archive, and reported that --exclude wasn't working. 1818Peter McConalogue pointed out that cp/mv -i prompt should 1819default N, and mv should only prompt when stdin is a tty. 1820Patrick Oppenlander made rtcwake and hwclock default to UTC if /dev/adjtime 1821isn't available, suggested watchdog catch SIGINT, and caught an 1822uninitialized offset in blkdiscard which gcc apparently didn't. 1823Ryan Pritchard reported that <b>file</b> wasn't getting gif heights right, 1824and Elliott added gif version output so TEST_HOST provided similar output. 1825Martin Stjernholm fixed <b>cp -P</b> to not follow symlinks and updated 1826the help text to say it's not the default. 1827Antoni Villalonga added more --long asiases to <b>chgrp/chown/rmdir</b>. 1828 1829<p>The symlink indirection code subtly broke xexec() so it would still try to call 1830a builtin when given a path to a command (fixed now). 1831The code to trip \n off xgetline() was using the length of the allocation 1832instead of the length of the read. 1833Netcat no longer leaks sockfd into child processes. 1834Patch "fuzz" support was outputting context lines from the hunk, not the 1835file. Chrisrfq fixed i2cdetect parameter reading. 1836Chris Sarra taught dd not to throw an error trying to truncate device files, 1837added a misshing fflush() to lib/password.c, fixed a parse error in logger 1838preventing local facility logging from working right, taught init 1839to reload inittab when it receives SIGHUP, and fixed a memory leak in tar.</p> 1840 1841<p><u>Library</u>: 1842New xvdaemon() function daemonizing on nommu systems, to wean commands 1843off daemon() which requires TOYBOX_FORK. 1844Teach sendfile_len() to use copy_file_range() when available, with compile 1845time probe. Fix xsignal_all_killers() to install the correct handler. 1846Rob switched dirtree_path() to a non-recursive implementation, and 1847taught dirtree that a top level entry named "" is equivalent to 1848"." but should not show up in dirtree_path(), 1849read_password() now reads from tty device rather than stdin, and handles ctrl-c and ctrl-d, and 1850human_readable() now has HR_NODOT so it can print single digit numbers without going "0.0". 1851Petri Gynther increased the vmstat column sizes so it didn't fit 1852in 80 columns anymore. 1853Elliott added UTC offset support to xparsedate().</p> 1854 1855<p><u>Infrastructure</u>: 1856mkroot now only passes --no-preserve-owner to cpio (so initramfs files 1857belong to root rather than whichever user ran the build) when using toybox's 1858version, because the other one is brain-damaged and errors out not 1859understanding it. (It understands it for extract, but not create. Yes really.) 1860New PENDING variable you can set to enable more than just sh and route out 1861of pending. 1862The "make distclean" no longer deletes root_download (where mkroot's "extra plumbing" 1863file saves source code downloads it tarballs, such as dropbear).</p> 1864 1865<p><u>Cleanup</u>: 1866Cleanup openvt (35 lines added, 78 lines removed) 1867Cleanup blkdiscard so lib/args.c parses the -o and -l numbers, which means 186832 bit systems are limited to 2 gigabyte sizes which is a TODO item systemwide. 1869oneit now uses flag macros and mentions -rn in the help. demo-utf8towc 1870now has main.c call setlocale for it. 1871Some cleanup on stty.</p> 1872 1873<a name="11-05-2020" /><a href="#11-05-2020"><hr><h2><b>May 11, 2020</b></h2></a> 1874<blockquote> 1875<p>Ford: Ah. A 1876<a href=https://www.youtube.com/watch?v=WSsR419HBpQ>towel</a>. 1877Keep this and guard it with your life.</p> 1878<p>Arthur: Huh?</p> 1879<p>Ford: Listen, it's a tough universe. There's all sorts of people and things 1880trying to do you, kill you, rip you off, everything. If you're going to 1881survive out there, you've really got to know where your 1882<a href=towel.jpg>towel</a> is.</p> 1883<p>- The Hitchhiker's Guide to the Galaxy</p> 1884</blockquote> 1885 1886<p>Despite everything <a href=downloads/toybox-0.8.3.tar.gz>Toybox 0.8.3</a> 1887(<a href=https://github.com/landley/toybox/releases/tag/0.8.3>git commit</a>) 1888is finally out, with new commands <b>rtcwake</b> from Elliott Hughes and 1889<b>blkdiscard</b> from Patrick Oppenlander. 1890The big news is "<b>make root</b>" now boots to a shell prompt, 1891with toysh making it all the way through toyroot's init script. 1892(Some people were looking forward to <b>patch --fuzz</b> support too.)</p> 1893 1894<p><u>Toyroot</u>: <b>make root</b> now does what it says on the tin, it 1895builds a bootable toybox-based Linux system using two source 1896packages (toybox and linux). The trivial version is "make root && sudo chroot 1897root/host/fs /init". Here's 1898a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-April/011667.html>post with instructions</a> if you want to know how to build the 1899cross compilers for testing the various architectures. The self-contained 1900<a href=https://github.com/landley/toybox/blob/0.8.3/scripts/mkroot.sh>script 1901that builds a simple bootable Linux system</a> is 250 lines long, and 1902should be easy to read if you want to know how it works.</p> 1903 1904<p>It also has basic module support, meaning arguments that aren't X=Y variable 1905assignments are the names of scripts to run to build more 1906packages at the end of the build. I checked in an <a href=https://github.com/landley/toybox/blob/master/scripts/root/dropbear>example package</a>, 1907and there's generic "download and extract source tarballs" 1908<a href=https://github.com/landley/toybox/blob/master/scripts/root/plumbing>plumbing</a> available to such modules. 1909(The Makefile doesn't know how to pass module names through to the script, 1910so instead of "make root" you have to 1911call the script directly, ala "scripts/mkroot.sh CROSS=sh4 LINUX=~/linux dropbear".)</p> 1912 1913<p>The resulting root filesystem now uses /root as the home 1914dir for UID 0, and creates /dev/fd and /dev/shm in devtmpfs. The build works 1915around a kernel 1916build bug where "make distclean" doesn't work in a "cp -sfR" symlink 1917directory. (The bug meant if you pointed LINUX= at unclean source, it was 1918unhappy, so now it distcleans the source directory before copying it. This 1919modifies said source directory, which is not ideal, but as usual the kernel guys 1920<a href=http://lkml.iu.edu/hypermail/linux/kernel/2002.2/00083.html>ignored 1921the bug report</a>, so a workaround was required.) 1922The CROSS=all build announces each target in the title bar, puts 1923its logs into root/log, and has better trap handling to stop with one 1924ctrl-c. 1925Since last release it uses a more concise config format for the various kernel 1926arch targets (which shrank the script by about 50%), and merged 1927the old (now removed) scripts/cross.sh into the main script so 1928"make CROSS=armv7l LINUX=~/linux" is 1929literally just a call to "scripts/mkroot.sh CROSS=armv7l LINUX=~/linux" now.</p> 1930 1931<p>As for <b>scripts/mcm-buildall.sh</b> building cross compilers, 1932of COURSE gcc 8.3 requires a different configuration to build the same 1933toolchain as previous versions, it's gcc. It now builds a proper 1934nommu libc for sh2eb without a broken fork() call in it that can never 1935be used but prevents compile-time probes from detecting nommu, 1936and checks that the cross compiler completed before trying to build 1937the native compiler.</p> 1938 1939<p>This was all tested with a recent 1940<a href=https://github.com/richfelker/musl-cross-make/commit/5086175f2902>version</a> 1941of musl-cross-make with the top level Makefile 1942edited to use BINUTILS_VER = 2.32 because the newer one 1943<a href=https://www.spinics.net/lists/linux-sh/msg56844.html>breaks the kernel 1944build</a> 1945and LINUX_VER = 4.19.90 because the default musl-cross-make config uses an 1946out-of-tree headers package for some reason (those who forget history are 1947<a href=https://lkml.org/lkml/2006/4/28/194>doomed to repeat it</a>) which 1948breaks m68k and s390x. I won't post binaries of the resulting toolchains 1949because they're GPLv3, but <a href=https://musl.cc/>Thalheim might</a>. 1950(I've also <a href=http://lists.landley.net/pipermail/toybox-landley.net/2020-May/011673.html>test built</a> 1951with the Android NDK, but bionic's startup code currently segfaults before 1952calling main() if it can't open /dev/null, and the kernel guys 1953<a href=https://lkml.org/lkml/2017/9/13/651>ignored my patches</a> to 1954make CONFIG_DEVTMPFS_MOUNT work in initramfs.)</p> 1955 1956<p>At the moment toyroot cheats and uses two commands out of pending: toysh 1957is about 80% of the way to being useful but still missing some 1958obvious features like function support, job and terminal control, command 1959history, $((math)), wildcard expansion... plus a lot of bash features like array 1960variables, so it isn't out of pending yet. And route needs to be redone to use the 1961netlink interface that can handle multiple routing tables. For the moment 1962scripts/mkroot.sh adds both of them to defconfig if you haven't already 1963got a .config when you run it. (If you build and can't boot, your .config 1964probably hasn't got CONFIG_SH switched on. Fix it in menuconfig, or 1965rm .config and try gain.)</p> 1966 1967<p><u>New toybox features</u>: 1968Elliott taught <b>patch</b> to understand [FILE [PATCH]] arguments, made 1969<b>cal</b> highlight the current day, added -T to <b>cp</b>/<b>mv</b>, 1970did a lot of work on <b>lsattr</b>/<b>chattr</b> (including adding -p 1971and "chattr ="), added tar --absolute-names, taught <b>id</b> to 1972support numeric lookup and handle unknown groups, made -G show all 1973groups, and removed context= from -Z. 1974Rob added <b>patch -F</b> (fuzz factor) support and <b>help -u</b> (usage only), 1975taught <b>echo -e</b> about bash extensions like \uXXXX unicode escapes, 1976<b>netcat -L</b> no longer automatically includes stderr (new -E 1977option does that), 1978<b>setsid</b> now uses -c (like the man page says) instead of -t, and added 1979-w (wait) and -d (detach from tty). 1980Andrew Ilijic added <b>ls -w</b>, removed trailing whitespace on output, 1981and added tests for -C and -x.</p> 1982 1983<p><u>Library</u>: 1984another fix to 32 bit option parsing of long long optflag values, 1985remove getdirname() and just use the libc function (which modifies 1986its argument, but we don't have to free a malloc), 1987dlist_terminate() can now be called repeatedly on the same list, 1988new relative_path() function finds path from one directory to another, 1989and readfd() works like readfile() but on an already open fd. 1990Use MPATH macros in mkpathat(). Elliott added macOS versions of 1991dev_minor()/dev_major()/makedev() to portability.c, 1992implemented posix_fallocate() for macOS, removed a 1993leftover uClibc workaround in fallocate that was breaking macOS, 1994and moved the table of ELF architectures from file to lib/lib.c (so 1995readelf can share). 1996Park Ju Hyung pointed out the fast path of fdlength() was commented out, 1997and we switched it to the 64 bit API while we were there (and then Elliott 1998added macOS support). 1999Joeky taught file to recognize 7z archives.</p> 2000
2001<p><u>Pending</u>: 2002The shell got a bunch of work: standalone "make sh" now includes the 2003multiplexer for builtin commands like "exit" and "cd". Added MAYFORK annotation 2004for commands that exist in the $PATH but can also be run within the shell 2005process (and sometimes have different behavior within the shell): 2006currently applied to help, echo, false, kill, printf, pwd, test, time, and 2007true.</p> 2008 2009<p>Elliott added new commands <b>getopt</b> and <b>readelf</b>. 2010Jarno Mäkipää taught wget how to follow http 301 and 302 redirects, 2011and did lots of work on vi. (Elliott also did work on vi.) 2012Gavin Howard fixed a comparison bug in bc. 2013Ethan Sommer fixed numerous small issues (including several build 2014warnings and FLAG() macro conversions).</p> 2015 2016<p><u>Bugfixes</u>: 2017<b>tar</b> extract now deletes files and symlinks where it's making a directory, 2018<b>find -L -type -l</b> now finds dangling symlinks, extra #ifdefs in 2019portability.h prevent old gcc versions from barfing on __has_include(), 2020xgetline() now returns NULL at EOF, tee with no arguments was 2021writing to stdout twice, setsid works on nommu (I.E. vfork) now, 2022<b>netcat -L</b> no longer accumulates zombie processes, 2023<b>sntp</b> now uses adjtimex instead of adjtime (to build on bionic). 2024xcreate_stdio() was checking WARN_ONLY in the wrong field (and tar was 2025passing it in the wrong field, so it worked there). 2026Several people wrestled with the <b>xargs</b> "argument too long" problem. 2027Alessio Balsini fixed memory leaks in loopback_setup() and "wayling" 2028added a missing continue to losetup. 2029Elliott fixed xargs -E, various things in modinfo, 2030added an error check to gzip when using zlib (which copies non-gzip data to 2031the output verbatim for some reason), 2032found an case where dirtree could use 2033uninitialized data and silenced "Invalid argument" warnings (triggered by 2034Android's selinux policies making stat() and readlink() fail), 2035fixed locale support in macOS (both in iconv 2036and in main.c), taught stat to show filesystem time on macOS, 2037fixed a 32 bit truncation in sntp, fixed a memory access one byte outside 2038of its array in patch.c, removed the cpio --trailer option, 2039and widened the pid display fields in ps to 20406 digits. Rich Felker fixed find.c assuming 2041time_t is a long (y2038 issue on 32 bit). 2042Greg Kaiser fixed a thinko in get_block_device_size(). 2043Jarno Mäkipää fixed utf8 support in cut -C, and cp --parents. 2044David Legault complained that dir/.* tells rm to delete dir/.. and we'd do it. 2045JakeSFR pointed out a bug in file's identification of broken symlinks. 2046William Haddon fixed cp to treat a directory with a trailing slash 2047teh same as one without. 2048Denys Nykula fixed rm -i not to prompt for an empty "" argument. 2049SebiderSushi reported that chmod g+s wasn't working. 2050The linux kernel doesn't let an O_PATH fd work with fgetxattr(), so 2051Elliott switched <b>ls</b> to use the path-based functions now (which is racy, 2052it means you can stat() one inode and get the xattrs for a different one, 2053but nobody in kernel land seems to use xattrs much so they're not fully 2054supported by the API, and who's crazy enough to use xattrs for security 2055anyway). Derick Pallas pointed out an xclearenv() bug. 2056Atatsulo (or possibly luolongjuna) did a lot of work on the <b>ping</b> 2057command: pointed out min/range/max were out of order, 2058that we shouldn't print a summary unless we received at least one 2059reply packet, and implemented ttl support.</p> 2060 2061<p><u>Build plumbing</u>: 2062The "make root CROSS=all" build announces each target in the title bar, puts 2063its logs into root/log, and has better trap handling to stop with one 2064ctrl-c.</p> 2065 2066<p>Fixed scripts/single.sh to work when PREFIX has no trailing slash, 2067make silentoldconfig no longer feeds "y" to kconfig (which puts menus in a 2068loop) and instead just feeds in newlines to accept whatever the default is. 2069The non-git version number I keep forgetting to update each release moved 2070from main.c to toys.h. 2071Fixed a couple different errors in mkflags.c (one of which caused ls --color 2072to set all the other flags).</p> 2073 2074<p>Elliott added more macos support and tests, and added 2075fallocate, cp, mktemp, and mv to the macOS defconfig.</p> 2076 2077<p><u>Cleanup</u>: 2078Rob did some cleanup on xargs, ls, md5sum, and sort, tidied up main.c a bit, 2079made cp, base64, dmesg, and free use FLAG macros, 2080and switched fallocate to new style global names. 2081Elliott Hughes switched rfkill from the old byte at a time get_line() 2082to libc getline(), made du use FLAG() macros, and mad chattr use 2083standard toybox argument parsing for -v and -p. 2084Several commands (help, cp) had sub-options removed from menuconfig. 2085Merged realpath into readlink.c, and use xabspath() instead of libc realpath().</p> 2086 2087<p><u>Documentation</u>: 2088New roadmap section about <a href=roadmap.html#packages>other packages</a> 2089that toybox can (eventually) replace.</p> 2090 2091<p>Update roadmap to note that posix-2008 moved to a different URL (content 2092at the old URL undergoes random changes), and link to the IETF RFCs. 2093design.html explains more of the history of environment sizes on links 2094and has a #bits anchor tag.</p> 2095 2096<p>Update the <a href=design.html#bits>LP64 section</a> of design.html to 2097fish the documents out of archive.org now that unix.org is gone, 2098and show the actual size table locally.</p> 2099 2100<p>Elliott improved the help of <b>date</b> and <b>chattr</b>, and 2101made the usage: lines in the posix directory more consistent.</p> 2102 2103<p>Shrank the sed help from 150 lines to 90-ish.</p> 2104 2105<p><u>Tests</u>: 2106New "txpect" performs interactive tests, running through a sequence of 2107writes to a command's stdin and reads from stdout and stderr, when failing it 2108reports the first non-matching step. (This for example lets sh.test check 2109the shell line continuation logic prompts with $PS1 and $PS2 appropriately 2110with various unfinished input lines, and that "echo hello; if" doesn't 2111output hello before prompting for the next line of input.) Added 2112VERBOSE=xpect to print out each read and write successfully performed by txpect.</p> 2113 2114<p>runtest.sh only creates an "input" file when the input argument isn't empty, 2115EVAL doesn't supply -- (you have to provide your own if you want that).</p> 2116 2117<p>Rob added basic <b>stat</b>, <b>patch</b>, and <b>tee</b> tests. 2118Jarno added a bunch of <b>vi</b> tests. 2119Elliott fixed tests for <b>ifconfig</b>, <b>lsattr</b>, <b>chattr</b>, 2120and <b>date</b>, added tests for find, id, xargs, and made 2121the id, iconv, env, file, printf and cat tests work on macOS (and skipped 2122the du tests there). Rob cleaned up chmod tests.</p> 2123 2124<p><u>Sheer pedantry</u>: 2125true and false now have usage: lines (which you have to "help false" to see 2126because they (intentionally!) don't support --help. 2127Renamed get_chunk()/dump_chunk() to read_chunk()/write_chunk() in tail.c 2128Elliott fixed some typos.</p> 2129 2130<a name="18-10-2019" /><a href="#18-10-2019"><hr><h2><b>October 18, 2019</b></h2></a> 2131<blockquote> 2132<p>"In those days spirits were brave, the stakes were high, men were real 2133men, women were real women and small furry creatures from Alpha Centauri 2134were real small furry creatures from Alpha Centauri." 2135</p> <p>- The Hitchhiker's Guide to the Galaxy</p> 2136</blockquote> 2137 2138<p><a href=downloads/toybox-0.8.2.tar.gz>Toybox 0.8.2</a> 2139(<a href=https://github.com/landley/toybox/releases/tag/0.8.2>git commit</a>) 2140is finally out (only two months late).</p> 2141 2142<p>The new commands this time are minor variants of existing ones 2143(<b>dnsdomainname</b> is hostname -d, <b>arch</b> is uname -m), but about 21441/3 of toysh got implemented in pending, <a href=https://github.com/landley/mkroot>mkroot</a> got merged as a "make root" target (see Build), 2145and Android's <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-August/010845.html>hermetic build</a> 2146work continued resulting in a lot of fixes.</p> 2147 2148<p>The android-specific <b>getprop</b>, <b>setprop</b>, <b>start</b>, and <b>stop</b> 2149commands were removed from toybox because they grew dependencies on android 2150libraries, and since toybox tries to avoid mandatory external dependencies 2151they moved to another package.</p> 2152 2153<p><u>Build</u>: 2154The new "make root" target builds a simple toybox root filesystem, 2155using scripts/mkroot.sh which is a stripped 2156down version of the external <a href=https://github.com/landley/mkroot>mkroot</a> 2157project merged into toybox. (Alas, at the moment it wants sh and route out 2158of pending, which aren't quite load bearing yet.)</p> 2159 2160<p>The new "scripts/mcm-buildall.sh" convenience script builds cross 2161compilers based on gcc+musl multiple targets (arm, x86, mips, m68k, s390, 2162microblaze...) 2163using <a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a>, 2164and a new scripts/cross.sh to easily cross compile for one or more targets 2165using the resulting compilers. (This works with "make root", "make 2166toybox", and other builds understanding the CROSS_COMPILE environment variable.)</p> 2167 2168<p>Setting the environment variable ASAN=1 enables the 2169llvm address sanitizer. (You may need to 2170export ASAN_SYMBOLIZER_PATH on debian to use it, they add the version number to 2171the filename in /usr/bin which confuses llvm.)</p> 2172 2173<p>Patrick Oppenlander fixed make -j (our make is a wrapper around 2174scripts/make.sh which was already doing a parallel build, but if you called 2175make with -j it wouldn't wait for make.sh to finish and the build broke).</b> 2176 2177<p>Lots of fixes for the MacOS build.</p> 2178 2179<p><u>New options</u>: 2180We upgraded <b>tar</b> to extract some older tarballs, <b>file</b> added -b and -s 2181options and can also recognize older tarballs, 2182some xz archives, .otf fonts, perf/simpleperf data files, 2183android boot images, and dtb files. Upgraded <b>file</b> to show the device type 2184for block and char devices, the target for symlinks, and it says _why_ it can't 2185open a file.</p> 2186 2187<p>Several commands ignore some options rather than erroring on them: 2188<b>patch</b> ignores the -f -g# and --no-backup-if-mismatch options 2189(all of which we were already doing by default anyway), and <b>xargs</b> 2190ignores -P# (which requests parallel execution, maybe add it later but 2191single threading works), and Denys Nykula made <b>wget</b> ignore 2192--no-check-ceritificate and <b>gzip</b> ignore -n (again, we weren't doing 2193either anyway) to satisfy pkgsrc.</p> 2194 2195<p>William Haddon taught <b>diff</b> and <b>patch</b> to support special characters in path 2196names (via quoting) and more timestamp formats. 2197Eric Molitor implemented <b>nl</b> -v (with negative and zero starting values), 2198and fixed a MacOS compatibility issue in the build scripts. 2199Tom Cherry added <b>netcat</b> -U (UNIX domain socket support). 2200Denys Nykula taught <b>wget</b> to call ftpget for ftp:// URLs. 2201Ethan Sommer added ln -t and rmdir --ignore-fail-on-non-empty. 2202Elliott Hughes added <b>grep</b> -R, <b>killall</b> -w, tar --mode, 2203 fixed <b>ls</b> -Z and spacing in -lZ and --full-time, 2204added <b>xargs</b> --max-args and removed xargs -I (which we never actually 2205supported) and tweaked the size calculation to more closely match other 2206versions, 2207added xargs -o and made -p read from /dev/tty, added <b>blkid</b> -s, SEC_TYPE for ext3, 2208msdos, and ntfs, f2fs and ntfs LABEL, fixed vfat/ntfs UUID, and switched 2209the endianness for UUID of msdos/ntfs/vfat, simplified the octal mode 2210display in <b>lsof</b>, added <b>pidof</b> -x (thus making the default behavior _not_ 2211be -x).</p> 2212 2213<p>When readdir() succeeds but stat() fails, <b>ls</b> now prints "??? ? ? filename" 2214entries instead of "cannot access" errors. (Still trying to make ls / on 2215android work more gracefully with the SELinux weirdness they added in 9.)</p> 2216 2217<p>Added %C to <b>stat</b> and made it show device type in its default output, 2218<b>cmp</b> now works 2219with 1 argument (implicit - as second argument reading from stdin). 2220<b>sed</b> added exit codes to the q command (q123), added Q, and fixed a 2221bad error message with "!". 2222Removed TAIL_SEEK config option (it just always does that now), 2223<b>find</b> grew several new options (%Z, -true, -false, and -newerXY), <b>ln</b> added -T. 2224The argument to <b>mktmp</b> --tmpdir is now optional.</p> 2225 2226<p><u>Bugfixes</u>: 2227The unshare probe bit-rotted so it was impossible to build <b>unshare</b> 2228and <b>nsenter</b> on some distros. 2229Commit 771e94e2a08 broke toybox's ability to say "unknown command" when 2230you pointed a symlink at it that it that didn't correspond to a command, 2231pointing a symlink at another symlink 2232to make toybox run a command under a name it doesn't recognize 2233(ln -s toybox sh; ln -s sh ash) had an off by one error, 2234<b>ifconfig</b> no longer error_exits if the "read mac address" ioctl fails (Android N 2235and later block that for non-root users to prevent device tracking). 2236When the libcrypto library was disabled, md5sum was building commands it 2237hasn't got built-in support for yet (like sha3sum) which acted like redundant 2238copies of md5sum, 2239<b>tar</b> wasn't correctly reading or writing sparse files with a hole at 2240the end, and tar tzf blah.tar.xz was calling xz when it meant xzcat, 2241<b>xargs</b> would spuriously fail on large memory systems (due to a missing 2242"unsigned" in a typecast), 2243<b>basename</b> can now remove suffixes starting with a dash, 2244<b>cp</b> -r unlinks() an existing symlink before trying to 2245recreate the symlink (otherwise cp -r into an existing directory will 2246try to follow the existing symlink and create another symlink at its 2247destination, failing if a file already exists there, which 2248was confusing gentoo's package manager), <b>timeout</b> now produces the 2249right exit code when the timed command intercepts the exit signal, 2250and <b>hostname</b> -d no longer segfaults on machines with no domain name.</p> 2251 2252<p>Denys Nykula fixed the <b>ftpget</b> freeze (it was sending REST instead 2253of RETR), made <b>mv</b> work with a trailing slash in the source, 2254fixed a <b>find</b> segfault, 2255and made <b>env</b> always exec rather than recursing to a builtin 2256(on the theory #!/usr/bin/env is expected to search the $PATH).</p> 2257 2258<p>Elliott improved <b>killall</b>'s handling of long filenames, made <b>kill</b> work 2259when killall5 is disabled (missing FORCE_FLAGS), fixed error reporting in 2260<b>hostname</b>, fixed several things in <b>xargs</b>, 2261made <b>printf</b> \c and \0 octal escapes work, fixed a <b>find</b> -name corner 2262case (find src/*.c -name file.c) and find dangling symlink behavior, 2263made some <b>rm</b> error messages consistent, fixed \x behavior in echo and 2264printf, fixed linestack.c's buffer length for utf8 U+XXXX escape printing, 2265fixed a race in <b>losetup</b>, fixed <b>grep</b> -F with multiple matches on the same 2266line and an -I failure in the existing grep tests, made <b>readlink</b> notice 2267when it has more than one argument, and made various 2268fixes to <b>modinfo</b>.</p> 2269 2270<p>Ethan Sommer reported that <b>stat</b> of an suid/sticky file output 5 digits of 2271octal stat date instead of 4. 2272Rob fixed an unaligned access in <b>grep</b>. Alessio Balsini removed an incorrect 2273null termination in <b>losetup</b>. 2274Jarno Mäkipää fixed <b>hexedit</b> 2275scrolling up and down on older terminals, and fixed <b>ls</b> segfaulting with a broken 2276(non-UTF8) locale (didn't expect wcrtomb to return -1). 2277Andrew Ilijic made ls put 2 spaces between columns like other implementations 2278(which helps when terminals get confused and apply UTF8 combining characters 2279to the space after a filename). 2280</p> 2281 2282<p><u>Library</u>: 2283Added dlist_lpop() to use a dlist as a stack (it removes the last entry 2284instead of the first), and dlist_pop() now works on a dlist_terminate()d list. 2285DIRTREE_STATLESS returns entries we couldn't stat() (with a zeroed ->st 2286field, and ->again |= 2), but filling in the file type from readdir(). 2287Coversions between signal names and numbers now include all the Linux signals, 2288not just posix ones. The functions sig_to_num() and num_to_sig() moved into 2289lib/portability.h, because MacOS has different signals. 2290TOYFLAG_MAYFORK allows commands to run in toysh's process without forking, 2291but also makes them accessable from the toybox multiplexer or standalone. 2292A MAYFORK command has to clean up after itself (even in error paths) and 2293can't discard anything we need to keep (such as closing stdout).</p> 2294 2295<p>Elliott did a big xbind/xconnect cleanup (adding xbindany and xconnectany), 2296taught xregcomp that an empty regex matches the whole line 2297(because FreeBSD, and thus MacOS, doesn't already do that), and 2298replaced several uses of get_line() (which does single byte reads) 2299with getline() (which doesn't).</p> 2300 2301<p><u>Plumbing</u>: 2302"make clean" doesn't produce a screenful of unnecessary output now, 2303rm, ln, cp, kill, and netcat use the FLAG() macros now, makedevs uses the new 2304GLOBALS naming format, timeout now uses xwaitpid(), and we fixed an 2305off by one error in xwaitpid().</p> 2306 2307<p>Elliott removed a workaround for old NDK versions from getconf and 2308moved sort off of get_rawline() (which let us remove it). 2309Rob undid some loop unrolling in md5sum/sha1sum (we have libcrypto if you want 2310an assembly optimized version instead of an understandable version).</p> 2311 2312<p>Android ndk-r20 doesn't define the __ANDROID_NDK__ symbol, so we switched 2313to using clang's __has_include() (which is an undefined macro and thus a NOP 2314resolving to false on gcc).</p> 2315 2316<p><u>Pending</u>: 2317Rob did a lot of work on toysh: continuation support works (it knows when to ask 2318for another line of text and when to run what it's got), as do 2319nested if/else/elif statements, and while and for loops (supporting "for i" 2320and "for i in", but not for ((;;)) yet). 2321It parses pipes and redirections but doesn't perform them yet. 2322The next big todo chunk is environment variable resolution. 2323</p> 2324 2325<p>Gavin Howard fixed some bugs in bc, such as not returning error when raising 2326zero to a negative power. 2327Jarno Mäkipää continues to extend vi, adding yank and push, support for tabs, 2328rewriting delete, changing the drawing code, etc. Elliott also added several 2329fixes and features to vi, 2330fixed host to cope with large DNS responses, switched crontab to 2331use getline(), did work on man, fixed fixed several things in dd, 2332implemented diff --strip-trailing-cr and made diff not need /tmp. 2333Mike Bennett added extended attribute support to ip.c's route change.</p> 2334 2335<p><u>Test suite</u>: The killall and pgrep tests were having occasional 2336spurious failures due to a race condition (the test script would fork() 2337to call sleep and for just a moment there were two instances of the test 2338script, reporting an extra process if killall or pgrep ran then). 2339 2340<p>Elliott Hughes did a lot of work on the test suite: add kill -l tests, 2341make losetup.test notice (and skip) if loopback 2342devices area already in use, blkid.test can TEST_HOST e2fsprogs blkid now, 2343modinfo.test notices (and skips) if there's no /proc/modules or modules 2344directory, ifconfig.test skips if "dummy0 up" fails and disabled the 2345pointtopoint tests entirely (the kernel never supported it?), 2346truncate.test is less 2347confused by SELinux, hostname.test can run some tests as non-root, 2348addressed a race condition in killall.test on slow systems, 2349made chgrp.test, chown.test, and tar.test handle an empty /etc/passwd or /etc/group 2350(we were checking whether they were _there_ but some android devics have 2351empty ones), 2352fixed a couple things in env.test, extended killall.test to explicitly cover 2353long and short names (and not assume the shell is /bin/sh), taught split.test not to rely on bash process substitution, 2354added missing TZ=utc to touch.test, fixed cp.test to pass for root 2355or with a restrictive umask, and fixed the FAILCOUNT arithmetic to use more portable 2356$(()). Elliott also made some tests more flexible (find.test can handle any 2357error message about symlink loops, etc), and disabled others (such as du) 2358when SELinux is enabled.</p> 2359 2360<p>Rob taught testcmd not to use the shell builtin for TEST_HOST unless there 2361isn't one in the $PATH, work if $PWD has a space in it, 2362and make kill.test stop testing the shell builtin 2363insead of the command, quieted tests that were producing output 2364other than their PASS/FAIL line, and made NOSPACE tests ignore the 2365presence/absence of leading/trailing whitespace as well as differences 2366in the amount of space.</p> 2367 2368<p><u>Documentation</u>: 2369The <a href=roadmap.html>roadmap</a> got freshened up a bit, with updates 2370for android from Elliott. 2371In <a href=design.html>the design page</a> the Features section starts with 23724 new paragraphs about the scope of toybox, and a new License section talks 2373about 0BSD. 2374The "./toybox --help" output now explains how to do install_flat by hand.</p> 2375 2376<p><u>Sheer pedantry</u>: 2377Elliott fixed a double close in loopfiles_lines() so strace doesn't 2378have an EBADF in it, and added a missing "static" in lib/net.c.</p> 2379 2380<a name="21-05-2019" /><a href="#21-05-2019"><hr><h2><b>May 21, 2019</b></h2></a> 2381<blockquote> 2382<p>"I like the cover," he said. "'Don't Panic.' It's the first helpful or intelligible thing anybody's said to me all day."</p> <p>- The Hitchhiker's Guide to the Galaxy</p> 2383</blockquote> 2384 2385<p><a href=downloads/toybox-0.8.1.tar.gz>Toybox 0.8.1</a> 2386(<a href=https://github.com/landley/toybox/releases/tag/0.8.1>git commit</a>) 2387is out.</p> 2388 2389<p>Android is now using toybox as part of a <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-November/009824.html>"hermetic" build</a> 2390(meaning Android is building under toybox, which is halfway to Android 2391building under Android). See Android's <a href=https://android.googlesource.com/platform/prebuilts/build-tools/+log>build-tools</a> and <a href=https://android.googlesource.com/platform/build/soong/+log>soong</a> repositories to follow along with development, or check the list for the <a href=http://lists.landley.net/pipermail/toybox-landley.net/2019-May/010485.html>current status</a>.</p> 2392 2393<p><u>New commands</u>: Isaac Dunham added <b>mcookie</b>, which is used by 2394X11. Elliott Hughes added <b>devmem</b>, which replaces toolbox's "r".</p> 2395 2396<p><u>Promoted</u>: <b>tar</b>, <b>gunzip</b>, and <b>zcat</b> were cleaned 2397up and moved out of pending. 2398Tar is a near-rewrite with support for --owner --group and --sparse, 2399capable of autodetecting "wget blah.tar.bz2 -O - | tar xv", a --restrict option 2400that only allows files to extract into a single subdirectory, and so on.</p> 2401 2402<p><u>Added to pending</u>: 2403Jarno Mäkipää contributed a new <b>vi</b> implementation. Makepost added 2404<b>man</b>.</p> 2405 2406<p><u>Updates</u>: Elliott added <b>find</b> -wholename/-iwholename, -printf, 2407and -context, <b>grep</b> --exclude-dir, and <b>echo</b> -E. 2408Rob added <b>blkid</b> -U and -L, <b>mount</b> now has UUID= support via blkid -U, 2409you can now specify TZ="blah" in the <b>date</b> -d input string (although 2410why the double quotes are required is an open question), and <b>sed</b> now 2411understands ,+N relative ranges. Makepost added <b>rm</b> -v. 2412The human_readable fields in <b>ps</b>/<b>top</b>/<b>iotop</b> now use all 2413available space to show more precision, <b>iotop</b> displays swap bandwidth 2414properly (it was showing a signed field as unsigned), and 2415<b>touch</b> -t and -d autodetect the time format.</p> 2416 2417<p><u>Cleanups</u>: In pending Rob did some bc cleanup, 2418Colin Davidson did some arp cleanup, 2419Uros Prestor fixed route argument parsing, and 2420in bc Daniel Rosenberg fixed bc_vec_concat()'s allocation and 2421Gavin Howard fixed an overflow bug in bc_num_ulong().</p> 2422 2423<p>Elliott made scan_key() detect cursor keys and such with several more 2424terminal types, made more better about directories and non-tty output, 2425cleaned up inconsistent uses of ARRAY_LEN(), taught file 2426to recognize a c-sky ELF machine, and fixed timezone and daylight 2427savings time issues in date and its tests. 2428Rob simplified netcat and ping using new lib/net.c infrastructure. 2429When run as non-PID 1 (for a chroot) oneit's exit code now indicates 2430which signal it's exiting due to. 2431Hadrian Węgrzynowski #defined constants for mkpathat. 2432Date now uses the current time for unspecified fields (instead of zeroing 2433them), and should set the weekday properly. 2434Sort's -k2.3,4.5 syntax is now documented and behaves more closely to 2435other implementations (silly as that behavior is). 2436Significant speedup to sed s/a/b/g on a megabyte long line (it was 2437O(n^2) and now it's quite snappy).</p> 2438 2439<p>Several cases of warning about stdout failing to write were silenced 2440because things like piping the output to head means stdout is a special 2441case: its output is often discarded and that's fine. And xprintf() 2442and friends only checks stdout for errors so it can exit early ("yes | head" 2443should not spin endlessly) without actually flushing anything. 2444There's an ongoing design debate about line buffering stdin and stdout 2445that'll have to be resolved next dev cycle.</p> 2446 2447<p><u>Bugs</u>: 2448Elliott sent in a lot of fixes for building AOSP with toybox: 2449support for more date formats, sed substitution of empty capture groups 2450and an endless loop with -EPIPE, fixed find -path/-ipath, made loggit() 2451only feed LOG_AUTH to syslog when we have a tty, gave grep a bunch 2452of --longname synonyms for short options, added getfattr --only-values, 2453and timeout --foreground --preserve-status and --signal.</p> 2454 2455<p>Top got a lot of tweaks: it no longer collates thread's CPU usage into the 2456parent thread (in -H mode), hides the cursor in interactive mode, updates the 2457display properly in batch mode, and accepts ENTER 2458as a synonym for SPACE when refreshing the display. The header lines 2459now abbreviate long numbers with units (via human_readable()), 2460more reliably spans the terminal width, and says "Thread" instead 2461of "Task" when it's showing threads. The task/thread count now adds up 2462(understanding more kernel states and treating unknown ones as "sleeping"). 2463Top now treats -o "" as an error (previously it displayed an empty screen).</p> 2464 2465<p>William Djupström pointed out login needs to chown() the tty, and made it 2466clear the environment when -p is absent. 2467Vidar Holen fixed inverted find -exec exit status. 2468Rob made chgrp() stop relying on libc's printf("%s", NULL) printing "(null)" 2469because a <a href=https://github.com/varnishcache/varnish-cache/issues/2879>newly added gcc bug</a> turns that into a build break, and 2470fixed an outright embarassing xstrndup() bug. 2471Liwugang fixed a segfault when stat's format string had a trailing %. 2472A missing else in grep meant -x was sometimes ignored.</p> 2473 2474<p><u>Infrastructure</u>: 2475in the build scripts/portability.sh now tests that the compiler it's trying 2476to use exists up front. We no longer link against librt because on glibc that's 2477pulling in libpthread for some unknown reason.</p> 2478 2479<p>TOYFLAG_ARGFAIL() lets commands return custom exit values when option parsing 2480fails (because AOSP build scripts cared), so timeout --unknown returns 125 2481and so on: chroot, diff, cmp, env, grep, nohup, and sort also use it now with 2482various values.</p> 2483 2484<p>New FLAG() macros: FLAG(x) expands to (toys.optflags&FLAG_x) and returns 2485zero if the command line argument's bit isn't set and nonzero if it is 2486(meaning -x was or wasn't encountered on the command line this time by 2487lib/args.c). Lots of commands got converted to use them.</p> 2488 2489<p>New lib/env.c with reset_env(), xsetenv(), and xunsetenv() which frees 2490the old environment variable when it wasn't inherited from exec(). This 2491lets long-running loops reset environment variables without memory leaks.</p> 2492 2493<p>New xnotify() plumbing in lib/portability.c with Linux and Mac versions 2494(which should also work on BSD).</p> 2495 2496<p>New xparsedate() and xvali_date() functions allow date, tar --mtime, and 2497touch to understand the same date formats, 2498xpopen_both() now knows how to use existing filehandles for the child's 2499stdin/stdout, and no longer redirects stderr (which hides errors), 2500new nanomove(), nanodiff(), union socksaddr, xrecvwait(), 2501and human_readable_long() to show more than 3 digits of output.</p> 2502 2503<p>Taught xgetaddrinfo() to return a wildcard address for a NULL host, 2504xconnbind() sets SO_REUSEADDR by default, 2505fileunderdir() returns the abspath to the file now (since we had to work 2506it out anyway), and 2507xabspath() uses O_PATH now (because Android's SELinux rules got more aggressive).</p> 2508 2509<p>Several new xsendfile() variants, including sendfile_len() to copy 2510a specific amount of data and xsendfile_pad() to append zeroes to the 2511output when there isn't enough input.</p> 2512 2513<p>regexec0() is now using libc's REG_STARTEND supported by every libc out 2514there _except_ musl. (It originated in BSD and was adopted by glibc, bionic, 2515and uClibc 15 years ago, and is also supported in macosx. Musl's maintainer 2516<a href=https://www.openwall.com/lists/musl/2013/01/15/26>turned down</a> 2517the feature request as an invalid use case he didn't think they should 2518be trying to do.)</p> 2519 2520<p><u>Docs</u>: 2521the <a href=license.html>license page</a> now has an SPDX identifier, and links to more 2522history/context. The actual LICENSE file had its title line removed 2523so <a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-463746027>github could recognize it</a> as 0BSD. (We're still waiting for 2524"<a href=https://github.com/github/choosealicense.com/pull/643#issuecomment-464380789>a while</a>" to pass before github actually does, though.)</p> 2525 2526<p>A new www/doc/mount.txt file describes how mount works under the covers.</p> 2527 2528<p>Rob consistently misspelled "canonical" and "millisecond", mountpoint 2529has a synopsis now, 2530Kevin van der Kamp fixed a typo in netstat's help text, and Elliott 2531Hughes fixed typos everywhere and made the tense, capitalization, and 2532punctuation of help text more consistent.</p> 2533 2534<p>New paragraph in the code style part of design.html explaining that 2535"char* a, b;" is not how C works, and since you wind up with "char *a, b;" 2536anyway you might as well say that.</p> 2537 2538<p>Given dreamhost's tendency to repeatedly delete the mailing list 2539archive, the nav bar on the left links to a backup web archive now too.</p> 2540 2541<p>The FAQ now says why we do time based releases.</p> 2542 2543<p><u>Testing</u>: Added VERBOSE=nopass to not show passing tests. 2544The test suite now does an rm -rf on testdir between each command so 2545debris files don't accumulate. 2546New skipnot function skips the next test if a command line fails, and 2547toyonly function only runs command when we're testing the toybox version 2548of command. (This isn't the same as TEST_HOST, the host version could be 2549toybox.) 2550The find -newer test calls "sleep .1" to avoid intermittently failing with an 2551identical timestamp. 2552Elliott added the start of a file.test rogues gallery. 2553No longer call lsattr in "make tests" because the behavior differs by 2554filesystem and it needs fixing somehow.</p> 2555 2556<a name="08-02-2019" /><a href="#08-02-2019"><hr><h2><b>February 8, 2019</b></h2></a> 2557<blockquote> 2558<p>"Most readers get as far as the Future Semiconditionally Modified 2559Subinverted Plagal Past Subjunctive Intentional before giving up; and 2560in fact in later editions of the book all pages beyond this point have 2561been left blank to save on printing costs."</p> 2562<p>- The Hitchhiker's Guide to the Galaxy</p> 2563</blockquote> 2564 2565<p><a href=downloads/toybox-0.8.0.tar.gz>Toybox 0.8.0</a> 2566(<a href=https://github.com/landley/toybox/releases/tag/0.8.0>git commit</a>) 2567is out.</p> 2568 2569<p>Toybox now builds on MacOS and FreeBSD, thanks to the efforts of Elliott 2570Hughes and Ed Maste respectively. Use the "make macos_defconfig" and "make 2571freebsd_defconfig" targets to enable the set of commands that compile on 2572each so far.</p> 2573 2574<p><u>New commands</u>: Added an <b>sntp</b> client/server (RFC 4330 Simple Network Time 2575Protocol, a compatible subset of ntp). The <b>test</b> command was rewritten 2576and promoted out of pending.</p> 2577 2578<p><u>New options</u>: <b>grep</b> now has --color, supports embedded NUL bytes in its input, 2579recognizes binary files, and passes all of grep.test (in case you 2580needed the middle numerical field of -bB, etc). 2581Josh Gao added ipv6 and UDP support to <b>netcat</b>. 2582Volodymyr Medvid reported that install -d doesn't honor -o or -g. 2583Elliott Hughes did a lot of work to support hermetic Android/AOSP builds: 2584<b>mkdir</b> accepts both --parent and --parents as synonyms for -p, <b>touch</b> ignores -f, 2585<b>basename</b> added -s to remove a trailing suffix, <b>dirname</b> now supports multiple 2586arguments, <b>cmp</b> accepts --quiet and --silent as synonyms for -s, <b>hostname</b> 2587added -sfd, <b>head</b> added --bytes as a synonym for -c and --lines as a synonym 2588for -n, <b>mktemp</b> added -t and fixed -u, <b>sed</b> added -z and -iEXT to keep backup files, 2589<b>md5sum</b> and sha1sum added --status and --check as synonyms -s and -c, 2590<b>readlink</b> added --canonicalize as a synonym for -f, <b>sort</b> grew -V, 2591<b>patch</b> added -s its synonym --quiet, <b>stat</b> added --format as 2592a synonym for -c, <b>xargs</b> added -p -t -r, 2593Eduardas Meile asked 2594that <b>umount</b> ignore -c. Reverend Homer added a small optimization to file.c, 2595and Elliott taught <b>file</b> to recognize riscv ELF binaries. Peter Collingbourne 2596taught <b>ls</b> -t to use the nanoseconds field. 2597<b>patch</b> has better support for patching a file with a tab in the name.</p> 2598 2599<p><u>Bugfixes</u>: 2600<b>cp</b> --preserve was segfaulting when you didn't specify _what_ to 2601preserve (it now correctly defaults to "mot") and didn't get the permissions 2602right when copying a symlink's contents as a regular file, 2603<b>sort</b> -x didn't work when 2604attached to a key, <b>host</b> didn't allocate a big enough buffer for worst 2605case ipv6 address size, <b>sed</b> needed a ; between b and } when other 2606implementations don't (we're already well past what posix says but a script 2607out in the field broke...), 2608and several fixes to <b>hostname</b> in a container, <b>ps</b>/<b>top</b> were 2609misreading the VIRT and SHM fields.</p> 2610 2611<p><u>Pending</u>: Gavin Howard updated <b>bc</b> and added more tests, 2612Marius Adaskevicius pointed out <b>mdev</b> -s should follow symlinks, 2613Yangchun Fu reported a <b>dhcp</b> checksum bug, <b>modprobe</b> needed errno reset 2614to avoid reporting spurious errors in verbose mode and no longer 2615exits with status 1 if it can't find /etc/modprobe.conf, the <b>more</b> command 2616wasn't always flushing stdout when it exited.</p> 2617 2618<p><u>Build</u>: 2619<b>chrt</b> no longer #warns about the musl sched_get_priority_min() bug, but 2620instead works around it. 2621We were mixing setjmp/siglongjmp (harmless in some contexts but it kind 2622of annoyed FreeBSD), and 2623make install_airlock now adds sha256sum (because the Linux 4.20 build now 2624needs that for the s390x target).</p> 2625 2626<p><u>Coding style</u>: 2627Rob converted the rest of the option GLOBALS() to the new single letter 2628coding style, and the new FLAG(x) macro is a slightly tidier way to say 2629"toys.optflags&FLAG_x". 2630Removed CFG_SORT_BIG (the sort command always 2631has the full functionality now. The general future direction or toybox 2632is to either have a command or not have it; multiple versions of the 2633same command aren't worth the complexity in testing, documentation, 2634or system adminstration).</p> 2635 2636<p><u>Library code</u>: 2637The mkflags plumbing can now support arbitrary punctuation as option 2638names via an escape syntax (because mkfs.vfat specifies an offset with -@), 2639and lib/args.c now only sets FLAGS_NODASH when the first argument 2640didn't have a dash (allowing "ps ax" and "ps -ax" to behave differently).</p> 2641 2642<p>Added an xrename() function, 2643xchdir() has better error reporting, xconnect()/xbind() had their 2644implementations merged, xsendto() moved from ping to lib, xpoll() now 2645measures elapsed time and only waits for the remaining period when restarted, 2646and Eduardas Meile converted several error_exit() to error_exit_raw() (because 2647yocto 2.6 buils with -Werror=format-security by default).</p> 2648 2649<p>There's a scripts/portability.sh now which lets the build use gsed and gmake 2650(when available) in more places to avoid broken host versions on MacOS and FreeBSD. 2651The syntax of the "noreturn" attribute 2652changed slightly, some calls to strnstr() were replaced with strcasestr(), 2653we work around MacOS' lack of features.h, portability.h can now do Apple's 2654endianness macros, and so on.</p> 2655 2656<p>Following the <a href=faq.html#support_horizon>seven year rule</a> 2657Elliott removed support or glibc 2.10, and also 2658reformatted a lot of --help text for consistency (removing trailing 2659periods and such).</p> 2660 2661<p><u>Test suite</u>: 2662The test suite now has its own version of "pending", specifically "make tests" 2663skips test files without the executable bit set (unless you export $TEST_ALL). 2664This eliminates "expected failures" (I.E. files with unresolved todo items) 2665from the global regression test. Additional tests can be added to the global 2666regression test with chmod +x as the todo items they document get resolved.</p> 2667 2668<p>Several places the test suite got fluffed out, including guards to skip 2669root-only tests when run as a normal user. The testcmd function prints the 2670short name instead of the full path to the command. 2671The tests for the test command 2672("test.test") no longer test the shell builtin but correctly test the 2673toybox command. (Now say that ten times fast.)</p> 2674 2675<p>Bash version skew required replacing "continue" with "return" to stop 2676tests early in bash 4.4 (for VERBOSE=fail).</p> 2677 2678 2679<a name="31-10-2018" /><a href="#31-10-2018"><hr><h2><b>October 31, 2018</b></h2></a> 2680<blockquote><p> 2681"In the beginning the Universe was created. 2682This has made a lot of people very angry and been widely regarded as a bad move."</p><p>- The Hitchhiker's Guide to the Galaxy 2683</p> 2684</blockquote> 2685 2686<p><a href=downloads/toybox-0.7.8.tar.gz>Toybox 0.7.8</a> 2687(<a href=https://github.com/landley/toybox/releases/tag/0.7.8>git commit</a>) 2688is out (a month late).</p> 2689 2690<p>Toybox <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-October/009769.html>now builds</a> with the Android NDK (r18 release), almost out of the box. 2691You have to add an "llvm-cc" symlink to "clang", then use CROSS_COMPILE=llvm- 2692with the appropriate $PATH (or absolute path in CROSS_COMPILE) for the 2693target you want to build for. (To run it on a non-android host, you probably 2694want "LDFLAGS=--static" too.)</p> 2695 2696<p><u>New commands</u>: <b>i2cdetect</b>, <b>i2cdump</b>, <b>i2cget</b>, and <b>i2cset</b> 2697courtesy of Elliott/Android. The <b>watch</b> command got a complete (tty-aware) 2698rewrite, and was promoted out of pending. The <b>prlimit</b> command is back 2699(a configuration bug was always disabling it), and <b>ascii</b> now defaults y 2700in defconfig (an oversight).</p> 2701 2702<p><u>New options</u>: 2703The <b>ifconfig</b> command added -S (short view) giving one line of 2704information per interface, in "name ipv4/mask macaddr [ipv6/mask@type...]" 2705format. <b>grep</b> grew binary file detection, with -I and -a options to 2706ignore them or force treating them as ascii (really utf8). 2707Zach Van Rijn asked for <b>find -empty</b>. 2708In the <b>file</b> command, Elliott improved java 2709.class support and added .wav, .bmp, and android .dex. 2710Elliott also added "getconf -a" and a lot of missing getconf symbols the 2711AOSP build uses, including support for pathconf(3) symbols requiring two 2712arguments. This means getconf's arguments work more like ubuntu's 2713now (one or two arguments, not iterating through the supplied list and 2714processing each like last release), and -l grew section identifiers (so 2715the "getconf -l" output is no longer a directly consumable list of symbols 2716it takes). 2717Command line options specifying durations (like top -d or ping -i) can 2718now use fractions and units, like "1.5" and ".1m". See "toybox --help" for 2719details. 2720Toybox now dereferences one layer of symlinks if it doesn't recognize the 2721name it's called under (so if you "ln -s $(which sleep) blah" and then 2722"./blah 30", it should figure out you mean sleep 30).</p> 2723 2724<p><u>Pending</u>: 2725Gavin Howard contributed a large new <b>bc</b> implementation to pending, 2726which covers everything the kernel build needs (and more) but is going to 2727take a while to review. 2728In <b>diff</b> the android guys added --color and timestamps on the 2729+++/--- lines. In <b>mdev</b> Faustas Azuolas Bagdonas added support 2730for mdev.conf's fourth field. Reverend Homer suggested an error message 2731improvement in <b>wget</b>, Kevin Spiteri did a cleanup pass on test.c, 2732and there was a cleanup pass on the passwd command.</p> 2733 2734<p><u>Bugfixes</u>: 2735Elliott responded to a bug in <b>getconf</b>'s sed-based header generation by 2736replacing it with a more conventional array of #defined symbols, with #ifdefs 2737for missing symbols on the three libc implementations (glibc, bionic, musl) 2738we've tested so far. (This requires portability.h work to add new supported 2739build environments, but at least it works on bionic now.) 2740The <b>file</b> command now fails gracefully when reading pathologically 2741broken ELF files (the tests for integer overflow missed one), and 2742<b>date</b> no longer shows a meaningless errno when it can't parse the 2743date format. 2744Several fixes in <b>ping</b> (-w and -W didn't work, -c kept sending packets 2745while waiting for replies, and when no reply packets were received the summary 2746data was printed twice). 2747Elliott fixed <b>wc</b> column widths (traditional reality doesn't match 2748posix), fixed <b>modprobe</b>'s error handling (reported by Wen Xie), and 2749also fixed the units on blockdev --blkraget (reported by Martijn Coenen). 2750Several small fixes to the <b>stat</b> command's output (help text, quoting 2751style, leading zeroes, switch -f from %t to %T), <b>oneit</b> no longer 2752reboots the system when it's not called as PID 1. 2753Last release's commit letting $STRIP be redefined had a typo. 2754Nick Kralevich added an istty() check before calling TIOCGWINSZ (which 2755triggered Android's strict ioctl filtering on non-filesystem fifos). 2756An unstripped build (where the "strip" command fails for a given 2757CROSS_COMPILE) will now overwrite the (read-only) output file instead of 2758failing a rebuild. The dirtree plumbing no longer prints a "No" on front 2759of the filename in error messages (such as "permission denied").</p> 2760 2761<p><u>Docmentation</u> 2762Elliott updated the roadmap with Android AOSP status, towards turning that 2763into a hermetic build (hopefully with an eventual self-hosting option, work 2764is <a href=b33d37d6f735>already underway</a> on the airlock step). 2765Rob added a little more context for LSB and described the move from Aboriginal 2766Linux to mkroot. 2767The <b>ps</b> help text changed: now "ps -o help" shows the fields and the normal 2768--help is much more manageable. Ifconfig's --help text also got tidied up. 2769Our LICENSE file has the SPDX identifier (0BSD) now, and the note about 2770the kconfig/ subdirectory still having (build only, non-shipping) GPL code 2771in it moved into the kconfig subdirectory.</p> 2772 2773<p><u>Coding style change:</u> 2774GLOBALS() now uses the argument letter as the variable 2775name for variables automatically set by command line arguments via lib/args.c, 2776and no longer requires they be one per line. 2777Removed use of the gcc 2778\e extension in string constants, replacing it with \033.</p> 2779 2780<p><u>Library code:</u> 2781lib/interestingtimes.c was renamed to lib/tty.c. 2782The deflate code from toys/pending/compress.c moved to lib/deflate.c, 2783which gunzip.c now uses when it's not configured to use zlib. (This reduces 2784the bootstrap dependencies for the "airlock" step of a hermetic build.) 2785The compression side remains a todo item. 2786Added "%" to lib/args.c reading seconds (including fractions/units) into a 2787long milliseconds argument. Redid xparsetime() to not need floating point, 2788added xparsemillitime() for milliseconds common case, and 2789parsetime() now treats leading garbage as an error instead of returning zero. 2790The bufgetpwuid()/pufgetgrgid() functons can now handle more than 4k of group 2791data. The set_terminal() function can now set serial speed, and microcom.c 2792uses it. Added xsignal_flags(), and more consistently use xsignal() (which wraps 2793sigaction()) instead of signal(). 2794A new xgetrandom() function calls the new getrandom() system call, falling 2795back to /dev/urandom on older kernel/libc (made to actually _work_ by 2796Patrick Oppenlander).</p> 2797 2798<p><u>Build infrastructure:</u> 2799Updated dependency detection so "make distclean defconfig toybox" now works 2800all on the same command line. (This is nice when $CROSS_COMPILE changes). 2801scripts/config2help.c no longer includes parts of lib/ and just copies what 2802it needs into itself, improving build portability. 2803The build now checks the specified cross compiler exists, hopefully providing 2804a more informative error message when it doesn't. Similarly, when .config is 2805missing the error message suggests running make defconfig.</p> 2806 2807<a name="23-06-2018" /><a href="#23-06-2018"><hr><h2><b>June 23, 2018</b></h2></a> 2808<blockquote> 2809<p>There is a theory which states that if ever anyone discovers exactly what 2810the Universe is for and why it is here, it will instantly disappear and be 2811replaced by something even more bizarre and inexplicable. There is another 2812theory which states that this has already happened.</p> 2813<p> - The Hitchhiker's Guide to the Galaxy</p> 2814</blockquote> 2815 2816<p><a href=downloads/toybox-0.7.7.tar.gz>Toybox 0.7.7</a> 2817(<a href=https://github.com/landley/toybox/releases/tag/0.7.7>git commit</a>) 2818is out.</p> 2819 2820<p><u>New commands</u>: Rob Landley added <b>getconf</b>, Elliott Hughes added 2821<b>uuidgen</b>, and <b>ping</b> and <b>fmt</b> were promoted out of pending.</p> 2822 2823<p><u>New options</u>: The <b>cp</b> command added the --parents option, 2824<b>readlink</b> now has -m to show where a missing path would be, 2825<b>netstat</b> got some updates (better nommu support, it can use /etc/services 2826names, more command line options are bounds checked), 2827Ng Zhi An added --getra and --setra (get/set readahead) to <b>blockdev</b>, 2828Elliott Hughes added <b>xxd</b> -o and made <b>top</b> -d understand fractional 2829sections (because the Linux Test Project uses that), and 2830Minghui Liu added <b>watch</b> -b.</p> 2831 2832<p><u>Bugfixes</u>: Several fixes to <b>vmstat</b>: Elliott Hughes made it work on a 283348-core machine 2834and fixed a glitch with -n, Haroon Maqsood pointed out "vmstat 1 1" didn't 2835stop and "vmstat -n" didn't print the first theader line. 2836Minghui Liu fixed a bug where <b>cp</b> -p didn't preserve 2837timestamps, and pointed out the b, c, and d suffixes on numbers weren't 2838working. Ryan Pritchard pointed out that <b>du</b> -d0 should act like du -s. 2839Filip Perich made an RFC2347 OACK compliance fix to <b>tftp</b> in pending. 2840Zach van Rijn fixed a bug in <b>xxd</b> causing incorrect translation of 2841uppercase characters. 2842Elliott fixed several things in <b>top</b> (removed spurious '\r' characters from -b 2843output, removed interactive flicker, made running processes bold), and 2844pushed Rob to make <b>file</b> work better recognizing things on stdin 2845("cat /bin/ls | file -" still won't work but "file - < /bin/ls" should). 2846Rob fixed a bug in <b>netstat</b> on 64 bit big endian systems, 2847and fixed <b>cut</b> -DF 2848(a posix compliance fix broke its ability to act as a decent awk replacment 2849because outputting all delimiterless lines isn't what you want there, -D 2850now disables that too).</p> 2851</u></p> 2852 2853<p><u>Documentation</u>: Rob added a buildroot section to the roadmap with the 2854prequisites that needs to run, and reformatted more of the ps --help text 2855to two columns.</p> 2856 2857<p><u>Library</u>: 2858FLAGS_NODASH is now set in toys.optargs when an optstring starting 2859with & has no dash in its first argument. (This lets "ps -ax" and "ps ax" 2860behave differently.) Factored out xtestfile() into lib/. 2861The comma-separated-list parsing infrastructure moved to lib/commas.c. 2862Added mkpath() for the common case of mkpathat() and #defined MKPATHAT_* 2863constants for the uncommon cases. Elliott added better error checking 2864to xparsetime() and fixed a bug in names_to_pid() (used in <b>pidof</b> 2865and <b>killall</b> and such). 2866Rob inlined the old toys/e2fs.h into pending's mke2fs, 2867and removed a rogue semicolon from the WOULD_EXIT() macro that screwed 2868up if() statement levels. The do_lines() semantics changed adding a 2869callback(0,0) at the end of each file.</p> 2870 2871<p><u>Tests</u>: 2872The test_* commands under toys/example were all renamed demo_* so 2873they don't share a namespace with running a standalone command 2874through the test suite (ala "make test_sed"), then 2875demo_human_readable became demo_number so it can also test atolx() 2876suffixes. The readlink.test was getting confused by a 2877behavior difference between toybox and ubuntu (Ubuntu's ln -sf wouldn't 2878replace a symlink that pointed to itself, toybox's would), now there's 2879an explicit rm so it works on ubuntu too. The test suite plumbing's 2880error message for a test with the wrong number of arguments now 2881correctly identifies which test it's complaining about, and it can now 2882run a test on a command name beginning with "-" (none yet but it 2883comes up in toysh).</p> 2884 2885<p><u>Building</u>: You can now override strip with $STRIP. The set of 2886"make install_airlock" commands got tweaked for the 4.17 kernel (which 2887needs bison and flex because the new kconfig plumbing doesn't have _shipped 2888versions of those generated files like the old kconfig did), and for 2889the new release (remove ping, toybox provides it now). 2890$LDFLAGS now applies to library probing (the android NDK provides 2891dynamic libraries without corresponding --static versions). 2892Elliott took a CFG_TOYBOX_DEBUG check out of an error_exit() to make 2893clang stop complaining about an error that should never happen.</p> 2894 2895<a name="24-02-2018" /><a href="#24-02-2018"><hr><h2><b>February 24, 2018</b></h2></a> 2896<blockquote><p> 2897"Arthur," said Ford.<br /> 2898"Hello? Yes?" said Arthur.<br /> 2899"Just believe everything I tell you, and it will all be very, very simple."<br /> 2900"Ah, well, I'm not sure I believe that."<br /> 2901</p> 2902<p> - The Hitchhiker's Guide to the Galaxy</p> 2903</blockquote> 2904 2905<p><a href=downloads/toybox-0.7.6.tar.gz>Toybox 0.7.6</a> 2906(<a href=https://github.com/landley/toybox/releases/tag/0.7.6>git commit</a>) 2907is out, a month and change late.</p> 2908 2909<p>This release adds a <b>crc32</b> command and 2910cleans up and promotes <b>iconv</b>, <b>logger</b>, and <b>setfattr</b>. 2911Pending added <b>stty</b> and <b>fmt</b>, the other half of <b>ping</b>, 2912and got a lot of cleanup on <b>gzip</b>/<b>gunzip</b>/<b>zcat</b>.</p> 2913 2914<p>The <b>cut</b> command got a complete rewrite adding support for multiple 2915ranges, utf8/unicode characters, and new -CDFO options. Use -C to measure 2916columns rather than characters (for double width chars), and "cut -DF 3-5" 2917selects words (like "awk '{print $3 $4 $5}'").</p> 2918 2919<p>This release adds <b>top -m</b> and <b>iotop -H</b> (and fixed top width truncating for narrow terminal windows), 2920added <b>iconv -c</b>, fixed <b>ps -T 1234</b> to show threads belonging to 2921that PID, improved <b>file</b>'s executable identification (properly recognizing 2922both endiannesses and adding sh/frv fdpic, bpf, and new microblaze 2923identifiers), and <b>blkid</b> now only shows LABEL= when it isn't blank.</p> 2924 2925<p>Running ./configure now does "make defconfig", help_exit() now shows the 2926"See %s --help" message on the same line, and several web pages 2927(about, code, roadmap, cleanup) got updates.</p> 2928 2929<p>Elliott and the Android devs added <b>df -i</b>, <b>uptime -p</b>, 2930<b>time -v</b>, and <b>xxd -i -ri</b>, 2931fixed <b>id -G</b> to only output the group IDs, fixed the <b>xargs -0 -n</b> 2932combination, made <b>find -exec +</b> obey ARG_MAX (just like xargs), made 2933<b>killall</b> kill scripts too, fixed an off by one error in <b>pidof</b> 2934that could cause false positive matches in 15 character names, added 2935gzip/gunzip/zcat tests, and taught <b>file</b> to identify more stuff 2936(ogg, TrueType font, LLVM bitcode, PEM certs, and PE executables), and 2937for some reason needed TOYBOX_VENDOR in the version stuff. 2938Android <b>getprop</b>/<b>setprop</b> now allows ro. properties to have 2939arbitrary lengths, <b>losetup</b> can now find loop devices under /dev/block (where android puts 2940them for historical reasons), and Andreas Gampe pointed out a thinko in 2941<b>top</b> that triggered llvm's address sanitizer checks.</p> 2942 2943<p>Gael PORTAY asked for mdev to work when Linux has the 2944block layer configured out, Patrick Oppenlander pointed out a problem 2945with make bloatcheck's build dependencies, and 2946xuphung on github fixed config2help building on MacOS X.</p> 2947 2948<p>Commands with locale support now setlocale(LC_TYPE, "C.UTF-8") for more 2949targeted locale support (enable character parsing with 2950utf8 support, use "C" semantics otherwise). The toys/example/test_*.c 2951commands got renamed demo_*.c.</p> 2952 2953<p>In lib.c the millitime() function returns current unix time in milliseconds 2954(which as a 64 bit number is good for + or - 292 million years from 2955midnight Jan 1 1970), and the old xgetty() function became tty_fd() returning 2956-1 instead of erroring out.</p> 2957 2958<p>The "make help" text moved from the Makefile to scripts/help.txt, 2959ls.c now lists some of its deviations from posix, and work is underway 2960to make ps.c more intelligible (and figure out how to break it up into 2961multiple files).</p> 2962 2963<a name="12-10-2017" /><a href="#12-10-2017"><hr><h2><b>October 12, 2017</b></h2></a> 2964<blockquote><p>Would it save you a lot of time if I just gave up and went 2965mad now?</p> 2966<p> - The Hitchhiker's Guide to the Galaxy</p> 2967</blockquote> 2968 2969<p><a href=downloads/toybox-0.7.5.tar.gz>Toybox 0.7.5</a> 2970(<a href=https://github.com/landley/toybox/releases/tag/0.7.5>git commit</a>) 2971is out.</p> 2972 2973<p>This is another <b>mostly bugfix release</b>. I delayed it a bit trying to get some 2974of the many half-finished projects (dd, ping, lsof, iconv, cut, sysconf, 2975the cp --parents option...) finished/promoted/tested, but decided adding big 2976things at the end of the dev cycle would be too destabilizing.</p> 2977 2978<p>A couple <b>new features</b> slipped in anyway. Ilya Kuzmich added head -c 2979and strings -t (and corrected strings' output in a couple places). Elliott 2980Hughes taught file to recognize Macintosh (Mach-O) binaries.</p> 2981 2982<p><b>NOTE</b>: the chrt command is broken when built against musl-libc 2983because that project's maintainer decided he didn't like the system calls 2984it depends on, so he 2985<a href=https://git.musl-libc.org/cgit/musl/commit/?id=1e21e78bf7a5>removed 2986them from his libc</a>. The prebuilt binaries are built against musl, 2987so the chrt command there will always say -ENOSYS. (It works fine built against 2988glibc, and presumably bionic or uClibc.)</p> 2989 2990<p><b>Bugfixes</b>: Remounting (mount -o remount,rw) should work properly now, 2991and we fixed another place ps was segfaulting when /proc 2992entries vanished out from under us (a hard to hit race condition 2993mostly noticeable in long-running "top" exiting; error value was treated 2994as a pointer). Elliott Hughes fixed a -Wformat warning in expr.c, 2995and made xargs split the command 2996line at ARG_MAX (which is 1/4 the stack size ulimit, which normal users 2997can arbitrarily increase but execve() will complain if they don't). 2998Josh Gao made netcat -l exit after handling a request, and reported a 2999seq bug (seq "1000000 10000001" output 1e+06, fixed now). Zach Riggle fixed a 3000broken URL in the README. Rob fixed env -0 checking the flag wrong so
3001it didn't work in combination with other flags (typo).</p> 3002 3003<p><b>Plumbing</b>: 3004Replaced mbrtowc() with a new utf8towc() that doesn't have a context 3005struct or care about locale. A bugfix in comma_scan() (wasn't removing an entry at the end 3006of the list) is why remount _sometimes_ worked (depending on argument 3007order), xgetaddrinfo() is now separate from xconnect(), 3008atolx_range() learned the w (word=2) suffix, and b is now (block=512) instead 3009of (byte=1).</p> 3010 3011<p><b>Build</b>: Patrick Oppenlander added a workaround for a bug in config2help.c 3012that resulted in segfaults on newer toolchains. (It was actually a use 3013after free error; the build infrastructure isn't nearly as heavily audited as 3014code that gets installed on the target; oops.) A new GITHASH 3015environment variable can force the build version from 3016the command line, and the build checks for .git in the top directory 3017before asking git what our version is (so it isn't confused by ../../.git 3018in an enclosing directory). Added a build #warning about musl intentionally 3019breaking chrt (as with nommu fork() musl now provides a broken 3020stub function so compile-time probes for its existence think it's there, and 3021you can't run the result to test behavior when cross compiling).</p> 3022 3023<a name="19-06-2017" /><a href="#19-06-2017"><hr><h2><b>June 19, 2017</b></h2></a> 3024<blockquote><p>It is a well-known fact that those people who most want to rule people are, ipso facto, those least suited to do it. 3025To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job.</p> 3026<p>- The Hitchhiker's Guide to the Galaxy</p> 3027</blockquote> 3028 3029<p><a href=downloads/toybox-0.7.4.tar.gz>Toybox 0.7.4</a> 3030(<a href=https://github.com/landley/toybox/releases/tag/0.7.4>git commit</a>) 3031is out. No new commands this time, but 3032<b>chrt</b> and <b>dmesg</b> got promoted out of pending.</p> 3033 3034<p><u>New features</u>: 3035Rob rewrote paste, which should work much better now, and added grep 3036-M and -S to match and skip wildcards respectively (useful with -r). 3037Elliott's updated dmesg has -T and --color. The file 3038command can recognize gzip now, uptime grew -s, date grew %N, env knows - as a first argument 3039means -i (posix!) and grew -0, ls defaults to -b 3040instead of -q now when there's a tty, and ls has a new -ll option (with 3041--full-time as a compatibility synonym) showing nanoseconds and (for some 3042reason) timezone. (Why do individual files have timezones?) Elliott added 3043"uudecode -o -" support. Illya Kuzmich taught head -v and -q. The cpio 3044code no longer adds the "TRAILER!!!" entry by default (initramfs extractor 3045doesn't care) without which you can concatenate cpio archives with "cat". 3046(Use the new --trailer option if you want the legacy behavior.) 3047In pending, fdisk compiles now and tar understands bzip2.</p> 3048 3049<p><u>Build</u>: 3050The "make install_airlock" target now symlinks bc from the host because 3051the kernel <a href=https://landley.net/notes-2013.html#28-03-2013>inexplicably</a> 3052needs that to build. This was motivated by 3053<a href=https://github.com/landley/mkroot>mkroot</a>, which builds under 3054a toybox airlock directory.</p> 3055 3056<p>Lots of work on the test suite, mostly from the Android guys who are now 3057running it under Android. This fixed several existing tests that didn't 3058pass, made more tests run on a toybox-only system, and so on. The test suite 3059infrastructure now has a second testing function, 3060"testcmd", which supplies the command name being tested (bypassing 3061shell builtins).</p> 3062 3063<p>Various android build and config fixes, getting closer to being able 3064to let android someday use scripts/make.sh instead of generated/* snapshots. 3065Also more work into building under android's NDK; not quite there yet 3066but much closer. 3067Use nproc in scripts/make.sh detect available processors (so you can control the SMP level with taskset). 3068Removed the old uClibc compatibility glue, it's been 5 years since their 3069<a href=http://lists.busybox.net/pipermail/buildroot/2016-December/180102.html>last release</a>.</p> 3070 3071<p>The new config option TOYBOX_PEDANTIC_ARGS checks arguments when there 3072are no arguments, so things like "uptime" no longer silently ignore arguments 3073you pass but instead refuse to run.</p> 3074 3075<p><u>Docs</u>: 3076The FAQ now has more than one entry. Commands no longer output the full 3077help text for argument errors but instead just say "See %s --help" with the 3078command name (in addition to the actual error message). 3079Elliott did a big period-ectomy on all the --help text, and 3080we cleaned up some tab/space inconsistency. The 3081non-html help -a output now has separators with the command name. 3082The top/iotop and pkill/pgrep help text now describe a lot more of what 3083the commands can do. Twitter's code of conduct page went down so we 3084mirrored the text locally.</p> 3085 3086<p><u>Bugfixes</u>: 3087Fixed a race condition in ps/top where a process that exited right as we 3088read its data returned a different error value than we were expecting (which 3089was causing long-running top instances to occasionally exit), 3090mount now gives an error if it can't autodetect the filesystem 3091type, ps no longer queries the terminal size when output isn't to a tty 3092(so "ps -A | cat" doesn't vary), date's chkmktime() was replaced with 3093simple range checks for fields (to avoid false positives from things like 3094timezones and daylight savings time), removed %s from date's help (we 3095didn't implement it, we have @seconds[.nanoseconds] instead), fixed 3096zcat's buffer flush logic (which was always failing on files larger 3097than 32k), and factor now detects requests for numbers >64 bits and fails 3098loudly instead of producing incorrect answers. 3099Elliott fixed touch -a/-m (they were backwards), and allowed ':' in 3100setprop's property names. Grep now exits with 2 for errors (so -q can 3101distinguish "didn't find" from "didn't work"), doesn't stop on symlinks 3102that point nowhere (there was an error_exit() that should just be a warning), 3103and provides error messages for files we could open but not read.</p> 3104 3105<p><u>Library</u>: 3106New library functions: strend() complements strstart(), minof()/maxof() 3107are min/max macros that evalute arguments once and autodetect type (why 3108isn't this in libc?), xmmap() checks MAP_FAILED (which is not NULL).</p> 3109 3110<a name="21-02-2017" /><a href="#21-02-2017"><hr><h2><b>February 21, 2017</b></h2></a> 3111<blockquote><p>Only six people in the Galaxy knew that the job of the 3112Galactic President was not to wield power but to attract attention 3113away from it. Zaphod Beeblebrox was amazingly good at his job.</p> 3114<p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 3115 3116<p>Despite everything, <a href=downloads/toybox-0.7.3.tar.gz>Toybox 0.7.3</a> 3117(<a href=https://github.com/landley/toybox/releases/tag/0.7.3>git commit</a>) 3118is out. The <u>new commands</u> this time are <b>ftpget</b>, <b>ftpput</b>, <b>microcom</b>, and <b>ascii</b>.<p> 3119 3120<p>We also had two command _demotions_ out of defconfig: 3121<b>hostid</b> got moved to toys/example and 3122switched to "default n" because despite <a href=http://pubs.opengroup.org/onlinepubs/9699919799/functions/gethostid.html>still being in posix</a> 3123the concept of a unique 32 bit number identifying a system is something 3124Linux outgrew about the time Pauline Middelink wrote the first IP 3125Masquerading code. And Elliott did a complete rewrite of <b>dmesg</b> introducing 3126two codepaths that I didn't get a chance to unify and didn't want to 3127hold up the release for, so that's back in pending.</p> 3128 3129<p><u>New features</u>: Rob added units to <b>find</b> -atime and friends 3130(with the legacy -amin alias). Elliott added color and -w to dmesg, fallocate 3131-o, and improved file's ELF parsing. Steve Muckle added -d and finit_module 3132support to modprobe. Rob and Elliott tweaked the 3133ps/top display format a bit more (extending the USER field from 8 to 18 chars 3134and putting + at the end of string fields that got truncated). 3135df -a isn't entirely new, but wasn't documented and needed a bugfix.</p> 3136 3137<p><u>Bugfixes</u>: 3138Last release broke oneit because -c didn't get moved to xopen_stdio() (oops). 3139Rob and Elliott simultaneously spotted ps padding each line to 99999 3140chars when there's no tty (serial console or adb); now it pads to 80 in 3141that case but also switches on -w to avoid field truncation. The "tty" 3142field also sometimes had trailing debris (that's fixed now). And "top" was 3143endlessly redrawing with out tty because receipt of the ANSI size probe 3144results would set SIGWINCH, and handling that sent another ansi probe. (Sigh.) 3145And while we're there, replace "ADDR" with "BIT" in ps -l so there are 3146more than 4 chars left for the "CMD" field on 64 bit systems.</p> 3147 3148<p>Izabera pointed out that split -b and -l can't mix, and suggested seq should 3149multiply to avoid accumulating rounding errors from repeated fractional 3150increments. Wang Xiao Jian fixed a bug in sort -k. 3151Elliott let getprop use the @ character in property names, and 3152Dimitry Ivanov removed the name length limit for system properties. 3153Elliott also improved some error reporting and improved top -H's display 3154of thread names. 3155 3156<p>Josh Gao pointed out that recursive operations on . and .. could be ignored 3157in chmod -R (and the resulting generic fix to dirtree_notdotdot() fixed 3158it in several other places).</p> 3159 3160<p>Justin Cormack caught tar producing a warning to stdout that screwed up 3161"tar c" to stdout. 3162Rob fixed an option parsing bug (where switching off a --longopt in menuconfig 3163confused the parser), and another one where an option excluding itself 3164(ala "abc[-ab][!abc]" with "command -a -b") would segfault.</p> 3165 3166<p>There's some sort of gcc stack over-optimization bug where musl-libc's 3167version of vfork() doesn't get marked with attribute(returns_twice) so 3168stack varabiles in the same function after that get semi-randomly overwritten 3169when the optimizer decides to reclaim the space. So add the attribute 3170to the function the XVFORK() wrapper macro calls. (It's a nommu thing.)</p> 3171 3172<p>Fixed a couple variable size mismatch bugs that were only tested on 64 bit 3173(printf %x 64) or only tested on 32 bit (modprobe), removed some 3174unnecessary casts in stat.</p> 3175 3176<p>Continuing attempts to build under Android NDK brought up that posix 3177defines the global 'stdout' as a macro, which older versions of bionic 3178turned into an array member, but a function was using it as an argument 3179name. (This worked in the AOSP build because it only builds against current 3180bionic, where there's a global 'stdout'.)</p> 3181 3182<p>Several commits argued with clang's warning generation, eventually 3183settling on a variant of __attribute__((__shut_up__)).</p> 3184 3185<p>Android should no longer give spurious error messages 3186when you "ps -A | head" about EPIPE on output. (Older versions of bionic 3187set an error handler on SIGPIPE, but it shouldn't do that now. More recent 3188versions of adb set the SIGPIPE handler to SIGIGN instead of SIGDFL, 3189leading to write returning an error message instead of silently killing 3190the program. So we set it back to the default.)</p> 3191 3192<p><u>Docs</u>: 3193Removed website link to the gmane archive (which didn't survive gmane's 3194change of ownership). The FAQ now answers a _second_ question. (Woo!) 3195Some roadmap updates.</p> 3196 3197<p><u>Build tweaks</u>: 3198Upgraded "make install_airlock" target to only warn about missing 3199commands (unless $PEDANTIC is set) when it sets up the hermetic build 3200path. (The plan is still to implement everything but the toolchain 3201binaries in toybox, but in the meantime we're symlinking other stuff from 3202the $HOST that isn't ready yet. See 3203<a href=https://github.com/landley/mkroot>mkroot</a> for an example using 3204this.)</p> 3205 3206<p>Elliott and Rob continue to poke at building toybox with Android's NDK, 3207but it's a work in progress (<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-December/008767.html>thread</a>). Various changes 3208removing libcutils dependencies and adding an selinux dependency to getprop 3209are fallout from this.</p> 3210 3211<p>Cross-compiling from Macs needs to use "gsed" instead of apple's 3212version, so teach the build to use that name if it exists in the $PATH. 3213If you try to build without running config first, you should get better 3214error reporting now. Added a workaround for Centos' broken "which" command 3215producing output when it _can't_ find a name in the $PATH.</p> 3216 3217<p><u>Library</u>: 3218The new dirtree flag DIRTREE_PROC skips non-numeric entries so things 3219like ps and top can scan /proc more efficiently.</p> 3220 3221<a name="21-10-2016" /><a href="#21-10-2016"><hr><h2><b>October 21, 2016</b></h2></a> 3222<blockquote><p>Probability factor of one to one. We have normality. I repeat, 3223we have normality. Anything you still can't cope with is therefore your 3224own problem.</p><p> - The Hitchhiker's Guide to the Galaxy</p></blockquote> 3225 3226<p><a href=downloads/toybox-0.7.2.tar.gz>Toybox 0.7.2</a> 3227(<a href=https://github.com/landley/toybox/releases/tag/0.7.2>git commit</a>) 3228is out.</p> 3229 3230<p>During this development cycle, Elliott Hughes <a href=http://androidbackstage.blogspot.com/2016/07/episode-53-adb-on-adb.html>got interviewed 3231on the ADB podcast</a> 3232and Rob Landley <a href=http://linuxluddites.com/shows/episode-88/>got interviewed on Linux Luddites</a> (<a href=http://linuxluddites.com/shows/episode-11/>again</a>). 3233Both talk about toybox and many other things. 3234The web page also grew a new <a href=faq.html>FAQ page</a>, currently with 3235just the one.</p> 3236 3237<p>New comands added to defconfig are <b>tunctl</b>, 3238<b>log</b>, <b>start</b>, <b>stop</b>, and <b>sendevent</b>. 3239The commands <b>file</b> and <b>netstat</b> got promoted out of pending. 3240Pending added <b>chrt</b>, <b>setfattr</b>, and <b>getfattr</b>, and saw 3241a lot of cleanups to diffstat and dd but not enough to promote them to 3242defconfig yet. A new toys/net directory was added, moving ifconfig, netcat, 3243netstat, rfkill, and tunctl there so far.</p> 3244 3245<p><b>Upgrades</b>: All commands now parse --version when they understand 3246--help, but "true" and "false" should now ignore their arguments entirely. 3247We taught stat to handle "%12x" and "%.12x" printf-style escapes, which 3248apparently other versions do. The ifconfig output now shows the interface's 3249device driver. Added patch -d and --dry-run, wc can now do -cm together, 3250find has a NOP -noleaf so scripts that use that don't break, add -c to md5sum 3251and sha1sum. Elliott taught ps to treat extra aguments as additional -p 3252pids, implemented xxd -s, did a number of upgrades to file (added -HL, 3253support for ar files, improved ELF support to report android API level 3254and stripped/not stripped and it no longer prints a guessed build ID type). 3255Elliott also added optional build-time support for using openssl's 3256assembly-optimized md5sum/sha1sum implementations (leading to a new 3257<a href=design.html>design</a> policy on shared libraries).</p> 3258 3259<p><b>Bugfixes</b>: Too many fixes to "ps" and "touch" to list, from both 3260Elliott and Rob. Rob taught sed to handle s/[[:space:]/]// type sequences 3261properly, switched grep to a better 3262workaround for <a href=https://sourceware.org/bugzilla/show_bug.cgi?id=17829>glibc bug 17829</a>, made sed -i preserve ownership when run as root, 3263made du max out at 2 terabytes instead of 2 gigabytes on 32-bit systems 3264(it was always designed to, but was missing a typecast), 3265fixed the option parsing infrastructure (config options that remove command 3266line options got the placeholders wrong), fix to printf for printing 3267octal digits and handling the (posix-mandated) difference between %b and 3268non-%b octal output, reading from "-" no longer closes stdin when done, 3269netcat -L works with nommu (although it may need more portability work), 3270and you can now "make test_scankey" if you want to. Several commands 3271(stat, makedeves, chgrp, cp, find) handled user name lookup failure badly 3272(stat was segfaulting if you interrogated a file belonging to a nonexistent 3273user, "chown 12345 file" errored out if you didn't have that user 3274in /etc/passwd... now they should all print/accept the number when 3275appropriate). "LC_ALL=C ls -Cs --color" produces the same output 3276as other versions (two spaces padding, -k hardwired on).<p> 3277 3278<p>Kyungsik Lee fixed a bug 3279in cp (readlink() doesn't actually null terminate the string it reads in), 3280Elliott Hughes made pgrep/pkill return success/failure, fixed trailing 3281whitespace in netstat, fixed a SMACK symbol conflict due to linux/xattr.h 3282changing, fixed ls -sh, and added a lot of 3283stuff to the <a href=roadmap.html>roadmap page</a>. 3284Izabera pointed out cmp -l and -s can't be selected at the same time, 3285that timeout was never actually checking -v, that ls should default to -q 3286when output is to a tty, and that "file -" would sometimes try to open "-" 3287instead of stdin. 3288Usischev Yury pointed out a use after free error, and that id shouldn't 3289call exit() directly. Matthias Urhahn pointed out that stat(2) returns 3290hardwired 512-byte units, so stat.c was wrong. David Hedges pointed out 3291that route could only handle 10 character interface names when the kernel 3292can do 15 (it's still in pending for a reason, but fixed). Evgenii 3293Stepanov found and helped diagnose one of the more subtle ps bugs fixed 3294this time around.</p> 3295 3296<p>Calling "make test_blah" no longer causes make to error out if the last 3297test fails (and thus returns a nonzero error code). 3298Building single commands and the multiplexer used to require a "make clean" 3299between them (because they had different config files both of which were older 3300than generated/config.h so it didn't get rebuilt; now it just always 3301rebuilds it).</p> 3302 3303<p>The defconfig build is now slightly less broken on older centos versions 3304(although <a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-September/008664.html>the consensus</a> is that Centos is just generally broken).<p> 3305 3306<p>Several commands were over-using xprintf(), which flushes its output 3307to check for error (something you only need to do maybe once per line, 3308and even then maybe only in loops because xexit() flushes and checks 3309ferror() for you and adjusts the exit code if we wrote stuff to stdout 3310that couldn't be printed). Lots of little flushes are inefficient, 3311so most things can use normal printf(). (Retransmission 3312of short writes is presumably libc's problem since it's buffering the 3313output and all.)</p> 3314 3315<p><b>Library:</b> 3316New library functions readlink0() and readlinkat0() which properly null 3317terminates the symlink value (which the stock libc function inexplicably 3318doesn't). 3319The new do_lines() function interates reading lines from a filehandle 3320and calling a function on each line. 3321New function pollinate() factoring out netcat's poll() loop so things 3322like telnet can use it. 3323New functions getusername() and getgroupname() return a 3324char * given a uid/gid (and return a string representation of the number 3325if the lookup fails), and xgetpwnamid/xgetgrnamid were renamed to 3326xgetuid/xgetgid and now return an integer instead of a struct (also helping 3327handle lookup failures, you can still return the uid/get for "12345").</p> 3328 3329<p>Switched atolx() to use long long internally. 3330Renamed xopen() to xopen_stdio() and made a new xopen() that never returns 3331stdin, stdout, or stderr (duping /dev/null into the filehandles as necessary). 3332New function xopenro() opens a file read only with one less argument, and 3333understands that "-" means stdin. New flag WARN_ONLY tells these functions 3334to just print a warning on failure, and return -1 instead of exiting. 3335Misc new functions like openro() which defaults to the WARN_ONLY behavior 3336and notstdio() which dup()s a filehandle up beyond stdin/out/err backfilling 3337with /dev/null as necessary. The WARN_ONLY flag let us remove the failok 3338argument from loopfiles().</p> 3339 3340<p>New TOYFLAG_NOHELP disables --help processing (which "true" and "false" 3341should not do).</b> 3342 3343<p>The test suite now has NOSPACE=1 to ignore whitespace (using diff -b to 3344check results), which helps TEST_HOST pass the same tests as toybox. 3345Fixes to chattr and date tests. It also has a new variable $C with the 3346absolute path to the command being tested (bypassing shell builtins), 3347and a function testcmd() which is just like testing() except it prepends 3348the command name ($C) to the test command line as well as the test 3349description.</p> 3350 3351<a name="02-06-2016" /><a href="#02-06-2016"><hr><h2><b>June 2, 2016</b></h2></a> 3352<blockquote><p>When the 'Drink' button is pressed it makes an instant but 3353highly detailed examination of the subject's taste buds, a spectroscopic 3354analysis of the subject's metabolism, and then sends tiny experimental 3355signals down the neural pathways to the taste centres of the subject's 3356brain to see what is likely to be well received. However, no-one knows 3357quite why it does this because it then invariably delivers a cupful of 3358liquid that is almost, but not quite, entirely unlike tea.</p> 3359<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote> 3360 3361<p><a href=downloads/toybox-0.7.1.tar.gz>Toybox 0.7.1</a> 3362(<a href=https://github.com/landley/toybox/releases/tag/0.7.1>git commit</a>) 3363is out. (Yes, I forgot to update the --version string, but I already 3364uploaded the <a href=downloads/binaries/0.7.1>binaries</a>.)</p> 3365 3366<p>The website has https support now, you can "make cat ps ls" 3367to get standalone commands (and "make list list_pending" to see what's 3368available), and a whole lot of bugfixes and new options to existing 3369commands.</p> 3370 3371<h2>New Commands</h2> 3372<p>Rob implemented <b>ulimit</b>. In pending, Elliott Hughes implemented 3373file. and Lipi Lee implemented a simple wget. (Pending also had minor 3374cleanups to more and lsof, but no promotions this time around.)</p> 3375 3376<h2>New Options</h2> 3377<p>Izabera implemented env -u, suggested adding seq -w, made factor 3378use full unsigned 64 bit math even on 32 bit platforms, pointed out base64 3379-w0 should disable wrapping, and sped up wc -c. 3380Elliott Hughes added mount -o relatime, xxd -p -r, and od -w. 3381Sameer Pradhan (or possibly Bilal Qureshi) suggested adding stat -tL -c %m%t%T. 3382Tom Cherry added getprop -Z. Paul Barker added hostname -b and -F. 3383Rob added ls -b, made ls -q work with utf8, 3384made sed -f - read from stdin, and added top -O (like ps -O).</p> 3385 3386<p>Elliott and Rob <b>added Thread support to ps and top</b>, 3387with -o TID, TNAME. We also added -o PCY (android scheduling policy), 3388-o BIT (process is 32 or 64 bit), and -o TNAME now shows the parent 3389command name for threads.</p> 3390 3391<h2>Documentation</h2> 3392 3393<p>Rob added the sed invocations to convert tabs/spaces and back to 3394design.html. Isaac Dunham updated hexedit's help text. 3395Jakob Flierl pointed out a broken URL in the README.</p> 3396 3397<p>Rob also redid the naming scheme of sed's pattern manipulation code to 3398remove the gratuitous references to Roger Zelazny's "Amber" series, since it 3399was confusing people.</p> 3400 3401<h2>Bugfixes</h2> 3402 3403<p>Grep -H and -n should now work properly with -ABC. Andy Chu pointed out an 3404out of bounds access for zero length lines in rev, fixed a buffer overflow 3405in diff -r, and fixed operator precedence in expr (although Rob is rewriting 3406chunks of expr so toysh can use its plumbing for $(( )) ). 3407Patrick Ohly fixed the too-aggressive suid permission dropping logic. 3408Josh Gao fixed a segfault when find -iname got no argument, and 3409made tail -f work right with just one file argument. 3410Tom Marshall cleaned up tar's long filename support and improved 3411the tar tests, and reported another find bug (with -iname -o -iname not 3412tracking copy lifetimes properly) that got fixed.</p> 3413 3414<p>Elliott Hughes fixed wc -c to not trust zero length files to actually be 3415zero length (/proc does that), fixed "mount -o rw,remount /system" on 3416Android, removed trailing spaces on ps -o cmdline, fixed pkill -9 and 3417the corresponding tests, made "insmod -" work, fixed top -b and tail -NUM, 3418pointed out that ps shouldn't trim numeric fields for display size limits, 3419and added some more 3420explicit "sort" calls to make pipelines so build tempfiles are easier to cache. 3421Rob <a href=https://github.com/landley/toybox/commit/32b3587af261>fixed an insane sed thing</a> the perl 5.22 build was doing. 3422Fixed mount -o to properly pass in leftover string data, and 3423<a href=http://lists.landley.net/pipermail/toybox-landley.net/2016-March/004790.html>documented how to use toybox to mount nfs</a> (warning: kernel patch 3424to fix some bitrot in the kernel NFS driver's string parsing attached 3425to that message).</p> 3426 3427<p>George Burgess IV corrected some variable types in traceroute. 3428Base64 now wraps == properly. Fixed two bzcat segfaults reported by 3429John Regehr. Andy Chu found a segfault in "sed -e 'c\'" with no trailing 3430line, and implemented mv -n and cp -n. The cyanogenmod guys pointed out that 3431cp -a shouldn't complain if a non-root user can't chown, and we added 3432the output path to cp -r error messages while we were there (before was just 3433the filename).</p> 3434 3435<p>Samuel Holland fixed blkid's handling of vfat labels, and 3436fixed a segfault when basename was passed an empty string and an empty 3437suffix. Davis Mosenkovs fixed touch -t seconds parsing. 3438Rob fixed a bunzip bug reported by John Regehr (the bad CRC 3439error message was printing a NUL argument).</p> 3440 3441<p>Not all of the commands build standalone, but more of them do now; 3442scripts/single.sh can now build a "mv" that isn't actually "cp". 3443The dependencies are more granualr, so "make top; make ps" no longer 3444produces a broken ps that ignores -A (because ps.o wasn't getting rebuilt 3445even though top had the FLAG macros for -A zeroed).</p> 3446 3447<h2>Build</h2> 3448<p>Rob added a <b>warning when building commands out of 3449pending</b>. (The pending directory is full of code that hasn't been 3450properly vetted. Use at your own risk.)</p> 3451 3452<p><b>New build targets let you build individual commands by name</b>, ala 3453"make ls cat ps", and you can run the test suite for each standalone 3454command with "make test_ls" and such. 3455"make list" shows all such standalone commands in defconfig, and 3456"make list_pending" shows unfinished commands from toys/pending 3457("make list list_pending" shows both together). "make clean" now deletes 3458these filenames at the top level, and the corresponding unstripped files 3459live in the directory generated/unstripped.</p> 3460 3461<p>Nicholas Boichat suggested switching make.sh to use $! for process 3462enumeration during parallel builds (which is both more efficient and more 3463portable), and suggested shell builtin replacements for wc/awk/sed so the 3464build loop has fewer forks now.</p> 3465 3466<p>Lots of work on the test suite, much of it due to Andy Chu. It now 3467consistently prints the name of the command being tested at the start of each 3468test (and the common infrastructure does that, not each individual test), and 3469"make tests" actually runs all the available tests now. 3470Seperated pgrep and pkill tests, split lsattr/chattr, added fstype and base64 3471tests. The "tests/files" directory now collects files for tests to 3472use, with blkid, bzcat, and utf8 subdirectories: the $FILES variable 3473gives a path to it, so "$FILES/blkid/ext2.bz2" and so on. 3474The testsuite now has test files with 3 different types of "not utf8 output" 3475sequences that require escaping, plus some combining character torture 3476tests, direction reversals, and so on.</p> 3477 3478<p>Added dependencies on TOYBOX_FORK to various pending commands that need 3479nommu conversion (which should fix the allyesconfig build).</p> 3480 3481<p>Static builds with selinux should work again.</p> 3482 3483<h2>Library</h2> 3484 3485<p>New bufgetgrgid() and bufgetpwuid() functions cache previous lookup info 3486rather than repeatedly traversing /etc/passwd and /etc/group (which is slow). 3487Added xpipe() to lib to catch pipe creation failure. 3488The HELP_ macros generated by config2help.c now use a capital prefix 3489to avoid collicing with help_exit() and such.</p> 3490 3491<p>The dirtree infrastructure got a cleanup pass in preparation for adding 3492infinite recursion depth support (needed by rm -r), updated the 3493<a href=code.html#ib_dirtree>documentation</a> to describe the new 3494semantics (removing dirtree_start() and adding dirtree_flagread()). 3495Now dirtree_recurse() takes the new dirfd as an argument.</p> 3496 3497<p>Split out _xexit() from xexit() and let sigatexit() set multiple 3498callbacks.</p> 3499 3500<p>For years the man pages have said to #include <sys/types.h> to get 3501major/minor/makedev but now that glibc 3502<a href=https://sourceware.org/ml/libc-alpha/2015-11/msg00253.html>has vowed 3503to break existing programs</a> and replace it with another nonstandard header 3504not in posix or lsb, we added our own functions to lib/ to do the transform 3505ourselves (based on what the kernel actually expects).</p> 3506 3507<h2>Portability</h2> 3508 3509<p>Debian unstable started needing an extra header #include for some reason, and 3510although printf("%.*s", INT_MAX, s) worked fine on Ubuntu 12.04 it 3511didn't on 14.04, so added a workaround for that. Typecast a printf because 3512wchar_t isn't a rigidly defined size. RLIMIT_RTTIME was 3513added to the kernel in 2008 but you can't expect uClibc to have noticed yet, 3514nor did it #define MS_RELATIME (added in 2006), or prlimit (2010)... 3515(Given the improvements in musl and bionic, uClibc support may be dropped 3516in a future release.) Given that the xattr functions were added during 3517linux 2.5, we can #include its header unconditionally.</p> 3518 3519<p>Renamed basename_r() to something else to avoid conflicting with freebsd's 3520libc, and both scripts/install.h and scripts/config2help.c no longer include 3521toys.h (to make cross-compiling from systems we don't run on easier).</p> 3522 3523<p>Debian bug 635570 did something unspeakably nonportable, depending on 3524"sed -e 'a\'" (with no next line of the pattern, so an unterminated 3525continuation) to add a newline to the last line of input if and only if 3526that last line of the input didn't have a newline, and to take no other 3527action. This is well into "depending on a bug" territory, but we implemented 3528it because otherwise Debian's install broke. (Of course this behavior 3529is undocumented, non-obvious, and doesn't really make logical sense.)</p> 3530 3531<p>CONFIG_TOYBOX_NORECURSE now disables the stack measuring logic (which 3532was giving some "security" code fits). Also we typecast pointers to (long) 3533before comparing them to avoid spurious compiler "optimizations" that 3534break the code.</p> 3535 3536<a name="02-02-2016" /><a href="#02-02-2016"><hr><h2><b>February 2, 2016</b></h2></a> 3537<blockquote><p>"I checked it very thoroughly," said the computer, "and that 3538quite definitely is the answer. I think the problem, to be quite honest with 3539you, is that you've never actually known what the question is." 3540</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 3541 3542<p><a href=downloads/toybox-0.7.0.tar.gz>Toybox 0.7.0</a> 3543(<a href=https://github.com/landley/toybox/releases/tag/0.7.0>git commit</a>) 3544is out.</p> 3545 3546<p>The new commands in defconfig are <b>iotop</b>, <b>top</b>, <b>pgrep</b>, 3547and <b>pkill</b> 3548(most replacing corresponding versions from pending). Added grep -ABC, 3549swapon -d (discard), mkswap -L (label) and UUID support, and find -delete. 3550Izabera added free -h and unshare -f. Josh Gao implemented tail -f. 3551Jose Bollo submitted cp --preserve=context,attr. Kylie McClain added 3552mktemp -u.</p> 3553 3554<p>In pending there's the start of a vi command, and Sameer Pradhan contributed 3555a new dhcp6. This cycle saw several rounds of route cleanup and a little dhcp 3556cleanup, but neither are complete yet. Lipi Lee did some cleanup to netstat.c 3557and Elliott Hughes removed warnings from traceroute.</p> 3558 3559<p>Lots of updates to ps: several new -o options, -k (--sort) -O and -M, 3560improved compatibility with Android's historical behavior, and 3561extensive internal code cleanup (including the removal of all 3562the magic constants).</p> 3563 3564<h3><b>Website</b></h3> 3565 3566<p>Dreamhost restored the <a href="#12-21-2015">missing 11 months</a> 3567to the mailing list archive, in the process deleting the month after 3568that. Now they've asked if I have mbox files archiving the new 3569gap (between December 20, 2015 to January 21, 2016, and presumably they 3570could also fill in the gap from December 14, 2014 to January 3, 2015 that's 3571been there since the last time they did this), 3572but due to some gmail filtering I've 3573<a href=http://landley.net/notes-2012.html#15-10-2012>never 3574been able to disable</a>, my copy of those files is spread among 3 different 3575mbox files I'd have to sort/filter/collate. (It's on the todo list.)</p> 3576 3577<p>Added a code of conduct to the README (we're 3578<a href=https://engineering.twitter.com/opensource/code-of-conduct>borrowing twitter's</a>) because somebody 3579made it necessary.</p> 3580 3581<h3><b>Bugfixes</b></h3> 3582<p>Fixed another sed bug where any ] right after [ was skipped (not just the 3583first one in the range, so [[] didn't terminate). Fixed sort -f and added test cases. 3584Assume 80 columns in "ls -m | cat", ls -L is no longer backwards, 3585and ls of files with no paths no longer uses an uninitialized (zero) dirfd. 3586Several bugfixes 3587to find (Gilad Arnold fixed -perm, Daniel K. Levy fixed "find . -exec echo {}", 3588and while we're there I fixed find --prune, made "find . -execdir 3589echo {} + -execdir ls {} +" work, and ripped out the environment size 3590measuring code that checked for a 128k limit removed back in linux 2.6.22). 3591Elliott Hughes fixed the date command's parsing of 4 digit 3592years and documented the %s escape, fixed hwclock -u, and pointed out 3593that runcon needs to exec to do its job (not recursively call another 3594command_main() in the same process). Tom Marshall reported that blkid was 3595handling ext2 wrong. Mike Moreton corrected cpio extraction's uid and gid 3596values, and added a --no-preserve-owner option. Fixed the SUID permission 3597dropping logic (which was a bit over-zealous, preventing some commands from 3598running at all).</p> 3599 3600<p>I'm told that debian-testing broke its libc so the nsenter build breaks, 3601but my attempts to install the debian-testing network cd image under 3602qemu keep breaking. Maybe someday they'll fix it enough I can actually 3603reproduce the problem. (Debootstrap under unbuntu builds an ubuntu-flavored 3604chroot in which toybox builds fine.)</p> 3605 3606<h3><b>Documentation</b></h3> 3607<p>Rewrite of the about.html page, tweaks to design.html, and a re-triage of 3608sbase in roadmap.html. Update to mkstatus.py to collate multiple span 3609tags with the same id, resulting in a larger status.html page (which 3610was previously ignoring some commands in the roadmap).</p> 3611 3612<p>Expanded the defconfig/allyesconfig/allnoconfighelp text in "make help" 3613to explain what they're for.</p> 3614 3615<h3><b>infrastructure</b></h3> 3616<ul> 3617<li><p>Expanded toys.optargs to 64 bits so a command can have more than 32 options.</p></li> 3618<li><p>Added NOEXIT() wrapper to turn xwrap() functions into warning versions 3619using the existing longjump(toys.rebound) infrastructure.</p></li> 3620<li><p>Renamed dirtree->data to dirfd and stopped storing symlink length 3621into it (this fixed a bug where following symlinks to directories 3622didn't give a valid directory filehandle, noticeable with ls -Z).</p></li> 3623<li><p>New TAGGED_ARRAY() infrastructure generates index and bitmask macros 3624for arrays of structures starting with a name string.</p></li> 3625<li><p>New lib/linestack.c for utf8 fontmetrics (draw_str() and utf8len() 3626and so on), and for tracking multiple lines of text 3627(vi, less, shell history) that need wordwrapping and scrolling up/down.</p></li> 3628<li><p>Upgrades to lib/interestingtimes.c: scan_key() now has a timeout 3629in milliseconds and recognizes more sequences including ANSI 3630window size probes. New utf8 test files in tests/files/utf8 including 3631sequence reversing, stacked combining chars, and all three types of 3632unprintable sequences (low ascii <32 ala ^X, invalid utf8 sequences ala 3633<AB><CD>, and invalid unicode code points ala U+1234).</p></li> 3634<li><p>More comma handling code in lib.c: comma_args()</p></li> 3635<li><p>Added error_msg_raw() to shut up fortify's endless static checking false 3636positives.</p></li> 3637<li><p>readfileat() can now realloc() in a loop to read long files 3638("zcat | insmod" needed it).</p></li> 3639</ul> 3640 3641<h3><b>Roadmap</b></h3> 3642<p>We're getting close to having a self-hosting development environment 3643using toybox for the command line. The remaining busybox commands in 3644<a href=http://landley.net/aboriginal/about.html>Aboriginal Linux</a> are:</p> 3645 3646<blockquote><p><b> 3647awk bunzip2 bzcat bzip2 dd diff expr fdisk ftpd ftpget ftpput gunzip gzip 3648less ping route sh sha512sum tar test tr unxz vi wget xzcat zcat 3649</b></p></blockquote> 3650 3651<p>And the remaining non-busybox commands in Aboriginal Linux's build/host 3652directory (from the distcc, genext2fs, e2fsprogs, zlib, and squashfs packagesi) 3653are:</p> 3654 3655<blockquote><p><b> 3656mke2fs fsck.ext2 resize2fs distcc genext2fs unsquashfs distccd mksquashfs tune2fs 3657</b></p></blockquote> 3658 3659<p>Squashfs and distcc are probably out of scope for toybox, but mke2fs, 3660fsck.ext2, resize2fs, genext2fs, and tune2fs should all be added to the 3661above "busybox" replacement list.</p> 3662 3663<p>Remind me to include this countdown in future releases. Once they've all 3664been replaced, the next goal is <a href=http://landley.net/aboriginal/about.html#selfhost>building AOSP under itself</a>.</p> 3665 3666<p>See the full <a href=roadmap.html>roadmap</a> and <a href=status.html>status</a> 3667pages for more details.</p> 3668 3669<a name="12-21-2015" /><a href="#12-21-2015"><hr><h2><b>December 21, 2015</b></h2></a> 3670 3671<p>Yes, 11 months have gone missing from the mailing list web archive.</p> 3672 3673<p>Yesterday evening Dreamhost's mailman server went down (timing out trying 3674to connect). I poked them about it, they 3675<a href=https://twitter.com/landley/status/678781271670149121>blamed 3676DNS</a>, I explained that the hang was _after_ the DNS lookup and 3677entered the dig info into the trouble ticket showing the IPs the DNS 3678queries were returning, they reinstalled the server at that IP from what I 3679assume was their most recent backup, and that's how 11 months of messages 3680vanished out of the archive.</p> 3681 3682<p>I've <a href=https://twitter.com/landley/status/679114451975467008>continued to poke them about it</a> but I honestly believe that's the best they 3683can do. <a href=http://landley.net/dreamhost.txt>Last time</a> 3684this sort of thing <a href=http://landley.net/dreamhost2.txt>happened</a> 3685we went back and forth for months, so I added a link to a backup web 3686archive (in the nav bar on the left) that isn't controlled by dreamhost, 3687and thus doesn't gratuitously lose data on a regular basis. (I note 3688the earlier hole in Dreamhost's archive was never fixed either. That 3689was data never getting archived, this is a year's worth of data that 3690was in the archive until yesterday vanishing after the fact.)</p> 3691 3692<p>If you're wondering why the <a href=http://lists.landley.net>top level</a> 3693list page has been "temporarily disabled" for multiple years now... you'd have 3694to ask Dreamhost. I know I have. More than a dozen times.</p> 3695 3696<a name="03-11-2015" /><a href="#03-11-2015"><hr><h2><b>November 3, 2015</b></h2></a> 3697<blockquote><p>"Alright," said Ford. "How would you react if I said that I'm 3698not from Guildford after all, but from a small planet somewhere in the vicinity 3699of Betelgeuse?" Arthur shrugged in a so-so sort of way. "I don't know," he 3700said, taking a pull of beer. "Why - do you think it's the sort of thing you're 3701likely to say?"</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 3702 3703<p><a href=downloads/toybox-0.6.1.tar.gz>Toybox 0.6.1</a> 3704(<a href=https://github.com/landley/toybox/releases/tag/0.6.1>git commit</a>) 3705is out.</p> 3706 3707<p>We have a new <b>ps</b> command with all the -o fields posix wants (although 3708it doesn't accept BSD non-dash option syntax yet), and <b>bunzip2</b> (not just 3709bzcat but the proper extract-in-place command). 3710Sameer Pradhan added <b>hostid</b> and <b>fsync</b>. 3711Elliott Hughes added <b>flock</b>. 3712 3713<p>The people waiting for <b>human readable number support</b> (du -hH, ls -h, 3714and so on) can thank Elliott Hughes for implementing it. (Our output doesn't 3715exactly match others' because we our "binary" mode will say 1.0G instead of 37161024M, which is a bug in the other one we didn't emulate.)</p> 3717 3718<p>The other big news is <b>nommu support</b>, tested on the new 3719<a href=http://nommu.org/jcore>jcore</a> processor but presumaby working 3720on any nommu system. A few commands don't support nommu yet, but those 3721are disabled by dependencies on TOYBOX_FORK in menuconfig when building 3722for nommu. The roadmap now has a large section analyzing the uClinux 3723project (note that <a href=http://nommu.org>nommu.org</a> is slowly replacing 3724<a href=http://uclinux.org>uclinux.org</a> as the standard repository of 3725all knowledge and wisdom about nommu. The old site <a href=#12-02-2012>contains 3726much that is apocryphal</a>, or at least wildly inaccurate, and the new one 3727is trying to improve on that).</p> 3728 3729<p>Both "make change" and scripts/single.sh (for building standalone commands 3730without the multiplexer logic) now use the top level .config 3731for toybox global settings such as Linux Security Blanket Module selection, 3732(so make defconfig before change now).</p> 3733 3734<p>Documentation updates to the <a href=code.html>code</a> and 3735<a href=roadmap.html>roadmap</a> pages.</p> 3736 3737<h3>pending</h3> 3738 3739<p>In the pending directory Sameer Pradhan added tftp, 3740and Elliott Hughes sent lsof. Isaac Dunham upgraded mdev, 3741reboot, init, login, and modprobe, and fixed a distro-specific build break in 3742scripts/mkflags.h. Elliott Hughes and Lipi Lee made netstat -p handle 3743command lines longer than 21 characters, and Elliott fixed netstat -e and 3744some build warnings. Yeongdeok Suh fixed a warning in dhcpd. 3745I started cleanup on pgrep/pkill.</p> 3746 3747<h3>Command updates, bugfixes, and infrastructure</h3> 3748 3749<p>The multiplexer's "command not found" error exit is now 127, so now you can't 3750distinguish between a command not being found in the multiplexer and 3751the multiplexer itself not being found by the shell, because people wanted 3752that for some reason.</p> 3753 3754<p>Elliott Hughes made date reject invalid dates rather 3755than set the clock to something weird (setting the clock 100 years into the 3756future makes most Linux desktops surprisingly unhappy, and ntpdate won't fix it 3757either), fixed several ls -l display issues (user/group field ordering, 3758make user/group/lsmcontext left aligned), did the aforementioned 3759extensive work on human readable number output, fixed ionice's default 3760class, fixed a mv overwrite bug, made df's columns auto-size, added 3761--ppid and -Z to ps, and teamed up with Daniel K. Levy to fix 3762a segfault in find's handling of -newer -group or -user.</p> 3763 3764<p>Hyejin Kim added stat -c %T support. Colin Cross worked 3765on vmstat fixing 3766a header printing bug and calculating the bi and bo columns in the right 3767units. Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra 3768newline and that grep -w '\(x\)\1' didn't work, both now fixed. 3769Alistair Strachan fixed several problems with switch_root. Kylie McClain 3770pointed out env should be able to clear variables via NAME= syntax. 3771Dima Krasner added support for running blkid without a partition (so it shows 3772all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p> 3773 3774<p>Isabella Parakiss reported that sed -e "/x/c\" -e "y" added an extra 3775newline and that grep -w '\(x\)\1' didn't work, both now fixed. 3776Alistair Strachan fixed several problems with switch_root. Kylie McClain 3777pointed out env should be able to clear variables via NAME= syntax. 3778Dima Krasner added support for running blkid without a partition (so it shows 3779all partitions). Hyejin Kim sent in a bunch of static analysis bug reports.</p> 3780 3781<p>Two large thinko fixes in oneit: -3 was always enabled (which would 3782eventually block if the child never read the exiting PID numbers from its file 3783descriptor #3 until the pipe filled up), and the signal handlers weren't 3784set up right (for requesting semi-graceful halt/poweroff/reboot). 3785Calling install without a mode is now 0755, and install -g 0 no longer clashes 3786with cp --preserve. Better error message for ls -r on unreadable 3787directories, and ls -Z now uses O_PATH (with the /proc/self/fd/%d 3788workaround for kernel stupidity as necessary).</p> 3789 3790<p>Date now understands @unixtime[.fraction] and uses -D for 3791the set-side format (matching busybox's extension for this). The seq -f 3792string now checks that it's got exactly one %f escape with the correct 3793attributes (and a whole bunch of test cases for it). Fixed a bug 3794in od that screwed up the position indicator on arm and mips. 3795In stat the d/h units moved from %d %D to the default string. 3796And patch can now correctly apply hunks with trailing context to the start of 3797the file.</p> 3798 3799<p>The prompt argument moved out of yesno() (the caller can print the prompt 3800themselves). Replaced toys.exithelp with help_exit(). Added new 3801XVFORK() macro, and xpopen_both() calls /proc/self/exe when passed 3802a NULL argv (see cpio -p for example usage). Replaced toys.recurse 3803with toys.stacktop so the recurse or re-exec decision is now based 3804on bytes of stack space used. Marked a bunch of command-local functions 3805static.</p> 3806 3807<p>New additions to lib/ include strlower(), xconnect(), and the 3808aforementioned help_exit(). 3809The testsuite now has some infrastructure tests based on "example" 3810commands such as toys/examples/test_human_readable.c. 3811The login command finally got a long-overdue cleanup (it's one of the 3812commands that predate the "pending" directory but were part of the reason 3813for it). Hexedit had an 3814uninitialized variable (of course gcc didn't spot it, it was too busy 3815warning about "may be used uninitialized but never actually is" variables).</p> 3816 3817<p>Tweaked makefile so 3818"make CROSS_COMPILE=prefix-" (as well as "CROSS_COMPILE=prefix- make", 3819which still works). Toybox is now installed chmod -w so broken installers 3820(like the bunzip2 package's) that try to overwrite existing binaries won't 3821knock out the whole of toybox. 3822GCC 5.2.0 stopped being able to compile Linux 2.6.12's kconfig, but 3823we added a workaround. You can now build uptime without utmpx.h. 3824Alejandro Joya pointed out that enabling smack required smack on the host 3825as well as target when cross compiling, which is now fixed.</p> 3826 3827<p>Note: toybox can autodetect nommu support when building with a uClibc 3828toolchain such as <a href=http://landley.net/aboriginal/downloads/binaries/old/1.4.3/cross-compiler-sh2eb.tar.gz>the one from Aboriginal Linux</a>, 3829but <a href=http://github.com/richfelker/musl-cross-make>with musl-libc</a> 3830you'll have to enable CONFIG_TOYBOX_MUSL_NOMMU_IS_BROKEN to work around the 3831fact they provide a non-functional fork() implementation that always returns 3832-ENOSYS, to prevent you from compile-time probing for nommu support when 3833cross-compiling. Unfortunately "preventing you from probing" seems to be 3834an explicit policy with musl, they also don't provide an "#ifdef __MUSL__" 3835because their library is perfect and you're only ever allowed to work around 3836other people's bugs, not theirs. So we have to use menuconfig to manually 3837enable musl-specific bug workarounds.</p> 3838 3839<a name="23-07-2015" /><a href="#23-07-2015"><hr><h2><b>July 23, 2015</b></h2></a> 3840<p>I recreated the <a href=downloads/toybox-0.6.0.tar.gz>0.6.0 source tarball</a> 3841(new sha1sum 08fb1c23f520c25a15f262a8a95ea5b676a98d54) 3842because I forgot to add --prefix to the git archive command when I updated 3843my release script from mercurial, so the files weren't in an enclosing 3844directory. (Ooops.)</p> 3845 3846<a name="19-07-2015" /><a href="#19-07-2015"><hr><h2><b>July 19, 2015</b></h2></a> 3847<blockquote><p> 3848The reason why it was published in the form of a micro sub meson electronic 3849component is that if it were printed in normal book form, an interstellar 3850hitchhiker would require several inconveniently large buildings to carry it 3851around in.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 3852 3853<p><a href=downloads/toybox-0.6.0.tar.gz>Toybox 0.6.0</a> 3854(<a href=https://github.com/landley/toybox/releases/tag/0.6.0>git commit</a>) 3855is out. (Yes, git. See the <a href=#05-04-2015>previous news entry</a>.)</p> 3856 3857<p>Sorry for the unusually long gap between releases. Since last release Ye 3858Olde Project Maintainer traveled to japan twice and had two more "once 3859a century" floods at home. (Probably a coincidence.) Still catching up.</p> 3860 3861<h3><b>CELF/ELC talk and Wikipedia[citation needed] article</b></h3> 3862 3863<p>I gave another State Of The Toybox talk 3864(<a href=https://www.youtube.com/watch?v=04XwAbtPmAg>video</a> 3865<a href=http://landley.net/talks/celf-2015.txt>outline</a>), in which I 3866repeat my <a href=http://landley.net/notes-2013.html#07-11-2013>perennial</a> 3867<a href=https://twitter.com/landley/status/557309224535851009>complaint</a> 3868that Wikipedia[citation needed] 3869<a href=http://en.wikipedia.org/wiki/Toybox>still</a> 3870<a href=https://en.wikipedia.org/wiki/BusyBox#Controversy_over_Toybox>says</a> 3871toybox was relicensed before its hiatus, when relicensing was why 3872the hiatus ended.</p> 3873 3874<p>Since Wikipedia[citation needed] seems unable to do the 3875<a href=#15-11-2011>most</a> 3876<a href=http://landley.net/hg/toybox/log/tip/LICENSE>basic</a> 3877<a href=http://landley.net/notes-2011.html#13-11-2011>research</a> on 3878this point, and has stuck to an incorrect sequence of events for years, 3879I've been gradually escalating my attempts to correct them. Toybox 3880came out of mothballs in November 2011 <b>because</b> it could be 3881relicensed. That's what opened up a new niche busybox wasn't already 3882filling with a 10 year headstart.</p> 3883 3884<a name="asterisk_back" /> 3885<p>The article has plenty of smaller issues<a href=#asterisk>*</a>, but 3886given that I gave an entire talk at Ohio LinuxFest in 2013 3887(<a href=http://landley.net/talks/ohio-2013.txt>outline</a>, 3888<a href=https://archive.org/download/OhioLinuxfest2013/24-Rob_Landley-The_Rise_and_Fall_of_Copyleft.mp3>audio</a>) on why I switched away from GPL for 3889my projects, that one bugs me.</p> 3890 3891<h3><b>New stuff this release</b></h3> 3892 3893<p>There's a new android menu in menuconfig, and rather a lot of Linux 3894Security Module support (Smack for Tizen from Xavier Roche and José Bollo, 3895and SELinux for Android from Elliott Hughes; see 3896the Security Blanket menu under global settings in menuconfig) has 3897trickled in, although there's still more to come.</p> 3898 3899<p><b>New commands:</b> Added reset, nproc, ionice, and iorenice. 3900Elliott Hughes contributed xxd, runcon, 3901restorecon, load_policy, getenforce, setenforce, getprop, and setprop. 3902Promoted shred, nsenter, and hwclock.</p> 3903 3904<p>You can once again build catv now the flag infrastructure's been updated to 3905let it coexist with cat -v. 3906And on a long plane flight I wrote 3907hexedit, an interactive hex editor that implements the start of 3908cursor control infrastructure (for eventual use by less and vi and shell 3909command history and so on).</p> 3910 3911<p><b>New options:</b> Added sed -E as a BSD-compatible synonym for -r. 3912Upgraded oneit with -r (restart), -3 (send exiting PID values to child), 3913and signal handling. Added -v option to timeout, -m to mknod, -u to shred, 3914-t to dmesg, and -123 to head and tail. Added implicit "." to grep -r without 3915any files to work on. Hyejin Kim requested prefix support for truncate -s. 3916Greg Hackman added -inum to find. 3917Jan Cybulski added the smack side of ls -Z support. Various patches also 3918added -Z to mkdir, mknod, and mkfifo. 3919Basic cp --preserve support went in, but not yet the xattr/LSM parts.</p> 3920 3921<p>The toybox command now has a --version option, 3922which uses "git describe" if available.</p> 3923 3924<p><b>Build infrastructure:</b> 3925The "make change" target now saves the output of each failed standalone 3926command build in a .bad file, and "make defconfig" is quieter now.</p> 3927 3928<p>Paul Barker submitted a large patch changing command install paths so 3929"toybox can be installed alongside busybox without confusing 3930update-alternatives". (There's some argument over 3931what the right paths should be, and I'm waiting for 3932people to tell me what else needs fixing because I have no idea. I've 3933been symlinking /bin to /usr/bin since 2002 3934<a href=http://landley.net/writing/hackermonthly-issue022-pg33.pdf>for 3935historical reasons</a>.)</p> 3936 3937<p><b>Docs:</b> The repository link now goes to github, with another link 3938to the commit rss feed.</p> 3939 3940<p>Elliott Hughes updated the Android section of the roadmap 3941(and he would know). Redid bits of scripts/mkstatus.py to make updating 3942status.html easier, and the README is larger.</p> 3943 3944<p>More description of option parsing in code.html, which now describes the 3945FLAG_x macros, switching flag macro sets with FOR_newcommand, how 3946configuration zeroes flag macros and using FORCE_FLAGS to suppress the 3947zeroing of options shared between commands. Also added description of ";" 3948to make --longopts take an optional =value part, and more about TOYBOX_DEBUG 3949to check NEWTOY() option strings (otherwise a bad option string makes 3950lib/args.c obviously segfault, but doesn't explain why).</p> 3951 3952<p>Added a "Why 0BSD?" section to license.html when submitting zero clause bsd 3953to SPDX (according to the pending license spreadsheet, it's been approved for 3954SPDX 2.2).</p> 3955 3956<p>The old list of commands needing cleanup but not in pending was 3957removed from toys/pending/README and instead the issues were added 3958as TODO comments in the individual commands.</p> 3959 3960<p><b>Bugfixes:</b> 3961Fixed mount -a segfaulting without -O (reported by Janus Troelsen), 3962and made it try a "become rw" ioctl() on the block device before falling 3963back to mounting read only (because Android expects that). 3964Fixed printf -- and printf ---. Lots of tweaks to ls -l spacing with 3965different options. Make touch -d and -t actually set time when you don't 3966specify nanoseconds. 3967Fixed a subtle bug where recursive calls (toybox commands that run other 3968toybox commands) weren't resetting all their state. (This manifested as 3969a "no }" error from "find | xargs sed", but could cause other problems.) 3970And David Halls reported another sed bug trying to compile libiconv (which 3971left extra \ at the start of lines in a generated shell script, breaking 3972the build). Output an error message for "cat /mnt".</p> 3973 3974<p>Kylie McClain reported that mktemp broke when $TMPDIR was set to an empty 3975string (which is not the same as unset), that install/find didn't support 3976numeric uid/gids, and that sort -z affects both input and output. 3977Isabella Parakiss fixed a printf.c bug. 3978David Halls fixed bugs in install -D and find -exec. Samuel Holland 3979fixed unshare -r. Hyejin Kim fixed makedevs with a count of 1, fold -w 3980range checking, an error path in scripts/mkflags.c, added -i to dhcpd, 3981and stopped su from prompting the root user for the new user's password. 3982Jan Cybulski spotted wrong indentation when combining ls -s and -i with -C and 3983-x. José Bollo fixed stat %G. Sameer Pradhan fixed a bug in mkfifo -Z.</p> 3984 3985<p>Elliott Hughes asked for a default SIGPIPE handler to disable 3986the signal handler bionic's dynamic loader installs (yes really). Still not 3987100% sure what the correct behavior is there. (Posix is 3988(<a href=http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/10915>actively unhelpful</a>, but at least they're taking 3989<a href=http://austingroupbugs.net/view.php?id=789#c1976>years to 3990make up their mind</a>. Elliott also sent patches to fix a typo in 3991useradd.test, add missing arguments to error_exit() calls and clean up 3992printf() format strings, fix an off by one error in human_readable(), 3993fix dmesg -c error reporting, fix a segfault in comma_scan where the option 3994was the last item in optlist (triggered by mount -o ro,remount), fix 3995hwclock -w, made ifconfig print lowercase MAC addresses (it was bothering 3996him), and make terminal_size() read the right environment variable 3997(LINES, not ROWS). And he suggested the test suite notice high command exit 3998values (corresponding to segfault or other signals).</p> 3999 4000<p>People are apparently using toys/pending commands, despite the police tape
4001and flashing lights, so added louder warnings to toys/pending/README. 4002Elliott Hughes fixed various problems with tar, dd, more, and top. 4003Hyejin Kim cleaned up syslogd and dumpleases. Isaac Dunham added hotplug 4004support to mdev. Yeongdeok Suh added RFC-3315 ipv6 support to dhcpd.</p> 4005 4006<p>I rewrote ps.c from scratch (in pending), but it's not ready for real use 4007yet.</p> 4008 4009<p><b>Portability:</b> 4010On the portability front Bernhard Rosenkranzer fixed a problem where the 4011menuconfig code wouldn't compile in C99 mode. (This led to me documenting 4012the craptacular nature of kconfig in a README, and the plan to replace it 4013sometime before 1.0.) Some extra flags to shut up overzealous llvm warnings 4014were added (and have to be probed for because gcc complains about 4015arguments it doesn't recognize even when they switch stuff _off_ using 4016a standard syntax). Don't depend on malloc(0) to return non-null in ls. 4017David Halls fixed some mac/ios portability issues, 4018implying somebody's built at least part of toybox on a mac.</p> 4019 4020<p>Added basename_r() to lib/lib.c because the posix semantics for basename() 4021are stupid but what the gnu guys did to it was appalling. 4022Turns out bionic already had a basename_r(), but posix still doesn't. 4023Fixed it up in portability.h, but this 4024could break more stuff in future. (Correct fix is to lobby posix to add it, 4025which would probably take about 15 years...)</p> 4026 4027<p><b>Infrastructure:</b> 4028The build now checks $LDFLAGS for linker-only flags, and allows the strip 4029command to fail (binflt toolchains provide a strip that doesn't work). 4030Since time.c uses floating point, added TOYBOX_FLOAT dependency in config.</p> 4031 4032<p>There's a lib/lsm.h defining varous inline functions for linux 4033security modules stuff, if (lsm_enabled()) should turn into a compile-time 4034constant 0 and let code drop out when TOYBOX_LSM_NONE selected, but 4035testing against CFG_TOYBOX_LSM_NONE or derived symbols is still useful 4036becuase when it _is_ enabled the probe turns into a system call you 4037don't want to repeat too much.</p> 4038 4039<p>Switched a bunch of commands from signal() to xsignal(). Factored out 4040xgetgrnamid() and xgetpwnamid() into xwrap.c. Make time.c depend on 4041TOYBOX_FLOAT (since it always uses float so shouldn't be available on 4042build targets without even software float). Added readfileat() to lib/lib.c.</p> 4043 4044<p>The dirtree infrastructure now passes in full flags for the old symlink 4045field, and the new DIRTREE_SHUTUP flag disables warnings if a file vanishes 4046out from under you during traverse. New dirtree_start() wrapper to 4047create dirtree root with only two arguments.</p> 4048 4049<p>The not-curses infrastructure introduced by hexedit mostly moved to 4050lib/interestingtimes.c.</p> 4051 4052<a name="asterisk" /> 4053<a href="#asterisk_back" />Asterisk:</a> such when 4054Tim contacted me (my blog says a couple days before nov 13, 2011, I.E. 405511/11/11 not some specific day 2 months later) to ask if I wanted to work 4056on a new project he was proposing called 4057<a href=http://www.elinux.org/Busybox_replacement_project>BentoBox</a> 4058(because I used to do busybox, he'd forgotten toybox existed 4059until I brought it up). And don't ask me what "focuses not on compatibility 4060with its GNU counterparts" means when CP_MORE adds 7 non-posix options 4061and toys/other has 84 commands in neither posix nor LSB. I think they're 4062struggling to explain the difference having dismissed "licensing" as being 4063the reason it started up again after a long hiatus? The reason I don't think 4064GNU is special is there are a half-dozen other independent 4065implementations of the same unix command tools out there (AT&T, 4066BSD, Coherent, Minix, plan 9, busybox, toybox, and several more analyzed in 4067the <a href=roadmap.html>roadmap</a>, and that's ignoring the implementations 4068written for DOS or in assembly over the years). But I do care what 4069Linux From Scratch expects, and if it's 4070<a href=http://archive.linuxfromscratch.org/lfs-museum/7.6/LFS-BOOK-7.6-NOCHUNKS.html#ch-tools-gcc-pass1>calling mv -v</a> 4071then I impelement mv -v 4072even if <a href=http://landley.net/toybox/roadmap.html>posix hasn't got 4073it</a>. And I don't know why "gnu counterparts" would describe this when 4074util-linux isn't a gnu package, nor are info-zip, e2fsprogs, kmod, less, 4075procps, shadow, sysklogd, vim, zlib, sudo, dhcpcd...</p> 4076 4077<a name="05-04-2015" /><a href="#05-04-2015"><hr><h2><b>April 5, 2015</b></h2></a> 4078<p>Since <a href=https://android.googlesource.com/platform/external/toybox/>android</a> and 4079<a href=https://git.tizen.org/cgit/platform/upstream/toybox.git>tizen</a> 4080and <a href=https://github.com/kraj/meta-musl/tree/master/recipes-core/toybox>openembedded</a> 4081and <a href=https://packages.gentoo.org/package/sys-apps/toybox>gentoo</a> 4082and so on have all been using Georgi Chorbadzhiyski's git mirror rather 4083than the mercurial repository, I bit the bullet and switched the project's repo 4084<a href=https://github.com/landley/toybox>to git</a>. Georgi's 4085<a href=https://github.com/gfto/toybox>mirror</a> is now pulling from that.</p> 4086 4087<a name="25-02-2015" /><a href="#25-02-2015"><hr><h2><b>February 25, 2015</b></h2></a> 4088<blockquote><p>"A common mistake that people make when trying to design 4089something completely foolproof is to underestimate the ingenuity of 4090complete fools."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 4091 4092<p><a href=downloads/toybox-0.5.2.tar.gz>Toybox 0.5.2</a> 4093(<a href=/hg/toybox/shortlog/1702>commit 1702</a>) is out.</p> 4094 4095<p>New promoted commands: sed (finally fixed enough it builds Linux From 4096Scratch), printf (cleaned up and promoted), shred and 4097base64 (the Tizen guys wanted them), getenforce, setenforce, and chcon (android), 4098mix (promoted with fixes from Isaac Dunham), nsenter (from 4099Andy Lutomirski, merged into unshare).</p> 4100 4101<p>Elliott Hughes submited a bunch of patches to support Android (to 4102both toybox and Bionic libc, which he maintains). On toybox's end this 4103involved a lot of fixups to portability.[ch] and fixes to over a dozen 4104commands, plus several new ones. Other portability fixes included working 4105with buildroot's uclibc fork and building for nommu targets.</p> 4106 4107<p>The new "make change" target builds each toybox command as a standalone 4108binary. Rather a lot of commands that didn't build by themselves (mv depending 4109on cp and so on) were hit with a large rock until they built standalone. 4110This involved rewriting bits of option parsing, more elaborate dependency 4111generation, making each command have its own config 4112symbol and main() function (even when it's just a wrapper calling another 4113command's main()), and so on. Also, some commands can't be built standalone 4114at a conceptual level: "help" describes other enabled commands and "sh" 4115has a number of bulitin commands (cd, exit, set) that require the 4116multiplexer infrastructure, so "make change" filters them out.</p> 4117 4118<p>The mailing list's web archive is still screwed up. Dreamhost has 4119been trying to fix it since approximately September. There are 4120<a href=http://www.mail-archive.com/toybox@lists.landley.net/>two</a> 4121<a href=http://news.gmane.org/gmane.linux.toybox>other</a> less broken 4122archives, but neither has quite the same UI as mailman.</p> 4123 4124<h3>Bugfixes and tweaks</h3> 4125 4126<p>Cynt Rynt sent in tests for ifconfig, 4127Robert Thompson taught factor to accept whitespace separated arguments, 4128Hyejin Kim pointed out that some of mktemp's longopts were attached to 4129the wrong short options, 4130Luis Felipe Strano Moraes fixed a wrong free() call in bootchartd in pending. 4131Patches from Ashwini Sharma to make "df /dev/node" work, prevent du from 4132looping endlessly following symlinks, and to make expr.c 4133(in pending) understand == and regex matches. (Speaking of expr, it gets 4134priority groupings wrong but the bug was actually in the posix spec's 4135HTML conversion. They fixed the posix spec upstream for us. Still need 4136to fix the expr code, but it's in pending for a reason...)</p> 4137 4138<p>Some commands grew new option flags, such as cp --remove-destination 4139and touch -h.</p> 4140 4141<p>The parallel build has better error reporting now. When toybox needs to 4142re-exec itself to regain suid root permissions and hasn't got the suid bit, 4143it now gives the right error message ("not root" instead of "no such command"). 4144 4145<p>Added a test to "mount" to not mount the same device/directory combination 4146over itself (the OS catches this for block devices, but not for tmpfs). 4147Make blkid distinguish ext3 from ext4. Added catv back into cat (because 4148the Android guys wanted it, and they have historical usage on their side, 4149so...). Handle nanoseconds in touch.</p> 4150 4151<p>Fixed a segfault when CP_MORE was disabled (the resulting option flag list 4152no longer defined -d but still had it in option groups at the end). 4153Workaround for glibc redefining dirname() and basename() to random non-posix 4154semantics because gnu. (They could have created dirname_r() but didn't want 4155to.)</p> 4156 4157<p>Fix an ifconfig test that was preventing assigning an ipv4 address to 4158interface aliases. Several cleanup passes on hwclock but not quite 4159promoted out of pending yet.<p> 4160 4161<p>Fixed a wrong error message in rm (if you had a chmod 000 directory and 4162did rm -r on it without -f, after the prompt it would complain it was a 4163directory, which was not the problem).</p> 4164 4165<p>The gzip compression code now does "store only" output to stdout, for 4166what that's worth.</p> 4167 4168<p>Cleanup mountpoint and expand, and remove them from toys/pending/README 4169(a list of commands that predate the toys/pending directory but needed 4170another pass).</p> 4171 4172<h3>Library and infrastructure:</h3> 4173 4174<p>Reworked the option parsing infrastructure so more commands build 4175standalone (via scripts/single.sh or "make change"). The option flag bit 4176values are no longer packed, it leaves spaces where currently disabled 4177flags go, and you can #define FORCE_FLAGS so disabled flags aren't zeroed. 4178This allows multiple commands to more easily share infrastructure, even if 4179your current flag context is for a disabled command (switched off in config), 4180you can force them to stay on and as long as the flags read the same right 4181to left they'll have the same values.</p> 4182 4183<p>We've started removing use of strncpy() because it's a hugely broken 4184standard C function: the length is the maximum length to _append_, not 4185the size of the destination buffer. It memsets the remaining space it didn't 4186copy ala "memset(dest+strlen(dest), 0, len);" so 4187if you think len is the size of dest you're guaranteed to stomp memory off the 4188end). And if it runs out of space it won't null terminate because reasons. 4189(Meanwhile sprintf("%*s", len, str) is counting wide characters in your current 4190locale, so if you set a locale other than "C" it will also go past your 4191allocated buffer size. Whoever is maintining the C library standards is really 4192bad at strings.) 4193Instead we have xstrncat() which will error_exit() if src+dest+1 doesn't 4194fit in the buffer. (Because randomly truncating input data isn't necessarily 4195an improvement.) And there's always xmprintf().</p> 4196 4197<p>Similarly, strtol() doesn't return an error indicator on overflow, 4198you have to clear and then check errno. So new xstrtol() that cares 4199about overflow.</p> 4200 4201<p>The bionic and musl guys agree faccessat(AT_SYMLINK_NOFOLLOW) is not 4202supported, so stop using it.</p> 4203 4204<p>Fixed toy_exec() to detect when argc is in optargs, so we don't 4205need a separate xexec_optargs().</p> 4206 4207<a name="18-02-2015" /><a href="#18-02-2015"><hr><h2><b>February 18, 2015</b></h2></a> 4208<p>Dreamhost continues to be unable to make mailing list archives work, so 4209here's <a href=http://www.mail-archive.com/toybox@lists.landley.net/>another 4210list archive</a> with a less awkward interface than gmane.</p> 4211 4212<p>(Neither gives you the convenient historical monthly views of mailman, 4213but I still have hopes dreamhost will someday figure out what they're doing 4214wrong. They've only been trying since October. Last month they did a 4215<a href=http://www.dreamhoststatus.com/2015/01/14/discussion-list-hardware-maintenance/>hardware upgrade to fix a software problem</a>, and the stale 4216data loads much faster now, so that's something.)</p> 4217 4218<p>Update (Feb 19): the archive started updating again, by discarding 4219all the pending data. So there are now _two_ giant holes in Dreamhost's 4220web archive, from Dec 15-Jan 3, and then another hole from Jan 16-Feb 18. 4221The relevant messages are in both of the other archives. Here's hoping 4222the chronic archive constipation problem won't happen a sixth time.</p> 4223 4224<a name="30-12-2014" /><a href="#30-12-2014"><hr><h2><b>December 30, 2014</b></h2></a> 4225<p>Due to Dreamhost's <a href=http://landley.net/dreamhost.txt>ongoing</a> 4226<a href=http://landley.net/dreamhost2.txt>inability</a> to make mailman 4227work reliably, I've added a link to a backup web archive at 4228<a href=http://news.gmane.org/gmane.linux.toybox>gmane</a> to the nav bar 4229on the left.</p> 4230 4231<p>You still subscribe to the list through 4232<a href=http://lists.landley.net/listinfo.cgi/toybox-landley.net>the first link</a>.</p> 4233 4234<p>Update (January 27, 2015): they're <a href=https://twitter.com/landley/status/558428839462703104>still working on it</a>.</p> 4235 4236<a name="19-11-2014" /><a href="#19-11-2014"><hr><h2><b>November 19, 2014</b></h2></a> 4237 4238<blockquote><p>"This time it was right, it would work, and no one would have to get nailed to anything."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 4239 4240<p><a href=downloads/toybox-0.5.1.tar.bz2>Toybox 0.5.1</a> 4241(<a href=/hg/toybox/shortlog/1566>commit 1566</a>) is out.</p> 4242 4243<p>It's an interim release, mostly bugfixes. There are several new commands, 4244but they're all in pending.</p> 4245 4246<h3>Development</h3> 4247 4248<p>Finally implemented sed, which is still in pending because although 4249it's feature complete according to posix, and even passes the parts of 4250Busybox's sed test suite that aren't explicitly testing for gnu bugs we 4251don't want to copy, it's not yet good enough to build Linux From Scratch. 4252(The ./configure stages use very long sed scripts. 20 commits worth of 4253implementation and debugging, just under 1000 lines of code, and there's 4254still more to do. We're definitely up to some of the "fiddly" commands now. 4255Did you know "echo hello | sed p - -" segfaults gnu sed in Ubuntu 12.04? 4256Yeah...)</p> 4257 4258<p>Talked with the Tizen developers to follow up on their desire to 4259make toybox a part of the base Tizen system, and got a list of commands 4260to add to the roadmap. The tizen todo list is:</p> 4261 4262<blockquote><p> 4263wget, sha256*, gzip, gunzip, bunzip2, rsync, zdiff*, 4264less, ar, arch, base64, csplit, dir, fmt, join, 4265nproc, shred, shuf, stdbuf, stty, test, tr, unexpand, 4266users, vdir, diff3, sdiff, dosfsck (fsck.vfat), awk, fdisk 4267</p></blockquote> 4268 4269<p>(Most of which was already on the todo list, but it helps prioritize.)</p> 4270 4271<p>Fixed md5sum and sha1sum on big endian systems (reported by James McMechan). 4272Andy Lutomirski fixed unshare's help text and option parsing, 4273and submitted nsenter (a tool to use setns(2)) to pending. 4274Isaac Dunham implemented acpi -ctV options, and spotted the bug that ls -d 4275was inappropraitely following command line symlinks without -H or -L (it 4276should act like ls -l does), and ls -F handles symlinks wrong too. 4277Lukasz Szpakowski sent in two bugfixes to tail.c. Cynt Rynt spotted an 4278unnecessary assignment in lib/password.c.</p> 4279 4280<p>Ashwini Sharma's team was as busy as usual, submitting tr, crontab, and 4281ipcrm, and hwclock to pending, more features to the pending ip.c, and a 4282pile of bugfixes (to chgrp, killall, ifconfig, insmod, 4283losetup, comm, cp, id, xwrap, netcat, modprobe, nohup...) mostly found by 4284static analysis. (These fixes are mostly to seldom-used codepaths like the 4285TOYBOX_FREE config option, but test coverage is always appreciated.) Ashwini 4286also suggested upgrading ln -f to leave the original target alone if link 4287creation fails, and reported that mv -f and -i weren't implemented (now fixed).</p> 4288 4289<p>New config option: TOYBOX_NORECURSE prevents xexec() from making internal 4290function calls (for nommu systems with a finite stack).</p> 4291 4292<p>The "toybox" multiplexer command no longer adds a trailing space to each 4293line of command names, so things like "./toybox | tr ' \n' '|'" to create 4294a grep pattern snippet are easier to do. (Why you'd want to is your business, 4295but the output is tidier now.)</p> 4296 4297<h3>Infrastructure</h3> 4298 4299<p>Isaac Dunham added Android support to portability.h, including compile 4300probes for functions missing from bionic-libc, and annotated the commands that 4301use those functions. We haven't really tested building against bionic, 4302but in theory it's possible now.</p> 4303 4304<p>Running the test suite now color codes the PASS/SKIP/FAIL notifications 4305if output is to a tty. (And in case you missed it last time, VERBOSE=fail 4306to stop at the first failure is really useful.)</p> 4307 4308<p>In loopfiles_rw() use O_CLOEXEC instead of O_RDONLY to request the loop 4309function close filehandles for us. (Otherwise the callback function must 4310close each supplied filehandle itself.)</p> 4311 4312<p>The printf-style escape parsing ("\n" and friends) got factored out into 4313a new unescape() function.</p> 4314 4315<a name="02-10-2014" /><a href="#02-10-2014"><hr><h2><b>October 2, 2014</b></h2></a> 4316<blockquote><p>"There is an art, it says, or rather, a knack to flying. 4317The knack lies in learning how to throw yourself at the ground and miss... 4318Clearly, it is this second part, the missing, which presents the 4319difficulties."</p><p>- The Hitchhiker's Guide to the Galaxy<p></blockquote> 4320 4321<p><a href=downloads/toybox-0.5.0.tar.bz2>Toybox 0.5.0</a> 4322(<a href=/hg/toybox/shortlog/1512>commit 1512</a>) is out.</p> 4323 4324<h3>New commands</h3> 4325 4326<p>The new commands are find, install, factor, and mount. Promoted commands 4327(cleaned up and moved out of "pending") are lspci, inotifyd, and blockdev.</p> 4328 4329<p>cp now implements -HL and -F to force delete of pending files, cpio now 4330ignores -m and implements -p, ls -C now has utf8 support (using wcwidth 4331instead of strlen), and umount got a number of upgrades involving 4332looking things up in /proc/mounts. Other minor cleanups happend to 4333cut, touch, free, and id.</p> 4334 4335<p>In pending: Bradley Controy submitted mix (adjusts OSS sound volume). Ashwini 4336Sharma submitted diff, userdel, blockdev, ipcs, and crond, upgraded 4337fdisk, fsck, and ftpget, and ran a static analyzer on a lot of other code. 4338Partial cleanup was done to useradd, userdel, groupadd, and groupdel.</p> 4339 4340<h3>Build infrastructure</h3> 4341 4342<p><b>Parallel builds</b></p> 4343 4344<p>The build now takes advantage of SMP, autodetecting the number of 4345processors. (Export the environment variable CPUS to pick a specific number.) 4346Other build changes: split out $LDOPTIMIZE because old compilers complain 4347about linker options passed with -c, and the entire "generated" directory now 4348gets deleted by clean (the README that was in there got merged into code.html).</p> 4349 4350<p><b>Standalone builds</b></p> 4351 4352<p>The standalone build infrastructure (scripts/single.sh) got upgraded to 4353build more commands as standalone executables. In make.sh the source file 4354selection uses a regex to find the source files with the NEWTOY/OLDTOY macro 4355for the command. It enables each command's 4356sub-options (so CP has CP_MORE), enables I18N and FLOAT support to build 4357full-featured commands, and includes --help text (at least when 4358the command doesn't use another command's help). The OLDTOY() macro 4359now produces (redundant) function prototypes so you can build an OLDTOY 4360without the NEWTOY</p> 4361 4362<p>It doesn't quite have complete coverage yet, the defconfig entries that 4363aren't building standalone yet are:</p> 4364 4365<blockquote><p>chown, egrep, fgrep, fstype, halt, mv, nc, poweroff, unix2dos, 4366whoami</p></blockquote> 4367 4368<p>The main reason for standalone build failures is NEWTOY() or OLDTOY() 4369entries that don't have their own config symbol. Another problem is entries 4370that depend on another entry in kconfig, usually because common infrastructure 4371is using one command's flags (which the other commands copy): if that command 4372is disabled, the FLAG macros become 0 so dead code elimination can remove the 4373code. It's <a href=http://landley.net/hg/toybox/rev/1503>possible 4374to untangle</a> this, but a bit awkward. (It boils down to conflicting 4375design goals in the two contexts.)</p> 4376 4377<p>Standalone builds are used by the test suite when testing individual 4378commands. 4379 4380<p><b>Snapshot builds</b></p> 4381 4382<p>A new addition to the "generated" directory is generated/build.sh 4383containing a single compiler command line to build toybox in its current 4384configuration. Combined with the generated/*.{h,sh} files from an 4385exisiting build, this may let you build on a new system that hasn't quite 4386got enough OS bits working to run a full configureand make.</p> 4387 4388<h3>Internals</h3> 4389 4390<p>Library code: xcreate/xopen now O_CLOEXEC by default to avoid leaking 4391filehandles to child processes. DIRTREE_COMEAGAIN's second callback is now 4392done with the directory filehandle still open (new dir->again variable added 4393to distinguish first from second callback, and requesting DIRTREE_RECURSE now 4394requires passing in the specific macro value, not just a true/false). 4395Use daemon() out of libc instead of hand-rolled daemonize() in various 4396pending commands. string_to_mode() now passes through type bits so you can 4397use it to more easily modify a file's existing mode. 4398Split xpopen() into xpopen_both(), xopen(), and xrun() depending on whether 4399we want to redirect both, one, or neither of stdin/stdout.</p> 4400 4401<p>Bugfixes: Better error message when TOYBOX_SUID option can't drop priviliges 4402(which happens when you suid something _other_ than root). 4403The old pending version of nbd_client.c wasn't deleted when the 4404command was promoted (and the build would break if both were enabled), 4405toy_exec() sometimes needs to re-exec from $PATH rather than recurse 4406internally (to gain dropped root permissions or limit stack depth), 4407always call setlocale() when I18N is enabled to switch it back _off_ when 4408we run commands that expect sscanf("%n") to return bytes, 4409dirtree() had a memory leak in an error path, patch.c had some bugs in 4410error paths (didn't report problem clearly). Ashwini Sharma spotted an 4411option parsing bug where [-abc] would forget _all_ command line arguments 4412saved in the GLOBALS() block (not just the ones for options being switched 4413off), plus various minor fixes to nbd_client and cpio. 4414Lukasz Szpakowski fixed rm -f on a broken symlink (failed), and killall 4415with no arguments (segfaulted).</p> 4416 4417<p><b>Portability</b></p> 4418 4419<p>A somewhat fiddly fix to rm -rf (which needs to chmod directories to u+rwx 4420to descend into them) which hit a musl bug in faccessat() which the musl 4421maintainer refuses to fix. (He literally wants the man page changed 4422instead, despite other libcs working.) Added an #ifdef __MUSL__ section 4423to portability.h with a workaround, you may need CFLAGS=-D__MUSL__ in your 4424build if your musl build's features.h doesn't #define that. (I may do 4425a different workaround in future, but sometimes you've just got to make 4426it work so you can ship. Also, toybox grep with multiple patterns 4427requires <a href=http://landley.net/hg/aboriginal/rev/1692>a patch 4428to musl's regex engine</a>, which applies to 1.1.4 but not to the current 4429musl source control.)</p> 4430 4431<p>More portability.h fixes for uClibc too. (I don't expect that to ever have 4432another release, so locally patching around posix-2008 violations is silly).</p> 4433 4434<p><b>Change to username filtering</b></p> 4435 4436<p>Posix recommends the username creation logic filter usernames to a small 4437allowed set of characters (which even Red Hat breaks by explicitly allowing 4438"$" at the end), but this prevents UTF-8 usernames. Posix' stated logic 4439is to allow filesystems to create the user's home directory, but Linux 4440filesystems can accept any character but NUL and "/". The only characters 4441we actually _need_ to filter out are ":" (field separator in passwd), 4442newline (line separator in passwd), and "/" (directory separator in 4443filesystem).</p> 4444 4445<h3>Documentation</h3> 4446 4447<p>Web pages updated: cleanup.html documents more cleanup, code.html 4448documents more code, and about.html now capitalizes "toybox" consistently 4449(it's just a word, capitalize at start of sentence).</p> 4450 4451<p>The pending/README file now lists commands that needed review/cleanup 4452before the pending directory was added.</p> 4453 4454<h3>Test Suite</h3> 4455 4456<p>Moved out of scripts/test into top level "tests" directory, and the 4457testing.sh script is now in scripts rather than mixed into the *.test files.</p> 4458 4459<p>Johan Bergström requested VERBOSE=fail to make tests (telling it to 4460stop at the first failure), and spotted a build bug where using gnu 4461sort on the host broke in non-C locales.</p> 4462 4463<p>Divya Kothari submitted tests for chmod, link, tar, bzcat, xzcat, zcat, 4464and hostname. (And more, but that's all that's merged so far.)</p> 4465 4466<a name="07-07-2014" /><a href="#07-07-2014"><hr><h2><b>July 7, 2014</b></h2></a> 4467<blockquote><p>"This planet has - or rather had - a problem, which was this: 4468most of the people living on it were unhappy for pretty much of the time. Many 4469solutions were suggested for this problem, but most of these were largely 4470concerned with the movement of small green pieces of paper, which was odd 4471because on the whole it wasn't the small green pieces of paper that were 4472unhappy."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 4473 4474<p><a href=downloads/toybox-0.4.9.tar.bz2>Toybox 0.4.9</a> (<a href=/hg/toybox/shortlog/1385>commit 1385</a>) is out.</p> 4475 4476<p><b>New commands</b> added to pending include: 4477lsattr, chattr, inotifyd, rfkill, sulogin, strings, makedevs, 4478killall5, and tar from Ashwini Sharma, arp from Kyungwan Han, 4479sysctl by Bilal Qureshi, partprobe from Bertold Van den Bergh, 4480host from Rich felker, and I did nbd-client and the first 2/3 of mount.</p> 4481 4482<p>Finished cleanups (commands promoted out of pending): 4483sysctl, rfkill, strings, mkpasswd, makedevs, partprobe, killall5, 4484fallocate, and nbd-client.</p> 4485 4486<p>(Along the way partial cleanups got made to: last, fold, lspci, ps, 4487bootchartd, init, fsck, telnetd, telnet, vconfig, toysh, iconv, useradd, 4488login, host, openvt, deallocvt, getty, tftpd, and modprobe. But there's 4489still more to do on all of those.)</p> 4490 4491<p>This time around the <a href=bin>static binaries</a> are linked against 4492musl instead of uClibc. (That's why there's no sparc version, musl doesn't 4493support that target yet.)</p> 4494 4495<p><b>Documentation:</b></p> 4496 4497<p>The help text parser expects lower case "usage:" lines with 4498a blank line after them, so go through and regularize those. Expand the 4499"coding style" section in the docs and move it to design.html. (Not a show 4500stopper for incoming 4501contributions, just an explanation of some of the things I'll do to them 4502during cleanup.) The help text for the "toybox" command now includes 4503the shell script snippet to install symlinks to the toybox binary.</p> 4504 4505<p>The <a href=cleanup.html>cleanup page</a> now has descriptions for the 4506full ifconfig cleanup series, among others.</p> 4507 4508<p>The new toys/examples directory contains hello.c and skeleton.c. The first is 4509a simple hello world program in toybox style, the second is a much more 4510elaborate example program using showing how to use the command line option 4511parsing and how to provide multiple commands in the same C file.</p> 4512 4513<p><b>Fixes</b>:</p> 4514 4515<p>Fix od bug reported by Samuel Holland ("od -v -b" was appending the default 4516output type even though an output type was specified). Ashwini Sharma reported 4517bugs where readfile() was incorrectly freeing its buffer, and where toy_init() 4518was zeroing the wrong data because the field it was using to measure (rebound) 4519had moved (when I moved it back I added a comment why the field needs to be 4520there), fixed a segfault in the dhcp client, and made a 0 length read at 4521the start of password entry count as EOF. Make the "we are not root" test 4522in the init code show the help text. Posix implies that fflush() can return 4523success even when the stream's error bit is set, so call both fflush() and 4524ferror() from xprintf().</p> 4525 4526<p>Isaac Dunham pointed out that bloatcheck couldn't deal with diff 4527implementations that only implement "unified diff" format, and that some 4528diff implementations can't handle nonseekable input (I.E. reading from 4529a pipe). Bugfix so "help -a" works again. Option parsing on nohup now stops 4530at first nonoption argument. Fix segfault in "which" if PATH wasn't set, 4531which was actually a bug in lib function find_in_path(). Made rm -rf of 4532chmod 000 directories actually remove them.</p> 4533 4534<p>The build now passes the same $CFLAGS to the library probe as the final 4535build, because arch linux is so broken it provides different sets of 4536libraries for static and dynamic linking.</p> 4537 4538<p>It turns out sprintf("%.123s", str) is counting characters, not bytes, 4539so globally enabling locale support opens stack smashing vulnerabilities. 4540So there's a new TOYFLAGS_LOCALE you set in toyflags when you want the 4541setup code to setlocale().</p> 4542 4543<p><b>Upgrades:</b></p> 4544 4545<p>Isaac Dunham extended cpio to archive unreadable empty files, and I taught it 4546to set uid/gid and timestamp when extracting archives. Isaac also 4547added tests for cpio, link, and du, added lspci -i, made the pci database 4548parsing skip # comment lines, merged logname and whoami into id.</p> 4549 4550<p>Daniel Verkamp sped up md5sum about 30% with some loop unrolling, making 4551it actually smaller in the process. I added -b flags to md5sum and sha1sum 4552for "brief" output that's just the hash with no filename. (I'm aware other 4553implementations use that for MSDOS "binary" mode, and don't care.)</p> 4554 4555<p>When building standalone commands (scripts/singleconfig.sh commandname), 4556the build now switches on all the sub-options of the command so we get 4557a standalone version with all the bells and whistles enabled.</p> 4558 4559<p>Add -ds flags to date and document +FORMAT escapes. Add the shell NOP 4560command ":" as an alias for true (for toysh).</p> 4561 4562<p>Add uClibc probe for iconv() and fallocate. (The fact it didn't always 4563build against uClibc is why fallocate wasn't enabled in defconfig before.)</p> 4564 4565<p>The umount command now does an losetup -d on the device by default, so 4566we don't leak loopback devices. Bugfix to losetup so "losetup /dev/loop0 4567filename" actually works again.</p> 4568 4569<p>Divya Kothari sent in test suite entries for ls, ln, rm, mv, printf, dd, 4570and renice. Then a second round for lsattr/chattr, mount, chmod, pgrep/pkill, 4571groupadd, groupdel, and useradd. Several of these uncovered bugs, still 4572working to fix them.</p> 4573 4574<p>There are now free() functions for the predefined llist types and a 4575dlist_terminate() function to break doubly linked lists. The new 4576generic_signal() handler either sets "toys.signal" or writes a byte 4577to toys.signalfd with the signal number if signalfd isn't -1 (which it's 4578initialized to in toy_init).</p> 4579 4580<p>The option parsing logic can now detect when a double fits in a long and 4581use the more precise type for floating point arguments (the FLOAT macro 4582contains the type used). The human_readable() function now just outputs 4583decimal kilo/mega/gigabytes (so when du -u says 5.0G it means 5.0 billion 4584bytes). The build infrastructure now notices duplicate commands (so if you 4585cp toys/pending/command.c toys/other/command.c and forget to delete the 4586first one, the build break is now more informative).</p> 4587 4588<a name="20-04-2014" /><a href="#20-04-2014"><hr><h2><b>April 20, 2014</b></h2></a> 4589<blockquote><p>And to this end they built themselves a stupendous supercomputer 4590which was so amazingly intelligent that even before the data banks 4591had been connected up it had started from "I think therefore I am" and got as 4592far as the existence of rice pudding and income tax before anyone managed to 4593turn it off.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 4594 4595<p><a href=downloads/toybox-0.4.8.tar.bz2>Toybox 0.4.8</a> is based on 4596<a href=http://landley.net/hg/toybox/shortlog/1262>commit 1262</a>. And 4597about time too.</p> 4598 4599<p>The big news is that the build no longer needs python to generate help.h, 4600that's now done in C. The help text generation is also collating help text 4601from multiple options, merging command line option blocks and usage: lines. 4602There's even a new <a href=help.html>help web page</a>.</p> 4603 4604<p><b>New commands:</b> Ifconfig, cpio, and su were cleaned up the rest of the 4605way and promoted out of pending. That saga is mostly explained on the 4606<a href=cleanup.html>cleanup page</a>. Vivek Bhagat's freeramdisk, 4607Isaac Dunham's fsfreeze, and Felix Janda's iconv are also new.</p> 4608 4609<p><b>In pending:</b> 4610Ashwini Sharma's team submitted tcpsvd, udpsvd, telnet, telnetd, last, more, 4611groupdel/delgroup, arping, brctl, ftpget, ftpput, printf, reset, and added 4612ipv6 support to traceroute. Kyungwan Han's team submitted modprobe and getty. 4613Vivek Bhagat submitted openvt and deacllocvt. Samuel Holland submitted fold. 4614I wrote a new inflate (zip/zlib/gzip decompression) implementation in 4615compress.c, and still need to do a corresponding deflate (compression-side) 4616and plug them into gzip and zip and so on. (Right now it does zcat.)</p> 4617 4618<p>Several commands (vmstat, login, du, vconfig, mountpoint, free, chroot, 4619cut, touch, modinfo, expand) predate the "pending" directory, and are thus 4620in other directories but still need cleanup. Of these, vmstat got some 4621work this time (which would be much easier other vmstat implementations 4622documented what their output actually meant).</p> 4623 4624<p><b>Upgrades:</b> Ifconfig grew /prefix netmask support (ala 1.2.3.4/24). Grep now has -zZ to 4625handle null terminated data, cksum grew -H for hex output. Upgraded od so the 4626fields align better when producing multiple output types. Help has -a and -h 4627options (all commands, html output). 4628Bugfix to blkid building for a 32 bit target. The date command can actually 4629set dates now. The O_NOFOLLOW compile time probe didn't work with cross 4630compiling, so it's back to an #ifdef test in portability.h. Nathan McSween 4631sent in a bugfix to od and a portability fix in the common library code. 4632Ashwini Sharma spotted a bug in pidof -o, and added verbose (-v) options 4633to mkdir and ln, and suggested killall should have an -s option and 4634allow -l to take zero arguments. Ashwini Sharma and Felix Janda upgraded 4635tftpd. Fixed dumpleases still using toynet.h after 4636that was removed. Corrected killall return code and error reporting. 4637Isacc Dunham fixed bugs all over the tree, did cleanup on a bunch of 4638pending commands (getty, ftpget, init, openvt, modprobe...), and clarified 4639find's help text. Tom Sparrow ran three different static analyzers on 4640the code, which resulted in a few cleanups. The peek()/poke() functions 4641now use "volatile" to prevent broken compiler "optimizations" to do with 4642aliasing.</p> 4643 4644<p><b>Build stuff:</b> Each FOR_xxxx macro now has a complementary CLEANUP_xxxx macro, so you 4645can put multiple commands with different command line options in the same 4646.c file, so they can share infrastructure outside of lib. (This let the 4647bunzip logic move out of lib into bzcat.c.) See XXX for example. 4648i 4649<p>The headers #included in toys.h are now grouped by standard, and headers 4650not listed in Posix or LSB were moved to portability.h. The old xregcomp.h 4651was folded into lib.h because it's posix (and supporting oddball uClibc 4652configurations isn't as important as it once was).</p> 4653 4654<p>Regression tested against Ubuntu 8.04 to fix up bit-rot in defconfig 4655build on older systems. (We depend on Posix-2008, but not necessarily 4656the absolute latest build environment.)</p> 4657 4658<p><b>In lib</b>: lib/xwrap.c added xgetpwnam(), xchroot(), and lib/lib.c now has names_to_pid(). 4659xsetuid() was replaced with xsetuser() which takes a struct passwd 4660and sets both gid and uid, mkpathat() got factored out into a library command, 4661get_int_value() became atolx_range(), and 4662xmsprintf() is now just xmprintf(). The bunzip2 logic moved from lib into 4663bzcat.c.</p> 4664 4665<p><b>Documentation</b>: new <a href=help.html>help page</a> with the 4666help text for all the defconfig commands, using the new help -ah output. 4667The <a href=code.html>source code walkthrough</a> now says more about 4668#including header files, and how the generated/* directory works. The 4669<a href=design.html>design page</a> has some new paragraphs about trading 4670of different kinds of simplicity, and why comments aren't a substitute for 4671good code. The README no longer trails off into obvious unfinished confusion 4672at the end. Each page on the website should now have its own title.</p> 4673 4674<a name="18-11-2013" /><a href="#18-11-2013"><hr><h2><b>November 18, 2013</b></h2></a> 4675<blockquote><p>"Space," it says, "is big. Really big. You just won't believe how vastly, hugely, mindbogglingly big it is. I mean, you may think it's a long way down the street to the chemist's, but that's just peanuts to space."</p><p>- 4676The Hitchhiker's Guide to the Galaxy</p></blockquote> 4677 4678<p><a href=downloads/toybox-0.4.7.tar.bz2>Toybox 0.4.7</a> is based on 4679<a href=http://landley.net/hg/toybox/shortlog/1122>commit 1122</a>.</p> 4680 4681<p>New commands: Brad Conroy submitted blkid. Elie De Brauwer submitted 4682reboot, halt, and poweroff. Strake's nl got cleaned up and promoted from 4683pending to posix. In addition, the existing chvt and vconfig got some 4684cleanup.</p> 4685 4686<p>That said, I haven't nearly kept up with the flood of new commands going 4687into pending: Ashwini Sharma's team submitted 4688dd, dumpleases, traceroute, top, useradd, groupadd, mkpasswd, tftpd, and 4689an fsck wrapper (with no filesystem drivers yet). Isaac Dunham sent in cpio.</p> 4690 4691<p>Bugfixes: Jeroen van Rijn added a user count to uptime. Elie De Brauwer 4692added -e to watch, removed a memory leak, and fixed a terminal size problem. 4693William Haddon made xargs call its command line once even with blank input 4694(the standard is vague, but builds expect it), and fixed an off by one bug 4695where grep didn't malloc enough space with -E (leading to a segfault). 4696I fixed a glitch in bunzip2 (same one as went into busybox since they're using 4697the code I wrote), in od to fix -t co, -J, and -c options. Add uname -o as a 4698synonym for -s. Build fix to never use $CC without prefixing it with 4699$CROSS_COMPILE (since $HOSTCC could be different). Anca Emanuel spotted 4700a typo in the web page.</p> 4701 4702<p>The compile-time command line option parsing got rewritten (ported from 4703bash to C), which should speed up builds a bit and allow code controlled by 4704--longopts to drop out properly when disabled in the configuration. Terminal 4705querying got refactored. Patch's -x option is now more informative (a 4706debug thing if you're trying to figure out why a patch didn't apply). 4707The "toynet.h" file got folded into toys.h since musl supports it and 4708micromanging uClibc options isn't very interesting anymore. The test suite 4709now uses scripts/single.sh when testing a single command.</p> 4710 4711<a name="17-09-2013" /><a href="#17-09-2013"><hr><h2><b>September 17, 2013</b></h2></a> 4712<blockquote><p>"Think of a number," said the computer, "any number." 4713Arthur told the computer the telephone number of King's Cross railway 4714station passenger inquiries, on the grounds that it must have some function, 4715and this might turn out to be it.</p><p>- The Hitchhiker's Guide to the Galaxy</p> 4716</blockquote> 4717 4718<p><a href=downloads/toybox-0.4.6.tar.bz2>Toybox 0.4.6</a> is based on 4719<a href=http://landley.net/hg/toybox/shortlog/1068>commit 1068</a>.</p> 4720 4721<p>This release adds 4722several new commands: Felix Janda wrote paste and fallocate, Kyungwan Han 4723submitted eject, Strake contributed grep, Ashwini Sharma added pmap (and 4724a testsuite entry for grep), Lukasz Skalski sent pwdx, Isaac Dunham posted 4725acpi, and I did timeout and umount.</p> 4726 4727<p>The ls command now has a --color=auto option (suggested by Rich Felker). 4728The multiplexer now has a --help option so you can say "./toybox --help blah" 4729instead of using the built-in "help" command. (Which is a shell built-in. 4730Try it on your command line, it's like man for shell builtins. But a certain 4731other project has conditioned people to expect --help, so...) I forget who 4732heehooman at gmail is but they pointed out unshare needed PID and UID 4733namespace support.</p> 4734 4735<h3>Pending</h3> 4736 4737<p>A lot of new commands in toys/pending, to the point the next release should 4738probably just focus on cleanup and review of this backlog. We've got klogd, 4739dhcp, dhcpd, watch, route, and ps from 4740Ashwini Sharma (and an fsck wrapper but no fsck.fstype engines yet), 4741syslogd, pgrep, and pkill from Madhur Verma, netstat by Ranjan Kumar, 4742test by Felix Janda, lspci by Isaac Dunham, nl, su, and renice by strake (I.E. 4743M. Farkas-Dyck), and sysvinit by Kyungwan Han.</p> 4744 4745<p>Some cleanup work on existing pending commands that aren't 4746ready to promote yet: I did a few more rounds on ifconfig 4747and Isaac Dunham's did several cleanups to xzcat, Felix Janda cleaned up 4748logger and syslogd...</p> 4749 4750<p>Also some cleanup work on commands that predate the pending directory, 4751but weren't quite polished when they went in, most prominently du, 4752expand, and touch.</p> 4753 4754<h3>Infrastructure</h3> 4755 4756<p>The new scripts/single.sh builds a standalone command without the 4757multiplexer, although not all commands can be built that way yet (NEWTOY yes, 4758OLDTOY no) and the space savings aren't anything to write home about. (If a 4759command needs the option parsing logic at all, it needs all of it.) If 4760you're curious, you can do:</p> 4761 4762<blockquote><pre> 4763make defconfig 4764make 4765mkdir singles 4766for i in $(./toybox) 4767do 4768 echo $i 4769 PREFIX=singles/ scripts/single.sh $i || break 4770done 4771</pre> 4772<p>(And then wait a long time and watch almost half the builds fail.)</p> 4773</blockquote> 4774 4775<p>There is now libbuf analogous to toybuf, another global 4k buffer this 4776time for use by lib/ code instead of command code.</p> 4777 4778<p>The lib directory got split up a bit, lib/pending.c contains functions 4779not yet used by anything outside of toys/pending/*, and lib/xwrap.c contains 4780functions that wrap other functions and handle failures (via error_exit). 4781This leaves lib/lib.c containing actual new functions.</p> 4782 4783<p>General improvements and bug fixes to argument parsing. The [-abc] exclude 4784logic should now clear arguments slots when disabling options. Bare --longopts 4785should work now and be able to report errors using their name, the new ; 4786option allows optional arguments to longopts only suppliable with = (I.E. 4787--color and --color=auto but not --color auto).</p> 4788 4789<p>I'm gradually weaning the code off of itoa()/utoa() because sprintf 4790does this already. In this case "simple" probably means "let libc do it 4791for us".</p> 4792 4793<p>Rewrote for_each_pid_with_name_in() and renamed it to just names_to_pid(). 4794It shouldn't get confused trying to compare absolute and relative paths quite 4795so much anymore.</p> 4796 4797<p>lib/llist.c grew a new dlist_pop() function for removing a doubly 4798linked list entry while maintaining a circular list; tail and patch are 4799using it now.</p> 4800 4801<p>The musl guys suggested a new optimization flag 4802(-fno-asynchronous-unwind-tables) 4803that shaves about 10% off the binary size by removing a C++ism that crept 4804into gcc's idea of C. While I don't normally try to micromanage the compiler, 4805"-fstop-being-stupid" is a thing you have to hit gcc with from time to time.</p> 4806 4807<h3>Bugfixes</h3> 4808 4809<p>Felix Janda and I did a largeish rewrite of tail to 4810finally make it work right (we think). Still need to implement tail -f 4811someday (the tricky bit is making -f follow multiple files at once). 4812Felix also reported a bug in xpidfile.</p> 4813 4814<p>Juhani Haverinen pointed out that 4815python 3 doesn't work with config2help.py, so the detection logic looks 4816for python2 (until I get around to rewriting that in C). Elie De Brauwer 4817then fixed our first attempt at this, and also fixed uname's help string.</p> 4818 4819<p>Ashwini Sharma 4820pointed out the build was making a FLAG_ macro for " " which broke some 4821configurations. (That's a control character, not a command line option.)</p> 4822 4823<p>Jacek Bukarewicz pointed out a bug in chdir permission handling, and 4824a way to make env segfault. Both should be fixed now.</p> 4825 4826<p>The new function xexec_optargs() 4827replaces calls to xexec(toys.optargs) to avoid freeing and reusing optargs 4828during option parsing screwing stuff up (such as netcat's exec mode).</p> 4829 4830<p>The stat command's %a output was padded with leading zeroes, which 4831didn't match anybody else's behavior and thus made the test suite hiccup 4832between TEST_HOST and testing toybox. (If you go "TEST_HOST=1 scripts/test.sh 4833command" it sanity checks the tests against the host implementation.)</p> 4834 4835<p>Last release, "mkdir sub/sub && chmod 007 sub/sub && rm -rf sub" didn't 4836delete sub and didn't exit with an error either. Neither was correct, rm 4837should now be fixed.</p> 4838 4839<p> 4840<a name="26-07-2013" /><a href="#26-07-2013"><hr><h2><b>July 26, 2013</b></h2></a> 4841<p>Georgi Chorbadzhiyski maintains a <a href=https://github.com/gfto/toybox>git 4842mirror</a> of the repository on github, automatically updated from the 4843mercurial every 6 hours. The mirror is read only, but you can generate patches 4844against it and post them to the list.</p> 4845 4846<a name="02-07-2013" /><a href="#02-07-2013"><hr><h2><b>July 2, 2013</b></h2></a> 4847<blockquote><p>"Time is an illusion. Lunchtime doubly so." "Very deep. You 4848should send that in to the Reader's Digest. They've got a page for people 4849like you."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 4850 4851<p><a href=downloads/toybox-0.4.5.tar.bz2>Toybox 0.4.5</a> is based on 4852<a href=http://landley.net/hg/toybox/shortlog/941>commit 941</a>. It adds 4853uuencode and uudecode from Erich Plondke, and enables Luis Morales' "who" by 4854default. Felix Janda and I cleaned up last year's "stat" submission and 4855enabled it. Ivo van Poorten added "groups". 4856Andre Renaud added "lsusb". I implemented "split", "pivot_root", and "mv". 4857</p> 4858 4859<p>The "help" command is implemented differently now (lib/help.c) and 4860each command can now understand --help (including both "toybox --help" 4861and "toybox --help command" in the multiplexer).</p> 4862 4863<p>The "pending" directory has several commands (find, xzcat, nbd-client, 4864logger, expr) which work but are not enabled by default pending further cleanup. 4865Ifconfig is enabled, but still in pending because it's only 2/3 cleaned up. 4866(It's an awkward halfway state but I'm not holding up the release for it.)</p> 4867 4868<p>I'm <a href=cleanup.html>documenting the cleanups</a> to teach 4869more people to do it, but the writeups aren't caught up yet. The 4870<a href=roadmap.html>roadmap</a> also got updated a bit with further analysis 4871of other projects, and the README and about pages got updated.</p> 4872 4873<p>Fixed _another_ "ls -C" segfault when terminal size can't be detected, 4874condensed the ls help text to fit on one page, implented --color, and taught 4875-l to print the major, minor numbers when showing block/char devices. 4876Argument parsing now handles "--" properly (to end option checking), 4877and the infrastructure can now handle bare --longopts that have no 4878corresponding short option (both were implemented before but didn't work). 4879Fixed an old bug in "patch", chmod grew -f, who grew -a. Isaac Dunham 4880fixed "-" vs "_" handling in modinfo, added a "firmware" output 4881field, added -b and -k support, and taught it that the ".ko" extension means 4882to look for the file at the specified path instead of under /lib. Felix Janda 4883moved file permission display code to lib so ls and 4884stat could share it. Ashwini Sharma spotted a bug in xabspath when the 4885last path component exists but we haven't got permissions to open it 4886(ala readlink -f /dev/sda as a normal user). 4887</p> 4888 4889<p>In the build infrastructure, scripts/findglobals.sh finds leaked global 4890variables. (Leaked means they aren't part of the global union: Other than glibc 4891debris, toybox should define "this", "toy_list", "toybuf", and "toys", and 4892that's it; the rest add memory footprint to every command for the benefit of 4893just one command; use GLOBALS() to stick 'em in the union.) Static linking 4894against libraries other than the host's libc now applies to feature probes 4895for unshare and such. Neuter stupid internationalization support that makes 4896various host "sort" commands put things in an order other than alphabetical 4897(breaking the multiplexer's binary search on command names). 4898 4899<p>You should now be able to build from a source control snapshot on a build 4900system that hasn't got python: if you disable CONFIG_TOYBOX_HELP. (The 4901release tarballs ship generated/help.h, but it's not in source control. 4902Eventually I should rewrite that python script in C.)</p> 4903</p> 4904 4905<p><b>LICENSE TWEAK</b>: After <a href=http://lists.landley.net/pipermail/toybox-landley.net/2013-March/000794.html>discussion</a> on the mailing list the "2 clause 4906BSD" <a href=license.html>license</a> got slightly simplified so the first 4907paragraph now says:</p> 4908 4909<blockquote><p>Permission to use, copy, modify, and/or distribute this 4910software for any purpose with or without fee is hereby granted.</p></blockquote> 4911 4912<p>It used to continue "provided that the above copyright notice and this 4913permission notice appear in all copies", but A) what's the point? B) does "all 4914copies" mean binaries, or just source code, or what? C) lots of projects 4915that consider BSD and GPL compatible have <a href=https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/crypto/aes_generic.c>files with 4916both license notices</a> on them (sometimes at <a href=http://git.busybox.net/busybox/tree/shell/ash.c>opposite ends of the file</a> to make the conflict 4917less obvious) because "all copies must include this function" would violate 4918the GPL but "all copies must include this magic text blob" somehow don't?</p> 4919 4920<p>I don't want to have to care about this anymore. The tweaked version is more 4921or less public domain with a liability disclaimer, but we're still calling it 4922BSD (sometimes "0 clause BSD") to avoid explaining.</p> 4923 4924<a name="21-03-2013" /><a href="#21-03-2013"><hr><h2><b>March 21, 2013</b></h2></a> 4925<p>Video of my ELC talk 4926"<a href=http://youtu.be/SGmtP5Lg_t0>Why is Toybox?</a>" 4927is up on youtube. Related materials include the 4928<a href=http://landley.net/talks/celf-2013.txt>talk outline</a> and an 4929<a href=/aboriginal/about.html#selfhost>android self-hosting writeup</a>.</p> 4930 4931<p>[Updated June 4] The following links jump to specific topics in the video. (Sorry about 4932the ads, it's The Linux Foundation.)</p> 4933 4934<ul> 4935<li>0m29s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=0m29s>The smartphone is replacing the PC</a></li> 4936 <ul> 4937 <li>4m22s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=4m22s>Software needed to become self-hosting</a></li> 4938 <li>6m20s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=6m20s>Do we care if android or iphone wins?</a></li> 4939 </ul> 4940<li>9m45s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=9m45s>Android not vanilla: oppose or accept?</a></li> 4941 <ul> 4942 <li>11m30s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=11m30s>Open source can't do User Interfaces</a></li> 4943 </ul> 4944<li>15m09s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=15m09s>Android is not copyleft: oppose or accept?</a></li> 4945<li>18m23s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=18m23s>Security issues</a></li> 4946<li>21m15s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=21m15s>Solutions to the software problems</a></li> 4947 <ul> 4948 <li>22m55s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=22m55s>What toybox needs to be/do</a></li> 4949 <li>28m17s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m17s>What is toybox?</a></li> 4950 <ul> 4951 <li>28m58s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=28m58s>Why toybox started...</a></li> 4952 <li>37m50s <a href=http://www.youtube.com/watch?v=SGmtP5Lg_t0#t=37m50s>What does toybox actually implement?</a></li> 4953 </ul> 4954 </ul> 4955</ul> 4956</span> 4957 4958 4959<a name="14-03-2013" /><a href="#14-03-2013"><hr><h2><b>March 14, 2013</b></h2></a> 4960<blockquote><p>"Ford, you're turning into a penguin. Stop it." - 4961</p><p>The Hitchhiker's Guide to the Galaxy</p></blockquote> 4962 4963<p><a href=downloads/toybox-0.4.4.tar.bz2>Toybox 0.4.4</a> is based on 4964<a href=http://landley.net/hg/toybox/shortlog/813>commit 813</a>, adding 4965the "time" and "readahead" commands, plus some bugfixes.</p> 4966 4967<p>The "cp" command now implements the -s symlink option, plus bugfixes 4968getting various corner cases right as used in actual package builds. 4969"id -Gn root" should now print root's groups 4970instead of the current user's. Several build fixes so toybox builds under 4971Ubuntu 8.04 again (which is about as old a build environment as you 4972can expect to find posix-2008 features in).</p> 4973 4974<p>Unfinished commands have generally been moved to "toys/pending". 4975Everything else should "default y" to participate in make defconfig. 4976Several of those pending commands got some basic cleanup so allyesconfig 4977should at least compile (although defconfig is still what's useful).</p> 4978 4979<p>Significant roadmap updates, checking several other multicall binaries 4980(klibc, sash, sbase, s6...) to see what commands they include.</p> 4981 4982<a name="18-01-2013" /><a href="#18-01-2013"><hr><h2><b>January 18, 2013</b></h2></a> 4983<blockquote><p>This must be Thursday. I never could get the hang of Thursdays.</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 4984 4985<p><a href=downloads/toybox-0.4.3.tar.bz2>Toybox 0.4.3</a> is based on 4986<a href=http://landley.net/hg/toybox/shortlog/793>commit 793</a>. There 4987are now exactly 100 commands in defconfig (of a little over 220 on the 4988<a href=roadmap.html>todo list</a>).</p> 4989 4990<p>Elie De Brauwer added the rev command, cleaned up tac, implemented the -s 4991and -f flags for seq, added -v and -i to killall (and fixed killall not to 4992kill itself before finishing its pid list), and added to the test suite. 4993Felix Janda added -m to mkdir, pwd -L and -P, and more test suite entries.</p> 4994 4995<p>Rob Landley added the losetup command, and fixed the existing ls, cp, and 4996readlink commands. The segfault in ls 4997happened when it couldn't determine the screen size (last release changed the 4998default to -C and a screen size of 0 made column view unhappy), and cp got an 4999extensive rewrite bringing it up to date with the dirtree changes and fixing 5000a number of things it never did right in the first place. The xabspath()
5001code in the library now handles a symlink after ".." properly (and the test 5002suite checks for it).</p> 5003 5004<p>Infrastructure-wise the code is better about automatically setting the 5005error return code properly. Now error_msg() sets the exit code to 1 if it's 5006still defaulting to 0, and the global exit path does a fflush(NULL) with error 5007bit check rather than trying to be quite so granular about flushing. (That 5008means if we use printf() instead of xprintf() it still exits with the right 5009error code, it just doesn't end the program early on an output error.) 5010Minor bugfix so TOYBOX_DEBUG 5011doesn't always warn about the lack of suid bit when toybox is built with 5012at least one STAYROOT command. Bugfix for the option [grouping] logic 5013(and then further fixes to the error reporting pointed out by Ashwini Sharma). 5014dirtree_handle_callback() now has a prefix like the rest of the dirtree 5015functions. A lot of stuff doing manual path handling was switched to using 5016libc basename() (including, embarassingly, the basename command), which means 5017it now correctly detects "/trailing/slash/" which the previous code didn't.</p> 5018 5019<p>Also, last release included some accidentally checked in debug code that 5020disabled compiler optimization, so the binary size bloated a bit. It's back 5021to -Os by default now.</p> 5022 5023<a name="15-12-2012" /><a href="#15-12-2012"><hr><h2><b>December 15, 2012</b></h2></a> 5024<blockquote><p>"The major difference between a thing that might go wrong and a 5025thing that cannot possibly go wrong is that when a thing that cannot possibly 5026go wrong goes wrong it usually turns out to be impossible to get at or repair." 5027</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 5028 5029<p><a href=downloads/toybox-0.4.2.tar.bz2>Toybox 0.4.2</a> is based on 5030<a href=http://landley.net/hg/toybox/shortlog/749>commit 749</a> and is 5031just a resync. Linux 3.7 came out, meaning it's time to do an Aboriginal 5032Linux release, and that should use a stable version of toybox. So here's 5033a new stable version.</p> 5034 5035<p>The new commands are cut (from Jason Kyungwan Han), touch 5036(from Choubey Ji), expand (from Jonathan Clairembault, and he fixed a 5037bug in login), and rm (from Rob Landley). Felix Janda added UTF-8 5038support infrastructure (for non-ascii character sets) with a config option. 5039Elie De Brauwer added tests for cat and sha1sum, and -so options to pidof. 5040The "ls" command defaults to -C (column view) now, and "readlink" now supports 5041-fenq.</p> 5042 5043<p>Portability work: toybox should now build against the musl C library, 5044and against older glibc versions (circa 2008, much before that and kernel 5045features we depend on start to drop out).</p> 5046 5047<p>The whole codebase got reindented from "one tab" to "two spaces" per 5048level. The option parsing logic now understands [groups] of commands (when more 5049than one in a group is selected it can switch the others off, or error out, 5050or other things). The error_exit() infrastructure can now longjmp back to an 5051earlier point instead of exiting. Each toys/* directory now has a README, 5052the first line of which is the fancy name menuconfig uses for the directory 5053(so no more hardwired directory list in scripts/genconfig.sh).</p> 5054 5055<p>Fixed a filehandle leak in getmountlist(). 5056Pass parent pointer to dirtree_add_node() so it can give error messages with 5057full path. The yesno() function now always reads from stdin and writes to 5058stderr (we can retry tty checking complexity once we've got commands needing 5059it).</p> 5060 5061<p>The open group broke their website so the 5062<a href=http://opengroup.org/onlinepubs/9699919799>old links</a> to POSIX 2008 5063now <a href=http://pubs.opengroup.org/onlinepubs/9699919799>need to start with 5064pubs</a>. Some of the links in the tree have been updated, others haven't while 5065I wait to see if their webmaster notices and fixes it.</p> 5066 5067<p>(I note that the current rm implementation is not technically posix compliant 5068because the standard requires infinite recursion depth and the current 5069implementation uses one filehandle per level. I can add a config option 5070to do it Posix's way, which is more brittle and needs extra security checks, 5071but am waiting for somebody to complain first. The default "ulimit -n" is 1024 5072filehandles, so drilling down over 1000 nested subdirectories).</p> 5073 5074<a name="13-11-2012" /><a href="#13-11-2012"><hr><h2><b>November 13, 2012</b></h2></a> 5075<blockquote><p>"Rule Six: The winning team shall be the first team that wins." 5076</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 5077 5078<p><a href=downloads/toybox-0.4.1.tar.bz2>Toybox 0.4.1</a> is based on 5079<a href=http://landley.net/hg/toybox/shortlog/691>commit 691</a>.</p> 5080 5081<p>Elie De Brauwer contributed usleep, Ashwini Kumar contributed du, and 5082Kyungwan Han contributed vconfig. Other new commands include switch_root and 5083md5sum, and the remaining shell wrappers are now proper commands (dos2unix, 5084unix2dos).</p> 5085 5086<p>The patch command now supports -l, and gethostname is now enabled by 5087default. The df command follows symlinks to get the actual device name. 5088Felix Janda added -m support to wc (for utf8).</p> 5089 5090<p>On the infrastructure side, the commands have now been grouped into 5091"posix", "lsb", and "other" subdirectories (for things required by Posix-2008, 5092the Linux Standard Base 4.1, and commands in neither). This affects menuconfig 5093and the actual source layout (toys/cp.c is now toys/posix/cp.c, and so on). 5094An android directory is planned (see the updated 5095<a href=roadmap.html#android>android roadmap analysis</a>).</p> 5096 5097<p>The FLAG_ macros for command option parsing and TT alias for the command's 5098global block are now automatically generated, commands should 5099#define FOR_commandname before #including <toys.h> to get the macros for that 5100command.</p> 5101 5102<p>An upgrade to the build infrastructure now allows commands with _ and - 5103in them, such as switch_root.</p> 5104 5105<p>Bugfixes: Avery Pennarun spotted a case where ls showed uid twice instead of 5106uid and gid, and that nice was using the wrong range of numbers. 5107The ls command also recursed inappropriately last time (not quite 5108properly converted for the dirtree changes last release), and now it's 5109fixed. Roy Tam pointed out a glitch in sh, and fixed df's percentage 5110calculation to match the POSIX spec. The kernel build didn't like our mktemp 5111and it does now. The wc command wasn't quite posix compliant (trailing spaces 5112break stuff). The ls command recursed inappropriately last time (not quite 5113properly converted for the dirtree changes last release), and now it's 5114fixed. The catv command wasn't displaying byte 255 correctly. Some lib 5115fixes (thinko in xpidfile). Fixed uname -m when running a 32 bit x86 binary 5116on an x86-64 host (it lies and says the system is i686, i586, or i486 depending 5117on what the toolchain that built the binary supported. This makes builds in 5118a 32 bit chroot on a 64 bit kernel break less.) The df command was checking 5119partitions in the wrong order (displaying undermounts instead of overmounts: 5120this used to work but some library code changed out from under it and it 5121wasn't updated to match until now). Felix Janda filled out the test suite 5122some more. The patch file creation logic got tweaked several times to 5123successfully apply more patches. Support for older (pre 2.10) glibc 5124versions was added to portability.h.</p> 5125 5126<p>Miscelaneous cleanups all around (mknod, sha1sum, logname), including a 5127rewrite of taskset to be less dependent on libc getting the headers right. All 5128the command headers should now point to the current relevant standards 5129document, where applicable.</p> 5130 5131<p>This news page had old news entries from before the relaunch moved into 5132a separate <a href=oldnews.html>oldnews</a> page.</p> 5133 5134<p>I forgot to create <a href=bin>static binaries</a> last time, but they're 5135back now.</p> 5136</span> 5137 5138<a name="23-07-2012" /><a href="#23-07-2012"><hr><h2><b>July 23, 2012</b></h2></a> 5139<blockquote><p>"Ford", Arthur said. "There's an infinite number of monkeys 5140out here who want to talk to us about this script for Hamlet they've worked 5141out."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 5142 5143<p><a href=downloads/toybox-0.4.0.tar.bz2>Toybox 0.4.0</a> is based on 5144<a href=http://landley.net/hg/toybox/shortlog/640>commit 640</a>.</p> 5145 5146<p>The new <a href=status.html>status page</a> is calculated from 5147the roadmap info, and should be easier to keep up to date in future.</p> 5148 5149<p>Andre Renaud contributed od and modinfo. Elie De Brauwer contributed 5150taskset, bugfixes to cmp and tail, and tests for sort and tail. Kyungwan Han 5151contributed passwd. Gaurang Shastri contributed w. Ashwini Sharma spotted a 5152case where dirtree was adding extra slashes to a path.</p> 5153 5154<p>I rewrote od, cleaned up comm, documented the 5155<a href=code.html#lib_llist>llist</a> and 5156<a href=code.html#lib_dirtree>dirtree</a> infrastructure, added an -r option 5157to date (and fixed a bug where -u wouldn't override /etc/localtime), 5158fixed bugs in chmod +stw, fixed ls to show suid bits properly when the 5159corresponding executable bit wasn't set, and worked around a longstanding 5160glibc bug where static linking prevents stdout from automatically flushing 5161pending output on exit.</p> 5162 5163<a name="25-06-2012" /><a href="#25-06-2012"><hr><h2><b>June 25, 2012</b></h2></a> 5164<blockquote><p>"For a moment, nothing happened. Then, after a second or so, nothing continued to happen."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 5165 5166<p><a href=downloads/toybox-0.3.1.tar.bz2>Toybox 0.3.1</a> is based on commit 5167<a href=http://landley.net/hg/toybox/shortlog/607>commit 607</a>. It's 5168mostly a bugfix release for ls -l (which was unhappy on targets other than 5169x86-64), plus a new "date" from Andre Renaud and rewritten chgrp/chown which 5170now support the full set of posix flags, plus a little work on the test 5171suite and some more header tweaks towards eventual compatability with the 5172musl libc.</p> 5173 5174<p>The todo list runneth over, but "release early, release often", so here 5175it is. The roadmap and documentation are a bit behind, and I've got ~40 5176pending submissions to review. I need to catch up...</p> 5177</span> 5178 5179<a name="12-06-2012" /><a href="#12-06-2012"><hr><h2><b>June 12, 2012</b></h2></a> 5180<blockquote><p>"For instance, on the planet Earth, man had always assumed that 5181he was more intelligent than dolphins because he had achieved so much - the 5182wheel, New York, wars and so on - whilst all the dolphins had ever done was 5183muck about in the water having a good time. But conversely, the dolphins had 5184always believed that they were far more intelligent than man - for precisely 5185the same reasons."</p><p>- The Hitchhiker's Guide to the Galaxy</p></blockquote> 5186 5187<p>It's well past time for <a href=downloads/toybox-0.3.0.tar.bz2>toybox 0.3.0</a>, 5188so here it is, based 5189on <a href=http://landley.net/hg/toybox/shortlog/595>commit 595</a>, and the 5190statically linked <a href=downloads/binaries>prebuilt binaries</a> should 5191actually be statically linked this time (thanks Ashwini Sharma for spotting 5192that).</p> 5193 5194<p>It's hard to figure out where to cut a release, because development 5195doesn't stop. "Long before now" is the obviuos answer, of course. 5196The project's maintainer also moved house during this development cycle, which 5197threw things off for a bit (so many boxes). Releases should hopefully be a bit 5198more frequent from here on.</p> 5199 5200<p>The big things Rob worked on this time were the new dirtree (directory 5201tree traversal) infrastructure, and a complete rewrite of ls using that 5202which should now implement all 26 posix options.</p> 5203 5204<p>Georgi Chorbadzhiyski added printenv, whoami, mkdir, mkfifo, chmod, chown, 5205chgrp, and uniq. He also added fraction and extension support to sleep (so if 5206you need a quarter-second sleep, it can do that now), and fixed a build bug 5207on slackware.</p> 5208 5209<p>Daniel Walter contributed a string to mode_t parser (in use by chmod and 5210mkdir -m). Ilya Kuzmich contributed comm. Elie De Brauwer added mountpoint, 5211vmstat, logname, login, and mktemp. Kevin Chase did some portability cleanups. 5212Pere Orga fixed some documentation.</p> 5213 5214<p>The "tac" and "clear" commands are now normal commands instead of shell 5215wrappers, and the header #includes have been cleaned up a bit to remove 5216deprecated functions and attempt to increase compatability with the bionic and 5217musl C libraries, "tail" should now use lseek() for large files, and "id" got 5218some cleanups and bugfixes.</p> 5219 5220<p>The new TOYBOX_FLOAT configuration option selects whether or not 5221to include floating point support (for embedded targets where that's 5222problematic).</p> 5223 5224<p>Several random bugfixes: unshare() might actually build portably now, 5225yes 'n' | cp -i should no longer bypass stdin and prompt via the tty, the 5226SUID support no longer drops permissions going through the toybox 5227multiplexer command, and a bugfix to xargs -0 means it should no longer 5228segfault. (I have a pending bug report about xargs not doing the full 5229posix whitespace handling that -0 obsoleted, but I'll deal with that next 5230release.)</p> 5231 5232<p>The build infrastructure is now automatically generating FLAG_ macros 5233for the options, but currently with the wrong names. Some more macro glue 5234is necessary, which I haven't quite figured out how to do yet.</p> 5235 5236<p>A defconfig toybox at the start of the $PATH has successfully built 5237Linux From Scratch (in my Aboriginal Linux project). The commands that 5238'default n' in the config are often still broken, cleanup is ongoing. 5239(The new dirtree stuff broke several of them that haven't been converted 5240yet, but if I wait until everything works we won't have a release before 52411.0, so here's a checkpoint.)</p> 5242 5243 5244<a name="03-03-2012" /><a href="#03-03-2012"><hr><h2><b>March 3, 2012</b></h2></a> 5245 5246<blockquote><p>"They went unnoticed at Goonhilly, passed over Cape Canaveral 5247without a blip, and Woomera and Jodrell Bank looked straight through them. 5248Which was a pity, because it was exactly the sort of thing they'd been looking 5249for all these years."</p><p>- The Hitchhiker's Guide to the Galaxy</p> 5250</p></blockquote> 5251 5252<p>Here's <a href=downloads/toybox-0.2.1.tar.bz2>toybox 0.2.1</a> based 5253on <a href=http://landley.net/hg/toybox/shortlog/512>commit 512</a>. This 5254time around, there are statically linked <a href=downloads/binaries>prebuilt 5255binaries</a> for various embedded targets.</p> 5256 5257<p>It's been a busy few weeks, almost entirely due to new contributors. (I 5258have not quite been keeping up.)</p> 5259 5260<p>Elie De Brauwer contributed free, uptime, swapon, swapoff, lsmod, mknod, 5261insmod, rmmod, and fixed a bug in basename. Andre Renaud contributed ls, ln, 5262realpath, and hostname. Andres Heck contributed pidof and killall. Daniel 5263Walter wrote kill and extended id. Timothy Elliott contributed tail and tests 5264for cmp. Frank Bergmann sent a warning fix. Bryce Fricke added -i to cp. 5265Nathan McSween pointed out an optimization. Georgi Chorbadzhiyski fixed 5266cross compiling to work more reliably.</p> 5267 5268<p>(My own contribution this time around was just tightening up other people's 5269code, a build fix to unshare, some random bugfixes, and so on. My only new 5270code this time around was writing a bash replacement for the existing python 5271bloat-o-meter.)</p> 5272 5273<p>Last time (the 0.2.0 release) included the first pass at an id command from 5274Tim Bird, env and basename from Tryn Mirell, cmp and head from Timothy Elliott, 5275more bugfixes from Nathan McSween and Elie De Brauwer, and Luis Felipe Strano 5276Moraes did a first pass at the who command plus other bugfixes and 5277optimizations.</p> 5278 5279<p>(For that release I did xargs, cal, truncate, unlink, nohup, tty, wc, link, 5280dirname, unshare, and various infrastructure tweaks, but it took me 3 months 5281and those guys did their stuff in a week or so.)</p> 5282 5283 5284<a name="12-02-2012" /><a href="#12-02-2012"><hr><h2><b>February 12, 2012</b></h2></a> 5285<blockquote><p> 5286"for though it has many omissions and contains much that is apocryphal, or at 5287least wildly inaccurate, it scores over the older, more pedestrian work in two 5288important respects..."</p> 5289<p> - The Hitchhiker's Guide to the Galaxy</p></blockquote> 5290 5291<p>Here's the first BSD licensed release, 5292<a href=downloads/toybox-0.2.0.tar.bz2>toybox-0.2.0</a>, more a synchronization 5293point than anything particularly useful. 47 commands in a reasonably 5294ready-to-use state (what "make defconfig" builds), another ten or so partially 5295finished stubs ("make allyesconfig"), and several 5296patches pending on the mailing list I need to review and merge.</p> 5297 5298<p>More to come...</p> 5299 5300<hr> 5301<a name="15-11-2011" /><a href="#15-11-2011"><hr><h2><b>November 15, 2011</b></h2></a> 5302- Back from the dead, Toybox is now under a 2 5303clause BSD license, and aiming to become the default command line 5304implementation of Android systems everywhere.</p> 5305 5306<p>More to come...</p> 5307 5308<hr> 5309 5310<p><a href=oldnews.html>Old news</a> from before the relaunch.</p> 5311 5312<!--#include file="footer.html" --> 5313