1.. SPDX-License-Identifier: GPL-2.0+ 2.. Copyright (c) 2016 Google, Inc 3 4Introduction 5============ 6 7Firmware often consists of several components which must be packaged together. 8For example, we may have SPL, U-Boot, a device tree and an environment area 9grouped together and placed in MMC flash. When the system starts, it must be 10able to find these pieces. 11 12Building firmware should be separate from packaging it. Many of the complexities 13of modern firmware build systems come from trying to do both at once. With 14binman, you build all the pieces that are needed, using whatever assortment of 15projects and build systems are needed, then use binman to stitch everything 16together. 17 18 19What it does 20------------ 21 22Binman reads your board's device tree and finds a node which describes the 23required image layout. It uses this to work out what to place where. 24 25Binman provides a mechanism for building images, from simple SPL + U-Boot 26combinations, to more complex arrangements with many parts. It also allows 27users to inspect images, extract and replace binaries within them, repacking if 28needed. 29 30 31Features 32-------- 33 34Apart from basic padding, alignment and positioning features, Binman supports 35hierarchical images, compression, hashing and dealing with the binary blobs 36which are a sad trend in open-source firmware at present. 37 38Executable binaries can access the location of other binaries in an image by 39using special linker symbols (zero-overhead but somewhat limited) or by reading 40the devicetree description of the image. 41 42Binman is designed primarily for use with U-Boot and associated binaries such 43as ARM Trusted Firmware, but it is suitable for use with other projects, such 44as Zephyr. Binman also provides facilities useful in Chromium OS, such as CBFS, 45vblocks and and the like. 46 47Binman provides a way to process binaries before they are included, by adding a 48Python plug-in. 49 50Binman is intended for use with U-Boot but is designed to be general enough 51to be useful in other image-packaging situations. 52 53 54Motivation 55---------- 56 57As mentioned above, packaging of firmware is quite a different task from 58building the various parts. In many cases the various binaries which go into 59the image come from separate build systems. For example, ARM Trusted Firmware 60is used on ARMv8 devices but is not built in the U-Boot tree. If a Linux kernel 61is included in the firmware image, it is built elsewhere. 62 63It is of course possible to add more and more build rules to the U-Boot 64build system to cover these cases. It can shell out to other Makefiles and 65build scripts. But it seems better to create a clear divide between building 66software and packaging it. 67 68At present this is handled by manual instructions, different for each board, 69on how to create images that will boot. By turning these instructions into a 70standard format, we can support making valid images for any board without 71manual effort, lots of READMEs, etc. 72 73Benefits: 74 75 - Each binary can have its own build system and tool chain without creating 76 any dependencies between them 77 - Avoids the need for a single-shot build: individual parts can be updated 78 and brought in as needed 79 - Provides for a standard image description available in the build and at 80 run-time 81 - SoC-specific image-signing tools can be accommodated 82 - Avoids cluttering the U-Boot build system with image-building code 83 - The image description is automatically available at run-time in U-Boot, 84 SPL. It can be made available to other software also 85 - The image description is easily readable (it's a text file in device-tree 86 format) and permits flexible packing of binaries 87 88 89Terminology 90----------- 91 92Binman uses the following terms: 93 94- image - an output file containing a firmware image 95- binary - an input binary that goes into the image 96 97 98Relationship to FIT 99------------------- 100 101FIT is U-Boot's official image format. It supports multiple binaries with 102load / execution addresses, compression. It also supports verification 103through hashing and RSA signatures. 104 105FIT was originally designed to support booting a Linux kernel (with an 106optional ramdisk) and device tree chosen from various options in the FIT. 107Now that U-Boot supports configuration via device tree, it is possible to 108load U-Boot from a FIT, with the device tree chosen by SPL. 109 110Binman considers FIT to be one of the binaries it can place in the image. 111 112Where possible it is best to put as much as possible in the FIT, with binman 113used to deal with cases not covered by FIT. Examples include initial 114execution (since FIT itself does not have an executable header) and dealing 115with device boundaries, such as the read-only/read-write separation in SPI 116flash. 117 118For U-Boot, binman should not be used to create ad-hoc images in place of 119FIT. 120 121 122Relationship to mkimage 123----------------------- 124 125The mkimage tool provides a means to create a FIT. Traditionally it has 126needed an image description file: a device tree, like binman, but in a 127different format. More recently it has started to support a '-f auto' mode 128which can generate that automatically. 129 130More relevant to binman, mkimage also permits creation of many SoC-specific 131image types. These can be listed by running 'mkimage -T list'. Examples 132include 'rksd', the Rockchip SD/MMC boot format. The mkimage tool is often 133called from the U-Boot build system for this reason. 134 135Binman considers the output files created by mkimage to be binary blobs 136which it can place in an image. Binman does not replace the mkimage tool or 137this purpose. It would be possible in some situations to create a new entry 138type for the images in mkimage, but this would not add functionality. It 139seems better to use the mkimage tool to generate binaries and avoid blurring 140the boundaries between building input files (mkimage) and packaging then 141into a final image (binman). 142 143 144Using binman 145============ 146 147Example use of binman in U-Boot 148------------------------------- 149 150Binman aims to replace some of the ad-hoc image creation in the U-Boot 151build system. 152 153Consider sunxi. It has the following steps: 154 155 #. It uses a custom mksunxiboot tool to build an SPL image called 156 sunxi-spl.bin. This should probably move into mkimage. 157 158 #. It uses mkimage to package U-Boot into a legacy image file (so that it can 159 hold the load and execution address) called u-boot.img. 160 161 #. It builds a final output image called u-boot-sunxi-with-spl.bin which 162 consists of sunxi-spl.bin, some padding and u-boot.img. 163 164Binman is intended to replace the last step. The U-Boot build system builds 165u-boot.bin and sunxi-spl.bin. Binman can then take over creation of 166sunxi-spl.bin (by calling mksunxiboot, or hopefully one day mkimage). In any 167case, it would then create the image from the component parts. 168 169This simplifies the U-Boot Makefile somewhat, since various pieces of logic 170can be replaced by a call to binman. 171 172 173Example use of binman for x86 174----------------------------- 175 176In most cases x86 images have a lot of binary blobs, 'black-box' code 177provided by Intel which must be run for the platform to work. Typically 178these blobs are not relocatable and must be placed at fixed areas in the 179firmware image. 180 181Currently this is handled by ifdtool, which places microcode, FSP, MRC, VGA 182BIOS, reference code and Intel ME binaries into a u-boot.rom file. 183 184Binman is intended to replace all of this, with ifdtool left to handle only 185the configuration of the Intel-format descriptor. 186 187 188Running binman 189-------------- 190 191First install prerequisites, e.g:: 192 193 sudo apt-get install python-pyelftools python3-pyelftools lzma-alone \ 194 liblz4-tool 195 196Type:: 197 198 binman build -b <board_name> 199 200to build an image for a board. The board name is the same name used when 201configuring U-Boot (e.g. for sandbox_defconfig the board name is 'sandbox'). 202Binman assumes that the input files for the build are in ../b/<board_name>. 203 204Or you can specify this explicitly:: 205 206 binman build -I <build_path> 207 208where <build_path> is the build directory containing the output of the U-Boot 209build. 210 211(Future work will make this more configurable) 212 213In either case, binman picks up the device tree file (u-boot.dtb) and looks 214for its instructions in the 'binman' node. 215 216Binman has a few other options which you can see by running 'binman -h'. 217 218 219Enabling binman for a board 220--------------------------- 221 222At present binman is invoked from a rule in the main Makefile. You should be 223able to enable CONFIG_BINMAN to enable this rule. 224 225The output file is typically named image.bin and is located in the output 226directory. If input files are needed to you add these to INPUTS-y either in the 227main Makefile or in a config.mk file in your arch subdirectory. 228 229Once binman is executed it will pick up its instructions from a device-tree 230file, typically <soc>-u-boot.dtsi, where <soc> is your CONFIG_SYS_SOC value. 231You can use other, more specific CONFIG options - see 'Automatic .dtsi 232inclusion' below. 233 234 235Using binman with OF_BOARD 236-------------------------------------------- 237 238Normally binman is used with a board configured with OF_SEPARATE or OF_EMBED. 239This is a typical scenario where a device tree source that contains the binman 240node is provided in the arch/<arch>/dts directory for a specific board. 241 242However for a board configured with OF_BOARD, no device tree blob is provided 243in the U-Boot build phase hence the binman node information is not available. 244In order to support such use case, a new Kconfig option BINMAN_STANDALONE_FDT 245is introduced, to tell the build system that a standalone device tree blob 246containing binman node is explicitly required. 247 248Note there is a Kconfig option BINMAN_FDT which enables U-Boot run time to 249access information about binman entries, stored in the device tree in a binman 250node. Generally speaking, this option makes sense for OF_SEPARATE or OF_EMBED. 251For the other OF_CONTROL methods, it's quite possible binman node is not 252available as binman is invoked during the build phase, thus this option is not 253turned on by default for these OF_CONTROL methods. 254 255Access to binman entry offsets at run time (symbols) 256---------------------------------------------------- 257 258Binman assembles images and determines where each entry is placed in the image. 259This information may be useful to U-Boot at run time. For example, in SPL it 260is useful to be able to find the location of U-Boot so that it can be executed 261when SPL is finished. 262 263Binman allows you to declare symbols in the SPL image which are filled in 264with their correct values during the build. For example:: 265 266 binman_sym_declare(ulong, u_boot_any, image_pos); 267 268declares a ulong value which will be assigned to the image-pos of any U-Boot 269image (u-boot.bin, u-boot.img, u-boot-nodtb.bin) that is present in the image. 270You can access this value with something like:: 271 272 ulong u_boot_offset = binman_sym(ulong, u_boot_any, image_pos); 273 274Thus u_boot_offset will be set to the image-pos of U-Boot in memory, assuming 275that the whole image has been loaded, or is available in flash. You can then 276jump to that address to start U-Boot. 277 278At present this feature is only supported in SPL and TPL. In principle it is 279possible to fill in such symbols in U-Boot proper, as well, but a future C 280library is planned for this instead, to read from the device tree. 281 282As well as image-pos, it is possible to read the size of an entry and its 283offset (which is the start position of the entry within its parent). 284 285A small technical note: Binman automatically adds the base address of the image 286(i.e. __image_copy_start) to the value of the image-pos symbol, so that when the 287image is loaded to its linked address, the value will be correct and actually 288point into the image. 289 290For example, say SPL is at the start of the image and linked to start at address 29180108000. If U-Boot's image-pos is 0x8000 then binman will write an image-pos 292for U-Boot of 80110000 into the SPL binary, since it assumes the image is loaded 293to 80108000, with SPL at 80108000 and U-Boot at 80110000. 294 295For x86 devices (with the end-at-4gb property) this base address is not added 296since it is assumed that images are XIP and the offsets already include the 297address. 298 299 300Access to binman entry offsets at run time (fdt) 301------------------------------------------------ 302 303Binman can update the U-Boot FDT to include the final position and size of 304each entry in the images it processes. The option to enable this is -u and it 305causes binman to make sure that the 'offset', 'image-pos' and 'size' properties 306are set correctly for every entry. Since it is not necessary to specify these in 307the image definition, binman calculates the final values and writes these to 308the device tree. These can be used by U-Boot at run-time to find the location 309of each entry. 310 311Alternatively, an FDT map entry can be used to add a special FDT containing 312just the information about the image. This is preceded by a magic string so can 313be located anywhere in the image. An image header (typically at the start or end 314of the image) can be used to point to the FDT map. See fdtmap and image-header 315entries for more information. 316 317 318Map files 319--------- 320 321The -m option causes binman to output a .map file for each image that it 322generates. This shows the offset and size of each entry. For example:: 323 324 Offset Size Name 325 00000000 00000028 main-section 326 00000000 00000010 section@0 327 00000000 00000004 u-boot 328 00000010 00000010 section@1 329 00000000 00000004 u-boot 330 331This shows a hierarchical image with two sections, each with a single entry. The 332offsets of the sections are absolute hex byte offsets within the image. The 333offsets of the entries are relative to their respective sections. The size of 334each entry is also shown, in bytes (hex). The indentation shows the entries 335nested inside their sections. 336 337 338Passing command-line arguments to entries 339----------------------------------------- 340 341Sometimes it is useful to pass binman the value of an entry property from the 342command line. For example some entries need access to files and it is not 343always convenient to put these filenames in the image definition (device tree). 344 345The -a option supports this:: 346 347 -a <prop>=<value> 348 349where:: 350 351 <prop> is the property to set 352 <value> is the value to set it to 353 354Not all properties can be provided this way. Only some entries support it, 355typically for filenames. 356 357 358Image description format 359======================== 360 361The binman node is called 'binman'. An example image description is shown 362below:: 363 364 binman { 365 filename = "u-boot-sunxi-with-spl.bin"; 366 pad-byte = <0xff>; 367 blob { 368 filename = "spl/sunxi-spl.bin"; 369 }; 370 u-boot { 371 offset = <CONFIG_SPL_PAD_TO>; 372 }; 373 }; 374 375 376This requests binman to create an image file called u-boot-sunxi-with-spl.bin 377consisting of a specially formatted SPL (spl/sunxi-spl.bin, built by the 378normal U-Boot Makefile), some 0xff padding, and a U-Boot legacy image. The 379padding comes from the fact that the second binary is placed at 380CONFIG_SPL_PAD_TO. If that line were omitted then the U-Boot binary would 381immediately follow the SPL binary. 382 383The binman node describes an image. The sub-nodes describe entries in the 384image. Each entry represents a region within the overall image. The name of 385the entry (blob, u-boot) tells binman what to put there. For 'blob' we must 386provide a filename. For 'u-boot', binman knows that this means 'u-boot.bin'. 387 388Entries are normally placed into the image sequentially, one after the other. 389The image size is the total size of all entries. As you can see, you can 390specify the start offset of an entry using the 'offset' property. 391 392Note that due to a device tree requirement, all entries must have a unique 393name. If you want to put the same binary in the image multiple times, you can 394use any unique name, with the 'type' property providing the type. 395 396The attributes supported for entries are described below. 397 398offset: 399 This sets the offset of an entry within the image or section containing 400 it. The first byte of the image is normally at offset 0. If 'offset' is 401 not provided, binman sets it to the end of the previous region, or the 402 start of the image's entry area (normally 0) if there is no previous 403 region. 404 405align: 406 This sets the alignment of the entry. The entry offset is adjusted 407 so that the entry starts on an aligned boundary within the containing 408 section or image. For example 'align = <16>' means that the entry will 409 start on a 16-byte boundary. This may mean that padding is added before 410 the entry. The padding is part of the containing section but is not 411 included in the entry, meaning that an empty space may be created before 412 the entry starts. Alignment should be a power of 2. If 'align' is not 413 provided, no alignment is performed. 414 415size: 416 This sets the size of the entry. The contents will be padded out to 417 this size. If this is not provided, it will be set to the size of the 418 contents. 419 420pad-before: 421 Padding before the contents of the entry. Normally this is 0, meaning 422 that the contents start at the beginning of the entry. This can be used 423 to offset the entry contents a little. While this does not affect the 424 contents of the entry within binman itself (the padding is performed 425 only when its parent section is assembled), the end result will be that 426 the entry starts with the padding bytes, so may grow. Defaults to 0. 427 428pad-after: 429 Padding after the contents of the entry. Normally this is 0, meaning 430 that the entry ends at the last byte of content (unless adjusted by 431 other properties). This allows room to be created in the image for 432 this entry to expand later. While this does not affect the contents of 433 the entry within binman itself (the padding is performed only when its 434 parent section is assembled), the end result will be that the entry ends 435 with the padding bytes, so may grow. Defaults to 0. 436 437align-size: 438 This sets the alignment of the entry size. For example, to ensure 439 that the size of an entry is a multiple of 64 bytes, set this to 64. 440 While this does not affect the contents of the entry within binman 441 itself (the padding is performed only when its parent section is 442 assembled), the end result is that the entry ends with the padding 443 bytes, so may grow. If 'align-size' is not provided, no alignment is 444 performed. 445 446align-end: 447 This sets the alignment of the end of an entry with respect to the 448 containing section. Some entries require that they end on an alignment 449 boundary, regardless of where they start. This does not move the start 450 of the entry, so the contents of the entry will still start at the 451 beginning. But there may be padding at the end. While this does not 452 affect the contents of the entry within binman itself (the padding is 453 performed only when its parent section is assembled), the end result 454 is that the entry ends with the padding bytes, so may grow. 455 If 'align-end' is not provided, no alignment is performed. 456 457filename: 458 For 'blob' types this provides the filename containing the binary to 459 put into the entry. If binman knows about the entry type (like 460 u-boot-bin), then there is no need to specify this. 461 462type: 463 Sets the type of an entry. This defaults to the entry name, but it is 464 possible to use any name, and then add (for example) 'type = "u-boot"' 465 to specify the type. 466 467offset-unset: 468 Indicates that the offset of this entry should not be set by placing 469 it immediately after the entry before. Instead, is set by another 470 entry which knows where this entry should go. When this boolean 471 property is present, binman will give an error if another entry does 472 not set the offset (with the GetOffsets() method). 473 474image-pos: 475 This cannot be set on entry (or at least it is ignored if it is), but 476 with the -u option, binman will set it to the absolute image position 477 for each entry. This makes it easy to find out exactly where the entry 478 ended up in the image, regardless of parent sections, etc. 479 480expand-size: 481 Expand the size of this entry to fit available space. This space is only 482 limited by the size of the image/section and the position of the next 483 entry. 484 485compress: 486 Sets the compression algortihm to use (for blobs only). See the entry 487 documentation for details. 488 489missing-msg: 490 Sets the tag of the message to show if this entry is missing. This is 491 used for external blobs. When they are missing it is helpful to show 492 information about what needs to be fixed. See missing-blob-help for the 493 message for each tag. 494 495no-expanded: 496 By default binman substitutes entries with expanded versions if available, 497 so that a `u-boot` entry type turns into `u-boot-expanded`, for example. The 498 `--no-expanded` command-line option disables this globally. The 499 `no-expanded` property disables this just for a single entry. Put the 500 `no-expanded` boolean property in the node to select this behaviour. 501 502The attributes supported for images and sections are described below. Several 503are similar to those for entries. 504 505size: 506 Sets the image size in bytes, for example 'size = <0x100000>' for a 507 1MB image. 508 509offset: 510 This is similar to 'offset' in entries, setting the offset of a section 511 within the image or section containing it. The first byte of the section 512 is normally at offset 0. If 'offset' is not provided, binman sets it to 513 the end of the previous region, or the start of the image's entry area 514 (normally 0) if there is no previous region. 515 516align-size: 517 This sets the alignment of the image size. For example, to ensure 518 that the image ends on a 512-byte boundary, use 'align-size = <512>'. 519 If 'align-size' is not provided, no alignment is performed. 520 521pad-before: 522 This sets the padding before the image entries. The first entry will 523 be positioned after the padding. This defaults to 0. 524 525pad-after: 526 This sets the padding after the image entries. The padding will be 527 placed after the last entry. This defaults to 0. 528 529pad-byte: 530 This specifies the pad byte to use when padding in the image. It 531 defaults to 0. To use 0xff, you would add 'pad-byte = <0xff>'. 532 533filename: 534 This specifies the image filename. It defaults to 'image.bin'. 535 536sort-by-offset: 537 This causes binman to reorder the entries as needed to make sure they 538 are in increasing positional order. This can be used when your entry 539 order may not match the positional order. A common situation is where 540 the 'offset' properties are set by CONFIG options, so their ordering is 541 not known a priori. 542 543 This is a boolean property so needs no value. To enable it, add a 544 line 'sort-by-offset;' to your description. 545 546multiple-images: 547 Normally only a single image is generated. To create more than one 548 image, put this property in the binman node. For example, this will 549 create image1.bin containing u-boot.bin, and image2.bin containing 550 both spl/u-boot-spl.bin and u-boot.bin:: 551 552 binman { 553 multiple-images; 554 image1 { 555 u-boot { 556 }; 557 }; 558 559 image2 { 560 spl { 561 }; 562 u-boot { 563 }; 564 }; 565 }; 566 567end-at-4gb: 568 For x86 machines the ROM offsets start just before 4GB and extend 569 up so that the image finished at the 4GB boundary. This boolean 570 option can be enabled to support this. The image size must be 571 provided so that binman knows when the image should start. For an 572 8MB ROM, the offset of the first entry would be 0xfff80000 with 573 this option, instead of 0 without this option. 574 575skip-at-start: 576 This property specifies the entry offset of the first entry. 577 578 For PowerPC mpc85xx based CPU, CONFIG_SYS_TEXT_BASE is the entry 579 offset of the first entry. It can be 0xeff40000 or 0xfff40000 for 580 nor flash boot, 0x201000 for sd boot etc. 581 582 'end-at-4gb' property is not applicable where CONFIG_SYS_TEXT_BASE + 583 Image size != 4gb. 584 585align-default: 586 Specifies the default alignment for entries in this section, if they do 587 not specify an alignment. Note that this only applies to top-level entries 588 in the section (direct subentries), not any subentries of those entries. 589 This means that each section must specify its own default alignment, if 590 required. 591 592Examples of the above options can be found in the tests. See the 593tools/binman/test directory. 594 595It is possible to have the same binary appear multiple times in the image, 596either by using a unit number suffix (u-boot@0, u-boot@1) or by using a 597different name for each and specifying the type with the 'type' attribute. 598 599 600Sections and hierachical images 601------------------------------- 602 603Sometimes it is convenient to split an image into several pieces, each of which 604contains its own set of binaries. An example is a flash device where part of 605the image is read-only and part is read-write. We can set up sections for each 606of these, and place binaries in them independently. The image is still produced 607as a single output file. 608 609This feature provides a way of creating hierarchical images. For example here 610is an example image with two copies of U-Boot. One is read-only (ro), intended 611to be written only in the factory. Another is read-write (rw), so that it can be 612upgraded in the field. The sizes are fixed so that the ro/rw boundary is known 613and can be programmed:: 614 615 binman { 616 section@0 { 617 read-only; 618 name-prefix = "ro-"; 619 size = <0x100000>; 620 u-boot { 621 }; 622 }; 623 section@1 { 624 name-prefix = "rw-"; 625 size = <0x100000>; 626 u-boot { 627 }; 628 }; 629 }; 630 631This image could be placed into a SPI flash chip, with the protection boundary 632set at 1MB. 633 634A few special properties are provided for sections: 635 636read-only: 637 Indicates that this section is read-only. This has no impact on binman's 638 operation, but his property can be read at run time. 639 640name-prefix: 641 This string is prepended to all the names of the binaries in the 642 section. In the example above, the 'u-boot' binaries which actually be 643 renamed to 'ro-u-boot' and 'rw-u-boot'. This can be useful to 644 distinguish binaries with otherwise identical names. 645 646 647Image Properties 648---------------- 649 650Image nodes act like sections but also have a few extra properties: 651 652filename: 653 Output filename for the image. This defaults to image.bin (or in the 654 case of multiple images <nodename>.bin where <nodename> is the name of 655 the image node. 656 657allow-repack: 658 Create an image that can be repacked. With this option it is possible 659 to change anything in the image after it is created, including updating 660 the position and size of image components. By default this is not 661 permitted since it is not possibly to know whether this might violate a 662 constraint in the image description. For example, if a section has to 663 increase in size to hold a larger binary, that might cause the section 664 to fall out of its allow region (e.g. read-only portion of flash). 665 666 Adding this property causes the original offset and size values in the 667 image description to be stored in the FDT and fdtmap. 668 669 670Hashing Entries 671--------------- 672 673It is possible to ask binman to hash the contents of an entry and write that 674value back to the device-tree node. For example:: 675 676 binman { 677 u-boot { 678 hash { 679 algo = "sha256"; 680 }; 681 }; 682 }; 683 684Here, a new 'value' property will be written to the 'hash' node containing 685the hash of the 'u-boot' entry. Only SHA256 is supported at present. Whole 686sections can be hased if desired, by adding the 'hash' node to the section. 687 688The has value can be chcked at runtime by hashing the data actually read and 689comparing this has to the value in the device tree. 690 691 692Expanded entries 693---------------- 694 695Binman automatically replaces 'u-boot' with an expanded version of that, i.e. 696'u-boot-expanded'. This means that when you write:: 697 698 u-boot { 699 }; 700 701you actually get:: 702 703 u-boot { 704 type = "u-boot-expanded'; 705 }; 706 707which in turn expands to:: 708 709 u-boot { 710 type = "section"; 711 712 u-boot-nodtb { 713 }; 714 715 u-boot-dtb { 716 }; 717 }; 718 719U-Boot's various phase binaries actually comprise two or three pieces. 720For example, u-boot.bin has the executable followed by a devicetree. 721 722With binman we want to be able to update that devicetree with full image 723information so that it is accessible to the executable. This is tricky 724if it is not clear where the devicetree starts. 725 726The above feature ensures that the devicetree is clearly separated from the 727U-Boot executable and can be updated separately by binman as needed. It can be 728disabled with the --no-expanded flag if required. 729 730The same applies for u-boot-spl and u-boot-spl. In those cases, the expansion 731includes the BSS padding, so for example:: 732 733 spl { 734 type = "u-boot-spl" 735 }; 736 737you actually get:: 738 739 spl { 740 type = "u-boot-expanded'; 741 }; 742 743which in turn expands to:: 744 745 spl { 746 type = "section"; 747 748 u-boot-spl-nodtb { 749 }; 750 751 u-boot-spl-bss-pad { 752 }; 753 754 u-boot-spl-dtb { 755 }; 756 }; 757 758Of course we should not expand SPL if it has no devicetree. Also if the BSS 759padding is not needed (because BSS is in RAM as with CONFIG_SPL_SEPARATE_BSS), 760the 'u-boot-spl-bss-pad' subnode should not be created. The use of the expaned 761entry type is controlled by the UseExpanded() method. In the SPL case it checks 762the 'spl-dtb' entry arg, which is 'y' or '1' if SPL has a devicetree. 763 764For the BSS case, a 'spl-bss-pad' entry arg controls whether it is present. All 765entry args are provided by the U-Boot Makefile. 766 767 768Compression 769----------- 770 771Binman support compression for 'blob' entries (those of type 'blob' and 772derivatives). To enable this for an entry, add a 'compress' property:: 773 774 blob { 775 filename = "datafile"; 776 compress = "lz4"; 777 }; 778 779The entry will then contain the compressed data, using the 'lz4' compression 780algorithm. Currently this is the only one that is supported. The uncompressed 781size is written to the node in an 'uncomp-size' property, if -u is used. 782 783Compression is also supported for sections. In that case the entire section is 784compressed in one block, including all its contents. This means that accessing 785an entry from the section required decompressing the entire section. Also, the 786size of a section indicates the space that it consumes in its parent section 787(and typically the image). With compression, the section may contain more data, 788and the uncomp-size property indicates that, as above. The contents of the 789section is compressed first, before any padding is added. This ensures that the 790padding itself is not compressed, which would be a waste of time. 791 792 793Automatic .dtsi inclusion 794------------------------- 795 796It is sometimes inconvenient to add a 'binman' node to the .dts file for each 797board. This can be done by using #include to bring in a common file. Another 798approach supported by the U-Boot build system is to automatically include 799a common header. You can then put the binman node (and anything else that is 800specific to U-Boot, such as u-boot,dm-pre-reloc properies) in that header 801file. 802 803Binman will search for the following files in arch/<arch>/dts:: 804 805 <dts>-u-boot.dtsi where <dts> is the base name of the .dts file 806 <CONFIG_SYS_SOC>-u-boot.dtsi 807 <CONFIG_SYS_CPU>-u-boot.dtsi 808 <CONFIG_SYS_VENDOR>-u-boot.dtsi 809 u-boot.dtsi 810 811U-Boot will only use the first one that it finds. If you need to include a 812more general file you can do that from the more specific file using #include. 813If you are having trouble figuring out what is going on, you can uncomment 814the 'warning' line in scripts/Makefile.lib to see what it has found:: 815 816 # Uncomment for debugging 817 # This shows all the files that were considered and the one that we chose. 818 # u_boot_dtsi_options_debug = $(u_boot_dtsi_options_raw) 819 820 821Updating an ELF file 822==================== 823 824For the EFI app, where U-Boot is loaded from UEFI and runs as an app, there is 825no way to update the devicetree after U-Boot is built. Normally this works by 826creating a new u-boot.dtb.out with he updated devicetree, which is automatically 827built into the output image. With ELF this is not possible since the ELF is 828not part of an image, just a stand-along file. We must create an updated ELF 829file with the new devicetree. 830 831This is handled by the --update-fdt-in-elf option. It takes four arguments, 832separated by comma: 833 834 infile - filename of input ELF file, e.g. 'u-boot's 835 outfile - filename of output ELF file, e.g. 'u-boot.out' 836 begin_sym - symbol at the start of the embedded devicetree, e.g. 837 '__dtb_dt_begin' 838 end_sym - symbol at the start of the embedded devicetree, e.g. 839 '__dtb_dt_end' 840 841When this flag is used, U-Boot does all the normal packaging, but as an 842additional step, it creates a new ELF file with the new devicetree embedded in 843it. 844 845If logging is enabled you will see a message like this:: 846 847 Updating file 'u-boot' with data length 0x400a (16394) between symbols 848 '__dtb_dt_begin' and '__dtb_dt_end' 849 850There must be enough space for the updated devicetree. If not, an error like 851the following is produced:: 852 853 ValueError: Not enough space in 'u-boot' for data length 0x400a (16394); 854 size is 0x1744 (5956) 855 856 857Entry Documentation 858=================== 859 860For details on the various entry types supported by binman and how to use them, 861see entries.rst which is generated from the source code using: 862 863 binman entry-docs >tools/binman/entries.rst 864 865.. toctree:: 866 :maxdepth: 2 867 868 entries 869 870 871Managing images 872=============== 873 874Listing images 875-------------- 876 877It is possible to list the entries in an existing firmware image created by 878binman, provided that there is an 'fdtmap' entry in the image. For example:: 879 880 $ binman ls -i image.bin 881 Name Image-pos Size Entry-type Offset Uncomp-size 882 ---------------------------------------------------------------------- 883 main-section c00 section 0 884 u-boot 0 4 u-boot 0 885 section 5fc section 4 886 cbfs 100 400 cbfs 0 887 u-boot 138 4 u-boot 38 888 u-boot-dtb 180 108 u-boot-dtb 80 3b5 889 u-boot-dtb 500 1ff u-boot-dtb 400 3b5 890 fdtmap 6fc 381 fdtmap 6fc 891 image-header bf8 8 image-header bf8 892 893This shows the hierarchy of the image, the position, size and type of each 894entry, the offset of each entry within its parent and the uncompressed size if 895the entry is compressed. 896 897It is also possible to list just some files in an image, e.g.:: 898 899 $ binman ls -i image.bin section/cbfs 900 Name Image-pos Size Entry-type Offset Uncomp-size 901 -------------------------------------------------------------------- 902 cbfs 100 400 cbfs 0 903 u-boot 138 4 u-boot 38 904 u-boot-dtb 180 108 u-boot-dtb 80 3b5 905 906or with wildcards:: 907 908 $ binman ls -i image.bin "*cb*" "*head*" 909 Name Image-pos Size Entry-type Offset Uncomp-size 910 ---------------------------------------------------------------------- 911 cbfs 100 400 cbfs 0 912 u-boot 138 4 u-boot 38 913 u-boot-dtb 180 108 u-boot-dtb 80 3b5 914 image-header bf8 8 image-header bf8 915 916 917Extracting files from images 918---------------------------- 919 920You can extract files from an existing firmware image created by binman, 921provided that there is an 'fdtmap' entry in the image. For example:: 922 923 $ binman extract -i image.bin section/cbfs/u-boot 924 925which will write the uncompressed contents of that entry to the file 'u-boot' in 926the current directory. You can also extract to a particular file, in this case 927u-boot.bin:: 928 929 $ binman extract -i image.bin section/cbfs/u-boot -f u-boot.bin 930 931It is possible to extract all files into a destination directory, which will 932put files in subdirectories matching the entry hierarchy:: 933 934 $ binman extract -i image.bin -O outdir 935 936or just a selection:: 937 938 $ binman extract -i image.bin "*u-boot*" -O outdir 939 940 941Replacing files in an image 942--------------------------- 943 944You can replace files in an existing firmware image created by binman, provided 945that there is an 'fdtmap' entry in the image. For example: 946 947 $ binman replace -i image.bin section/cbfs/u-boot 948 949which will write the contents of the file 'u-boot' from the current directory 950to the that entry, compressing if necessary. If the entry size changes, you must 951add the 'allow-repack' property to the original image before generating it (see 952above), otherwise you will get an error. 953 954You can also use a particular file, in this case u-boot.bin:: 955 956 $ binman replace -i image.bin section/cbfs/u-boot -f u-boot.bin 957 958It is possible to replace all files from a source directory which uses the same 959hierarchy as the entries:: 960 961 $ binman replace -i image.bin -I indir 962 963Files that are missing will generate a warning. 964 965You can also replace just a selection of entries:: 966 967 $ binman replace -i image.bin "*u-boot*" -I indir 968 969 970Logging 971------- 972 973Binman normally operates silently unless there is an error, in which case it 974just displays the error. The -D/--debug option can be used to create a full 975backtrace when errors occur. You can use BINMAN_DEBUG=1 when building to select 976this. 977 978Internally binman logs some output while it is running. This can be displayed 979by increasing the -v/--verbosity from the default of 1: 980 981 0: silent 982 1: warnings only 983 2: notices (important messages) 984 3: info about major operations 985 4: detailed information about each operation 986 5: debug (all output) 987 988You can use BINMAN_VERBOSE=5 (for example) when building to select this. 989 990 991Technical details 992================= 993 994Order of image creation 995----------------------- 996 997Image creation proceeds in the following order, for each entry in the image. 998 9991. AddMissingProperties() - binman can add calculated values to the device 1000tree as part of its processing, for example the offset and size of each
1001entry. This method adds any properties associated with this, expanding the 1002device tree as needed. These properties can have placeholder values which are 1003set later by SetCalculatedProperties(). By that stage the size of sections 1004cannot be changed (since it would cause the images to need to be repacked), 1005but the correct values can be inserted. 1006 10072. ProcessFdt() - process the device tree information as required by the 1008particular entry. This may involve adding or deleting properties. If the 1009processing is complete, this method should return True. If the processing 1010cannot complete because it needs the ProcessFdt() method of another entry to 1011run first, this method should return False, in which case it will be called 1012again later. 1013 10143. GetEntryContents() - the contents of each entry are obtained, normally by 1015reading from a file. This calls the Entry.ObtainContents() to read the 1016contents. The default version of Entry.ObtainContents() calls 1017Entry.GetDefaultFilename() and then reads that file. So a common mechanism 1018to select a file to read is to override that function in the subclass. The 1019functions must return True when they have read the contents. Binman will 1020retry calling the functions a few times if False is returned, allowing 1021dependencies between the contents of different entries. 1022 10234. GetEntryOffsets() - calls Entry.GetOffsets() for each entry. This can 1024return a dict containing entries that need updating. The key should be the 1025entry name and the value is a tuple (offset, size). This allows an entry to 1026provide the offset and size for other entries. The default implementation 1027of GetEntryOffsets() returns {}. 1028 10295. PackEntries() - calls Entry.Pack() which figures out the offset and 1030size of an entry. The 'current' image offset is passed in, and the function 1031returns the offset immediately after the entry being packed. The default 1032implementation of Pack() is usually sufficient. 1033 1034Note: for sections, this also checks that the entries do not overlap, nor extend 1035outside the section. If the section does not have a defined size, the size is 1036set large enough to hold all the entries. 1037 10386. SetImagePos() - sets the image position of every entry. This is the absolute 1039position 'image-pos', as opposed to 'offset' which is relative to the containing 1040section. This must be done after all offsets are known, which is why it is quite 1041late in the ordering. 1042 10437. SetCalculatedProperties() - update any calculated properties in the device 1044tree. This sets the correct 'offset' and 'size' vaues, for example. 1045 10468. ProcessEntryContents() - this calls Entry.ProcessContents() on each entry. 1047The default implementatoin does nothing. This can be overriden to adjust the 1048contents of an entry in some way. For example, it would be possible to create 1049an entry containing a hash of the contents of some other entries. At this 1050stage the offset and size of entries should not be adjusted unless absolutely 1051necessary, since it requires a repack (going back to PackEntries()). 1052 10539. ResetForPack() - if the ProcessEntryContents() step failed, in that an entry 1054has changed its size, then there is no alternative but to go back to step 5 and 1055try again, repacking the entries with the updated size. ResetForPack() removes 1056the fixed offset/size values added by binman, so that the packing can start from 1057scratch. 1058 105910. WriteSymbols() - write the value of symbols into the U-Boot SPL binary. 1060See 'Access to binman entry offsets at run time' below for a description of 1061what happens in this stage. 1062 106311. BuildImage() - builds the image and writes it to a file 1064 106512. WriteMap() - writes a text file containing a map of the image. This is the 1066final step. 1067 1068 1069External tools 1070-------------- 1071 1072Binman can make use of external command-line tools to handle processing of 1073entry contents or to generate entry contents. These tools are executed using 1074the 'tools' module's Run() method. The tools generally must exist on the PATH, 1075but the --toolpath option can be used to specify additional search paths to 1076use. This option can be specified multiple times to add more than one path. 1077 1078For some compile tools binman will use the versions specified by commonly-used 1079environment variables like CC and HOSTCC for the C compiler, based on whether 1080the tool's output will be used for the target or for the host machine. If those 1081aren't given, it will also try to derive target-specific versions from the 1082CROSS_COMPILE environment variable during a cross-compilation. 1083 1084 1085Code coverage 1086------------- 1087 1088Binman is a critical tool and is designed to be very testable. Entry 1089implementations target 100% test coverage. Run 'binman test -T' to check this. 1090 1091To enable Python test coverage on Debian-type distributions (e.g. Ubuntu):: 1092 1093 $ sudo apt-get install python-coverage python3-coverage python-pytest 1094 1095 1096Concurrent tests 1097---------------- 1098 1099Binman tries to run tests concurrently. This means that the tests make use of 1100all available CPUs to run. 1101 1102 To enable this:: 1103 1104 $ sudo apt-get install python-subunit python3-subunit 1105 1106Use '-P 1' to disable this. It is automatically disabled when code coverage is 1107being used (-T) since they are incompatible. 1108 1109 1110Debugging tests 1111--------------- 1112 1113Sometimes when debugging tests it is useful to keep the input and output 1114directories so they can be examined later. Use -X or --test-preserve-dirs for 1115this. 1116 1117 1118Running tests on non-x86 architectures 1119-------------------------------------- 1120 1121Binman's tests have been written under the assumption that they'll be run on a 1122x86-like host and there hasn't been an attempt to make them portable yet. 1123However, it's possible to run the tests by cross-compiling to x86. 1124 1125To install an x86 cross-compiler on Debian-type distributions (e.g. Ubuntu):: 1126 1127 $ sudo apt-get install gcc-x86-64-linux-gnu 1128 1129Then, you can run the tests under cross-compilation:: 1130 1131 $ CROSS_COMPILE=x86_64-linux-gnu- binman test -T 1132 1133You can also use gcc-i686-linux-gnu similar to the above. 1134 1135 1136Writing new entries and debugging 1137--------------------------------- 1138 1139The behaviour of entries is defined by the Entry class. All other entries are 1140a subclass of this. An important subclass is Entry_blob which takes binary 1141data from a file and places it in the entry. In fact most entry types are 1142subclasses of Entry_blob. 1143 1144Each entry type is a separate file in the tools/binman/etype directory. Each 1145file contains a class called Entry_<type> where <type> is the entry type. 1146New entry types can be supported by adding new files in that directory. 1147These will automatically be detected by binman when needed. 1148 1149Entry properties are documented in entry.py. The entry subclasses are free 1150to change the values of properties to support special behaviour. For example, 1151when Entry_blob loads a file, it sets content_size to the size of the file. 1152Entry classes can adjust other entries. For example, an entry that knows 1153where other entries should be positioned can set up those entries' offsets 1154so they don't need to be set in the binman decription. It can also adjust 1155entry contents. 1156 1157Most of the time such essoteric behaviour is not needed, but it can be 1158essential for complex images. 1159 1160If you need to specify a particular device-tree compiler to use, you can define 1161the DTC environment variable. This can be useful when the system dtc is too 1162old. 1163 1164To enable a full backtrace and other debugging features in binman, pass 1165BINMAN_DEBUG=1 to your build:: 1166 1167 make qemu-x86_defconfig 1168 make BINMAN_DEBUG=1 1169 1170To enable verbose logging from binman, base BINMAN_VERBOSE to your build, which 1171adds a -v<level> option to the call to binman:: 1172 1173 make qemu-x86_defconfig 1174 make BINMAN_VERBOSE=5 1175 1176 1177Building sections in parallel 1178----------------------------- 1179 1180By default binman uses multiprocessing to speed up compilation of large images. 1181This works at a section level, with one thread for each entry in the section. 1182This can speed things up if the entries are large and use compression. 1183 1184This feature can be disabled with the '-T' flag, which defaults to a suitable 1185value for your machine. This depends on the Python version, e.g on v3.8 it uses 118612 threads on an 8-core machine. See ConcurrentFutures_ for more details. 1187 1188The special value -T0 selects single-threaded mode, useful for debugging during 1189development, since dealing with exceptions and problems in threads is more 1190difficult. This avoids any use of ThreadPoolExecutor. 1191 1192 1193History / Credits 1194----------------- 1195 1196Binman takes a lot of inspiration from a Chrome OS tool called 1197'cros_bundle_firmware', which I wrote some years ago. That tool was based on 1198a reasonably simple and sound design but has expanded greatly over the 1199years. In particular its handling of x86 images is convoluted. 1200 1201Quite a few lessons have been learned which are hopefully applied here. 1202 1203 1204Design notes 1205------------ 1206 1207On the face of it, a tool to create firmware images should be fairly simple: 1208just find all the input binaries and place them at the right place in the 1209image. The difficulty comes from the wide variety of input types (simple 1210flat binaries containing code, packaged data with various headers), packing 1211requirments (alignment, spacing, device boundaries) and other required 1212features such as hierarchical images. 1213 1214The design challenge is to make it easy to create simple images, while 1215allowing the more complex cases to be supported. For example, for most 1216images we don't much care exactly where each binary ends up, so we should 1217not have to specify that unnecessarily. 1218 1219New entry types should aim to provide simple usage where possible. If new 1220core features are needed, they can be added in the Entry base class. 1221 1222 1223To do 1224----- 1225 1226Some ideas: 1227 1228- Use of-platdata to make the information available to code that is unable 1229 to use device tree (such as a very small SPL image). For now, limited info is 1230 available via linker symbols 1231- Allow easy building of images by specifying just the board name 1232- Support building an image for a board (-b) more completely, with a 1233 configurable build directory 1234- Detect invalid properties in nodes 1235- Sort the fdtmap by offset 1236- Output temporary files to a different directory 1237 1238-- 1239Simon Glass <sjg@chromium.org> 12407/7/2016 1241 1242.. _ConcurrentFutures: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor 1243