1 The QEMU build system architecture 2 ================================== 3 4This document aims to help developers understand the architecture of the 5QEMU build system. As with projects using GNU autotools, the QEMU build 6system has two stages, first the developer runs the "configure" script 7to determine the local build environment characteristics, then they run 8"make" to build the project. There is about where the similarities with 9GNU autotools end, so try to forget what you know about them. 10 11 12Stage 1: configure 13================== 14 15The QEMU configure script is written directly in shell, and should be 16compatible with any POSIX shell, hence it uses #!/bin/sh. An important 17implication of this is that it is important to avoid using bash-isms on 18development platforms where bash is the primary host. 19 20In contrast to autoconf scripts, QEMU's configure is expected to be 21silent while it is checking for features. It will only display output 22when an error occurs, or to show the final feature enablement summary 23on completion. 24 25Adding new checks to the configure script usually comprises the 26following tasks: 27 28 - Initialize one or more variables with the default feature state. 29 30 Ideally features should auto-detect whether they are present, 31 so try to avoid hardcoding the initial state to either enabled 32 or disabled, as that forces the user to pass a --enable-XXX 33 / --disable-XXX flag on every invocation of configure. 34 35 - Add support to the command line arg parser to handle any new 36 --enable-XXX / --disable-XXX flags required by the feature XXX. 37 38 - Add information to the help output message to report on the new 39 feature flag. 40 41 - Add code to perform the actual feature check. As noted above, try to 42 be fully dynamic in checking enablement/disablement. 43 44 - Add code to print out the feature status in the configure summary 45 upon completion. 46 47 - Add any new makefile variables to $config_host_mak on completion. 48 49 50Taking (a simplified version of) the probe for gnutls from configure, 51we have the following pieces: 52 53 # Initial variable state 54 gnutls="" 55 56 ..snip.. 57 58 # Configure flag processing 59 --disable-gnutls) gnutls="no" 60 ;; 61 --enable-gnutls) gnutls="yes" 62 ;; 63 64 ..snip.. 65 66 # Help output feature message 67 gnutls GNUTLS cryptography support 68 69 ..snip.. 70 71 # Test for gnutls 72 if test "$gnutls" != "no"; then 73 if ! $pkg_config --exists "gnutls"; then 74 gnutls_cflags=`$pkg_config --cflags gnutls` 75 gnutls_libs=`$pkg_config --libs gnutls` 76 libs_softmmu="$gnutls_libs $libs_softmmu" 77 libs_tools="$gnutls_libs $libs_tools" 78 QEMU_CFLAGS="$QEMU_CFLAGS $gnutls_cflags" 79 gnutls="yes" 80 elif test "$gnutls" = "yes"; then 81 feature_not_found "gnutls" "Install gnutls devel" 82 else 83 gnutls="no" 84 fi 85 fi 86 87 ..snip.. 88 89 # Completion feature summary 90 echo "GNUTLS support $gnutls" 91 92 ..snip.. 93 94 # Define make variables 95 if test "$gnutls" = "yes" ; then 96 echo "CONFIG_GNUTLS=y" >> $config_host_mak 97 fi 98 99 100Helper functions 101---------------- 102 103The configure script provides a variety of helper functions to assist 104developers in checking for system features: 105 106 - do_cc $ARGS... 107 108 Attempt to run the system C compiler passing it $ARGS... 109 110 - do_cxx $ARGS... 111 112 Attempt to run the system C++ compiler passing it $ARGS... 113 114 - compile_object $CFLAGS 115 116 Attempt to compile a test program with the system C compiler using 117 $CFLAGS. The test program must have been previously written to a file 118 called $TMPC. 119 120 - compile_prog $CFLAGS $LDFLAGS 121 122 Attempt to compile a test program with the system C compiler using 123 $CFLAGS and link it with the system linker using $LDFLAGS. The test 124 program must have been previously written to a file called $TMPC. 125 126 - has $COMMAND 127 128 Determine if $COMMAND exists in the current environment, either as a 129 shell builtin, or executable binary, returning 0 on success. 130 131 - path_of $COMMAND 132 133 Return the fully qualified path of $COMMAND, printing it to stdout, 134 and returning 0 on success. 135 136 - check_define $NAME 137 138 Determine if the macro $NAME is defined by the system C compiler 139 140 - check_include $NAME 141 142 Determine if the include $NAME file is available to the system C 143 compiler 144 145 - write_c_skeleton 146 147 Write a minimal C program main() function to the temporary file 148 indicated by $TMPC 149 150 - feature_not_found $NAME $REMEDY 151 152 Print a message to stderr that the feature $NAME was not available 153 on the system, suggesting the user try $REMEDY to address the 154 problem. 155 156 - error_exit $MESSAGE $MORE... 157 158 Print $MESSAGE to stderr, followed by $MORE... and then exit from the 159 configure script with non-zero status 160 161 - query_pkg_config $ARGS... 162 163 Run pkg-config passing it $ARGS. If QEMU is doing a static build, 164 then --static will be automatically added to $ARGS 165 166 167Stage 2: makefiles 168================== 169 170The use of GNU make is required with the QEMU build system. 171 172Although the source code is spread across multiple subdirectories, the 173build system should be considered largely non-recursive in nature, in 174contrast to common practices seen with automake. There is some recursive 175invocation of make, but this is related to the things being built, 176rather than the source directory structure. 177 178QEMU currently supports both VPATH and non-VPATH builds, so there are 179three general ways to invoke configure & perform a build. 180 181 - VPATH, build artifacts outside of QEMU source tree entirely 182 183 cd ../ 184 mkdir build 185 cd build 186 ../qemu/configure 187 make 188 189 - VPATH, build artifacts in a subdir of QEMU source tree 190 191 mkdir build 192 cd build 193 ../configure 194 make 195 196 - non-VPATH, build artifacts everywhere 197 198 ./configure 199 make 200 201The QEMU maintainers generally recommend that a VPATH build is used by 202developers. Patches to QEMU are expected to ensure VPATH build still 203works. 204 205 206Module structure 207---------------- 208 209There are a number of key outputs of the QEMU build system: 210 211 - Tools - qemu-img, qemu-nbd, qga (guest agent), etc 212 - System emulators - qemu-system-$ARCH 213 - Userspace emulators - qemu-$ARCH 214 - Unit tests 215 216The source code is highly modularized, split across many files to 217facilitate building of all of these components with as little duplicated 218compilation as possible. There can be considered to be two distinct 219groups of files, those which are independent of the QEMU emulation 220target and those which are dependent on the QEMU emulation target. 221 222In the target-independent set lives various general purpose helper code, 223such as error handling infrastructure, standard data structures, 224platform portability wrapper functions, etc. This code can be compiled 225once only and the .o files linked into all output binaries. 226 227In the target-dependent set lives CPU emulation, device emulation and 228much glue code. This sometimes also has to be compiled multiple times, 229once for each target being built. 230 231The utility code that is used by all binaries is built into a 232static archive called libqemuutil.a, which is then linked to all the 233binaries. In order to provide hooks that are only needed by some of the 234binaries, code in libqemuutil.a may depend on other functions that are 235not fully implemented by all QEMU binaries. To deal with this there is a 236second library called libqemustub.a which provides dummy stubs for all 237these functions. These will get lazy linked into the binary if the real 238implementation is not present. In this way, the libqemustub.a static 239library can be thought of as a portable implementation of the weak 240symbols concept. All binaries should link to both libqemuutil.a and 241libqemustub.a. e.g. 242 243 qemu-img$(EXESUF): qemu-img.o ..snip.. libqemuutil.a libqemustub.a 244 245 246Windows platform portability 247---------------------------- 248 249On Windows, all binaries have the suffix '.exe', so all Makefile rules 250which create binaries must include the $(EXESUF) variable on the binary 251name. e.g. 252 253 qemu-img$(EXESUF): qemu-img.o ..snip.. 254 255This expands to '.exe' on Windows, or '' on other platforms. 256 257A further complication for the system emulator binaries is that 258two separate binaries need to be generated. 259 260The main binary (e.g. qemu-system-x86_64.exe) is linked against the 261Windows console runtime subsystem. These are expected to be run from a 262command prompt window, and so will print stderr to the console that 263launched them. 264 265The second binary generated has a 'w' on the end of its name (e.g. 266qemu-system-x86_64w.exe) and is linked against the Windows graphical 267runtime subsystem. These are expected to be run directly from the 268desktop and will open up a dedicated console window for stderr output. 269 270The Makefile.target will generate the binary for the graphical subsystem 271first, and then use objcopy to relink it against the console subsystem 272to generate the second binary. 273 274 275Object variable naming 276---------------------- 277 278The QEMU convention is to define variables to list different groups of 279object files. These are named with the convention $PREFIX-obj-y. For 280example the libqemuutil.a file will be linked with all objects listed 281in a variable 'util-obj-y'. So, for example, util/Makefile.obj will 282contain a set of definitions looking like 283 284 util-obj-y += bitmap.o bitops.o hbitmap.o 285 util-obj-y += fifo8.o 286 util-obj-y += acl.o 287 util-obj-y += error.o qemu-error.o 288 289When there is an object file which needs to be conditionally built based 290on some characteristic of the host system, the configure script will 291define a variable for the conditional. For example, on Windows it will 292define $(CONFIG_POSIX) with a value of 'n' and $(CONFIG_WIN32) with a 293value of 'y'. It is now possible to use the config variables when 294listing object files. For example, 295 296 util-obj-$(CONFIG_WIN32) += oslib-win32.o qemu-thread-win32.o 297 util-obj-$(CONFIG_POSIX) += oslib-posix.o qemu-thread-posix.o 298 299On Windows this expands to 300 301 util-obj-y += oslib-win32.o qemu-thread-win32.o 302 util-obj-n += oslib-posix.o qemu-thread-posix.o 303 304Since libqemutil.a links in $(util-obj-y), the POSIX specific files 305listed against $(util-obj-n) are ignored on the Windows platform builds. 306 307 308CFLAGS / LDFLAGS / LIBS handling 309-------------------------------- 310 311There are many different binaries being built with differing purposes, 312and some of them might even be 3rd party libraries pulled in via git 313submodules. As such the use of the global CFLAGS variable is generally 314avoided in QEMU, since it would apply to too many build targets. 315 316Flags that are needed by any QEMU code (i.e. everything *except* GIT 317submodule projects) are put in $(QEMU_CFLAGS) variable. For linker 318flags the $(LIBS) variable is sometimes used, but a couple of more 319targeted variables are preferred. $(libs_softmmu) is used for 320libraries that must be linked to system emulator targets, $(LIBS_TOOLS) 321is used for tools like qemu-img, qemu-nbd, etc and $(LIBS_QGA) is used 322for the QEMU guest agent. There is currently no specific variable for 323the userspace emulator targets as the global $(LIBS), or more targeted 324variables shown below, are sufficient. 325 326In addition to these variables, it is possible to provide cflags and 327libs against individual source code files, by defining variables of the 328form $FILENAME-cflags and $FILENAME-libs. For example, the curl block 329driver needs to link to the libcurl library, so block/Makefile defines 330some variables: 331 332 curl.o-cflags := $(CURL_CFLAGS) 333 curl.o-libs := $(CURL_LIBS) 334 335The scope is a little different between the two variables. The libs get 336used when linking any target binary that includes the curl.o object 337file, while the cflags get used when compiling the curl.c file only. 338 339 340Statically defined files 341------------------------ 342 343The following key files are statically defined in the source tree, with 344the rules needed to build QEMU. Their behaviour is influenced by a 345number of dynamically created files listed later. 346 347- Makefile 348 349The main entry point used when invoking make to build all the components 350of QEMU. The default 'all' target will naturally result in the build of 351every component. The various tools and helper binaries are built 352directly via a non-recursive set of rules. 353 354Each system/userspace emulation target needs to have a slightly 355different set of make rules / variables. Thus, make will be recursively 356invoked for each of the emulation targets. 357 358The recursive invocation will end up processing the toplevel 359Makefile.target file (more on that later). 360 361 362- */Makefile.objs 363 364Since the source code is spread across multiple directories, the rules 365for each file are similarly modularized. Thus each subdirectory 366containing .c files will usually also contain a Makefile.objs file. 367These files are not directly invoked by a recursive make, but instead 368they are imported by the top level Makefile and/or Makefile.target 369 370Each Makefile.objs usually just declares a set of variables listing the 371.o files that need building from the source files in the directory. They 372will also define any custom linker or compiler flags. For example in 373block/Makefile.objs 374 375 block-obj-$(CONFIG_LIBISCSI) += iscsi.o 376 block-obj-$(CONFIG_CURL) += curl.o 377 378 ..snip... 379 380 iscsi.o-cflags := $(LIBISCSI_CFLAGS) 381 iscsi.o-libs := $(LIBISCSI_LIBS) 382 curl.o-cflags := $(CURL_CFLAGS) 383 curl.o-libs := $(CURL_LIBS) 384 385If there are any rules defined in the Makefile.objs file, they should 386all use $(obj) as a prefix to the target, e.g. 387 388 $(obj)/generated-tcg-tracers.h: $(obj)/generated-tcg-tracers.h-timestamp 389 390 391- Makefile.target 392 393This file provides the entry point used to build each individual system 394or userspace emulator target. Each enabled target has its own 395subdirectory. For example if configure is run with the argument 396'--target-list=x86_64-softmmu', then a sub-directory 'x86_64-softmu' 397will be created, containing a 'Makefile' which symlinks back to 398Makefile.target 399 400So when the recursive '$(MAKE) -C x86_64-softmmu' is invoked, it ends up 401using Makefile.target for the build rules. 402 403 404- rules.mak 405 406This file provides the generic helper rules for invoking build tools, in 407particular the compiler and linker. This also contains the magic (hairy) 408'unnest-vars' function which is used to merge the variable definitions 409from all Makefile.objs in the source tree down into the main Makefile 410context. 411 412 413- default-configs/*.mak 414 415The files under default-configs/ control what emulated hardware is built 416into each QEMU system and userspace emulator targets. They merely 417contain a long list of config variable definitions. For example, 418default-configs/x86_64-softmmu.mak has: 419 420 include pci.mak 421 include sound.mak 422 include usb.mak 423 CONFIG_QXL=$(CONFIG_SPICE) 424 CONFIG_VGA_ISA=y 425 CONFIG_VGA_CIRRUS=y 426 CONFIG_VMWARE_VGA=y 427 CONFIG_VIRTIO_VGA=y 428 ...snip... 429 430These files rarely need changing unless new devices / hardware need to 431be enabled for a particular system/userspace emulation target 432 433 434- tests/Makefile 435 436Rules for building the unit tests. This file is included directly by the 437top level Makefile, so anything defined in this file will influence the 438entire build system. Care needs to be taken when writing rules for tests 439to ensure they only apply to the unit test execution / build. 440 441 442- po/Makefile 443 444Rules for building and installing the binary message catalogs from the 445text .po file sources. This almost never needs changing for any reason. 446 447 448Dynamically created files 449------------------------- 450 451The following files are generated dynamically by configure in order to 452control the behaviour of the statically defined makefiles. This avoids 453the need for QEMU makefiles to go through any pre-processing as seen 454with autotools, where Makefile.am generates Makefile.in which generates 455Makefile. 456 457 458- config-host.mak 459 460When configure has determined the characteristics of the build host it 461will write a long list of variables to config-host.mak file. This 462provides the various install directories, compiler / linker flags and a 463variety of CONFIG_* variables related to optionally enabled features. 464This is imported by the top level Makefile in order to tailor the build 465output. 466 467The variables defined here are those which are applicable to all QEMU 468build outputs. Variables which are potentially different for each 469emulator target are defined by the next file... 470 471It is also used as a dependency checking mechanism. If make sees that 472the modification timestamp on configure is newer than that on 473config-host.mak, then configure will be re-run. 474 475 476- config-host.h 477 478The config-host.h file is used by source code to determine what features 479are enabled. It is generated from the contents of config-host.mak using 480the scripts/create_config program. This extracts all the CONFIG_* variables, 481most of the HOST_* variables and a few other misc variables from 482config-host.mak, formatting them as C preprocessor macros. 483 484 485- $TARGET-NAME/config-target.mak 486 487TARGET-NAME is the name of a system or userspace emulator, for example, 488x86_64-softmmu denotes the system emulator for the x86_64 architecture. 489This file contains the variables which need to vary on a per-target 490basis. For example, it will indicate whether KVM or Xen are enabled for 491the target and any other potential custom libraries needed for linking 492the target. 493 494 495- $TARGET-NAME/config-devices.mak 496 497TARGET-NAME is again the name of a system or userspace emulator. The 498config-devices.mak file is automatically generated by make using the 499scripts/make_device_config.sh program, feeding it the 500default-configs/$TARGET-NAME file as input. 501 502 503- $TARGET-NAME/Makefile 504 505This is the entrypoint used when make recurses to build a single system 506or userspace emulator target. It is merely a symlink back to the 507Makefile.target in the top level. 508