1Virtio devices and migration 2============================ 3 4Copyright 2015 IBM Corp. 5 6This work is licensed under the terms of the GNU GPL, version 2 or later. See 7the COPYING file in the top-level directory. 8 9Saving and restoring the state of virtio devices is a bit of a twisty maze, 10for several reasons: 11- state is distributed between several parts: 12 - virtio core, for common fields like features, number of queues, ... 13 - virtio transport (pci, ccw, ...), for the different proxy devices and 14 transport specific state (msix vectors, indicators, ...) 15 - virtio device (net, blk, ...), for the different device types and their 16 state (mac address, request queue, ...) 17- most fields are saved via the stream interface; subsequently, subsections 18 have been added to make cross-version migration possible 19 20This file attempts to document the current procedure and point out some 21caveats. 22 23 24Save state procedure 25==================== 26 27virtio core virtio transport virtio device 28----------- ---------------- ------------- 29 30 save() function registered 31 via VMState wrapper on 32 device class 33virtio_save() <---------- 34 ------> save_config() 35 - save proxy device 36 - save transport-specific 37 device fields 38- save common device 39 fields 40- save common virtqueue 41 fields 42 ------> save_queue() 43 - save transport-specific 44 virtqueue fields 45 ------> save_device() 46 - save device-specific 47 fields 48- save subsections 49 - device endianness, 50 if changed from 51 default endianness 52 - 64 bit features, if 53 any high feature bit 54 is set 55 - virtio-1 virtqueue 56 fields, if VERSION_1 57 is set 58 59 60Load state procedure 61==================== 62 63virtio core virtio transport virtio device 64----------- ---------------- ------------- 65 66 load() function registered 67 via VMState wrapper on 68 device class 69virtio_load() <---------- 70 ------> load_config() 71 - load proxy device 72 - load transport-specific 73 device fields 74- load common device 75 fields 76- load common virtqueue 77 fields 78 ------> load_queue() 79 - load transport-specific 80 virtqueue fields 81- notify guest 82 ------> load_device() 83 - load device-specific 84 fields 85- load subsections 86 - device endianness 87 - 64 bit features 88 - virtio-1 virtqueue 89 fields 90- sanitize endianness 91- sanitize features 92- virtqueue index sanity 93 check 94 - feature-dependent setup 95 96 97Implications of this setup 98========================== 99 100Devices need to be careful in their state processing during load: The 101load_device() procedure is invoked by the core before subsections have 102been loaded. Any code that depends on information transmitted in subsections 103therefore has to be invoked in the device's load() function _after_ 104virtio_load() returned (like e.g. code depending on features). 105 106Any extension of the state being migrated should be done in subsections 107added to the core for compatibility reasons. If transport or device specific 108state is added, core needs to invoke a callback from the new subsection. 109