1Virtio devices and migration 2============================ 3 4Copyright 2015 IBM Corp. 5 6This work is licensed under the terms of the GNU GPL, version 2 or later. See 7the COPYING file in the top-level directory. 8 9Saving and restoring the state of virtio devices is a bit of a twisty maze, 10for several reasons: 11- state is distributed between several parts: 12 - virtio core, for common fields like features, number of queues, ... 13 - virtio transport (pci, ccw, ...), for the different proxy devices and 14 transport specific state (msix vectors, indicators, ...) 15 - virtio device (net, blk, ...), for the different device types and their 16 state (mac address, request queue, ...) 17- most fields are saved via the stream interface; subsequently, subsections 18 have been added to make cross-version migration possible 19 20This file attempts to document the current procedure and point out some 21caveats. 22 23 24Save state procedure 25==================== 26 27virtio core virtio transport virtio device 28----------- ---------------- ------------- 29 30 save() function registered 31 via register_savevm() 32virtio_save() <---------- 33 ------> save_config() 34 - save proxy device 35 - save transport-specific 36 device fields 37- save common device 38 fields 39- save common virtqueue 40 fields 41 ------> save_queue() 42 - save transport-specific 43 virtqueue fields 44 ------> save_device() 45 - save device-specific 46 fields 47- save subsections 48 - device endianness, 49 if changed from 50 default endianness 51 - 64 bit features, if 52 any high feature bit 53 is set 54 - virtio-1 virtqueue 55 fields, if VERSION_1 56 is set 57 58 59Load state procedure 60==================== 61 62virtio core virtio transport virtio device 63----------- ---------------- ------------- 64 65 load() function registered 66 via register_savevm() 67virtio_load() <---------- 68 ------> load_config() 69 - load proxy device 70 - load transport-specific 71 device fields 72- load common device 73 fields 74- load common virtqueue 75 fields 76 ------> load_queue() 77 - load transport-specific 78 virtqueue fields 79- notify guest 80 ------> load_device() 81 - load device-specific 82 fields 83- load subsections 84 - device endianness 85 - 64 bit features 86 - virtio-1 virtqueue 87 fields 88- sanitize endianness 89- sanitize features 90- virtqueue index sanity 91 check 92 - feature-dependent setup 93 94 95Implications of this setup 96========================== 97 98Devices need to be careful in their state processing during load: The 99load_device() procedure is invoked by the core before subsections have 100been loaded. Any code that depends on information transmitted in subsections 101therefore has to be invoked in the device's load() function _after_ 102virtio_load() returned (like e.g. code depending on features). 103 104Any extension of the state being migrated should be done in subsections 105added to the core for compatibility reasons. If transport or device specific 106state is added, core needs to invoke a callback from the new subsection. 107