1================ 2ARM CPU Features 3================ 4 5Examples of probing and using ARM CPU features 6 7Introduction 8============ 9 10CPU features are optional features that a CPU of supporting type may 11choose to implement or not. In QEMU, optional CPU features have 12corresponding boolean CPU proprieties that, when enabled, indicate 13that the feature is implemented, and, conversely, when disabled, 14indicate that it is not implemented. An example of an ARM CPU feature 15is the Performance Monitoring Unit (PMU). CPU types such as the 16Cortex-A15 and the Cortex-A57, which respectively implement ARM 17architecture reference manuals ARMv7-A and ARMv8-A, may both optionally 18implement PMUs. For example, if a user wants to use a Cortex-A15 without 19a PMU, then the `-cpu` parameter should contain `pmu=off` on the QEMU 20command line, i.e. `-cpu cortex-a15,pmu=off`. 21 22As not all CPU types support all optional CPU features, then whether or 23not a CPU property exists depends on the CPU type. For example, CPUs 24that implement the ARMv8-A architecture reference manual may optionally 25support the AArch32 CPU feature, which may be enabled by disabling the 26`aarch64` CPU property. A CPU type such as the Cortex-A15, which does 27not implement ARMv8-A, will not have the `aarch64` CPU property. 28 29QEMU's support may be limited for some CPU features, only partially 30supporting the feature or only supporting the feature under certain 31configurations. For example, the `aarch64` CPU feature, which, when 32disabled, enables the optional AArch32 CPU feature, is only supported 33when using the KVM accelerator and when running on a host CPU type that 34supports the feature. 35 36CPU Feature Probing 37=================== 38 39Determining which CPU features are available and functional for a given 40CPU type is possible with the `query-cpu-model-expansion` QMP command. 41Below are some examples where `scripts/qmp/qmp-shell` (see the top comment 42block in the script for usage) is used to issue the QMP commands. 43 44(1) Determine which CPU features are available for the `max` CPU type 45 (Note, we started QEMU with qemu-system-aarch64, so `max` is 46 implementing the ARMv8-A reference manual in this case):: 47 48 (QEMU) query-cpu-model-expansion type=full model={"name":"max"} 49 { "return": { 50 "model": { "name": "max", "props": { 51 "sve1664": true, "pmu": true, "sve1792": true, "sve1920": true, 52 "sve128": true, "aarch64": true, "sve1024": true, "sve": true, 53 "sve640": true, "sve768": true, "sve1408": true, "sve256": true, 54 "sve1152": true, "sve512": true, "sve384": true, "sve1536": true, 55 "sve896": true, "sve1280": true, "sve2048": true 56 }}}} 57 58We see that the `max` CPU type has the `pmu`, `aarch64`, `sve`, and many 59`sve<N>` CPU features. We also see that all the CPU features are 60enabled, as they are all `true`. (The `sve<N>` CPU features are all 61optional SVE vector lengths (see "SVE CPU Properties"). While with TCG 62all SVE vector lengths can be supported, when KVM is in use it's more 63likely that only a few lengths will be supported, if SVE is supported at 64all.) 65 66(2) Let's try to disable the PMU:: 67 68 (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"pmu":false}} 69 { "return": { 70 "model": { "name": "max", "props": { 71 "sve1664": true, "pmu": false, "sve1792": true, "sve1920": true, 72 "sve128": true, "aarch64": true, "sve1024": true, "sve": true, 73 "sve640": true, "sve768": true, "sve1408": true, "sve256": true, 74 "sve1152": true, "sve512": true, "sve384": true, "sve1536": true, 75 "sve896": true, "sve1280": true, "sve2048": true 76 }}}} 77 78We see it worked, as `pmu` is now `false`. 79 80(3) Let's try to disable `aarch64`, which enables the AArch32 CPU feature:: 81 82 (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"aarch64":false}} 83 {"error": { 84 "class": "GenericError", "desc": 85 "'aarch64' feature cannot be disabled unless KVM is enabled and 32-bit EL1 is supported" 86 }} 87 88It looks like this feature is limited to a configuration we do not 89currently have. 90 91(4) Let's disable `sve` and see what happens to all the optional SVE 92 vector lengths:: 93 94 (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"sve":false}} 95 { "return": { 96 "model": { "name": "max", "props": { 97 "sve1664": false, "pmu": true, "sve1792": false, "sve1920": false, 98 "sve128": false, "aarch64": true, "sve1024": false, "sve": false, 99 "sve640": false, "sve768": false, "sve1408": false, "sve256": false, 100 "sve1152": false, "sve512": false, "sve384": false, "sve1536": false, 101 "sve896": false, "sve1280": false, "sve2048": false 102 }}}} 103 104As expected they are now all `false`. 105 106(5) Let's try probing CPU features for the Cortex-A15 CPU type:: 107 108 (QEMU) query-cpu-model-expansion type=full model={"name":"cortex-a15"} 109 {"return": {"model": {"name": "cortex-a15", "props": {"pmu": true}}}} 110 111Only the `pmu` CPU feature is available. 112 113A note about CPU feature dependencies 114------------------------------------- 115 116It's possible for features to have dependencies on other features. I.e. 117it may be possible to change one feature at a time without error, but 118when attempting to change all features at once an error could occur 119depending on the order they are processed. It's also possible changing 120all at once doesn't generate an error, because a feature's dependencies 121are satisfied with other features, but the same feature cannot be changed 122independently without error. For these reasons callers should always 123attempt to make their desired changes all at once in order to ensure the 124collection is valid. 125 126A note about CPU models and KVM 127------------------------------- 128 129Named CPU models generally do not work with KVM. There are a few cases 130that do work, e.g. using the named CPU model `cortex-a57` with KVM on a 131seattle host, but mostly if KVM is enabled the `host` CPU type must be 132used. This means the guest is provided all the same CPU features as the 133host CPU type has. And, for this reason, the `host` CPU type should 134enable all CPU features that the host has by default. Indeed it's even 135a bit strange to allow disabling CPU features that the host has when using 136the `host` CPU type, but in the absence of CPU models it's the best we can 137do if we want to launch guests without all the host's CPU features enabled. 138 139Enabling KVM also affects the `query-cpu-model-expansion` QMP command. The 140affect is not only limited to specific features, as pointed out in example 141(3) of "CPU Feature Probing", but also to which CPU types may be expanded. 142When KVM is enabled, only the `max`, `host`, and current CPU type may be 143expanded. This restriction is necessary as it's not possible to know all 144CPU types that may work with KVM, but it does impose a small risk of users 145experiencing unexpected errors. For example on a seattle, as mentioned 146above, the `cortex-a57` CPU type is also valid when KVM is enabled. 147Therefore a user could use the `host` CPU type for the current type, but 148then attempt to query `cortex-a57`, however that query will fail with our 149restrictions. This shouldn't be an issue though as management layers and 150users have been preferring the `host` CPU type for use with KVM for quite 151some time. Additionally, if the KVM-enabled QEMU instance running on a 152seattle host is using the `cortex-a57` CPU type, then querying `cortex-a57` 153will work. 154 155Using CPU Features 156================== 157 158After determining which CPU features are available and supported for a 159given CPU type, then they may be selectively enabled or disabled on the 160QEMU command line with that CPU type:: 161 162 $ qemu-system-aarch64 -M virt -cpu max,pmu=off,sve=on,sve128=on,sve256=on 163 164The example above disables the PMU and enables the first two SVE vector 165lengths for the `max` CPU type. Note, the `sve=on` isn't actually 166necessary, because, as we observed above with our probe of the `max` CPU 167type, `sve` is already on by default. Also, based on our probe of 168defaults, it would seem we need to disable many SVE vector lengths, rather 169than only enabling the two we want. This isn't the case, because, as 170disabling many SVE vector lengths would be quite verbose, the `sve<N>` CPU 171properties have special semantics (see "SVE CPU Property Parsing 172Semantics"). 173 174SVE CPU Properties 175================== 176 177There are two types of SVE CPU properties: `sve` and `sve<N>`. The first 178is used to enable or disable the entire SVE feature, just as the `pmu` 179CPU property completely enables or disables the PMU. The second type 180is used to enable or disable specific vector lengths, where `N` is the 181number of bits of the length. The `sve<N>` CPU properties have special 182dependencies and constraints, see "SVE CPU Property Dependencies and 183Constraints" below. Additionally, as we want all supported vector lengths 184to be enabled by default, then, in order to avoid overly verbose command 185lines (command lines full of `sve<N>=off`, for all `N` not wanted), we 186provide the parsing semantics listed in "SVE CPU Property Parsing 187Semantics". 188 189SVE CPU Property Dependencies and Constraints 190--------------------------------------------- 191 192 1) At least one vector length must be enabled when `sve` is enabled. 193 194 2) If a vector length `N` is enabled, then, when KVM is enabled, all 195 smaller, host supported vector lengths must also be enabled. If 196 KVM is not enabled, then only all the smaller, power-of-two vector 197 lengths must be enabled. E.g. with KVM if the host supports all 198 vector lengths up to 512-bits (128, 256, 384, 512), then if `sve512` 199 is enabled, the 128-bit vector length, 256-bit vector length, and 200 384-bit vector length must also be enabled. Without KVM, the 384-bit 201 vector length would not be required. 202 203 3) If KVM is enabled then only vector lengths that the host CPU type 204 support may be enabled. If SVE is not supported by the host, then 205 no `sve*` properties may be enabled. 206 207SVE CPU Property Parsing Semantics 208---------------------------------- 209 210 1) If SVE is disabled (`sve=off`), then which SVE vector lengths 211 are enabled or disabled is irrelevant to the guest, as the entire 212 SVE feature is disabled and that disables all vector lengths for 213 the guest. However QEMU will still track any `sve<N>` CPU 214 properties provided by the user. If later an `sve=on` is provided, 215 then the guest will get only the enabled lengths. If no `sve=on` 216 is provided and there are explicitly enabled vector lengths, then 217 an error is generated. 218 219 2) If SVE is enabled (`sve=on`), but no `sve<N>` CPU properties are 220 provided, then all supported vector lengths are enabled, which when 221 KVM is not in use means including the non-power-of-two lengths, and, 222 when KVM is in use, it means all vector lengths supported by the host 223 processor. 224 225 3) If SVE is enabled, then an error is generated when attempting to 226 disable the last enabled vector length (see constraint (1) of "SVE 227 CPU Property Dependencies and Constraints"). 228 229 4) If one or more vector lengths have been explicitly enabled and at 230 at least one of the dependency lengths of the maximum enabled length 231 has been explicitly disabled, then an error is generated (see 232 constraint (2) of "SVE CPU Property Dependencies and Constraints"). 233 234 5) When KVM is enabled, if the host does not support SVE, then an error 235 is generated when attempting to enable any `sve*` properties (see 236 constraint (3) of "SVE CPU Property Dependencies and Constraints"). 237 238 6) When KVM is enabled, if the host does support SVE, then an error is 239 generated when attempting to enable any vector lengths not supported 240 by the host (see constraint (3) of "SVE CPU Property Dependencies and 241 Constraints"). 242 243 7) If one or more `sve<N>` CPU properties are set `off`, but no `sve<N>`, 244 CPU properties are set `on`, then the specified vector lengths are 245 disabled but the default for any unspecified lengths remains enabled. 246 When KVM is not enabled, disabling a power-of-two vector length also 247 disables all vector lengths larger than the power-of-two length. 248 When KVM is enabled, then disabling any supported vector length also 249 disables all larger vector lengths (see constraint (2) of "SVE CPU 250 Property Dependencies and Constraints"). 251 252 8) If one or more `sve<N>` CPU properties are set to `on`, then they 253 are enabled and all unspecified lengths default to disabled, except 254 for the required lengths per constraint (2) of "SVE CPU Property 255 Dependencies and Constraints", which will even be auto-enabled if 256 they were not explicitly enabled. 257 258 9) If SVE was disabled (`sve=off`), allowing all vector lengths to be 259 explicitly disabled (i.e. avoiding the error specified in (3) of 260 "SVE CPU Property Parsing Semantics"), then if later an `sve=on` is 261 provided an error will be generated. To avoid this error, one must 262 enable at least one vector length prior to enabling SVE. 263 264SVE CPU Property Examples 265------------------------- 266 267 1) Disable SVE:: 268 269 $ qemu-system-aarch64 -M virt -cpu max,sve=off 270 271 2) Implicitly enable all vector lengths for the `max` CPU type:: 272 273 $ qemu-system-aarch64 -M virt -cpu max 274 275 3) When KVM is enabled, implicitly enable all host CPU supported vector 276 lengths with the `host` CPU type:: 277 278 $ qemu-system-aarch64 -M virt,accel=kvm -cpu host 279 280 4) Only enable the 128-bit vector length:: 281 282 $ qemu-system-aarch64 -M virt -cpu max,sve128=on 283 284 5) Disable the 512-bit vector length and all larger vector lengths, 285 since 512 is a power-of-two. This results in all the smaller, 286 uninitialized lengths (128, 256, and 384) defaulting to enabled:: 287 288 $ qemu-system-aarch64 -M virt -cpu max,sve512=off 289 290 6) Enable the 128-bit, 256-bit, and 512-bit vector lengths:: 291 292 $ qemu-system-aarch64 -M virt -cpu max,sve128=on,sve256=on,sve512=on 293 294 7) The same as (6), but since the 128-bit and 256-bit vector 295 lengths are required for the 512-bit vector length to be enabled, 296 then allow them to be auto-enabled:: 297 298 $ qemu-system-aarch64 -M virt -cpu max,sve512=on 299 300 8) Do the same as (7), but by first disabling SVE and then re-enabling it:: 301 302 $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve512=on,sve=on 303 304 9) Force errors regarding the last vector length:: 305 306 $ qemu-system-aarch64 -M virt -cpu max,sve128=off 307 $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve128=off,sve=on 308 309SVE CPU Property Recommendations 310-------------------------------- 311 312The examples in "SVE CPU Property Examples" exhibit many ways to select 313vector lengths which developers may find useful in order to avoid overly 314verbose command lines. However, the recommended way to select vector 315lengths is to explicitly enable each desired length. Therefore only 316example's (1), (4), and (6) exhibit recommended uses of the properties. 317 318