Attempts to perform a live migration of an instance fails with the following error.
Migration pre-check error: CPU doesn't have compatibility. internal error: Unknown CPU feature ssbd Refer to http://libvirt.org/html/libvirt-libvirt-host.html#virCPUCompareResult
- Platform9 Managed OpenStack - All Versions
- Red Hat Enterprise Linux
The instance being migrated may require a feature not available on the destination host. In this case, ssbd refers to "Speculative Store Bypass Disable", a Spectre vulnerability mitigation technique available in certain Red Hat kernels and presented as a CPU feature or capability on patched kernels.
When a virtual machine instance is created on a compute node with a kernel that is patched against a side-channel attack using speculative store bypass, subsequent migrations or resizes to other compute nodes may fail if that host has not been patched.
Required features for a virtual machine instance can be found using the
virsh dumpxml <domain> command shown here.
[root@host1 ~]# virsh dumpxml 101 | grep require
<feature policy='require' name='ss'/>
<feature policy='require' name='stibp'/>
<feature policy='require' name='ssbd'/>
<feature policy='require' name='pdpe1gb'/>
<feature policy='require' name='hypervisor'/>
- Verify that all hosts have the
ssbdCPU flag present.
If patched, the output from cat /proc/cpuinfo will reflect
ssbdas a CPU flag.
[root@host1 ~]# cat /proc/cpuinfoIf patched, the output from
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 79
model name : Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz
stepping : 1
microcode : 0xb00002e
cpu MHz : 3200.012
cache size : 35840 KB
physical id : 0
siblings : 28
core id : 0
cpu cores : 14
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 20
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand la
hf_lm abm 3dnowprefetch epb cat_l3 cdp_l3 intel_ppin intel_pt ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap xsaveop
t cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts spec_ctrl intel_stibp flush_l1d
bogomips : 5187.86
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
virsh capabilitieswill also reflect the
[root@host1 ~]# virsh capabilities | grep ssbdUnpatched hosts will not reflect the feature.
[root@host2 ~]# virsh capabilities | grep ssbd
If patched, a file system path can be used to determine the state of Speculative Store Bypass:[root@host1 ~]# cat /sys/devices/system/cpu/vulnerabilities/spec_store_bypassUnpatched hosts may reflect two different states, depending on the kernel version.
Mitigation: Speculative Store Bypass disabled via prctl and seccomp
[root@host2 ~]# cat /sys/devices/system/cpu/vulnerabilities/spec_store_bypass
cat: /sys/devices/system/cpu/vulnerabilities/spec_store_bypass: No such file or directorynon-zero return code
[root@host3 ~]# cat /sys/devices/system/cpu/vulnerabilities/spec_store_bypass
- If necessary, upgrade the kernel to a patched version.
The following kernels have been observed as providing Speculative Store Bypass mitigation.
3.10.0-862.14.4.el7.x86_64Unpatched kernels may include the following.
3.10.0-693.el7.x86_64Patched kernels may be provided during RHEL upgrades or installed manually. Upgrading from Red Hat Enterprise Linux 7.4 to Red Hat Enterprise Linux 7.5 should provide a patched kernel and allow migrations and resizes that previously failed due to this issue to complete properly.
For more information on how Red Hat addresses kernel side-channel attacks using Speculative Store Bypass, please refer to Kernel Side-Channel Attack using Speculative Store Bypass - CVE-2018-3639.
If a kernel upgrade is not possible, or if the error is unrelated to the ssbd CPU feature, an instance's XML file can be modified manually to allow for a successful migration between hosts with different capabilities. Please refer to Live Migration Fails With Error: "Unacceptable CPU Info: CPU Doesn't Have Compatibility."