In virtualized environments, virtual CPUs (vCPUs) are commonly oversubscribed on physical CPUs (pCPUs) to utilize CPU resources efficiently. However, excessive vCPU spinning, which occurs when a vCPU is waiting in a spin loop for an event from a descheduled vCPU, greatly degrades application performance in virtualized environments. VM-agnostic hypervisors aim to prevent excessive vCPU spinning by rescheduling vCPUs when an excessive spin is detected by hardware support for virtualization. We investigate the effectiveness of the KVM vCPU scheduler and show that it fails to avoid excessive vCPU spinning under various situations. We identify three problems: 1) scheduler mismatch, 2) aggressive limitation of candidate vCPUs, and 3) IPI context misuse. The first problem stems from the mismatch between the KVM vCPU scheduler and the Linux scheduler. The second and third problems come from failures in choosing candidate vCPUs to be scheduled next. Our in-depth analysis reveals simple modification to KVM (89 LoC) can mitigate excessive vCPU spinning. Our simple modification reduces excessive vCPU spinning by up to 96% and improves benchmark performance by up to 2.6×. Part of the proposed mitigation has been integrated with KVM from Linux KVM v5.13 onward.

In virtualized environments, oversubscribing virtual CPUs (vCPUs) on physical CPUs (pCPUs) is common to utilize CPU resources efficiently. Unfortunately, excessive vCPU spinning, which occurs when a vCPU is waiting in a spin loop for an event from a descheduled vCPU, causes serious performance degradation. Usually, the VM-agnostic hypervisor tries to prevent excessive vCPU spinning by rescheduling vCPUs when an excessive spin is detected by hardware support for virtualization.