Semantic top-K selection with cross-encoder rerankers underpins on-device AI services, such as retrieval-augmented generation, agent memory, and personalized recommendation. However, its latency and memory demands dominate end-to-end budgets on edge hardware. Revisiting the objective of top-K selection, we reveal that only relative rankings matter, not exact per-candidate scores. We further observe sequence-level sparsity: relative rankings progressively stabilize in intermediate layers, enabling early pruning prior to completing full inference.

With the popularity of confidential computing, confidential virtual machines (CVMs) have been widely adopted and they guarantee strong security by hardware. However, there still exist some problems in memory management in CVMs. Since private memory pages of CVMs are encrypted and cannot be accessed by hypervisors, existing CVMs employ static page management to avoid crashes due to the relocation of encrypted memory pages, leading cloud platforms managing CVMs to face more severe memory management pressures than before. Memory ballooning, as an efficient, flexible, and highly compatible memory management mechanism in virtualization, is not available in CVMs based on SEV (Secure Encrypted Virtualization). In this paper, we analyze the design of SEV CVMs and memory ballooning, and enable memory ballooning on SEV CVMs by substituting static page management with dynamic page management and addressing communication issues between the guest frontend and the host backend. Besides, we propose three performance optimization strategies for memory ballooning on SEV CVMs, including asynchronous reclaiming on the host side, an additional shadow vCPU on the guest side, and accelerating cache flushing operations in the host kernel. Experiments show that the time cost of reclaiming memory from SEV CVMs by memory ballooning can be reduced by up to 38 times and up to 55% overhead caused by delaying reclamation in real-world applications like MYSQL can be eliminated.

With the rise of cloud computing, the use of virtual machines in data centers has become genuinely common. To achieve load balance and disaster recovery between different hosts when a virtual machine is providing services, the concept of live migration has been proposed and has now become one of the essential capabilities in data centers. On the other hand, users’ concerns about data privacy when applications are deployed on cloud vendors’ servers are also rising, and for this reason, confidential virtual machines, a new type of virtualization hardware, have been proposed. However, since the memory of confidential VMs is usually encrypted, existing live migration solutions cannot be applied to confidential VMs directly, so hardware vendors have come up with various solutions to support this critical feature. For example, the mainstream confidential virtualization hardware platform – AMD-SEV, adds a series of interfaces in their secure firmware to assist the hypervisor to achieve live migration capability. However, this solution can cause the live migration time to be too long due to the limited number of secure processors and computing power. In this paper, we propose a new software design: PROMETHEUSMIGRATE. PROMETHEUSMIGRATE presents a novel software abstraction that bypasses SEV’s secure firmware with guaranteed security, thereby dramatically accelerating the total live migration time as well as downtime of the AMD-SEV platform.

Device pass-through has been widely adopted by cloud service providers to achieve near bare-metal I/O performance in virtual machines (VMs). However, this approach requires static pinning of VM memory, making on-demand paging unavailable. The hardware device I/O page fault (IOPF) capability offers an optimal solution to this limitation. Current IOPF approaches, using either standard IOMMU capabilities (ATS+PRI) or devices with independent IOMMU implementations, have not gained widespread adoption in public Infrastructure-as-a-Service clouds. This is due to high costs, platform dependency, and significant impacts on performance and service level objectives (SLOs). We present the Virtualized Page Request Interface (VPRI), a novel IOPF system developed through software-hardware collaboration. VPRI is not only platform-independent, free from address translation complexities, but also cost-effective, and designed to minimize SLO impact. Our work enables large-scale deployment of IOPF capability in Alibaba Cloud with negligible impact on SLOs. When integrated with memory management software, it significantly enhances memory utilization in public IaaS clouds, effectively overcoming the static memory pinning restriction associated with pass-through devices.

Confidential VM, which offers an isolated execution environment for cloud tenants with limited trust in the cloud provider, has recently been deployed in major clouds such as AWS and Azure. However, while ARM has become increasingly popular in cloud data centers, existing confidential VM designs mainly leverage specialized x86 hardware extensions (e.g., AMD SEV and Intel TDX) to isolate VMs upon a shared hypervisor.

Microkernels have been extensively studied over decades. However, IPC (Inter-Process Communication) is still a major factor of runtime overhead, where fine-grained isolation usually leads to excessive IPCs. The main overhead of IPC comes from the involvement of the kernel, which includes the direct cost of mode switches and address space changes, as well as indirect cost due to the pollution of processor structures.