As large language models (LLMs) become more capable, many applications are shifting from a single LLM call to multi-agent systems.Manually designed or automatically optimized workflows often include multiple verification and testing stages.These stages can improve accuracy but also introduce substantial latency and increased token consumption.We find that many requests do not require such heavyweight processing and are solvable by a single strong agent.To address this inefficiency, we propose LLM-as-Scheduler (LAS), a system that dynamically routes each query through a workflow.LAS uses a two-stage cascade: a lightweight gate that quickly checks each agent's output, and an LLM-based scheduler that makes fine-grained routing decisions using query features and gate signals.Experiments show that LAS reduces token usage by 50.5% and end-to-end latency by over 36% on average, with at most a 1.4 percentage-point drop in accuracy compared with a strong fixed workflow.The code will be public at https://gi thub.com/YoshuaDavy/LLM-as-Scheduler

Multi-cloud deployed multi-agent systems powered by Large Language Models (LLMs) introduce critical security challenges. While Intel Trust Domain Extensions (TDX)-based Confidential Virtual Machines (CVMs) provide strong isolation and boot-time attestation, they lack dynamic runtime integrity verification, a capability essential for trusted agent systems that frequently load new models and coordinate across distributed services. We present TrustWeave, a runtime integrity measurement and attestation framework that extends the Linux Integrity Measurement Architecture (IMA) with support for Intel TDX's Runtime Measurement Registers (RTMRs). TrustWeave enables userspace attestation of dynamically loaded agent components throughout their life-cycle, providing runtime trust guarantees for secure and scalable LLM agent deployments.

The KV cache in current LLM serving system is primarily used to accelerate processing within a single request and is aggressively deleted once the response is generated. However, in scenarios like virtual assistants and multi-turn conversations, the KV cache can be reused across requests, which can dramatically reduce computation costs and improve serving latency. Caching historical tokens, however, significantly increases memory requirements. Furthermore, existing serving systems treat the request scheduler and KV cache separately, despite their tight coupling.MCaM is a multi-tier cache system that enables the KV cache reuse and sharing across requests. It leverages DRAM as slow- tier memory for storing the KV cache of historical prompts. To efficiently utilize fast-tier Hign Bandwidth Memory(HBM) on GPU, we co-designed the KV cache manager and scheduler to coordinate request scheduling and token placement across tiers. To hide the reload time, MCaM employs a pipeline prefetcher that overlaps communication and computation. Additionally, MCaM incorporates a quality-aware sparsification algorithm to heterogeneously compress the KV cache in each layer. This approach not only reduces data transfer size but also decreases the overall KV cache size. To remove data offloading from a request’s critical path, we designed an asynchronous offload engine that swaps data from HBM to DRAM in the background. Our experiments show that MCaM can reduce TTFT by up to 69% and improve prompt prefilling throughput by 3.3X. It can also reduce the end-to-end latency of LLM inference by up to 58% when request length increase to 4096 tokens.

Carbon Nanotubu field-effect transistor (CNFET) that promises both higher clock speed and energy efficiency becomes an attractive alternative to the conventional power-hungry CMOS cache. We observe that the CNFET-based cache constructed with typical SRAM cells has distinct energy consumption when reading/writing 0 and 1 from/to it. For instance, the energy consumption of writing 1 to an SRAM cell is almost 10X higher than writing 0. With this observation, we propose an energy-efficient cache design called CNT-Cache to take advantage of this feature. It predicts the cache line access pattern based on the latest cache line access history. On top of the prediction, it decides the optimal cache line encoding to match the cache operation preferences at runtime. According to our experiments on a set of benchmark programs, the optimized CNFET-based D-Cache reduces the dynamic power consumption by 22.2% on average compared to the baseline CNFET cache.