Large Language Models (LLMs) present significant challenges when deployed on client devices, primarily due to their large weight sizes and computational demands. Recent work has identified model sparsity as a promising opportunity to mitigate these issues, yet existing system supports often require custom ML operators, sophisticated IO pipelining, or sacrifice portability across platforms.

In an autonomous driving system (ADS), the perception module is crucial to driving safety and efficiency. Unfortunately, the perception in today's ADS remains oblivious to driving decisions, contrasting to how humans drive. Our idea is to refactor ADS so that (1) the ADS guides its perception with the driving knowledge in situ; (2) the perception differentiates between awareness and attention. We propose a system called AnA with three novel mechanisms: (1) a query interface for the planning to express its interest in perception; (2) a query executor that maps queries to an optimal set of perception tasks; (3) a monitor for handling abnormal task executions with driving knowledge. On challenging driving benchmarks, AnA outperforms competitive baselines: it responds to adversarial events timely, reducing collisions by 2x; it reduces compute usage by 44% without compromising driving safety. We attribute AnA's efficacy to its attentive driving, a human-like behavior that improves resource proportionality.

Proto is a new instructional OS that runs on commodity, portable hardware. It showcases modern features, including per-app address spaces, threading, commodity filesystems, USB, DMA, multicore support, self-hosted debugging, and a window manager. It supports rich applications such as 2D/3D games, music and video players, and a blockchain miner. Unlike traditional instructional systems, Proto emphasizes engaging, media-rich apps that go beyond basic terminal programs. Our method breaks down a full-featured OS into a set of incremental, self-contained prototypes. Each prototype introduces a minimal set of OS mechanisms, driven by the needs of specific apps. The construction process then progressively enables these apps by bringing up one mechanism at a time.

Natural Language Processing (NLP) inference is seeing increasing adoption by mobile applications, where on-device inference is desirable for crucially preserving user data privacy and avoiding network roundtrips. Yet, the unprecedented size of an NLP model stresses both latency and memory, creating a tension between the two key resources of a mobile device. To meet a target latency, holding the whole model in memory launches execution as soon as possible but increases one app’s memory footprints by several times, limiting its benefits to only a few inferences before being recycled by mobile memory management. On the other hand, loading the model from storage on demand incurs IO as long as a few seconds, far exceeding the delay range satisfying to a user; pipelining layerwise model loading and execution does not hide IO either, due to the high skewness between IO and computation delays.