While on-device small language models (SLMs) enable responsive inference, they often lack the accuracy required for complex reasoning tasks; conversely, server-based large language models (LLMs) deliver high accuracy but incur prohibitive latency for real-time mobile applications. Collaborative inference approaches, such as split computing and speculative decoding, promise to bridge this gap, yet both require repeated network synchronization during generation, accumulating delays that negate their computational benefits. We present SAIL, a collaborative inference framework that achieves both high accuracy and low latency through a strategy we term Prefix Handoff Inference (PHI). In PHI, a server LLM generates the difficult early tokens and hands off completion to the mobile SLM, consolidating communication into a single handoff while providing sufficient context for the SLM to maintain accuracy. SAIL operationalizes PHI through three core modules: the handoff decision module determining the optimal prefix length tailored to each query's difficulty, the branch prediction module speculatively generating candidate continuations on-device while awaiting the server prefix, and the adaptive control module coordinating hand-off and branch prediction under fluctuating network and server conditions to meet service level objectives. Together, these modules enable SAIL to deliver up to 76% accuracy improvement over mobile SLMs with 98.9% server LLM accuracy retention, while strictly meeting latency constraints across translation, math, and expert level QA.

While Vision Transformers (ViTs) have emerged as foundation models for visual recognition, their high computational demands hinder deployment on edge platforms. Temporal redundancy across video frames offers a natural opportunity to reuse prior computations; however, existing methods remain far from ideal, often relying on simple frame-difference signals. To address this, we propose Ouroboros, a framework that encompasses geometric redundancy from spatial displacement of content. We achieve this by aligning invariant content to consistent coordinates across frames, enabled by warping each frame into a global coordinate system via motion vectors from a hardware-accelerated encoder. Yet, this design raises two key challenges: (i) preserving content that drifts out of the limited coordinate system and (ii) maintaining spatial continuity at frame borders. Ouroboros resolves these challenges by introducing a toroidal (i.e., wrap-around) input space and reassigning positional encodings to track displaced content. Leveraging the significant patch reduction via a system-efficient partial computation scheme, our approach accelerates inference by up to 2.61× and reduces energy consumption by 64.5% on NVIDIA Jet-son Orin devices, with <1% accuracy loss on object detection and instance segmentation, outperforming prior methods. Designed to process only non-redundant patches, Ouroboros also excels as an offloading system, yielding higher accuracy at lower bandwidth compared to prior schemes. The source code is available at https://github.com/ckswjd99-lab/Ouroboros.

Collaborative inference is the current state-of-the-art solution for mobile-server neural network inference offloading. However, we find that existing collaborative inference solutions only focus on partitioning the DNN computation, which is only a small part of achieving an efficient DNN offloading system. What ultimately determines the performance of DNN offloading is how the execution system utilizes the characteristics of the given DNN offloading task on the mobile, network, and server resources of the offloading environment. To this end, we design CoActo, a DNN execution system built from the ground up for mobile-server inference offloading. Our key design philosophy is Coactive Inference Offloading, which is a new, improved concept of DNN offloading that adds two properties, 1) fine-grained expression of DNNs and 2) concurrency of runtime resources, to existing collaborative inference. In CoActo, system components go beyond simple model splitting of existing approaches and operate more proactively to achieve the coactive execution of inference workloads. CoActo dynamically schedules concurrent interleaving of the mobile, server, and network operations to actively increase resource utilization, enabling lower end-to-end latency. We implement CoActo for various mobile devices and server environments and evaluate our system with distinct environment settings and DNN models. The experimental results show that our system achieves up to 2.1 times speed-up compared to the state-of-the-art collaborative inference solutions.