Efficient LLM inference on resource-constrained devices presents significant challenges in compute and memory utilization. Due to limited GPU memory, existing systems offload model weights to CPU memory, incurring substantial I/O overhead between the CPU and GPU. This leads to two major inefficiencies: (1) GPU cores are underutilized, often remaining idle while waiting for data to be loaded; and (2) GPU memory has low impact on performance, as reducing its capacity has minimal effect on overall throughput.In this paper, we propose SpecOffload, a high-throughput inference engine that embeds speculative decoding into offloading. Our key idea is to unlock latent GPU resources for storing and executing a draft model used for speculative decoding, thus accelerating inference at near-zero additional cost. To support this, we carefully orchestrate the interleaved execution of target and draft models in speculative decoding within the offloading pipeline, and propose a planner to manage tensor placement and select optimal parameters. Compared to the best baseline, SpecOffload improves GPU core utilization by 4.49x and boosts inference throughput by 2.54x. Our code is available at https://github.com/MobiSense/SpecOffload-public .

Time-Sensitive Networking (TSN) is foreseen as a foundational technology that enables Industry 4.0. It offers deterministic data transmission over Ethernet for critical applications such as industrial control and automotive systems. However, TSN is susceptible to hardware and software errors, necessitating an effective diagnostic system. Traditional network diagnostic tools are inadequate for TSN fault localization and classification due to the tightly coupled traffic and high precision requirements in TSN. In response, this paper presents TSNCard, a cross-cycle postcard-based diagnostic system tailored for Time-Aware Shaper (IEEE 802.1 Qbv) in TSN. TSNCard introduces a novel telemetry protocol that leverages the cyclical nature of TSN networks for data collection at each node. This protocol, coupled with dedicated analytic algorithms and hardware innovations within switches, forms a comprehensive system for TSN monitoring, fault localization and classification. Extensive experiments on both simulation and physical testbeds show that TSNCard can 100% detect fault location and type of the TSN misbehavior while adhering to industrial bandwidth restrictions. TSNCard not only bridges the gap in the TSN protocol stack, but also serves as a versatile toolkit for time-synchronized network analysis, paving the way for future research. The code is available athttps://github.com/MobiSense/TSNCard

Time-Sensitive Networking (TSN) is an enabling technology for Industry 4.0. Traffic scheduling plays a key role for TSN to ensure low-latency and deterministic transmission of critical traffic. As industrial network scales, TSN networks are expected to support a rising number of both time-triggered and event-triggered critical traffic (TCT and ECT). In this work, we present InNetScheduler, the first in-network TSN scheduling paradigm that boosts the throughput, i.e., number of scheduled data flows, of both traffic types. Different from existing approaches that conduct entire scheduling on the server, InNetScheduler leverages the computation resources on switches to promptly schedule latency-critical ECT, and delegate the computational-intensive TCT scheduling to server. The key innovation of InNetScheduler includes a Load-Aware Optimizer to mitigate ECT conflicts, a Relaxated ECT Scheduler to accelerate in-network computation, and End-to-End Determinism Guarantee to lower scheduling jitter. We fully implement a suite of InNetScheduler-compatible TSN switches with hardwaresoftware co-design. Extensive experiments are conducted on both simulation and physical testbeds, and the results demonstrate InNetScheduler’s superior performance. By unleashing the power of in-network computation, InNetScheduler points out a direction to extend the capacity of existing industrial networks.

Time-Sensitive Networking (TSN) is foreseen as a foundational technology that enables Industry 4.0. It offers deterministic data transmission over Ethernet for critical applications such as industrial control and automotive systems. However, TSN is susceptible to hardware and software errors, necessitating an effective diagnostic system. Traditional network diagnostic tools are inadequate for TSN fault localization due to the unique characteristics of TSN. In response, this paper presents TSNCard, a cross-cycle postcard-based diagnostic system tailored for TSN. TSNCard introduces a novel telemetry protocol that leverages the cyclical nature of TSN networks for data collection at each node. This protocol, coupled with dedicated analytic algorithms and hardware innovations within switches, forms a comprehensive system for TSN monitoring and fault localization. Extensive experiments on both simulation and physical testbeds show that TSNCard can 100% localize the root cause of the TSN misbehavior while adhering to industrial bandwidth restrictions. TSNCard not only bridges the gap in the TSN protocol stack, but also serves as a versatile toolkit for time-synchronized network analysis, paving the way for future research.

Drones have witnessed extensive popularity among diverse smart applications, and visual Simultaneous Localization and Mapping (SLAM) technology is commonly used to estimate the six-degrees-of-freedom pose for drone flight control systems. However, traditional image-based SLAM cannot ensure the flight safety of drones, especially in challenging environments such as high-speed flight and high dynamic range scenarios. The event camera, a new vision sensor, holds the potential to enable drones to overcome these challenging scenarios if fused with the image-based SLAM. Unfortunately, the computational demands of event-image fusion SLAM have grown manifold compared with image-based SLAM. Existing research on visual SLAM acceleration cannot achieve real-time operation of event-image fusion SLAM on on-board computing platforms for drones. To fill this gap, we present TrinitySLAM , a high-accuracy, real-time, low-energy consumption event-image fusion SLAM acceleration framework utilizing Xilinx Zynq, an on-board heterogeneous computing platform. The key innovations of TrinitySLAM include a fine-grained computation allocation strategy, several novel hardware–software co-acceleration designs, and an efficient data exchange mechanism. We fully implement TrinitySLAM on the latest Zynq UltraScale+ platform and evaluate its performance on one custom-made drone dataset and four official datasets covering various scenarios. Comprehensive experiments show that TrinitySLAM improves the pose estimation accuracy by 28% with half end-to-end latency and 1.2× energy consumption reduction compared with the most comparable state-of-the-art heterogeneous computing platform acceleration baseline.

Time-Sensitive Networking (TSN) has been considered the most promising network paradigm for time-critical applications (e.g., industrial control) and traffic scheduling is the core of TSN to ensure low latency and determinism. With the demand for flexible production increases, industrial network topologies and settings change frequently due to pipeline switches. As a result, there is a pressing need for a more efficient TSN scheduling algorithm. In this paper, we propose DeepScheduler, a fast and scalable flow-aware TSN scheduler based on deep reinforcement learning. In contrast to prior work that heavily relies on expert knowledge or problem-specific assumptions, DeepScheduler automatically learns effective scheduling policies from the complex dependency among data flows. We design a scalable neural network architecture that can process arbitrary network topologies with informative representations of the problem, and decompose the problem decision space for efficient model training. In addition, we develop a suite of TSN-compatible testbeds with hardware-software co-design and DeepScheduler integration. Extensive experiments on both simulation and physical testbeds show that DeepScheduler runs >150/5 times faster and improves the schedulability by 36%/39% compared to state-of-the-art heuristic/expert-based methods. With both efficiency and effectiveness, DeepScheduler makes scheduling no longer an obstacle towards flexible manufacturing.

Rapid advances in programmable network devices catalyzed the development of in-network computing, which is foreseen as a key enabler to empower the intelligence of production lines and mechanical arms in Industry 4.0. Various pioneering approaches have demonstrated the significant benefits of moving simple yet delay-sensitive industrial control tasks performed by servers to network switches. However, our detailed field study at a top-tier auto glass factory reveals that current practice fails to achieve a real-time and deterministic intelligent decision closure as leaving those complex yet essential planning tasks still on edge or cloud. In this paper, we design and implement a brand-new industrial switch, named Netopia, on a commercial Zynq platform through software and hardware co-design. Netopia enables planning and control to simultaneously perform on a network switch during communication. At the core of Netopia are three simple yet effective modules - a determinism guarantee mechanism, a computing acceleration scheme, and a packet deterministic forwarding framework that work hand-in-hand to ensure mechanical arms obtain intelligent control commands with low and deterministic latency. Comprehensive evaluations in industrial environments demonstrate that Netopia achieves an average end-to-end intelligent decision latency of 3.0ms with a jitter < 0.4ms, reduced by > 86% over existing works.