Large language models (LLMs) are expensive to serve because model parameters, attention computation, and KV caches impose substantial memory and latency costs.We present GRASPrune, a structured pruning framework applied after pretraining that jointly prunes FFN channels and KV head groups under a single global budget.Instead of learning importance scores without constraints and applying the budget only after training, GRASPrune learns lightweight gate scores with a projected straight-through estimator that enforces a hard mask satisfying the budget at every step while keeping the backbone weights frozen.After the mask is fixed, we calibrate scaling factors on the retained units to mitigate scale mismatch caused by pruning, and fold these factors into the pruned weights to obtain a smaller dense checkpoint with no extra parameters at inference.On LLaMA-2-7B, GRASPrune removes 50% of parameters and achieves 12.18 perplexity on WikiText-2 while maintaining competitive average zero-shot accuracy on five benchmarks, using four epochs on 512 unlabeled calibration sequences on a single NVIDIA A100 80GB GPU without any full model finetuning. 1

Multi-sensor data fusion plays a crucial role in modern autonomous systems, enabling them to perceive the surroundings from multiple dimensions. However, the accuracy of fusion output can be compromised by the temporal inconsistency of input messages from different sources. ROS provides algorithms of message synchronization to mitigate this misalignment before the data fusion process. Nevertheless, this introduces additional latency in processing each message, which influences the real-time performance of ROS systems. Previous research [1] is the first to analyze and bound the reaction latency of the ApproximateTime synchronization policy in ROS, which is essential for analyzing the system-level end-to-end reaction time. However, their bound is overly pessimistic. In this paper, we propose a safe and tight reaction latency upper bound for the ApproximateTime policy. We conduct experiments to validate its accuracy and assess its improvements compared to [1].

Sensor fusion plays a critical role in modern robotics and autonomous systems. In reality, the sensor data destined for the fusion algorithm may have substantially different sampling times. Without proper management, this could lead to poor sensor fusion quality. Robot operating system (ROS) is the most popular robotic software framework, providing essential mechanisms for synchronizing messages to mitigate timing inconsistencies during sensor fusion. Recently, ROS introduced a new LatestTime message synchronization policy. In this article, we formally model the behavior of the LatestTime policy and analyze its worst-case real-time performance. Our investigation uncovers a defect of the LatestTime policy that may cause infinite latency in publishing subsequent outputs. We propose a solution to address this defect and develop safe and tight upper bounds on worst-case real-time performance, in terms of both the maximal temporal inconsistency of its outputs and the incurred latency. Experiments are conducted to evaluate the precision, safety and robustness of our theoretical results.

Robot Operating System (ROS) is a prevailing software framework for robotic appliscation development. Graphics Processing Unit (GPU) is widely used in many ROS applications as a first-order computation resource. Unfortunately, ROS does not do any resource management for GPU, and different components in a ROS application directly submit their GPU workload without coordinating with each other, which may cause severe problems in both general performance and realtime capability. This paper presents ROSGM, a real-time ROS 2G PUM anagement framework. Instead of providing a fixed GPU management policy for all scenarios, ROSGM allows the addition of any management policy as a plug-in and dynamic switching among different management policies at run-time, which is helpful since GPU management policies are typically device-dependent, and different applications or the same application in different modes may need different GPU management policies. Besides, ROSGM supports dynamic task loading and unloading for integrating additional functionalities when required at run-time. We conduct experiments to evaluate ROSGM. The results show that by properly managing the GPU resource using ROSGM, we can significantly improve the performance of ROS 2 applications. The flexibility of adding management policies as plug-ins, dynamic switching of management policies, and dynamic task loading and unloading helps improve the adaptability of ROSGM.

Multi-sensor data fusion is crucial for modern autonomous systems to accurately perceive their surrounding environments and make intelligent decisions. However, as different sensor sources may have significant time disparity, it is necessary to synchronize their data before sending them to the fusion algorithm, in order to control such differences and get meaningful fusion results. This paper discusses the message synchronization policy in ROS, a popular framework for robotic systems. The ROS message synchronization policy has proven to be highly effective in reducing the time disparity, but it introduces a certain level of latency. Therefore, to use it for real-time systems, it is essential to establish an upper bound for the worst-case latency that may occur. Specifically, we analyze two key latency metrics of the ROS message synchronization policy, the passing latency and reaction latency, which are needed to analyze the end-to-end delay and reaction time on the system level. We conduct experiments under different settings to evaluate the precision of our proposed latency upper bounds against the maximum observed latency in real execution.

ROS (Robot Operating System) is currently one of the most popular development frameworks for robotic software, which is usually subject to hard real-time constraints in safe-critical domains. Designers must formally model and analyze its timing behaviors to guarantee that real-time constraints are always honored at run-time. This paper studies real-time scheduling and analysis under a multi-threaded executor in ROS 2. We present a formal description of the scheduling model of multi-threaded executors, and develop response time analysis techniques for processing chains executing on it. Moreover, we identify a risk of increasing the response time of chains that may be caused by improper design when deploying systems on multi-threaded executors, which provides a useful guidance to designers. We conduct experiments with both randomly generated workloads and case studies on a realistic ROS 2 platform to evaluate and demonstrate our results.

Multi-sensor data fusion is essential in autonomous systems to support accurate perception and intelligent decisions. To perform meaningful data fusion, input data from different sensors must be sampled at time points in close propinquity to each other, otherwise the result cannot accurately reflect the status of the physical environment. ROS (Robotic Operating System), a popular software framework for autonomous systems, provides message synchronization mechanisms to address the above problem, by buffering messages carrying data from different sensors and grouping those with similar timestamps. Although message synchronization is widely used in applications developed based on ROS, little knowledge is known about its actual behavior and performance, so it is hard to guarantee the quality of data fusion. In this paper, we model the message synchronization policy in ROS and formally analyze its worst-case time disparity (maximal difference among the timestamps of the messages grouped into the same output set). We conduct experiments to evaluate the precision of the proposed time disparity upper bound against the maximal observed time disparity in real execution, and compare it with the synchronization policy in Apollo Cyber RT, another popular software framework for autonomous driving systems. Experiment results show that our analysis has good precision and ROS outperforms Apollo Cyber RT in terms of both observed worst-case time disparity and the theoretical bound.

Trajectory prediction can aid in target tracking, automatic system navigation, social behavior prediction, analysis and other computer vision tasks. When people walk in crowded spaces such as sidewalks, subway and airports, etc., they naturally adjust their walking style according to the scene context and follow common social etiquette, such as maintaining separation and avoiding collisions. Accurate prediction of trajectories is a big challenge in a crowded scenario where interaction between targets may cause complex societal dynamics. Unlike the prediction of a single person trajectory, it is difficult to capture the real motion of multiple people by only considering the historical positions of each individual separately. Benefit from the recent success of graph neural networks, we propose a model called GNN-TP for pedestrian trajectory prediction. GNN-TP is a purely data-driven model that simultaneously infers the interactions between pedestrians in an unsupervised way and predicts their future trajectories jointly in crowded scenes. On the one hand, GNN-TP infers the interactions employing the observed historical trajectories. We transfer pedestrians' information on the graph-structured data and classify the interaction type based on edges' features. On the other hand, it learns the dynamical model and predicts future trajectories based on the inferred interactions and the observations. Extensive experiments show that our trajectory prediction model achieves efficient and state-of-the-art performance on several public datasets.

Recently, the emerging bio-inspired event cameras have demonstrated potentials for a wide range of robotic applications in dynamic environments. In this paper, we propose a novel fast and asynchronous event-based corner detection method which is called FA-Harris. FA-Harris consists of several components, including an event filter, a Global Surface of Active Events (G-SAE) maintaining unit, a corner candidate selecting unit, and a corner candidate refining unit. The proposed G-SAE maintenance algorithm and corner candidate selection algorithm greatly enhance the real-time performance for corner detection, while the corner candidate refinement algorithm maintains the accuracy of performance by using an improved event-based Harris detector. Additionally, FA-Harris does not require artificially synthesized event-frames and can operate on asynchronous events directly. We implement the proposed method in C++ and evaluate it on public Event Camera Datasets. The results show that our method achieves approximately 8× speed-up when compared with previously reported event-based Harris detector, and with no compromise on the accuracy of performance.