An autonomous vehicle generates several terabytes of sensor data per day. A significant portion of this data consists of 3D point clouds produced by depth sensors such as LiDAR. This data is transferred to cloud storage, where it is utilized for training machine learning models or conducting analyses, e.g., forensic investigations in the event of an accident. To reduce network and storage costs, this paper introduces DejaView that searches for and uses redundancies on larger temporal scales (days and months) for more effective compression. We designed DejaView with the insight that the operating area of autonomous vehicles is limited and that vehicles mostly traverse the same routes daily. Consequently, the daily collected 3D data is likely similar to the data they’ve captured in the past. To capture this, the core of DejaView is a diff operation that compactly represents point clouds as delta w.r.t. 3D data from the past. Using two months of LiDAR data, DejaView can compress point clouds by a factor of 210 at a reconstruction error of only 15 cm.

To overcome line-of-sight limitations in 3D sensors, cooperative perception shares sensor information between vehicles in real time. Core to cooperative perception is the alignment of sensor data in multiple coordinate systems. Existing techniques use 3D maps and GPS for alignment, but these can lead to inaccurate alignments. However, improving alignment accuracy incurs additional compute latency, which is not desirable. In this paper, we present ARC, a system that carefully navigates the trade-off between latency and accuracy for point cloud alignment. ARC uses an anchor-based alignment technique to align vehicles to a common coordinate system. It minimizes latency by using grid-based spatial reasoning to perform alignment only in overlapping regions of the point clouds. ARC reuses the same spatial reasoning to selectively share only the most relevant data among vehicles. ARC, on traces collected from the real world and simulations, can fuse point clouds from up to 40 vehicles with an accuracy of under 7 cm and achieve a mean compute latency of 20 ms.

Real-time mobile applications such as AR/VR, cloud gaming, and collaborative robotics rely on edge computing to maintain ultra-low latency, yet they can suffer noticeable service interruptions when users move because the application's session state must migrate to a new edge site. We present EdgeWarp, a system that delivers instant mobility for stateful edge applications over 5G while laying the groundwork for 6G. EdgeWarp employs a two-step synchronization protocol that proactively mirrors application state at edge sites predicted to serve the user next, sharply reducing transfer delays, and it signals each session's latency budget to the 5G control plane so that critical flows are prioritized during handover. Experiments with real applications and 4G/5G radio traces show that EdgeWarp cuts mobility-induced downtime by up to 15.4×, charting a path toward zero-downtime edge computing—a prerequisite for emerging 6G scenarios such as holographic telepresence, tactile Internet, and large-scale digital twins. We have made our anonymized code publicly accessible here.

On-vehicle 3D sensing technologies, such as LiDARs and stereo cameras, enable a novel capability, 3D traffic reconstruction. This produces a volumetric video consisting of a sequence of 3D frames capturing the time evolution of road traffic. 3D traffic reconstruction can help trained investigators reconstruct the scene of an accident. In this paper, we describe the design and implementation of RECAP, a system that continuously and opportunistically produces 3D traffic reconstructions from multiple vehicles. RECAP builds upon prior work on point cloud registration, but adapts it to settings with minimal point cloud overlap (both in the spatial and temporal sense) and develops techniques to minimize error and computation time in multi-way registration. On-road experiments and trace-driven simulations show that RECAP can, within minutes, generate highly accurate reconstructions that have 2× or more lower errors than competing approaches.

Autonomous vehicles and human drivers are prone to line-of-sight limitations. Road-side mounted 3D sensors like LiDARs can augment a vehicle's on-board perception. However, this entails fusing 3D frames at low latency and high accuracy. Road-side and vehicle 3D frames are captured from different viewpoints. This adversely affects alignment accuracy and can be computationally expensive. To this end, VRF optimizes for both latency and accuracy by decoupling the alignment process into indirect and direct alignments. First, VRF indirectly aligns the 3D frames by aligning them to a common reference point i.e., a vehicle's on-board 3D map. Then, it directly aligns the two point clouds to refine this alignment. To ensure high accuracy, it incorporates novel offline registration and alignment accuracy forecasting modules. To ensure low latency, it uses a fast fusion pipeline that caches previous and offline computations. To our knowledge, VRF is the first vehicle road-side cooperative system to ensure cm-level accuracy and end-to-end latency less than 20 ms. Most importantly, its latency is below the 100 ms threshold required for autonomous vehicles to react to external events. Finally, VRF can improve reaction time to external events by as much as 5 seconds1.

Tracking the position and orientation, or pose, of a viewing device enables AR applications to accurately embed virtual content in physical spaces. Mobile OSs track pose by matching device camera images against street-level imagery. Thus, pose tracking is often unavailable at off-street pedestrian locations. UbiPose enables pose tracking at such locations using aerial meshes, generated from satellite imagery, that are likely to be more widely available at these locations. However, matching a camera image against an aerial mesh can be error-prone, even with modern neural matchers. These neural components are also compute-intensive. UbiPose contains a novel pose tracking pipeline that runs entirely on a mobile device using fast-path optimizations designed to accept or reject pose estimates in many cases, without sacrificing accuracy. Experiments on real-world traces show that it achieves tracking accuracy comparable to AR pose tracking in iOS in places where that is available, and is able to track pose accurately in places where it is not.

Disaster and emergency response operations require rapid situational assessment of the affected area for timely and efficient rescue operations. A 3D map, collected after a disaster, can provide such awareness, but constructing this map quickly is a significant challenge. In this paper, we explore the design of a capability called QuickSketch that rapidly builds 3D representations of an unknown environment using crowdsourcing. QuickSketch employs multiple vehicles equipped with 3D sensors (stereo cameras) to explore different areas of an unknown territory and then combines 3D data from all the vehicles to build a single 3D map. QuickSketch annotates the 3D map with important landmarks and enables rapid contextualization of visual intelligence (photos) received from first responders and disaster victims to guarantee timely backup and rescue operations. Our evaluation results show that QuickSketch can stitch a 3D map for a large campus with sub-meter mapping accuracy under certain conditions, position landmarks an order of magnitude more accurately than other image matching techniques, and contextualize visual intelligence accurately.

Autonomous vehicle prototypes today come with line-of-sight depth perception sensors like 3D cameras. These 3D sensors are used for improving vehicular safety in autonomous driving, but have fundamentally limited visibility due to occlusions, sensing range, and extreme weather and lighting conditions. To improve visibility and performance, not just for autonomous vehicles but for other Advanced Driving Assistance Systems (ADAS), we explore a capability called Augmented Vehicular Reality (AVR). AVR broadens the vehicle's visual horizon by enabling it to wirelessly share visual information with other nearby vehicles, but requires the design of novel relative positioning techniques, new perspective transformation methods, approaches to isolate and predict the motion of dynamic objects in order to hide latency, and adaptive transmission strategies to cope with wireless bandwidth variability. We show that AVR is feasible using off-the-shelf wireless technologies, and it can qualitatively change the decisions made by autonomous vehicle path planning algorithms. Our AVR prototype achieves positioning accuracies that are within a few percent of car lengths and lane widths, and is optimized to process frames at 30fps.

Traffic congestion is not only a cause of nuisance for general commuters but also a factor that has a measurable impact on the economy if not handled proactively. When congestion increases, the waiting time for commuters increases which results in wasted fuel and wasted time. Wasted fuel adds to the import bill of a country and lost time results in loss of productivity. Traffic can be regulated at various points in order to reduce congestion and eliminate bottleneck areas. In this paper, we propose the use of conventional scheduling to regulate traffic at intersections in order to reduce congestion. We propose minimum destination distance first (MDDF) and minimum average destination distance first (MADDF) algorithms and compare them with some of the relevant existing scheduling algorithms. The proposed algorithms can not only be easily implemented on low cost hardware but also show better performance and outperform the existing algorithms that are considered, based on simulation results. Simulation results show that the MDDF and MADDF algorithms reduce the traffic congestion at intersections by up to 80% in some cases compared to static traffic lights.

Wireless Sensor Networks (WSN) have emerged as a suitable solution for real-time vehicle data collection in many ITS applications. We propose the use of magnetometers in conjunction with WSN for real-time vehicle data collection. Magnetometers when used as sensor nodes offer advantages over other vehicle sensing technologies that include cost-effectiveness, energy efficiency, resistance to changes in environmental conditions, ease of re-deployment and flexibility. In this paper we present our prototype of a wireless magnetic sensor, algorithms for vehicle detection, vehicle speed estimation and vehicle length estimation and quantify their performance. Results from field evaluations prove the feasibility of our proposed solution for accurate vehicle data collection.