LiDAR technology holds vast potential across various sectors, including robotics, autonomous driving, and urban planning. However, the performance of current LiDAR sensors is hindered by limited field of view (FOV), low resolution, and lack of flexible focusing capability. We introduce αLiDAR, an innovative LiDAR system that employs controllable actuation to provide a panoramic FOV, high resolution, and adaptable scanning focus. The core concept of αLiDAR is to expand the operational freedom of a LiDAR sensor through the incorporation of a controllable, active rotational mechanism. This modification allows the sensor to scan previously inaccessible blind spots and focus on specific areas of interest in an adaptive manner. By modeling uncertainties in LiDAR rotation process and estimating point-wise uncertainty, αLiDAR can correct point cloud distortions resulted from significant rotation. In addition, by optimizing LiDAR's rotation trajectory, αLiDAR can swiftly adapt to dynamic areas of interest. We developed several prototypes of αLiDAR and conducted comprehensive evaluations in various indoor and outdoor real-world scenarios. Our results demonstrate that αLiDAR achieves centimeter-level pose estimation accuracy, with an average latency of only 37 ms. In two typical LiDAR applications, αLiDAR significantly enhances 3D mapping accuracy, coverage, and density by 8.5×, 2×, and 1.6× respectively, compared to conventional LiDAR sensors. Additionally, αLiDAR's adaptive rotation improves the effective sensing distance by 1.8× and increases the number of perceived objects by 1.9×. A video demonstration of αLiDAR's in action in real world is available at https://youtu.be/x4zc_I_xTaw. The code is available at https://github.com/HViktorTsoi/alpha_lidar.

We present αLiDAR, an innovative LiDAR system that incorporates a controllable active rotational mechanism to broaden the field of view (FOV), enhance resolution, and provide adaptable focusing. This system addresses the inherent limitations of traditional LiDAR sensors, such as narrow FOV, low resolution, and lack of flexible focusing capability. By scanning blind spots and dynamically focusing on areas of interest, αLiDAR significantly surpasses conventional LiDAR sensors. Our prototypes, tested under varied real-world conditions, have demonstrated marked improvements in typical LiDAR applications. Specifically, αLiDAR enhances 3D mapping accuracy, coverage, and density by factors of 8.5, 2, and 1.6, respectively. Furthermore, the adaptive rotational mechanism of αLiDAR extends the effective sensing distance by 1.8× and increases object detection by 1.9×. To see αLiDAR in action, visit our video demonstration at https://youtu.be/x4zc_I_xTaw. Both the hardware and software implementations of αLiDAR are open-sourced at https://github.com/HViktorTsoi/alpha_lidar.

Manga is a fashionable Japanese-style comic form that is composed of black-and-white strokes and is generally displayed as raster images on digital devices. Typical mangas have simple textures, wide lines, and few color gradients, which are vectorizable natures to enjoy the merits of vector graphics, e.g., adaptive resolutions and small file sizes. In this paper, we propose MARVEL (MAnga’s Raster to VEctor Learning), a primitive-wise approach for vectorizing raster gray-level mangas by Deep Reinforcement Learning (DRL). Unlike previous learning-based methods which predict vector parameters for an entire image, MARVEL introduces a new perspective that regards an entire manga as a collection of basic primitives—stroke lines, and designs a DRL model to decompose the target image into a primitive sequence for achieving accurate vectorization. To improve vectorization accuracies and decrease file sizes, we further propose a stroke accuracy reward to predict accurate stroke lines, and a pruning mechanism to avoid generating erroneous and repeated strokes. Extensive subjective and objective experiments show that our MARVEL can generate impressive results and reaches the state-of-the-art level.

Infrastructure-assisted autonomous driving is an emerging paradigm that expects to significantly improve the driving safety of autonomous vehicles. The key enabling technology for this vision is to fuse LiDAR results from the roadside infrastructure and the vehicle to improve the vehicle's perception in real time. In this work, we propose VIPS, a novel lightweight system that can achieve decimeter-level and real-time (up to 100 ms) perception fusion between driving vehicles and roadside infrastructure. The key idea of VIPS is to exploit highly efficient matching of graph structures that encode objects' lean representations as well as their relationships, such as locations, semantics, sizes, and spatial distribution. Moreover, by leveraging the tracked motion trajectories, VIPS can maintain the spatial and temporal consistency of the scene, which effectively mitigates the impact of asynchronous data frames and unpredictable communication/compute delays. We implement VIPS end-to-end based on a campus smart lamppost testbed. To evaluate the performance of VIPS under diverse situations, we also collect two new multi-view point cloud datasets using the smart lamppost testbed and an autonomous driving simulator, respectively. Experiment results show that VIPS can extend the vehicle's perception range by 140% within 58 ms on average, and delivers a 4X improvement in perception fusion accuracy and 47X data transmission saving over existing approaches. A video demo of VIPS based on the lamppost dataset is available at https://youtu.be/zW4oi_EWOu0.

Traffic camera is one of the most ubiquitous traffic facilities, providing high coverage of complex, accident-prone road sections such as intersections. This work leverages traffic cameras to improve the perception and localization performance of autonomous vehicles at intersections. In particular, vehicles can expand their range of perception by matching the images captured by both the traffic cameras and on-vehicle cameras. Moreover, a traffic camera can match its images to an existing high-definition map (HD map) to derive centimeter-level location of the vehicles in its field of view. To this end, we propose AutoMatch - a novel system for real-time image registration, which is a key enabling technology for traffic camera-assisted perception and localization of autonomous vehicles. Our key idea is to leverage landmark keypoints of distinctive structures such as ground signs at intersections to facilitate image registration between traffic cameras and HD maps or vehicles. By leveraging the strong structural characteristics of ground signs, AutoMatch can extract very few but precise landmark keypoints for registration, which effectively reduces the communication/compute overhead. We implement AutoMatch on a testbed consisting of a self-built autonomous car, drones for surveying and mapping, and real traffic cameras. In addition, we collect two new multi-view traffic image datasets at intersections, which contain images from 220 real operational traffic cameras in 22 cities. Experimental results show that AutoMatch achieves pixel-level image registration accuracy within 88 milliseconds, and delivers an 11.7× improvement in accuracy, 1.4× speedup in compute time, and 17.1× data transmission saving over existing approaches.

The century-old development of the automotive industry has spawned one of the greatest Cyber-Physical Systems (CPSs) in the future-unmanned vehicles. The vehicle can obtain environmental information through different sensors, map it to the virtual coordinate system of the vehicle body to make decisions, and finally generate control instructions. However, a series of factors, such as complex road scenes, defective and irregular target sparse sampling, and large coding space, pose challenges to accurate, efficient, and stable perception results. To overcome the most challenging problem of dynamic target tracking, this paper designs a two-stage detection model based on non-uniform polar voxelization sampling of irregular 3D point cloud, which is used with local registration-based search to achieve efficient multi-target tracking. Non-uniform voxelization not only balances the spatial sampling and encoding efficiency of the point cloud for the backbone, but also adapts to the feature aggregation of the detection head, thereby achieving double acceleration. Finally, we tested our model on KITTI Tracking data. The comparison results show that the calculation speed of the final model is greatly improved and the tracking accuracy is competitive in all categories.

Object detection is the most critical and foundational sensing module for the autonomous movement platform. However, most of the existing deep learning solutions are based on GPU servers, which limits their actual deployment. We present an efficient multi-sensor fusion based object detection model that can be deployed on the off-the-shelf edge computing device for the vehicle platform. To achieve real-time target detection, the model eliminates a large number of invalid point clouds through ground filtering algorithm, and then adds texture information (fused from camera image) through point cloud coloring to enhance features. The proposed PV-EncoNet efficiently encodes both the spatial and texture features of each colored point through point-wise and voxel-wise encoding, and then predicts the position, heading and class of the objects. The final model can achieve about 17.92 and 24.25 Frame per Second (FPS) on two different edge computing platforms, and the detection accuracy is comparable with the state-of-the-art models on the KITTI public dataset (i.e., 88.54% for cars, 71.94% for pedestrians and 73.04% for cyclists). The robustness and generalization ability of the PV-EncoNet for the 3D colored point cloud detection task is also verified by deploying it on the local vehicle platform and testing it on real road conditions.

Manga is a world popular comic form originated in Japan, which typically employs black-and-white stroke lines and geometric exaggeration to describe humans' appearances, poses, and actions. In this paper, we propose MangaGAN, the first method based on Generative Adversarial Network (GAN) for unpaired photo-to-manga translation. Inspired by the drawing process of experienced manga artists, MangaGAN generates geometric features and converts each facial region into the manga domain with a tailored multi-GANs architecture. For training MangaGAN, we collect a new data-set from a popular manga work with extensive features. To produce high-quality manga faces, we propose a structural smoothing loss to smooth stroke-lines and avoid noisy pixels, and a similarity preserving module to improve the similarity between domains of photo and manga. Extensive experiments show that MangaGAN can produce high-quality manga faces preserving both the facial similarity and manga style, and outperforms other reference methods.