Roadside multi-sensor networks, such as RGB-D camera networks, play a crucial role in Intelligent Transportation Systems (ITS). These systems rely on accurate extrinsic parameters (i.e., the relative positions and orientations) of each camera in the network. However, achieving fast and accurate large-scale extrinsic parameter calibration is challenging, especially when the overlap between camera views is limited due to cost constraints. To address this issue, we propose an automated, scalable, and marker-free calibration method that requires no human intervention. Our approach leverages dynamic rigid bodies, such as moving vehicles, as bridges to establish associations among all cameras without manual placement or supervision. The proposed method consists of three main stages: 1) calibrating camera height, roll, and pitch using the ground plane; 2) estimating each camera’s 2D position and yaw angle based on trajectories; and 3) refining these estimates by matching features on the road surface using SuperPoint and SuperGlue. Real-world experiments involving 58 roadside RGB-D cameras deployed in a parking lot demonstrate that our method significantly improves calibration efficiency while maintaining high accuracy, making it well-suited for large-scale RGB-D camera network deployment.

Infrastructure-based vehicle localization, in comparison to single-agent approaches, offers several advantages including reduced system cost, extended perception range, enhanced data fusion capabilities, and energy savings. Many conventional approaches impose limitations on the types of objects due to the need for specific object-end modifications, such as applying perceptual markers like color-labeled plates and reflective balls. LiDAR presents a solution in terms of object arbitrariness, as it addresses the challenges of feature-free object modeling and continuous registration. However, achieving complete environmental coverage with LiDAR remains prohibitively expensive, particularly in extensive areas. Hence, this study proposes a cross-modal localization approach using adaptive modeling, employing LiDAR for object modeling and cost-effective sensor cameras for object tracking through image-point-cloud registration. Accurate correspondence between the model and observation can be estimated in real-time. The experiments are conducted in a typical scenario that requires adaptive modeling: Autonomous Valet Parking (AVP). Results demonstrate that the proposed system achieves comparable performance with significantly reduced system costs, highlighting its potential for large-scale deployment.

In indoor environments where the Global Navigation Satellite System (GNSS) isn’t available, the infrastructure-based LiDAR-camera joint array can provide high-precision localization for mobile robots, such as Autonomous Valet Parking (AVP). The primary challenge in employing the infrastructure-based LiDAR-camera joint array is the extrinsic calibration between the LiDAR and the camera. Moreover, to handle interference deviation caused by vibrations or inadequate mounting stiffness during operation, the calibration’s extrinsic parameters must be automatically updated online, presenting higher demands for infrastructure-based LiDAR-camera extrinsic calibration. This paper proposes an infrastructure LiDAR-camera online automatic calibration method based on prior knowledge of cross-modal target registration. This method requires no manual targets and initial pose guesses and can achieve extrinsic calibration. The object-prior model based on a lightweight object detection algorithm can rapidly detect scenes favorable for extrinsic calibration in sub-images of camera images. This creates favorable conditions for the registration of cross-modal networks and poses optimization of the LiDAR camera. Additionally, because a lightweight algorithm is used, the process does not compromise efficiency or consume excessive computational resources. Experimental results demonstrate that the proposed calibration method is suitable for calibrating infrastructure-based LiDAR-camera, with comparable accuracy and the ability to perform online calibration. Comparative experiments also show that the object-prior model can indeed select better scenes for LiDAR-camera extrinsic calibration, thus improving the accuracy and stability of extrinsic calibration to some extent.

Localization is a key problem for autonomous driving in indoor parking. There have been some previously proposed methods based on UWB, LiDAR, fisheye cameras, etc. However, most of these methods have some drawbacks such as high cost or dependency on light conditions. To address these challenges, this paper proposes a novel Time-of-Flight (ToF) camera based mapping and localization system for indoor parking lots leveraging Manhattan World Regulation. ToF cameras are low-cost and can actively generate dense point clouds of the environment without external light sources. To overcome the shortcoming of ToF camera small field of view, the proposed system utilizes the structural information of the ceiling of indoor parking lots and Manhattan World Regulation. We track the surface normals on the unit sphere for drift-free rotation estimation. Based on this drift-free rotation, we can effectively calculate 6-DOF pose with decoupled rotation and translation estimation during mapping or global localization. This new system runs in real-time on limited computation resources and is demonstrated on two different challenging indoor parking, achieving real-time performance at 10 Hz and localization error less than 0.1 meter.

This paper presents a Graph Matching Pose SLAM to build the perception map which is a prerequisite of localization and environment perception of the intelligent vehicle. Graph-based simultaneous localization and mapping (SLAM) is widely used to build a map with global consistency and requires loop closure to eliminate the accumulative errors. However, loop closure forces the vehicle to re-visit a previously entered area. In this paper, a road network-based graph SLAM method without the requirement of loop closure is proposed to build a consistent map for the real environment. The estimated poses from the original SLAM are linked with the available road network based on the graph matching. Thus the available road network introduces the real topology of the environment to Pose SLAM regularly. In this way, the map is optimized by using the factor graph inference frequently. The proposed method is validated on the KITTI dataset and the real-world experiments. This method outperforms the state-of-the-art methods.

A novel scheme of edge detection based on the physical law of diffusion is presented in this paper. Though the most current studies are using data based methods such as deep neural networks, these methods on machine learning need big data of labeled ground truth as well as a large amount of resources for training. On the other hand, the widely used traditional methods are based on the gradient of the grayscale or color of images with using different sorts of mathematical tools to accomplish the mission. Instead of treating the outline of an object in an image as a kind of gradient of grayscale or color, our scheme deals with the edge detection as a character of an energy diffusing in the space of media such as charge-coupled device. By using the characteristic function of diffusion, the information of the energy will be extracted. The scheme preserves the structural information of images very well. Because it comes from the inhere law of images' physical property, it has a unified mathematical framework for images' edge detection under different conditions, for example, multiscales, diferent light conditions, and so on. Moreover, it has low computational complexity.

Graphical avatars have gained popularity in many application domains such as three-dimensional (3D) animation movies and animated simulations for product design. However, the methods to edit avatars' behaviors in the 3D graphical environment remained to be a challenging research topic. Since the hand-crafted methods are time-consuming and inefficient, the automatic actions of the avatars are required. To achieve the autonomous behaviors of the avatars, artificial intelligence should be used in this research area. In this paper, we present a novel approach to construct a system of automatic avatars in the 3D graphical environments based on the machine learning techniques. Specific framework is created for controlling the behaviors of avatars, such as classifying the difference among the environments and using hierarchical structure to describe these actions. Because of the requirement of simulating the interactions between avatars and environments after the classification of the environment, Reinforcement Learning is used to compute the policy to control the avatar intelligently in the 3D environment for the solution of the problem of different situations. Thus, our approach has solved problems such as where the levels of the missions will be defined and how the learning algorithm will be used to control the avatars. In this paper, our method to achieve these goals will be presented. The main contributions of this paper are presenting a hierarchical structure to control avatars automatically, developing a method for avatars to recognize environment and presenting an approach for making the policy of avatars' actions intelligently.

Classification the 3D models in the graphical environment is a key problem with applications in computer graphics, virtual reality, especially the intelligent virtual human for humanoid animation. The challenging aspects of this problem are to find a suitable shapes' feature that can be used to compare them quickly and a proper method to classify the models on the features. We propose a method of classifying shapes' features for surface-based 3D shape models based on their shape similarity. The features of shape of 3D models are computed by first converting an input surface based model into an oriented point set model and then computing features histograms of distance and shape distributions. Then, support vector machines (SVM) are used to classify the features of the models. By the classification the models can be given semantic meanings in the 3D environment.