This work presents an exhaustive analysis and a quantitative performance comparison between the use of information from infrastructure and vehicle mounted sensors for 3D object detection in autonomous driving environments. To do so, LiDAR point clouds have been used as the main data input and the most popular and well-established models have been considered for this task: Second, PointPillars and PV-RCNN. They have all been trained on the DAIR-V2X cooperative dataset, since it offers both the infrastructure and vehicle perspective. The broader spatial context and greater field of vision from an elevated point of view demonstrate superior performance by mitigating occlusions and overcoming the inherent limitations of a reduced perception range from onboard a vehicle. However, this comes with its own challenges to avoid losing detection capabilities for smaller objects. The main objective of this work is to provide a like-for-like comparison of the real performance difference, isolating the point of view as the only modified variable.

Solving Person Re-Identification (Re-Id) through Deep Convolutional Neural Networks is a daunting challenge due to the small size and variety of the training data, especially in Single-Shot Re-Id, where only two images per person are available. The lack of training data causes the overfitting of the deep neural models, leading to degenerated performance. This article explores a wide assortment of neural architectures that have been commonly used for object classification and analyzes their suitability in a Re-Id model. These architectures have been trained through a Triplet Model and evaluated over two challenging Single-Shot Re-Id datasets, PRID2011 and CUHK. This comparative study is aimed at obtaining the best-performing architectures and some concluding guidance to optimize the features embedding for the Re-Identification task. The obtained results present Inception-ResNet and DenseNet as potentially useful models, especially when compared with other methods, specifically designed for solving Re-Id.

Most sensor setups for onboard autonomous perception are composed of LiDARs and vision systems, as they provide complementary information that improves the reliability of the different algorithms necessary to obtain a robust scene understanding. However, the effective use of information from different sources requires an accurate calibration between the sensors involved, which usually implies a tedious and burdensome process. We present a method to calibrate the extrinsic parameters of any pair of sensors involving LiDARs, monocular or stereo cameras, of the same or different modalities. The procedure is composed of two stages: first, reference points belonging to a custom calibration target are extracted from the data provided by the sensors to be calibrated, and second, the optimal rigid transformation is found through the registration of both point sets. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups. In order to assess the performance of the proposed method, a novel evaluation suite built on top of a popular simulation framework is introduced. Experiments on the synthetic environment show that our calibration algorithm significantly outperforms existing methods, whereas real data tests corroborate the results obtained in the evaluation suite. Open-source code is available athttps://github.com/beltransen/velo2cam_calibration.

The rapid development of embedded hardware in autonomous vehicles broadens their computational capabilities, thus bringing the possibility to mount more complete sensor setups able to handle driving scenarios of higher complexity. As a result, new challenges such as multiple detections of the same object have to be addressed. In this work, a siamese network is integrated into the pipeline of a well-known 3D object detector approach to suppress duplicate proposals coming from different cameras via re-identification. Additionally, associations are exploited to enhance the 3D box regression of the object by aggregating their corresponding LiDAR frustums. The experimental evaluation on the nuScenes dataset shows that the proposed method outperforms traditional NMS approaches.

This paper proposes a method to detect abnormal motions in real vehicle situations based on trajectory data. Our approach uses a Gaussian process (GP) regression that facilitates to approximate expected vehicle's movements over a whole environment based on sparse observed data. The main contribution of this paper consists in decomposing the GP regression into spatial zones, where quasi-constant velocity models are valid. Such obtained models are employed to build a set of Kalman filters that encode observed vehicle's dynamics. This paper shows how proposed filters enable the online identification of abnormal motions. Detected abnormalities can be modeled and learned incrementally, automatically by intelligent systems. The proposed methodology is tested on real data produced by a vehicle that interacts with pedestrians in a closed environment. Automatic detection of abnormal motions benefits the traffic scene understanding and facilitates to close the gap between human driving and autonomous vehicle awareness.

Robust real-time obstacle detection/avoidance is a challenging problem especially for micro and small aerial vehicles due to the limited number of the on-board sensors due to the battery constraint and low payload. Usually lightweight sensors such as CMOS camera are the best choice comparing with laser or radar sensors. For real-time applications, most studies focus on using stereo cameras to reconstruct a 3D model of the obstacles or to estimate their depth. Instead, in this paper, a method that mimics the human behavior of detecting the state of the approaching obstacles using single camera is proposed. During the flight, this method is able to detect the changes of the size area of the obstacles. First, the method detects the feature points of the obstacles, and then extracts the obstacles that has probability of getting close. In addition, by comparing the changes in the area ratios of the obstacle in the image sequence, the method can decide if it is obstacle or not. Finally, by estimating the obstacle 2D position in the image and combining with the tracked waypoints, the UAV can take the action of avoidance.

The work presented describes a tool for driver behavior analysis. The proposal offers a contribution based on Gaussian Mixture Model (GMM) modeling technique. GMM is a powerful tool for the statistical modeling, which allows to identify specific patterns of behavior in comparison with available data. Data are obtained by the in-vehicle sensors using Controller Area Network bus (CAN bus), an Inertial Measurement Unit (IMU) and a GPS. These data allow to provide driver behavior analysis and aggressive behavior identification. This development has been tested in real-traffic situations with different drivers.

During the last decade, ground mobile robots that are able to drive autonomously in off-road environments have received a great deal of attention. Autonomous navigation in unstructured environments faces many new challenges compared to structured urban environments, these challenges increase the complexity of the localization, obstacle detection, path planning and navigation commands. Accordingly this paper presents a fusion system for stereo-vision and laser-rangefinder outdoor obstacle detection, which is implemented as an application for autonomous off-road navigation. The test platform is an electric golf-cart that is modified mechanically and electrically to operate in driver-less mode. This vehicle is equipped with binocular camera, laser-rangefinder, electronic compass and on-board embedded computer, which operates using Robotic Operating System (ROS) architecture. The proposed architecture gathers the data from all different sensors, in order to make navigation decisions from one point to another, avoiding obstacles in the path. Experimental results indicate the high performance of the proposed approaches, they show that the perception from the stereo-vision detection enhances the laser-rangefinder detection, which consequently makes a better decision in maneuvering the obstacle and returns back to the original path.

Ground detection algorithms are important for locating the area where a vehicle can travel during autonomous navigation or for collision avoidance Advanced Driver Assistance Systems (ADAS). Although many algorithms rely on a single camera to detect the ground, they don't consider the possibility of combining transformation approaches with additional information that could be obtained from another camera mounted on the vehicle. In this paper, we propose an algorithm for ground detection that allows the use of disparity maps and the combination of infrared information to enhance road detection. To this end we propose an original approach based on a thermal stereo system to analyze the thermal features of the road that are then complemented by the depth information obtained from the disparity map. We tested the algorithm with a variety of complex scenarios and results showed that the 2D and 3D data enabled accurate detection of the road independent of the weather conditions.

This paper presents an autocalibration method to determine the pose of a stereo vision system based on knowing the geometry of the ground in front of the cameras. This pose changes considerably while the vehicle is driven, therefore it is good to know constantly the pose of the camera for several applications based on computer vision, such as advanced driver assistance systems, autonomous vehicles or robotics. These constant changes of the pose make interesting to be able to detect constantly the variations in its extrinsic parameters (height, pitch, roll). The validation of the autocalibration method is accomplished by a visual odometry implementation. A study of the improvement of the results of the visual odometry estimation taking into account the changes of the camera pose is presented, demonstrating the advantages of the autocalibration method.

A solution for driver monitoring and event detection based on 3-D information from a range camera is presented. The system combines 2-D and 3-D techniques to provide head pose estimation and regions-of-interest identification. Based on the captured cloud of 3-D points from the sensor and analyzing the 2-D projection, the points corresponding to the head are determined and extracted for further analysis. Later, head pose estimation with three degrees of freedom (Euler angles) is estimated based on the iterative closest points algorithm. Finally, relevant regions of the face are identified and used for further analysis, e.g, event detection and behavior analysis. The resulting application is a 3-D driver monitoring system based on low-cost sensors. It represents an interesting tool for human factor research studies, allowing automatic study of specific factors and the detection of special event related to the driver, e.g, driver drowsiness, inattention, or head pose.

Fusion is becoming a classic topic in Intelligent Transport System (ITS) society. The lack of trustworthy sensors requires the combination of several devices to provide reliable detections. In this paper a novel approach, that takes advantage of the Joint Probabilistic Data Association technique (JPDA) for data association, is presented. The approach uses one of the most powerful techniques of Multiple Target Tracking theory and adapts it to fulfill the strong requirements of road safety applications. The different test performed proved that a powerful association technique can enhance the capacity of Advance Driver Assistance Systems. Two main sensors are used for pedestrian detection: laser scanner and computer vision. Furthermore, the approach takes advantage of the availability of other information sources i.e. context information and online information (GPS). The detections are fused using JPDA, enhancing the capacities of classical pedestrian detection systems, mainly based in visual information. The test performed also showed that JPDA improved the results offered by other data association techniques, e.g. Global Nearest Neighbors.

In this paper the authors propose a pedestrian detection system based on discrete features in infrared images. Unique keypoints are searched for in the images around which a descriptor, based on the histogram of the phase congruency orientation, is extracted. These descriptors are matched with defined regions of the body of a pedestrian. In case of a match, it creates a region of interest in the image, which is classified as a pedestrian / non-pedestrian by an SVM classifier. The pedestrian detection system has been tested in an advanced driver assistance system for urban driving.

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.

The movement of the vehicle provides useful information for different applications, such as driver assistant systems or autonomous vehicles. This information can be known by means of a GPS, but there are some areas in urban environments where the signal is not available, as tunnels or streets with high buildings. A new method for 2D visual ego motion estimation in urban environments is presented in this paper. This method is based on a stereo-vision system where the feature road points are tracked frame to frame in order to estimate the movement of the vehicle, avoiding outliers from dynamic obstacles. The road profile is used to obtain the world coordinates of the feature points as a unique function of its left image coordinates. For these reasons it is only necessary to search feature points in the lower third of the left images. Moreover, the Kalman filter is used as a solution for filtering problem. That is, in some cases, it is necessary to filter raw data due to noise acquisition of time series. The results of the visual ego motion are compared with raw data from a GPS.

In this paper a new contrast invariant descriptor for human detection in long-wave infrared images is proposed. It exploits local information histogram of orientations of phase coherence. Contrast in infrared images depends on the temperature of the object and the background, which makes gradient based descriptors less robust, especially in daylight conditions. The objective is to obtain a scale, brightness and contrast invariant descriptor that can successfully detect pedestrians in images taken with a cheap, temperature-sensitive, uncooled microbolometer. The descriptor, packed into grids is feed to a Support Vector Machine classifier. The algorithm has been tested in night and day sequences and its performance is compared with a day only descriptor: the histogram of oriented features (HOG).

The lack of trustworthy sensors makes ADAS applications a tough task. In this paper a fusion approach that uses two sensors: far infrared camera vision and a laser scanner to detect pedestrian for ADAS applications is presented. Both sensors have different field of view thus different detection zones are created according to the number of sensors available. The proposed algorithm combines low and high level information, providing a hybrid fusion scheme. Experimental results show that pedestrian detections are improved thanks to the use of several sensors.

This paper deals with the extraction of part of the visual information presented in streets, roads, and motorways. This information, provided by either traffic or road signs and route-guidance signs, is extremely important for safe and successful driving. An automatic system that is capable of extracting and identifying these signs automatically would help human drivers enormously; navigation would be easier and would allow him or her to concentrate on driving the vehicle. The system would indicate to the driver the presence of a sign in advance, so that some incorrect human decisions could be avoided. A deformable model scheme allows us to include the knowledge used while designing the signs in the algorithm and is used for their detection and identification. Two techniques to find the minimum in the energy function are shown: simulated annealing and genetic algorithms. Some problems are addressed, such as uncontrolled lighting conditions; occlusions; and variations in shape, size, and color.

This paper presents practical, high-level topological navigation tasks, making use of our general purpose landmark learning and detection system, which includes the possibility of reading text or icons inside detected landmarks. Room identification from inside, without any initialization, is achieved through its landmark signature. Room search along a corridor is done by reading the content of room nameplates placed around for human use; this allow the robot to take high-level decisions, and results in a higher integration degree of mobile robotics in real life.

Deformable models have been intensively studied in image analysis through the last decade, and are used for detection and recognition of flexible or rigid templates under diverse viewing conditions. Genetic algorithm (GA) based deformable models are used for generic visual landmark detection and interpretation. The developed system allows topologic localization and navigation using natural and artificial landmarks, exploiting deformable models' ability for handling landmark perspective variations. The resulting perception module has been integrated successfully in a complex navigation system. Various experimental results in real environments are presented on this paper, showing the effectiveness and capacity of the landmark detection and reading system.

This paper describes a vision-based landmark recognition system for use with mobile robot navigation tasks. A search algorithm based on genetic techniques for pattern recognition in digital images is presented. The developed system allows the topologic localization of a mobile robot using natural and artificial landmarks. Text strings inside landmarks can be read and interpreted, if present. The resulting system was tested onboard a B21 mobile robot and proved useful. The presented experimental results show the effectiveness of the proposed algorithm.

This paper presents a fuzzy perception planner that takes into account the time cost, the suitability of every landmark detection and the different uncertainties the robot encounters along its path for mobile robot localization. The sensor used is a camera with a motorized zoom on a pan and tilt platform and the artificial landmarks are circles detected through normalized gray scale correlation. An extended Kalman filter is used to correct the position and orientation of the vehicle. The resulting self-localization module has been integrated successfully in a more complicated navigation system.