This work presents an exhaustive analysis and a quantitative performance comparison between the use of information from infrastructure and vehicle mounted sensors for 3D object detection in autonomous driving environments. To do so, LiDAR point clouds have been used as the main data input and the most popular and well-established models have been considered for this task: Second, PointPillars and PV-RCNN. They have all been trained on the DAIR-V2X cooperative dataset, since it offers both the infrastructure and vehicle perspective. The broader spatial context and greater field of vision from an elevated point of view demonstrate superior performance by mitigating occlusions and overcoming the inherent limitations of a reduced perception range from onboard a vehicle. However, this comes with its own challenges to avoid losing detection capabilities for smaller objects. The main objective of this work is to provide a like-for-like comparison of the real performance difference, isolating the point of view as the only modified variable.

Smart cities introduce the strategies of solving the problems faced by the current cities; such as traffic jams, resource management, and climate change. Traffic flow monitoring is considered as one of the major challenges to be solved. Cooperative systems introduced a suitable solution for sharing information between the road elements, which provides a helpful tool for monitoring the traffic. However, most of the solutions are based on the sensors or cameras mounted in fixed locations, which have a limited range of vision and produce blind areas. In order to deal with the problem, this paper presents a cooperative system based on using Unmanned Aerial Vehicles (UAVs); to monitor the traffic flow. This system proposed a lightweight semantic neural network; that takes the RGB images as input and generates a segmented image of the vehicles and its 2D real-world positions. This data then is sent to each vehicle; to have full information about its surroundings. The proposed method has been validated with several experiments of different scenarios and situations, and the obtained results show its robustness and efficiency of the system, illustrating its functionality in traffic monitoring purposes.

Odometry is a crucial task in the design of intelligent vehicles and there are many novel approaches with different sensors in Intelligent Transportation Systems (ITS) field. Accordingly, this leads to the use of multiple methods and sensors for identifying the vehicle pose in the environment, hence the necessity of multiple odometry sensors fusion methods. Quantifying the uncertainties of the sensors used in the vehicle is essential for utilizing effective fusion systems. Since the identification of the true values of the uncertainties is an exhaustive task, this paper introduces an online adaptive covariance estimation approach for drift suffering proprioceptive sensors, using an exteroceptive sensor with known uncertainty. To validate the proposed approach, three scenarios were selected and various experiments were carried out under different conditions. Though the use of multiple odometry sensors fusion algorithm, a comparative study was conducted between the adaptive covariance and several constant covariances based on the true variances. The obtained results show high performance of the proposed approach, in terms of four evaluation metrics for both translation and orientation mean errors.

This paper presents an innovative smartphone application which takes advantage of the availability of smartphone technologies to develop a highly accurate pedestrian detection and tracking application, based on monocular camera and embedded sensors. Monocular pedestrian detection is performed based on well-known vision approaches, later pose estimation is used to correct the location of the pedestrian, based on the information provided by the mobile device, and finally movement is estimated by means of a Kalman Filter. The information from the internal sensors corrects the distance given by the pin-hole model. Test performed proved the viability of the system as well as its accuracy.

Robust real-time obstacle detection/avoidance is a challenging problem especially for micro and small aerial vehicles due to the limited number of the on-board sensors due to the battery constraint and low payload. Usually lightweight sensors such as CMOS camera are the best choice comparing with laser or radar sensors. For real-time applications, most studies focus on using stereo cameras to reconstruct a 3D model of the obstacles or to estimate their depth. Instead, in this paper, a method that mimics the human behavior of detecting the state of the approaching obstacles using single camera is proposed. During the flight, this method is able to detect the changes of the size area of the obstacles. First, the method detects the feature points of the obstacles, and then extracts the obstacles that has probability of getting close. In addition, by comparing the changes in the area ratios of the obstacle in the image sequence, the method can decide if it is obstacle or not. Finally, by estimating the obstacle 2D position in the image and combining with the tracked waypoints, the UAV can take the action of avoidance.

During the last decade, ground mobile robots that are able to drive autonomously in off-road environments have received a great deal of attention. Autonomous navigation in unstructured environments faces many new challenges compared to structured urban environments, these challenges increase the complexity of the localization, obstacle detection, path planning and navigation commands. Accordingly this paper presents a fusion system for stereo-vision and laser-rangefinder outdoor obstacle detection, which is implemented as an application for autonomous off-road navigation. The test platform is an electric golf-cart that is modified mechanically and electrically to operate in driver-less mode. This vehicle is equipped with binocular camera, laser-rangefinder, electronic compass and on-board embedded computer, which operates using Robotic Operating System (ROS) architecture. The proposed architecture gathers the data from all different sensors, in order to make navigation decisions from one point to another, avoiding obstacles in the path. Experimental results indicate the high performance of the proposed approaches, they show that the perception from the stereo-vision detection enhances the laser-rangefinder detection, which consequently makes a better decision in maneuvering the obstacle and returns back to the original path.

This paper presents an autocalibration method to determine the pose of a stereo vision system based on knowing the geometry of the ground in front of the cameras. This pose changes considerably while the vehicle is driven, therefore it is good to know constantly the pose of the camera for several applications based on computer vision, such as advanced driver assistance systems, autonomous vehicles or robotics. These constant changes of the pose make interesting to be able to detect constantly the variations in its extrinsic parameters (height, pitch, roll). The validation of the autocalibration method is accomplished by a visual odometry implementation. A study of the improvement of the results of the visual odometry estimation taking into account the changes of the camera pose is presented, demonstrating the advantages of the autocalibration method.

A solution for driver monitoring and event detection based on 3-D information from a range camera is presented. The system combines 2-D and 3-D techniques to provide head pose estimation and regions-of-interest identification. Based on the captured cloud of 3-D points from the sensor and analyzing the 2-D projection, the points corresponding to the head are determined and extracted for further analysis. Later, head pose estimation with three degrees of freedom (Euler angles) is estimated based on the iterative closest points algorithm. Finally, relevant regions of the face are identified and used for further analysis, e.g, event detection and behavior analysis. The resulting application is a 3-D driver monitoring system based on low-cost sensors. It represents an interesting tool for human factor research studies, allowing automatic study of specific factors and the detection of special event related to the driver, e.g, driver drowsiness, inattention, or head pose.

Fusion is becoming a classic topic in Intelligent Transport System (ITS) society. The lack of trustworthy sensors requires the combination of several devices to provide reliable detections. In this paper a novel approach, that takes advantage of the Joint Probabilistic Data Association technique (JPDA) for data association, is presented. The approach uses one of the most powerful techniques of Multiple Target Tracking theory and adapts it to fulfill the strong requirements of road safety applications. The different test performed proved that a powerful association technique can enhance the capacity of Advance Driver Assistance Systems. Two main sensors are used for pedestrian detection: laser scanner and computer vision. Furthermore, the approach takes advantage of the availability of other information sources i.e. context information and online information (GPS). The detections are fused using JPDA, enhancing the capacities of classical pedestrian detection systems, mainly based in visual information. The test performed also showed that JPDA improved the results offered by other data association techniques, e.g. Global Nearest Neighbors.

In this paper the authors propose a pedestrian detection system based on discrete features in infrared images. Unique keypoints are searched for in the images around which a descriptor, based on the histogram of the phase congruency orientation, is extracted. These descriptors are matched with defined regions of the body of a pedestrian. In case of a match, it creates a region of interest in the image, which is classified as a pedestrian / non-pedestrian by an SVM classifier. The pedestrian detection system has been tested in an advanced driver assistance system for urban driving.

Trustworthiness is a key point when dealing with vehicle safety applications. In this paper an approach to a real application is presented, able to fulfill the requirements of such demanding applications. Most of commercial sensors available nowadays are usually designed to detect front vehicles but lack the ability to detect overtaking vehicles. The work presented here combines the information provided by two sensors, a Stop&Go radar and a camera. Fusion is done by using the unprocessed information from the radar, and computer vision based on optical flow. The basic capabilities of the commercial systems are upgraded giving the possibility to improve the front vehicles detection system, by detecting overtaking vehicles with a high positive rate.

The movement of the vehicle provides useful information for different applications, such as driver assistant systems or autonomous vehicles. This information can be known by means of a GPS, but there are some areas in urban environments where the signal is not available, as tunnels or streets with high buildings. A new method for 2D visual ego motion estimation in urban environments is presented in this paper. This method is based on a stereo-vision system where the feature road points are tracked frame to frame in order to estimate the movement of the vehicle, avoiding outliers from dynamic obstacles. The road profile is used to obtain the world coordinates of the feature points as a unique function of its left image coordinates. For these reasons it is only necessary to search feature points in the lower third of the left images. Moreover, the Kalman filter is used as a solution for filtering problem. That is, in some cases, it is necessary to filter raw data due to noise acquisition of time series. The results of the visual ego motion are compared with raw data from a GPS.

In this paper a new contrast invariant descriptor for human detection in long-wave infrared images is proposed. It exploits local information histogram of orientations of phase coherence. Contrast in infrared images depends on the temperature of the object and the background, which makes gradient based descriptors less robust, especially in daylight conditions. The objective is to obtain a scale, brightness and contrast invariant descriptor that can successfully detect pedestrians in images taken with a cheap, temperature-sensitive, uncooled microbolometer. The descriptor, packed into grids is feed to a Support Vector Machine classifier. The algorithm has been tested in night and day sequences and its performance is compared with a day only descriptor: the histogram of oriented features (HOG).

The lack of trustworthy sensors makes ADAS applications a tough task. In this paper a fusion approach that uses two sensors: far infrared camera vision and a laser scanner to detect pedestrian for ADAS applications is presented. Both sensors have different field of view thus different detection zones are created according to the number of sensors available. The proposed algorithm combines low and high level information, providing a hybrid fusion scheme. Experimental results show that pedestrian detections are improved thanks to the use of several sensors.

This paper deals with the extraction of part of the visual information presented in streets, roads, and motorways. This information, provided by either traffic or road signs and route-guidance signs, is extremely important for safe and successful driving. An automatic system that is capable of extracting and identifying these signs automatically would help human drivers enormously; navigation would be easier and would allow him or her to concentrate on driving the vehicle. The system would indicate to the driver the presence of a sign in advance, so that some incorrect human decisions could be avoided. A deformable model scheme allows us to include the knowledge used while designing the signs in the algorithm and is used for their detection and identification. Two techniques to find the minimum in the energy function are shown: simulated annealing and genetic algorithms. Some problems are addressed, such as uncontrolled lighting conditions; occlusions; and variations in shape, size, and color.

This paper presents practical, high-level topological navigation tasks, making use of our general purpose landmark learning and detection system, which includes the possibility of reading text or icons inside detected landmarks. Room identification from inside, without any initialization, is achieved through its landmark signature. Room search along a corridor is done by reading the content of room nameplates placed around for human use; this allow the robot to take high-level decisions, and results in a higher integration degree of mobile robotics in real life.

Deformable models have been intensively studied in image analysis through the last decade, and are used for detection and recognition of flexible or rigid templates under diverse viewing conditions. Genetic algorithm (GA) based deformable models are used for generic visual landmark detection and interpretation. The developed system allows topologic localization and navigation using natural and artificial landmarks, exploiting deformable models' ability for handling landmark perspective variations. The resulting perception module has been integrated successfully in a complex navigation system. Various experimental results in real environments are presented on this paper, showing the effectiveness and capacity of the landmark detection and reading system.

This paper describes a vision-based landmark recognition system for use with mobile robot navigation tasks. A search algorithm based on genetic techniques for pattern recognition in digital images is presented. The developed system allows the topologic localization of a mobile robot using natural and artificial landmarks. Text strings inside landmarks can be read and interpreted, if present. The resulting system was tested onboard a B21 mobile robot and proved useful. The presented experimental results show the effectiveness of the proposed algorithm.

This paper presents a fuzzy perception planner that takes into account the time cost, the suitability of every landmark detection and the different uncertainties the robot encounters along its path for mobile robot localization. The sensor used is a camera with a motorized zoom on a pan and tilt platform and the artificial landmarks are circles detected through normalized gray scale correlation. An extended Kalman filter is used to correct the position and orientation of the vehicle. The resulting self-localization module has been integrated successfully in a more complicated navigation system.