Explaining the decision made by end-to-end autonomous driving is a difficult task. These approaches take raw sensor data and compute the decision as a black box with large deep learning models. Understanding the output of deep learning is a complex challenge due to the complicated nature of explainability; as data passes through the network, it becomes untraceable, making it difficult to understand. Explainability increases confidence in the decision by making the black box that drives the vehicle transparent to the user inside. Achieving a Level 5 autonomous vehicle necessitates the resolution of that challenging task. In this work, we propose a model that leverages the driver’s attention to obtain explainable decisions based on an attention map and the scene context. Our novel architecture addresses the task of obtaining a decision and its explanation from a single RGB sequence of the driving scene ahead. We base this architecture on the Transformer architecture with some efficiency tricks in order to use it at a reasonable frame rate. Moreover, we integrate in this proposal our previous ARAGAN model, which obtains SOTA attention maps, to improve the performance of the model thanks to understand the sequence as a human does. We train and validate our proposal on the BDD-OIA dataset, achieving on-pair results or even better than other state-of-the-art methods. Additionally, we present a simulation-based proof of concept demonstrating the model’s performance as a copilot in a close-loop vehicle to driver interaction.

Motion Prediction (MP) of multiple surroundings agents is a crucial task in arbitrarily complex environments, from simple robots to Autonomous Driving Stacks (ADS). Current techniques tackle this problem using end-to-end pipelines, where the input data is usually a rendered top-view of the physical information and the past trajectories of the most relevant agents; leveraging this information is a must to obtain optimal performance. In that sense, a reliable ADS must produce reasonable predictions on time. However, despite many approaches use simple ConvNets and LSTMs to obtain the social latent features, State-of-the-Art (SOTA) models might be too complex for real-time applications when using both sources of information (map and past trajectories) as well as little interpretable, specifically considering the physical information. Moreover, the performance of such models highly depends on the number of available inputs for each particular traffic scenario, which are expensive to obtain, particularly, annotated High-Definition (HD) maps. In this work, we propose several efficient baselines for the well-known Argoverse 1 Motion Forecasting Benchmark. We aim to develop compact models using SOTA techniques for MP, including attention mechanisms and GNNs. Our lightweight models use standard social information and interpretable map information such as points from the driveable area and plausible centerlines by means of a novel physics-based heuristic step based on kinematic constraints, in opposition to black-box CNN-based or too-complex graphs methods for map encoding, to generate plausible multi-modal trajectories achieving up-to-pair accuracy with less operations and parameters than other SOTA methods. Our code is publicly available at https://github.com/Cram3r95/mapfe4mp.

Predicting driver’s attention in complex driving scenarios is becoming a hot topic due to it helps the design of some autonomous driving tasks, optimizing visual scene understanding and contributing knowledge to the decision making. We introduce ARAGAN, a driver attention estimation model based on a conditional Generative Adversarial Network (cGAN). This architecture uses some of the most challenging and novel deep learning techniques to develop this task. It fuses adversarial learning with Multi-Head Attention mechanisms. To the best of our knowledge, this combination has never been applied to predict driver’s attention. Adversarial mechanism learns to map an attention image from an RGB traffic image while mapping the loss function. Attention mechanism contributes to the deep learning paradigm finding the most interesting feature maps inside the tensors of the net. In this work, we have adapted this concept to find the saliency areas in a driving scene. An ablation study with different architectures has been carried out, obtained the results in terms of some saliency metrics. Besides, a comparison with other state-of-the-art models has been driven, outperforming results in accuracy and performance, and showing that our proposal is adequate to be used on real-time applications. ARAGAN has been trained in BDDA and tested in BDDA and DADA2000, which are two of the most complex driver attention datasets available for research.

Autonomous vehicle (AV) is one of the most challenging engineering tasks of our era. High-Definition (HD) maps are a fundamental tool in the development of AVs, being considered as pseudo sensors that provide a trusted baseline that other sensors cannot. Our approach is focused on the use of OpenDRIVE standard based HD maps in order to conduct the different mapping and planning tasks involved in Autonomous Driving (AD). In this paper we present a method for exploiting the HD map potential for two specific purposes: i) Global Path Planning and ii) Monitoring the relevant lanes and regulatory elements around the ego-vehicle to support the perception module. Mapping and planning modules are connected to the other modules of the AV stack by using ROS (Robot Operating System). Our AD architecture has been validated both in local and CARLA Autonomous Driving Leaderboard cloud, where we can appreciate a considerable improvement in the metrics by incorporating information from the HD map, not only used to conduct the Global Path Planning task but also providing prior information to the Perception module. Code is available in https://github.com/AlejandroDiazD/opendrive-mapping-planning.

Abstract In this work, we present a complete system to produce an automatic linguistic reporting about the customer activity patterns inside open malls, a mixed distribution of classical malls joined with the shops on the street. These reports can assist to design marketing campaigns by means of identifying the best places to catch the attention of customers. Activity patterns are estimated with process mining techniques and the key information of localization. Localization is obtained with a parallelized solution based on WiFi fingerprint system to speed up the solution. In agreement with the best practices for human evaluation of natural language generation systems, the linguistic quality of the generated report was evaluated by 41 experts who filled in an online questionnaire. Results are encouraging, since the average global score of the linguistic quality dimension is 6.17 (0.76 of standard deviation) in a 7-point Likert scale. This expresses a high degree of satisfaction of the generated reports and validates the adequacy of automatic natural language textual reports as a complementary tool to process model visualization.

An indoor positioning fusion sensor composed of a five infrared detector set and one camera is presented in this work. The position of the target is obtained by hyperbolic trilateration from phase-difference measurements with the infrared sensor and by an homography with the camera. Subsequent fusion is carried out with a maximum likelihood estimator. A model is proposed for the infrared and camera observation variances and their propagation to the fusion estimation covariance matrix. The system shows cm-level accuracy in infrared multipath-free conditions and good matching with the model. Real measurements are conducted to assess sensor performance, also compared with Monte Carlo simulations for model validation. The evaluation of the fusion sensor performance is specially focused on its dependence on the camera resolution, tested at three resolution levels.

The number of applications for smartphones and tablets is growing exponentially in the last years. Many of these applications are supported by the so-called Location Based Services, which are expected to provide reliable real-time localization anytime and anywhere, no matter either outdoors or indoors. Even though outdoors world-wide localization has been successfully developed through the well-known Global Navigation Satellite System technology, its counterpart large-scale deployment indoors is not available yet. In previous work, we have already introduced a novel technology for indoor localization supported by a WiFi fingerprint approach. In this paper, we describe how to enhance such approach through the combination of hierarchical localization and fuzzy classifier ensembles. It has been tested and validated at the University of Edinburgh, yielding promising results.

Indoor localization is increasingly required for applications like deployment of rescue teams in emergency situations, proactive care for the elders, and so on. The quick growing of coverage of WiFi networks makes WiFi technology a very promising choice for indoor localization. But, this localization should be linked to a map to be useful. This work presents an open-access software designed for that purpose. It is composed of two different applications, a desktop software for research purposes and an Android application for user friendly localization. We address the localization task as a high dimensional classification problem. So far, we have developed classifiers based on the classic Nearest Neighbour, Support Vector Machines (SVM) and fuzzy rule-based classifiers. This work is made in the context of the ABSYNTHE project which is aimed at creating human-robot teams. We show a use case of the new software in one of the scenarios of the ABSYNTHE project.

Localization and mapping in indoor environments, such as airports and hospitals, are key tasks for almost every robotic platform. Some researchers suggest the use of RO (Range Only) sensors based on WiFi (Wireless Fidelity) technology with SLAM (Simultaneous Localization And Mapping) techniques. The current state of the art in RO SLAM is mainly focused on the filtering approach, while the study of smoothing approach with RO sensors is quite incomplete. This paper presents a comparison between a filtering algorithm, the EKF, and a smoothing algorithm, the SAM (Smoothing And Mapping). Experimental results are obtained, first in an outdoor environment using two types of RO sensors and then in an indoor environment with WiFi sensors. The results demonstrate the feasibility of the smoothing approach with WiFi sensors in indoors.

This work presents a general framework for people indoor activity recognition. Firstly, a Wireless Fidelity (WiFi) localization system implemented as a Fuzzy Rule-based Classifier (FRBC) is used to obtain an approximate position at the level of discrete zones (office, corridor, meeting room, etc). Secondly, a Fuzzy Finite State Machine (FFSM) is used for human body posture recognition (seated, standing upright or walking). Finally, another FFSM combines both WiFi localization and posture recognition to obtain a robust, reliable, and easily understandable activity recognition system (working in the desk room, crossing the corridor, having a meeting, etc). Each user carries with a personal digital agenda (PDA) or smart-phone equipped with a WiFi interface for localization task and accelerometers for posture recognition. Our approach does not require adding new hardware to the experimental environment. It relies on the WiFi access points (APs) widely available in most public and private buildings. We include a practical experimentation where good results were achieved.

The goal of this paper is to study some of the most important WiFi signal variations, large and small scale variations and how they affect to WiFi localization systems. Moreover, the paper shows how to use soft computing techniques to deal with these uncertainties in WiFi localization systems. This work describes how to reduce uncertainty produced by small scale variations in indoor environments using fuzzy techniques. Some experimental results and conclusions are presented.

In this paper we present a new real-time hierarchical (topological/metric) Visual SLAM system focusing on the localization of a vehicle in large-scale outdoor urban environments. It is exclusively based on the visual information provided by both a low-cost wide-angle stereo camera and a low-cost GPS. Our approach divides the whole map into local sub-maps identified by the so-called fingerprint (reference poses). At the sub-map level (low level SLAM), 3D sequential mapping of natural landmarks and the vehicle location/orientation are obtained using a top-down Bayesian method to model the dynamic behavior. A higher topological level (high level SLAM) based on references poses has been added to reduce the global accumulated drift, keeping real-time constraints. Using this hierarchical strategy, we keep local consistency of the metric sub-maps, by mean of the EKF, and global consistency by using the topological map and the MultiLevel Relaxation (MLR) algorithm. GPS measurements are integrated at both levels, improving global estimation. Some experimental results for different large-scale urban environments are presented, showing an almost constant processing time.

This paper presents a new real-time hierarchical (topological/metric) simultaneous localization and mapping (SLAM) system. It can be applied to the robust localization of a vehicle in large-scale outdoor urban environments, improving the current vehicle navigation systems, most of which are only based on Global Positioning System (GPS). Then, it can be used on autonomous vehicle guidance with recurrent trajectories (bus journeys, theme park internal journeys, etc.). It is exclusively based on the information provided by both a low-cost, wide-angle stereo camera and a low-cost GPS. Our approach divides the whole map into local submaps identified by the so-called fingerprints (vehicle poses). In this submap level (low-level SLAM), a metric approach is carried out. There, a 3-D sequential mapping of visual natural landmarks and the vehicle location/orientation are obtained using a top-down Bayesian method to model the dynamic behavior. GPS measurements are integrated within this low-level improving vehicle positioning. A higher topological level (high-level SLAM) based on fingerprints and the multilevel relaxation (MLR) algorithm has been added to reduce the global error within the map, keeping real-time constraints. This level provides nearly consistent estimation, keeping a small degradation with GPS unavailability. Some experimental results for large-scale outdoor urban environments are presented, showing an almost constant processing time.

This paper presents a new method for real-time SLAM calculation applied to autonomous robot navigation in large environments without restrictions. It is exclusively based on the information provided by a cheap wide-angle stereo camera. Our approach divide the global map into local sub- maps identified by the so-called SIFT fingerprint. At the sub- map level (low level SLAM), 3D sequential mapping of natural land-marks and the robot location/orientation are obtained using a top-down Bayesian method to model the dynamic behavior. A high abstraction level to reduce the global accumulated drift, keeping real-time constraints, has been added (high level SLAM). This uses a SIFT correction method based on the sub-maps' fingerprints. A comparison of the low SLAM level using our method and SIFT features has been carried out. Some experimental results using a real large environment are presented.

This paper describes a comprehensive combination of feature extraction methods for vision-based pedestrian detection in Intelligent Transportation Systems. The basic components of pedestrians are first located in the image and then combined with a support-vector-machine-based classifier. This poses the problem of pedestrian detection in real cluttered road images. Candidate pedestrians are located using a subtractive clustering attention mechanism based on stereo vision. A components-based learning approach is proposed in order to better deal with pedestrian variability, illumination conditions, partial occlusions, and rotations. Extensive comparisons have been carried out using different feature extraction methods as a key to image understanding in real traffic conditions. A database containing thousands of pedestrian samples extracted from real traffic images has been created for learning purposes at either daytime or nighttime. The results achieved to date show interesting conclusions that suggest a combination of feature extraction methods as an essential clue for enhanced detection performance

The framework of this paper is the robotics navigation inside buildings using WiFi signal strength measure. This navigation is achieved using a partially observable Markov decision process (POMDP). In the localization phase we used WiFi signal strength and ultrasound measures as observations. The localization system works in two stages: map construction and localization stage. The map construction stage usually requires a great effort, therefore in this paper we address the problem of minimizing this calibration effort using an automatic training method. We describe the method based on simultaneous localization and mapping (SLAM) techniques and in a robust local navigation task. This automatic method is compared with a manual method to obtain a deterministic map. Also we demonstrate that using this one in a on-line training stage the system is able to adapt the WiFi map to the variations of the WiFi signal measure. Additionally, we analyze the optimal parameters for this automatic training system. The system has been tested in a real environment using two commercial robotic platforms. Some experimental results and the conclusions are presented

This paper presents a new method for real-time ego-motion calculation applied to the location/orientation of a cheap wide-angle stereo camera in a 3D environment. The objective is to apply it to a mobile robot navigation system. To achieve that, the goal is to solve the simultaneous localization and mapping (SLAM) problem. Our approach consists in the 3D sequential mapping of natural landmarks by means of a stereo camera, which also provides means to obtain the camera location/orientation. The dynamic behavior is modeled using a top-down Bayesian method. The results show a comparison between our system and a monocular visual SLAM system using a hand-waved camera. Several improvements related to no priori environment knowledge requirements, lower processing time (real-time constrained) and higher robustness is presented

This paper presents a robot navigation system for indoor environments using a partially observable Markov decision process (POMDP) based on WiFi signal strength and ultrasound observations. The paper represents the first one in using WiFi sensor readings as an observation in a POMDP. We present an algorithm based on an EM-SLAM that we called WSLAM (Wifi simultaneous localization and mapping) that is able to learn the observation and transition matrix in autonomous mode. With this algorithm we obtain a minimum calibration effort. We demonstrate that this system is useful to navigate in indoor environments with a real robot. Some experimental results are shown. Finally, the conclusions and future works are presented.