Autonomous vehicles must be capable of driving safely and having some level of social compliance with human drivers. Acting egoistically can make other drivers to take undesirable actions, such as performing hard brakes to avoid collisions. Designing a proper behavior involves dealing with antagonistic objectives, such as increasing speed and avoiding rear-end collisions. However, weighting those objectives in a reward or cost function is an error-prone and time-consuming task, and can become very hard as more features are added to the problem. In this regard, the main objective of this paper is to use learning from demonstration to compute trajectories that mimic human expert behavior without the need of manually tuning a reward function. On this subject, we present a variation of the well-known Maximum Entropy Inverse Reinforcement Learning algorithm in order to deal with continuous state spaces: instead of exactly computing the gradients, they are estimated by sampling trajectories in regions with higher rewards using a Monte Carlo Tree Search based sampler. The sampler is applied to solve an interaction-aware Markov Decision Problem capable of dealing with the inherent interaction and uncertainty present in surrounding vehicles motion. Experiments are performed in a merging scenario considering real data, showing that the proposed method can generate trajectories similar to the ones executed by human drivers. Additionally, favorable results are achieved when compared to traditional baseline methods and also to a variant of Inverse Reinforcement Learning that uses a polynomial-curve trajectory sampler.

The dynamic identification of a robot manipulator is a canonical problem in robotics and is usually solved by employing the least squares technique, considering the so-called base parameters of the robot. Such parameters, however, differ from those to which we really want to have access, such as the center of mass and the moments of inertia with respect to this center. Furthermore, the solution obtained by this technique can lead to unsuitable parameters from a physical point of view, which requires a later stage of physical feasibility analysis. In this article, we seek to overcome these drawbacks by identifying the parameters of a 7-DoF lightweight robot using only population-based metaheuristics. We make a study between different techniques and compare them with Bayesian optimization, another black-box approach. We also propose a new algorithm called Probabilistic Differential optimization and show that this algorithm is more efficient than the others considered and can find a physically feasible solution because of its probabilistic exploration ability. We also study the need for more than one excitation trajectory to overcome measurement noise and overfitting, and finally obtain accurate parameters of the Kinova Gen3 arm equipped with a Robotiq 2F-85 gripper.

The ability to deal with systems parametric uncertainties is an essential issue for heavy self-driving vehicles in unconfined environments. In this sense, robust controllers prove to be efficient for autonomous navigation. However, uncertainty matrices for this class of systems are usually defined by algebraic methods which demand prior knowledge of the system dynamics. In this case, the control system designer depends on the quality of the uncertain model to obtain an optimal control performance. This work proposes a robust recursive controller designed via multiobjective optimization to overcome these shortcomings. Furthermore, a local search approach for multiobjective optimization problems is presented. The proposed method applies to any multiobjective evolutionary algorithm already established in the literature. The results presented show that this combination of model-based controller and machine learning improves the effectiveness of the system in terms of robustness, stability and smoothness.

Multi-Task Learning is a learning paradigm that uses correlated tasks to improve performance generalization. A common way to learn multiple tasks is through the hard parameter sharing approach, in which a single architecture is used to share the same subset of parameters, creating an inductive bias between them during the training process. Due to its simplicity, potential to improve generalization, and reduce computational cost, it has gained the attention of the scientific and industrial communities. However, tasks often conflict with each other, which makes it challenging to define how the gradients of multiple tasks should be combined to allow simultaneous learning. To address this problem, we use the idea of multi-objective optimization to propose a method that takes into account temporal behaviour of the gradients to create a dynamic bias that adjusts the importance of each task during backpropagation. The result of this method is to give more attention to tasks that are diverging or not being benefited during the last iterations, ensuring that the simultaneous learning is heading to the performance maximization of all tasks. As a result, we empirically show that the proposed method outperforms the state-of-the-art approaches on learning conflicting tasks. Unlike the adopted baselines, our method ensures that all tasks reach good generalization performances.

To autonomously navigate in traffic roads, an Autonomous Vehicle must take into account perception information, as well as the topological and geometric structure of the environment it is inserted in. Specifically in urban scenarios, the vehicle has to plan its path across intersections, roundabouts and perform lane changes to obey traffic rules. In addition, the planning algorithm must also consider the kinematics constraints of the vehicle and comfort parameters to the passengers. This paper proposes a road network model based on clothoids, which embraces the geometric and topological representation of the environment in a compact data structure. Piecewise linear continuous-curvature paths composed of clothoids, circular arcs, and straight lines are used for this purpose. The proposed approaches are evaluated in an urban scenario composed of curved and straight roads with single and double lanes, roundabouts, and intersections. As a result, a navigation architecture for Autonomous Vehicles was developed using the model, including global planning with continuous-curvature paths.

Advanced driver assistance systems are responsible for assisting decision making and can play an important role in safety and traffic efficiency. Such systems require robust perception methods to handle complex urban scenes, and one way to achieve this is through instance segmentation. However, due to the difficulty in separating overlapping objects into different instances, this task becomes very challenging. For this, several authors proposed CNN-based methods and used depth information to enhance the instance segmentation performance. A promising way to explore this information is by adopting a multi-task learning approach, in which multiple tasks are learned simultaneously by sharing the same architecture. Usually, this combination is made by the weighted sum of loss functions, in which the weight of each task is defined manually. Nonetheless, when tasks have different natures with variation in the order of magnitude, performing this combination during training so that all tasks converge towards their optimal solution is not trivial. Aiming to get the best possible solution, we modeled the multi-task learning as a multiobjective optimization problem and, as the main contribution of this paper, we proposed a greedy approach to find the weighting coefficients for each task, performing a trade-off between tasks that allow the optimization of multiple loss functions. Experimental results showed that it is possible to enhance instance segmentation when depth information is properly explored. Moreover, not only did depth information help instance segmentation, but also did the instance segmentation help the depth estimations, achieving better performance compared to single-task models.

Deep Reinforcement Learning (DRL) represents an interesting class of algorithms, since its objective is to learn a behavioral policy through interaction with the environment, leveraging the function approximation properties of neural net-works. Nonetheless, for episodic problems, it is usually modeled to deal with a unique goal. In this sense, some works showed that it is possible to learn multiple goals when using a Universal Value Function Approximator (UVFA), i.e. a method to learn a universal policy by taking information about the current state of the agent and the goal. Their results are promising but show that there is still space for new contributions regarding the integration of the goal information into the model. For this reason, we propose using the Hadamard product or the Gated-Attention module in the UVFA architecture for visual-based problems. Also, we propose a hybrid exploration strategy based on the ε-greedy and the categorical probability distribution, namely ε-categorical. By systematically comparing different architectures of UVFA for different exploration strategies, and applying or not the Trust Region Policy Optimization (TRPO), we demonstrate through experiments that, for visual topologic navigation, combining visual information of the current and goal states through Hadamard product or Gated-Attention module allows the network learning near-optimal navigation policies. Also, we empirically show that the ε-categorical policy helps to avoid local minimums during the training, which facilitates the convergence to better results.

In the automation of many kinds of processes, the observable outcome can often be described as the combined effect of an entire sequence of actions, or controls, applied throughout the process execution. In these cases, strategies to optimise control policies for individual stages of the process are not applicable, and instead the whole policy needs to be optimised at once. On the other hand, the cost to evaluate the policy's performance might also be high, being desirable that a solution can be found with as few interactions as possible with the real system. We consider the problem of optimising control policies to allow a robot to complete a given race track within a minimum amount of time. We assume that the robot has no prior information about the track or its own dynamical model, just an initial valid driving example. Localisation is only applied to monitor the robot and to provide an indication of its position along the track's centre axis. With that in mind, we propose a method for finding a policy that minimises the time per lap while keeping the vehicle on the track using a Bayesian optimisation (BO) approach over a reproducing kernel Hilbert space. We apply an algorithm to search more efficiently over high-dimensional policy-parameter spaces with BO, by iterating over each dimension individually, in a sequential coordinate descent-like scheme. Experiments demonstrate the performance of the algorithm against other methods in a simulated car racing environment.

Intelligent vehicles require an efficient way to compute a feasible path that connects its current localization to a destination point. To achieve that, the knowledge about the road network, including its geometry, is important since connections between roads can be used in the path planning. This work consists on computing a feasible global path for autonomous vehicles with kinematic constraints. Piecewise linear continuous-curvature paths composed of clothoids, circular arcs, and straight lines are used for this purpose. Low curvature derivatives provide comfort to passengers. This approach provides a compact road network representation as only the parameters of the curves are stored. A real urban scenario with straight and curved roads, multiple lanes, intersections, and roundabouts is modeled and the proposed approach is validated. As a result of the proposed approaches, door-to-door global continuous-curvature paths are generated considering the shortest traveled distance.

Robust and stable control is a requirement for navigation of self-driving cars. Some approaches in the literature depend on a high number of parameters that are often difficult to estimate. A poor selection of these parameters often reduces considerably the efficiency of the control algorithms. In this paper we propose a simplified control system for autonomous vehicles that depends on a reduced number of parameters that can be easily set. This control system is composed of longitudinal and lateral controllers. The longitudinal controller is responsible for regulating the vehicle's cruise velocity while the lateral controller steers the vehicle's wheels for path tracking. Simulated and experimental tests have been carried out with the CaRINA II platform in the university campus with positive results.

The development of autonomous vehicles is a highly relevant research topic in mobile robotics. Road recognition using visual information is an important capability for autonomous navigation in urban environments. Over the last three decades, a large number of visual road recognition approaches have been appeared in the literature. This paper proposes a novel visual road detection system based on multiple artificial neural networks that can identify the road based on color and texture. Several features are used as inputs of the artificial neural network such as: average, entropy, energy and variance from different color channels (RGB, HSV, YUV). As a result, our system is able to estimate the classification and the confidence factor of each part of the environment detected by the camera. Experimental tests have been performed in several situations in order to validate the proposed approach.

We describe the development and assessment of a computer controlled wheelchair called the SMARTCHAIR. A shared control framework with different levels of autonomy allows the human operator to stay in complete control of the chair at each level while ensuring her safety. The framework incorporates deliberative motion plans or controllers, reactive behaviors, and human user inputs. At every instant in time, control inputs from these three different sources are blended continuously to provide a safe trajectory to the destination, while allowing the human to maintain control and safely override the autonomous behavior. In this paper, we present usability experiments with 50 participants and demonstrate quantitatively the benefits of human-robot augmentation.

We describe the development and assessment of a computer controlled wheelchair equipped with a suite of sensors and a novel interface, called the SMARTCHAIR. The main focus of this paper is a shared control framework which allows the human operator to interact with the chair while it is performing an autonomous task. At the highest level, the autonomous system is able to plan paths using high level deliberative navigation behaviors depending on destinations or waypoints commanded by the user. The user is able to locally modify or override previously commanded autonomous behaviors or plans. This is possible because of our hierarchical control strategy that combines three independent sources of control inputs: deliberative plans obtained from maps and user commands, reactive behaviors generated by stimuli from the environment, and user-initiated commands that might arise during the execution of a plan or behavior. The framework we describe ensures the user's safety while allowing the user to be in complete control of a potentially autonomous system.