International audience

This paper proposes a reinforcement learning (RL)-based approach for dynamically selecting the most suitable control method according to changing operating conditions. Using the nominal model of the actual system, several feedback controllers are developed, each offering different levels of performance depending on the scenario. The RL algorithm is employed to determine and apply the optimal control strategy within a given operational range. Four control methods are investigated: Linear Parameter Varying (LPV), Ultra-local Model-based (ULM), Linear Quadratic Regulator (LQR), and a kinematic model-based controller. The performance and effectiveness of the proposed approach are assessed through three test scenarios using the high-fidelity vehicle simulation platform, CarMaker.

In the recent research of autonomous vehicle (AV) control methods, machine learning techniques are gaining more and more popularity due to their possible performance advantages over classical model based methods. As lane keeping or trajectory tracking through steering intervention is one of the key feature of the autonomous vehicle, several methods had been proposed and utilized both with model and machine learning based controllers. Present paper focuses on the integration of a robust Linear Parameter Varying (LPV) and a Reinforcement Learning (RL) controller in a supervisor structure, with the aim to guarantee stability against disturbances not considered during the training process of the RL agent. Hence, with the supervision of the proposed LPV controller, noisy or faulty GPS signals can be handled safely, without compromising the trajectory tracking performance of the RL agent during normal operation of the autonomous vehicle. For the demonstration of the proposed LPV supervised RL control method, highway simulations have been carried out and compared in CarSim simulation environment.

In the motion estimation of self-driving vehicles, the three main requirements are accuracy, robustness, and cost-effectiveness. The generally applied sensors and methods are the GNSS, inertial, and visual-odometry, but the contradictory requirements demand the integration of new ideas. The wheel odometry could be an adequate choice since the method is robust and cost-effective, but the accuracy of the estimation is limited by the parameter uncertainty, thus a calibration method should be included as well. However, the general parameter identification of a nonlinear model in the presence of noise has not been solved yet. The presented method is based on the assumption that noisy, but several measurements of GNSS and IMU sensors are available in a self-driving vehicle. In the proposed architecture, nonlinear least squares and optimal control techniques are combined in a unique way to compensate for the noise of the orientation and wheel rotation signals to achieve unbiased model calib ration. The performance of the developed algorithm and the accuracy of parameter estimation are demonstrated with detailed validation and a test with a real vehicle.

This paper presents a velocity planning method for autonomous vehicles (AVs) to guarantee safe interactions with pedestrians at unsignalized crosswalks and with surrounding vehicles on the AV’s route. The method is structured within a hierarchical framework that includes robust control, a learning-based component, and a supervisory element. The learning-based component is trained using reinforcement learning techniques to reduce traveling time, minimize control interventions, and set the priority ratio between the AV and pedestrians. The supervisory element employs scenario optimization, using statistical data on pedestrian motions to ensure collision avoidance. A complex game-theory-based pedestrian model is formulated and analyzed in order to evaluate the effectiveness of the proposed velocity planning method. Extensive simulations are performed using the high-precision traffic simulator software SUMO. These simulations evaluate various aspects of the velocity planner, including computation time, traveling time, control interventions, and parameter settings. The results demonstrate the method’s ability to achieve real-time implementation while maintaining safety and performance objectives.

The paper presents a novel observer design method for estimating the front and rear wheel slips of the vehicle. The proposed observer design technique consists of two parts: A simple linear observer algorithm, which uses a reformulated lateral vehicle model to estimate the tire slips. The second part is based on an ultra-local model. The main goal of the ultra-local model is to eliminate the nonlinear, unmodeled, uncertain dynamics of the lateral vehicle model. In this way, the performance level of the linear observer can be significantly increased especially under critical circumstances such as high lateral acceleration maneuvers or driving on a low µ surface. The proposed observer algorithm is implemented in MATLAB/Simulink environment connected to the high-fidelity simulation software, CarMaker. The operation and the effectiveness of the proposed observer are demonstrated through several simulation examples.

The online self-calibration is a required capability from an autonomous vehicle that should operate lifelong in a safe manner. This paper introduces relative weighting by a neural network into the batch Gauss–Newton calibration method of the wheel odometry model. A wheel odometry model with accurately estimated parameters could improve the motion estimation task of an autonomous vehicle, but the online parameter identification from only onboard measurements is a challenge due to the noises and the nonlinear behavior of the dynamic system. A possible solution to deal with the effect of noises is to calibrate the model with more segments at once forming a batch, but this can only reduce the distortion effect, not eliminate it. Our proposed algorithm improves this batch formulation by integrating relative weights for the segments to mitigate the distorting effect of noisy measurements. The method applies an AI-based tool to extract a proper weighting strategy from the previously recorded data that utilizes only online signals during operation. With the usage of the proposed architecture, the calibration accuracy significantly increased with the reduction of the distortion effect of faulty measurements, while the same amount of data is used as the raw batch estimation. The performance of the method is demonstrated with real measurements in a city driving with a passenger vehicle, where the calibration signals come from the equipped automotive-grade type of Global Navigation Satellite System and Inertial Measurement Unit.

This paper proposed a road adaptive semi-active suspension control method, where a trade-off between driving comfort and vehicle stability/road-holding is accomplishable in order to achieve desirable performance results at different road irregularities and velocities by modifying the scheduling variable that is designed by Fuzzy Logic Control. The proposed semi-active controller is founded on the Linear Parameter-Varying framework. Hungarian highway route data has been implemented into the TruckSim simulation environment based on real geographical data having road irregularities in order to compare the proposed adaptive method with a non-adaptive scenario. Simulation results show that all performances have been improved with the proposed method in different road irregularities and velocities.

Reinforcement learning applications are spreading among different domains, including autonomous vehicle control. The diverse situations that can happen during, for instance, at a highway commute are infinite, and with labeled data, the perfect coverage of all use-cases sounds ambitious. However, with the complex tasks and complicated scenarios faced during an autonomous vehicle system design, the credit assignment problem arises. How to construct appropriate objectives for the artificial intelligence to learn and the preferences between the different goals also matter of the designer’s choice. This work attempts to tackle the problem by utilizing successor features and providing a possible decomposition of the reward functions, guiding the agent’s actions. This method makes the training easier for the agent and enables immediate, profound performance on new combined tasks. Furthermore, with the optimal composition, the desired behavior can be fine-tuned, and as an auxiliary gain, the decomposition empowers different driving styles and makes driving preferences rapidly changeable. We introduce the adaptation of FastRL algorithm to autonomous vehicle domain, meanwhile developing a stabilizing way of using Successor Features, namely DoubleFastRL. We compare our solution for a highway driving scenario with basic agents such as Q-learning having multi-objective training.

In the era of self-driving vehicles, state estimation has 3 main contradictory requirements, such as accuracy, robustness, and cost-effectiveness. To satisfy all of them, the integration of the wheel encoder measurements is a proper choice besides the generally applied GNSS, inertial and visual-odometry methods. The wheel odometry is a robust and cost-effective method, but the accuracy of the estimation is limited by the proper knowledge of the vehicle parameter values. However, the calibration of the nonlinear odometry model in the presence of noise remains an open problem in the context of autonomous vehicles yet. This paper presents an algorithm that takes advantage of the assumption that more measurements are available in a self-driving vehicle for accurate parameter estimation. With the proposed architecture, the measurements with distortion effects are detected, and also the noise is corrected to reach unbiased model calibration. The performance of the developed algorithm and the accuracy of the parameter estimation are demonstrated with detailed validation and test with a real vehicle.

Recommender systems generate items that should be interesting for the customers. However, recommenders usually fail in the cold-start scenario - when a new item or a new customer appears. In our work, we study the cold-start problem for a new customer. For a cold-start customer we find the most similar customers and use a “their” pre-trained collaborative filtering model to recommend. We compare several recommendation approaches and similarity metrics to analyze the accuracy and computational performance.

The paper presents a motion planning solution which combines classic control techniques with machine learning.For this task, a reinforcement learning environment has been created, where the quality of the fulfilment of the designed path by a classic control loop provides the reward function.System dynamics is described by a nonlinear planar single track vehicle model with dynamic wheel mode model.The goodness of the planned trajectory is evaluated by driving the vehicle along the track.The paper shows that this encapsulated problem and environment provides a one-step reinforcement learning task with continuous actions that can be handled with Deep Deterministic Policy Gradient learning agent.The solution of the problem provides a real-time neural network-based motion planner along with a tracking algorithm, and since the trained network provides a preliminary estimate on the expected reward of the current state-action pair, the system acts as a trajectory feasibility estimator as well.Highly automated and autonomous driving is expected to enhance the quality of road transportation in multiple aspects, such as increasing the level of safety while reducing fuel consumption and emissions.The development potential makes the topic one of the most intense research fields both for vehicle industry and related academic institutions.This paper deals with the problem of feasible motion planning, i.e. the design and evaluation of the trajectory that the vehicle must follow.Many different approaches have been evolved over the years to solve the motion planning problem for wheeled vehicles, all having advantages and drawbacks as well.Geometric approaches assemble the path of the vehicle from geometric curves as clothoids, circular arcs and splines.A popular choice is to define curvature as function of arc length (Li et al., 2015).They are often used in simple lowa

Recommender systems aim at enhancing user experience on the Web by employing the results of users behavior analysis for recommending items. However, user behavior is usually influenced by various aspects. Even though visual stimuli greatly influence almost every part of our life, it is yet poorly reflected in the domain of recommendation. In our work, we study the impact of visual stimuli (specifically images) on recommendation process on the Web. We focus on the domains where the impact of images is substantial (e.g.,~movies and shopping). First results of our experiments suggest that features extracted from the images are able to improve the ranking of the current recommendation approaches.

There is a relationship between the traffic flow and the look-ahead control; they strongly interact with each other. Thus, this paper develops a design method for the look-ahead control, in which the influences of the traffic flow are considered. A sensitivity analysis of the parameter variation in the look-ahead control is also performed. If the traffic information is also considered in the look-ahead control, an undesirable side effect on the traffic flow may occur. An optimization method is also developed in order to calculate the optimum speed, which handles the individual vehicle energy optimization and its impact on the traffic flow. The method is illustrated through a complex simulation example based on the CarSim software.

One of the basis of the development in future intelligent transportation systems is the vehicle itself, whose provided features and communication possibilities are continuously expanding. One of the main areas in development of the vehicle of the future is mechatronics. Teaching this topic is a complex challenge because of the multidisciplinary nature of the topic. On the other hand, the requirements of the smart city or the smart mobility of the future demand complex solutions from next generation of engineers. The topics include electrical, mechanical engineering, control theory, and information and communications technology, in which fields the students must deepen their knowledge. Tasks and solving problems in mechatronics require cognitive and operational knowledge and practical experience in systems design and analysis. There is a strong demand for project-based teaching using certain kinds of practical material. This paper presents two educational frameworks designed for students with a specialization in vehicle mechatronics. Both are aimed at the emulation of real vehicle behaviors: the first is on the electric control unit level, whereas the second is on the vehicle level.

This paper is focusing on rollover prevention to provide a heavy vehicle with the ability to resist overturning moments generated during cornering. A combined yaw-roll model including the roll dynamics of unsprung masses is studied. This model is nonlinear with respect to the velocity of the vehicle. In our model the velocity is handled as an LPV scheduling parameter. The linear parameter-varying model of the heavy vehicle is transformed into a proper polytopic form by Tensor Product model transformation. The Hinfingain-scheduling based control is immediately applied to this form for the stabilization. The effectiveness of the designed controller is demonstrated by numerical simulation.