In this work, we propose a novel Decision Transformer-based framework for tuning the parameters of Model Predictive Control (MPC). First, we show that an MPC scheme with quadratic cost, linear constraints, and a nominal linear model can reproduce the optimal solution of an infinite-horizon nonlinear regulation problem when the cost and constraint parameters are tuned based on the history of states and control inputs. Then, we formulate parameter tuning as a sequence modeling problem and develop a Decision Transformer-based framework, referred to as MPC-Decisioner, which leverages the attention mechanism of Decision Transformers to exploit historical and contextual information and generate MPC parameters online, conditioned on trajectories of costs, states, and past parameters. The resulting framework offers interpretability through the attention scores of the Transformer and achieves improved closed-loop performance compared to baseline MPC parameter tuning methods. Its effectiveness is demonstrated through simulation studies.

Driving simulators are essential tools to guide automotive research and development. Their motion system requires a motion cueing algorithm (MCA) to keep the simulator motion platform within its physical boundaries, while simultaneously aiming to recreate the sensation of real vehicle motion. While traditional, filter-based approaches are still predominant, optimization-based MCAs have been at the center of MCA research for over a decade due to their ability to systematically improve motion cueing quality through explicit cost function design and constraints handling. However, despite their demonstrated advantages, these optimization-based methods have not yet achieved widespread adoption in driving simulation. This paper therefore provides a comprehensive review of optimization-based MCAs for driving simulation, categorizing and comparing algorithms, describing their key developments and core characteristics. The current limited real-time capability, lack of accurate evaluation methods, challenges in cost function design and its tuning, and the current lack of accurate future reference predictions are identified as key barriers to the practical deployment and widespread use of optimization-based MCAs. These theoretical and practical challenges are further reviewed, providing guidelines to advance the theory and application of optimization-based MCAs. Central in these advancements are a better understanding of which motions constitute a realistic motion experience, a framework allowing to compare the achieved motion fidelity of MCAs across papers, the design of the cost function focusing on human motion perception, and techniques for easing up the tuning process to swiftly reach high quality tunings for different simulators, scenarios, and use cases. We identify the need for improving the real-time capability, and providing high quality motion reference predictions using learning-based approaches on diverse datasets, along with techniques to handle existing uncertainties. Following these guidelines, new foundations for optimization-based algorithms in driving simulation can be achieved, which will significantly impact the research and development of automotive systems.

Grasping and manipulating various kinds of objects cooperatively is the core skill of a dual-arm robot when deployed as an autonomous agent in a human-centered environment. This requires fully exploiting the robot's versatility and dexterity. In this work, we propose a general framework for dual-arm manipulators that contains two correlative modules. The learning-based dexterity-reachability-aware perception module deals with vision-based bimanual grasping. It employs an end-to-end evaluation network and probabilistic modeling of the robot's reachability to deliver feasible and dexterity-optimum grasp pairs for unseen objects. The optimization-based versatility-oriented control module addresses the online cooperative manipulation control by using a hierarchical quadratic programming formulation. Self-collision avoidance and dual-arm manipulability ellipsoid tracking with high reliability and fidelity are simultaneously achieved based on a learned lightweight distance proxy function and a speed-level tracking technique on Riemannian manifold. Intrinsic system safety is guaranteed, and a novel interface for skill transfer is enabled. A long-horizon rearrangement experiment, a bimanual turnover manipulation, and multiple comparative performance evaluation verify the effectiveness of the proposed framework.

This paper presents a model predictive control (MPC) method for redundant robots controlling multiple hierarchical tasks formulated as multi-layer constrained optimal control problems (OCPs). The proposed method, named hierarchical incremental MPC (HIMPC), is robust to dynamic uncertainties, untethered from kinematic/algorithmic singularities, and capable of handling input and state constraints such as joint torque and position limits. To this end, we first derive robust incremental systems that approximate uncertain system dynamics without computing complex nonlinear functions or identifying model parameters. Then the constrained OCPs are cast as quadratic programming problems which result in linear MPC, where dynamically-consistent task priority is achieved by deploying equality constraints and optimal control is attained under input and state constraints. Moreover, hierarchical feasibility and recursive feasibility are theoretically proven. Since the computational complexity of HIMPC drastically decreases compared with nonlinear MPC-based methods, it is implemented under the sampling frequency of 1 kHz for physical experiments with redundant manipulator setups, where robustness (high tracking accuracy and enhanced dynamic consistency), admissibility of multiple constraints, and singularity-avoidance nature are demonstrated and compared with state-of-the-art task-prioritized controllers.

For the safe application of reinforcement learning algorithms to high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning (SRL) framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. By employing an online adaptation method, the low-dimensional representation is updated using the feedback data to obtain more accurate safety estimates. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is illustrated using the example of a quadcopter. The results demonstrate a more reliable and representative low-dimensional representation of the safe region compared with previous works, which extends the applicability of the SRL framework.

Although the state-of-the-art learning approaches exhibit impressive results for dynamical systems, only a few applications on real physical systems have been presented. One major impediment is that the intermediate policy during the training procedure may result in behaviors that are not only harmful to the system itself but also to the environment. In essence, imposing safety guarantees for learning algorithms is vital for autonomous systems acting in the real world. In this article, we propose a computationally effective and general safe learning framework, specifically for complex dynamical systems. With a proper definition of the safe region, a supervisory control strategy, which switches the actions applied on the system between the learning-based controller and a predefined corrective controller, is given. A simplified system facilitates the estimation of the safe region for the high-dimensional dynamical system. During the learning phase, the belief of the safe region is updated with the actual execution results of the corrective controller, which in turn enables the learning-based controller to have more freedom in choosing its actions. Two examples are given to demonstrate the performance of the proposed framework, one simple inverted pendulum to illustrate the online adaptation method, and one quadcopter control task to show the overall performance.

We introduce a novel concept based on maneuver templates, which are formalized collaborative maneuvers, to select cooperative driving strategies. The approach is based on the exclusion principle, where we derive provable conditions to discard unsafe cooperative maneuvers for a given traffic situation. We thereby consider the full action set of all cooperative vehicles and do not discretize the action space. We demonstrate the applicability of our approach with numerical examples.

This paper introduces an online motion planning algorithm and a motion generation methodology for underactuated dynamic planar walking on uneven terrain. The key idea is to utilize a database of Motion Primitives and use them as training examples in a regression methodology, which is utilized when there is no match between the terrain variation and the Motion Primitives in the database. Among the key features which enable the algorithm to be suitable for real-time purposes is the proposed best first graph search approach and the small inference time of the regression methodology, which in this paper is the Gaussian Process.

This paper introduces a novel way to improve the settling time of transitions between different walking controllers. This improvement is achieved by commanding a sequence of intermediate transitions to the target controller. As a result, the state of the system enters the domain of attraction of the target controller closer to the fixed point of the Poincaré Map. The method is applicable to any walking robot with one degree of underactuation. The problem is expressed as a Markov Decision Process and then solved with Reinforcement Learning. In order to simplify the stability analysis of underactuated walking the Hybrid Zero Dynamics framework is utilized. Another advantage of using the Hybrid Zero Dynamics is the dimensionality reduction of the state representation in the Markov Decision Process. The experimental results suggest that the proposed methodology performs better than a one-step transition for 84.34% of all the considered transitions for a simulated walking robot matching the parameters of RABBIT [1].

We propose an online compensation scheme for rollover prevention of mobile manipulators based on the invariance control framework, and that makes use of recursively computed analytic gradients of the zero-moment point (ZMP) function. Our controller relaxes many of the assumptions made in existing approaches, and enhances robustness as well as effectiveness through the use of exact gradient information. Several case studies demonstrate the improved performance of our controller over existing rollover prevention schemes.