In the realm of control systems, model predictive control (MPC) has exhibited remarkable potential; however, its reliance on accurate models and substantial computational resources has hindered its broader application, especially within real-time nonlinear systems. This study presents an innovative control framework to enhance the practical viability of the MPC. The developed safe data-driven predictive control aims to eliminate the requirement for precise models and alleviate computational burdens in the nonlinear MPC (NMPC). This is achieved by learning both the system dynamics and the control policy, enabling efficient data-driven predictive control while ensuring system safety. The methodology involves a spatial temporal filter (STF)-based concurrent learning for system identification, a robust control barrier function (RCBF) to ensure the system safety amid model uncertainties, and a RCBF-based NMPC policy approximation. An online policy correction mechanism is also introduced to counteract performance degradation caused by the existing model uncertainties. Demonstrated through simulations on two applications, the proposed approach offers comparable performance to existing benchmarks with significantly reduced computational costs.

Data-enabled predictive control (DeePC) has garnered significant attention for its ability to achieve safe, data-driven optimal control without relying on explicit parametric models. Traditional DeePC methods use pre-collected input/output (I/O) data to construct a Hankel matrix offline and then formulate a predictive control framework online for linear, weakly nonlinear, and weakly stochastic systems. However, in systems with evolving dynamics, incorporating real-time data into the DeePC framework becomes crucial to enhance control performance. This paper proposes an online DeePC framework designed for strongly nonlinear and/or time-varying systems (i.e., systems with evolving dynamics), enabling the algorithm to update the Hankel matrix online by adding real-time informative signals. By exploiting the minimum non-zero singular value of the Hankel matrix, the developed online DeePC selectively integrates informative data and effectively captures evolving system dynamics. Additionally, a numerical singular value decomposition technique is introduced to reduce the computational complexity for updating a reduced-order Hankel matrix. Simulation results on three cases, linear time-varying system, vehicle anti-rollover control, and Li-ion battery fast charging, demonstrate the effectiveness of the proposed online reduced-order DeePC framework.

Optimal control schemes have achieved remarkable performance in numerous engineering applications. However, they typically require high computational cost, which has limited their use in real-world engineering systems. To address this challenge, Neighboring Extremal (NE) has been developed to adapt a pre-computed nominal control solution to perturbations from the nominal trajectory. The resulting control law is a time-varying feedback gain that can be pre-computed along with the original optimal control problem, and it takes negligible online computation. However, existing NE frameworks only deal with state perturbations while in modern applications, optimal controllers frequently incorporate preview information. Therefore, a new NE framework is needed to adapt to such preview perturbations. In this work, an extended NE (ENE) framework is developed to systematically adapt the nominal control to both state and preview perturbations. We show that the derived ENE law is two time-varying feedback gains on the state and preview perturbations. We also develop schemes to handle nominal non-optimal solutions and large perturbations to retain optimal performance and constraint satisfaction. Case study on nonlinear model predictive control is presented due to its popularity but it can be easily extended to other optimal control schemes. Promising simulation results on the cart inverted pendulum problem demonstrate the efficacy of the ENE algorithm. Note to Practitioners—Due to the vast success in predictive control and advancement in sensing, modern control applications have frequently been incorporating preview information in the control design. For example, the road profile preview obtained from vehicle crowdsourcing is exploited for simultaneous suspension control and energy harvesting, demonstrating a significant performance enhancement using the preview information despite noises in the preview (Hajidavalloo et al., 2022). Another example is thermal management for cabin and battery of hybrid electric vehicles, where traffic preview is employed in hierarchical model predictive control to improve energy efficiency (Amini et al., 2019). In Laks et al. (2011), light detection and ranging systems are used to provide wind disturbance preview to enhance the controls of turbine blades. In Yazdandoost et al. (2022), virtual water preview is employed using integrated water resources management modelling to optimize agricultural patterns and control level of water in lakes. In this work, we develop an extended neighboring extremal framework that can adapt a nominal control law to state and preview perturbations simultaneously. This setup is widely applicable as in many applications, a nominal preview is available while the preview signal can also be measured or estimated online.

Data-driven predictive control of connected and automated vehicles (CAVs) has received increasing attention as it can achieve safe and optimal control without relying on explicit dynamical models. However, employing the data-driven strategy involves the collection and sharing of privacy-sensitive vehicle information, which is vulnerable to privacy leakage and might further lead to malicious activities. In this paper, we develop a privacy-preserving data-enabled predictive control scheme for CAVs in a mixed traffic environment, where human-driven vehicles (HDVs) and CAVs coexist. We tackle external eavesdroppers and honest-but-curious central unit eavesdroppers who wiretap the communication channel of the mixed traffic system and intend to infer the CAVs’ state and input information. An affine masking-based privacy protection method is designed to conceal the true state and input signals, and an extended form of the data-enabled predictive leading cruise control under different data matrix structures is derived to achieve privacy-preserving optimal control for CAVs. Numerical simulations demonstrate that the proposed scheme can protect the privacy of CAVs against attackers without affecting control performance or incurring heavy computations.

Cloud computing is an emerging paradigm to enable computation and data-intensive automotive systems for improved safety and drivability. In this article, we propose a hierarchical, decentralized, and auction-based resource allocation model for cloud-enabled automotive vehicles. In this model, cloud-enabled vehicles bid for resources at a high level, inducing a multiplayer game; at a low level, each vehicle performs an onboard resource optimization to allocate its obtained resources to its cloud-based applications. The Nash equilibrium of the induced game is defined, and we show the existence and uniqueness of the equilibrium. A constrained optimization problem is solved for onboard resource allocation. A distributed update mechanism is considered: asynchronized update where only a subset of vehicles updates their bid at each iteration. This mechanism shares desired features of requiring little communication and being secure. Convergence to Nash equilibrium is proved for the proposed update mechanism. Furthermore, the robustness to stochastic task arrival rate is characterized in terms of total variance distance. Numerical simulations are presented to demonstrate the efficacy of the proposed framework.

Due to labor shortage and rising labor cost for the apple industry, there is an urgent need for the development of robotic systems to efficiently and autonomously harvest apples. In this paper, we present a system overview and algorithm design of our recently developed robotic apple harvester prototype. Our robotic system is enabled by the close integration of several core modules, including visual perception, planning, and control. This paper covers the main methods and advancements in deep learning-based multi-view fruit detection and localization, unified picking and dropping planning, and dexterous manipulation control. Indoor and field experiments were conducted to evaluate the performance of the developed system, which achieved an average picking rate of 3.6 seconds per apple. This is a significant improvement over other reported apple harvesting robots with a picking rate in the range of 7–10 seconds per apple. The current prototype shows promising performance towards further development of efficient and automated apple harvesting technology. Finally, limitations of the current system and future work are discussed.

It is important to accurately obtain the motion information of the ego vehicle and surrounding vehicles for autonomous vehicles to plan safe trajectories in complicated traffic environments. In this paper, a LiDAR-based estimation method is developed to simultaneously identify the pose and the velocity information of an ego vehicle and its surrounding moving obstacles. Specifically, a pose estimation network is designed to estimate the poses of both the ego vehicle and the obstacles only with the continuous point clouds obtained by the LiDAR mounted on the ego vehicle. In the network, PointNet++ is utilized as the backbone to extract point-wise features and divide the points into the static part and the moving part. The former is used to estimate the ego vehicle’s pose, while the latter is applied for the obstacle pose identification. Then, a reduced-order observer is designed to estimate the velocities, whose convergence is proved with the Lyapunov theory. Finally, both simulation and experiment results are provided to show the effectiveness of the proposed method.

In this article, the eye-in-hand visual trajectory tracking control problem of wheeled mobile robots (WMRs) is considered. Different from the conventional vision-based approaches, the monocular camera is not required to be mounted at the center of WMR, and thus the derived visual model is subject to not only the nonholonomic constraint but also the unknown camera extrinsic parameters. To ensure the WMR can track a desired trajectory effectively, a combined observation/control strategy is proposed. First, a concurrent learning observer is designed to identify the camera extrinsic parameters with measurable visual signals. Then, with the aid of the estimated parameters, a nonlinear controller is presented to achieve the tracking task. The closed-loop system stability is analyzed with Lyapunov methods, showing that both the estimation and tracking errors are asymptotically convergent to zero. Simulation and experimental results are provided to validate the developed approach.

In this paper, a 2-point algorithm is proposed to estimate the relative pose as well as the absolute scale between two vehicle-mounted cameras efficiently. The system model is deduced by combining a two-view geometric model and the planar motion constraint to reduce the degrees of freedom. 2-point correspondences are utilized to calculate the rotation information independently from the translation information, which indicates that the proposed algorithm can deal with pure rotation scenes. Besides, provided that the camera's configuration satisfies a certain condition, the absolute scale can be recovered. An approximation algorithm is developed and combined with the random sample and consensus scheme to deal with the uneven ground surfaces in practice. As only 2-point correspondences are required, less iterations are demanded in the estimating procedure compared with many other existing related algorithms. Both simulation and experiments are implemented to evaluate the proposed algorithm, in which the synthetic data, virtual robot experimentation platform, KITTI Vision Benchmark, and SUMMIT-XL platform are acquired. According to the results, the proposed algorithm performs better than many related algorithms including the well-known 5-point algorithm in many cases, especially when the camera's trajectory contains sharp corners.

In this paper, a visual servoing approach is developed for the trajectory tracking control and depth estimation problem of a mobile robot without a priori knowledge about desired velocities. By exploiting the multiple images captured by the on-board camera, the current and desired poses (i.e., scaled translation and orientation) of the mobile robot are reconstructed to define system errors. Then, an adaptive time-varying controller is proposed to achieve the trajectory tracking task in the presence of nonholonomic constraint and unknown depth parameters. Most of previous works require the measurement of the desired velocity information to facilitate the controller design, leading to tedious offline computation. In this paper, to eliminate this requirement, the desired velocities are estimated in real-time based on a reduced order observer. Moreover, an augmented update law is designed to compensate for the unknown depth parameters and identify the inverse depth constant. The Lyapunov-based method is employed to prove that the proposed controller achieves asymptotic tracking, and the inverse depth estimate converges to its actual value provided that a persistent excitation condition is satisfied. Subsequently, a robust data-driven algorithm is introduced to ensure the convergence of the inverse depth estimate under a relaxed finite excitation condition. Simulation and experimental results are provided to demonstrate the effectiveness of the proposed approach.

This paper presents a 2-point algorithm for relative pose estimation of a monocular camera mounted on a ground vehicle. Specifically, the geometric model is developed by combining two-view geometry with planar motion constraint. Based on the proposed model, the relative camera pose information can be estimated efficiently with at least 2 feature correspondences. Besides, the proposed algorithm can recover rotation independently with translation so that pure rotation can be computed stably under planar motion. Both experiments on synthetic data and in real world are presented. In the synthetic data experiments, the proposed algorithm is compared with other well-known algorithms to illustrate its high accuracy and robustness to image noise. In addition, the great robustness of the proposed algorithm to nonplanar motion is demonstrated as well. Furthermore, experiment are carried out based on KITTI Vision Benchmark, and the result indicates that the proposed algorithm has better performance than 5-point algorithm for monocular visual odometry and has a wide range of applicability.

In this paper, a trifocal tensor-based approach is proposed for the visual trajectory tracking task of a nonholonomic mobile robot equipped with a roughly installed monocular camera. The desired trajectory is expressed by a set of prerecorded images, and the robot is regulated to track the desired trajectory using visual feedback. Trifocal tensor is exploited to obtain the orientation and scaled position information used in the control system, and it works for general scenes owing to the generality of trifocal tensor. In the previous works, the start, current, and final images are required to share enough visual information to estimate the trifocal tensor. However, this requirement can be easily violated for perspective cameras with limited field of view. In this paper, key frame strategy is proposed to loosen this requirement, extending the workspace of the visual servo system. Considering the unknown depth and extrinsic parameters (installing position of the camera), an adaptive controller is developed based on Lyapunov methods. The proposed control strategy works for almost all practical circumstances, including both trajectory tracking and pose regulation tasks. Simulations are made based on the virtual experimentation platform (V-REP) to evaluate the effectiveness of the proposed approach.

This paper presents a trifocal tensor based approach for the visual trajectory tracking task of a nonholonomic mobile robot, which is equipped with a roughly installed monocular camera. A set of pre-recorded images are used to express the desired trajectory, and the robot is regulated to track the desired trajectory using visual feedback. Trifocal tensor is exploited to obtain the orientation and scaled position information, which is used in the control system. Conventional methods require that there exist enough corresponding feature points in the start, current and final images, while this requirement can be easily violated in large workspace, especially for perspective cameras with limited field of view. In this paper, key frame strategy is proposed to loosen this requirement, extending the workspace of the system. Considering the unknown depth and extrinsic parameters, an adaptive controller is developed based on Lyapunov methods. Simulation results are provided to show the effectiveness of the proposed approach.