Considering the potential of using multi-frame information to solve the occlusion problem, we introduce a novel idea of multi-frame information integration, which uses the attention mechanism to fuse the temporal information from the previous frame. The idea can effectively improve the estimation accuracy in occluded regions and optimize the inference speed under multi-frame settings. Meanwhile, we suggest the concept of attention confidence to provide an explicit value criterion for the model to utilize useful attention information more efficiently. Furthermore, we propose an Efficient Temporal Attention network (ETA), which achieves promising results on Sintel and KITTI benchmarks, especially with a 9.4% error reduction compared to the baseline method GMA on Sintel (test) Clean.

Tracking control, a traditional subject in the industrial domain, has garnered extensive research over the past several decades and has been broadly applied in many different contexts, encompassing vehicle path tracking tasks and robotic control. In this study, a novel multi-objective suboptimal control strategy for discrete-time nonlinear tracking control systems is presented, integrating the robust fuzzy model predictive control (RFMPC) approach with an adaptive dynamic programming (ADP) method. First, the original system model is reformulated into a T-S fuzzy system, and an infinite-horizon RFMPC algorithm is developed to guarantee input-to-state stability (ISS) of the system. Subsequently, the policy iteration technique is utilized to perform multi-objective optimization for the RFMPC in the Pareto sense, resulting in the determination of a Pareto optimal tracking controller. The efficacy and benefits of the proposed algorithm are validated through a series of comparative simulations.

This study develops a quantized control framework for nonlinear systems with state delays, external disturbances, and bounded parametric uncertainties. First, a static logarithmic quantizer is integrated into the controller architecture, where its participation form within the actual computations is determined through theoretical analysis based on the sector bound approach. The nonlinear dynamics are subsequently decomposed into a group of linear subsystems via the Takagi-Sugeno (T-S) fuzzy modeling method. By formulating a Lyapunov-Krasovskii functional (LKF), the input-to-state stability (ISS) is demonstrated, leading to the development of a robust fuzzy model predictive control (RFMPC) algorithm supported by comprehensive proofs. Numerical simulations corroborate the assertions, illustrating the efficacy of the proposed strategy.

Considering that scene flow estimation has the capability of the spatial domain to focus but lacks the coherence of the temporal domain, this study proposes long-term scene flow estimation (LSFE), a comprehensive task that can simultaneously capture the fine-grained and long-term 3D motion in an online manner. We introduce SceneTracker, the first LSFE network that adopts an iterative approach to approximate the optimal 3D trajectory. The network dynamically and simultaneously indexes and constructs appearance correlation and depth residual features. Transformers are then employed to explore and utilize long-range connections within and between trajectories. With detailed experiments, SceneTracker shows superior capabilities in addressing 3D spatial occlusion and depth noise interference, highly tailored to the needs of the LSFE task. We build a real-world evaluation dataset, LSFDriving, for the LSFE field and use it in experiments to further demonstrate the advantage of SceneTracker in generalization abilities.

Generative models are gaining momentum in both academic and industrial applications driven by the availability of large-scale datasets, especially in tasks involving Image-to-Image Translation. Meanwhile, poor human perception of nighttime environment has led to a demand for translation from night-vision infrared to day-vision RGB images. However, collecting such cross-modal training data at the same time is impossible due to the thermal imaging properties of infrared cameras, the challenge lies in constructing image pairs during the day and at night respectively, where the requirement for data alignment poses significant difficulties. In this paper, we propose a Pixel-level aligned Image Pair generation framework PIP to explore efficient colorization of high-resolution infrared images. Specifically, we first construct a 3D high-precision point cloud map for the purpose of establishing the correlation between day and night scenes. Corresponding point clouds of modal images are collected simultaneously during data acquisition to obtain image sensor poses by Global Matching with the map, which allows us to calculate the transformation relationship from infrared to RGB image coordinate systems based on the sensor parameters and depth information of the map. Leveraging the relationship, the pixel values of RGB image is projected onto the infrared image followed by optimization as the colored image. Accordingly, we present a dataset NUDT-PIP, the first of its kind containing large-scale pixel-level aligned cross-time infrared-RGB image pairs of complicated real road scenes. Experimental results demonstrate the reliability and strong applicability of our dataset in Image-to-Image Translation. Our code will be released at https://github.com/wjjjjyourFA/NUDT-PIP.

In this paper, we propose a multi-environment, multi-LiDAR, multi-task dataset to promote the grid map-based navigation capability for autonomous vehicles. The dataset comprises structured and unstructured environmental data captured by different types of LiDAR and contains various challenging scenarios, including moving objects, negative obstacles, steep slopes, cliffs, overhangs, etc. Further, we have devised an innovative method for generating ground truth, facilitating the creation of dense, accurate, and stable grid maps with a minimal requirement for human annotation efforts. A new baseline method and two existing approaches are evaluated on this dataset. Results indicate that existing approaches perform much worse than the proposed baseline. The dataset will be made publicly available at https://github.com/guanglei96/M3-GMN.

Autonomous decision making is a critical and difficult task for intelligent vehicles in dynamic transportation environments. In this paper, a reinforcement learning approach with value function approximation and feature learning is proposed for autonomous decision making of intelligent vehicles on highways. In the proposed approach, the sequential decision making problem for lane changing and overtaking is modeled as a Markov decision process with multiple goals, including safety, speediness, smoothness, etc. In order to learn optimized policies for autonomous decision-making, a multiobjective approximate policy iteration (MO-API) algorithm is presented. The features for value function approximation are learned in a data-driven way, where sparse kernel-based features or manifold-based features can be constructed based on data samples. Compared with previous RL algorithms such as multiobjective Q-learning, the MO-API approach uses data-driven feature representation for value and policy approximation so that better learning efficiency can be achieved. A highway simulation environment using a 14 degree-of-freedom vehicle dynamics model was established to generate training data and test the performance of different decision-making methods for intelligent vehicles on highways. The results illustrate the advantages of the proposed MO-API method under different traffic conditions. Furthermore, we also tested the learned decision policy on a real autonomous vehicle to implement overtaking decision and control under normal traffic on highways. The experimental results also demonstrate the effectiveness of the proposed method.

This paper presents a parameterized batch reinforcement learning algorithm for near-optimal longitudinal control of autonomous land vehicles (ALVs). The proposed approach uses an actor-critic architecture, where parameterized feature vectors based on kernels are learned from collected samples for approximating the value functions and policies. One difference between the parameterized batch actor-critic (PBAC) algorithm and previous actor-critic learning approaches is that the critic and actor in PBAC share the same linear features, which has been theoretically proved to be a beneficial property for the convergence of actor-critic learning approaches. In order to obtain better learning efficiency, least-squares-based batch updating rules are designed for the critic and actor, respectively. Based on the PBAC learning algorithm, a data-driven longitudinal control method is presented for ALVs to obtain near-optimal control policies which adaptively tune the fuel/brake control signals to track different speeds. A multiobjective reward function is designed so that both tracking precision and driving smoothness are considered. Extensive experiments were conducted on a real ALV platform while driving on flat, slippery, sloping, and bumpy roads. The experimental results illustrate the superiority of the PBAC-based self-learning controller over conventional longitudinal control methods such as proportional-integral (PI) control and learning-based PI control.

The accurate information of vehicle states that could not be obtained directly by onboard sensors is virtual important for vehicle active safety systems. This paper presents the nonlinear full-order observer which is used to estimate the longitudinal velocity, lateral velocity and road grade estimation with autonomous driving. Firstly, this paper established a simplified vehicle dynamics model for the Hongqi HQ430 which could characterize the performance of autonomous driving vehicle on highway, and the estimator was designed. Secondly, in order to verify the effectiveness of the proposed nonlinear full-order observer, we do some simulation experiments, simulation experiments were carried out under different running conditions, the simulation results showed that the estimation method have certain validity and accuracy.

In the last decade, autonomous driving technology has become an important research topic due to its potential economic and social benefits. There has been considerable research activities contributed to make the autonomous driving system adapt to complex environments. Motion control is vital to the overall autonomous driving system, especially when the autonomous vehicle is driving in complex off-road environments. The aim of our work in this paper is to develop a high-precision motion controller for autonomous driving system running on rugged mountain roads and sand roads. Different from most existing methods in which the motion control problem is decoupled into lateral control and longitudinal control. In this work, a coupling controller is designed for solving the motion control problem of autonomous driving system. Experiments in the real-world rugged mountain road and sand road environment have been conducted to demonstrate the high-precision performance and efficiency of the proposed motion controller.

This paper presents a practical trajectory planning framework towards fully autonomous driving in urban environments. Firstly, based on the behavioral decision commands, a reference path is extracted from the digital map using the LIDAR-based localization information. The reference path is refined and interpolated via a nonlinear optimization algorithm and a parametric algorithm, respectively. Secondly, the trajectory planning task is decomposed into spatial path planning and velocity profile planning. A closed-form algorithm is employed to generate a rich set of kinematically-feasible spatial path candidates within the curvilinear coordinate framework. At the same time, the velocity planning algorithm is performed with considering safety and smoothness constraints. The trajectory candidates are evaluated by a carefully developed objective function. Subsequently, the best collision-free and dynamically-feasible trajectory is selected and executed by the trajectory tracking controller. We implemented the proposed trajectory planning strategy on our test autonomous vehicle in the realistic urban traffic scenarios. Experimental results demonstrated its capability and efficiency to handle a variety of driving situations, such as lane keeping, lane changing, vehicle following, and static and dynamic obstacles avoiding, while respecting traffic regulations.

Path tracking issues of autonomous ground vehicles (AGVs) have attracted more attention in recent years with the intelligent and electrified development of vehicles. In order to make AGVs path tracking problem more flexible, regional path tracking problem is discussed in this manuscript based on model predictive control (MPC) method, where the front wheel steering angle is regarded as the control variable. The feasible region for AGVs running is determined first according to the detected road boundaries. In the following, AGVs running in this region is considered using kinematic model. Then, in order to make the actual trajectory of AGVs keep in the region and satisfy the safety requirements, MPC method is employed to design path tracking controller considering the vehicle dynamics, the actuator and state constraints. In order to verify the effectiveness of the proposed algorithm, simulations under various test conditions are carried out using a high fidelity vehicle simulator veDYNA, where the Hongqi vehicle HQ430 parameters are matched. The results obtained from the simulation illustrate that the proposed algorithm obtains good performance in dealing with the regional path tracking problem.

In this paper, a state space sampling-based local trajectory generation framework for autonomous vehicles driving along a reference path is proposed. The presented framework employs a two-step motion planning architecture. In the first step, a Support Vector Machine based approach is developed to refine the reference path through maximizing the lateral distance to boundaries of the constructed corridor while ensuring curvature-continuity. In the second step, a set of terminal states are sampled aligned with the refined reference path. Then, to satisfy system constraints, a model predictive path generation method is utilized to generate multiple path candidates, which connect the current vehicle state with the sampling terminal states. Simultaneously the velocity profiles are assigned to guarantee safe and comfort driving motions. Finally, an optimal trajectory is selected based on a specified objective function via a discrete optimization scheme. The simulation results demonstrate the planner's capability to generate dynamically-feasible trajectories in real time and enable the vehicle to drive safely and smoothly along a rough reference path while avoiding static obstacles.

To address the path tracking problem of autonomous land vehicle, a new vehicle-road model named “Ribbon Model” is constructed under the constraints of road width and vehicle geometry structure. A new vehicle-road evaluation algorithm is developed based on this model, and new path tracking controller is designed. The difficulties of preview distance selection and parameters tuning with speed of pure following controller are avoided in this controller. Performance of the novel method is verified by simulation and vehicle experiments.