The application of object detection in industrial transportation has witnessed substantial advancements, yielding significant enhancements in both safety and efficiency. While Transformer-based detectors have demonstrated remarkable success in object detection for autonomous driving, their quadratic computational complexity and limited small-object perception capabilities remain significant challenges. To address these limitations, we propose lightweight visual Mamba small object detection (LVMSOD) method, a novel selective state space model designed for small object detection task. The proposed framework employs a multi-level cascade of dual-layer nested Mamba modules to comprehensively capture contextual information across different scales, coupled with a hierarchical feature fusion strategy to enhance multi-scale feature integration for improved small object detection. The LVMSOD framework incorporates two key components to enhance feature representation: Visual Mamba Detection (VDM) block that preserves fine-grained image details, and Lightweight Gated Multi-Layer Perceptron (LGMLP) designed to model local feature dependencies efficiently. Furthermore, we propose an optimized feature extraction mechanism employing depthwise and pointwise convolutions with distribution factors, significantly reducing computational overhead while maintaining detection accuracy. Comprehensive evaluations on benchmark datasets demonstrate the effectiveness of our approach, achieving [email protected] scores of 93.3% on KITTI and 45.3% on VisDrone while maintaining superior computational efficiency. These results highlight the potential of LVMSOD for efficient and accurate small object detection.

Autonomous on-ramp merging requires a safe and efficient decision and planning framework to navigate dynamic and complex traffic scenarios. Reinforcement Learning (RL) offers adaptability in such environments but struggles to ensure safety in unseen situations, whereas Model Predictive Control (MPC) can enforce safety constraints but relies on accurate models. To leverage the strengths of both, we propose a hierarchical Safety-Augmented RL and MPC integration approach, i.e., SARMI, for decision and planning in the on-ramp merging scenario. The high-level decision-making layer employs a discrete Soft Actor-Critic (SAC-Discrete) algorithm enhanced with the Augmented Lagrangian method to generate actions. MPC generates reference trajectories based on RL actions and feeds predicted states back to RL for reward and cost design, ensuring consistency between RL and MPC. Our key innovations include: i) incorporating a Gaussian-based risk field model into the cost design of constrained RL, which quantifies collision risk based on MPC-predicted states, enabling agents to make proactive decisions; ii) an Augmented Lagrangian SAC-Discrete method with barrier-like quadratic penalties to promote compliance with safety constraints and alleviate oscillations in dual gradient descent; iii) theoretical analysis proving the equivalence between the optimal solutions of the primal and dual problems in Augmented Lagrangian SAC-Discrete and iv) a dual safety mechanism combining action masking (to filter invalid actions) and action shielding (to replace unsafe actions in RL), enhancing safety during the exploration and execution stages, respectively. Experiments demonstrate the superiority of our method over the baseline SAC-Discrete, showing improved safety and efficiency.

Four-wheel independent steering (4WIS) vehicles have attracted increasing attention for their superior maneuverability. Human drivers typically choose to cross or drive over low-profile obstacles (e.g., plastic bags) to efficiently navigate through narrow spaces, while existing planners neglect obstacle attributes, leading to suboptimal efficiency or planning failures. To address this issue, we propose a novel multimodal trajectory planning framework that employs a neural network for scene perception, integrates 4WIS hybrid A* search to generate a warm start, and formulates an optimal control problem (OCP) for trajectory optimization. Specifically, a multimodal perception network fusing visual information and vehicle states is employed to capture semantic and contextual scene information, enabling the planner to adapt the strategy according to scene complexity (hard or easy planning task). For hard tasks, guided points are introduced to decompose complex tasks into local subtasks, improving search efficiency. The multiple steering modes of 4WIS vehicles—Ackermann, diagonal, and zero-turn—are also incorporated as kinematically feasible motion primitives. Moreover, a hierarchical obstacle handling strategy, which categorizes obstacles as “non-traversable”, “crossable”, and “drive-over”, is incorporated into the node expansion process, explicitly linking obstacle attributes to planning actions to enable efficient decisionmaking. Furthermore, to address dynamic obstacles with motion uncertainty, we introduce a probabilistic risk field model, constructing risk-aware driving corridors that serve as linear collision constraints in the OCP. Experimental results demonstrate the proposed framework’s effectiveness in generating safe, efficient, and smooth trajectories for 4WIS vehicles, especially in constrained environments.

Switching communication topologies can cause instability in vehicle platoons, as vehicle information may be lost during the dynamic switching process. This highlights the need to design a controller capable of maintaining the stability of vehicle platoons under dynamically changing topologies. However, capturing the dynamic characteristics of switching topologies and obtaining complete vehicle information for controller design while ensuring stability remains a significant challenge. In this study, we propose an observer-based distributed model predictive control (DMPC) method for vehicle platoons under directed Markovian switching topologies. Considering the stochastic nature of the switching topologies, we model the directed switching communication topologies using a continuous-time Markov chain. To obtain the leader vehicle's information for controller design, we develop a fully distributed adaptive observer that can quickly adapt to the randomly switching topologies, ensuring that the observed information is not affected by the dynamic topology switches. Additionally, a sufficient condition is derived to guarantee the mean-square stability of the observer. Furthermore, we construct the DMPC terminal update law based on the observer and formulate a string stability constraint based on the observed information. Numerical simulations demonstrate that our method can reduce tracking errors while ensuring string stability.

It has been challenging to model the complex temporal-spatial dependencies between agents for trajectory prediction. As each state of an agent is closely related to the states of adjacent time steps, capturing the local temporal dependency is beneficial for prediction, while most studies often overlook it. Besides, learning the high-order motion state attributes is expected to enhance spatial interaction modeling, but it is rarely seen in previous works. To address this, we propose a lightweight framework, i.e., LTMSformer, to extract temporal-spatial interaction features for multi-modal trajectory prediction. Specifically, we introduce a Local Trend-Aware Attention mechanism to capture the local temporal dependency by leveraging a convolutional attention mechanism with hierarchical local time boxes. Next, to model the spatial interaction dependency, we build a Motion State Encoder to incorporate high-order motion state attributes, such as acceleration, jerk, heading, etc. To further refine the trajectory prediction, we propose a Lightweight Proposal Refinement Module that leverages Multi-Layer Perceptrons for trajectory embedding and generates the refined trajectories with fewer model parameters. Experiment results on the Argoverse 1 dataset demonstrate that our method outperforms the baseline HiVT-64, reducing the minADE by approximately 4.35%, the minFDE by 8.74%, and the MR by 20%. We also achieve higher accuracy than HiVT-128 with a 68% reduction in model size.

This paper proposes a novel sub-polytope integrated approach (sPIA) that features an intrinsic bump-free transition, aiming to reduce conservatism in the design of path tracking control for autonomous vehicles with large range time-varying longitudinal velocity. The approach encapsulates the interdependent time-varying parameters associated with longitudinal velocity as a set of finite-vertex sub-polytopes interconnected via junction points, thereby reducing the conservatism induced by modeling overbounding. The integration of junction points and the formulation of sub-region activation rules provide a theoretical foundation for avoiding abrupt changes in feedback gains, ensuring a bump-free transition between sub-regions. A gain-scheduling state feedback controller is designed, employing parameter-dependent Lyapunov functions to further attenuate design conservatism. The effectiveness of the proposed method in reducing design conservatism is demonstrated by a comparative analysis of the optimal H∞performance indices across various sub-polytope integration schemes. Furthermore, the superiority of the method is exemplified via simulations within real-world driving scenarios, utilizing the high-fidelity CarSim-Simulink platform. The results indicate that the proposed sPIA outperforms traditional polytopic methods in path tracking performance. This improvement, together with the effective avoidance of bumps during sub-regional transitions, confirms the efficacy of the proposed approach. Moreover, the real-time performance of the method is verified by hardware-in-the-loop experiments.

Accurate localization and mapping play a critical role in intelligent vehicles for advanced driver assistance systems and autonomous driving. For the rapidly developing field of autonomy, autonomous systems require an accurate LiDAR SLAM system to adapt to different specifications of LiDARs in various complex scenarios. Therefore, this paper proposes a highly accurate and universal LiDAR SLAM method, which is innovative in the design of LiDAR odometry (LO) and loop detection descriptors. The odometry is designed in two key ways. Firstly, this paper focuses on the 3D spatial information of the point cloud and unifies its representation through voxel gridding, enhancing adaptability to different LiDARs and reducing computation time for feature extraction. Secondly, leveraging the “near dense, far sparse” characteristic of LiDAR, a dynamic registration function is designed to improve the stability and accuracy of the odometry. In addressing loop detection, we have designed a novel dual-channel correlation global descriptor, integrating point cloud height and height difference characteristics. This innovative descriptor reduces the sensitivity to translation and improves the precision and recall of loop detection. We have thoroughly tested our method using various public datasets with different LiDARs in a variety of scenarios and compared it with other well-known methods. Experimental results show that our SLAM technology improves stability and accuracy while maintaining universal compatibility. On the KITTI benchmark, our method ranks among the highest in terms of accuracy and real-time performance, achieving a relative average translation error (RATE) of 0.59 and an average time of 40 ms per scan.

Scene text editing aims to replace the source text with the target text while preserving the original background. Its practical applications span various domains, such as data generation and privacy protection, highlighting its increasing importance in recent years. In this study, we propose a novel Scene Text Editing network with Explicitly-decoupled text transfer and Minimized background reconstruction, called STEEM. Unlike existing methods that usually fuse text style, text content, and background, our approach focuses on decoupling text style and content from the background and utilizes the minimized background reconstruction to reduce the impact of text replacement on the background. Specifically, the text-background separation module predicts the text mask of the scene text image, separating the source text from the background. Subsequently, the style-guided text transfer decoding module transfers the geometric and stylistic attributes of the source text to the content text, resulting in the target text. Next, the background and target text are combined to determine the minimal reconstruction area. Finally, the context-focused background reconstruction module is applied to the reconstruction area, producing the editing result. Furthermore, to ensure stable joint optimization of the four modules, a task-adaptive training optimization strategy has been devised. Experimental evaluations conducted on two popular datasets demonstrate the effectiveness of our approach. STEEM outperforms state-of-the-art methods, as evidenced by a reduction in the FID index from 29.48 to 24.67 and an increase in text recognition accuracy from 76.8% to 78.8%.

The tire-road friction coefficient (TRFC) is the critical parameter that significantly improves the control performance of distributed electric vehicles. Nonetheless, achieving precise TRFC estimation during straight stationary driving maneuvers, characterized by constant longitudinal speed (e.g., where the longitudinal acceleration is nearly zero) on a straight road, poses a particularly formidable challenge. In the paper, we propose a new learning strategy that leverages multi-domain fusion feature extraction in both the time domain and time-frequency domain to estimate the TRFC during straight stationary driving maneuvers. Specifically, the frequency response function of the in-wheel-motor-drive system first is inferred from the longitudinal dynamics model and single wheel dynamics model. Then, the input selection of learning strategy is determined through frequency response characteristics analysis and explainable dynamics mechanism. In addition, a parallel spatial-temporal convolutional neural network (PSTCNN) is built to extract features in both the time domain and in the time-frequency domain, respectively. Finally, the TRFC learning strategy is verified by experimental tests on different road surfaces. Our results demonstrate that the proposed methodology is capable of estimating the TRFC with a lower error than the traditional learning-based method and the classical slip-slope method.

In the foreseeable Intelligent Transportation System, Intelligent Connected Vehicles (ICVs) will play an important role in improving travel efficiency and safety. However, it is challenging for ICVs to support the resource-hungry autonomous driving applications due to the limitation of hardware computing power. Fortunately, the emergence of Multi-access Edge Computing helps overcome this limitation effectively. This paper addresses the vehicle-to-edge server computation offloading conundrum by optimizing the trade-offs in partial offloading and resource allocation. Proposing a distributed approach, this study confronts the multi-variable non-convex challenge directly by decoupling variables and deriving constraint-based bounds that guide the decisions for offloading and allocation. A novel low-complexity distributed algorithm is introduced that not only tends toward optimal but also demonstrates superior real-time applicability and efficiency, illustrated through enhanced performances both in simulated trials and genuine vehicular edge computing settings. The algorithm’s practical effectiveness addresses a notable gap between the theoretical models for computation offloading and actual real-life execution, reinforcing the soundness and relevance of the proposed method. Furthermore, its advanced integration with federated learning frameworks marks a leading-edge application, substantiating significant enhancements in computational efficiency and robustness.

Well-trained visual object detectors are generally confronted with a severe performance decline when deployed in a novel driving scenario due to the impact of domain shift. Despite excellent improvements in unsupervised domain adaptive object detection achieved by adversarial training, those approaches fail to capture the transfer core underlying the holistic scenes. To solve this problem, we propose a progressive critical region transfer framework for cross-domain visual object detection. Specifically, we exploit a potential foreground mining (PFM) module and a semantic-specific RoI aggregation (SRA) module to improve the robustness of the cross-domain detection framework. Upon the critical regions in the broad sense, the PFM module first highlights the foreground regions by reweighting the hierarchical feature maps in sequence, and then modifies location biases at the downstream position of the backbone network for more accurate upstream predictions. Deep into the critical regions in the narrow sense, the SRA module concentrates on establishing an appropriate matching between batch-wise RoIs and all semantic centers, and further strengthens the aggregation of cross-domain identical semantic with the complement of context references. Together these modules are obligated to transform the adaptation importance from the whole scope to the latent foreground areas, and afterward to the informative regions of interest along the detection pipeline. Experiments show that our progressive critical region transfer framework achieves a state-of-the-art performance in adverse weather, camera configuration, and complicated scene adaptation, which outperforms the baselines by 19.4%, 5.0%, and 6.1%, respectively.

Multi-modal fusion can take advantage of the LiDAR and camera to boost the robustness and performance of 3D object detection. However, there are still of great challenges to comprehensively exploit image information and perform accurate diverse feature interaction fusion. In this paper, we proposed a novel multi-modal framework, namely Point-Pixel Fusion for Multi-Modal 3D Object Detection (PPF-Det). The PPF-Det consists of three submodules, Multi Pixel Perception (MPP), Shared Combined Point Feature Encoder (SCPFE), and Point-Voxel-Wise Triple Attention Fusion (PVW-TAF) to address the above problems. Firstly, MPP can make full use of image semantic information to mitigate the problem of resolution mismatch between point cloud and image. In addition, we proposed SCPFE to preliminary extract point cloud features and point-pixel features simultaneously reducing time-consuming on 3D space. Lastly, we proposed a fine alignment fusion strategy PVW-TAF to generate multi-level voxel-fused features based on attention mechanism. Extensive experiments on KITTI benchmarks, conducted on September 24, 2023, demonstrate that our method shows excellent performance.

This work deals with the Multi-Vehicle Trajectory Planning (MVTP) problem in unstructured conflict areas. Compared with MVTP on structured roads, the more complex intersection and the more conflicting motions of vehicles, set higher requirements for the real-time of the planner. To address the issue, this work proposes a centralized decision-making distributed planning framework, to generate trajectories for multiple vehicles navigating conflict areas on unstructured roads. In the multi-vehicle collaboration process, vehicles are assigned a priority upon entering the control area, with higher-priority vehicles treated as dynamic obstacles to be avoided. The distributed planning is divided into three stages: 1) trajectory search utilizing Velocity-Hybrid A* (V-Hybrid A*), 2) path optimization with respect to both dynamic and static obstacles, and 3) speed optimization leveraging the convex space created by the initial solution. In the trajectory search of individual vehicles, each expansion node of V-Hybrid A* is given an acceleration to get a coarse trajectory with discrete velocity. Moreover, a box constraint is established to optimize the path of the coarse trajectory. Then, according to the homotopy class provided by the coarse trajectory, speed optimization is carried out to obtain the final single vehicle trajectory. In our simulation experiments, we randomized the generation of numerous vehicles assigned with varied tasks to evaluate the algorithm’s effectiveness. Ultimately, real-vehicle experiments substantiated the algorithm’s practical feasibility. The results indicate that the proposed method can significantly improve the traffic efficiency in conflict areas and has practical applicability.

In this article, an adaptive dynamic programming (ADP) scheme utilizing a costate function is proposed for optimal control of unknown discrete-time nonlinear systems. The state-action data are obtained by interacting with the environment under the iterative scheme without any model information. In contrast with the traditional ADP scheme, the collected data in the proposed algorithm are generated with different policies, which improves data utilization in the learning process. In order to approximate the cost function more accurately and to achieve a better policy improvement direction in the case of insufficient data, a separate costate network is introduced to approximate the costate function under the actor-critic framework, and the costate is utilized as supplement information to estimate the cost function more precisely. Furthermore, convergence properties of the proposed algorithm are analyzed to demonstrate that the costate function plays a positive role in the convergence process of the cost function based on the alternate iteration mode of the costate function and cost function under a mild assumption. The uniformly ultimately bounded (UUB) property of all the variables is proven by using the Lyapunov approach. Finally, two numerical examples are presented to demonstrate the effectiveness and computation efficiency of the proposed method.

Cooperation among multiple connected vehicles (CVs) brings swarm intelligence to our transportation systems and helps improve their performance. This study proposes a fuel economy optimization approach for a platooning vehicle swarm via distributed economic model predictive control (DEMPC). Within the DEMPC framework, each CV shares its assumed trajectory in the predictive horizon with its neighboring CVs at each control loop. With its neighbors’ and its own assumed trajectories, each CV first solves an open-loop control optimization problem for platoon formation, and then solves an open-loop economic optimization problem for direct fuel economy improvement. In particular, the optimal cost of the former optimization problem is used in the latter one to build an upperbound constraint for stability guarantee. The asymptotic convergence of assumed terminal states is given in the analysis, and the recursive feasibility of the two optimization problems are proved with an explicit constraint on the weight matrices in the open-loop control optimization problem. Based on these analyses, the asymptotic stability of the closed-loop system is finally proved through Lyapunov analysis. Numerical simulation results validate the effectiveness of the proposed approach in terms of closed-loop stability and fuel economy improvement. Note to Practitioners—This work aims to optimize the fuel economy for a swarm of vehicles running in a platoon. Existing approaches on platoon control mainly focus on the accurate tracking of the following vehicles, but this may cause aggressive control and hence lead to poor fuel economy. Therefore, this paper proposes a distributed economic model predictive control approach to explicitly optimize the fuel consumption rate of each vehicle. The proposed method can improve fuel economy and reduce communication requirements and burdens for platooning vehicle swarms. The paper assumes no communication time delay and packet drops, so the impact of unreliable communication will be studied in the future work.

Vehicle platoon control based on vehicle-to-vehicle (V2V) communication is one of the promising technologies to improve the performance of transportation systems. This paper presents a distributed controller to optimize a vehicle platoon’s fuel consumption by combining the switching feedback control and economic model predictive control (EMPC) methods. The closed-loop dynamics involving switching feedback gains with the constant time headway (CTH) policy are established firstly, and the multiple-predecessor following (MPF) communication topology is considered. In order to obtain the economy optimal feedback gain, we design a local optimal control problem for each vehicle, based on which the average dwell time is defined and the distributed EMPC algorithm is designed. Based on linear matrix inequalities (LMIs) and the Lyapunov theorem, the feedback gain selection method and the lower bound for average dwell time that guarantees asymptotic stability are analyzed rigorously. Then a modified algorithm that can ensure string stability is designed. Numerical simulations show a maximum of 6.84% fuel benefit compared with pure tracking-oriented methods.

Lane keeping is a key task in driving, and it plays an important role in staying safe while driving. However, conventional lane-keeping assistance systems (LKASs) have a limited level of automation, while fully autonomous lane-keeping systems have shortcomings regarding reliability. To address these issues, an advanced LKAS with two switchable assistance modes, namely, the lane departure prevention mode and the lane-keeping co-pilot mode, is proposed in this paper. First, the system structure is constructed. Then, the functions and control strategies for the two assistance modes are defined. Next, the controller algorithms are designed using the learning-based model predictive control (LBMPC) method to compensate for modeling errors. Within the framework of LBMPC, an oracle is built to learn the unmodeled dynamics using an extended Kalman filter. Moreover, optimization problems are formulated to achieve the control objectives regarding driving safety and driver acceptance. Finally, driver-in-the-loop experiments carried out on a driving simulator prove that both assistance modes are effective. Furthermore, the lane-keeping co-pilot mode can help reduce the driving burden in the sense of avoiding frequent steering correction.