Compared to conventional decomposition methods that use ellipses or polygons to represent free space, starshaped representation can better capture the natural distribution of sensor data, thereby exploiting a larger portion of traversable space. This paper introduces a novel motion planning and control framework for navigating robots in unknown and cluttered environments using a dynamically constructed starshaped roadmap. Our approach generates a starshaped representation of the surrounding free space from real-time sensor data using piece-wise polynomials. Additionally, an incremental roadmap maintaining the connectivity information is constructed, and a searching algorithm efficiently selects short-term goals on this roadmap. Importantly, this framework addresses dead-end situations with a graph updating mechanism. To ensure safe and efficient movement within the starshaped roadmap, we propose a reactive controller based on Dynamic System Modulation (DSM). This controller facilitates smooth motion within starshaped regions and their intersections, avoiding conservative and short-sighted behaviors and allowing the system to handle intricate obstacle configurations in unknown and cluttered environments. Comprehensive evaluations in both simulations and real-world experiments show that the proposed method achieves higher success rates and reduced travel times compared to other methods. It effectively manages intricate obstacle configurations, avoiding conservative and myopic behaviors. The source code will be released on website11Available at: github.com/kkkkkaiai/starshaped_roadmap.

This paper introduces RoboDexVLM, an innovative framework for robot task planning and grasp detection tailored for a collaborative manipulator equipped with a dexterous hand. Previous methods focus on simplified and limited manipulation tasks, which often neglect the complexities associated with grasping a diverse array of objects in a long-horizon manner. In contrast, our proposed framework utilizes a dexterous hand capable of grasping objects of varying shapes and sizes while executing tasks based on natural language commands. The proposed approach has the following core components: First, a robust task planner with a task-level recovery mechanism that leverages vision-language models (VLMs) is designed, which enables the system to interpret and execute open-vocabulary commands for long sequence tasks. Second, a language-guided dexterous grasp perception algorithm is presented based on robot kinematics and formal methods, tailored for zero-shot dexterous manipulation with diverse objects and commands. Comprehensive experimental results validate the effectiveness, adaptability, and robustness of RoboDexVLM in handling long-horizon scenarios and performing dexterous grasping. These results highlight the framework’s ability to operate in complex environments, showcasing its potential for open-vocabulary dexterous manipulation. Our open-source project page can be found at https://henryhcliu.github.io/robodexvlm.

To address the intricate challenges of cooperative scheduling and motion planning in Autonomous Mobility-on-Demand (AMoD) systems, this paper introduces LMMCoDrive, a novel cooperative driving framework that leverages a Large Multimodal Model (LMM) to improve traffic efficiency and passenger experience in dynamic urban environments. This framework seamlessly integrates scheduling and motion planning processes to ensure the effective operation of Cooperative Autonomous Vehicles (CAVs). The spatial relationship between CAVs and passenger requests is abstracted into a Bird’s-Eye View (BEV) image to fully exploit the potential of the multimodal understanding ability of LMMs. Besides, trajectories are cautiously refined for each CAV while ensuring collision avoidance through safety constraints. A decentralized optimization strategy, facilitated by the Alternating Direction Method of Multipliers (ADMM) within the LMM framework, is proposed to drive the graph evolution of CAVs. Simulation results in diverse urban scenarios demonstrate the pivotal role and significant impact of LMM in optimizing CAV scheduling and seamlessly serving a decentralized cooperative optimization process for each CAV. This marks a substantial stride towards practical, efficient, and safe AMoD systems that are poised to revolutionize urban transportation. The code is available at https://github.com/henryhcliu/LMMCoDrive.

Current autonomous driving systems often struggle to balance decision-making and motion control while ensuring safety and traffic rule compliance, especially in complex urban environments. Existing methods may fall short due to separate handling of these functionalities, leading to inefficiencies and safety compromises. To address these challenges, we introduce UDMC, an interpretable and unified Level 4 autonomous driving framework. UDMC integrates decision-making and motion control into a single optimal control problem (OCP), considering the dynamic interactions with surrounding vehicles, pedestrians, road lanes, and traffic signals. By employing innovative potential functions to model traffic participants and regulations, and incorporating a specialized motion prediction module, our framework enhances on-road safety and rule adherence. The integrated design allows for real-time execution of flexible maneuvers suited to diverse driving scenarios. High-fidelity simulations conducted in CARLA exemplify the framework’s computational efficiency, robustness, and safety, resulting in superior driving performance when compared against various baseline models. Our open-source project is available athttps://github.com/henryhcliu/udmc_carla.git.

Ensuring safe driving and real-time execution is a crucial requirement in the motion planning process for autonomous vehicles. Hence, there is a compelling demand for advanced motion planning algorithms that exhibit effective management of inequality constraints and exceptional computational performance. This paper investigates a diffeomorphism-transformed iterative linear quadratic regulator (DTiLQR) algorithm for addressing constrained motion planning problems in autonomous vehicles with nonlinear dynamics and multiple inequality constraints. With regard to the state and input constraints, a novel state-and-input diffeomorphism is proposed to transform the constrained state/input space into an unconstrained one. Subsequently, these inequality constraints are systematically incorporated into the vehicle dynamics, thereby leading to the newly constructed system in this context. Then, we reformulate and incorporate the obstacle avoidance constraint into the objective function using state diffeomorphism and logarithmic barrier function. With this, the original optimization problem is converted to the unconstrained counterpart, adhering only to the constructed system dynamics. In this sense, featuring a streamlined single-loop architecture (which is essentially different from the dual-loop algorithmic design of existing constrained iLQR algorithms), DTiLQR is used to solve the optimization problem effectively while maintaining motion performance and constraint satisfaction for the resulting optimal trajectory. Ultimately, case studies across various driving situations showcase the effectiveness and exceptional computational efficiency of the proposed DTiLQR algorithm.

The development of connected autonomous vehicles (CAVs) facilitates the enhancement of traffic efficiency in complicated scenarios. Difficulties remain unsolved in developing an effective and efficient coordination strategy for CAVs. In this paper, we formulate the cooperative autonomous driving task of CAVs as an optimal control problem with safety conditions enforced as hard constraints, and propose a computationally-efficient parallel optimization framework to generate strategies for CAVs with the travel efficiency improved and the hard safety constraints satisfied. Specifically, all constraints involved are addressed appropriately with convex approximation, such that the convexity property of the reformulated optimization problem is exhibited. Then, a parallel optimization algorithm is presented to solve the reformulated optimization problem, with an embodied iterative nearest neighbor search strategy to determine the optimal passing sequence. It is noteworthy that the travel efficiency is enhanced and the computation burden is considerably alleviated with the proposed innovation development. We also examine the proposed method in CARLA simulator and perform thorough comparisons to demonstrate the effectiveness and efficiency of the proposed approach.

In the context of spatial-temporal autonomous driving, the accurate and real-time trajectory prediction of the surrounding vehicle (SV) is crucial. This paper aims to design an efficient, accurate, and interpretable unimodal trajectory prediction approach. To achieve this objective, we employ Sparse Gaussian Process Regression (SGPR), which enables large dataset learning and efficient inference of future trajectories. This approach ensures accurate predictions while maintaining high computational efficiency. To further enhance the robustness of the prediction module, we propose the translation and rotation transformation strategy, which effectively simplifies the prediction problem. Additionally, we utilize an instant evaluation algorithm to assess the prediction performance and maintain a streaming dataset for incremental learning, capable of adapting to dynamic driving environments. In our experimental evaluation, we compare our proposed trajectory prediction approach with a series of existing methods. The results demonstrate that our work achieves superior prediction accuracy while requiring less inference time. It is noteworthy that, the proposed SGPR-based trajectory prediction approach with rotation equivalence is able to swiftly infer and incrementally learn from dynamic environments, which makes it a promising tool for enhancing safety and efficiency in autonomous driving systems.