Multi-Sensor Fusion (MSF) based perception systems have become the foundation supporting many industrial applications and domains, such as self-driving cars, robotic arms, and unmanned aerial vehicles. With the rapid development of data-driven artificial intelligence (AI), the perception capabilities of MSF have been comprehensively enhanced, especially in understanding complex, dynamic external environments. Similar to traditional software, AI-enabled MSF systems also require rigorous testing. However, existing testing methods are still limited to evaluating the frame-level perception capabilities (e.g., object detection in static scenes) of singlesensor systems (e.g., image-based and point cloud-based systems). Given that many safety-critical intelligent systems, such as selfdriving cars, are operated in dynamic environments where perception systems play an important role, there comes an urgent need to assess their dynamic perception capabilities in understanding and responding to external environmental variations in real-time.To bridge this gap, we design and implement DRIVENCE, an automated metamorphic testing tool for testing the dynamic perception capabilities of MSF-based systems. DRIVENCE accounts for various real-world physical constraints to generate realistic multi-modal test sequences by inserting multiple dynamic traffic participants into the background image and point cloud driving sequences. To diversify testing sequences, we incorporate six driving patterns derived from real-world common driving behaviors into the testing process. We conduct experiments with five SOTA MSF-based tracking systems to evaluate DRIVENCE from the perspectives of (1) generated test cases’ realism, (2) fault detection capabilities, and (3) test efficiency. The results show that DRIVENCE can generate realistic and modality-consistent test driving sequences and effectively detect various dynamic perception errors within MSF systems.

Safe reinforcement learning (SRL) aims to realize a safe learning process for deep reinforcement learning (DRL) algorithms by incorporating safety constraints. However, the efficacy of SRL approaches often relies on accurate function approximations, which are notably challenging to achieve in the early learning stages due to data insufficiency. To address this issue, we introduce, in this work, a novel generalizable safety enhancer (GenSafe) that can overcome the challenge of data insufficiency and enhance the performance of SRL approaches. Leveraging model order reduction techniques, we first propose an innovative method to construct a reduced order Markov decision process (ROMDP) as a low-dimensional approximator of the original safety constraints. Then, by solving the reformulated ROMDP-based constraints, GenSafe refines the actions of the agent to increase the possibility of constraint satisfaction. Essentially, GenSafe acts as an additional safety layer for SRL algorithms. We evaluate GenSafe on multiple SRL approaches and benchmark problems. The results demonstrate its capability to improve safety performance, especially in the early learning phases, while maintaining satisfactory task performance. Our proposed GenSafe not only offers a novel measure to augment existing SRL methods but also shows broad compatibility with various SRL algorithms, making it applicable to a wide range of systems and SRL problems.

Motivated by the substantial achievements of Large Language Models (LLMs) in the field of natural language processing, recent research has commenced investigations into the application of LLMs for complex, long-horizon sequential task planning challenges in robotics. LLMs are advantageous in offering the potential to enhance the generalizability as task-agnostic planners and facilitate flexible interaction between human instructors and planning systems. However, task plans generated by LLMs often lack feasibility and correctness. To address this challenge, we introduce ISR-LLM, a novel framework that improves LLM-based planning through an iterative self-refinement process. The framework operates through three sequential steps: preprocessing, planning, and iterative self-refinement. During preprocessing, an LLM translator is employed to convert natural language input into a Planning Domain Definition Language (PDDL) formulation. In the planning phase, an LLM planner formulates an initial plan, which is then assessed and refined in the iterative self-refinement step by a validator. We examine the performance of ISR-LLM across three distinct planning domains. Our experimental results show that ISR-LLM is able to achieve markedly higher success rates in sequential task planning compared to state-of-the-art LLM-based planners. Moreover, it also preserves the broad applicability and generalizability of working with natural language instructions.

Grasping and manipulating various kinds of objects cooperatively is the core skill of a dual-arm robot when deployed as an autonomous agent in a human-centered environment. This requires fully exploiting the robot's versatility and dexterity. In this work, we propose a general framework for dual-arm manipulators that contains two correlative modules. The learning-based dexterity-reachability-aware perception module deals with vision-based bimanual grasping. It employs an end-to-end evaluation network and probabilistic modeling of the robot's reachability to deliver feasible and dexterity-optimum grasp pairs for unseen objects. The optimization-based versatility-oriented control module addresses the online cooperative manipulation control by using a hierarchical quadratic programming formulation. Self-collision avoidance and dual-arm manipulability ellipsoid tracking with high reliability and fidelity are simultaneously achieved based on a learned lightweight distance proxy function and a speed-level tracking technique on Riemannian manifold. Intrinsic system safety is guaranteed, and a novel interface for skill transfer is enabled. A long-horizon rearrangement experiment, a bimanual turnover manipulation, and multiple comparative performance evaluation verify the effectiveness of the proposed framework.

For the safe application of reinforcement learning algorithms to high-dimensional nonlinear dynamical systems, a simplified system model is used to formulate a safe reinforcement learning (SRL) framework. Based on the simplified system model, a low-dimensional representation of the safe region is identified and used to provide safety estimates for learning algorithms. However, finding a satisfying simplified system model for complex dynamical systems usually requires a considerable amount of effort. To overcome this limitation, we propose a general data-driven approach that is able to efficiently learn a low-dimensional representation of the safe region. By employing an online adaptation method, the low-dimensional representation is updated using the feedback data to obtain more accurate safety estimates. The performance of the proposed approach for identifying the low-dimensional representation of the safe region is illustrated using the example of a quadcopter. The results demonstrate a more reliable and representative low-dimensional representation of the safe region compared with previous works, which extends the applicability of the SRL framework.

This article proposes an off-policy risk-sensitive reinforcement learning (RL)-based control framework to jointly optimize the task performance and constraint satisfaction in a disturbed environment. The risk-aware value function, constructed using the pseudo control and risk-sensitive input and state penalty terms, is introduced to convert the original constrained robust stabilization problem into an equivalent unconstrained optimal control problem. Then, an off-policy RL algorithm is developed to learn the approximate solution to the risk-aware value function. During the learning process, the associated approximate optimal control policy is able to satisfy both input and state constraints under disturbances. By replaying experience data to the off-policy weight update law of the critic neural network, the weight convergence is guaranteed. Moreover, online and offline algorithms are developed to serve as principled ways to record informative experience data to achieve a sufficient excitation required for the weight convergence. The proofs of system stability and weight convergence are provided. The Simulation results reveal the validity of the proposed control framework.

Although the state-of-the-art learning approaches exhibit impressive results for dynamical systems, only a few applications on real physical systems have been presented. One major impediment is that the intermediate policy during the training procedure may result in behaviors that are not only harmful to the system itself but also to the environment. In essence, imposing safety guarantees for learning algorithms is vital for autonomous systems acting in the real world. In this article, we propose a computationally effective and general safe learning framework, specifically for complex dynamical systems. With a proper definition of the safe region, a supervisory control strategy, which switches the actions applied on the system between the learning-based controller and a predefined corrective controller, is given. A simplified system facilitates the estimation of the safe region for the high-dimensional dynamical system. During the learning phase, the belief of the safe region is updated with the actual execution results of the corrective controller, which in turn enables the learning-based controller to have more freedom in choosing its actions. Two examples are given to demonstrate the performance of the proposed framework, one simple inverted pendulum to illustrate the online adaptation method, and one quadcopter control task to show the overall performance.