The classical Model Predictive Path Integral (MPPI) control framework, while effective in many applications, lacks reliable safety features since it relies due to its reliance on arisk-neutraltrajectory evaluation technique, which can present challenges for safety-critical applications such as autonomous driving. Furthermore, if when the majority of MPPI sampled trajectories concentrate in high-cost regions, it may generate aninfeasiblecontrol sequence. To address this challenge, we propose the U-MPPI control strategy, a novel methodology that can effectively manage system uncertainties while integrating a more efficient trajectory sampling strategy. The core concept is to leverage the Unscented Transform (UT) to propagate not only the mean but also the covariance of the system dynamics, going beyond the traditional MPPI method. As a result, it introduces a novel and more efficient trajectory sampling strategy, significantly enhancing state-space exploration and ultimately reducing the risk of being trapped in local minima. Furthermore, by leveraging the uncertainty information provided by UT, we incorporate arisk-sensitivecost function that explicitly accounts for risk or uncertainty throughout the trajectory evaluation process, resulting in a more resilient control system capable of handling uncertain conditions. By conducting extensive simulations of 2D aggressive autonomous navigation in both known and unknown cluttered environments, we verify the efficiency and robustness of our proposed U-MPPI control strategy compared to the baseline MPPI. We further validate the practicality of U-MPPI through real-world demonstrations in unknown cluttered environments, showcasing its superior ability to incorporate both the UT and local costmap into the optimization problem without introducing additional complexity.

Bounded rational agents often make decisions by evaluating a finite selection of choices, typically derived from a reference point termed the ‘default policy,’ based on previous experience. However, the inherent rigidity of the static default policy presents significant challenges for agents when operating in unknown environment, that are not included in agent’s prior knowledge. In this work, we introduce a context-generative default policy that leverages the region observed by the robot to predict unobserved part of the environment, thereby enabling the robot to adaptively adjust its default policy based on both the actual observed map and the imagined unobserved map. Furthermore, the adaptive nature of the bounded rationality framework enables the robot to manage unreliable or incorrect imaginations by selectively sampling a few trajectories in the vicinity of the default policy. Our approach utilizes a diffusion model for map prediction and a sampling-based planning with B-spline trajectory optimization to generate the default policy. Extensive evaluations reveal that the context-generative policy outperforms the baseline methods in identifying and avoiding unseen obstacles. Additionally, real-world experiments conducted with the Crazyflie drones demonstrate the adaptability of our proposed method, even when acting in environments outside the domain of the training distribution.

Robot data collected in complex real-world scenarios are often biased due to safety concerns, human preferences, and mission or platform constraints. Consequently, robot learning from such observational data poses great challenges for accurate parameter estimation. We propose a principled causal inference framework for robots to learn the parameters of a stochastic motion model using observational data. Specifically, we leverage the de-biasing functionality of the potential-outcome causal inference framework, the Inverse Propensity Weighting (IPW), and the Doubly Robust (DR) methods, to obtain a better parameter estimation of the robot's stochastic motion model. The IPW is a re-weighting approach to ensure unbiased estimation, and the DR approach further combines any two estimators to strengthen the unbiased result even if one of these estimators is biased. We then develop an approximate policy iteration algorithm using the bias-eliminated estimated state transition function. We validate our framework using both simulation and real-world experiments, and the results have revealed that the proposed causal inference-based navigation and control framework can correctly and efficiently learn the parameters from biased observational data.

Biological agents, such as humans and animals, are capable of making decisions out of a very large number of choices in a limited time. They can do so because they use their prior knowledge to find a solution that is not necessarily optimal but good enough for the given task. In this work, we study the motion coordination of multiple drones under the above-mentioned paradigm, Bounded Rationality (BR), to achieve cooperative motion planning tasks. Specifically, we design a prior policy that provides useful goal-directed navigation heuristics in familiar environments and is adaptive in unfamiliar ones via Reinforcement Learning augmented with an environment-dependent exploration noise. Integrating this prior policy in the game-theoretic bounded rationality framework allows agents to quickly make decisions in a group considering other agents' computational constraints. Our investigation assures that agents with a well-informed prior policy increase the efficiency of the collective decision-making capability of the group. We have conducted rigorous experiments in simulation and in the real world to demonstrate that the ability of informed agents to navigate to the goal safely can guide the group to coordinate efficiently under the BR framework.

Although providing exceptional asymptotic performance, trial-and-error based reinforcement learning suffers from being data inefficient, i.e., requiring an excessive amount of trial data for training. In this paper, we propose a data-efficient model-based reinforcement learning algorithm based on the Koopman operator theory. By representing the environment dynamics as a linear dynamical system in a high-dimensional space, the Koopman operator theory allows incorporating effective optimal control methods to produce high-quality trials thus accelerating learning. However, when applying the theory for reinforcement learning, with the sparse and unevenly distributed trial data, it is difficult to learn globally linear representations thus leading to serious model bias. To overcome this problem, we devise a local Koopman operator approach that is tailored for the setup of reinforcement learning. By coupling with deep neural networks and the linear quadratic regulator control, we propose the first Koopman operator model-based reinforcement learning algorithm called deep Koopman reinforcement learning (DKRL). The simulation experiment results show the proposed method can outperform existing approaches and can learn robotics control tasks with a small amount of data.

Imitation learning holds the promise to address challenging robotic tasks such as autonomous navigation. It however requires a human supervisor to oversee the training process and send correct control commands to robots without feedback, which is always prone to error and expensive. To minimize human involvement and avoid manual labeling of data in the robotic autonomous navigation with imitation learning, this paper proposes a novel semi-supervised imitation learning solution based on a multi-sensory design. This solution includes a suboptimalsensor policybased on sensor fusion to automatically label states encountered by a robot to avoid human supervision during training. In addition, arecording policyis developed to throttle the adversarial affect of learning too much from the suboptimal sensor policy. As a result, this solution allows the robot to learn a navigation policy in a self-supervised manner without human intervention after the initial data collection. With extensive experiments in indoor environments, this solution can achieve near human performance in most of the tasks and even surpasses human performance in case of unexpected events such as hardware failures or human operation errors. To best of our knowledge, this is the first work that synthesizes sensor fusion and imitation learning to enable robotic autonomous navigation in the real world without human supervision.

We investigate the autonomous navigation of a mobile robot in the presence of other moving vehicles under time-varying uncertain environmental disturbances. We first predict the future state distributions of other vehicles to account for their uncertain behaviors affected by the time-varying disturbances. We then construct a dynamic-obstacle-aware reachable space that contains states with high probabilities to be reached by the robot, within which the optimal policy is searched. Since, in general, the dynamics of both the vehicle and the environmental disturbances are nonlinear, we utilize a nonlinear Gaussian filter – the unscented transform – to approximate the future state distributions. Finally, the forward reachable space computation and backward policy search are iterated until convergence. Extensive simulation evaluations have revealed significant advantages of this proposed method in terms of computation time, decision accuracy, and planning reliability.

Smart health calls for novel approaches to detect vital signs in non- contact, non-invasive and non-intrusive matters. In this work, we design a solution that monitors the rates of heartbeats and respiration simultaneously by using a Frequency Modulated Continuous Wave (FMCW) radar with multiple antennas. This solution measures the reflections from heartbeats and respiration at a high frequency of 4 K H z to capture fine dynamics of motions with big data. It employs multiple antennas and superposition to reduce the interference noises from unwanted motions in the background and any detection defects. The heart and respiration rates are detected in the frequency domains after a chain of preprocessing techniques on the sensed big data. With extensive experiments in a lab office, this system demonstrates high accuracies in various cases: 98% in the still case, 95% with finger motions and 96% with body motions. The tests also confirm that multiple antennas and signal superposition improve the detection accuracy and reliability.

Deep imitation learning enables robots to learn from expert demonstrations to perform tasks such as lane following or obstacle avoidance. However, in the traditional imitation learning framework, one model only learns one task, and thus it lacks of the capability to support a robot to perform various different navigation tasks with one model in indoor environments. This paper proposes a new framework, Shared Multi-headed Imitation Learning (SMIL), that allows a robot to perform multiple tasks with one model without switching among different models. We model each task as a sub-policy and design a multi-headed policy to learn the shared information among related tasks by summing up activations from all sub-policies. Compared to single or non-shared multi-headed policies, this framework is able to leverage correlated information among tasks to increase performance. We have implemented this framework using a robot based on NVIDIA TX2 and performed extensive experiments in indoor environments with different baseline solutions. The results demonstrate that SMIL has doubled the performance over non-shared multi-headed policy.

Indoor human activity recognition has been extensively investigated. However, most of the solutions require sensors e.g. 9-axis IMU be equipped on human body or use image processing that presents privacy issues. This work proposes an ambient radar sensor based a solution to recognize the activities that humans normally perform in indoor environments. This solution uses a 7.8 GHz radar to emit 16 pulse signals every second and samples the reflected signals at 128 KHz to capture the fine dynamics of human activities. This solution designs a set of data preprocessing algorithms, including a data refining algorithm to filter outlier data, a contrastive divergence algorithm to remove background static reflection, and a transformation algorithm to convert the signal data into feature- rich spatial location changes. This solution also develops schemes to separate a collection of various activities into individuals. A lowpass frequency filter is designed to remove unwanted noisy data and the motion intensity is used to classify the activities into two high-level groups. It uses a slope-based approach and a k- means clustering to further finely recognize each activity. This solution has been extensively evaluated in a spacious research lab room and shows outstanding accuracy.

The world has entered into a “smart” era. One area becoming smart is the place where we live-homes. Smart homes are expected to be equipped with numerous sensors to continually monitor, sense, and actuate the space. The data from these sensors can be used to provide various types of services by automating common tasks while causing minimal disruption to daily life. In order to provide these services, a system must have sufficient intelligence to predict future events based on its observations. This paper first examines the requirements for smart home predictions. It then comprehensively reviews prediction algorithms and variations that have been proposed and investigated in smart environments, such as smart homes. It is these prediction algorithms that provide the intelligence required by a smart home. Comparisons are also made upon these prediction algorithms on their features and models.