Articulated objects are ubiquitous in daily environments, and effective manipulation of these objects is essential for advancing open-world robotics. Existing approaches, which rely heavily on large-scale data collection or simulation, often face limitations in real-world applications, including issues with generalization and the sim-to-real gap. In this paper, we introduce the Automated Articulated Object Parameter Learning (AAOPL) framework, which autonomously learns the articulation parameters of real-world articulated objects through direct interaction. This approach enables robots to generate precise manipulation trajectories without relying on predefined object models or extensive human demonstration data. To accelerate the learning process, we develop Accelerated Single-Step Gradient (ASSG) algorithm, which efficiently refines the articulation parameters by leveraging real-time execution feedback. Experimental results demonstrate that AAOPL can learn accurate articulation parameters within 30 minutes (80 trials) and generate robust manipulation trajectories, outperforming baseline methods in terms of both task completion and force efficiency. Our approach eliminates the need for large-scale training datasets and can adapt to various articulated objects in real-world environments, offering a scalable solution for autonomous robotic manipulation in unstructured settings.

The data-driven paradigm has shown great potential in solving many decision-making tasks. In the robot navigation realm, it also sparked a new trend. People believe powerful data-driven methods can learn efficient and general navigation policies from a vast offline dataset. However, robot navigation tasks differ from common planning tasks and present unique challenges. It often involves multi-objective optimization to meet arbitrary and ever-changing human preferences. It should also overcome the short-sighted problem to obtain globally optimal performance. Furthermore, high planning frequency is needed to address real-time demands. These factors obstruct the application of data-driven methods in robot navigation. To address these challenges, we integrate one of the most powerful data-driven methods, the diffusion model, into robot navigation. Our proposed approach, NaviDiffuser, utilizes a novel classification label to guide the diffusion model in capturing the complex connections between navigation and human preferences. Its Transformer network backbone outputs action sequences to alleviate short-sightedness. It also includes special distillation skills to boost the planning speed and quality. We conduct experiments in both simulated and real-world scenarios to evaluate our approach. In these experiments, NaviDiffuser not only demonstrates an extremely high arrival rate but also adjusts its navigation policy to align with different human preferences.

Dual-arm robotic systems hold great potential for complex bimanual tasks that require intricate and coordinated manipulation, such as holding and transporting a tray with a cup of coffee while navigating through cluttered environments. However, these tasks pose significant challenges due to the inherent closed-chain constraints between the arms and the object, as well as the need for real-time collision avoidance, especially in real-world applications. To address these challenges, we introduce a hierarchical framework that combines learning-based planning with classical control theory to ensure whole-body collision avoidance movement while maintaining the kinematic relationship. In addition, we present a novel, efficient, and cost-free data generation method specifically designed for dual-arm cooperative tasks, overcoming the lack of sufficient training data. Extensive experiments in both simulation and real-world scenarios demonstrate that our approach improves the success rate by 26.3% compared to existing planning methods and by 54.7% compared to end-to-end methods. These results highlight the advantages of our method in whole-body collision avoidance and environmental adaptability, making it a promising solution for dual-arm cooperative tasks.

Robot navigation using deep reinforcement learning (DRL) has shown great potential in improving the performance of mobile robots. Nevertheless, most existing DRL-based navigation methods primarily focus on training a policy that directly commands the robot with low-level controls, like linear and angular velocities, which leads to unstable speeds and unsmooth trajectories of the robot during the long-term execution. An alternative method is to train a DRL policy that outputs the navigation path directly. Then the robot can follow the generated path smoothly using sophisticated velocity-planning and path-following controllers, whose parameters are specified according to the hardware platform. However, two roadblocks arise for training a DRL policy that outputs paths: (1) The action space for potential paths often involves higher dimensions comparing to low-level commands, which increases the difficulties of training; (2) It takes multiple time steps to track a path instead of a single time step, which requires the path to predicate the interactions of the robot w.r.t. the dynamic environment in multiple time steps. This, in turn, amplifies the challenges associated with training. In response to these challenges, we propose PathRL, a novel DRL method that trains the policy to generate the navigation path for the robot. Specifically, we employ specific action space discretization techniques and tailored state space representation methods to address the associated challenges. Curriculum learning is employed to expedite the training process, while the reward function also takes into account the smooth transition between adjacent paths. In our experiments, PathRL achieves better success rates and reduces angular rotation variability compared to other DRL navigation methods, facilitating stable and smooth robot movement. We demonstrate the competitive edge of PathRL in both real-world scenarios and multiple challenging simulation environments.

Reinforcement learning has shown great potential in improving the performance of robot navigation. In response to the increasing deployments of mobile robots within various scenarios, a data-driven paradigm of navigation approach with safety verification is preferred where one can train RL algorithms with large amounts of prior data, keep learning continuously, and ensure safe navigation in applications. Conventional end-to-end reinforcement learning navigation paradigms have encountered multiple challenges in meeting these demands. In this work, we introduce a novel robot navigation approach termed NaviFormer. This approach handles navigation tasks based on sequence modeling to obtain the data-driven ability. It also integrates rule-based verification for safety insurance. We conduct a series of experiments to validate the data-driven ability of our approach and to compare it with existing navigation methods. We also perform quantitative tests on a real-world robot platform, TurtleBot. The experimental results show our method’s outstanding data-driven ability and highlight its superior arrival rate and generalization compared to other state-of-the-art methods like the PPO-based navigation method.

Existing navigation policies for autonomous robots tend to focus on collision avoidance while ignoring human-robot interactions in social life. For instance, robots can pass along the corridor safer and easier if pedestrians notice them. Sounds have been considered as an efficient way to attract the attention of pedestrians, which can alleviate the freezing robot problem. In this work, we present a new deep reinforcement learning (DRL) based social navigation approach for autonomous robots to move in pedestrian-rich environments with interaction capacity. Most existing DRL based methods intend to train a general policy that outputs both navigation actions, i.e., expected robot's linear and angular velocities, and interaction actions, i.e., the beep action, in the context of reinforcement learning. Different from these methods, we intend to train the policy via both supervised learning and reinforcement learning. In specific, we first train an interaction policy in the context of supervised learning, which provides a better understanding of the social situation, then we use this interaction policy to train the navigation policy via multiple reinforcement learning algorithms. We evaluate our approach in various simulation environments and compare it to other methods. The experimental results show that our approach outperforms others in terms of the success rate. We also deploy the trained policy on a real-world robot, which shows a nice performance in crowded environments.

It is challenging for a mobile robot to navigate through human crowds. Existing approaches usually assume that pedestrians follow a predefined collision avoidance strategy, like social force model (SFM) or optimal reciprocal collision avoidance (ORCA). However, their performances commonly need to be further improved for practical applications, where pedestrians follow multiple different collision avoidance strategies. In this paper, we propose a map-based deep reinforcement learning approach for crowd-aware robot navigation with various pedestrians. We use the sensor map to represent the environmental information around the robot, including its shape and observable appearances of obstacles. We also introduce the pedestrian map that specifies the movements of pedestrians around the robot. By applying both maps as inputs of the neural network, we show that a navigation policy can be trained to better interact with pedestrians following different collision avoidance strategies. We evaluate our approach under multiple scenarios both in the simulator and on an actual robot. The results show that our approach allows the robot to successfully interact with various pedestrians and outperforms compared methods in terms of the success rate.