The soft robotic hand exhibits a wide range of manipulation capabilities, which are attributed to the dexterity of its soft fingers and their coordinated movements. Therefore, designing a versatile soft hand requires careful consideration of both the characteristics of the individual fingers, such as degree of freedom (DOF), and their strategic arrangements to improve performance for specific target tasks. This work presents a modularized high DOF tendon-driven soft finger and a customized design of a soft robotic hand for diverse dexterous manipulation tasks. Furthermore, an all-in-one module is developed that integrates both the 4-way tendon-driven soft finger body and drive parts. Its high DOF enables multidirectional actuations with a wide actuation range, thereby expanding possible manipulation modes. The modularity of the system expands the design space for finger arrangements, which enables the diverse configuration of robotic hands and facilitates the customization of task-oriented platforms. To achieve sophisticated control of these complex configurations, we employ neural network-planned trajectories, enabling the precise execution of complicated tasks. The performance of a single finger is validated, including dexterity and payload, and several real-world manipulation tasks are demonstrated, including writing, grasping, rotating, and spreading, using motion primitives of diverse soft hands with distinctive finger arrangements. These demonstrations showcase the system’s versatility and precision in various tasks. We expect that our system will contribute to the expansion of possibilities in the field of soft robotic manipulation.

This work presents a motion retargeting approach for legged robots, aimed at transferring the dynamic and agile movements to robots from source motions. In particular, we guide the imitation learning procedures by transferring motions from source to target, effectively bridging the morphological disparities while ensuring the physical feasibility of the target system. In the first stage, we focus on motion retargeting at the kinematic level by generating kinematically feasible whole-body motions from keypoint trajectories. Following this, we refine the motion at the dynamic level by adjusting it in the temporal domain while adhering to physical constraints. This process facilitates policy training via reinforcement learning, enabling precise and robust motion tracking. We demonstrate that our approach successfully transforms noisy motion sources, such as hand-held camera videos, into robot-specific motions that align with the morphology and physical properties of the target robots. Moreover, we demonstrate terrain-aware motion retargeting to perform BackFlip on top of a box. We successfully deployed these skills to four robots with different dimensions and physical properties in the real world through hardware experiments.

Recent successes in robot learning have significantly enhanced autonomous systems across a wide range of tasks. However, they are prone to generate similar or the same solutions, limiting the controllability of the robot to behave according to user intentions. These limited robot behaviors may lead to collisions and potential harm to humans. To resolve these limitations, we introduce a semi-autonomous teleoperation framework that enables users to operate a robot by selecting a high-level command, referred to as option. Our approach aims to provide effective and diverse options by a learned policy, thereby enhancing the efficiency of the proposed framework. In this work, we propose a quality-diversity (QD) based sampling method that simultaneously optimizes both the quality and diversity of options using reinforcement learning (RL). Additionally, we present a mixture of latent variable models to learn multiple policy distributions defined as options. In experiments, we show that the proposed method achieves superior performance in terms of the success rate and diversity of the options in simulation environments. We further demonstrate that our method outperforms manual keyboard control for time duration over cluttered real-world environments.

In this paper, we focus on the problem of efficiently locating a target object described with free-form text using a mobile robot equipped with vision sensors (e.g., an RGBD camera). Conventional active visual search predefines a set of objects to search for, rendering these techniques restrictive in practice. To provide added flexibility in active visual searching, we propose a system where a user can enter target commands using free-form text; we call this system Zero-shot Active Visual Search (ZAVIS). ZAVIS detects and plans to search for a target object inputted by a user through a semantic grid map represented by static landmarks (e.g., desk or bed). For efficient planning of object search patterns, ZAVIS considers commonsense knowledge-based co-occurrence and predictive uncertainty while deciding which landmarks to visit first. We validate the proposed method with respect to SR (success rate) and SPL (success weighted by path length) in both simulated and real-world environments. The proposed method outperforms previous methods in terms of SPL in simulated scenarios, and we further demonstrate ZAVIS with a Pioneer-3AT robot in real-world studies.