Behavioral cloning from human demonstrations has succeeded in programming a robot to generate fine-grained motion, but it is still challenging to learn multimodal trajectories such as with various speeds. This restricts the use of a robot dataset collected by multiusers because the different proficiency of robot operators makes the dataset have diverse distributions of speed. To tackle this issue, we develop Hierarchical Action Chunking Transformer with Vector-quantization (HACT-Vq) to efficiently learn temporal multimodality in addition to fine-grained motion. The proposed hierarchical model consists of a high-level policy to make planning for a latent subgoal and style, and a low-level policy to predict an action chunk conditioned with the latent subgoal and style. The latent subgoal and style are trained as discrete representations so that high-level policy can efficiently learn multimodal distributions of demonstrations and retrieve the mode of fast behavior. In experiments, we set up bimanual robots in both simulation and real-world environments, and collected demonstrations with various speeds. The proposed model with the quantized subgoal and style showed the highest success rates with fast imitation behavior. Our code is available at https://github.com/SamsungLabs/hierarchical-act.

This paper proposes a novel approach to address the technical challenges of stable object grasping, particularly in the context of handling tableware in a home environment. Handling tableware is particularly important, yet challenging, due to the flat nature of most tableware objects and the need to maintain a stable posture to prevent spills. To address these challenges, we present three key contributions: 1) a large-scale tableware dataset, not commonly found in the previous datasets; 2) a novel sampling method for stable grasp pose generation; and 3) a multi-modal fusion grasp network that effectively learns 6- DoF grasp pose, including flat objects. Our dataset contains over 45 million grasp poses and 1 million RGBD images captured in 800 scenes, which include randomly selected 10–18 tableware objects under 4 different lighting conditions. The grasp poses in the dataset are generated using a novel sampling method that incorporates geometric analysis to ensure stable grasping with minimal object movement. Furthermore, we design an RGBD fusion grasp network (RGBD-FGN) that can combine information from RGB and depth images considering each characteristic. Our experimental results demonstrate the superior performance of our approach over existing techniques, which is a significant contribution towards developing a multitasking home robot. Our dataset and source code can be accessed at https://github.com/SamsungLabs/RGBD-FGN.

We propose a novel model-free optimal estimation and sensor placement framework for a high-DOF (degree-of-freedom) EKC (elastic kinematic chain) with only a limited number of IMU (inertial measurement unit) sensors based on POD (proper orthogonal decomposition) and MAP (maximum a posteriori) estimation. First, we (off-line) excite the system richly enough, collect the data and perform the POD to extract dominant and non-dominant modes. We then decide the minimum number of IMUs according to the dominant modes, and construct the prior distribution of the output (i.e., top-end position of EKC) based on the singular value of each POD mode. We also formulate the MAP estimation given the prior distribution and different placements of the IMUs and choose the optimal IMU placement to maximize the posterior probability. This optimal placement is then used for real-time output estimation of the EKC. Experiments are also performed to verify the theory.

Electrical motor and hydraulic actuation widely-used in robotics are “internal actuation” with their actuators sitting at the joint between two links. This internal actuation is fundamentally limiting to construct a large-size dexterously-articulated robot, since any external force (and its own link weight) is to be accumulated to the base multiplied by the moment arm length, requiring extremely strong/sturdy base actuator/structure as the system size increases. In this paper, we propose a novel robotic system, LASDRA (large-size aerial skeleton with distributed rotor actuation), which, by utilizing distributed rotors as “external actuation”, can overcome this limitation of internal actuation and enables us to realize large-size dexterously-articulated robots. We present its design and modeling, joint locking strategy to increase its loading capability, and also a novel decentralized control scheme to allow for compliant operation with scalability against the number of links. Trajectory tracking and valve turning experiments are also performed to validate the theory.