This paper proposes a novel approach to address the technical challenges of stable object grasping, particularly in the context of handling tableware in a home environment. Handling tableware is particularly important, yet challenging, due to the flat nature of most tableware objects and the need to maintain a stable posture to prevent spills. To address these challenges, we present three key contributions: 1) a large-scale tableware dataset, not commonly found in the previous datasets; 2) a novel sampling method for stable grasp pose generation; and 3) a multi-modal fusion grasp network that effectively learns 6- DoF grasp pose, including flat objects. Our dataset contains over 45 million grasp poses and 1 million RGBD images captured in 800 scenes, which include randomly selected 10–18 tableware objects under 4 different lighting conditions. The grasp poses in the dataset are generated using a novel sampling method that incorporates geometric analysis to ensure stable grasping with minimal object movement. Furthermore, we design an RGBD fusion grasp network (RGBD-FGN) that can combine information from RGB and depth images considering each characteristic. Our experimental results demonstrate the superior performance of our approach over existing techniques, which is a significant contribution towards developing a multitasking home robot. Our dataset and source code can be accessed at https://github.com/SamsungLabs/RGBD-FGN.

We propose a novel real-time physically-accurate simulator for long flexible cable manipulation. We first discretize the cable into multiple rigid link segments, each with complementarity-based contact model and inter-segment compliant coupling; and partition the cable into a number of subsystems, each composed with a number of consecutive links. We then formulate the inter-subsystem consistency constraint as a certain analytical condition among the inter-subsystem coupling and the contact impulses; and solve each subsystem dynamics in parallel with the contact model together with this consistency condition in an iterative manner, achieving both the speed and the accuracy of the simulation. A novel post-regulation scheme is also proposed to further speed up the simulation. Experimental validation/demonstration are also performed to show the theory.

We propose a novel real-time physically-accurate simulation framework for the snap connection process. For this, we first notice the peculiarities of the process, namely, small/smooth deformation, stiff connector and segmented contact. We then design our simulation to fully exploit these peculiarities by adopting the following strategies: 1) the technique of passive midpoint integration (PMI [1]), which allows for stable simulation of arbitrarily light/stiff system by enforcing discrete-time passivity; 2) linear finite element method (FEM [2]) modeling, which is adequate to deal with the small snap connector deformation while providing much faster speed as compared to nonlinear FEM; 3) segmentation of the snap connector FEM model and solving of each segment individually with their coupling analytically eliminated, thereby, further speeding up the simulation; 4) balanced model reduction (BMR [3]) to further reduce the dimension of each segment purely analytically without any prior experiment or simulation; and 5) parallelized data-driven collision detection, which turns out to further significantly speed up our simulation. Experimentally-verified simulations are also performed to show the efficacy of our proposed simulation framework.

We propose a novel model-free optimal estimation and sensor placement framework for a high-DOF (degree-of-freedom) EKC (elastic kinematic chain) with only a limited number of IMU (inertial measurement unit) sensors based on POD (proper orthogonal decomposition) and MAP (maximum a posteriori) estimation. First, we (off-line) excite the system richly enough, collect the data and perform the POD to extract dominant and non-dominant modes. We then decide the minimum number of IMUs according to the dominant modes, and construct the prior distribution of the output (i.e., top-end position of EKC) based on the singular value of each POD mode. We also formulate the MAP estimation given the prior distribution and different placements of the IMUs and choose the optimal IMU placement to maximize the posterior probability. This optimal placement is then used for real-time output estimation of the EKC. Experiments are also performed to verify the theory.

We propose a novel data-driven learning-based contact clustering (i.e., of contact points and contact normals) framework for rigid-body robot simulation, with its accuracy established/verified by real experimental data. We first construct an experimental robotic setup with force/torque (F/T) sensors to collect real contact motion/force data. We then design a multilayer perceptron (MLP) network for the contact clustering based on the full motion and force/torque information of the contacts. We also adopt the constraint-based optimization contact solver to facilitate the learning of our MLP network during the training. Our proposed data-driven/learning-based contact clustering framework is then verified against the experimental setup, compared with other techniques/simulators and shown to significantly (or meaningfully) enhance the accuracy of contact simulation as compared to them.

We propose a novel fast simulation framework for soft objects/robots with intermittent contacts, whose contact areas/locations can be varying. We first perform a balanced model reduction of the full-order FEM (finite element method) model for each contact mode with the contact forcing as the input and the shape of the object/robot as the output. We then devise the strategy of passive model reduction and passive model switching of these reduced-order models (each with its contact mode) utilizing the techniques of our recently-proposed passive mid-point integration (for the passivity of each reduced-order model) and simultaneous diagonalization (for the passivity of model reduction and model switching). The efficacy of the theory is then demonstrated with simulation and experimental results.

We propose a novel control framework to enable nonholonomic wheeled mobile robots (WMRs) to autonomously drive in an environment with the speed fast enough so that the dynamics effect (e.g., Coriolis effect) is not negligible, yet, still less than a certain threshold to prevent slippage at the wheels. For this, instead of the Newtonian vehicle modeling, we adopt Lagrange-D'Alembert formulation, which then allows us to explicitly relate the system's state/control with the constraint force, so that we can predict/detect possibility of a given motion's violating the no-slip condition. We present a scheme to generate a no-slip/collision-free timed-trajectory for the WMRs using this Lagrange-D'Alembert formulation. We also propose a backstepping-based control law, which enables the WMR to track the generated trajectory while respecting its nonholonomic constraints. Experiment, using a modified commercial radio-controlled car, is performed to verify the theory.