Pouring snacks and moving containers with beverages are challenging for a service robot. To obtain accurate content properties for planning robotic motion, tactile sensing can provide information about the pressure distribution of the contact surface, which is not obvious by visual observation. In this work, we focus on estimating the content properties of various content materials in distinct deformable containers through tactile interactions. We propose a learning-based model that can estimate content properties by using the tactile data collected by slightly squeezing a container with the content of interest. We analyzed an uncalibrated tactile sensor and collected a dataset consisting of 1125 sets of tactile sequences, which are combinations of five types of deformable containers and eleven types of content materials in different content heights. Experiments were conducted on content estimation with known contents and containers, unknown contents, and unknown containers. For unknown contents, our model can still achieve 8.5% height relative error and 79.7% state of matter accuracy. Furthermore, we analyzed that the tactile features of contents with similar content properties are close in the latent snace to show the effectiveness of our model.

In this article, we focus on rearranging an object by pushing it to any target planar pose. We identify the essential elements to guarantee that the target pose can be reached at an acceptable precision. We present a simple rearrangement algorithm that relies on only a few known straight-line pushes for some novel object and requires no analytical models, force sensors, or large training datasets. We derive the step upper bound, which relates the initial pose of the object, stopping criterion, and quality of the set of pushes, to facilitate the estimation of the maximum number of required steps without the need to perform a task. We experimentally verified the performance of our algorithm at different noise levels, stopping criteria, and task difficulties on datasets containing several types of objects. By applying combinations of only nine known pushes, our simple algorithm performed successfully in real-world experiments with challenging objects, including partially deformable objects that are difficult to model analytically; it achieved the precise stopping criterion (7.5 mm, 5$^\circ$) in various rearrangement tasks.

Studies on object rearrangement planning typically consider known objects. Some learning-based methods can predict the movement of an unknown object after single-step interaction, but require intermediate targets, which are generated manually, to achieve the rearrangement task. In this work, we propose a framework for unknown object rearrangement. Our system first models an object through a small-amount of identification actions and adjust the model parameters during task execution. We implement the proposed framework based on a low-cost tabletop robot (under 180 USD) to demonstrate the advantages of using a physics engine to assist action prediction. Experimental results reveal that after running our adaptive learning procedure, the robot can successfully arrange a novel object using an average of five discrete pushes on our tabletop environment and satisfy a precise 3.5 cm translation and 5° rotation criterion.

Structured-light and stereo cameras, which are widely used to construct point clouds for robotic applications, have different limitations on estimating depth values. Structured-light cameras fail in black, transparent, and reflective objects, which influence the light path; stereo cameras fail in texture-less objects. In this work, we propose a depth fusion model that complements these two types of methods to generate high-quality point clouds for short-range robotic applications. The model first determines the fusion weights from the two input depth images and then refines the fused depth using color features. We construct a dataset containing the aforementioned challenging objects and report the performance of our proposed model. The results reveal that our method reduces the average L1 distance on depth prediction by 75% and 52% compared with the original depth output of the structured-light camera and the stereo model, respectively. A noticeable improvement on the Iterative Closest Point (ICP) algorithm can be achieved by using the refined depth images output from our method.

This paper presents an object-centric method for efficiently performing two types of challenging pick-and-place tasks, namely sequential pick and place and object sorting. We propose multiclass dense object nets (MCDONs) for learning object-centric dense descriptors that maintain not only intra-class variations but also inter-class separation. Intra-class consistency is also inherently learned and is useful for our pick-and-place tasks. All the tasks only require a single demonstration from users, which can then be generalized to all class instances. A dataset containing eight classes and a total of 52 objects was provided in this study. We obtained a task success rate of 93.33% on a five-block stacking task and 97.41% on a three-class object sorting task.