Synthetic image generation plays a crucial role in the development of robot vision algorithms, circumventing manual data collection. However, the realism of synthetic images could affect the performance of the algorithms when applied in real-world settings. In this study, we propose a framework to quantitatively assess the realism of synthetic images using a set of realism metrics as a means of performance characterization. We use a commercial rendering engine as a test-bed for generating synthetic images and ascertain that a set of rendering parameters affect specific image metrics through statistical hypothesis testing. We demonstrate that this framework can be used to optimize rendering parameter values and generate synthetic datasets with improved performance on downstream robot vision tasks such as instance segmentation.

A single shepherd dog can herd a flock of sheep to a gate. Despite a heuristic algorithm of a dog based on adaptive switching between collecting the sheep when they are too dispersed and driving them once they are aggregated, it remains unknown how the dog learns the algorithm of switching. In fact, reinforcement learning models have not succeeded so far in reproducing the switching algorithm without explicitly making two strategies. Here, we show that an imitation learning model can reproduce the switching algorithm, that is, the dog learns the algorithm from demonstrations by an expert. We also confirmed that the dog does not simply copy the demonstrations but learns the required task by showing that it can herd more sheep than those in the given demonstrations.

A robust, easy-to-deploy robot for service tasks in a real environment is difficult to construct. Record-and-playback (R&P) is a method used to teach motor-skills to robots for performing service tasks. However, R&P methods do not scale to challenging tasks where even slight changes in the environment, such as localization errors, would either require trajectory modification or a new demonstration. In this paper, we propose a Sequence-to-Sequence (Seq2Seq) based neural network model to generate robot trajectories in configuration space given a context variable based on real-world measurements in Cartesian space. We use the offset between a target pose and the actual pose after localization as the context variable. The model is trained using a few expert demonstrations collected using teleoperation. We apply our proposed method to the task of toilet cleaning where the robot has to clean the surface of a toilet bowl using a compliant end-effector in a constrained toilet setting. In the experiments, the model is given a novel offset context and it generates a modified robot trajectory for that context. We demonstrate that our proposed model is able to generate trajectories for unseen setups and the executed trajectory results in cleaning of the toilet bowl.

Robotic solutions to clothing assistance can significantly improve quality of life for the elderly and disabled. Real-time estimation of the human-cloth relationship is crucial for efficient learning of motor skills for robotic clothing assistance. The major challenge involved is cloth-state estimation due to inherent nonrigidity and occlusion. In this study, we present a novel framework for real-time estimation of the cloth state using a low-cost depth sensor, making it suitable for a feasible social implementation. The framework relies on the hypothesis that clothing articles are constrained to a low-dimensional latent manifold during clothing tasks. We propose the use of manifold relevance determination (MRD) to learn an offline cloth model that can be used to perform informed cloth-state estimation in real time. The cloth model is trained using observations from a motion capture system and depth sensor. MRD provides a principled probabilistic framework for inferring the accurate motion-capture state when only the noisy depth sensor feature state is available in real time. The experimental results demonstrate that our framework is capable of learning consistent task-specific latent features using few data samples and has the ability to generalize to unseen environmental settings. We further present several factors that affect the predictive performance of the learned cloth-state model.

Real-time estimation of human-cloth relationship is crucial for efficient learning of motor skills in robotic clothing assistance. However, cloth state estimation using a depth sensor is a challenging problem with inherent ambiguity. To address this problem, we propose the offline learning of a cloth dynamics model by incorporating reliable motion capture data and applying this model for the online tracking of human-cloth relationship using a depth sensor. In this study, we evaluate the performance of using a shared Gaussian Process Latent Variable Model in learning the dynamics of clothing articles. The experimental results demonstrate the effectiveness of shared GP-LVM in capturing cloth dynamics using few data samples and the ability to generalize to unseen settings. We further demonstrate three key factors that affect the predictive performance of the trained dynamics model.

In this study, we propose a novel method for the real-time estimation of Human-Cloth relationship, which is crucial for efficient motor skill learning in Robotic Clothing Assistance. This system relies on the use of low cost depth sensor, which provides color and depth images without requiring an elaborate setup making it suitable for real-world applications. We present an efficient algorithm to estimate the parameters that represent the topological relationship between human and the clothing article. At the core of our approach are low dimensional representation of Human-Cloth relationship using topology coordinates for fast learning of motor skills and a unified ellipse fitting algorithm for the compact representation of the state of clothing articles. We conducted experiments that illustrate the robustness of these feature representations. Furthermore, we evaluated the performance of our proposed method by applying it to real-time clothing assistance tasks and compared the estimates provided by our method with the ground truth.