Visual deep reinforcement learning (RL) enables robots to acquire skills from visual input for unstructured tasks. However, current algorithms suffer from low sample efficiency, limiting their practical applicability. In this work, we present MENTOR, a method that improves both the *architecture* and *optimization* of RL agents. Specifically, MENTOR replaces the standard multi-layer perceptron (MLP) with a mixture-of-experts (MoE) backbone and introduces a task-oriented perturbation mechanism. MENTOR outperforms state-of-the-art methods across three simulation benchmarks and achieves an average of 83\% success rate on three challenging real-world robotic manipulation tasks, significantly surpassing the 32% success rate of the strongest existing model-free visual RL algorithm. These results underscore the importance of sample efficiency in advancing visual RL for real-world robotics. Experimental videos are available at https://suninghuang19.github.io/mentor_page/.

Catching objects in flight (i.e., thrown objects) is a common daily skill for humans, yet it presents a significant challenge for robots. This task requires a robot with agile and accurate motion, a large spatial workspace, and the ability to interact with diverse objects. In this paper, we build a mobile manipulator composed of a mobile base, a 6-DoF arm, and a 12-DoF dexterous hand to tackle such a challenging task. We propose a two-stage reinforcement learning framework to efficiently train a whole-body-control catching policy for this high-DoF system in simulation. The objects' throwing configurations, shapes, and sizes are randomized during training to enhance policy adaptivity to various trajectories and object characteristics in flight. The results show that our trained policy catches diverse objects with randomly thrown trajectories, at a high success rate of about 80 % in simulation, outperforming the baselines significantly. The policy trained in simulation can be deployed seamlessly to the real world with only onboard sensing and computation, which achieves catching sandbags in various shapes, randomly thrown by humans. Our video and code are available at https://mobile-dex-catch.github.io.

Due to its heterogeneous property, multi-view data has been widely concerned over single-view data for performance improvement. Unfortunately, some instances may be with partially available information because of some uncontrollable factors, for which the incomplete multi-view clustering (IMVC) problem is raised. IMVC aims to partition unlabeled incomplete multi-view data into their clusters by exploiting the heterogeneity of multi-view data and overcoming the difficulty of data loss. However, most existing IMVC methods like BSV, MIC, OMVC, and IVC tend to conduct basic completion processing on the input data, without taking advantage of the correlation between samples and information redundancy. To overcome the above issue, we propose one novel IMVC method named data completion-guided unified graph learning (DCUGL), which could complete the data of missing views and fuse multiple learned view-specific similarity matrices into one unified graph. Specifically, we first reduce the dimension of the input data to learn multiple view-specific similarity matrices. By stacking all view-specific similarity matrices, DCUGL constructs a third-order tensor with the low-rank constraint, such that sample correlation within and between views can be well explored. Finally, by dividing the original data into observed data and unobserved data, DCUGL can infer and complete the missing data according to the view-specific similarity matrices, and obtain a unified graph, which can be directly used for clustering. To solve the proposed model, we design an iterative algorithm, which is based on the alternating direction method of multipliers framework. The proposed model proves to be superior by benchmarking on six challenging datasets compared with state-of-the-art IMVC methods.

We present ClothesNet: a large-scale dataset of 3D clothes objects with information-rich annotations. Our dataset consists of around 4400 models covering 11 categories annotated with clothes features, boundary lines, and keypoints. ClothesNet can be used to facilitate a variety of computer vision and robot interaction tasks. Using our dataset, we establish benchmark tasks for clothes perception, including classification, boundary line segmentation, and keypoint detection, and develop simulated clothes environments for robotic interaction tasks, including rearranging, folding, hanging, and dressing. We also demonstrate the efficacy of our ClothesNet in real-world experiments. Supplemental materials and dataset are available on our project webpage at https://sites.google.com/view/clothesnet.