We propose Co-op, a novel method for accurately and robustly estimating the 6DoF pose of objects unseen during training from a single RGB image. Our method requires only the CAD model of the target object and can precisely estimate its pose without any additional fine-tuning. While existing model-based methods suffer from inefficiency due to using a large number of templates, our method enables fast and accurate estimation with a small number of templates. This improvement is achieved by finding semi-dense correspondences between the input image and the pre-rendered templates. Our method achieves strong generalization performance by leveraging a hybrid representation that combines patch-level classification and offset regression. Additionally, our pose refinement model estimates probabilistic flow between the input image and the rendered image, refining the initial estimate to an accurate pose using a differentiable PnP layer. We demonstrate that our method not only estimates object poses rapidly but also outperforms existing methods by a large margin on the seven core datasets of the BOP Challenge, achieving state-of-the-art accuracy.

Despite the progress of learning-based methods for 6D object pose estimation, the tradeoff between accuracy and scalability for novel objects still exists. Specifically, previous methods for novel objects do not make good use of the target object's 3D shape information since they focus on generalization by processing the shape indirectly, making them less effective. We present GenFlow, an approach that enables both accuracy and generalization to novel objects with the guidance of the target object's shape. Our method predicts optical flow between the rendered image and the observed image and refines the 6D pose iteratively. It boosts the performance by a constraint of the 3D shape and the generalizable geometric knowledge learned from an end-to-end differentiable system. We further improve our model by designing a cascade network architecture to exploit the multi-scale correlations and coarse-to-fine refinement. GenFlow ranked first on the unseen object pose estimation benchmarks in both the RGB and RGB-D cases. It also achieves performance competitive with existing state-of-the-art methods for the seen object pose estimation without any fine-tuning.

In human body pose estimation, manifold learning is a useful method for reducing the dimension of 2D images and 3D body configuration data. Most commonly, body pose is estimated from silhouettes derived from images or image sequences. A major problem when applying manifold estimation, however, is its vulnerability to silhouette variation. In this paper, we propose a novel approach to solving viewpoint-induced silhouette variation by introducing biased label distances for learning manifolds that are able to represent variations in viewpoint, pose, and 3D body configuration. We demonstrate the effectiveness of the approach on a synthetic and a real-world dataset.

In human body pose estimation, manifold learning is a popular technique for reducing the dimension of 2D images and 3D body configuration data. This technique, however, is especially vulnerable to silhouette variation such as caused by viewpoint changes. In this paper, we propose a novel approach that combines three separate manifolds for representing variations in viewpoint, pose and 3D body configuration. We use biased manifold learning to learn these manifolds with appropriately weighted distances. A set of four mapping functions are then learned by a generalized regression neural network for added robustness. Despite using only three manifolds, we show that this method can reliably estimate 3D body poses from 2D images with all learned viewpoints.