Mask-guided matting networks have achieved significant improvements and have shown great potential in practical applications in recent years. However, simply learning matting representation from synthetic and lack-of-real-world-diversity matting data, these approaches tend to overfit low-level details in wrong regions, lack generalization to objects with complex structures and real-world scenes such as shadows, as well as suffer from interference of background lines or textures. To address these challenges, in this paper, we propose a novel auxiliary learning framework for mask-guided matting models, incorporating three auxiliary tasks: semantic segmentation, edge detection, and background line detection besides matting, to learn different and effective auxiliary representations from different types of data and annotations. Our framework and model introduce the following key aspects: 1) to learn real-world adaptive semantic representation for objects with diverse and complex structures under real-world scenes, we introduce extra semantic segmentation and edge detection tasks on more diverse real-world data with segmentation annotations; 2) to avoid overfitting on low-level details, we propose a module to utilize the inconsistency between learned segmentation and matting representations to regularize detail refinement; 3) we propose a novel background line detection task into our auxiliary learning framework, to suppress interference of background lines or textures. In addition, we propose a high-quality matting benchmark, Plant-Mat, to evaluate matting methods on complex structures. Extensively quantitative and qualitative results show that our approach outperforms state-of-the-art mask-guided methods.

Maintaining content consistency is a critical point in unpaired image-to-image translation. Contrastive learning is an essential framework in unpaired image-to-image translation to maximize the mutual information between corresponding patches of input and generated images. However, ignoring the importance of background information, the previous CL-based methods suffer from the problems that the background of generated images, as well as the regions where the object and background intersect, appear to be blurred. To tackle these problems, we propose cluster-guided contrastive learning in the I2I task. We leverage the cluster information of features to mine hard negative samples and distinguish the concept and the detail without additional parameters. We further provide rigorous proof of mutual information of our proposed method, which proves that our method preserves the lower bound of previous work and has a lower variance. Our proposed method, CGCUT, achieves state-of-the-art performance on most metrics on three benchmark datasets.