Natural image matting is a widely used image processing technique that extracts foreground by predicting the alpha values of the unknown region based on the alpha values of the known foreground and background regions. However, existing image matting methods may not yield the most optimal results when applied to images containing transparent objects because the known foreground region is small or even absent. To address this shortcoming, in this paper, we propose a novel method named Transparent Object Matting using Predicted Definite Foreground and Background (TOM-PDFB), which can explore and utilize the definite foreground and background in the unknown region. For this purpose, a newly developed foreground-background confidence estimator is applied to predict the confidence level of the definite foreground and the definite background, thus providing the priors required for transparent object matting. Next, foreground-background guided progressive refinement network developed as a part of this work is adopted to incorporate the estimated definite foreground and background into the alpha matte refinement process. Extensive experimental results demonstrate that the TOM-PDFB outperforms state-of-the-art methods when applied to transparent objects. Project page:https://github.com/yihuiliang/TOM-PDFB.

High-resolution natural image matting plays an important role in image editing, film-making and remote sensing due to its ability of accurately extract the foreground from a natural background. However, due to the complexity brought about by the proliferation of resolution, the existing image matting methods cannot obtain high-quality alpha mattes on high-resolution images in reasonable time. To overcome this challenge, we introduce a high-resolution image matting framework based on alpha matte refinement from low-resolution to high-resolution (HRIMF-AMR). The proposed framework transforms the complex high-resolution image matting problem into low-resolution image matting problem and high-resolution alpha matte refinement problem. While the first problem is solved by adopting an existing image matting method, the latter is addressed by applying the Detail Difference Feature Extractor (DDFE) designed as a part of our work. The DDFE extracts detail difference features from high-resolution images by measuring the image feature difference between high-resolution images and low-resolution images. The low-resolution alpha matte is refined according to the extracted detail difference feature, providing the high-resolution alpha matte. In addition, the Matte Detail Resolution Difference (MDRD) loss is introduced to train the DDFE, which imposes an additional constraint on the extraction of detail difference features with mattes. Experimental results show that integrating HRIMF-AMR significantly enhances the performance of existing matting methods on high-resolution images of Transparent-460 and Alphamatting. Project page: https://github.com/yexianmin/HRAMR-Matting.

Abstract Trimap is required for most image matting algorithms, as it provides partial regions with known opacity. As capturing elaborate trimaps is a time‐consuming process, in practice, users prefer to provide coarse trimaps with large unknown regions. However, extant image matting algorithms cannot provide high‐quality alpha mattes based on coarse trimaps. Although some matting algorithms include trimap expansion in the pre‐processing stage, if this is done by directly comparing the similarity of image features between pixels, errors and omissions can easily occur. To overcome this issue, in this paper, a coarse trimap expansion model based on one‐class classification is presented, in which the problem is treated as a process of reclassifying pixels in unknown regions. For this purpose, a coarse trimap expansion method denoted as CTE‐OC is proposed, in which the similarity between pixels is reliably determined by measuring semantic features, allowing newly developed one‐class classifiers to adequately classify pixels in entire unknown regions. The validity of these strategies is tested experimentally, and the results show that CTE‐OC can significantly improve the quality of alpha mattes obtained by extant image matting methods when provided with coarse trimaps.

Image matting is evolving for a wide range of applications including image/video editing. Sampling-based image matting aims to estimate the opacity of foreground objects by properly selecting a pair of foreground and background pixels for every unknown pixel. Sampling-based image matting is essentially an uncertain multicriteria optimization problem (UMCOP). It shows unique advantages in parallelization and handling spatially disconnected regions. However, sampling-based approaches encounter difficulty in accurately evaluating pixel pairs and efficiently optimizing the large-scale UMCOP. To address these two problems, a fuzzy multicriteria evaluation (FMCE) and a multiobjective evolutionary algorithm based on multicriteria decomposition (MOEA-MCD) are proposed. We model three fuzzy membership functions for three selection criteria and aggregate them by Einstein and averaging operators providing FMCE for pixel pairs. MOEA-MCD uses the heuristic information for each criterion by multicriteria decomposition that divides the single objective into multiple objectives and optimizes them simultaneously using a multiobjective optimizer with neighborhood grouping strategy. Experimental results show that FMCE accurately evaluates pixel pairs even in uncertain cases with low satisfaction degree of some evaluation criteria, and the heuristic information for each criterion enhances the population diversity of MOEA-MCD. MOEA-MCD outperforms state-of-the-art large-scale optimization approaches and sampling-based image matting approaches.

In sampling-based matting methods, the alpha is estimated by choosing the best pair of foreground and background color samples. The lack of true samples is the major obstacle in obtaining high-quality alpha mattes. Regrettably, several proposed approaches did not address the conflicts among multiple sampling criteria and the effects of incomplete sample spaces. To address this issue, we propose a pixel-level discrete multiobjective sampling (PDMS) method. The color sampling process at each unknown pixel is formalized as a multiobjective optimization problem (MOP). The strength of PDMS includes its ability to minimize both color difference and spatial distance between unknown and known pixels, and its capacity to adaptively make trade-offs among conflicting sampling criteria. To mitigate the effects of incomplete sample spaces, the sample space is extended to complete known regions in PDMS, which means that the colors of all known pixels can be sampled, instead of mean colors of superpixels. Our experimental results show that PDMS collects a small set of samples while achieving smaller minimum absolute difference in alpha estimation. Moreover, PDMS implements pixel-level sampling by using the proposed multiobjective optimization algorithm to efficiently solve sampling MOPs. The PDMS-based matting method provides high-quality alpha mattes with sharp boundaries and thus outperforms those prior image matting methods in terms of gradient error.