In this paper, we propose a self-training based method for semi-supervised semantic segmentation. Our method utilizes k perturbed images of each unlabeled image to generate a new mask through the proposed vote operation which in turn is used as a supervision signal to train the model. The k predicted masks of perturbed images can provide nontrivial knowledge that is not captured by a single prediction and the proposed vote operation enables the model to output a low entropy prediction. Consistency regularization is applied between generated mask and k masks in terms of MSE loss in our network. Extensive experiments are conducted on different datasets to evaluate the effectiveness of our method. We show that the proposed method surpasses previous state-of the-art semi-supervised methods on ISIC 2017 dataset and achieves competitive performance on PASCAL VOC 2012 dataset.

An instance with a bad mask might make a composite image that uses it look fake. This encourages us to learn segmentation by generating realistic composite images. To achieve this, we propose a novel framework that exploits a new proposed prior called the independence prior based on Generative Adversarial Networks (GANs). The generator produces an image with multiple category-specific instance providers, a layout module and a composition module. Firstly, each provider independently outputs a category-specific instance image with a soft mask. Then the provided instances’ poses are corrected by the layout module. Lastly, the composition module combines these instances into a final image. Training with adversarial loss and penalty for mask area, each provider learns a mask that is as small as possible but enough to cover a complete category-specific instance. Weakly supervised semantic segmentation methods widely use grouping cues modeling the association between image parts, which are either artificially designed or learned with costly segmentation labels or only modeled on local pairs. Unlike them, our method automatically models the dependence between any parts and learns instance segmentation. We apply our framework in two cases: (1) Foreground segmentation on category-specific images with box-level annotation. (2) Unsupervised learning of instance appearances and masks with only one image of homogeneous object cluster (HOC). We get appealing results in both tasks, which shows the independence prior is useful for instance segmentation and it is possible to unsupervisedly learn instance masks with only one image.

This paper presents a novel scheme to automatically and directly detect smoking events in video. In this scheme, a color-based ratio histogram analysis is introduced to extract the visual clues from appearance interactions between lighted cigarette and its human holder. The techniques of color re-projection and Gaussian Mixture Models (GMMs) enable the tasks of cigarette segmentation and tracking over the background pixels. Then, a key problem for event analysis is the non-regular form of smoking events. Thus, we propose a self-determined mechanism to analyze this suspicious event using HHM framework. Due to the uncertainties of cigarette size and color, there is no automatic system which can well analyze human smoking events directly from videos. The proposed scheme is compatible to detect the smoking events of uncertain actions with various cigarette sizes, colors, and shapes, and has capacity to extend visual analysis to human events of similar interaction relationship. Experimental results show the effectiveness and real-time performances of our scheme in smoking event analysis.

This paper presents a novel template matching method to efficiently match and search image patterns. The method using concentric sampling structures, boosting process, and coarse-to-fine framework, differs from the traditional pattern matching schemes of time-exhausting correlation. The time complexity at searching stage is invariant to the dimension of concerned patterns. The rotation-invariant collection of concentric sub-samples represents as a reliable relaxation process of weak beliefs to efficiently reject the impossible location candidates. The concentric sampling approximation of integral images and the hierarchical scheme enable sifting out the patterns to process with the reduced complexity. Experimental result demonstrates the real-time performance on efficient pattern detection and geometry parameter estimation and the flexibility (on translation-, scaling-, and rotation-variant patterns) for various image analysis applications.