This paper proposes an edge-based superpixel similarity measurement, which globally evaluates the similarity between superpixels by binary edge maps. The basic idea is to assess whether the superpixels are surrounded by the same edges. To this end, we first describe the edge spatial distributions by directional regions and then use the directional regions to represent the surrounding relationships of superpixels and edges by their traverse relationships, which form the histogram feature. Finally, the similarity is simply calculated by the distances between the features. To verify the proposed similarity measurement, we use our global similarity measurement to perform superpixel clustering. Two clustering methods, the directed graph clustering (DGC) and spectral clustering (ultrametric contour map) are combined to achieve the clustering process. The combination of our global similarity measurement and DGC to form a new three-layer-based superpixel generation method, which can quickly generate the superpixel from edge maps, is highlighted. We verify the global similarity measurement by the BSDS500 dataset. The experimental results demonstrate that the proposed global similarity measurement can improve the clustering accuracy in terms of larger intersection-over-union-criterion-based values. The code can be downloaded from https://github.com/FanmanMeng/Superpixel-Similarity-Measurement.

Fine-grained categorization is a challenging task mainly due to two factors: first, objects share similar appearances between different categories; second, objects present significant pose variation within the same category. To address these challenges, we propose a method to automatically detect discriminative and pose-invariant regions, which is referred to as a polygon-based classifier (PBC). In the first stage, we generate a set of polygons that are composed of multiple parts. For each polygon, a classifier is trained based on deep features of a convolutional network. Then, a greedy algorithm is employed to select the discriminative and complementary polygon-based classifiers that deliver highest classification accuracy for fine-grained object categories. In the second stage, the confusing classes of the first stage are selected and employed to train the polygon-based classifiers. Then, a greedy algorithm is employed to select discriminative classifiers. For the test images, we use the classifiers trained in the first stage to obtain a coarse result. Then, the classifiers of the second stage are adopted to distinguish the confusing classes of the coarse result. In our experiments, the proposed approach is evaluated on three well-known fine-grained datasets. The experiments show that our approach outperforms the state-of-the-art methods.

In this paper, we propose a video object segmentation method via global consistency aware query strategy. The aim is to obtain higher segmentation accuracy with less user annotation. Intuitively, we hope to annotate some frames to obtain better segmentation performance than to annotate other frames, which can be modeled by active learning framework. Specifically, we first generate a sample space of potential annotation regions via an object proposals method for each frame. Then, the annotation likelihood for the region is calculated in terms of annotation history and global consistency for the object in the video. Third, the segmentation result of the annotation region can be obtained by minimizing an MRF energy function. Fourth, the algorithm will provide the user with the most valuable frame to annotate, which has high annotation likelihood and large segmentation result change. Finally, the annotation is added to the framework to begin the next iteration. Experiments on a number of video sequences demonstrate that the proposed method can reduce the user effort and obtain the higher segmentation accuracy compared with the state-of-the-art methods.

This paper proposes a deep convolutional network for quality evaluation of object segmentation. In this work, we first present two large-scale datasets for object quality evaluation. The segmentation results of the proposed datasets are generated from simple and complex segmentation datasets respectively. Then, we propose three types of deep convolutional networks to learn object segmentation quality. The weighted mask layer is proposed to utilize both the information of foreground and background. One advantage of our network is that the segmentation quality is evaluated without using the groundtruth segmentation results during test period. This may help us obtain better segmentation result based on the predicted segmentation quality score. The promising results on the proposed datasets indicate the efficiency of our network.

Segment propagation transfers object priors among images, which is an important prior generation manner in image segmentation. The existing propagation methods focus on object foreground propagation, while the detailed part propagation is deficiency, which is caused by the challenges that not only the multiple part regions, but also their relationships need to be transferred. In this paper, a part propagation method is proposed. Two level propagations such as object level propagation, and part level propagation are successively used for the part propagation. The object level propagation is to transfer global shape information among images, which is formulated as graph matching based edge fragments matching problem, with dynamic programming solution. The part level propagation is to transfer the more detailed part labels, which is formulated as pixel level structure matching problem, and is efficiently solved by traditional dense pixel matching methods. The proposed method is verified on 15 challenging classes selected from PASCAL 2010 dataset, Bird dataset and Cat-Dog dataset. The experimental results demonstrate the effectiveness of the proposed method.

In this paper, we propose an efficient blind image quality assessment (BIQA) algorithm, which is characterized by a new feature fusion scheme and a k-nearest-neighbor (KNN)-based quality prediction model. Our goal is to predict the perceptual quality of an image without any prior information of its reference image and distortion type. Since the reference image is inaccessible in many applications, the BIQA is quite desirable in this context. In our method, a new feature fusion scheme is first introduced by combining an image's statistical information from multiple domains (i.e., discrete cosine transform, wavelet, and spatial domains) and multiple color channels (i.e., Y, Cb, and Cr). Then, the predicted image quality is generated from a nonparametric model, which is referred to as the label transfer (LT). Based on the assumption that similar images share similar perceptual qualities, we implement the LT with an image retrieval procedure, where a query image's KNNs are searched for from some annotated images. The weighted average of the KNN labels (e.g., difference mean opinion score or mean opinion score) is used as the predicted quality score. The proposed method is straightforward and computationally appealing. Experimental results on three publicly available databases (i.e., LIVE II, TID2008, and CSIQ) show that the proposed method is highly consistent with human perception and outperforms many representative BIQA metrics.

Most existing video segmentation methods are focused on extracting the primary objects in test video sequences. They assumed that only one object appeared through the whole video sequences, which is impractical in many applications. In this paper, we focus on the object segmentation from the long video sequences which consist of many different scenes, shot cuts and various motion patterns, etc. In order to solve this problem, we propose a framework to segment the objects in relative video shots, while discarding the irrelative video shots. A graph is constructed to model the video object detection and final segmentation is obtained by getting the superpixels in the detection boxes. We also introduce a new long video segmentation dataset which corresponds to the pixel-wise ground truth. The experiments demonstrate that our proposed method can deal with the object segmentation in long video sequence.

This paper proposes a new constrained directed graph clustering (DGC) method and segmentation propagation method for the multiple foreground cosegmentation. We solve the multiple object cosegmentation with the perspective of classification and propagation, where the classification is used to obtain the object prior of each class and the propagation is used to propagate the prior to all images. In our method, the DGC method is designed for the classification step, which adds clustering constraints in cosegmentation to prevent the clustering of the noise data. A new clustering criterion such as the strongly connected component search on the graph is introduced. Moreover, a linear time strongly connected component search algorithm is proposed for the fast clustering performance. Then, we extract the object priors from the clusters, and propagate these priors to all the images to obtain the foreground maps, which are used to achieve the final multiple objects extraction. We verify our method on both the cosegmentation and clustering tasks. The experimental results show that the proposed method can achieve larger accuracy compared with both the existing cosegmentation methods and clustering methods.

In this paper, we propose a framework to discover and segment favorite object from the natural images. The main idea is to first generate the shape based common template of the favorite object using the images collected from the web. Then, the common template is used to extract the favorite object from the original images. In the common template generation, co-segmentation is used to provide the initial segments. The median graph theory is employed to construct the common template. We also propose a new shape descriptor namely directional shape representation to handle shape variations. We test our method on the images collected from image datasets and web. Experimental results demonstrate the effectiveness of the proposed method.

In this paper, we develop a new algorithm to segment multiple common objects from a group of images. Our method consists of two aspects: directed graph clustering and prior propagation. The clustering is used to cluster the local regions of the original images and generate the foreground priors from these clusterings. The second step propagates the prior of each class and locates the common objects from the images in terms of foreground map. Finally, we use the foreground map as the unary term of Markov random field segmentation and segment the common objects by graph-cuts algorithm. We test our method on FlickrMFC and ICoseg datasets. The experimental results show that the proposed method can achieve larger accuracy compared with several state-of-arts co-segmentation methods.

In this paper, we present a robust texture representation by exploring an ensemble of binary codes. The proposed method, called Locally Enhanced Binary Coding (LEBC), is training-free and needs no costly data-to-cluster assignments. Given an input image, a set of features that describe different pixel-wise properties, is first extracted so as to be robust to rotation and illumination changes. Then, these features are binarized and jointly encoded into specific pixel labels. Meanwhile, the Local Binary Pattern (LBP) operator is utilized to encode the neighboring relationship. Finally, based on the statistics of these pixel labels and LBP labels, a joint histogram is built and used for texture representation. Extensive experiments have been conducted on the Outex, CUReT and UIUC texture databases. Impressive classification results have been achieved compared with state-of-the-art LBP-based and even learning-based algorithms.

In this paper, we exploit an algorithm for detecting the individual objects from multiple images in a weakly supervised manner. Specifically, we treat the object co-detection as a jointly dictionary learning and objects localization problem. Thus a novel low-rank and sparse representation dictionary learning algorithm is proposed. It aims to learn a compact and discriminative dictionary associated with the specific object category. Different from previous dictionary learning methods, the sparsity imposed on representation coefficients, the rank minimization of learned dictionary, data reconstruction error and the low-rank constraint of sample data are all incorporated in a unitized objective function. Then we optimize all the constraint terms via an extended version of augmented lagrange multipliers (ALM) method simultaneously. The experimental results demonstrate that the low-rank and sparse representation dictionary learning algorithm can compare favorably to other single object detection method.