Depth provides complementary information for salient object detection (SOD). However, the performance of RGB-D SOD methods is usually hindered by low quality depth map, semantic gap cross-modality and intrinsic gap between multi-level features. Although recent RGB-D SOD methods have been embedded into depth quality assessment, these methods do not consider the inconsistency of the depth format across datasets. In this paper, we propose an interpretable and effective mechanism called interference degree (ID) to assess depth quality and reweight the contribution of single-modality features without extra annotation. Then, a cross-modality interaction block (CMIB) is designed to reduce the semantic gap between RGB and depth features with the help of ID mechanism, and a mutually guided cross-level fusion (MGCF) module is designed to reduce the intrinsic gap among multi-level features. Finally, a refinement branch is proposed to enhance the salient regions and suppress the non-salient regions of fused features. Extensive experiments on six benchmark datasets show that the proposed depth-induced gap-reducing network (DIGR-Net) outperforms 20 recent state-of-the-art methods.

Salient instance segmentation (SIS) can be considered as the next generation task for the saliency detection community. Most of the existing state-of-the-art methods used for this novel challenging task are built on the mainstream Mask R-CNN architecture. However, this mechanism relies heavily on hand-designed anchors and NMS post-processing. In this paper, we provide a one stage SIS framework with transformers, termed Orientative Query Transformer (OQTR). To leverage the long-range dependencies of transformers, a cross fusion module is designed to efficiently fuse the global features in the encoder and salient query features for salient mask prediction. Furthermore, derived from the center prior in traditional saliency models, we propose an orientative query that is considered as the initial salient object query to accelerate convergence. In addition, to mitigate the issue of the lack of a large-scale dataset with salient instance labels, we collect a new SIS dataset (SIS10 K) containing over 10 K images elaborately annotated with both object- and instance-level labels to promote the community. Without any post-processing, our end-to-end OQTR framework significantly surpasses the top-1 RDPNet by an average of 13.1% AP scores across all three challenging datasets, demonstrating the strong performance of the proposed OQTR. The code and the dataset proposed in this work are available at:https://github.com/ssecv/OQTR.

Superpixel segmentation aims at dividing the input image into some representative regions containing pixels with similar and consistent intrinsic properties, without any prior knowledge about the shape and size of each superpixel. In this article, to alleviate the limitation of superpixel segmentation applied in practical industrial tasks that detailed boundaries are difficult to be kept, we regard each representative region with independent semantic information as a subspace, and correspondingly formulate superpixel segmentation as a subspace clustering problem to preserve more detailed content boundaries. We show that a simple integration of superpixel segmentation with the conventional subspace clustering does not effectively work due to the spatial correlation of the pixels within a superpixel, which may lead to boundary confusion and segmentation error when the correlation is ignored. Consequently, we devise a spatial regularization and propose a novel convex locality-constrained subspace clustering model that is able to constrain the spatial adjacent pixels with similar attributes to be clustered into a superpixel and generate the content-aware superpixels with more detailed boundaries. Finally, the proposed model is solved by an efficient alternating direction method of multipliers solver. Experiments on different standard datasets demonstrate that the proposed method achieves superior performance both quantitatively and qualitatively compared with some state-of-the-art methods.

Superpixel segmentation aims at grouping discretizing pixels into high-level correlative units and reducing the complexity of subsequent tasks, e.g., saliency detection and object tracking. Existing superpixel segmentation algorithms mainly focus on maintaining the geometrical information, while neglecting the irregular structure of superpixels. In this paper, a superpixel segmentation method is proposed to generate approximately structural superpixels with sharp boundary adherence and comprehensive semantic information. The superpixel segmentation is formulated as a square-wise asymmetric partition problem, where the semantic perceptual superpixels are recorded in a square level to preserve abundant semantic information and save storage simultaneously. Moreover, in order to achieve regular-shape superpixel units to better adhere to image boundaries and contours, a combinatorial optimization strategy is devised to achieve an optimal combination of squares and isolated pixels. Experimental comparisons with some state-of-the-art superpixel segmentation methods on the public benchmarks demonstrate the effectiveness of the proposed method quantitatively and qualitatively. In addition, we have applied the method to brain tissue segmentation to illustrate superior performance.

In this paper, a novel concept factorization (CF) method, called CF with adaptive neighbors (CFANs), is proposed. The idea of CFAN is to integrate an ANs regularization constraint into the CF decomposition. The goal of CFAN is to extract the representation space that maintains geometrical neighborhood structure of the data. Similar to the existing graph-regularized CF, CFAN builds a neighbor graph weights matrix. The key difference is that the CFAN performs dimensionality reduction and finds the neighbor graph weights matrix simultaneously. An efficient algorithm is also derived to solve the proposed problem. We apply the proposed method to the problem of document clustering on the 20 Newsgroups, Reuters-21578, and TDT2 document data sets. Our experiments demonstrate the effectiveness of the method.

In this paper, we propose a novel graph-based semisupervised learning framework, called joint sparse representation and embedding propagation learning (JSREPL). The idea of JSREPL is to join EPL with sparse representation to perform label propagation. Like most of graph-based semisupervised propagation learning algorithms, JSREPL also constructs weights graph matrix from given data. Different from classical approaches which build weights graph matrix and estimate the labels of unlabeled data in sequence, JSREPL simultaneously builds weights graph matrix and estimates the labels of unlabeled data. We also propose an efficient algorithm to solve the proposed problem. The proposed method is applied to the problem of semisupervised image clustering using the ORL, Yale, PIE, and YaleB data sets. Our experiments demonstrate the effectiveness of our proposed algorithm.

The binary feature descriptors such as binary robust independent elementary features (BRIEF), oriented rotated binary robust independent elementary features (ORB), and fast retina keypoint (FREAK) usually perform binarisation on the intensity comparisons, thus they lose some useful information. In this study, the authors propose an effective binary image descriptor which is called significant bit‐planes‐based local binary pattern for visual recognition. First, the authors divide an image into several sub regions according to the intensity orders to incorporate the spatial information. Then the authors extract the higher bit planes for all the sub regions and sort the adjacent neighbour bits based on the corresponding intensity orders, which make the descriptor invariant to rotation. In order to further improve the discriminative ability, the authors sample the multi‐scale neighbours and average the adjacent pixels and extract the feature descriptor from the higher bit planes. Since the authors directly perform operation on the significant bit planes without quantisation, the authors decrease the information loss to some extent. The descriptor has demonstrated a better performance over the state‐of‐the‐art binary descriptors as well as scale invariant feature transform on two recognition benchmarks (i.e. Kentucky and ETHZ) and PASCAL 2007 for image classification.

This letter proposes a bottom-up saliency model to predict eye fixation locations. Unlike traditional models that measure saliency by computing local or global distinctness, the proposed model considers saliency as the prediction error, because we believe that image patches or pixels with higher prediction error are more salient than others. The prediction error consists of both mispredicted error and unpredicted error. We propose a new algorithm called sparse residual to compute the mispredicted error. We then adopt outlier detection to compute the unpredicted error. Finally, we obtain the saliency map from merging the two results together via a guided filter. Extensive experiments on three benchmark databases show that our model is superior to 12 state-of-the-art models.

In this paper, a novel label propagation (LP) method is presented, called the manifold adaptive label propagation (MALP) method, which is to extend original LP by integrating sparse representation constraint into regularization framework of LP method. Similar to most LP, first of all, MALP also finds graph edges from given data and gives weights to the graph edges. Our goal is to find graph weights matrix adaptively. The key advantage of our approach is that MALP simultaneously finds graph weights matrix and predicts the label of unlabeled data. This paper also derives efficient algorithm to solve the proposed problem. Extensions of our MALP in kernel space and robust version are presented. The proposed method has been applied to the problem of semi-supervised face clustering using the well-known ORL, Yale, extended YaleB, and PIE datasets. Our experimental evaluations show the effectiveness of our method.

In this paper, a novel projective nonnegative matrix factorization (PNMF) method for enhancing the clustering performance is presented, called automated graph regularized projective nonnegative matrix factorization (AGPNMF). The idea of AGPNMF is to extend the original PNMF by incorporating the automated graph regularized constraint into the PNMF decomposition. The key advantage of this approach is that AGPNMF simultaneously finds graph weights matrix and dimensionality reduction of data. AGPNMF seeks to extract the data representation space that preserves the local geometry structure. This character makes AGPNMF more intuitive and more powerful than the original method for clustering tasks. The kernel trick is used to extend AGPNMF model related to the input space by some nonlinear map. The proposed method has been applied to the problem of document clustering using the well-known Reuters-21578, TDT2, and SECTOR data sets. Our experimental evaluations show that the proposed method enhances the performance of PNMF for document clustering.

A method of representing a binary image is developed. This method, called linear binary tree (LBT), is more effective than the linear quadtree (LQT) method. A LBT can be represented by encoding each black node with a binary integer whose digits reflect successive one-half subdivisions. The relation between LBT and LQT is discussed, and the space complexity and time complexity of some algorithms on LBT are compared to those of LQT. Methods for encoding LBT and for finding adjacent nodes for search, union and intersection operations are presented. The paper also shows that the space-efficiency of LBT is superior to that of LQT.>