Interactive segmentation task (IS) aims at taking into account the influence of user preferences on the basis of general semantic segmentation in order to obtain the specific target-of-interest. Given the fact that most of the related algorithms generate a single mask only, the robustness of which might be constrained due to the diversity of user intention in the early interaction stage, namely the vague selection of object part/whole object/adherent object, especially when there's only one click. To handle this, we propose a novel framework called Diversified Interactive Segmentation Network (DISNet) in which we revisit the peculiarity of first-click: given an input image, DISNet outputs multiple candidate masks under the guidance of first-click only, then a Dual-attentional Mask Correction (DAMC) module is utilized to measure the complex mutual effect within first-click, all-clicks and image features. Moreover, we design a new sampling strategy to generate GT masks with rich semantic relations. Performance analysis plus adequate ablation studies has demonstrated the efficacy of our methods, which further exemplifies the decisive role of first-click in the realm of IS.

In deep learning-based interactive segmentation, one of the key issues is how to effectively incorporate label prior into the network. The sparsity of clicks and the difference between images and label information limit the compatibility of user intent embedding in the deep long-path network. This paper aims to explore more effective embedding strategy of label prior in the neural network, and proposes a multi-branch U-net architecture for the interactive segmentation task. A principled fusion manner is adopted to progressively enhance the semantics of features based on the layer-by-layer promotion of the label prior guidance. Vast experiments on the popular GrabCut, Berkeley, DAVIS and SBD datasets verified the effectiveness of the proposed method. Our source code is available at:https://github.com/wbbbli/MBU-Net

Existing deep learning-based interactive image segmentation methods have significantly reduced the user's interaction burden with simple click interactions. However, they still require excessive numbers of clicks to continuously correct the segmentation for satisfactory results. This article explores how to harvest accurate segmentation of interested targets while minimizing the user interaction cost. To achieve the above goal, we propose a one-click-based interactive segmentation approach in this work. For this particularly challenging problem in the interactive segmentation task, we build a top-down framework dividing the original problem into a one-click-based coarse localization followed by a fine segmentation. A two-stage interactive object localization network is first designed, which aims to completely enclose the target of interest based on the supervision of object integrity (OI). Click centrality (CC) is also utilized to overcome the overlapping problem between objects. This coarse localization helps to reduce the search space and increase the focus of the click at a higher resolution. A principled multilayer segmentation network is then designed by a progressive layer-by-layer structure, which aims to accurately perceive the target with extremely limited prior guidance. A diffusion module is also designed to enhance the information flow between layers. Besides, the proposed model can be naturally extended to multiobject segmentation task. Our method achieves the state-of-the-art performance under one-click interaction on several benchmarks.

In interactive image segmentation methods, users can participate in and influence the segmentation process through their interactions, such as scribbles or bounding boxes. Similarly, the process of deep interactive segmentation utilizes users’ interactions to guide the network to learn the target of interest. This article mainly considers mouse clicking, which is the simplest interaction mode. Then, how to effectively characterize the click interaction (we call this “click encoding”) and fuse the click-related information with the network are the key issues in a deep interactive segmentation framework. However, the current click encoding method concentrates only on the spatial information of the clicks, so the region affected by each click is difficult to control, and the stability of the network is therefore reduced. Therefore, we propose a feature-interactive map that builds a close relationship between interaction information and target semantics. The affected region of the feature interactive map is determined by semantic information. Furthermore, we introduce an interactive nonlocal block by embedding a feature-interactive map into a nonlocal block, so that the long-range dependencies of the interaction information can be captured. Finally, based on the early fusion strategy, the features of the interactive nonlocal block are fused with the high-level features, thus amplifying the impacts of position and semantics on the final prediction results. Comprehensive experiments demonstrate that our click embedding approach significantly boosts the efficiency of the network and achieves state-of-the-art performance.

Recently, hashing has been successfully applied for large-scale remote sensing image retrieval (LSRSIR) due to its advantage in terms of computation and storage. In LSRSIR, existing hashing methods mainly focus on single-source remotely sensed data. They cannot effectively fuse multisource remotely sensed data, which has a large potential for LSRSIR. To fulfill this gap, this letter proposes a novel deep hashing method, dubbed Multiview Graph Convolutional Hashing (MGCH) that can successfully fuse multisource remote sensing image. Since graph convolutional network (GCN) has been applied as an effective means that expresses and integrates relationships into features, MGCH applies a GCN to explore inherent structural similarity among multiview data, which will help to generate discriminative hash codes. An asymmetric scheme is developed that optimizes the proposed deep model in an end-to-end manner to improve training efficiency. We evaluate the proposed method by fusing two different kinds of RS images, i.e., multispectral (MUL) image and panchromatic (PAN) image. The experimental results on the dual-source RS image data set (DSRSID) show that the proposed MGCH outperforms state-of-the-art multiview hashing methods.

Significant attention to multiple kernel graph-based clustering (MKGC) has emerged in recent years, primarily due to the superiority of multiple kernel learning (MKL) and the outstanding performance of graph-based clustering. However, many existing MKGC methods design a fat model that poses challenges for computational cost and clustering performance, as they learn both an affinity graph and an extra consensus kernel cumbersomely. To tackle this challenging problem, this article proposes a new MKGC method to learn a consensus affinity graph directly. By using the self-expressiveness graph learning and an adaptive local structure learning term, the local manifold structure of the data in kernel space is preserved for learning multiple candidate affinity graphs from a kernel pool first. After that, these candidate affinity graphs are synthesized to learn a consensus affinity graph via a thin autoweighted fusion model, in which a self-tuned Laplacian rank constraint and a top- k neighbors sparse strategy are introduced to improve the quality of the consensus affinity graph for accurate clustering purposes. The experimental results on ten benchmark datasets and two synthetic datasets show that the proposed method consistently and significantly outperforms the state-of-the-art methods.

This article explores the relationships between image element pairs and label pairs and extends label diffusion to label pair diffusion for the interactive image segmentation task. Compared with label diffusion, more accurate relationships between unlabeled and labeled data can be captured on a tensor product graph (TPG) by using higher order information, and more complex interactions among image elements and finer relationships between image element pairs and label pairs are explored in label pair diffusion (LPD) process. We first establish a prior label estimation framework to measure the label pair prior probability. Then, a probability learning process on TPG is designed to smooth the label prior. The learning process is equivalent to an iterative LPD process on the original graph, which makes the proposed algorithm maintain computational efficiency. Finally, the unary label probabilities can be obtained by a total-probability-theorem-based conversion from the binary relationships. Experiments on popular segmentation data sets demonstrate the superior performance of the proposed method.

This paper presents an interactive image segmen-tation algorithm, in which the segmentation problem is formulated as a global manifold learning process. Based on the principle that the label of each element depends on the influence of all the elements in the image, we extend the conventional local neighborhood or the long range regional relationships to the global relationships over the whole image. A probabilistic framework is established to measure the global effect of each element on all other elements. Based on two different manifold learning styles, the semi-global manifold learning (SGML) and the fully-global manifold learning (FGML) algorithms are proposed to capture the global geometry structure of the data manifold. SGML learns the intrinsic effects of each element separately, equivalent to a matrix diffusion process on an affinity graph. FGML learns the intrinsic effects of all elements together, equivalent to a label pair diffusion process on a higher-order tensor product graph. The global manifold learning helps to overcome the low contrast, weak boundary and texture problems. Extensive experiments on three public interactive segmentation datasets demonstrate the superior performance of the proposed algorithms both in accuracy and efficiency.

Multi-view hashing efficiently integrates multi-view data for learning compact hash codes, and achieves impressive large-scale retrieval performance. In real-world applications, multi-view data are often stored or collected in different locations, where hash code learning is more challenging yet less studied. To fulfill this gap, this paper proposes a novel supervised multi-view distributed hashing (SMvDisH) for hash code learning from multi-view data in a distributed manner. SMvDisH yields the discriminative latent hash codes by joint learning of latent factor model and classifier. With local consistency assumption among neighbor nodes, the distributed learning problem is divided into a set of decentralized sub-problems. The sub-problems can be solved in parallel, and the computational and communication costs are low. Experimental results on three large-scale image datasets demonstrate that SMvDisH achieves competitive retrieval performance and trains faster than state-of-the-art multi-view hashing methods.

Fully polarimetric synthetic aperture radar (PolSAR) images are generally expressed as the complex-valued (CV) matrix, whereas the convolutional neural networks (CNNs) have been successfully utilized for the PolSAR image classification. However, most 2D or 3D CNNs suffer from insufficient exploring the CV features or computationally expensive. To address these issues, this paper presents an efficient complex-valued spatial-scattering separated attention network (CVS3ANet) for PolSAR image classification. The CVS3ANet utilizes CV-3D convolutions to explore the features in both spatial and scattering dimensions of the PolSAR images, and reduces the parameters by factorizing the 3D convolution as a sequential process of 2D spatial convolution followed by 1D scattering convolution. Moreover, a squeeze and fusion attention unit is used to enhance the learning interpretation ability of the network by modeling correlations between channels with respect to attention probability. The experimental results demonstrate that the proposed method can obtain superior results over the state-of-the-art techniques.

This paper presents an interactive image segmentation approach in which we formulate segmentation as a probabilistic estimation problem based on the prior user intention. Instead of directly measuring the relationship between pixels and labels, we first estimate the distances between pixel pairs and label pairs using a probabilistic framework. Then, binary probabilities with label pairs are naturally converted to unary probabilities with labels. The higher order relationship helps improve the robustness to user inputs. To improve segmentation accuracy, a likelihood learning framework is proposed to fuse the region and the boundary information of the image by imposing a smoothing constraint on the unary potentials. Furthermore, we establish an equivalence relationship between likelihood learning and likelihood diffusion and propose an iterative diffusion-based optimization strategy to maintain computational efficiency. Experiments on the Berkeley segmentation data set and Microsoft GrabCut database demonstrate that the proposed method can obtain better performance than the state-of-the-art methods.

This paper presents an interactive image segmentation approach where the segmentation problem is formulated as a probabilistic estimation manner. Instead of measuring the distances between unseeded pixels and seeded pixels, we measure the similarities between pixel pairs and seed pairs to improve the robustness to the seeds. The unary prior probability of each pixel belonging to the foreground F and background B can be effectively estimated based on the similarities with label pairs (F, F),(F, B),(B, F) and (B, B). Then a likelihood learning framework is proposed to fuse the region and boundary information of the image by imposing the smoothing constraint on the unary potentials. Experiments on challenging data sets demonstrate that the proposed method can obtain better performance than state-of-the-art methods.

The combination of pixel and superpixel has been widely utilized in the interactive segmentation methods to overcome the sensitivity to the seeds' quantity and quality. However, because of the introduction of more variables and variables' interactions, the pixel-superpixel combination methods are still limited to the segmentation accuracy and computational complexity. To solve these problems, in this paper, we propose an interactive multilabel image segmentation method. In the proposed segmentation model, the multilayer relationships among the pixel layer, superpixel layer, and label layer are fused by the Markov random field framework to further improve the segmentation accuracy. During the optimization stage, the parallel partial optimality strategy is utilized to effectively solve the multilabel submodular energy function. Experimental results on challenging data sets demonstrate the competitiveness of the proposed method comparing with several state-of-the-art interactive algorithms.