In recent years, anchor and hash-based multi-view clustering methods have gained attention for their efficiency and simplicity in handling large-scale data. However, existing methods often overlook the interactions among multi-view data and higher-order cooperative relationships during projection, negatively impacting the quality of hash representation in low-dimensional spaces, clustering performance, and sensitivity to noise. To address this issue, we propose a novel approach named Tensor-Interacted Projection and Cooperative Hashing for Multi-View Clustering(TPCH). TPCH stacks multiple projection matrices into a tensor, taking into account the synergies and communications during the projection process. By capturing higher-order multi-view information through dual projection and Hamming space, TPCH employs an enhanced tensor nuclear norm to learn more compact and distinguishable hash representations, promoting communication within and between views. Experimental results demonstrate that this refined method significantly outperforms state-of-the-art methods in clustering on five large-scale multi-view datasets. Moreover, in terms of CPU time, TPCH achieves substantial acceleration compared to the most advanced current methods.

3D Gaussian Splatting (3DGS) has become one of the most promising 3D reconstruction technologies. However, label noise in real-world scenarios-such as moving objects, non-Lambertian surfaces, and shadows-often leads to reconstruction errors. Existing 3DGS-Bsed anti-noise reconstruction methods either fail to separate noise effectively or require scene-specific fine-tuning of hyperparameters, making them difficult to apply in practice. This paper re-examines the problem of anti-noise reconstruction from the perspective of epistemic uncertainty, proposing a novel framework, OCSplats. By combining key technologies such as hybrid noise assessment and observation-based cognitive correction, the accuracy of noise classification in areas with cognitive differences has been significantly improved. Moreover, to address the issue of varying noise proportions in different scenarios, we have designed a label noise classification pipeline based on dynamic anchor points. This pipeline enables OCSplats to be applied simultaneously to scenarios with vastly different noise proportions without adjusting parameters. Extensive experiments demonstrate that OCSplats always achieve leading reconstruction performance and precise label noise classification in scenes of different complexity levels.

Compared to static anchor selection, existing dynamic anchor learning could automatically learn more flexible anchors to improve the performance of large-scale multi-view clustering. Despite improving the flexibility of anchors, these methods do not pay sufficient attention to the alignment and fairness of learned anchors. Specifically, within each cluster, the positions and quantities of cross-view anchors may not align, or even anchor absence in some clusters, leading to severe anchor misalignment and imbalance issues. These issues result in inaccurate graph fusion and a reduction in clustering performance. Besides, in practical applications, missing information caused by sensor malfunctions or data losses could further exacerbate anchor misalignment and imbalance. To overcome such challenges, a novel Incomplete Multi-view Clustering withPaired and Balanced Dynamic Anchor Learning (PBDAL)is proposed to ensure the alignment and fairness of anchors. Unlike existing unsupervised anchor learning, we first design a paired and balanced dynamic anchor learning scheme to supervise dynamic anchors to be aligned and fair in each cluster. Meanwhile, we develop an enhanced bipartite graph tensor learning to refine paired and balanced anchors. Our superiority, effectiveness, and efficiency are all validated by performing extensive experiments on multiple public datasets.

Cross-modal hashing encodes different modalities of multimodal data into low-dimensional Hamming space for fast cross-modal retrieval. In multi-label cross-modal retrieval, multimodal data are often annotated with multiple labels, and some labels, e.g., "ocean" and "cloud," often co-occur. However, existing cross-modal hashing methods overlook label dependency that is crucial for improving performance. To fulfill this gap, this article proposes graph convolutional multi-label hashing (GCMLH) for effective multi-label cross-modal retrieval. Specifically, GCMLH first generates word embedding of each label and develops label encoder to learn highly correlated label embedding via graph convolutional network (GCN). In addition, GCMLH develops feature encoder for each modality, and feature fusion module to generate highly semantic feature via GCN. GCMLH uses teacher-student learning scheme to transfer knowledge from the teacher modules, i.e., label encoder and feature fusion module, to the student module, i.e., feature encoder, such that learned hash code can well exploit multi-label dependency and multimodal semantic structure. Extensive empirical results on several benchmarks demonstrate the superiority of the proposed method over existing state-of-the-arts.

A significant challenge facing current optical flow meth-ods is the difficulty in generalizing them well to the real world. This is mainly due to the lack of large-scale real-world datasets, and existing self-supervised methods are limited by indirect loss and occlusions, resulting in fuzzy outcomes. To address this challenge, we introduce a novel optical flow training framework: automatic data factory (ADF). ADF only requires RGB images as input to effectively train the optical flow network on the target data do-main. Specifically, we use advanced NeRF technology to reconstruct scenes from photo groups collected by a monoc-ular camera, and then calculate optical flow labels between camera pose pairs based on the rendering results. To elimi-nate erroneous labels caused by defects in the scene reconstructed by NeRF, we screened the generated labels from multiple aspects, such as optical flow matching accuracy, radiation field confidence, and depth consistency. The fil-tered labels can be directly used for network supervision. Experimentally, the generalization ability of ADF on KITTI surpasses existing self-supervised optical flow and monoc-ular scene flow algorithms. In addition, ADF achieves impressive results in real-world zero-point generalization evaluations and surpasses most supervised methods11Code: https://github.com/HanLingsgjk/UnifiedGeneralization.

Interactive segmentation task (IS) aims at taking into account the influence of user preferences on the basis of general semantic segmentation in order to obtain the specific target-of-interest. Given the fact that most of the related algorithms generate a single mask only, the robustness of which might be constrained due to the diversity of user intention in the early interaction stage, namely the vague selection of object part/whole object/adherent object, especially when there's only one click. To handle this, we propose a novel framework called Diversified Interactive Segmentation Network (DISNet) in which we revisit the peculiarity of first-click: given an input image, DISNet outputs multiple candidate masks under the guidance of first-click only, then a Dual-attentional Mask Correction (DAMC) module is utilized to measure the complex mutual effect within first-click, all-clicks and image features. Moreover, we design a new sampling strategy to generate GT masks with rich semantic relations. Performance analysis plus adequate ablation studies has demonstrated the efficacy of our methods, which further exemplifies the decisive role of first-click in the realm of IS.

Recently, tensor Schatten p-norm has achieved impressive performance for fast multi-view clustering [57]. This primarily ascribes the superiority of tensor Schatten p-norm in exploring high-order structure information among views. Whereas, 1) tensor Schatten p-norm treats different singular values equally, such that the larger singular values corresponding to certain significant feature information (i.e., prior information) have not been utilized fully; 2) tensor Schatten p-norm also ignore ranking the core entries of core tensor, which may contain noise information; 3) existing methods select fixed anchors or averagely update anchors to construct the neighbor bipartite graphs, greatly limiting the flexibility and expression of anchors. To break these limitations, we propose a novel Improved Weighted Tensor Schatten p-Norm for Fast Multi-view Graph Clustering (IWTSN-FMGC). Specifically, to eliminate the interference of the first two limitations, we propose an improved weighted tensor Schatten p-norm to dynamically rank core tensor and automatically shrink singular values. To this end, improved weighted tensor Schatten p-norm has the potential to more effectively leverage low-rank structures and prior information, thereby enhancing robustness compared to current tensor Schatten p-norm methods. Further, the designed adaptive neighbor bipartite graph learning can more flexibly and expressively encode the local manifold structure information than existing anchor selection and averaged anchor updating. Extensive experiments validate our effectiveness and superiority across multiple benchmark datasets.

The existing multikernel graph clustering (MKGC) methods have emerged with notable success on nonlinear clustering tasks since the graph learning can effectively capture graph structure similarity between sample pair. Ideally, a high-quality graph should enjoy the good block diagonal property, i.e. the intercluster similarities correspond to zeros, while intracluster similarities represent nonzeros. Meanwhile, the number of diagonal blocks for a graph equals to the number of clusters on the dataset. However, most of the existing MKGC methods design a corpulent three-parts graph leaning process that poses challenges for hyperparameter tuning, time cost, and clustering performance. To overcome these challenging issues, we propose an enforced block diagonal graph learning for multikernel clustering (EBDGL-MKC) method, where we pursue a high-quality block diagonal graph via well-designed one-part graph leaning scheme rather than three parts. Inspired by symmetric matrix factorization (SMF), we first design a one-part block diagonal graph learning scheme to learn multiple block diagonal graphs, by exploring an explicit theoretical connection between the clustering partition of kernel$k$-means and the excellent block diagonal graph. Then, these block diagonal graphs are stacked into a low-rank tensor for exploiting the high-order structure information hidden in the nonlinear data. After that, an effective alternate algorithm with convergence proof is performed on extensive experiments to demonstrate the superiority of EBDGL compared with the state-of-the-art multikernel clustering (MKC) methods.

With limited labeled samples, few-shot classification poses a challenge to standard deep models and has attracted a surge of concern. Metric learning based approaches stand out for the minimalist and efficient design, aiming to classify query samples using supervision of support sets in the metric space. Prototypical Network has pioneered the use of mean feature embeddings to represent each support class and leaned on the computed prototypes for query classification. However, inherent bias exists in the mean prototypes generating from scarce support samples versus the actual class prototypes, which induces subsequent inference deviation. In this paper, we propose to diminish the bias by leveraging the semantic information of query samples to guide prototype optimization. Specifically, we exploit the semantic correlation between the local of initial mean prototypes and the global of query samples to generate query-guided masks, thus tailoring optimized prototypes that vary by query samples. This exploration of correlation is first utilized to alleviate the prototype bias problem and shows great brevity compared to existing methods. Extensive experiments are conducted on three few-shot image classification benchmark datasets, and demonstrate the effectiveness of our proposed method.

For robust and effective tracking, most efforts strive to design a powerful representation target model, while we are inspired by the idea of“knowing oneself and knowing others”to major in both the target and non-target features. In this work, we propose a unit correlation with interactive feature tracker (UCIF), which utilizes feature interaction and independent correlation operation to improve robustness and effectiveness. Specifically, we first propose a feature integration network, in which the feature enhancement module concentrates on enhancing the tracker's representation ability for both target and non-target. The feature interaction module is in charge of completing the interactive learning between target and non-target features. Then, considering the potential risk of blurring spatial information in regular correlation operation, a unit correlation network is presented, where the convolution sampling strategy can integrate the target features as well as reduce the computation costs. The unit kernel for correlation operation can protect the target spatial information. The channel ranking module suppresses background interference via weight assignment. Extensive experiments are conducted on both the short-term and long-term challenging benchmarks, including OTB2015, NFS, UAV123, TrackingNet, GOT-10 k, TLP, LaSOT and VOT-LT2019. Our tracker achieves remarkable performance in robustness and effectiveness.

With the rapid advances in digital imaging and communication technologies, recently image set classification has attracted significant attention and has been widely used in many real-world scenarios. As an effective technology, the class-specific representation theory-based methods have demonstrated their superior performances. However, this type of methods either only uses one gallery set to measure the gallery-to-probe set distance or ignores the inner connection between different metrics, leading to the learned distance metric lacking robustness, and is sensitive to the size of image sets. In this article, we propose a novel joint metric learning-based class-specific representation framework (JMLC), which can jointly learn the related and unrelated metrics. By iteratively modeling probe set and related or unrelated gallery sets as affine hull, we reconstruct this hull sparsely or collaboratively over another image set. With the obtained representation coefficients, the combined metric between the query set and the gallery set can then be calculated. In addition, we also derive the kernel extension of JMLC and propose two new unrelated set constituting strategies. Specifically, kernelized JMLC (KJMLC) embeds the gallery sets and probe sets into the high-dimensional Hilbert space, and in the kernel space, the data become approximately linear separable. Extensive experiments on seven benchmark databases show the superiority of the proposed methods to the state-of-the-art image set classifiers.

Existing deep learning-based interactive image segmentation methods have significantly reduced the user's interaction burden with simple click interactions. However, they still require excessive numbers of clicks to continuously correct the segmentation for satisfactory results. This article explores how to harvest accurate segmentation of interested targets while minimizing the user interaction cost. To achieve the above goal, we propose a one-click-based interactive segmentation approach in this work. For this particularly challenging problem in the interactive segmentation task, we build a top-down framework dividing the original problem into a one-click-based coarse localization followed by a fine segmentation. A two-stage interactive object localization network is first designed, which aims to completely enclose the target of interest based on the supervision of object integrity (OI). Click centrality (CC) is also utilized to overcome the overlapping problem between objects. This coarse localization helps to reduce the search space and increase the focus of the click at a higher resolution. A principled multilayer segmentation network is then designed by a progressive layer-by-layer structure, which aims to accurately perceive the target with extremely limited prior guidance. A diffusion module is also designed to enhance the information flow between layers. Besides, the proposed model can be naturally extended to multiobject segmentation task. Our method achieves the state-of-the-art performance under one-click interaction on several benchmarks.

This paper address the problem of optical expansion (OE). OE describes the object scale change between two frames, widely used in monocular 3D vision tasks. Previous methods estimate optical expansion mainly from optical flow results, but this two-stage architecture makes their results limited by the accuracy of optical flow and less robust. To solve these problems, we propose the concept of 3D optical flow by integrating optical expansion into the 2D optical flow, which is implemented by a plug-and-play module, namely TPCV. TPCV implements matching features at the correct location and scale, thus allowing the simultaneous optimization of optical flow and optical expansion tasks. Experimentally, we apply TPCV to the RAFT optical flow baseline. Experimental results show that the baseline optical flow performance is substantially improved. Moreover, we apply the optical flow and optical expansion results to various dynamic 3D vision tasks, including motion-in-depth, time-to-collision, and scene flow, often achieving significant improvement over the prior SOTA. Code is available at https://github.com/HanLingsgjk/TPCV.

In practical scenarios, partial missing of multi-view data is very common, such as register information missing from social network analysis, which results in incomplete multi-view clustering (IMVC). How to fill missing data fast and efficiently plays a vital role in improving IMVC, carrying a significant challenge. Existing IMVC methods always use all observed data to fill in missing data, resulting in high complexity and poor imputation quality due to a lack of guidance from consistent distribution. To break the existing limitations, we propose a novel Distribution Consistency based Fast Anchor Imputation for Incomplete Multi-view Clustering (DCFAI-IMVC) method. Specifically, to eliminate the interference of redundant and fraudulent features in the original space, incomplete data are first projected into a consensus latent space, where we dynamically learn a small number of anchors to achieve fast and good imputation. Then, we employ global distribution information of the observed embedding representations to further ensure the consistent distribution between the learned anchors and the observed embedding representations. Ultimately, a tensor low-rank constraint is imposed on bipartite graphs to investigate the high-order correlations hidden in data. DCFAI-IMVC enjoys linear complexity in terms of sample number, which gives it great potential to handle large-scale IMVC tasks. By performing extensive experiments, our effectiveness, superiority, and efficiency are all validated on multiple public datasets with recent advances.

Adversarial attacks in visual object tracking aim to drop tracking performance through injecting imperceptible perturbations to the input of the tracker. Current methods usually superimpose perturbation maps on the input images, and advocate blinding the tracker via occluding the real targets to achieve attack effect. From the perspective of attraction, we alternatively propose a novel idea of attacking the tracker, which advocates using perturbation patches to act as fake targets to attract the tracker's attention. For this purpose, we establish a multi-conditional objective function to generate our ideal patch in an offline iterative manner. For invisibility, we integrate the constraint of patch value into the function for unified optimization. For universality, in addition to adopting large-scale and high-diversity training samples , we also incorporate the video-agnostic condition into this function. To make the patch attractive like a fake target, we elaborately design the non-overlapping area to determine the patch position, and generate matched fake labels to mislead the tracker to track the patch. In online attacking, it only needs to paste the optimized patch onto the video frames, the tracker will be successfully attracted by our patch, achieving attack effect. Extensive experimental results on 8 popular tracking datasets demonstrate that our method can obtain exceptional attack performance in both non-targeted and targeted attack. Additionally, the experiments on transferability illustrate our optimized patches can be directly applied to other trackers with different architectures.

Existing multi-kernel clustering (MKC) methods usually focus on constructing a fixed dimension consensus-partition from base kernels to demonstrate their superior in integrating complementary information. Despite their success, they still suffer from the following limitations: (1) The size of consensus-partition is always fixed as the upper bound ($k=n$) or lower bound ($k=c$), where$n$,$c$, and$k$are the number of samples, clusters, and partition dimension, respectively, resulting in suboptimal partition;(2)The learned consensus-partition cannot make full use of the global distribution information hidden in data. To address these issues, we propose a latent consensus-partition learning framework for MKC, namelyConsensus Cluster Center Guided Latent Multi-kernel Clustering(C3LMC), including two methods,i.e., C3LMCKand C3LMCH. For C3LMCK, we flexibly search for a more proper dimension of consensus-partition in a latent embedding space rather than the fixed partition dimension. Meanwhile, the generation of latent consensus-partition is guided by a consensus cluster center of base kernels, such that global distribution information hidden in base kernels can be captured fully. However, C3LMCKsuffers from$\boldsymbol {\mathcal {O}}(n^{2})$computational complexity and memory complexity. Thus, we further propose C3LMCHto handle large-scale data by reducing both kinds of complexities to$\boldsymbol {\mathcal {O}}(n)$. Two solvers with convergence proof are developed to validate our effectiveness, superiority, and efficiency on multiple public datasets with the recent advances.

Few-shot classification, which aims to recognize novel classes with the aid of learned knowledge and a few labeled samples, remains challenging and draws emerging concerns in computer vision. Metric-based methods incorporate the idea of metric learning into tackling the few-shot classification problem, i.e., the class membership of query samples can be determined in the latent space. Several recent methods propose to infer such memberships by feature reconstruction, which, however either exploit attention mechanisms or closed-form solutions. In this paper, we revisit the essence of feature reconstruction applied to few-shot classification and introduce a$l_{2,1}$-norm regularization to constraint it. As with the intention of attention mechanisms, we attempt to guide feature reconstruction towards focusing more on semantically rich target regions and diminishing the contribution of profitless features. We thereby obtain more discriminative query reconstructions for each class and perform classification based on these reconstruction errors. We formulate the proposal as a constrained optimization problem and design a simple and efficient iterative method as a solution. Furthermore, we provide the analysis on the convergence of proposed iterative method in theory. We conduct few-shot classification on two fine-grained and two general datasets. Extensive experimental results reveal that our method achieves new state-of-the-art performance.

Adversarial attacks in visual object tracking aims to fool trackers via injecting invisible perturbations for the video frames. Most adversarial methods advocate generating perturbations for each video frame, but frequent attacks may increase the computational load and the risk of exposure. Unfortunately, less works are about only attacking the initial frame and their attack effects are insufficient. To tackle this, we focus on the initialization phase of tracking and propose an only once attack framework. It can effectively fool the tracker via only generating invisible perturbations for the initial template, rather than each frame. Specifically, considering the tracking mechanism of the Siamese-based trackers, we design the minimum score-based and the minimum IoU-based loss functions. Both of them are used for training the UNet-based perturbation generator instead of the tracker, achieving the non-targeted attack. Additionally, we propose the location and direction offsets as the base attacks of sophisticated targeted attack. Combined with the two basic attacks, the tracker can be easily hijacked to move towards the fake target predefined by users. Extensive experimental results demonstrate that our only once attack framework costs the least number of attacks yet achieves better attack effect, with the maximum drop of 68.7%. The transferability experiments illustrate that our attack framework with good generalization ability can be directly applicable to the CNN-based, Siamese-based, deep discriminative-based and Transformer-based trackers, without retraining.

In interactive image segmentation methods, users can participate in and influence the segmentation process through their interactions, such as scribbles or bounding boxes. Similarly, the process of deep interactive segmentation utilizes users’ interactions to guide the network to learn the target of interest. This article mainly considers mouse clicking, which is the simplest interaction mode. Then, how to effectively characterize the click interaction (we call this “click encoding”) and fuse the click-related information with the network are the key issues in a deep interactive segmentation framework. However, the current click encoding method concentrates only on the spatial information of the clicks, so the region affected by each click is difficult to control, and the stability of the network is therefore reduced. Therefore, we propose a feature-interactive map that builds a close relationship between interaction information and target semantics. The affected region of the feature interactive map is determined by semantic information. Furthermore, we introduce an interactive nonlocal block by embedding a feature-interactive map into a nonlocal block, so that the long-range dependencies of the interaction information can be captured. Finally, based on the early fusion strategy, the features of the interactive nonlocal block are fused with the high-level features, thus amplifying the impacts of position and semantics on the final prediction results. Comprehensive experiments demonstrate that our click embedding approach significantly boosts the efficiency of the network and achieves state-of-the-art performance.

Modeling a sequence of video frames as a linear subspace on Grassmann manifold has recently become increasingly attractive in multiple computer vision applications. The success of such algorithms largely depends on a good distance measure, and learning an appropriate metric on Grassmann manifold remains a key challenge. Existing works address this by learning a discriminative mapping from the original Grassmann manifold to Hilbert space or a lower-dimensional, more discriminative Grassmann manifold. However, these approaches always highly rely on nearest neighbor matching of samples on the projected space, which is sensitive to noises and errors. Different from them, this paper proposes a Grassmann Reconstruction Metric Learning (GRML) algorithm guided by sparse representation-based classifier (SRC) for image set classification. SRC selects the coefficients associated with each class to reconstruct training samples, and then we employ it as a criterion to direct the design of a discriminant metric on Grassmann manifold. Specifically, GRML attempts to jointly maximize the inter-class reconstruction residual and minimize the intra-class reconstruction residual in the lower but more discriminative Grassmann manifold. To further explore the intrinsic geometry distance, we present a Grassmann Reconstruction Multiple Kernel Metric Learning (GRMKML) algorithm, which aims to jointly learn a metric and the corresponding kernel from a family of kernels for Grassmann manifold. Extensive experiments on eight benchmark datasets demonstrate that the proposed algorithms perform favorably against the state-of-the-art methods.

As a challenging visual task, visual object tracking has recently been composed of the classification and regression subtasks. The anchor-free regression network gets rid of the dependence on the anchors, but the redundant range makes it usually regress some samples involving non-target information. Evenly dividing a target by the regular receptive field often causes ambiguous target localization. To address these issues, we propose a regression-selective feature-adaptive tracker (RSFA), where the regression-selective subnetwork can not only free the regression task from anchors, but can also select more effective regression samples using the refined criterion. The proposed feature-adaptive strategy concentrates the classification subnetwork on target feature extraction via adaptively modifying the receptive field, and the attached centrality branch offers a correction for target localization by exploiting the spatial information. Additionally, the designed online update mechanism realizes the tracker's online optimization, improving robustness against target deformation. Extensive experiments are conducted on challenging benchmarks, including GOT10 K, OTB2015, UAV123, NFS, VOT2018, VOT2019 and VOT2020-ST. Our tracker achieves satisfactory tracking results, and the evaluations of its tracking performance rank first or second in comparison with the state-of-the-art tracking algorithms.

Incomplete multi-view clustering (IMVC) is deemed a significant research topic in multimedia to handle data loss situations. Current late fusion incomplete multi-view clustering methods have attracted intensive attention owing to their superiority in using consensus partition for effective and efficient imputation and clustering. However, 1) their imputation quality and clustering performance depend heavily on the static prior partition, such as predefined zeros filling, destroying the diversity of different views; 2) the size of base partitions is too small, which would lose advantageous details of base kernels to decrease clustering performance. To address these issues, we propose a novel IMVC method, named Dynamic Incomplete Multi-view Imputing and Clustering (DIMIC). Concretely, the observed views dynamically generate a consensus proxy with the guidance of a shared cluster matrix for more effective imputation and clustering, rather than a fixed predefined partition matrix. Furthermore, the proper size of base partitions is employed to protect sufficient kernel details for further enhancing the quality of the consensus proxy. By designing a solver with a linear computational and memory complexity on extensive experiments, our effectiveness, superiority, and efficiency are validated on multiple public datasets with recent advances.

This paper addresses the problem of normalized scene flow (NSF): given a pair of RGB video frames, estimating the 3D motion, which consisted of optical flow and motion-in-depth estimation. NSF is a powerful tool for action prediction and autonomous robot navigation, presenting the advantage of only needing a monocular and uncalibrated camera. However, most existing methods directly regress motion-in-depth from two RGB frames or optical flow, resulting in sub-accurate and non-robust results. Our key insight is the scale matching scheme-establishing correlations between two frames containing objects in different scales, to estimate dense and continuous motion-in-depth. Based on the scale matching, we propose a unified framework: Scale-flow, which combines scale matching and optical flow estimation. This combination makes optical flow estimation can use dense and continuous scale information for the first time, so that the moving foreground objects can be estimated more accurately. On KITTI, our monocular approach achieves the lowest error in the foreground scene flow task, even compared with the multi-camera method. Moreover, on the motion-in-depth estimation task, Scale-flow reduces the error by 34% compared with the best-published method. Code will be available.

Multiple kernel clustering (MKC) has demonstrated promising performance for handing non-linear data clustering. Positively, it can integrate complementary information of multiple base kernels and avoid kernel function selection. However, negatively, the main challenging is that the kernel matrix with the size n x n leads to O(n2) memory complexity and O(n3) computational complexity. To mitigate such a challenging, taking graph Laplacian as breakthrough, this paper proposes a novel and simple MKC method, dubbed as approximate shifted Laplacian reconstruction (ASLR). For each base kernel, we propose the r-rank shifted Laplacian reconstruction scheme by considering the energy losing of Laplacian reconstruction and the clustering information preserving of Laplacian decompose simultaneously. Then, by analyzing the eigenvectors of the reconstructed Laplacian, we impose some constrains to tame its solution within a Fantope. Accordingly, the byproduct (i.e. the most informative eigenvectors) contains the main clustering information, such that the clustering assignments can be obtained relying on simple k-means algorithm. Owe to the Laplacian reconstruction scheme, the memory and computational complexity can be reduced to O(n) and O<(n^2)$, respectively. As experimentally demonstrated on eight challenging MKC benchmark datasets, the results verify the effectiveness and efficiency of ASLR.

The dependency between global and local information can provide important contextual cues for semantic segmentation. Existing attention methods capture this dependency by calculating the pixel wise correlation between the learnt feature maps, which is of high space and time complexity. In this article, a new attention module, covariance attention, is presented, and which is interesting in the following aspects: 1) covariance matrix is used as a new attention module to model the global and local dependency for the feature maps and the local-global dependency is formulated as a simple matrix projection process; 2) since covariance matrix can encode the joint distribution information for the heterogeneous yet complementary statistics, the hand-engineered features are combined with the learnt features effectively using covariance matrix to boost the segmentation performance; 3) a covariance attention mechanism based semantic segmentation framework, CANet, is proposed and very competitive performance has been obtained. Comparisons with the state-of-the-art methods reveal the superiority of the proposed method.

Multi-view hashing (MvH) learns compact hash code by efficiently integrating multi-view data, and has achieved promising performance in large-scale retrieval task. In real-world applications, multi-view data is often stored or collected in different locations, and learning hash code in such case is more challenging yet less studied. In addition, unsupervised MvHs hardly achieve impressive retrieval performance due to absence of supervision. To fulfill this gap, this paper introduces a novel unsupervised multi-view distributed hashing (UMvDisH) to learn hash code from multi-view data, which is distributed in different nodes of a network. UMvDisH jointly performs latent factor model and spectral clustering to generate latent hash code and pseudo label respectively in each node. The consistency between hash code and pseudo label improves discrimination of hash code. The proposed distributed learning problem is divided into a set of decentralized subproblems by imposing local consistency among neighbor nodes. As such, the subproblems can be solved in parallel, and training time can be reduced. The communication cost is low due to no exchange of training data. Experimental results on four benchmark image datasets including a very large-scale image dataset show that UMvDisH achieves comparable retrieval performance and trains faster than state-of-the-art unsupervised MvHs in the distributed setting.

Deep learning has achieved promising performance in hyperspectral image (HSI) classification. Training deep models usually requires labeling massive HSIs, which however is prohibitively time-consuming and expensive. To fill in the gap, this paper proposes a novel meta-learning method for HSI few-shot classification that conducts HSI classification with a few labeled samples. Specifically, we introduce the Earth Mover’s Distance (EMD) as a metric. The designed EMD metric learning module aims to calculate the similarity of paired embedding features by decomposing embedding features into a set of local representations. The EMD metric aims to find the optimal matching flows between local representations that have the minimum matching cost. Furthermore, we attempt to learn class prototype representation for each hyperspectral class using the EMD metric. The proposed network effectively learns general knowledge from base HSIs and transfers such knowledge to the classification of novel HSIs. We conduct HSI few-shot classification by training on three base HSIs and classification on three novel HSIs. Extensive experimental results on three novel HSI datasets demonstrate that the proposed model outperforms existing state-of-the-art HSI methods, including two HSI few-shot methods.

In the field of image set classification, most existing works focus on exploiting effective latent discriminative features. However, it remains a research gap to efficiently handle this problem. In this paper, benefiting from the superiority of hashing in terms of its computational complexity and memory costs, we present a novel Discrete Metric Learning (DML) approach based on the Riemannian manifold for fast image set classification. The proposed DML jointly learns a metric in the induced space and a compact Hamming space, where efficient classification is carried out. Specifically, each image set is modeled as a point on Riemannian manifold after which the proposed DML minimizes the Hamming distance between similar Riemannian pairs and maximizes the Hamming distance between dissimilar ones by introducing a discriminative Mahalanobis-like matrix. To overcome the shortcoming of DML that relies on the vectorization of Riemannian representations, we further develop Bilinear Discrete Metric Learning (BDML) to directly manipulate the original Riemannian representations and explore the natural matrix structure for high-dimensional data. Different from conventional Riemannian metric learning methods, which require complicated Riemannian optimizations (e.g., Riemannian conjugate gradient), both DML and BDML can be efficiently optimized by computing the geodesic mean between the similarity matrix and inverse of the dissimilarity matrix. Extensive experiments conducted on different visual recognition tasks (face recognition, object recognition, and action recognition) demonstrate that the proposed methods achieve competitive performance in terms of accuracy and efficiency.

Deep supervised hashing has greatly improved retrieval performance with the powerful learning capability of deep neural network. In multi-label image retrieval, existing deep hashing simply indicates whether two images are similar by constructing a similarity matrix. However, it ignores the dependency among multiple labels that has been shown important in multi-label application. To fulfill this gap, this paper proposes Deep Co-Image-Label Hashing (DCILH) to discover label dependency. Specifically, DCILH regards image and label as two views, and maps the two views into a common deep Hamming space. DCILH proposes to learn prototype for each label, and preserve similarity among images, labels, and prototypes. To exploit label dependency, DCILH further employs the label-correlation aware loss on the predicted labels, such that predicted output on positive label is enforced to be larger than that on negative label. Extensive experiments on several multi-label benchmarks demonstrate the proposed DCILH outperforms state-of-the-art deep supervised hashing on large-scale multi-label image retrieval.

Kernel k-means (KKM) and spectral clustering (SC) are two basic methods used for multiple kernel clustering (MKC), which have both been widely used to identify clusters that are non-linearly separable. However, both of them have their own shortcomings: 1) the KKM-based methods usually focus on learning a discrete clustering indicator matrix via a combined consensus kernel, but cannot exploit the high-order affinities of all pre-defined base kernels; and 2) the SC-based methods require a robust and meaningful affinity graph in kernel space as input in order to form clusters with desired clustering structure. In this paper, a novel method, kernel k-means coupled graph tensor (KCGT), is proposed to graciously couple KKM and SC for seizing their merits and evading their demerits simultaneously. In specific, we innovatively develop a new graph learning paradigm by leveraging an explicit theoretical connection between clustering indicator matrix and affinity graph, such that the affinity graph propagated from KKM enjoys the valuable block diagonal and sparse property. Then, by using this graph learning paradigm, base kernels can produce multiple candidate affinity graphs, which are stacked into a low-rank graph tensor for capturing the high-order affinity of all these graphs. After that, by averaging all the frontal slices of the tensor, a high-quality affinity graph is obtained. Extensive experiments have shown the superiority of KCGT compared with the state-of-the-art MKC methods.

Spatial-spectral classification is an important research direction in hyperspectral image (HSI) processing, but HSI generally cannot provide enough spatial information because of its lower spatial resolution. To improve the classification accuracy on HSI, we propose an adaptive correction strategy on a cooperative classification framework under the assist of multispectral image (MSI), and this strategy adaptively selects the most suitable guided filter to do decision correction for each category. Experimental results demonstrate the effectiveness of the proposed method on cooperative classification.

Summary In this paper, we propose a composite nonlinear multiset canonical correlation projections (CNMCPs) framework where orthogonal constraints are imposed in each set. This makes CNMCP capable of learning uncorrelated low‐dimensional features with minimum redundancy in Hilbert space. With the CNMCP framework, we further present a particular algorithm called multikernel multiset canonical correlations or mKMCC, which introduces different weights into multiple nonlinear functions in all views. An alternating iterative optimization is designed for computational solution. Numerous experimental results on practical datasets have demonstrated the effectiveness and robustness of mKMCC, in contrast with existing kernel correlation learning approaches.

Significant attention to multiple kernel graph-based clustering (MKGC) has emerged in recent years, primarily due to the superiority of multiple kernel learning (MKL) and the outstanding performance of graph-based clustering. However, many existing MKGC methods design a fat model that poses challenges for computational cost and clustering performance, as they learn both an affinity graph and an extra consensus kernel cumbersomely. To tackle this challenging problem, this article proposes a new MKGC method to learn a consensus affinity graph directly. By using the self-expressiveness graph learning and an adaptive local structure learning term, the local manifold structure of the data in kernel space is preserved for learning multiple candidate affinity graphs from a kernel pool first. After that, these candidate affinity graphs are synthesized to learn a consensus affinity graph via a thin autoweighted fusion model, in which a self-tuned Laplacian rank constraint and a top- k neighbors sparse strategy are introduced to improve the quality of the consensus affinity graph for accurate clustering purposes. The experimental results on ten benchmark datasets and two synthetic datasets show that the proposed method consistently and significantly outperforms the state-of-the-art methods.

This article explores the relationships between image element pairs and label pairs and extends label diffusion to label pair diffusion for the interactive image segmentation task. Compared with label diffusion, more accurate relationships between unlabeled and labeled data can be captured on a tensor product graph (TPG) by using higher order information, and more complex interactions among image elements and finer relationships between image element pairs and label pairs are explored in label pair diffusion (LPD) process. We first establish a prior label estimation framework to measure the label pair prior probability. Then, a probability learning process on TPG is designed to smooth the label prior. The learning process is equivalent to an iterative LPD process on the original graph, which makes the proposed algorithm maintain computational efficiency. Finally, the unary label probabilities can be obtained by a total-probability-theorem-based conversion from the binary relationships. Experiments on popular segmentation data sets demonstrate the superior performance of the proposed method.

This paper presents an interactive image segmen-tation algorithm, in which the segmentation problem is formulated as a global manifold learning process. Based on the principle that the label of each element depends on the influence of all the elements in the image, we extend the conventional local neighborhood or the long range regional relationships to the global relationships over the whole image. A probabilistic framework is established to measure the global effect of each element on all other elements. Based on two different manifold learning styles, the semi-global manifold learning (SGML) and the fully-global manifold learning (FGML) algorithms are proposed to capture the global geometry structure of the data manifold. SGML learns the intrinsic effects of each element separately, equivalent to a matrix diffusion process on an affinity graph. FGML learns the intrinsic effects of all elements together, equivalent to a label pair diffusion process on a higher-order tensor product graph. The global manifold learning helps to overcome the low contrast, weak boundary and texture problems. Extensive experiments on three public interactive segmentation datasets demonstrate the superior performance of the proposed algorithms both in accuracy and efficiency.

Multiple kernel learning (MKL) is generally recognized to perform better than single kernel learning (SKL) in handling nonlinear clustering problem, largely thanks to MKL avoids selecting and tuning predefined kernel. By integrating the self-expression learning framework, the graph-based MKL subspace clustering has recently attracted considerable attention. However, the graph structure of data in kernel space is largely ignored by previous MKL methods, which is a key concept of affinity graph construction for spectral clustering purposes. In order to address this problem, a novel MKL method is proposed in this article, namely, structure-preserving multiple kernel clustering (SPMKC). Specifically, SPMKC proposes a new kernel affine weight strategy to learn an optimal consensus kernel from a predefined kernel pool, which can assign a suitable weight for each base kernel automatically. Furthermore, SPMKC proposes a kernel group self-expressiveness term and a kernel adaptive local structure learning term to preserve the global and local structure of the input data in kernel space, respectively, rather than the original space. In addition, an efficient algorithm is proposed to solve the resulting unified objective function, which iteratively updates the consensus kernel and the affinity graph so that collaboratively promoting each of them to reach the optimum condition. Experiments on both image and text clustering demonstrate that SPMKC outperforms the state-of-the-art MKL clustering methods in terms of clustering performance and computational cost.

Multi-view hashing efficiently integrates multi-view data for learning compact hash codes, and achieves impressive large-scale retrieval performance. In real-world applications, multi-view data are often stored or collected in different locations, where hash code learning is more challenging yet less studied. To fulfill this gap, this paper proposes a novel supervised multi-view distributed hashing (SMvDisH) for hash code learning from multi-view data in a distributed manner. SMvDisH yields the discriminative latent hash codes by joint learning of latent factor model and classifier. With local consistency assumption among neighbor nodes, the distributed learning problem is divided into a set of decentralized sub-problems. The sub-problems can be solved in parallel, and the computational and communication costs are low. Experimental results on three large-scale image datasets demonstrate that SMvDisH achieves competitive retrieval performance and trains faster than state-of-the-art multi-view hashing methods.

Real-time semantic segmentation is a challenging task as both segmentation accuracy and inference speed need to be considered at the same time. In this paper, we present a Dual Context Network (DCN et) to address this challenge. It contains two independent sub-networks: Region Context Network and Pixel Context Network. Region Context Network is main network with low-resolution input and feature re-weighting module to achieve sufficient receptive field. Meanwhile, utilizing Pixel Context Network with location attention module to capture the location dependencies of each pixel for assisting the main network to recover spatial detail. A contextual feature fusion is introduced to combine output features of these two subnetworks. The experiments show that DCNet can achieve high-quality segmentation while keeping a high speed. Specifically, for Cityscapes test dataset, we achieve 76.1 % Mean IOU with the speed of 82 FPS on a single GTX 2080Ti GPU when using ResNet50 as backbone, and 71.2% Mean IOU with the speed of 142 FPS when using ResNet18 as backbone.

Random noise in hyperspectral images (HSIs) may significantly degrade the image quality and further affect the subsequent image applications, such as land cover classification. To improve the performance of the existing classification methods on noisy HSIs, we propose a framework to take full advantages of the superpixel segmentation and the traditional pixel-wise classification methods. First, a novel superpixel model is proposed for HSI segmentation, where a new spectral similarity is defined in wavelet domain to make the superpixel model more robust to random noise; then, a simple but effective fusion strategy is designed to combine the superpixels with the pixel-wise classification results. Experimental results demonstrate the effectiveness of the proposed superpixel model and fusion strategy on noisy HSIs.

Fully polarimetric synthetic aperture radar (PolSAR) images are generally expressed as the complex-valued (CV) matrix, whereas the convolutional neural networks (CNNs) have been successfully utilized for the PolSAR image classification. However, most 2D or 3D CNNs suffer from insufficient exploring the CV features or computationally expensive. To address these issues, this paper presents an efficient complex-valued spatial-scattering separated attention network (CVS3ANet) for PolSAR image classification. The CVS3ANet utilizes CV-3D convolutions to explore the features in both spatial and scattering dimensions of the PolSAR images, and reduces the parameters by factorizing the 3D convolution as a sequential process of 2D spatial convolution followed by 1D scattering convolution. Moreover, a squeeze and fusion attention unit is used to enhance the learning interpretation ability of the network by modeling correlations between channels with respect to attention probability. The experimental results demonstrate that the proposed method can obtain superior results over the state-of-the-art techniques.

Three‐dimensional (3D) driver pose estimation is a promising and challenging problem for computer–human interaction. Recently convolutional neural networks have been introduced into 3D pose estimation, but these methods have the problem of slow running speed and are not suitable for driving scenario. In this study, the proposed method is based on two types of inputs, infrared image and point cloud obtained from time‐of‐flight camera. The authors propose a joint 2D–3D network incorporating image‐based and point‐based feature to promote the performance of 3D human pose estimation and run on a high speed. For point cloud with invalid points, the authors first do preprocess and then design a denoising module to handle this problem. Experiments on private driver data set and public Invariant‐Top View data set show that the proposed method achieves efficient and competitive performance on 3D human pose estimation.

Recently, sparse representation-based classification (SRC) has been widely studied and has produced state-of-the-art results in various classification tasks. Learning useful and computationally convenient representations from complex redundant and highly variable visual data is crucial for the success of SRC. However, how to find the best feature representation to work with SRC remains an open question. In this paper, we present a novel discriminative projection learning approach with the objective of seeking a projection matrix such that the learned low-dimensional representation can fit SRC well and that it has well discriminant ability. More specifically, we formulate the learning algorithm as a bilevel optimization problem, where the optimization includes an ℓ1-norm minimization problem in its constraints. Through the bilevel optimization model, the relationship between sparse representation and the desired feature projection can be explicitly exploited during the learning process. Therefore, SRC can achieve a better performance in the transformed subspace. The optimization model can be solved by using a stochastic gradient ascent algorithm, and the desired gradient is computed using implicit differentiation. Furthermore, our method can be easily extended to learn a dictionary. The extensive experimental results on a series of benchmark databases show that our method outperforms many state-of-the-art algorithms.

To defy the curse of dimensionality, the inputs are always projected from the original high-dimensional space into the target low-dimension space for feature extraction. However, due to the existence of noise and outliers, the feature extraction task for corrupted data is still a challenging problem. Recently, a robust method called low rank embedding (LRE) was proposed. Despite the success of LRE in experimental studies, it also has many disadvantages: 1) The learned projection cannot quantitatively interpret the importance of features. 2) LRE does not perform data reconstruction so that the features may not be capable of holding the main energy of the original “clean” data. 3) LRE explicitly transforms error into the target space. 4) LRE is an unsupervised method, which is only suitable for unsupervised scenarios. To address these problems, in this paper, we propose a novel method to exploit the latent discriminative features. In particular, we first utilize an orthogonal matrix to hold the main energy of the original data. Next, we introduce an 12,1-norm term to encourage the features to be more compact, discriminative and interpretable. Then, we enforce a columnwise 12,1-norm constraint on an error component to resist noise. Finally, we integrate a classification loss term into the objective function to fit supervised scenarios. Our method performs better than several state-of-the-art methods in terms of effectiveness and robustness, as demonstrated on six publicly available datasets.

3D driver pose estimation is a promising and challenging problem for computer-human interaction. Recently convolutional neural networks (CNNs) have been introduced into 3D pose estimation, but these methods have the problem of slow running speed and are not suitable for driving scenario. In this paper, our method is based on two type of inputs, IR image and point cloud obtained from TOF camera. We propose a Joint 2D-3D network incorporating image-based and point-based feature to promote the performance of 3D human pose estimation and run in a high speed. For point cloud with invalid points, we firstly do preprocess and then design a denoising module to handle this problem. Experiments on private Driver dataset and public ITOP dataset show that our method achieves efficient and competitive performance on 3D human pose estimation.

Face hallucination (FH) is an effective technique for super-resolving low-resolution (LR) face images. In real-world applications, a face image usually has multiple distinct low resolutions. Most existing FH methods can not effectively deal with multiple LR views simultaneously. To solve this issue, we present a multi-set partial least squares (MPLS) approach and its kernel extension for jointly learning the nonlinear consistency of multi-resolution facial features. With nonlinear MPLS, we present a novel simultaneous super-resolution coherent facial feature method for the face images with multiple LRs, which has capacity of jointly learning the nonlinear relationships between multiple facial resolutions. Experimental results demonstrate the effectiveness and robustness of our proposed FH method.

Two-directional two-dimensional canonical correlation analysis ((2D)2CCA) directly seeks linear relationship between different image data sets without reshaping images into vectors. However, it fails in finding the nonlinear correlation. In this letter, a novel method named as two-directional two-dimensional kernel canonical correlation analysis is proposed, which is a nonlinear version of (2D)2CCA and is able to find the nonlinear relationship between different image data sets. Experimental results in different expressions, illumination conditions and poses show the effectiveness of the proposed method.

This paper presents an interactive image segmentation approach in which we formulate segmentation as a probabilistic estimation problem based on the prior user intention. Instead of directly measuring the relationship between pixels and labels, we first estimate the distances between pixel pairs and label pairs using a probabilistic framework. Then, binary probabilities with label pairs are naturally converted to unary probabilities with labels. The higher order relationship helps improve the robustness to user inputs. To improve segmentation accuracy, a likelihood learning framework is proposed to fuse the region and the boundary information of the image by imposing a smoothing constraint on the unary potentials. Furthermore, we establish an equivalence relationship between likelihood learning and likelihood diffusion and propose an iterative diffusion-based optimization strategy to maintain computational efficiency. Experiments on the Berkeley segmentation data set and Microsoft GrabCut database demonstrate that the proposed method can obtain better performance than the state-of-the-art methods.

Embedding methods have shown promising performance in multi-label prediction, as they can discover the dependency of labels. Most embedding methods cannot well align the input and output, which leads to degradation in prediction performance. Besides, they suffer from expensive prediction computational costs when applied to large-scale datasets. To address the above issues, this paper proposes a Co-Hashing (CoH) method by formulating multi-label learning from the perspective of cross-view learning. CoH first regards the input and output as two views, and then aims to learn a common latent hamming space, where input and output pairs are compressed into compact binary embeddings. CoH enjoys two key benefits: 1) the input and output can be well aligned, and their correlations are explored; 2) the prediction is very efficient using fast cross-view kNN search in the hamming space. Moreover, we provide the generalization error bound for our method. Extensive experiments on eight real-world datasets demonstrate the superiority of the proposed CoH over the state-of-the-art methods in terms of both prediction accuracy and efficiency.

Today, Big Data brings benefits to many areas of scientific research. However, processing these large amounts of data often requires extensive computing time and a large storage space. Global feature analysis is considered to be universal in traditional super resolution methods, but it is not applicable to Big Data. There remains a viewpoint that it is not necessary to address all data equally and impartially. Focusing on useful information can make the massive data analysis possible and more effective. In this paper, we consider the significant regions, and thus, we propose a new super resolution approach that uses significant and general information. Under the framework of a convolutional neural network, the training process is performed on the significant parts of the training data set, and the reconstruction process considers significant parts separately; then, a super resolution image will be obtained according to each different demand. This concept is easy to understand, but it can be achieved only via the Big Data approach with many similar images on the Internet and the effective deep learning algorithm. Experiments show that our new approach can reduce the testing time and obtain a high-quality reconstructed image.

Great advances have been made for the deep networks, but relatively high memory and computation requirements limit their applications in the embedded device. In this paper, we introduce a class of efficient network architecture named ThinNet mainly for object detection applications on memory and computation limited platforms. The new architecture is based on two proposed modules: Front module and Tinier module. The Front module reduce the information loss from raw input images by utilizing more convolution layers with small size filters. The Tinier module use pointwise convolution layers before conventional convolution layer to decrease model size and computation, while ensuring the detection accuracy. Experimental evaluations on ImageNet classification and PASCAL VOC object detection datasets demonstrate the superior performance of ThinNet over other popular models. Our pretrained classification model(ThinNet_C) attains the same top-l and top-5 performance as the classic AlexNet but only with 1/50th the parameters. The detection model also obtains significant improvements over other detection methods, while requiring smaller model size to achieve high performance.

DLRC is an extension of LRC that extends LRC from conventional still image based method to the image set based method. DLRC has a demonstrated better performance on image set classification. However, when the image sets of different objects are not linear separable, or when the linear regression axes of class-specific samples of different classes have an intersection, DLRC may be failed for well classifying the image sets. In this paper, a new classification method, kernel dual linear regression classification (KDLRC), is proposed. KDLRC is a nonlinear version of DLRC and can overcome the drawback of DLRC. KDLRC first embeds the input data into a high-dimensional Hilbert space, then in the kernel space, the data become easier to classify. Extensive experiments on four well-known databases prove that the performance of KDLRC is better than that of DLRC and several state-of-the-art classifiers.

Recently, object classification of remote sensing images has attracted more and more research interests due to the development of satellite and aerial vehicle technologies. Hashing learning is an efficient method to handle the huge amount of the remote sensing data. In this paper, we proposed a novel hashing learning method named partial randomness supervised discrete hashing (PRSDH), which combines data-dependent methods and data-independent methods. It jointly learns a discrete binary codes generation and partial random constraint optimization model. By random projection, the computation complexity is reduced effectively. With the weight matrix derived from the training data, the semantic similarity between the data can be well preserved while generating the hashing codes. For the discrete constraint problem, this paper adopts the discrete cyclic coordinate descent (DCC) algorithm to optimize the codes bit by bit. The experimental results show that PRSDH outperforms other comparative methods and demonstrated that PRSDH has good adaptability to the characteristic of remote sensing object.

Network embedding aims to seek low-dimensional vector representations for network nodes, by preserving the network structure. The network embedding is typically represented in continuous vector, which imposes formidable challenges in storage and computation costs, particularly in large-scale applications. To address the issue, this paper proposes a novel discrete network embedding (DNE) for more compact representations. In particular, DNE learns short binary codes to represent each node. The Hamming similarity between two binary embeddings is then employed to well approximate the ground-truth similarity. A novel discrete multi-class classifier is also developed to expedite classification. Moreover, we propose to jointly learn the discrete embedding and classifier within a unified framework to improve the compactness and discrimination of network embedding. Extensive experiments on node classification consistently demonstrate that DNE exhibits lower storage and computational complexity than state-of-the-art network embedding methods, while obtains competitive classification results.

Summary There are two problems in global over‐complete dictionary: lack of adaptability to image local structure and low computational efficiency. Based on the study of the adaptive sparse representation reconstruction, this paper obtained a series of corresponding sub‐dictionaries by image block subset, then the optimal sub‐dictionary for each reconstruction image block is adaptively selected, which can be more accurately sparse represented modeled to improve the effect and efficiency of the algorithm. In order to promote the ability of sparse representation model, nonlocal self‐similarity prior item is introduced. Meanwhile, the nonlocal self‐similarity model is improved by using the idea of the bilateral filter, and the space distance restraint between pixels is introduced to better keep the image edge information. Moreover, the nonlocal self‐similar distance measure is improved to reduce the amount of calculation. Experimental results show that the proposed algorithm can effectively suppress noise effects and can maintain the image edge details, at the same time, there are certain advantages at both the peak signal to noise ratio (PSNR) and visual effects.

Hashing techniques have recently gained increasing research interest in multimedia studies. Most existing hashing methods only employ single features for hash code learning. Multiview data with each view corresponding to a type of feature generally provides more comprehensive information. How to efficiently integrate multiple views for learning compact hash codes still remains challenging. In this article, we propose a novel unsupervised hashing method, dubbed multiview discrete hashing (MvDH), by effectively exploring multiview data. Specifically, MvDH performs matrix factorization to generate the hash codes as the latent representations shared by multiple views, during which spectral clustering is performed simultaneously. The joint learning of hash codes and cluster labels enables that MvDH can generate more discriminative hash codes, which are optimal for classification. An efficient alternating algorithm is developed to solve the proposed optimization problem with guaranteed convergence and low computational complexity. The binary codes are optimized via the discrete cyclic coordinate descent (DCC) method to reduce the quantization errors. Extensive experimental results on three large-scale benchmark datasets demonstrate the superiority of the proposed method over several state-of-the-art methods in terms of both accuracy and scalability.

Embedding methods have shown promising performance in multilabel prediction, as they are able to discover the label dependence. However, most methods ignore the correlations between the input and output, such that their learned embeddings are not well aligned, which leads to degradation in prediction performance. This paper presents a formulation for multilabel learning, from the perspective of cross-view learning, that explores the correlations between the input and the output. The proposed method, called Co-Embedding (CoE), jointly learns a semantic common subspace and view-specific mappings within one framework. The semantic similarity structure among the embeddings is further preserved, ensuring that close embeddings share similar labels. Additionally, CoE conducts multilabel prediction through the cross-view $k$ nearest neighborhood ( $k$ NN) search among the learned embeddings, which significantly reduces computational costs compared with conventional decoding schemes. A hashing-based model, i.e., Co-Hashing (CoH), is further proposed. CoH is based on CoE, and imposes the binary constraint on continuous latent embeddings. CoH aims to generate compact binary representations to improve the prediction efficiency by benefiting from the efficient $k$ NN search of multiple labels in the Hamming space. Extensive experiments on various real-world data sets demonstrate the superiority of the proposed methods over the state of the arts in terms of both prediction accuracy and efficiency.

Large-scale clustering has been widely used in many applications, and has received much attention. Most existing clustering methods suffer from both expensive computation and memory costs when applied to large-scale datasets. In this paper, we propose a novel clustering method, dubbed compressed k-means (CKM), for fast large-scale clustering. Specifically, high-dimensional data are compressed into short binary codes, which are well suited for fast clustering. CKM enjoys two key benefits: 1) storage can be significantly reduced by representing data points as binary codes; 2) distance computation is very efficient using Hamming metric between binary codes. We propose to jointly learn binary codes and clusters within one framework. Extensive experimental results on four large-scale datasets, including two million-scale datasets demonstrate that CKM outperforms the state-of-the-art large-scale clustering methods in terms of both computation and memory cost, while achieving comparable clustering accuracy.

Accurate image segmentation is an important issue in image processing, where unsupervised clustering models play an important part and have been proven to be effective. However, most clustering methods suffer from limited segmentation accuracy without considering spatial information or bounded support region for practical data. In this paper, a bounded spatial probability based fuzzy clustering algorithm is proposed for image segmentation. A bounded distribution to fit the bounded data is utilized and a new conditional probability is constructed based on the immediate neighboring probabilities. Then a parameter-free mean template is presented to impose the spatial information more precisely. Finally, the negative logarithmical conditional probability is utilized as the dissimilarity function to describe the observed data. We evaluated our algorithm against several state-of-the-art segmentation approaches on brain magnetic resonance images. Our results suggest that the proposed algorithm is more robust to noise and textures, and can produce more accurate segmentation results.

This paper presents an interactive image segmentation approach where the segmentation problem is formulated as a probabilistic estimation manner. Instead of measuring the distances between unseeded pixels and seeded pixels, we measure the similarities between pixel pairs and seed pairs to improve the robustness to the seeds. The unary prior probability of each pixel belonging to the foreground F and background B can be effectively estimated based on the similarities with label pairs (F, F),(F, B),(B, F) and (B, B). Then a likelihood learning framework is proposed to fuse the region and boundary information of the image by imposing the smoothing constraint on the unary potentials. Experiments on challenging data sets demonstrate that the proposed method can obtain better performance than state-of-the-art methods.

Exact compressed sensing (CS) recovery theoretically depends on a large number of random measurements. In this study, the authors present a novel CS measurement technique based on the cellular automata chaos (CAC) model. The proposed method selects original signal thresholding (OST) as its initial seed to realise CS signal coding. The benefits of CS coding with CAC‐OST are that: (i) the signal compression ratio of this coding method can be far below the signal sparsity level and (ii) the signal can be recovered perfectly, even with slow CS measurements. This study reports some experiments that demonstrate the excellent performance of CAC‐OST in CS coding.

Due to the significant reduction in computational cost and storage, hashing techniques have gained increasing interests in facilitating large-scale cross-view retrieval tasks. Most cross-view hashing methods are developed by assuming that data from different views are well paired, e.g., text-image pairs. In real-world applications, however, this fully-paired multiview setting may not be practical. The more practical yet challenging semi-paired cross-view retrieval problem, where pairwise correspondences are only partially provided, has less been studied. In this paper, we propose an unsupervised hashing method for semi-paired cross-view retrieval, dubbed semi-paired discrete hashing (SPDH). In specific, SPDH explores the underlying structure of the constructed common latent subspace, where both paired and unpaired samples are well aligned. To effectively preserve the similarities of semi-paired data in the latent subspace, we construct the cross-view similarity graph with the help of anchor data pairs. SPDH jointly learns the latent features and hash codes with a factorization-based coding scheme. For the formulated objective function, we devise an efficient alternating optimization algorithm, where the key binary code learning problem is solved in a bit-by-bit manner with each bit generated with a closed-form solution. The proposed method is extensively evaluated on four benchmark datasets with both fully-paired and semi-paired settings and the results demonstrate the superiority of SPDH over several other state-of-the-art methods in term of both accuracy and scalability.

In recent years, sparse representation-based classification (SRC) is one of the most successful methods and has been shown impressive performance in various classification tasks. However, when the training data have a different distribution than the testing data, the learned sparse representation may not be optimal, and the performance of SRC will be degraded significantly. To address this problem, in this paper, we propose an optimal couple projections for domain-adaptive SRC (OCPD-SRC) method, in which the discriminative features of data in the two domains are simultaneously learned with the dictionary that can succinctly represent the training and testing data in the projected space. OCPD-SRC is designed based on the decision rule of SRC, with the objective to learn coupled projection matrices and a common discriminative dictionary such that the between-class sparse reconstruction residuals of data from both domains are maximized, and the within-class sparse reconstruction residuals of data are minimized in the projected low-dimensional space. Thus, the resulting representations can well fit SRC and simultaneously have a better discriminant ability. In addition, our method can be easily extended to multiple domains and can be kernelized to deal with the nonlinear structure of data. The optimal solution for the proposed method can be efficiently obtained following the alternative optimization method. Extensive experimental results on a series of benchmark databases show that our method is better or comparable to many state-of-the-art methods.

This paper presents a content-adaptively sparse reconstruction method for abnormal events detection by exploiting the low-rank property of video sequences. In dictionary learning phase, the bases which describe more important characteristics of the normal behavior patterns are assigned with lower reconstruction costs. Based on the low-rank property of the bases captured by the low-rank approximation, a weighted sparse reconstruction method is proposed to measure the abnormality of testing samples. Multiscale 3-D gradient features, which encode the spatiotemporal information, are adopted as the low level descriptors. The benefits of the proposed method are threefold: first, the low-rank property is utilized to learn the underlying normal dictionaries, which can represent groups of similar normal features effectively; second, the sparsity-based algorithm can adaptively determine the number of dictionary bases, which makes it a preferable choice for representing the dynamic scene semantics; and third, based on the weighted sparse reconstruction method, the proposed method is more efficient for detecting the abnormal events. Experimental results on the public datasets have shown that the proposed method yields competitive performance comparing with the state-of-the-art methods.

canonical correlations (M 2 CCs) framework for subspace learning. In the proposed framework, the input data of each original view are mapped into multiple higher dimensional feature spaces by multiple nonlinear mappings determined by different kernels. This makes M 2 CC can discover multiple kinds of useful information of each original view in the feature spaces. With the framework, we further provide a specific multi-view feature learning method based on direct summation kernel strategy and its regularized version. The experimental results in visual recognition tasks demonstrate the effectiveness and robustness of the proposed method.

Hashing techniques have been widely applied to large-scale cross-view retrieval tasks due to the significant advantage of binary codes in computation and storage efficiency. However, most existing cross-view hashing methods learn binary codes with continuous relaxations, which cause large quantization loss across views. To address this problem, in this letter, we propose a novel cross-view hashing method, where a common Hamming space is learned such that binary codes from different views are consistent and comparable. The quantization loss across views is explicitly reduced by two carefully designed regression terms from original spaces to the Hamming space. In our method, the l2,1-norm regularization is further exploited for discriminative feature selection. To obtain high-quality binary codes, we propose to jointly learn the codes and hash functions, for which an efficient iterative algorithm is presented. We evaluate the proposed method, dubbed Robust Cross-view Hashing (RCH), on two benchmark datasets and the results demonstrate the superiority of RCH over many other state-of-the-art methods in terms of retrieval performance and cross-view consistency.

Sparse representation-based classification (SRC) has been developed and shown great potential for real-world application. Based on SRC, Yang et al. devised an SRC steered discriminative projection (SRC-DP) method. However, as a linear algorithm, SRC-DP cannot handle the data with highly nonlinear distribution. Kernel sparse representation-based classifier (KSRC) is a non-linear extension of SRC and can remedy the drawback of SRC. KSRC requires the use of a predetermined kernel function and selection of the kernel function and its parameters is difficult. Recently, multiple kernel learning for SRC (MKL-SRC) has been proposed to learn a kernel from a set of base kernels. However, MKL-SRC only considers the within-class reconstruction residual while ignoring the between-class relationship, when learning the kernel weights. In this paper, we propose a novel multiple kernel sparse representation-based classifier, and then we use it as a criterion to design a multiple kernel sparse representation-based orthogonal discriminative projection method. The proposed algorithm aims at learning a projection matrix and a corresponding kernel from the given base kernels such that in the low dimension subspace the between-class reconstruction residual is maximized and the within-class reconstruction residual is minimized. Furthermore, to achieve a minimum overall loss by performing recognition in the learned low-dimensional subspace, we introduce cost information into the dimensionality reduction method. The solutions for the proposed method can be efficiently found based on trace ratio optimization method. Extensive experimental results demonstrate the superiority of the proposed algorithm when compared with the state-of-the-art methods.

The combination of pixel and superpixel has been widely utilized in the interactive segmentation methods to overcome the sensitivity to the seeds' quantity and quality. However, because of the introduction of more variables and variables' interactions, the pixel-superpixel combination methods are still limited to the segmentation accuracy and computational complexity. To solve these problems, in this paper, we propose an interactive multilabel image segmentation method. In the proposed segmentation model, the multilayer relationships among the pixel layer, superpixel layer, and label layer are fused by the Markov random field framework to further improve the segmentation accuracy. During the optimization stage, the parallel partial optimality strategy is utilized to effectively solve the multilabel submodular energy function. Experimental results on challenging data sets demonstrate the competitiveness of the proposed method comparing with several state-of-the-art interactive algorithms.

This paper presents a new hyperspectral image (HSI) classification method which is capable of automatic feature learning while achieving high classification accuracy. The method contains two major modules: the spectral classification module and the spatial constraint module. Spectral classification module uses a deep network named stacked denoising autoencoders (SdA) to learn feature representation of the data. Through SdA, the data are projected nonlinearly from its original hyperspectral space to some higher dimensional space where more compact distribution is obtained. An interesting aspect of this method is that it does not need a feature design/extraction process guided by human prior. The suitable feature for the classification is learned by the deep network itself. Superpixel is utilized to generate the spatial constraints to refine the spectral classification results. By exploiting the spatial consistency of neighborhood pixels, the accuracy of classification is further improved by a big margin. Experiments on the public datasets reveal the superior performance of the proposed method.

The existing non-local structure tensors utilize the isotropic nature of the neighborhoods and compare similarity of tensors by the Euclidean distance for tensor field regularization, thus resulting in limited performances in image analysis. In this paper, we present an anisotropic nonlocal tensor regularization method by using a directional projection based atomic decomposition scheme, which offers two advantages: better exploitation of spatial directional information for anisotropically regularizing tensor field, and straightforward employment of the Euclidean distance to compute smoothing weights without extending the original non-local means filter to tensor field. Experimental results show that the proposed anisotropic structure tensor is superior to existing representative nonlinear structure tensors, in terms of corner detection and image denoising.

Hashing techniques have attracted broad research interests in recent multimedia studies. However, most of existing hashing methods focus on learning binary codes from data with only one single view, and thus cannot fully utilize the rich information from multiple views of data. In this paper, we propose a novel unsupervised hashing approach, dubbed multi-view latent hashing (MVLH), to effectively incorporate multi-view data into hash code learning. Specifically, the binary codes are learned by the latent factors shared by multiple views from an unified kernel feature space, where the weights of different views are adaptively learned according to the reconstruction error with each view. We then propose to solve the associate optimization problem with an efficient alternating algorithm. To obtain high-quality binary codes, we provide a novel scheme to directly learn the codes without resorting to continuous relaxations, where each bit is efficiently computed in a closed form. We evaluate the proposed method on several large-scale datasets and the results demonstrate the superiority of our method over several other state-of-the-art methods.

In colour compressed sensing (CS) imaging, the current two bottlenecks for application are (1) high computation cost of sparse representation (SR) with over‐complete dictionary and (2) unsatisfactory imaging quality of CS recovery with l 1 ‐norm minimisation. Thus, this study proposes a novel colour CS imaging framework. In the framework, two improvements are achieved: (1) the authors present the sparse difference to reduce the computation cost of SR in RGB colour imaging; (2) the authors use fractal dimension instead of l 1 ‐norm as the object function to actualise high quality CS recovery. The feasibility of our colour CS imaging framework is proved by sseveral experiments.

Different from the photometric images, depth images resolve the distance ambiguity of the scene, while the properties, such as weak texture, high noise, and low resolution, may limit the representation ability of the well-developed descriptors, which are elaborately designed for the photometric images. In this paper, a novel depth descriptor, geodesic invariant feature (GIF), is presented for representing the parts of the articulate objects in depth images. GIF is a multilevel feature representation framework, which is proposed based on the nature of depth images. Low-level, geodesic gradient is introduced to obtain the invariance to the articulate motion, such as scale and rotation variation. Midlevel, superpixel clustering is applied to reduce depth image redundancy, resulting in faster processing speed and better robustness to noise. High-level, deep network is used to exploit the nonlinearity of the data, which further improves the classification accuracy. The proposed descriptor is capable of encoding the local structures in the depth data effectively and efficiently. Comparisons with the state-of-the-art methods reveal the superiority of the proposed method.

Fuzzy clustering algorithms have been widely used in brain magnetic resonance (MR) image segmentation. However, due to the existence of noise and intensity inhomogeneity, many segmentation algorithms suffer from limited accuracy. In this paper, we propose a fuzzy clustering algorithm with robust spatially constraint for accurate and robust brain MR image segmentation. A novel spatial factor is proposed by incorporating the spatial information amongst neighborhood pixels with a simple metric. A new weight factor, which utilizes the intensity information of the original image, is constructed to filter the posterior and prior probabilities in the spatial neighborhood. The proposed method can preserve more details and overcome the over-smoothing disadvantage. Finally, the fuzzy objective function is integrated with the bias field estimation model to overcome the intensity inhomogeneity in the image and segment the brain MR images. Experimental results demonstrate that the proposed algorithm can substantially improve the accuracy of brain MR image segmentation.

Accurate estimation of noise level in images plays an important role in different image processing applications. The current algorithms can precisely estimate noise with smooth images, but it is still the challenge to approximate noise level from richly textured images. In this paper, we proposed a new adaptive superpixel classification algorithm for noise estimation in complicated textured images. Firstly, our new superpixel algorithm adapts the finite Gaussian clustering approach, which can better approximate homogeneous patches in noisy images. Then noise information is obtained locally from each superpixel patch. Finally, the best estimation of noise level is calculated with a statistical approach. Experimental results with various kinds of images demonstrate that our method is more accurate and robust compared to the five existing common used algorithms.

There are lots of objective image quality assessment (IQA) algorithms to accurately assess the quality of images recently; however, the characteristics of Human Visual System (HVS) as the ultimate receiver of the visual signal are critical factors affecting IQA. Saliency as a feature reflecting quality has been studied deeply. Since HVS has intricate psychovisual structure and nature, the impact of saliency on IQA algorithms needs still further exploration. This paper focuses mainly on the influence of distortion information on saliency applied on IQA. We eliminated interference of algorithms to study the characteristics of different distortion types and degrees simply. We applied three different objective metrics adding natural scene saliency (NSS) with different adding strategies on the LIVE IQA database. Experimental results demonstrate that the variation in saliency highly depends on the distortion type and degree. IQA algorithms will achieve improving performances on their accuracy by applied proper saliency strategy.

Changes of illumination have a huge effect on image quality during imaging process. One method that compares high and low resolution images to compute MTF in image quality assessment requires images to be in the same illumination conditions. Thus, it is necessary to do illumination normalization. This letter presents a novel method by combining gradient domain method and improved singular value equalization, which can achieve a good result of illumination normalization. The gradient domain method can bring the contrasts of multiple images to the same level while the improved singular value equalization can make their intensity means close to each other. We also suggest a parameter named p to assess the illumination consistency quantitatively. Experimental results demonstrate that the proposed method has a good performance in visualization and quantitative assessment.

Multiset features extracted from the same patterns always represent different characteristics of data. Thus, it is very valuable to perform the extraction on multiple feature sets. This paper addresses the issue of multiset correlation feature extraction (MCFE) in multiple feature representations. A novel method is proposed to carry out the MCFE for classification, called multiset canonical correlations using globality-preserving projections (MCC-GPs), which can perform joint dimensionality reduction for high-dimensional data. MCC-GP integrates correlational characteristics of feature pairs and global geometric information of data in the transformed low-dimensional space. This makes MCC-GPs have better discriminant ability than a previous method proposed by the authors, called multiset integrated canonical correlation analysis (MICCA), which only considers correlations for recognition tasks. Furthermore, MCC-GP can subsume two popular feature extraction methods into its framework under some constraints. This also provides a new insight for these two methods. The proposed method is applied to pattern recognition and examined using the COIL-100 and ETH-80 object databases and AR, CMU PIE, and Yale face databases. Extensive experimental results show that MCC-GP outperforms MICCA and multiset canonical correlation analysis in terms of classification accuracy and efficiency.

This study presents a novel compressive sensing (CS) framework to solve the high dimensional mass spectrometry (MS) signal processing in Bioinformatics. As a hot research topic, CS has attracted a great deal of attention in many fields. In theory, high sparsity is one precondition for any CS framework. However, in Bioinformatics, one application bottleneck is that only a few MS data can be considered as sparse. So sparse representation (SR) become necessary. However, this will create a new problem that the SR computation cost will be too huge to MS signal because of its high data dimensionality (usually tens of thousands or more). Therefore the authors propose theconcept ofsparse difference (SD) to realise a new CS framework. Firstly, it canacquire the prior MS information through fractal and entropy recognition. Secondly, the original signal can be perfectly recovered by SD based on the previous recognition result. The feasibility and validity of this CS framework isproved by experiments.

Accurate brain tissue segmentation from magnetic resonance (MR) images is an essential step in quantitative brain image analysis. However, due to the existence of noise and intensity inhomogeneity in brain MR images, many segmentation algorithms suffer from limited accuracy. In this paper, we assume that the local image data within each voxel's neighborhood satisfy the Gaussian mixture model (GMM), and thus propose the fuzzy local GMM (FLGMM) algorithm for automated brain MR image segmentation. This algorithm estimates the segmentation result that maximizes the posterior probability by minimizing an objective energy function, in which a truncated Gaussian kernel function is used to impose the spatial constraint and fuzzy memberships are employed to balance the contribution of each GMM. We compared our algorithm to state-of-the-art segmentation approaches in both synthetic and clinical data. Our results show that the proposed algorithm can largely overcome the difficulties raised by noise, low contrast, and bias field, and substantially improve the accuracy of brain MR image segmentation.

This paper presents a new interactive image segmentation method. The key idea is to generate the object contour feature image according to a small number of user supplied object contour points, and then adopt the parametric active contour model for object segmentation. An image patch matching with rotation-invariance is utilized to generate object contour feature image. In order to prevent the evolving curve running into local optimal solution, the initial evolving curve of the parametric active contour is constructed by local intensity values of the object contour feature image. Medical image segmentation results indicate that the proposed method is superior to the traditional parametric active contour model, and is an effective semi-automatic image segmentation method.

This letter presents a two-stage object tracking method by combining a region-based method and a contour-based method. First, a kernel-based method is adopted to locate the object region. Then the diffusion snake is used to evolve the object contour in order to improve the tracking precision. In the first object localization stage, the initial target position is predicted and evaluated by the Kalman filter and the Bhattacharyya coefficient, respectively. In the contour evolution stage, the active contour is evolved on the basis of an object feature image generated with the color information in the initial object region. In the process of the evolution, similarities of the target region are compared to ensure that the object contour evolves in the right way. The comparison between our method and the kernel-based method demonstrates that our method can effectively cope with the severe deformation of object contour, so the tracking precision of our method is higher.

A new feature extraction method, based on feature fusion, according to the idea of canonical correlation analysis (CCA), is proposed in this paper. A framework of CCA used in pattern recognition is described. The overall process comprises: extracting two groups of feature vectors with the same pattern; establishing the correlation criterion function between the two groups of feature vectors, and extract their canonical correlation features in order to form effective discriminant vectors for recognition. The inherent essence of this method used in recognition is theoretically analyzed. This method uses correlation features between two groups of feature vectors as effective discriminant information, so it not only is suitable for information fusion, but also eliminates redundant information within features, a new way for classification is proposed. Experimental results of our method applying on Concordia University CENPARMI handwritten numeral database has shown that our recognition rate is higher than that of the algorithm adopting single feature or the existing fusion algorithm.