The hyperspectral anomaly detection (HAD) aims to identify potential anomalies from complex backgrounds. Most reconstruction-based autoencoders equally treat background pixels and anomalies or ignore potential spatial information. In this letter, we propose an HAD method based on multiscale memory autoencoder and spatial filtering, abbreviated as SFM2AE. Specifically, by introducing memory modules into different hidden layers of the autoencoder, multiscale reconstruction of background and anomaly pixels is achieved in the spectral domain. In addition, morphological filtering in the spatial domain is used to extract spatial structural information from anomalies. Joint spatial-spectral anomaly detection is achieved by combining multiscale memory autoencoder and spatial filtering. Experiments demonstrate superior detection performance of the proposed method over the state-of-the-art methods.

Hyperspectral image (HSI) clustering has attracted increasing attention in recent years, because it doesn’t rely on labeled pixels. However, it is a challenging task due to the complex spectral-spatial structure. The emergence of large-scale HSIs introduces a new challenge in terms of heightened computational complexity. To address the above challenges, in this paper, we propose a structured anchor projected clustering (SAPC) model for large-scale HSIs. Specifically, we exploit spatial information reflecting in the generated superpixels to perform denoising and generate anchors. Based on the preprocessing, we simultaneously learn a pixel-anchor graph and an anchor-anchor graph in a projected feature space. Meanwhile, the rank-constraint is imposed on the Laplacian matrix related to the anchor-anchor graph. To uncover the clustering structure, we design a clustering inference strategy to propagate clustering labels from anchors to pixels based on the dual graphs. Additionally, we propose an efficient optimization strategy for the formulated SAPC model with linear time complexity in terms of the number of pixels. Since the anchor-anchor graph is with much smaller size, it is high efficient to obtain the structured anchors with pseudo labels. Thus, the clustering process is significantly accelerated. Extensive experiments on multiple large-scale HSI datasets demonstrates the superiority of our SAPC over the state-of-the-art methods. The source code is released athttps://github.com/ZhangYongshan/SAPC.

Hyperspectral image (HSI) clustering is challenging to partition pixels into different clusters due to the complex spatial distribution and high-correlated spectrum. Subspace clustering is a representative learning paradigm and has shown competitive performance in HSIs. Most existing methods ignore potential spatial or structural information and show difficulties in dealing with large-scale HSIs. In this paper, we propose an elastic graph fusion subspace clustering (EGFSC) framework that can flexibly incorporate spectral, spatial and structural information for large HSI clustering. Instead of performing pixel-level learning, superpixel-level learning is conducted according to the generated superpixels to lessen computation burden and memory cost. To explore structural information in two perspectives, a superpixel graph and a band graph are constructed based on the superpixel features. Considering the incompatible sizes of the two graphs, we present three effective dual graph fusion strategies to fuse them in different ways. With these graph fusion strategies, EGFSC is able to improve clustering performance by simultaneously considering spatial and structural information. To solve the proposed framework, we present a closed-form solution for easy implementation. Experiments demonstrate that the proposed EGFSC obtains 70.08%, 75.76%, 87.28% and 77.23% clustering accuracies on the four HSI datasets and outperforms the state-of-the-art methods. The source code is released athttps://github.com/ZhangYongshan/EGFSC.

Pansharpening of multispectral images aims to merge multispectral (MS) and panchromatic (PAN) images to produce high-resolution multispectral (HRMS) images. Unsupervised pansharpening algorithms, which are widely used in deep learning for pansharpening tasks, commonly employ generative adversarial networks (GANs). However, existing unsupervised methods based on GANs have some limitations: 1) restricted ability for joint preservation of spatial and spectral information, and 2) the receptive field of general convolutions restricts the extraction of long-range dependencies. To address these issues, we propose a frequency-guided generative adversarial networks for unsupervised pansharpening (FG-GAN). Our unsupervised framework uses high-frequency and low-frequency information as prior constraints to guide the training of the FG-GAN’s generator and discriminator networks, thereby enhancing the joint preservation of spatial and spectral details. Furthermore, a graph convolution-based generator network is designed, in which long-range edge dependencies are extracted and propagated by learning the relationships between distant edge feature nodes. Extensive experiments on the Quickbird and Gaofen-2 datasets demonstrate the effectiveness of our design: our method enhances spatial details and reduces the spatial distortion index (Ds) by 44.8%, while achieving the fidelity of spatial-spectral information with a HQNR score of 0.99.

Hyperspectral image (HSI) classification is a key task in the field of remote sensing, aiming to assign category labels to each pixel by leveraging the spectral and spatial information in HSIs. Recently, many deep learning (DL) methods, such as convolutional neural networks (CNNs) and Transformers, have been applied to this task, achieving significant results. However, most existing patch-based DL methods often overlook the potential relationships between the central pixel and its surrounding pixels. Additionally, the unique spectral characteristics of HSIs, such as the high correlation between adjacent spectral bands and low dependence between distant bands, also require special attention. Based on this, we propose a novel dual-branch convolution-Transformer network with spectral-spatial attention (CTSSA), which can effectively aggregate both local and global spectral-spatial features. Specifically, CTSSA comprises two core modules: the Pyramid Spectral Attention Module (PSAM) and the Center Transformer Encoder (CenterTE). The former extracts highly discriminative spectral features through a hierarchical multi-scale attention mechanism, capturing subtle differences between adjacent spectral bands. The latter improves the original Transformer encoder (TE) by introducing a center-attention mechanism to model the global relationship between the central pixel and its surrounding pixels, thereby enhancing classification accuracy while reducing computational complexity. Experimental results on four public datasets (Salinas, Pavia University, Houston, and WHUHi-LongKou) demonstrate that, compared with nine other networks, CTSSA achieves satisfactory performance with fewer parameters and relatively high efficiency.

Optical motion capture (MoCap) is the "gold standard" for accurately capturing full-body motions. To make use of raw MoCap point data, the system labels the points with corresponding body part locations and solves the full-body motions. However, MoCap data often contains mislabeling, occlusion and positional errors, requiring extensive manual correction. To alleviate this burden, we introduce RoMo, a learning-based framework for robustly labeling and solving raw optical motion capture data. In the labeling stage, RoMo employs a divide-and-conquer strategy to break down the complex full-body labeling challenge into manageable subtasks: alignment, full-body segmentation and part-specific labeling. To utilize the temporal continuity of markers, RoMo generates marker tracklets using a K-partite graph-based clustering algorithm, where markers serve as nodes, and edges are formed based on positional and feature similarities. For motion solving, to prevent error accumulation along the kinematic chain, we introduce a hybrid inverse kinematic solver that utilizes joint positions as intermediate representations and adjusts the template skeleton to match estimated joint positions. We demonstrate that RoMo achieves high labeling and solving accuracy across multiple metrics and various datasets. Extensive comparisons show that our method outperforms state-of-the-art research methods. On a real dataset, RoMo improves the F1 score of hand labeling from 0.94 to 0.98, and reduces joint position error of body motion solving by 25%. Furthermore, RoMo can be applied in scenarios where commercial systems are inadequate. The code and data for RoMo are available at https://github.com/non-void/RoMo.

Anomaly detection for hyperspectral images (HSIs) is a challenging problem to distinguish a few anomalous pixels from a majority of background pixels. Most existing methods cannot simultaneously explore both structural and spatial information from global and local perspectives. In this letter, we propose a stacked graph fusion denoising autoencoder (SGFDAE) for hyperspectral anomaly detection. Specifically, the global and local graphs are constructed from an HSI to explore potential structural and spatial information. With the designed graph fusion strategy, an advanced graph denoising autoencoder with deep architecture is developed in a hierarchical manner. To achieve better reconstruction and detection, a greedy layerwise unsupervised pretraining strategy is presented for network training. Experiments show that SGFDAE achieves 97.17%, 98.43%, and 98.90% detection accuracies by averaging the results of the datasets from three different scenes and outperforms the state-of-the-art methods.

Multimodal remote sensing image recognition aims to identify a category of land cover for every pixel with consistency and complementary information provided by different modalities. Most existing methods perform land cover recognition in a supervised manner with explicit label guidance. It is challenging to perform recognition without label guidance due to the complex spatial distribution and modality incompatibility, especially for large-scale data. In this article, we propose a dual graph learning affinity propagation (DGLAP) method for multimodal remote sensing image clustering. Based on the consistent spatial distribution from local regions, the proposed method learns an$N \times M$consensus anchor graph from N denoised pixels and M anchors by adaptive weighting different modalities along with projection learning. Meanwhile, an optimal$M \times M$compressed consensus anchor graph is learned from the updated anchors in different modalities with diverse adaptive contributions and connectivity constraint. Since$M \ll N$, clustering results can be efficiently obtained according to affinity propagation from the pseudolabeled anchors to the pixels without additional steps. An alternating optimization algorithm is devised to solve the proposed formulation. This is the first attempt to propose a ultraefficient graph-based clustering method with linear time complexity$\mathcal {O}(N)$and low time cost for large-scale multimodal remote sensing data. Extensive experiments on three datasets demonstrate the superiority of the proposed method over the state-of-the-art methods in both efficacy and efficiency. The code is released athttps://github.com/ZhangYongshan/DGLAP.

Hyperspectral image (HSI) clustering seeks to assign each pixel to a specific class without trained labels. This is a challenging task owing to the spatial and spectral complexity. Recently, anchor graph-based clustering has attracted considerable attention due to its flexibility in handling large-scale HSI data. However, these methods typically disregard noisy bands and require post-processing. To tackle these issues, we propose a structured-anchor projected clustering (SAPC) model for HSIs. In SAPC, the projection clustering is introduced into anchor graph learning to suppress noise, and the Laplacian rank constraints can quickly obtain the structure of anchors. Thus, we can directly obtain the clustering results through the anchor graph and the structured anchors. Moreover, we propose an iterative optimization method to efficiently solve the SAPC model. Extensive experiments show that our model achieves superior results.

Hyperspectral anomaly detection (HAD) aims to discern the objects deviated dramatically from their surrounding pixels. Some deep learning-based models integrating with the low-rank representation (LRR) have been proposed recently. The process of constructing dictionary in these methods is complex and the stability of the models is hard to maintain. To address these problems, in this paper, we propose a low-rank constrained memory autoencoder (LRMAE) for HAD. Specifically, we first train the memory autoencoder with a sparsity regularizer and a spectral consistency constraint in an unsupervised learning fashion and the embedded memory module is used to acquire the dictionary atoms for the construction of dictionary. The low-rank optimization process is conducted in the low-dimensional manifold space to obtain the final detection map. Substantial experiments are performed on three different datasets, and the final detection results demonstrate the superiority of the proposed model over other state-of-the-art methods.

Hyperspectral band selection has been proved to be effective in reducing redundant information for hyperspectral images (HSIs). Most existing band selection methods simply consider the relationship between bands by reshaping them into vectors and destroying the spatial structure. Moreover, the converted band vectors are usually high-dimensional, making the learning processing very time-consuming. To solve these problems, we propose a tensor decomposition based latent feature clustering (TDLFC) model for band selection. We maintain the tensor structure of the HSI and use CANDECOMP/PARAFAC (CP) decomposition to learn the latent low-dimensional representation of the bands to preserve spatial and spectral information. To avoid overfitting, we introduce a regularization term for the CP decomposition model. To solve the proposed model, we present an effective optimization algorithm as solution. Finally, the k-means algorithm is applied to the latent representation to get the band clustering results for band selection. Extensive experiments on three public HSI datasets show the superiority of our proposed model over the state-of-the-art methods.

We present a novel locality-based learning method for cleaning and solving optical motion capture data. Given noisy marker data, we propose a new heterogeneous graph neural network which treats markers and joints as different types of nodes, and uses graph convolution operations to extract the local features of markers and joints and transform them to clean motions. To deal with anomaly markers (e.g. occluded or with big tracking errors), the key insight is that a marker’s motion shows strong correlations with the motions of its immediate neighboring markers but less so with other markers, a.k.a. locality, which enables us to efficiently fill missing markers (e.g. due to occlusion). Additionally, we also identify marker outliers due to tracking errors by investigating their acceleration profiles. Finally, we propose a training regime based on representation learning and data augmentation, by training the model on data with masking. The masking schemes aim to mimic the occluded and noisy markers often observed in the real data. Finally, we show that our method achieves high accuracy on multiple metrics across various datasets. Extensive comparison shows our method outperforms state-of-the-art methods in terms of prediction accuracy of occluded marker position error by approximately 20%, which leads to a further error reduction on the reconstructed joint rotations and positions by 30%. The code and data for this paper are available at https://github.com/non-void/LocalMoCap.

Hyperspectral image (HSI) clustering has attracted great attention in the field of remote sensing. General anchor-based clustering methods often suffer from the problems of unstable anchor selection and insufficient utilization of spatial information, resulting in poor clustering performance. In this letter, a spectral-spatial superpixel anchor graph-based clustering (S3AGC) method is proposed for HSIs. Specifically, S3AGC further improves the clustering performance by simultaneously considering spatial and structural information as well as an advanced anchor selection strategy. Based on the spatial distribution, a useful HSI denoising solution is presented to reduce noise interference, and an effective anchor selection strategy is raised to alleviate the instability of random or clustering selection. Besides, we use a graph convolution method to embed structural information of spectral bands into the proposed framework. Experiments on HSI datasets verify the effectiveness of S3AGC.

Dimensionality reduction (DR) is important for feature extraction and classification of hyperspectral images (HSIs). Recently proposed superpixel-based DR models have shown promising performance, where superpixel segmentation techniques were applied to segment an HSI and then DR models like principal component analysis (PCA) or linear discriminant analysis (LDA) were employed to extract the local and/or global features. However, superpixelwise PCA based local features are unsatisfactory because PCA aims to extract features with high variance, which could be inefficient in superpixels with mixed objects or strong noise/outliers. In addition, superpixelwise unsupervised LDA based global features may neglect local (spatial-contextual) information. To address these issues, we propose a new spectral-spatial and superpixelwise unsupervised LDA (S3-ULDA) model for unsupervised feature extraction from HSIs. Specifically, the HSI is first segmented into various superpixels with pseudo labels. Then, superpixel based local reconstruction for HSI denoising is conducted. Next, superpixelwise unsupervised LDA (SuperULDA) is performed on both the original HSI and locally reconstructed data to extract global features. Then, superpixelwise unsupervised local Fisher discriminant analysis (SuperULFDA) is developed for local feature extraction, where each superpixel and its adjacent superpixels (along with their pseudo-labels) are fed into local Fisher discriminant analysis (LFDA) to extract local features. The superpixel-level local manifold structures can be effectively modeled by the proposed SuperULFDA. Finally, by fusing the extracted global and local features, novel global-local and spectral-spatial features can be obtained. Our experimental results on several benchmark HSIs demonstrate the superiority of the proposed method over state-of-the-art methods. The code of the proposed model is available at https://github.com/XinweiJiang/S3-ULDA.

Hyperspectral band selection aims to identify an optimal sub-set of bands from hyperspectral images (HSIs). Most existing methods explore the relationships between pair-wise pixels in a fixed graph. However, the quality of the initial fixed graph may be influenced by noises and user-defined parameters that may not be optimal for HSI analysis. In this paper, we pro-pose a graph learning based autoencoder (GLAE) to achieve unsupervised hyperspectral band selection. Using the relationships of pair-wise pixels within HSIs, GLAE constructs the initial graph to characterize the geometric structures of HSIs and then adjusts the graph to adapt the band selection process. To solve the proposed model, we intoduce an alternative optimization algorithm. Experiments and comparisons on three HSI datasets demonstrate that the proposed GLAE achieves better results over the state-of-the-art methods.

Superpixel based label propagation models have been successfully used for Hyperspectral Images (HSIs) classification especially when the training data are limited. However, it is inevitable that there are segmentation errors leading to data in one superpixel containing samples from different classes which could decrease the classification accuracy. In order to address this issue, we propose Superpixel Correction based Label Propagation for HSIs classification. First, superpixel segmentation technique is adopted to segment a HSI into many superpixel blocks. Then, clustering model density peak is used to adaptively cluster the data in each superpixel block to correct the segmentation errors. Finally, based on the corrected superpixel segmentation we construct global-local and spatial-spectral similarity graphs which results into effective propagation matrix for label propagation. The proposed model is verified in two HSIs data sets, and the experimental results demonstrate that the proposed model is superior to several state-of-the-art methods in terms of classification accuracy, especially in the case of limited training samples.

Deep neural networks have gained increasing interest in hyperspectral image (HSI) processing. However, prior arts often neglect the high-order correlation among data points, failing to capture intraclass variations. In this letter, we present a unified neural network framework, termed as hypergraph-structured autoencoder (HyperAE), to leverage the high-order relationship among data and learn robust deep representation for downstream tasks. Technically, the proposed method adopts a deep autoencoder regularized by hypergraph structure as the backbone network, which is jointly trained with a task-specific branch, resulting in a multitask architecture. We separately combine the subspace clustering model and the softmax classifier into the HyperAE to deal with HSI unsupervised and semisupervised classification problems. Benefiting from the hypergraph, HyperAE endows traditional networks with the capacity of preserving the high-order structured information. We evaluate the proposed methods on three benchmarking HSI data sets, demonstrating that the proposed HyperAE dramatically outperforms many existing methods with significant margins in both unsupervised and semisupervised HSI classification problems.

Hyperspectral images (HSIs) consisting of abundant spectral bands could lead to the curse of dimensionality issue when performing HSIs classification. In this letter, an unsupervised dimensionality reduction (DR) method termed Laplacian regularized collaborative representation projection (LRCRP) is proposed, where Laplacian regularization and local enhancement are introduced into collaborative representation (CR) to construct adjacent graph and then to reduce the spectral dimension in graph embedding framework. As the constructed graph simultaneously preserves the local manifold and global information in HSIs, the proposed LRCRP could be used to extract effective low-dimensional features for accurate HSIs classification. The experimental results on two HSI datasets demonstrate the effectiveness of the proposed model. The source code the proposed model is available athttps://github.com/XinweiJiang/LRCRP.

Autoencoder (AE) is an unsupervised neural network framework for efficient and effective feature extraction. Most AE-based methods do not consider spatial information and band correlations for hyperspectral image (HSI) analysis. In addition, graph-based AE methods often learn discriminative representations with the assumption that connected samples share the same label and they cannot directly embed the geometric structure into feature extraction. To address these issues, in this paper, we propose a dual graph autoencoder (DGAE) to learn discriminative representations for HSIs. Utilizing the relationships of pair-wise pixels within homogenous regions and pair-wise spectral bands, DGAE first constructs the superpixel-based similarity graph with spatial information and band-based similarity graph to characterize the geometric structures of HSIs. With the developed dual graph convolution, more discriminative feature representations are learnt from the hidden layer via the encoder-decoder structure of DGAE. The main advantage of DGAE is that it fully exploits both the geometric structures of pixels with spatial information and spectral bands to promote nonlinear feature extraction of HSIs. Experiments on HSI datasets show the superiority of the proposed DGAE over the state-of-the-art methods. The source code of DGAE is available athttps://github.com/ZhangYongshan/DGAE.

Convolutional neural networks (CNNs) are widely used in hyperspectral image (HSI) classification. However, the network architecture of CNNs is often designed manually, which requires careful fine-tuning. Recently, many techniques for neural architecture search (NAS) have been proposed to design the network automatically but most of the methods are only concerned with the overall classification accuracy and ignore the balance between the floating point operations per second (FLOPs) and the number of parameters. In this paper, we propose a new multi-objective optimization (MO) method called MO-CNN to automatically design the structure of CNNs for HSI classification. First, a MO method based on continuous particle swarm optimization (CPSO) is constructed, where the overall accuracy, floating point operations (FLOPs) and the number of parameters are considered, to obtain an optimal architecture from the Pareto front. Then, an auxiliary skip connection strategy is added (together with a partial connection strategy) to avoid performance collapse and to reduce memory consumption. Furthermore, an end-to-end band selection network (BS-Net) is used to reduce redundant bands and to maintain spectral-spatial uniformity. To demonstrate the performance of our newly proposed MO-CNN in scenarios with limited training sets, a quantitative and comparative analysis (including ablation studies) is conducted. Our optimization strategy is shown to improve the classification accuracy, reduce memory and obtain an optimal structure for CNNs based on unbiased datasets.

As the most classical unsupervised dimension reduction algorithm, principal component analysis (PCA) has been widely used in hyperspectral images (HSIs) preprocessing and analysis tasks. Recently proposed superpixelwise PCA (SuperPCA) has shown promising accuracy where superpixels segmentation technique was first used to segment an HSI to various homogeneous regions and then PCA was adopted in each superpixel block to extract the local features. However, the local features could be ineffective due to the neglect of global information especially in some small homogeneous regions and/or in some large homogeneous regions with mixed ground truth objects. In this article, a novel spectral–spatial and SuperPCA (S3-PCA) is proposed to learn the effective and low-dimensional features of HSIs. Inspired by SuperPCA we further adopt superpixels-based local reconstruction to filter the HSIs and use the PCA-based global features as the supplement of local features. It turns out that the global–local and spectral–spatial features can be well exploited. Specifically, each pixel of an HSI is reconstructed by the nearest neighbors’ pixels in the same superpixel block, which could eliminate the noise and enhance the spatial information adaptively. After the local reconstruction-based data preprocessing, PCA is performed on each region and the entire HSI to obtain local and global features, respectively. Then we simply concatenate them to get the global–local and spectral–spatial features for HSIs classification. The experimental results on two HSIs data sets demonstrate the superiority of the proposed method over the state-of-the-art methods. The source code of the proposed model is available athttps://github.com/XinweiJiang/S3-PCA.

Unsupervised band selection is an essential step in preprocessing hyperspectral images (HSIs) to select informative bands. Most existing methods exploit the spatial information from the entire HSI while ignoring the difference between diverse homogeneous regions. Moreover, traditional methods utilize the limited size of data for model training that may result in degraded generalization performance. In this article, we propose a marginalized graph self-representation (MGSR) method for unsupervised hyperspectral band selection. To explore the spatial information from diverse homogenous regions, MGSR generates the segmentations of an HSI by superpixel segmentation and records the relationships between adjacent pixels of the same segmentation in a structural graph. Meanwhile, to improve the generalization and robustness, infinite corrupted samples are obtained from the original pixels by introducing noises in spectral bands for model training. To solve the proposed formulation, we design an alternating optimization algorithm to marginalize out the corruption and search for the optimal solution. Experimental studies on HSI datasets demonstrate the effectiveness of the proposed MGSR and the superiority over the state-of-the-art methods. The source code is available athttps://github.com/ZhangYongshan/MGSR.

Unsupervised band selection aims to select informative spectral bands to preprocess hyperspectral images (HSIs) without using labels. Traditional band selection methods only work well on Euclidean data, but ignore structural information of pixels and spectral bands. Moreover, they treat each HSI as a whole to exploit latent spatial information while ignoring the difference of spatial distribution between diverse homogeneous regions. In this paper, we propose a robust dual graph self-representation (RDGSR) method for unsupervised band selection. RDGSR uses superpixel segmentation technique to generate homogenous regions of each HSI to extract spatial information. Based on the segmentation result, the superpixel-based similarity graph and band-based similarity graph are constructed from HSIs to record spatial and structural information. With this knowledge, the dual graph convolution is developed and thel2,1-norm is introduced in the loss function and regularization term to eliminate the noise in rows for robust and effective band selection. The novelty of RDGSR is the joint utilization of the geometric structure of pixels with spatial consistency and the geometric structure of spectral bands to enhance the performance of band selection in a robustl2,1-norm manner. An iterative optimization algorithm is designed to solve the proposed formulation. Substantial experiments on HSI datasets are conducted to verify the superiority of the proposed RDGSR over the state-of-the-art methods. The source code is available at https://github.com/ZhangYongshan/RDGSR.

We present a fully automatic system that can produce high-fidelity, photo-realistic three-dimensional (3D) digital human heads with a consumer RGB-D selfie camera. The system only needs the user to take a short selfie RGB-D video while rotating his/her head and can produce a high-quality head reconstruction in less than 30 s. Our main contribution is a new facial geometry modeling and reflectance synthesis procedure that significantly improves the state of the art. Specifically, given the input video a two-stage frame selection procedure is first employed to select a few high-quality frames for reconstruction. Then a differentiable renderer-based 3D Morphable Model (3DMM) fitting algorithm is applied to recover facial geometries from multiview RGB-D data, which takes advantages of a powerful 3DMM basis constructed with extensive data generation and perturbation. Our 3DMM has much larger expressive capacities than conventional 3DMM, allowing us to recover more accurate facial geometry using merely linear basis. For reflectance synthesis, we present a hybrid approach that combines parametric fitting andConvolutional Neural Networks (CNNs)to synthesize high-resolution albedo/normal maps with realistic hair/pore/wrinkle details. Results show that our system can produce faithful 3D digital human faces with extremely realistic details. The main code and the newly constructed 3DMM basis is publicly available.

Hyperspectral image (HSI) clustering is a challenging task due to the high complexity of HSI data. Subspace clustering has been proven to be powerful for exploiting the intrinsic relationship between data points. Despite the impressive performance in the HSI clustering, traditional subspace clustering methods often ignore the inherent structural information among data. In this article, we revisit the subspace clustering with graph convolution and present a novel subspace clustering framework called graph convolutional subspace clustering (GCSC) for robust HSI clustering. Specifically, the framework recasts the self-expressiveness property of the data into the non-Euclidean domain, which results in a more robust graph embedding dictionary. We show that traditional subspace clustering models are the special forms of our framework with the Euclidean data. On the basis of the framework, we further propose two novel subspace clustering models by using the Frobenius norm, namely efficient GCSC (EGCSC) and efficient kernel GCSC (EKGCSC). Each model has a globally optimal closed-form solution, making it easier to implement, train, and apply in practice. Extensive experiments strongly evidence that EGCSC and EKGCSC dramatically outperform current models on three popular HSI data sets consistently.

Traditional bilateral filtering (BF) cannot extract hyperspectral image (HSI) features well when the center pixel of the neighborhood pixel set is a noise point in the process of filtering the HSI. In this letter, a trilateral smooth filtering (TRSF) is presented. The proposed algorithm avoids the above-mentioned limitation problem in the BF algorithm. TRSF is successfully applied to the feature extraction of three actual HSIs. To prove the effectiveness of the proposed algorithm, support vector machines are used to classify the extracted features. Experimental results show that the proposed feature extraction method is simple and effective.

Dimensionality Reduction (DR) which tries to discover low-dimensional feature representation embedded into the high-dimensional observations are significant for data visualization and data preprocessing. However, most DR models are designed for vector-valued data while only few of them are for functional data where samples are considered as continuous data such as curves or surfaces compared to discrete vector-valued data. Motivated by Functional Principal Component Analysis (FPCA), which generalizes the idea of Principal Component Analysis (PCA) to the Hilbert space of square-integrable functions, in this paper we propose Functional Locality Preserving Projection (FLPP), where classic Locality Preserving Projection (LPP) is extended for functional data analysis. Different from FPCA which only focuses on the global structure, FLPP could preserve local manifold structure embedded into the functional data, thus FLPP is capable of dealing with noise data. Experimental results on both synthetic data and real-world data verify that FLPP outperforms FPCA and typical LPP.

Content centric networking (CCN) is a promising architecture for the future networks. In-networking caching of CCN can significantly improve the content delivery efficiency. Content popularity is one of the key factors considered in the design of the caching strategy. However, the existing research ignores the timeliness of content popularity statistics, which makes the changing of popular contents cached in the network lag behind the changing of the user preference. In this paper, a content clustering and popularity prediction based caching strategy (CPC) is proposed to solve this problem. Firstly, the massive contents are divided into different content types using cluster analysis. Then, the popularity of different content types is predicted by the autoregressive integrate moving average (ARIMA) model. Finally, based on the predicted content popularity, the process of the caching placement decision is given. The proposed caching strategy is a distributed caching management method, which can be implemented without centralized controller. Simulation results show that the proposed caching strategy can achieve better performance in terms of cache replacement rate, cache hit ratio and average hop count.

Representation-residual-based classifiers have attracted much attention in recent years in hyperspectral image (HSI) classification. How to obtain the optimal representa-tion coefficients for the classification task is the key problem of these methods. In this letter, spatial-aware collaborative representation (CR) is proposed for HSI classification. In order to make full use of the spatial-spectral information, we propose a closed-form solution, in which the spatial and spectral features are both utilized to induce the distance-weighted regularization terms. Different from traditional CR-based HSI classification algorithms, which model the spatial feature in a preprocessing or postprocessing stage, we directly incorporate the spatial information by adding a spatial regularization term to the representation objective function. The experimental results on three HSI data sets verify that our proposed approach outperforms the state-of-the-art classifiers.

Discriminative features are significant for hyper-spectral image (HSI) classification. In this letter, we apply the supervised dimensionality reduction (DR) model termed supervised latent linear Gaussian process latent variable model (SLLGPLVM) for feature extraction. As a semiparametric classification model, the new model has ability in simultaneous feature extraction and classification and demonstrates high classification accuracy with only a small training set. This is therefore suitable for HSI classification. Experimental results on six real HSI data sets show that the proposed SLLGPLVM outperforms several conventional supervised DR models and the support vector machine implemented in the original spectral space.

Face image super-resolution has attracted much attention in recent years. Many algorithms have been proposed. Among them, sparse representation (SR)-based face image super-resolution approaches are able to achieve competitive performance. However, these SR-based approaches only perform well under the condition that the input is noiseless or has small noise. When the input is corrupted by large noise, the reconstruction weights (or coefficients) of the input low-resolution (LR) patches using SR-based approaches will be seriously unstable, thus leading to poor reconstruction results. To this end, in this paper, we propose a novel SR-based face image super-resolution approach that incorporates smooth priors to enforce similar training patches having similar sparse coding coefficients. Specifically, we introduce the fused least absolute shrinkage and selection operator-based smooth constraint and locality-based smooth constraint to the least squares representation-based patch representation in order to obtain stable reconstruction weights, especially when the noise level of the input LR image is high. Experiments are carried out on the benchmark FEI face database and CMU+MIT face database. Visual and quantitative comparisons show that the proposed face image super-resolution method yields superior reconstruction results when the input LR face image is contaminated by strong noise.

Low Light Level Images (LLLIs) are captured with exceptionally low brightness and low contrast, and cannot be enhanced satisfactorily with ordinary methods. In this paper, we propose a LLLI enhancement method using coupled dictionary learning. During the training stage, a pair of dictionaries and a linear mapping function are learned simultaneously. The dictionary pair aims to describe the raw LLLIs and their enhanced versions, and the linear mapping function models the correspondence between the representations of the dictionary pair. In the enhancement process, the resulting image is generated through dictionary mapping from patches of the input LLLI. We adopt a clustering strategy to improve the robustness of coupled dictionary learning, and propose an improved algorithm for fast implementation. Experimental results on real images demonstrate the effectiveness of our method.

Unsupervised learning aims to discovery latent representation embedded in the observation, which is useful for data visualization, dimensionality reduction, and density modeling. Autoencoders have been successfully used to learn the latent variations in data, especially with the recent reintroduction by deep learning. For some specific tasks, there are supervised information or labels that can be used to further guide the unsupervised autoencoder model for finding latent representation. The Non-Parametrically Guided Autoencoder (NPGA) has been proved to be an effective model. It tries to utilize Gaussian Process Regression (GPR) to model the unknown mapping from unknown latent representation to extra supervised information. However for the discrete label information in classification tasks, using GPR could be unwise and inefficient. In this paper, we propose the Non-Parametrically Guided Autoencoder with Laplace Approximation (NPGA-LA) to effectively handle discrete labels. The idea of NPGA-LA is to make use of Gaussian Process Classification (GPC) rather than GPR to model the transformation between the latent space and the discrete label space. The experimental results verify the excellent performance of the newly developed method.

Image super resolution (SR) aims at recovering the missing high frequency details from single image or multiple images. Existing SR methods can be divided into three categories: interpolation-based, reconstructionbased and example learning-based. Our paper focuses on the third category. Example learning-based SRmethods [6] utilize the LR-HR image pair to infer the missing high-frequency details in the LR image and achieve state-of-the-art performance.Recently, in the field of example learningbased SR, more and more researchers resort to learn the LR-HR relationship directly, i.e. y = f (x), where x is the input LR image feature, y is the targeted HR image and f is the mapping function that transforms the LR feature into HR image. Instead of commonly used parametric models, non-parametric methods [3], especially gaussian process regression (GPR)-related methods [2, 4, 5] begin to emerge in the SR field. However, previous GPR-based SR methods simply learn all the GPR models independently and ignore the correlation between them. On the other hand, each pixel prediction can be treated as a task, so that inferring a HR patch can be regarded as a multi-task problem. In this paper, we focus on the multi-task gaussian process (MTGP) regression and apply it to superresolution problem. We first give a brief overview of MTGP proposed in [1]. Then we study how SR problem corresponds to MTGP and propose the multi-task gaussian process super-resolution (MTGPSR) framework. MTGP tries to solve the following problem: Given N distinct inputs x1, ...,xN we define the complete set of responses for M tasks as y = (y11, ...,yN1, ...,y12, ...,yN2, ...,y1M , ...,yNM) , where yi j is the response for the jth task on the ith input xi. We also denote the N ×M matrix Y such that y = vecY . Given a set of observations yo, which is a subset of y, we wish to predict the unobserved values of yu of some input points for some tasks. MTGP wishes to learn M related latent functions { fl} by placing a GP prior over { fl} and directly induce correlations between tasks. Assuming that the GPs have zero mean we define ⟨ fl(x) fk(x ′) ⟩ = K f lkk x(x,x′) (1)

Remote sensing images often suffer from shadow duo to partially or totally occludes direct light from an illumination source. In this paper, we propose a novel shadow removal algorithm. The trimap could be generated automatically by morphological subtraction method according to the result of shadow detection. Then, the weighted color and texture sample selection image matting method is applied in order to obtain the accurate shadow coefficient. The experiment results are illustrated with practical examples and verify the efficacy of this algorithm.

Nonnegative matrix factorization (NMF) has been applied to hyperspectral unmixing in recent years. Different constraints based on geometrical or statistical properties of end-member and abundance are incorporated into NMF model to improve unmixing result. In this paper, a new regularizer based on spectral cluster information is proposed to strengthen the constrained relationship between original image and abundance maps. The new algorithm makes abundances of similar pixels close and abundances of dissimilar pixels be separated completely. Additionally, L1/2sparsity constraint is adopted to make the solutions sparse. Comparative results on real and synthetic hyperspectral datasets prove our proposed method could improve the hyperspectral unmixing accuracy.

Hyperspectral Images often suffer from missing pixels due to acquisition system problem. Missing pixels in images are usually tackled by means of interpolation methods by using neighborhood known pixels, but once the missing region is large, there may not be sufficient information in the neighborhood to reconstruct well. Fortunately, hyperspectral sensors are also often flown with boresighted, higher resolution panchromatic sensors. We propose a novel hyperspectral data recovery algorithm based on multi-source image fusion, guided by the gradient field of coincident panchromatic image in inpainting processing. The experiment results are illustrated with practical examples and verify the efficacy of this algorithm.

Dimensionality reduction (DR) has been considered as one of the most significant tools for data analysis. One type of DR algorithms is based on latent variable models (LVM). LVM-based models can handle the preimage problem easily. In this paper we propose a new LVM-based DR model, named thin plate spline latent variable model (TPSLVM). Compared to the well-known Gaussian process latent variable model (GPLVM), our proposed TPSLVM is more powerful especially when the dimensionality of the latent space is low. Also, TPSLVM is robust to shift and rotation. This paper investigates two extensions of TPSLVM, i.e., the back-constrained TPSLVM (BC-TPSLVM) and TPSLVM with dynamics (TPSLVM-DM) as well as their combination BC-TPSLVM-DM. Experimental results show that TPSLVM and its extensions provide better data visualization and more efficient dimensionality reduction compared to PCA, GPLVM, ISOMAP, etc.

Dimensionality reduction (DR) has been considered as one of the most significant tools for data analysis. In this paper we propose a new latent variable model based on the thin plate splines, named Thin Plate Spline Latent Variable Model (TPSLVM). It has strong connection with the so-called Gaussian Process Latent Variable Model (GPLVM). We demonstrate that the proposed TPSLVM can be viewed as the GPLVM with a fairly peculiar covariance function. Moreover, compared to GPLVM, TPSLVM is more powerful especially when the dimensionality of the latent space is very low (e.g., 2D or 3D). One of main purposes of DR algorithms is to visualize data in 2D/3D spaces. Therefore, TPSLVM will benefit this process. Experimental results show that TPSLVM provides better data visualization and more efficient dimensionality reduction than GPLVM.

The Gaussian process (GP) latent variable model (GPLVM) has the capability of learning low-dimensional manifold from highly nonlinear data of high dimensionality. As an unsupervised dimensionality reduction (DR) algorithm, the GPLVM has been successfully applied in many areas. However, in its current setting, GPLVM is unable to use label information, which is available for many tasks; therefore, researchers proposed many kinds of extensions to the GPLVM in order to utilize extra information, among which the supervised GPLVM (SGPLVM) has shown better performance compared with other SGPLVM extensions. However, the SGPLVM suffers in its high computational complexity. Bearing in mind the issues of the complexity and the need of incorporating additionally available information, in this paper, we propose a novel SGPLVM, called supervised latent linear GPLVM (SLLGPLVM). Our approach is motivated by both SGPLVM and supervised probabilistic principal component analysis (SPPCA). The proposed SLLGPLVM can be viewed as an appropriate compromise between the SGPLVM and the SPPCA. Furthermore, it is also appropriate to interpret the SLLGPLVM as a semiparametric regression model for supervised DR by making use of the GP to model the unknown smooth link function. Complexity analysis and experiments show that the developed SLLGPLVM outperforms the SGPLVM not only in the computational complexity but also in its accuracy. We also compared the SLLGPLVM with two classical supervised classifiers, i.e., a GP classifier and a support vector machine, to illustrate the advantages of the proposed model.