LiDAR point cloud semantic segmentation has emerged as a fundamental technique for enabling intelligent perception in autonomous driving, robotics, and geospatial analysis. Point cloud segmentation methods are typically categorized into three types: point-based, voxel-based, and projection-based techniques. While point-based and voxel-based methods offer robust feature extraction, they face challenges related to computational efficiency and the handling of large-scale point clouds. Projection-based methods, on the other hand, project 3D point clouds into 2D representations, enabling the application of established 2D convolutional neural networks (CNNs) for segmentation tasks. Despite their advantages in efficiency, projection-based methods often suffer from the loss of spatial precision, leading to suboptimal segmentation performance, especially in complex and cluttered environments. In this paper, we propose a novel projection-based approach for semantic segmentation that addresses the limitations of existing methods. Our approach introduces a Multi-Scale Feature Embedding (MSFE) module to enhance feature extraction from the projected range images, combined with a Multi-Feature Fusion Module (MFFM) to integrate features at multiple scales. We further improve segmentation accuracy for challenging objects, such as pedestrians, traffic signs, and occluded structures, by utilizing a Dual Segmentation Head. Our experiments on the SemanticPOSS and SemanticKITTI datasets show significant improvements over existing methods, achieving a mean Intersection over Union (mIoU) of 53.6% on SemanticPOSS and 67.8% on SemanticKITTI. Notably, we achieve high performance on small and occluded objects, like trashcans (55.9% mIoU) and fences (49.5% mIoU), demonstrating the effectiveness of our approach for real-world applications like autonomous driving.

Weed and crop segmentation is becoming an increasingly integral part of precision farming that leverages the current computer vision and deep learning technologies. Research has been extensively carried out based on images captured with a camera from various platforms. Unmanned aerial vehicles (UAVs) and ground-based vehicles including agricultural robots are the two popular platforms for data collection in fields. They all contribute to site-specific weed management (SSWM) to maintain crop yield. Currently, the data from these two platforms is processed separately, though sharing the same semantic objects (weed and crop). In our paper, we have proposed a novel method with a new deep learning-based model and the enhanced data augmentation pipeline to train field images alone and subsequently predict both field images and UAV images for weed segmentation and mapping. The network learning process is visualized by feature maps at shallow and deep layers. The results show that the mean intersection of union (IOU) values of the segmentation for the crop (maize), weeds, and soil background in the developed model for the field dataset are 0.744, 0.577, 0.979, respectively, and the performance of aerial images from an UAV with the same model, the IOU values of the segmentation for the crop (maize), weeds and soil background are 0.596, 0.407, and 0.875, respectively. To estimate the effect on the use of plant protection agents, we quantify the relationship between herbicide spraying saving rate and grid size (spraying resolution) based on the predicted weed map. The spraying saving rate is up to 90% when the spraying resolution is at 1.78×1.78 cm2. The study shows that the developed deep convolutional neural network could be used to classify weeds from both field and aerial images and delivers satisfactory results. To achieve this performance, it is crucial to perform preprocessing techniques that reduce dataset differences between two distinct domains.

With the recent development of the joint classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data, deep learning methods have achieved promising performance owing to their locally sematic feature extracting ability. Nonetheless, the limited receptive field restricted the convolutional neural networks (CNNs) to represent global contextual and sequential attributes, while visual image transformers (VITs) lose local semantic information. Focusing on these issues, we propose a fractional Fourier image transformer (FrIT) as a backbone network to extract both global and local contexts effectively. In the proposed FrIT framework, HSI and LiDAR data are first fused at the pixel level, and both multisource feature and HSI feature extractors are utilized to capture local contexts. Then, a plug-and-play image transformer FrIT is explored for global contextual and sequential feature extraction. Unlike the attention-based representations in classic VIT, FrIT is capable of speeding up the transformer architectures massively and learning valuable contextual information effectively and efficiently. More significantly, to reduce redundancy and loss of information from shallow to deep layers, FrIT is devised to connect contextual features in multiple fractional domains. Five HSI and LiDAR scenes including one newly labeled benchmark are utilized for extensive experiments, showing improvement over both CNNs and VITs.

In this study, we propose a siamese recurrent residual refinement network (SR3Net) for change detection in high-resolution remote sensing images. SR3Net uses the siamese network to fully extract multi-scale features of bi-temporal images. The obtained multi-scale feature maps are input to the multi-level difference module (MDM) to generate difference feature maps. The residual refinement module (RRM) with residual refinement blocks (RRBs) learns the residual between the intermediate change map and the ground truth by alternately exploiting low-level and high-level integrated difference features. Moreover, RRBs can obtain complementary information of the intermediate predictions and add residuals to the intermediate prediction to refine the change map. Experiments on the WHU-CD dataset show that the proposed method outperforms state-of-the-art methods.

Deep learning has made great success in remote sensing scene classification since the powerful feature representation and complex nonlinear relationship learning. However, existing methods ignore the information redundancy and semantic ambiguity among them. To cope with this problem, we propose a multi-scale deep feature learning and semantic enhancement approach (MDFL-SE). First, we employ Pyramid Convolution PyConvResNet as the backbone to extract multilayer convolutional features. Then, a progressive deep feature aggregation module (PDFA) is designed to use high-level features to guide the low-level ones to choose the discriminative features. Finally, a global multiscale semantic extraction module (GMSE) and a grouped semantic extraction module (GSE) are combined to extract the channel and spatial information of multilayer fusion features. Experiments performed on AID and NWPURESISC45 RSSC datasets demonstrate that the proposed framework can obtain outstanding performance compared with state-of-the-art approaches.

Forest resources inventory and monitoring are essential for efficient management of natural resources. Remote sensing-based mapping technology provides geospatially explicit information on forest ecosystems. Generally, the attributes of forests derived from moderate satellite images show spatial and classification inaccuracy, introducing uncertainty in formulating forest dynamics and misleading inappropriate management plans. This study examines the efficiency of low-cost UAVs in generating orthoimages for differentiating bamboo from broadleaf species in forest succession. The results show that the generated very high-resolution orthoimage can reveal detailed spatial features of both bamboo and tree crowns, providing an excellent opportunity to differentiate individual bamboo crowns from trees and therefore help derive bamboo expansion and degradation over the forest.

Face detection is a fundamental task for numerous face-related applications (e.g., face recognition and age estimation), which directly affects the performance of the subsequent processing. Recent anchor-based face detectors have demonstrated the great potential by matching anchors and target boxes during training, which is crucial for high performance and training efficiency. However, existing anchor matching strategies still suffer from: 1) ignoring the inherent relationship between the targets and the anchors, which may cause unsuitable matched pairs, 2) adopting a fixed matching threshold, which cannot meet the varying demands of matched pairs for quality and quantity in different feature levels and training processes, and 3) the heuristic anchor setting, whose matching range is too narrow to capture about 20% target faces in the training dataset. This paper proposes an Adaptive Anchor Matching Strategy (AAMS) to address these issues, which selectively assigns proper targets to anchors in different feature levels by using adaptive matching thresholds and a robust anchor setting determined by the statistical characteristic of the training samples. Extensive experiments on popular benchmarks reveal that the proposed approach has significant improvements on anchor-based models and outperforms the recent state-of-the-arts methods in terms of both accuracy and generalization.

Complementary multimodal remote sensing (RS) data often leads to more robust and accurate classification performance. However, not all modal data can be available at the time of inference due to imaging conditions. To mitigate this issue, cross-modal knowledge distillation becomes an effective method, as it can leverage the complementary characteristics of multimodal data to guide cross-modal classification in cases with missing data. Therefore, this paper examines the shortcomings of traditional CNN cross-modal distillation methods in land cover classification: 1) insufficient knowledge representation; and 2) unstable knowledge transfer. Moreover, a novel cross-modal graph knowledge representation and distillation learning (CGKR-DL) framework is proposed to enhance land cover classification performance. The proposed CGKR-DL designs a single-stream joint feature learning network with convolutional neural network and graph convolutional network (CNN-GCN) to effectively construct the remote topology of data based on the strong correlation between land objects, thus enhancing the knowledge representation ability of the network. In addition, a multi-granularity graph distillation method is proposed to compensate for the inability of traditional CNN distillation in handling graph-structured information, where a feature distillation module based on graph discrimination (FD-GDM) is designed for stable graph feature distillation. We evaluate CGKR-DL on three publicly available multimodal RS datasets (HS-LiDAR, HS-SAR and HS-SAR-DSM) and achieve a significant improvement in comparison with several state-of-the-art methods.

To solve the limitation of labeled samples in hyperspectral image (HSI) classification, cross-scene learning methods are developed recently. However, the disparity caused by environmental variation between HSI scenes is still a challenge. As a supplement, light detection and ranging (LiDAR) data provides elevation and spatial information regardless the variations. In this paper, we propose a multisource cross-scene classification method using fractional fusion and spatial-spectral domain adaptation to reduce disparity between scenes. The spatial information of HSI is preserved by fractional differential masks (FrDM) firstly. Then the LiDAR data is utilized for spectral alignment of HSI. The utilization of LiDAR data reduces the pixel-level disparity between scenes. At last, a spatial-spectral domain adaptation network is proposed for feature extraction and classification. Experimental results on HSI and LiDAR scenes show 5% improvements in overall accuracy compared with state-of-the-art methods.

Focusing on joint classification of Hyperspectral image (HSI) and Light detection and ranging (LiDAR) data, a fractional Fourier image transformer (FrIT) is proposed as a backbone network in this paper. In the proposed FrIT, HSI and LiDAR data are firstly fused at pixel-level. Both multi-source and HSI feature extractors are utilized to capture local contexts. Then, a plug-and-play image transformer FrIT is explored for global contexts and sequential feature extraction. Unlike the attention-based representations in classic visual image transformer (VIT), FrIT is capable of speeding up the transformer architectures massively. To reduce the information loss from shallow to deep layers, FrIT is devised to connect contextual features in multiple fractional domains. At last, to evaluate the performance of FrIT, a new HSI and LiDAR benchmark is provided for extensive experiments, on which the proposed FrIT gains an improvement of 3% over state-of-the-art methods.

Pansharpening refers to a spatial–spectral contexts fusion procedure to produce high-quality multispectral (MS) images by retaining the fine spatial resolution of the panchromatic (PAN) images and the high spectral content of the MS images. This letter presents a novel end-to-end dual-branch deep learning-based fusion framework, exploiting the network to extract spatial and spectral contexts progressively in two separate branches level by level. For each level contexts extraction layer, a dual-branch weighted attentive fusion module is integrated to boost the important contexts aggregation and details injection while suppressing unimportant ones. Experimental results on two real datasets show that our method outperforms state-of-the-art methods in both objective metrics and image quality by visual appearance.

Band selection has become a significant issue for the efficiency of the hyperspectral image (HSI) processing. Although many unsupervised band selection (UBS) approaches have been developed in the last decades, a flexible and robust method is still lacking. The lack of proper understanding of the HSI data structure has resulted in the inconsistency in the outcome of UBS. Besides, most of the UBS methods are either relying on complicated measurements or rather noise sensitive, which hinder the efficiency of the determined band subset. In this article, an adaptive distance-based band hierarchy (ADBH) clustering framework is proposed for UBS in HSI, which can help to avoid the noisy bands while reflecting the hierarchical data structure of HSI. With a tree hierarchy-based framework, we can acquire any number of band subset. By introducing a novel adaptive distance into the hierarchy, the similarity between bands and band groups can be computed straightforward while reducing the effect of noisy bands. Experiments on four datasets acquired from two HSI systems have fully validated the superiority of the proposed framework.

Pansharpening is an image fusion procedure, which aims to produce a high spatial resolution multispectral image by combining a low spatial resolution multispectral image and a high spatial resolution panchromatic image. The most popular and successful paradigm for pansharpening is the framework known as detail injection, while it cannot fully exploit complex and non-linear complementary features of both images. In this paper, we propose a detail injection model inspired deep fusion network for pansharpening (DIM-FuNet). Firstly, by treating pansharpening as a complicated and non-linear details learning and injection problem, we establish a unified optimizing detail-injection model with triple detail fidelity terms: 1) a band-dependent spatial detail fidelity term, 2) a local detail fidelity term and 3) a complicated details synthesis term. Secondly, the model is optimized via the iterative gradient descent and unfolded into a deep convolutional neural network. Subsequently, the unrolling network has triple branches, in which, a point-wise convolutional sub-network, a depth-wise convolutional sub-network are corresponding to the former two detail constrained terms, and an adaptive weighted reconstruction module with a fusion sub-network to aggregate details of two branches and synthesis the final complicated details. Finally, the deep unrolling network is trained in end-to-end manners. Different from traditional deep fusion networks, the architecture design of DIM-FuNet is guided by the optimizing model and thus promotes better interpretability. Experimental results on reduced and full-resolution demonstrate the effectiveness of the proposed DIM-FuNet which achieves the best performance compared with the state-of-the-art pansharpening method.

Remote sensing using multisensor platforms has been systematically applied for monitoring and optimizing human activities. Several advanced techniques have been developed to enhance and extract the spatially and spectrally semantic information in the hyperspectral image (HSI) and light detection and ranging (LiDAR) data processing and analysis. However, an abundance of redundant information and sometimes a lack of discriminative features reduce the efficiency and effectiveness of multisource classification methods. This article proposes a fractional Gabor convolutional network (FGCN), focusing on efficient feature fusion and comprehensive feature extraction. First, the proposed FGCN uses Octave convolution layers to perform multisource information fusion and preserve discriminative information. Second, fractional Gabor convolutional (FGC) layers are proposed to extract multiscale, multidirectional, and semantic change features. The completeness and discrimination of the multisource features using different FGC kernels are improved, which yield robust feature extraction against semantic changes. Finally, the fractional Gabor feature and spectral feature are combined with two weighting factors which can be learned during the network training. Experimental results and comparisons with state-of-the-art multisource classification methods indicate the effectiveness of the proposed FGCN. With the FGCN, we can obtain an 89.90% overall accuracy on the challenging Muufl Gulfport (MUUFL) data set, with an improvement of 3% over state-of-the-art methods.

Existing methods for tensor completion (TC) have limited ability for characterizing low-rank (LR) structures. To depict the complex hierarchical knowledge with implicit sparsity attributes hidden in a tensor, we propose a new multilayer sparsity-based tensor decomposition (MLSTD) for the low-rank tensor completion (LRTC). The method encodes the structured sparsity of a tensor by the multiple-layer representation. Specifically, we use the CANDECOMP/PARAFAC (CP) model to decompose a tensor into an ensemble of the sum of rank-1 tensors, and the number of rank-1 components is easily interpreted as the first-layer sparsity measure. Presumably, the factor matrices are smooth since local piecewise property exists in within-mode correlation. In subspace, the local smoothness can be regarded as the second-layer sparsity. To describe the refined structures of factor/subspace sparsity, we introduce a new sparsity insight of subspace smoothness: a self-adaptive low-rank matrix factorization (LRMF) scheme, called the third-layer sparsity. By the progressive description of the sparsity structure, we formulate an MLSTD model and embed it into the LRTC problem. Then, an effective alternating direction method of multipliers (ADMM) algorithm is designed for the MLSTD minimization problem. Various experiments in RGB images, hyperspectral images (HSIs), and videos substantiate that the proposed LRTC methods are superior to state-of-the-art methods.

Due to the limitations of hyperspectral imaging systems, hyperspectral imagery (HSI) often suffers from poor spatial resolution, thus hampering many applications of the imagery. Hyperspectral super resolution refers to fusing HSI and MSI to generate an image with both high spatial and high spectral resolutions. Recently, several new methods have been proposed to solve this fusion problem, and most of these methods assume that the prior information of the point spread function (PSF) and spectral response function (SRF) are known. However, in practice, this information is often limited or unavailable. In this work, an unsupervised deep learning-based fusion method-HyCoNet-that can solve the problems in HSI-MSI fusion without the prior PSF and SRF information is proposed. HyCoNet consists of three coupled autoencoder nets in which the HSI and MSI are unmixed into endmembers and abundances based on the linear unmixing model. Two special convolutional layers are designed to act as a bridge that coordinates with the three autoencoder nets, and the PSF and SRF parameters are learned adaptively in the two convolution layers during the training process. Furthermore, driven by the joint loss function, the proposed method is straightforward and easily implemented in an end-to-end training manner. The experiments performed in the study demonstrate that the proposed method performs well and produces robust results for different data sets and arbitrary PSFs and SRFs.

Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named "structured sparse low-rank representation" (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.

To fuse hyperspectral and Light Detection And Ranging (LiDAR), we propose a semisupervised graph fusion (SSGF) approach. We apply morphological filters to LiDAR and the first few components of hyperspectral data to model the height and spatial information, respectively. Then, the proposed SSGF is used to project the spectral, elevation, and spatial features onto a lower subspace to obtain the new features. In particular, the objective of SSGF is to maximize the class separation ability and preserve the local neighborhood structure by using both labeled and unlabeled samples. Experimental results on the hyperspectral and LiDAR data from the 2013 IEEE Geoscience and Remote Sensing Society (GRSS) Data Fusion Contest demonstrated the superiority of the SSGF.

Spectral-spatial framework has been widely applied for hyperspectral image classification task. Some well-established models, such as group sparse representation (GSR), have gained a certain advance but still mainly focus on the usage of local spatial similarity and neglect the nonlocal spatial information. Recently, nonlocal self-similarity (NLSS) has been exploited to support the spatial coherence tasks. However, current NLSS-based methods are biased toward the direct use of nonlocal spatial information as a whole, while the underlying spectral information is not well exploited. In this article, we proposed a novel method to exploit local spectral similarity through nonlocal spatial similarity, with the integration of local spatial consistency in a single framework. Specifically, the proposed approach first exploits the NLSS by searching the nonoverlapped similar patches in defined scopes. Then, spectral similarity is determined locally within the found patches. After that, the found similar data and the original data are fused in a designed pattern. Finally, the GSR-based classifier (GSRC) is applied to process the fused data characterized by the manifold learning algorithm. The experimental results based on three real hyperspectral data sets demonstrate the efficiency of the proposed method, with improvements over the other related nonlocal or local similarity-based methods.

Earth observation using multisensor data is drawing increasing attention. Fusing remotely sensed hyperspectral imagery and light detection and ranging (LiDAR) data helps to increase application performance. In this article, joint classification of hyperspectral imagery and LiDAR data is investigated using an effective hierarchical random walk network (HRWN). In the proposed HRWN, a dual-tunnel convolutional neural network (CNN) architecture is first developed to capture spectral and spatial features. A pixelwise affinity branch is proposed to capture the relationships between classes with different elevation information from LiDAR data and confirm the spatial contrast of classification. Then in the designed hierarchical random walk layer, the predicted distribution of dual-tunnel CNN serves as global prior while pixelwise affinity reflects the local similarity of pixel pairs, which enforce spatial consistency in the deeper layers of networks. Finally, a classification map is obtained by calculating the probability distribution. Experimental results validated with three real multisensor remote sensing data demonstrate that the proposed HRWN significantly outperforms other state-of-the-art methods. For example, the two branches CNN classifier achieves an accuracy of 88.91% on the University of Houston campus data set, while the proposed HRWN classifier obtains an accuracy of 93.61%, resulting in an improvement of approximately 5%.

Conventional tensor completion (TC) methods generally assume that the sparsity of tensor-valued data lies in the global subspace. The so-called global sparsity prior is measured by the tensor nuclear norm. Such assumption is not reliable in recovering low-rank (LR) tensor data, especially when considerable elements of data are missing. To mitigate this weakness, this article presents an enhanced sparsity prior model for LRTC using both local and global sparsity information in a latent LR tensor. In specific, we adopt a doubly weighted strategy for nuclear norm along each mode to characterize global sparsity prior of tensor. Different from traditional tensor-based local sparsity description, the proposed factor gradient sparsity prior in the Tucker decomposition model describes the underlying subspace local smoothness in real-world tensor objects, which simultaneously characterizes local piecewise structure over all dimensions. Moreover, there is no need to minimize the rank of a tensor for the proposed local sparsity prior. Extensive experiments on synthetic data, real-world hyperspectral images, and face modeling data demonstrate that the proposed model outperforms state-of-the-art techniques in terms of prediction capability and efficiency.

Early detection of banana disease can limit the spread of disease, as well as reduce the treatment costs. However, the disease symptoms are so unapparent in the earlier stage that makes the labeled samples acquisition difficult and expensive. Meanwhile, it is much easier to obtain labeled samples at the late stage where the disease symptoms are obvious. In this paper, we exploit machine learning methods to use labeled samples from the late stage to train the model, then detect the banana disease in the earlier stage. Morphological openings and closings are utilized to extract the spectral-spatial features from banana leaves at both earlier and late stages, initial experimental results demonstrate significant improvements over using only spectral information.

In this work, we propose a semi-supervised method for classification of polarimetric synthetic aperture radar (PolSAR) images. In the proposed method, a 2-level mixture model is constructed by associating each component density with a unique Wishart mixture model (instead of a single Wishart distribution as that in the conventional Wishart mixture model). This modeling scheme facilitates the accurate description of data for the categories, each of which includes multiple subcategories. The learning algorithm for the proposed model is developed based on variational inference and all the update equations are obtained in closed form. In the learning algorithm, the spatial interdependencies are incorporated by imposing a Markov random field prior on the indicator variable to alleviate the speckle effect on the classification results. The experimental results demonstrate the improved performance of the proposed method compared with the unsupervised version and supervised version of the proposed model as well as an existing method for semi-supervised classification.

Large-scale Landsat image classification is essential for the production of land cover maps. The rise of convolutional neural networks (CNNs) provides a new idea for the implementation of Landsat image classification. However, pixels in Landsat images have higher uncertainty compared with high-resolution images due to its 30-m spatial resolution. In addition, the current deep learning methods tend to lose detailed information such as boundaries along with the stacking of convolutional and pooling layers. To solve these problems, we propose a new method called entropy and MRF model (EMM)-CNN based on Pyramid Scene Parsing Network. The EMM-CNN uses entropy to decrease the uncertainty of pixels. Then, the Markov random filed (MRF) model is employed to construct the connections between neighboring pixels and defined a prior distribution to prevent the cross entropy from sacrificing detailed information for the overall accuracy. Finally, transfer learning based on the pretrained ImageNet is introduced to overcome the shortage of training samples and boost the speed of the training process. Experimental results demonstrate that the proposed EMM-CNN is able to obtain classification results with fine structure by decreasing the uncertainty and retaining detailed information of the detected image.

Hyperspectral image (HSI) enjoys great advantages over more traditional image types for various applications due to the extra knowledge available. For the nonideal optical and electronic devices, HSI is always corrupted by various noises, such as Gaussian noise, deadlines, and stripings. The global correlation across spectrum (GCS) and nonlocal self-similarity (NSS) over space are two important characteristics for HSI. In this paper, a nonlocal low-rank regularized CANDECOMP/PARAFAC (CP) tensor decomposition (NLR-CPTD) is proposed to fully utilize these two intrinsic priors. To make the rank estimation more accurate, a new manner of rank determination for the NLR-CPTD model is proposed. The intrinsic GCS and NSS priors can be efficiently explored under the low-rank regularized CPTD to avoid tensor rank estimation bias for denoising performance. Then, the proposed HSI denoising model is performed on tensors formed by nonlocal similar patches within an HSI. The alternating direction method of multipliers-based optimization technique is designed to solve the minimum problem. Compared with state-of-the-art methods, the proposed algorithm can greatly promote the denoising performance of an HSI in various quality assessments.

This paper proposes an unsupervised classification method for multilook polarimetric synthetic aperture radar (Pol-SAR) data. The proposed method simultaneously deals with the heterogeneity and incorporates the local correlation in PolSAR images. Specifically, within the probabilistic framework of the Dirichlet process mixture model (DPMM), an observed PolSAR data point is described by the multiplication of a Wishartdistributed component and a class-dependent random variable (i.e., the textual variable). This modeling scheme leads to the proposed textured DPMM (tDPMM), which possesses more flexibility in characterizing PolSAR data in heterogeneous areas and from high-resolution images due to the introduction of the classdependent texture variable. The proposed tDPMM is learned by solving an optimization problem to achieve its Bayesian inference. With the knowledge of this optimization-based learning, the local correlation is incorporated through the pairwise constraint, which integrates an appropriate penalty term into the objective function so as to encourage the neighboring pixels to fall into the same category and to alleviate the "salt-and-pepper" classification appearance.We develop the learning algorithm with all the closed-form updates. The performance of the proposed method is evaluated with both low-resolution and high-resolution PolSAR images, which involve homogeneous, heterogeneous, and extremely heterogeneous areas. The experimental results reveal that the class-dependent texture variable is beneficial to PolSAR image classification and the pairwise constraint can effectively incorporate the local correlation in PolSAR images.

Pedestrian detection has achieved a remarkable progress in recent years, but challenges remain especially when occlusion happens.Intuitively, occluded pedestrian samples contain some characteristic occlusion appearance features that can help to improve detection.However, we have observed that most existing approaches intentionally avoid using samples of occluded pedestrians during the training stage.This is because such samples will introduce unreliable information, which affects the learning of model parameters and thus results in dramatic performance decline.In this paper, we propose a new framework for pedestrian detection.The proposed method exploits the use of occluded pedestrian samples to learn more robust features for discriminating pedestrians, and enables better performances on pedestrian detection, especially for the occluded pedestrians (which always happens in many real applications).Compared to some recent detectors on Caltech Pedestrian dataset, with our proposed method, detection miss rate for occluded pedestrians are significantly reduced.

The high dimensionality of hyperspectral images, the limited availability of ground-truth data as well as the low spatial resolution (causing pixels to contain mixtures of materials) hinder hyperspectral image classification. In this work we propose a novel hyperspectral classification method where we combine the outcome of spectral unmixing with the outcome of a supervised classifier. In particular, we consider fractional abundances obtained from a Sparse Unmixing method along with posterior probabilities acquired from a Multinomial Logistic Regression classifier. Both sources of information are fused using a Markov Random Field framework. We conducted experiments on publicly available real hyperspectral images: Indian Pines and University of Pavia using a very limited number of training samples. Our results indicate that the proposed decision fusion approach significantly improves the classification result over using the individual sources and outperforms the state of the art methods.

Early detection of banana disease can limit the spread of disease, as well as reduce the treatment costs. Current methods focus on either manually interpretation or calculation of spectral indices (e.g., the normalized difference vegetation index). In this paper, we exploit the fusion of close range hyperspectral (HS) image and high-resolution (HR) visible RGB image for potential disease detection in banana leaves. Our approach applies the joint bilateral filter to transfer the textural structures of HR RGB image to low-resolution HS image and obtain an enhanced HS image. Initial experimental results on Musa acuminata (banana) leaf images demonstrate the efficiency of our fusion approach, with significant improvements over either single data source or some conventional methods.

A framework based on color feature extraction of social media photos and correlation analysis with air quality parameters is proposed to monitor environmental change. More specifically, photos of the Beijing Olympic Park from Panoramio website have been analyzed as a case study. The aerosol optical depth data at 500 nm wavelength (AOD 500) obtained from sun-photometer observation network station has been used as reference. Results show a proof of concept that social media photos have an interesting potential for air pollution estimate and remote sensing parameter validation with a low cost.

Spectral-spatial classification has been widely exploited for hyperspectral imagery. However, current methods either focus on local spatial similarity or global nonlocal self-similarity (NLSS). In this paper, we propose novel methods to couple both global spatial similarity and local spectral similarity together in a single framework. In particular, our approaches exploit global spatial similarity by searching non-overlap nonlocal patches, whereas spectral similarity is determined locally within the found patches. Experimental results on two real hyperspectral data sets demonstrate the efficiency of the proposed methods, with 5%-7% (overall classification accuracy) improvements over approaches that only consider either global or local similarity.

This paper addresses the unsupervised classification problems for multilook Polarimetric synthetic aperture radar (PolSAR) images by proposing a patch-level spatially variant Wishart mixture model (SVWMM) with double constraints. We construct this model by jointly modeling the pixels in a patch (rather than an individual pixel) so as to effectively capture the local correlation in the PolSAR images. More importantly, a responsibility parameter is introduced to the proposed model, providing not only the possibility to represent the importance of different pixels within a patch but also the additional flexibility for incorporating the spatial information. As such, double constraints are further imposed by simultaneously utilizing the similarities of the neighboring pixels, respectively, defined on two different parameter spaces (i.e., the hyperparameter in the posterior distribution of mixing coefficients and the responsibility parameter). Furthermore, the variational inference algorithm is developed to achieve effective learning of the proposed SVWMM with the closed-form updates, facilitating the automatic determination of the cluster number. Experimental results on several PolSAR data sets from both airborne and spaceborne sensors demonstrate that the proposed method is effective and it enables better performances on unsupervised classification than the conventional methods.

Hyperspectral image (HSI) noise reduction is an active research topic in HSI processing due to its significance in improving the performance for object detection and classification. In this paper, we propose a joint spectral and spatial low-rank (LR) regularized method for HSI denoising, based on the assumption that the free-noise component in an observed signal can exist in latent low-dimensional structure while the noise component does not have this property. The proposed HSI denoising method not only considers the traditional LR property across the spectral domain but also leverages nonlocal LR property over the spatial domain. The main contribution of this paper is the incorporation of the low-rankness-based nonlocal similarity into sparse representation to characterize the spatial structure. Specially, the similar patches in each cluster usually contain similar sharp structure such as edges and textures; LR performed on cluster entitles to achieve a lower rank than that on the global spectral correlation. To make the proposed method more tractable and robust, we develop a variable splitting-based technique to solve the optimization problem. Experiment results on both simulated and real hyperspectral data sets demonstrate that the proposed method outperforms state-of-the-art methods with significant improvements both visually and quantitatively.

We propose a semi-supervised algorithm for processing and classification of hyperspectral imagery. For initialization, we keep 20% of the data intact, and use Principal Component Analysis to discard voxels from noisier bands and pixels. Then, we use either an Accelerated Proximal Gradient algorithm (APGL), or a modified APGL algorithm with a penalty term for distance between inpainted pixels and endmembers (APGL Hyp), on the initialized datacube to inpaint the missing data. APGL and APGL Hyp are distinguished by performance on datasets with full pixels removed or extreme noise. This inpainting technique results in band-by-band datacube sharpening and removal of noise from individual spectral signatures. We can also classify the inpainted cube by assigning each pixel to its nearest endmember via Euclidean distance. We demonstrate improved accuracy in classification over data-mining techniques like k-means, unmixing techniques like Hierarchical Non-Negative Matrix Factorization, and graph-based methods like Non-Local Total Variation.

Sparsity-based classification methods have been widely used in hyperspectral image (HSI) classification. These methods typically assumed Gaussian noise, neglecting the fact that HSIs are often corrupted by different types of noise in practice. In this paper, we develop a robust super-pixel level joint sparse representation classification model (RSJSRC) to address the mixed noise problem in sparsity-based HSI classification. Our method takes into account both Gaussian and sparse noise. Experimental results on simulated and real data demonstrate the efficiency of the proposed method and clear benefits from the introduced mixed-noise model.

The added value of multiple data sources on tree species mapping has been widely analyzed. In particular, fusion of hyperspectral (HS) and LiDAR sensors for forest applications is a very hot topic. In this paper, we exploit the use of multi-scale features to fuse HS and LiDAR data for tree species mapping. Hyperspectral data is obtained from the APEX sensor with 286 spectral bands. LiDAR data has been acquired with a TopoSys sensor Harrier 56 at full waveform. We generate multi-scale features on both HS and LiDAR data, by considering the diameter and the height layer of different tree species. Experimental results on a forested area in Belgium demonstrate the effectiveness of using multi-scale features for fusion of HS image and LiDAR data both visually and quantitatively.

Synthetic aperture radar (SAR) data are becoming more and more accessible and have been widely used in many applications. To effectively and efficiently represent multiple SAR images, we propose the mixture of Itakura-Saito (IS) divergence for non-negative matrix factorization (NMF) to perform the dimension reduction. Our proposed method incorporates the unit-mean Gamma mixture model into the NMF to model the multiplicative noise. To obtain the closed-form update equations as much as possible, we approximate the log-likelihood function with its lower bound. Finally, we apply Expectation-Maximization (EM) algorithm to estimate the parameters, resulting in the closed-form multiplicative update rules for the two matrix factors. Experimental results on real SAR dataset demonstrate the effectiveness of the proposed method and its applicability to post applications (e.g., classification) with improved performances over the conventional dimension reduction methods.

With the widespread application of hyperspectral image, restoration has become an important branch of hyperspectral data processing. Although principal component analysis (PCA), total variation (TV) and soft-thresholding algorithm (PCATV-ST) could restore image effectively, a computing bottleneck may occur with the increase of hyperspectral data volumes. In order to solve the restoration problem effectively and accurately, we put forward a distributed parallel cloud implementation of PCATV-STbased on Spark (PCATV-ST_DP). The proposed method optimizes the way of data transmission and improves the shared memory space, as well as optimizes matrix multiplication. Finally, our experimental results, conducted on real hyperspectral datasets, reveal very high performance for the proposed distributed parallel method.

This letter introduces a new spectral-spatial classification method for hyperspectral images. A multiscale superpixel segmentation is first used to model the distribution of classes based on spatial information. In this context, the original hyperspectral image is integrated with segmentation maps via a feature fusion process in different scales such that the pixel-level data can be represented by multiscale superpixel-level (MSP) data sets. Then, a subspace-based support vector machine (SVMsub) is adopted to obtain the classification maps with multiscale inputs. Finally, the classification result is achieved via a decision fusion process. The resulting method, called MSP-SVMsub, makes use of the spatial and spectral coherences, and contributes to better feature characterization. Experimental results based on two real hyperspectral data sets indicate that the MSP-SVMsub exhibits good performance compared with other related methods.

Hyperspectral images are high dimensional while the available number of training samples can be very low. For very small training sizes, classical supervised classification strategies may fail. In this work we propose an alternative, semi-supervised approach which is based on sparse unmixing. In this method, all training samples are gathered in a dictionary and serve as possible endmembers. Unmixing then reveals the relative contributions of the different training samples to an unlabeled sample. Since standard unmixing strategies as the Fully Constrained Linear Spectral Unmixing (FCLSU) typically assume only one endmember per class, we investigate the use of sparse unmixing. In this work, we apply the SunSAL algorithm. We show that this method outperforms SVM classification in the case of extremely small training sizes of only a few samples per class.

Recent advances in airborne light detection and ranging (LiDAR) technology allow us to rapid measure the topographical information over large areas. LiDAR remote sensed data has been widely used in many applications, e.g. forest management, urban planning, disaster predictions, etc. However, extracting useful information from LiDAR data remains challenging, especially in the urban remote sensing, where many objects have the same elevation and are connected, such as road and parking lots, trees and buildings. In this work, we present a new method to extract geometric and textural information from LiDAR data by using attribute filters with partial reconstruction. The proposed method can separate the connected objects and better model the geometric and textural information than traditional connected filters (e.g. attribute filters). Experimental results on LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using original LiDAR data or attribute profiles computed by traditional attribute filters, with the proposed method, overall classification accuracies were improved by 35% and 12%, respectively.

Hyperspectral and LiDAR data, can provide plentiful information about the objects on the Earths surface. However there are some shortages for each of them, where hyperspectral sensor is easily influenced by cloud and difficult to distinguish different objects contained same materials, LiDAR cannot discriminate different objects which are similar in altitude. Fusion of these multi-source data for reliable classification attracts increasing interests but remains challenging. In this paper, we propose a new framework to fuse multi-source data for classification. The proposed method contains three main works: 1) cloud shadows extraction; 2) feature fusion of spectral and spatial information extracted from hyperspectral image, elevation information extracted from LiDAR data; 3) decision fusion of cloud and non-cloud regions. Experimental results on real HSI and LiDAR data demonstrate effectiveness of the proposed method both visually and quantitatively.

Spectral unmixing is an important technology in hyperspectral image applications. Recently, sparse regression is widely used in hyperspectral unmixing. This paper proposes a double reweighted sparse regression method for hyperspectral unmixing. The proposed method enhances the sparsity of abundance fraction in both spectral and spatial domains through double weights, in which one is used to enhance the sparsity of endmembers in the spectral library, and the other to improve the sparseness of abundance fraction of every material. Experimental results on both synthetic and real hyperspectral data sets demonstrate effectiveness of the proposed method both visually and quantitatively.

Intersection detection is a critical aspect for both route planning and path optimization. In literature, intersections are detected indirectly using the road users' turning behaviors. This paper proposes a novel approach to detect intersections directly using their definition of connecting road segments. We first detect the Longest Common Sub-Sequences (LCSS) between each pair of GPS traces using dynamic programming approach. Second, we partition the longest nonconsecutive subsequences into consecutive substrings. The starting and ending points of each common substring are connecting points where two GPS traces split to different directions after they share a series of common locations. At last, we estimate Kernel Density (KD) of the connecting points and find the local maximas on the density map as intersections. Experimental results show our proposed method outperforms the state-of-the-art work with a high accuracy for intersection detection.

Dimensionality reduction (DR) is an important and helpful preprocessing step for hyperspectral image (HSI) classification. Recently, sparse graph embedding (SGE) has been widely used in the DR of HSIs. SGE explores the sparsity of the HSI data and can achieve good results. However, in most cases, locality is more important than sparsity when learning the features of the data. In this letter, we propose an extended SGE method: the weighted sparse graph based DR (WSGDR) method for HSIs. WSGDR explicitly encourages the sparse coding to be local and pays more attention to those training pixels that are more similar to the test pixel in representing the test pixel. Furthermore, WSGDR can offer data-adaptive neighborhoods, which results in the proposed method being more robust to noise. The proposed method was tested on two widely used HSI data sets, and the results suggest that WSGDR obtains sparser representation results. Furthermore, the experimental results also confirm the superiority of the proposed WSGDR method over the other state-of-the-art DR methods.

Extended attribute profiles (EAPs) have been widely used for the classification of high-resolution hyperspectral images. EAPs are obtained by computing a sequence of attribute operators. Attribute filters (AFs) are connected operators, so they can modify an image by only merging its flat zones. These filters are effective when dealing with very high resolution images since they preserve the geometrical characteristics of the regions that are not removed from the image. However, AFs, being connected filters, suffer the problem of “leakage” (i.e., regions related to different structures in the image that happen to be connected by spurious links will be considered as a single object). Objects expected to disappear at a certain threshold remain present when they are connected with other objects in the image. The attributes of small objects will be mixed with their larger connected objects. In this paper, we propose a novel framework for morphological AFs with partial reconstruction and extend it to the classification of high-resolution hyperspectral images. The ultimate goal of the proposed framework is to be able to extract spatial features which better model the attributes of different objects in the remote sensed imagery, which enables better performances on classification. An important characteristic of the presented approach is that it is very robust to the ranges of rescaled principal components, as well as the selection of attribute values. Our experimental results, conducted using a variety of hyperspectral images, indicate that the proposed framework for AFs with partial reconstruction provides state-of-the-art classification results. Compared to the methods using only single EAP and stacking all EAPs computed by existing attribute opening and closing together, the proposed framework benefits significant improvements in overall classification accuracy.

This paper proposes a semi-supervised graph-based fusion framework to couple dimensionality reduction and the fusion of multi-sensor data for classification. First, morphological features are used to model the elevation and spatial information contained in both LiDAR data and on the first few principal components (PCs) of the original hyperspectral (HS) image. Then, we fuse the features by projecting the spectral, spatial and elevation features onto a lower subspace through our proposed semi-supervised fusion graph. Experimental results on fusion of HS and LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using single data source or unsupervised graph fusion, with the proposed method, overall classification accuracies were improved by 9% and 4%, respectively.

Nowadays, we have diverse sensor technologies and image processing algorithms that allow one to measure different aspects of objects on the Earth [e.g., spectral characteristics in hyperspectral images (HSIs), height in light detection and ranging (LiDAR) data, and geometry in image processing technologies, such as morphological profiles (MPs)]. It is clear that no single technology can be sufficient for a reliable classification, but combining many of them can lead to problems such as the curse of dimensionality, excessive computation time, and so on. Applying feature reduction techniques on all the features together is not good either, because it does not take into account the differences in structure of the feature spaces. Decision fusion, on the other hand, has difficulties with modeling correlations between the different data sources. In this letter, we propose a generalized graph-based fusion method to couple dimension reduction and feature fusion of the spectral information (of the original HSI) and MPs (built on both HS and LiDAR data). In the proposed method, the edges of the fusion graph are weighted by the distance between the stacked feature points. This yields a clear improvement over an older approach with binary edges in the fusion graph. Experimental results on real HSI and LiDAR data demonstrate effectiveness of the proposed method both visually and quantitatively.

In this letter, we propose a novel approach for improving Random Forest (RF) in hyperspectral image classification. The proposed approach combines the ensemble of features and the semisupervised feature extraction (SSFE) technique. The main contribution of our approach is to construct an ensemble of RF classifiers. In this way, the feature space is divided into several disjoint feature subspaces. Then, the feature subspaces induced by the SSFE technique are used as the input space to an RF classifier. This method is compared with a regular RF and an RF with the reduced features by the SSFE on two real hyperspectral data sets, showing an improved performance in ill-posed, poor-posed, and well-posed conditions. An additional study shows that the proposed method is less sensitive to the parameters.

This paper proposes a method to combine feature fusion and decision fusion together for multi-sensor data classification. First, morphological features which contain elevation and spatial information, are generated on both LiDAR data and the first few principal components (PCs) of original hyper-spectral (HS) image. We got the fused features by projecting the spectral (original HS image), spatial and elevation features onto a lower subspace through a graph-based feature fusion method. Then, we got four classification maps by using spectral features, spatial features, elevation features and the graph fused features individually as input of SVM classifier. The final classification map was obtained by fusing the four classification maps through the weighted majority voting. Experimental results on fusion of HS and LiDAR data from the 2013 IEEE GRSS Data Fusion Contest demonstrate effectiveness of the proposed method. Compared to the methods using single data source or only feature fusion, with the proposed method, overall classification accuracies were improved by 10% and 2%, respectively.

This paper presents a two-stage denoising method for hyper-spectral image (HSI) by combining kernel principal component analysis (KPCA) and total variation (TV). In the first stage, we use KPCA denoising to reduce spectrally uncorre-lated noise. In the second stage, the information content is largely separated from the remaining noise by means of principal component analysis (PCA). The remaining noise is then efficiently removed by fast primal-dual TV denoising in low-energy PCA channels. Experimental results on simulated and real HSIs are very encouraging.

We propose a novel semisupervised local discriminant analysis method for feature extraction in hyperspectral remote sensing imagery, with improved performance in both ill-posed and poor-posed conditions. The proposed method combines unsupervised methods (local linear feature extraction methods and supervised method (linear discriminant analysis) in a novel framework without any free parameters. The underlying idea is to design an optimal projection matrix, which preserves the local neighborhood information inferred from unlabeled samples, while simultaneously maximizing the class discrimination of the data inferred from the labeled samples. Experimental results on four real hyperspectral images demonstrate that the proposed method compares favorably with conventional feature extraction methods.

A fast iterative Kernel Principal Component Analysis (KPCA) is proposed to extract features from hyperspectral images. The proposed method is a kernel version of the Candid Covariance-Free Incremental Principal Component Analysis, which solves the eigenvectors through iteration. Without performing eigen decomposition on Gram matrix, our method can reduce the space complexity and time complexity greatly. Experimental results were validated in comparison with the standard KPCA and linear version methods.

The theory of Chinese character intelligent formation considers that Chinese characters are formed by components according to character structure; all the components are the topological mapping of basic elements in the character structure. The mapping method of basic elements in different character structures is one of the key technologies. This paper carried on a thorough analysis to the transformation of basic elements, proposed the topological mapping method based on affine transformation. 27533 Chinese characters in GB18030-2000 standard were taken as experiment subject, a platform for Chinese character intelligent formation system was developed and all the characters were formed in the platform.