Balancing spectral, spatial, and temporal resolutions is a key challenge in spectral imaging. The Dual-Camera Coded Aperture Snapshot Spectral Imaging (DC-CASSI) system alleviates this trade-off but suffers from severely ill-posed reconstruction problems due to its high compression ratio. Existing methods are constrained by scene-specific tuning or excessive reliance on paired training data. To address these issues, we propose a Total Variation (TV) subgradient-guided multi-source fusion framework for DC-CASSI reconstruction, comprising three core components: (1) An end-to-end Single-Disperser CASSI (SD-CASSI) observation model based on the tensor-form Kronecker δ, which establishes a rigorous mathematical foundation for physical constraints while enabling efficient adjoint operator implementation; (2) An adaptive spatial reference generator that integrates SD-CASSI’s physical model and RGB subspace constraint, generating the reference image as reliable spatial prior; (3) A TV subgradient-guided regularization term that encodes local structural directions from the reference image into spectral reconstruction, achieving high-quality fused results. The framework is validated on simulated datasets and real-world datasets. Experimental results demonstrate that it achieves state-of-the-art reconstruction performance and robust noise resilience. This work not only establishes an interpretable theoretical foundation for subgradient-guided fusion but also provides a practical fusion-based paradigm for high-fidelity spectral image reconstruction in DC-CASSI systems. Source code: https://github.com/bestwishes43/ADMM-TVDS.

Fully supervised ultrasound segmentation requires extensive labeled data, which limits its clinical use. While existing semi-supervised methods leverage unlabeled data, they typically treat all regions equally-overlooking critical variations in learning difficulty. We find that boundary and low-contrast regions show persistently low confidence, indicating higher learning difficulty. To address this, we propose a novel framework featuring Bidirectional Replacement and Dynamic Reweighting. Our method constructs easy and hard samples by exchanging regions between images, then applies curriculum-inspired reweighting to adaptively balance their contributions during training. Extensive experiments on three ultrasound datasets demonstrate consistent superiority over state-of-the-art methods under varying supervision levels, confirming effectiveness and robustness.

High-precision spectral unmixing (SU) of hyperspectral image (HSI) faces a challenging problem in that it is difficult to distinguish the different objective classes presenting similar spectra without elevation information. A digital surface model (DSM) with the same spatial resolution as HSI can provide additional geometric information that can be useful for the HSI SU task. Existing SU methods that incorporate the DSM data miss the consideration that DSM data and hyperspectral data are actually different dimensions of information from different sensors observing the same scene. The intrinsic hyperspectral unmixing model can obtain the shading component and the reflectance component, which can be further decomposed into the endmember and the abundance. From a physical modeling perspective, the shadow component can be considered as the interaction of illumination and the geometric component from the DSM data; thus, the quality of hyperspectral unmixing under invariant illumination conditions can be significantly enhanced. Furthermore, the elevation information within the DSM data contributes to the unmixing process by distinguishing different objective classes with similar physiochemical properties at varying altitudes. Experimental validation is conducted using three HSI datasets. The results can indicate the robustness and superiority of the proposed unmixing method.

Hyperspectral images (HSI) have higher spectral resolution than multispectral images (MSI), but due to limitations of imaging equipment, their width is narrower than MSI. When using partially overlapping HSI-MSI to improve the classification capabilities of MSI, there may be unknown classes that do not exist in HSI-MSI overlapping regions. To solve this problem, this paper proposes a heterogeneous open-set cross-domain manifold embedding aligned method for HSI-MSI collaborative classification. The method designs manifold embedding to align HSI-MSI features to map into subspaces, and gradually selects target domain samples for pseudo-labeling through the designed strategy while rejecting unknown class samples. The feature alignment and pseudo-labeled sample selection are continuously iterated to promote each other, reducing the intra-class distance while pushing the rejected target data away from known classes. The experimental results verify the superiority of our method.

Variation in illumination conditions can give rise to divergences in the reflectance profiles corresponding to identical feature categories. This phenomenon has always been a very important challenge that cannot be avoided in spectral unmixing (SU) algorithms. Current methods primarily focus on modeling spectral errors or using spectral libraries for optimization. This paper presents a new multimodal hyperspectral unmixing method under spectral variability that incorporates DSM data to model external shading variations in complicated illumination conditions. The proposed model aims to reduce spectral variability caused by external imaging changes by fitting the shading information with the digital surface model (DSM) data and the illumination information. The reflectance information, which represents the properties of the features themselves, is used for spectral unmixing after introducing the intrinsic decomposition. The method proposed in this paper can effectively attenuate the effect of spectral variability, as demonstrated by experimental validation on real MUFFL multimodal datasets.

Foundation models offer a highly versatile and precise solution for intelligent interpretation of remote sensing images, thus greatly facilitating various remote sensing applications. Nevertheless, conventional remote sensing foundational models based on generative transformers neglect the consideration of multiscale features and frequency information, limiting their potential for dense prediction tasks in remote sensing scenarios. In this article, we make the first attempt to propose a generative convolutional neural network (ConvNet) foundation model tailored for remote sensing scenarios, which comprises two key components: First, a large dataset named GeoSense, containing approximately nine million diverse remote sensing images, is constructed to enhance the robustness and generalization of the foundation model during the pretraining phase. Second, a sparse modeling and low-frequency reconstruction (SMLFR) framework is designed for self-supervised representation learning of the ConvNet foundation model. Specifically, a sparse modeling strategy is proposed in masked image modeling (MIM), which allows ConvNet to process variable-length sequences by treating unmasked patches as voxels and sparsifying the encoder. In addition, a low-frequency reconstruction target is designed to guide the model’s attention toward essential ground object features in remote sensing images, while mitigating unnecessary detail interference. To evaluate the general performance of our proposed foundation model, comprehensive experiments have been carried out on five datasets across three downstream tasks. Experimental results demonstrate that our method consistently achieves state-of-the-art performance across all the benchmark datasets and downstream tasks. The code and pretrained models will be available athttps://github.com/HIT-SIRS/SMLFR.

Recent advancements in remote sensing foundation models have unveiled their tremendous potential in addressing earth observation tasks. Presently, when large-scale foundation models are transferred to downstream tasks, the prevalent approach is to adopt the full-tuning strategy, resulting in significant increases in storage demands and computational costs. Although the introduction of parameter-efficient fine-tuning (PEFT) has mitigated this issue to some extent, mainstream PEFT methods are primarily designed for classification tasks and often prove insufficient to meet the demands of dense prediction tasks. In order to overcome the aforementioned limitations, we propose a unified PEFT framework UPetu, encompassing two essential and complementary modules: the efficient quantization adapter module (EQAM) and the context-aware prompt module (CAPM). EQAM is specifically designed to enhance the correlation between fine-grained feature information and task-specific knowledge through the introduction of quantization linear layers and non-linear activation functions. Additionally, CAPM is introduced to acquire rich contextual features by incorporating trainable prompts into multi-scale features. The synergistic integration of both modules enhances the representation learning capability and generalization transferability of the foundation model. Extensive experiments on three remote sensing scene classification datasets demonstrate the superiority of UPetu over other fine-tuning methods. With the update of only 0.73% of ConvNeXt-B parameters, our UPetu achieves superior performance compared to full-tuning on the UCM-55, AID-28, and AID-55 datasets. Furthermore, experiments conducted on semantic segmentation and change detection tasks provide additional evidence of the effectiveness and generalization capabilities of the proposed UPetu.

The coordinated utilization of diverse types of satellite sensors provides a more comprehensive view of the Earth’s surface. However, due to the significant heterogeneity across modalities and the scarcity of high-quality labels, most existing methods face bottlenecks in the underutilization of massive unlabeled multimodal satellite data, making it challenging to understand the scene comprehensively. To this end, we propose a multimodal unified representation learning framework (MURLF) based on masked image modeling (MIM) for remote sensing (RS) images, aiming to make better use of massive unlabeled multimodal RS data. MURLF leverages the consistency and complementarity relationships among modalities to extract both common and distinctive features, mitigating the challenges faced by encoders due to significant heterogeneity across various data types. In addition, MURLF uses multilevel masking independently across different modalities, using visual tokens both within the same modality and across modalities to jointly recover masked pixels as the pretext task, facilitating comprehensive cross-modal information interaction. Furthermore, we design a preselected sensor-specific feature extractor (PSFE) to exploit the heterogeneous characteristics of various data sources, thereby extracting discriminative features. By integrating the multistage PSFE with the ViT backbone, MURLF can naturally extract multimodal hierarchical representations for downstream tasks, fully preserving valuable information from each modality. The proposed MURLF is not restricted to multimodal inputs but also supports single-modal inputs during the fine-tuning stage, significantly broadening the framework’s application. Extensive experiments across multiple tasks demonstrate the superiority of the proposed MURLF compared with several advanced multimodal models. The code will be released soon.

With the development of observation technology, multispectral (MS) images of large scenes are easy to obtain, but the low spectral resolution limits their classification ability. Moreover, the collection of training samples is difficult and time-consuming, and limited labeled samples are a challenge for the precise classification of large-scene MS images. This article attempts to use hyperspectral (HS) images with limited labels to help classify MS images of large scenes, so as to achieve better classification results. To solve this problem, a few-shot MS-HS image collaborative classification method combining feature distribution enhancement (FDE) and subdomain alignment is proposed. Specifically, a residual 3-D convolution network embedded with a 3-D FDE module is designed to improve the diversity of the feature distribution extracted by the network and increase the generalization ability of the model under the few-shot condition. Furthermore, the local domain alignment between the source and target domains is achieved by subdomain alignment, which better aligns the categories in the source domain and the target domain, and achieves the distribution alignment of the subdomains. In addition, the feature bias adjustment (FBA) module is introduced in the test phase to correct the bias of the MS image feature representation, and to alleviate the cross-domain problem to some extent. The few-shot learning (FSL) is applied in the source and target domains to learn better feature mapping. The results of comparative experiments on three datasets show that the proposed method is superior to the most advanced method in the case of limited labeled samples.

Hyperspectral images (HSIs) have higher spectral resolution than multispectral (MS) images, but they have a narrower swath than MS images. The limited spectral resolution of MS images constrains their classification capabilities, and annotating remote sensing data is time-consuming and laborious. In addition, large-scale MS images may contain unknown classes not present in the training data. This article attempts to use partially overlapping HS images with limited labels to assist in the classification of large-scene MS images. It can correctly distinguish known classes and simultaneously identify unknown classes, thereby achieving better classification results for MS images. To address this challenge, a few-shot open-set HS–MS image collaborative classification method is proposed. Specifically, a spectral–spatial feature interactive enhancement (SSFIE) module is designed for richer feature extraction and enhanced classification capabilities in the feature extraction stage. In the few-shot learning (FSL) stage, an adaptive joint similarity metric criterion is proposed to improve feature mapping between the source and target domains. Discriminative joint probability adaptation (DJPA) is used for domain adaptation and to enhance feature discriminability, while batch nuclear-norm maximization (BNM) is employed to increase the feature diversity. In the testing phase, the open-set classification module is designed to correctly classify samples of known classes while simultaneously distinguishing unknown classes. The experimental results on four cross-domain HS–MS data pairs demonstrate that our proposed method outperforms state-of-the-art methods.

Spectral reconstruction (SR) techniques can generate hyperspectral images (HSIs) from multispectral images (MSIs) with the same spatial resolution, thus alleviating the problem of limited availability and low spatial resolution of satellite HSIs. However, in scenarios where both HSIs and MSIs can be acquired simultaneously, spectral mapping relationship (SMR) among real images may not align with the sensor’s spectral response function (SRF), due to factors such as sensor noise and calibration errors. This mismatch can result in discrepancies in reflectivity between the reconstructed HSIs and the real HSIs. To solve the above problems, this article proposes a semi-supervised transfer learning SR (SSTSR) model based on gradient direction constraints. Through semi-supervised learning, SSTSR acquires precise SMRs in overlapping regions and extracts spectral trend information of ground objects from historical models in nonoverlapping regions. Experiments on two datasets demonstrate that the reconstructed HSIs closely resemble real HSIs, leading to impressive classification performance when employing a real HSI classifier.

Remote sensing cross-scene classification has always been an important research field, especially in the field of 3-D classification, which is of great significance. Considering the diversity of collection conditions, seasons, and regional styles, deep learning networks well-trained on one source domain dataset tend to suffer from severe performance degradation when applied to other target domain datasets. To tackle the issue, in this article, we propose a new cross-scene classification method, which combines pre-alignment and Shannon entropy constraint to accomplish unsupervised domain adaptive classification (PS-UDA). On the one hand, the pre-alignment employs$L_{2}$-paradigm constraint and Laplace matrix to pre-align the features. With the$L_{2}$-paradigm constraint, the originally distant features of the source and target domain are constrained to the same sphere surface, and it is easier to make the distribution alignment on the sphere surface. Further, the Laplace matrix is used to map the source and target domain. In this way, similar features of the source and target domain are further aligned, and dissimilar features become discrete from each other. On the other hand, this article employs the Shannon entropy constraint to motivate the network to obtain more high-confidence target domain pseudo-labels. In addition, to fully utilize the unlabeled target domain information, the target domain features are augmented using the adjacency matrix. Experimental results of two cross-scene multispectral point cloud classifications demonstrate that the proposed PS-UDA can effectively mitigate the spectral shift issue in cross-scene multispectral point clouds, achieving state-of-the-art performance.

Semantic segmentation is a crucial task in remote sensing and has been predominantly performed using convolutional neural networks (CNNs) for the past decade. Recently, transformers with self-attention mechanisms have demonstrated superior performance compared to CNNs. However, due to the locality of CNN and the high computational complexity and massive data resource requirements of transformer, neither of them can be well applied in resource-constrained practical remote sensing scenarios. Motivated by the limitations of using either convolutional neural networks (CNNs) or transformers alone in the task of semantic segmentation of remote sensing images, a novel cross-model knowledge distillation framework, named distilling segmenters from CNNs and transformers (DSCT), is proposed in this paper to harness the complementary advantages of both models. The framework utilizes a channel-weighted attention-guided feature distillation (CAFD) module to condense the feature from the teacher model and enhance the student model’s focus on the teacher-focused regions. Additionally, a target-nontarget knowledge distillation (TNKD) module is proposed that decouples logit distillation into target and nontarget knowledge distillation to guide the student model in learning the underlying representations and decision boundaries from the teacher model. By learning the complementary knowledge from the teacher, our proposed DSCT framework improves the student’s segmentation performance without adding trainable parameters. Experiments on four available remote sensing datasets (ISPRS Potsdam, Vaihingen, GID and LoveDA) indicate that the proposed DSCT outperforms the state-of-the-art knowledge distillation methods and demonstrates its effectiveness and robustness.

A critical requirement for the success of supervised deep learning lies in having numerous annotated images, which is often challenging to fulfill in remote sensing semantic segmentation tasks. Self-supervised contrastive learning (CL) offers a strategy for learning general feature representations by pre-training neural networks on vast amounts of unlabeled data and subsequently fine-tuning them on downstream tasks with limited annotations. However, the vast majority of CL methods are designed based on instance discriminative pretext tasks, focusing solely on learning the global representation of the entire image while disregarding the essential spatial and semantic correlations crucial for semantic segmentation tasks. To address the above issues, in this paper, we propose a spatial and semantic consistency contrastive learning (SSCCL) framework for the semantic segmentation task of remote sensing images. Specifically, a consistency branch in SSCCL is designed to learn feature representations with spatial and semantic consistency by maximizing the similarity of the overlapping regions of the two augmented views. Additionally, an instance branch is introduced to learn global representations by enforcing the similarity of two augmented views from one image. Through the integration of the consistency branch and instance branch, the proposed SSCCL framework can learn robust and informative feature representations for semantic segmentation in remote sensing scenarios. The proposed method was evaluated on three publicly available remote sensing semantic segmentation datasets, and the experimental results show that our method achieves superior segmentation performance with limited annotations compared to state-of-the-art CL methods as well as ImageNet pre-training method.

Spectral reconstruction based on satellite multispectral (MS) images can produce high spatial resolution hyperspectral (HS) images at a reasonable cost, significantly expanding the application of satellite-based HS remote sensing. As a challenging ill-posed problem, existing methods have difficulty making full use of local and global information of space and spectra to guide the reconstruction, resulting in limited accuracy in large-scale scenes with complex ground features and severe spectral mixing. In this article, we propose a novel convolution and Transformer joint network (CTJN) to address the challenge of high-accuracy spectral reconstruction in complex scenes. The CTJN is cascaded with shallow feature extraction modules (SFEMs) and deep feature extraction modules (DFEMs), which can explore local spatial features and global spectral features. Besides, a high-frequency Transformer block (HF-TB) is designed to highlight the detailed features of the images to prevent significant high-frequency information loss, which could improve the reconstruction results in regions with drastic feature changes. Moreover, a spatial–spectral recalibration block (SSRB) is proposed to perform explicit constraints on the reconstructed points by exploiting the correlation among neighboring pixels and adjacent spectra. Extensive experimental results on four HS–MS datasets and one MS dataset demonstrate that the proposed CTJN outperforms the state-of-the-art methods in large-scale and small-scale scenes.

Traditional spectral unmixing (SU) of satellite hyperspectral images (HSIs) faces two main challenges: One is that limited by the low resolution of satellite HSIs, it is difficult to guarantee the accuracy of endmember extraction due to severe spectral mixing; the other is that the spectral variability is unavoidable due to external factors such as atmospheric, illumination, and topographic variations, as well as internal factors such as physical changes of the features themselves. Unmanned aerial vehicle (UAV) HSIs of high spatial resolution can provide a highly accurate reflectance curves from regions of interest (ROIs), and the intrinsic image decomposition (IID) technique can reduce the spectral variability caused by external factors. Based on this, a novel IID embedded UAV-satellite spectral unmixing model is proposed. On the one hand, the spectral variability is solved by an embedded IID framework in the inverse problem of SU. The proposed method replaces the input,i.e., the original HSI, with the reflectance component, which is independent of the spectral variability caused by external factors. On the other hand, a UAV spectral library constructed from the UAV HSI is introduced to guarantee the accuracy of the endmember. Thus, by IID embedded in the framework of UAV-satellite collaborative spectral unmixing, the proposed method is able to address the aforementioned problems. Experimental validation is conducted using UAV HSI and three sets of satellite HSI from the Yellow River Delta region. The results indicate that the proposed method can effectively improve the robustness and superiority of the unmixing results.

The fine spectra of the hyperspectral (HS) images can fully reflect the subtle features of the spectra of different objects. However, due to the limitation of the imaging equipment, its swath is not as large as that of multispectral (MS) images. The acquisition of MS images is more convenient, but the discrimination of spectral features is relatively poor. This paper aims to investigate how partially overlapping HS images can be utilized to improve the classification accuracy of large-scene MS images. Due to the spectral mismatch existing between MS and HS features, traditional transfer learning methods cannot solve the problem of classification with heterogeneous features. To address this issue, a novel structure-preserving discriminative distribution adaptive MS-HS image collaborative classification method is proposed in this paper, which aims to improve the classification accuracy of large-scene MS images by discriminative features. Specifically, this method combines statistical properties and geometric constraints in transfer learning, and jointly maximizes the distance between different classes by discriminative least squares to maximize classification accuracy. Moreover, the source and target domains are probabilistically adaptive while maintaining the local structure of MS-HS features, so that the data distribution is fully aligned and the distance between different classes is increased. The learned mapping matrix enables the mapping of multi-scale spectral-spatial features of MS-HS images to subspaces for classification. Compared with related advanced methods, three sets of MS-HS data sets show that the proposed method can effectively reduce the differences between MS-HS data and achieve better classification results.

Deep learning-based intrinsic image decomposition (IID) has gained significant attention in computer vision due to the high efficiency and accuracy of learning-based methods. However, the development of deep learning-based IID methods in the remote sensing field has been limited by the lack of experimental datasets. This article proposes a two-stream encoder-decoder network for the single hyperspectral (HS) image IID task. The proposed network comprises one reflectance estimation subnetwork and one shading estimation subnetwork, which predict intrinsic properties separately. The proposed model introduces three physical losses to enhance performance: 1) In the reflectance estimation subnetwork, the self-similarity loss on the reflectance component is added to satisfy the basic assumption that pixels with similar intensity tend to have a similar reflectance property. 2) In the shading estimation subnetwork, the shading structure loss is added to ensure that the structure of the shading component conforms to physical observation. 3) Reconstruction loss connecting two subnetworks is required to ensure the estimated intrinsic components are physically correct. Finally, to avoid an unreasonable composition, the entire network is initialized by reflectance estimated by the physical model. The quantitative experimental results of intraclass consistency and classification metrics demonstrate that the proposed physical prior-driven unsupervised learning-based IID network outperforms the current available learning or optimization-based approaches.

Generating hyperspectral point cloud from hyperspectral images (HSIs) and light detection and ranging (LiDAR) has become more and more common in the remote sensing field and supported various applications. One challenge here is that hyperspectral imaging is a passive imaging method and is suffering from shadows in a natural scene. Intrinsic information recovery can effectively eliminate the spectral variation caused by illumination changes; however, it assumes a uniform light and neglects the shadows in the scene. In this article, we provide a novel hyperspectral point cloud intrinsic model that can detect the shaded regions and recover reflectance information in them. We first estimate the global illumination of the scene using an intrinsic information recovery method. Then, we perform supervoxel segmentation on hyperspectral point cloud to calculate the blocking relation of supervoxels and therefore accurately detect shaded regions. Finally, we estimate the illumination and reflectance of shaded regions based on an illumination-invariant spectral prior. The experimental results show that the proposed method can effectively detect shaded areas and robustly generate shadow-less intrinsic hyperspectral point cloud.

With the development of observation technology in remote sensing (RS), large-area multispectral (MS) images can be easily obtained. However, due to the limitation of imaging devices, only a limited range of hyperspectral (HS) images with higher spectral resolution can be obtained. This article mainly focuses on how to use limited HS images to improve the classification performance of MS images. In order to solve this problem, this article proposes an MS–HS image collaborative classification method, which integrates coupled dictionary learning and distance preserved probability distribution. First, image reconstruction based on coupled dictionary learning is performed, in which sparse representation and dictionary learning are used to generate HS images from MS images through spectral superresolution, so that the spectral features of the MS data and HS data are converted to the same feature space for feature space alignment. Second, the probability distribution is adapted, in which the marginal and conditional probabilities are adapted to further narrow the difference between the real HS data and the generated HS data. At the same time, the consistency of the data structure of the source domain before and after the mapping is maintained, so that the same class of data is more compact after the mapping and reduces the spacing within the same class. Compared with the state-of-the-art methods, this article conducts the experiments on three MS–HS RS datasets, which demonstrate the superiority of the proposed method.

The combination of spectral and 3-D elevation information provided by hyperspectral images (HSIs) and Light Detection and Ranging (LiDAR) has gained increased attention in the remote sensing field and enabled numerous applications. While various methods have been proposed to fuse these two data streams in pixel, feature, or decision level, a deeper view into the intrinsic relation of surface geometry, material reflectance, and environment illumination is still lacking. In this article, we present a novel supervoxel-based joint intrinsic decomposition framework for HSIs and LiDAR. First, we proposed a novel intrinsic scene model for HSIs and LiDAR point cloud, which tells how we can map LiDAR point cloud into HSI pixels with point-cloud-level normals, reflectance, and incident light direction. Then, we extract supervoxels from the LiDAR point cloud using a graph-based supervoxel method. Finally, we formulate the intrinsic decomposition problem within a supervoxel-based framework which can be optimized effectively and efficiently. The outputs of the proposed model are intrinsic scene properties like incident light direction and point-cloud-level hyperspectral reflectance, with which we can then generate intrinsic hyperspectral point cloud (IHSPC) where each point possesses not only 3-D coordinates and normals but also the reflectance over each wavelength. The performance of our approach is demonstrated with both synthetic and real data.

Hyperspectral satellite data has been widely applied in many fields due to its numerous bands. Along with the advantages of high spectral resolution, hyperspectral satellite data are still limited by some disadvantages of high acquisition cost, low revisiting capability, and low spatial resolution. Compared with hyperspectral satellites, multispectral satellites have a large number, large width, strong coverage and high spatial resolution. Therefore, multispectral data can be used as the input to the spectral reconstruction to obtain hyperspectral data with high temporal resolution. Better hyperspectral data can be obtained by spectral reconstructing with these continuous multi-temporal data than with single-temporal data. A multi-temporal spectral reconstruction network (MTSRN) is proposed in this paper, which is used to reconstruct hyperspectral images from multi-temporal multispectral images. The proposed MTSRN comprises multiple single-temporal spectral reconstruction networks (STSRN) for extracting temporal features and a multi-temporal fusion network (MTFN). The parallel component alternative (PA) post-processing method enhances the physical plausibility of reconstructed hyperspectral data. To demonstrate performance of the proposed method in aspects of multi-temporal reconstruction, experiments are conducted on four multi-temporal hyperspectral and multispectral satellite datasets. The experimental results prove that the proposed MTSRN obtains better spectral reconstruction results compared with the spectral reconstruction method based on single-temporal information.

Large-scene precise classification of multispectral images (MSIs) has become one of the hot topics in remote sensing field. MSIs usually have wide swath and a meter or even submeter level of spatial resolution, which make large-scene observation possible. However, the limited number of spectral bands leads to the confusion of land covers in classification, especially for the large-scene conditions with abundant land cover types. Therefore, overlapped hyperspectral images (HSIs) can be used to improve the precision degree of classification. To achieve this purpose, coupled dictionary learning has been proposed as a major means. Aiming at separating the class-specific characteristics and mutual patterns among different land covers, this paper proposed a separable coupled dictionary learning (SCDL) method, which converts the separation of mutual features into the construction of separable coupled dictionaries and learns both class-specific coupled dictionaries and mutual coupled dictionaries simultaneously with the aid of label information. More specifically, the proposed method uses the labels of training samples to construct class-specific reconstruction error constraint, class-specificity constraint and separable dictionary incoherence constraint as regularization terms, to make sure that the learned coupled dictionaries to be both compact and discriminative. The learned separable coupled dictionaries facilitate pixels belong to the same category to be represented by the mutual dictionary and the class-specific sub-dictionary of corresponding class. The experiments compared with several state-of-the-art methods on three pairs of HSI and MSI have shown better classification performance.

Coastal zone has become an important area for the study of global change. Remote sensing technology has been widely used in coastal zone monitoring due to its numerous advantages. Therefore, in this paper, Chinese GaoFen 1 Wide Field of View (GF1WFV) multitemporal data are used owe to its short revisiting period. Multitemporal change type identification in costal zone is achieved by firstly extracting the slowly changing features through slow feature analysis network (SFANet) and then identifying types of change with some category labels through long short-term memory (LSTM) network. Experimental results have shown the effectiveness of SFANet feature and LSTM. This study can provide technical feasible scheme and theoretical support for the analysis of coastal zone land change.

The miniaturization of hyperspectral sensors and the popularity of the unmanned aerial vehicle (UAV) make it possible to obtain a series of hyperspectral images (HSIs) in the same geographical area at different time-points by same or different sensors. When classifying these multi-temporal HSIs, temporal-adaptation is required to deal with the spectral drift and band inconsistency problems. Since most studies focus on semi-supervised domain adaptation (DA) strategy, and spatial features are usually absent during most of the DA procedure, an unsupervised temporal-adaptation method is realized by spatial-spectral multiple Geodesic Flow Kernels (S2-GFKs) to classify bi-temporal HSIs. Experiments conducted on two real HSI datasets and compared with several well-known methods demonstrate the availability of the proposed model.

Nowadays, many methods of spatial-spectral classification have been developed and achieved good results for classification with high-resolution remotely sensed images, especially superpixel-based methods. However, these methods generally consider a superpixel as a group of pixels instead of one entity, ignoring the spectral-spatial entirety in the third-order RSI data cube. In order to fully exploit the third-order spectral-spatial information, in this paper, we propose a superpixel-based tensor model for RSI classification, where a multiattribute superpixel tensor (MAST) model is constructed on the top of multiattribute superpixel maps based on the concept of extended morphological profiles (EMAPs). In order to manage the adaptive spatial nature of superpixels, we develop an increment strategy to augment all superpixels with filling up their own envelop rectangles including three different ways, i.e., 0 vector, mean vector of all the pixels within the superpixel, or original pixels. Then, we use CANDECOMP/PARAFAC (CP) decomposition to obtain the features of the unified dimension from the MASTs of various sizes. Especially, CP decomposition can deal with missing data, so we also got a fourth means of constructing the MAST. Finally, base kernels calculated, respectively, from the original spectral feature, EMAP features and MAST features are learned by multiple kernel learning methods, with the optimal kernel fed to a support vector machine to complete the classification task. The experiments conducted on four real RSIs and compared with several well-known methods demonstrate the effectiveness of the proposed model.

In this paper, a novel reflectance model is proposed to recover intrinsic images from remote sensing hyperspectral images (HSIs). Intrinsic image recovery is a well-known challenging and underconstrained problem in computer vision, and it becomes even more severely illposed for HSIs. To reduce the uncertainties and improve the recovery accuracy, two kinds of priors are introduced: 1) shading prior which describes the geometric relation between illuminate and object surface and 2) reflectance prior based on L1-graph coding, which describes the relation between pigment density with reflectance. These priors can effectively eliminate the reflectance inhomogeneity caused by surface normal changes or pigment density variations other than material changes. Then, a noniterative optimization method is proposed to combine the shading prior and reflectance prior, with which closed-form solutions can be derived and thus avoided falling into local optimums. The experimental results demonstrate that the proposed method can efficiently improve the spectral reflectance homogeneity within a class while preserving the image boundaries; it also produces a competitive performance with the state of the art when utilizing the extracted intrinsic hyperspectral reflectance feature in the task of HSI classification.

With the increasing acquisition ability of hyperspectral remote sensing images, unsupervised cross-temporal classification (UCTC) of hyperspectral images (HSIs) has attracted more and more attention. In this paper, we focus on cross-temporal HSI classification, i.e., using one labeled HSI to classify the other unlabeled HSI. A multiple geodesic flow kernel learning (MGFKL) framework is proposed to exploit both spatial and spectral features for UCTC with bitemporal HSIs and called S2-MGFKL. The proposed S2-MGFKL method first extracts extended multi-attribute profiles (EMAPs) from the original bitemporal HSIs. The spatial features of the bitemporal HSIs obtained by the same attribute filter are paired up, so are the original spectral features. Second, each pair of features from both source and target domains are used to construct multiple geodesic flows. According to the original definition of GFK, we can obtain the construction of Gaussian base GFKs. The base kernels consist of two parts, the spectral part is obtained base on the same geodesic flow (which is constructed on the bitemporal spectral features) by tuning the kernel scale, while the spatial part is obtained under the same kernel scale but different geodesic flows constructed on different spatial feature pairs. After that, the mean rule is adopted to acquire the combined kernel, which is fed into the supervised vector machine (SVM) to implement the cross-temporal classification task. Experiments are conducted on two real HSI data sets, and the results compared with several well-known methods demonstrate the effectiveness of the proposed method.

With the development of remote sensors, it is much easier to acquire large amount of remote sensing images (RSIs) with very high spatial resolution, which has made the spatial characteristics play an important role in classification task. Many work of spatial-spectral classification have been done and achieved good results, especially superpixel-based methods. However, these methods didn't take each superpixel as an entirety, which had ignored the relationship between spatial and spectral signature. It is well known that RSI can be treated as a third-order data cube, thus it can also be represented by a third-order tensor. This paper proposed a Multi-Attribute Superpixel Tensor (MAST) model to address the aforementioned problem. Experiments conducted on two real RSIs and compared with several well-known methods demonstrate the effectiveness of the proposed model.

Multi-temporal remote sensing images classification have attracted more and more attention in the last decade because of a wide range of applications of multi-temporal images in long-term environmental monitoring and land cover change detection and increasing multi-temporal data available. At present, most papers investigated two temporal remote-sensing images classification. In fact, there is lots of distinctive information to be unexploited between two or more temporal images which can enhance classification effect and improve ability of detecting change area. In this paper, we present an evidential fusion framework of manifold alignment to combine more than two multi-temporal remote sensing images. Embedding of multi-groups two temporal images pairs after MA can be intergraded based a layered structure of D-S theory. The proposed method was evaluated using five Landsat 8 images. Results confirmed that the proposed algorithm performed better than those with only two temporal images.

With the development of hyperspectral sensors, nowadays, we can easily acquire large amount of hyperspectral images (HSIs) with very high spatial resolution, which has led to a better identification of relatively small structures. Owing to the high spatial resolution, there are much less mixed pixels in the HSIs, and the boundaries between these categories are much clearer. However, the high spatial resolution also leads to complex and fine geometrical structures and high inner-class variability, which make the classification results very “noisy.” In this paper, we propose a multimorphological superpixel (MMSP) method to extract the spectral and spatial features and address the aforementioned problems. To reduce the difference within the same class and obtain multilevel spatial information, morphological features (multistructuring element extended morphological profile or multiattribute filter extended multi-attribute profiles) are first obtained from the original HSI. After that, simple linear iterative clustering segmentation method is performed on each morphological feature to acquire the MMSPs. Then, uniformity constraint is used to merge the MMSPs belonging to the same class which can avoid introducing the information from different classes and acquire spatial structures at object level. Subsequently, mean filtering is utilized to extract the spatial features within and among MMSPs. At last, base kernels are obtained from the spatial features and original HSI, and several multiple kernel learning methods are used to obtain the optimal kernel to incorporate into the support vector machine. Experiments conducted on three widely used real HSIs and compared with several well-known methods demonstrate the effectiveness of the proposed model.

Since the scan line corrector (SLC) of the Landsat ETM+ sensor failed permanently in 2003, about 22% of the pixels in an SLC-off image are missed. Traditional gap filling methods always need a SLC-on image for reference, but the most similar sensor (Landsat TM) closed at 2011. And the potential of multi-temporal was also neglected in traditional filling methods. In this paper, a multi-temporal Landsat ETM+ gap filling method is proposed without using SLC-on reference which has ability to increase the utilization efficiency of multi-temporal images. The proposed method is mainly based on neighborhood similar pixel interpolator (NSPI) and the major contribution are find an effective way to select valid temporal and conjunctive use the temporal advantage to calculate of the target pixel value. Similarity both in spatial and temporal can be obtained in our method. Real multi-temporal Landsat data and missing gap location are used to assess the efficacy of the proposed method. Both qualitative and quantitative evaluations results suggest that our proposed method can predict the missing values very accurately and improve the utilization efficiency of multi-temporal.

In this paper, we propose a novel multiple kernel learning (MKL) framework to incorporate both spectral and spatial features for hyperspectral image classification, which is called multiple-structure-element nonlinear MKL (MultiSE-NMKL). In the proposed framework, multiple structure elements (MultiSEs) are employed to generate extended morphological profiles (EMPs) to present spatial-spectral information. In order to better mine interscale and interstructure similarity among EMPs, a nonlinear MKL (NMKL) is introduced to learn an optimal combined kernel from the predefined linear base kernels. We integrate this NMKL with support vector machines (SVMs) and reduce the min-max problem to a simple minimization problem. The optimal weight for each kernel matrix is then solved by a projection-based gradient descent algorithm. The advantages of using nonlinear combination of base kernels and multiSE-based EMP are that similarity information generated from the nonlinear interaction of different kernels is fully exploited, and the discriminability of the classes of interest is deeply enhanced. Experiments are conducted on three real hyperspectral data sets. The experimental results show that the proposed method achieves better performance for hyperspectral image classification, compared with several state-of-the-art algorithms. The MultiSE EMPs can provide much higher classification accuracy than using a single-SE EMP.

In recent years, many studies on hyperspectral image classification have shown that using multiple features can effectively improve the classification accuracy. As a very powerful means of learning, multiple kernel learning (MKL) can conveniently be embedded in a variety of characteristics. This paper proposes a class-specific sparse MKL (CS-SMKL) framework to improve the capability of hyperspectral image classification. In terms of the features, extended multiattribute profiles are adopted because it can effectively represent the spatial and spectral information of hyperspectral images. CS-SMKL classifies the hyperspectral images, simultaneously learns class-specific significant features, and selects class-specific weights. Using an $L_{1}$-norm constraint (i.e., group lasso) as the regularizer, we can enforce the sparsity at the group/feature level and automatically learn a compact feature set for the classification of any two classes. More precisely, our CS-SMKL determines the associated weights of optimal base kernels for any two classes and results in improved classification performances. The advantage of the proposed method is that only the features useful for the classification of any two classes can be retained, which leads to greatly enhanced discriminability. Experiments are conducted on three hyperspectral data sets. The experimental results show that the proposed method achieves better performances for hyperspectral image classification compared with several state-of-the-art algorithms, and the results confirm the capability of the method in selecting the useful features.

Multiple kernel learning (MKL) combines multiple base kernels and is becoming more and more popular in machine learning. The choice of kernels is crucial importance for classification performance. In this paper, we propose a new RMKL (RMKBoost) framework for classification in hyperspectral images. The classification is performed in separate two steps. The key boosting strategy is embedded in the first step, which aims to learn an optimally or suboptimally linear combined kernel from the predefined base kernels. Then, the proposed boosting framework generates weak multiple kernel classifiers using a part of the base kernels randomly selected rather than using all base kernels with randomly training samples. Experiments are conducted on the real hyperspectral data set, and the corresponding experimental result shows that RMKBoost algorithm provides the best performances compared with the state-of-the-art kernel methods.