Improving hyperspectral image (HSI) semantic segmentation by exploiting complementary information from supplementary modalities (termed X-modality) is promising but challenging due to significant differences in imaging sensors, image content, and resolution. Existing methods often underutilize the unique spatial–spectral features of HSIs by processing them uniformly with X-modality data. In addition, current cross-modality fusion strategies often suffer from limited intermodal interaction or significantly increased model complexity. To address these limitations, we propose CoMiX, an asymmetric encoder-decoder architecture with deformable convolutions (DCNs) for HSI-X semantic segmentation. CoMiX includes an encoder with two parallel, interacting backbones and a lightweight all-multilayer perceptron (ALL-MLP) decoder. The encoder consists of four stages, each incorporating 2D DCN blocks for the X-modality to accommodate geometric variations and 3D DCN blocks for HSIs to adaptively capture spatial-spectral features. Each stage also incorporates a Cross-Modality Feature enhancement and eXchange (CMFeX) module and a feature fusion module (FFM). CMFeX exploits spatial-spectral correlations across modalities to recalibrate and enhance modality-specific and modality-shared features, while adaptively exchanging complementary information. Its outputs are subsequently fused in the FFM and propagated to the next stage for further learning. Finally, the ALL-MLP decoder aggregates the fused features from all stages to produce the final predictions. Extensive experiments demonstrate that CoMiX achieves state-of-the-art performance and generalizes well to various multimodal datasets. The CoMiX code will be released soon.

Hyperspectral image (HSI) classification has recently reached its performance bottleneck. Multimodal data fusion is emerging as a promising approach to overcome this bottleneck by providing rich complementary information from the supplementary modality (X-modality). However, achieving comprehensive cross-modal interaction and fusion that can be generalized across different sensing modalities is challenging due to the disparity in imaging sensors, resolution, and content of different modalities. In this study, we propose a local-to-global cross-modal attention-aware fusion (LoGoCAF) framework for HSI-X segmentation. LoGoCAF adopts a two-branch semantic segmentation architecture to learn information from HSI and X modalities. The pipeline of LoGoCAF consists of a local-to-global encoder and a lightweight all multilayer perceptron (ALL-MLP) decoder. In the encoder, convolutions are used to encode local and high-resolution fine details in shallow layers, while transformers are used to integrate global and low-resolution coarse features in deeper layers. The ALL-MLP decoder aggregates information from the encoder for feature fusion and prediction. In particular, two cross-modality modules, the feature enhancement module (FEM) and the feature interaction and fusion module (FIFM), are introduced in each encoder stage. The FEM is used to enhance complementary information by combining information from the other modality across direction-aware, position-sensitive, and channel-wise dimensions. With the enhanced features, the FIFM is designed to promote cross-modality information interaction and fusion for the final semantic prediction. Extensive experiments demonstrate that our LoGoCAF achieves superior performance and generalizes well on various multimodal datasets. Code is available athttps://github.com/xumzhang.

Transformer is a powerful tool for capturing long-range dependencies and has shown impressive performance in hyperspectral image (HSI) classification. However, such power comes with a heavy memory footprint and huge computation burden. In this paper, we propose two types of lightweight self-attention modules (a channel lightweight multi-head self-attention module and a position lightweight multi-head self-attention module) to reduce both memory and computation while associating each pixel or channel with global information. Moreover, we discover that transformers are ineffective in explicitly extracting local and multi-scale features due to the fixed input size and tend to overfit when dealing with a small number of training samples. Therefore, a lightweight transformer (LiT) network, built with the proposed lightweight self-attention modules, is presented. LiT adopts convolutional blocks to explicitly extract local information in early layers and employs transformers to capture long-range dependencies in deep layers. Furthermore, we design a controlled multi-class stratified sampling strategy to generate appropriately sized input data, ensure balanced sampling, and reduce the overlap of feature extraction regions between training and test samples. With appropriate training data, convolutional tokenization, and lightweight transformers, LiT mitigates overfitting and enjoys both high computational efficiency and good performance. Experimental results on several HSI datasets verify the effectiveness of our design.

This paper presents an attention based lightweight network (ALN) for hyperspectral image (HSI) classification, aiming to jointly optimize classification performance and parameter efficiency. It employs a light spectral feature extraction module to learn the spectral information of the input data, and then uses the proposed spatial self-attention module to aggregate the spatial information related to the center pixel. Therefore, it can extract more informative spectral-spatial features related to the classified pixels. Extensive experiments show that the proposed ALN significantly outperforms the state-of-the-art methods. The codes of this work will be available at https://github.com/xmzhang2018.

Canopy clumping index (CI) characterizes the extent of the nonrandom spatial distribution of foliage elements within a canopy and is critical for determining the radiative transfer, photosynthesis, and transpiration processes in the canopy. It is widely perceived that CI increases with zenith angle (θ), because between-crown gaps decrease in size and number with increasing θ. In this study, we demonstrate that this is not always true. Analytical equations between CI and θ are first developed based on widely-used forest canopy gap fraction theories. The results show that the zenith angular variation of CI is closely related to crown projected area or crown shapes (i.e., the ratio of the crown height to its diameter, RHD): CI increases with θ for canopies with “tower” crowns (RHD > 1), but decreases with θ for “umbrella” crowns (RHDin-situmeasurements and multi-angular remote sensing.

Over the past years, the acquisition of hyper-temporal data (HTD) from geostationary orbit remote sensing satellites (GEORSS) has provided numerous new research opportunities. Many factors influence the in-orbit dynamic modulation transfer function (MTF) of GEORSS, making it difficult to satisfy the requirements for space-borne cameras. The MTF compensation (MTFC) technique can effectively optimize the design of dynamic MTFs for GEORSS. The traditional MTFC methods mainly consider the sensor, atmosphere and relative motion of the satellite platform when improving GEORSS image quality. They will introduce new high frequency noise, resulting in image information loss. In this paper, a mixed sparse higher-order non-convex total variation (MS-HONCTV) model-aided MTFC method is proposed. By introducing the group sparse regularization (GSR) term into the MS-HONCTV model, it increases the robustness to noise and hence reduces the degeneration of the MTF produced by satellite’ low pointing stability. The MS-HONCTV model is then applied to solve the problem of image degradation. The quality of remote sensing data is improved by the proposed MTFC and this is achieved without modifying the aperture diameter, focal length, or detector size of the satellite’s optical system. Experimental results show that the proposed MS-HONCTV effectively improves the images’ MTF, SNR, gray mean gradient (GMG) and standard deviation (SD), as evidenced by subjective qualitative analysis and objective quantitative assessments of simulated data, laboratory data, and GF-4 satellite data. Compared with other methods, the SNR of the proposed method is increased by 30%, GMG by 14.21% and SD by 6.3% on average.

Geometric-optical (GO) model suitable for forest plantation (GOFP) is a GO model for forest plantations at the stand level developed in this study based on a four-scale GO model a Geometric-Optical Model for Sloping Terrains-II (GOST2), which simulates the bidirectional reflectance distribution function (BRDF) for natural forest canopies. In most previous GO models, tree distributions are often assumed to meet the Poisson or Neyman model in a forest; therefore, these models are suitable for simulating BRDF for natural forest canopies. However, in forest plantations, tree distributions are proven to meet the hypergeometric model rather than the Poisson or Neyman model at the stand level. GOFP, in which the tree distributions are described using the hypergeometric model, is proposed to simulate the bidirectional reflectance factor (BRF) of forest plantations at the stand level. The area ratios of the four scene components (sunlit foliage, sunlit ground, shaded foliage, and shaded ground) of GOFP compare well with those simulated by a 3-D canopy visualization technique. A comparison is also made against discrete anisotropic radiative transfer, showing that GOFP has the ability to simulate BRF of forest plantations. Another comparison is made against operational land imager and Moderate Resolution Imaging Spectroradiometer surface.

It is quite confusing to effectively monitor and precisely evaluate growing conditions of wheat by using normalized differential vegetation index based on pixel size (NDVIp) as they are significantly different when acquired by the wheat of same growth status with different types of soil background. The wheat canopy normalized differential vegetation index(NDVIc) acquired by one scene of multi-spectral remote sensing image with no soil disturbance, pure black background are similar while soil backgrounds are often different. In view of this situation, based on the fixed wheat canopy spectrum which means the NDVIc is a constant value, this paper selects 9 typical soil types in our country as background in order to study the influence of different soil background types on NDVIp of wheat and analyze the sensitivity of NDVIp of wheat to the vegetation coverage simulated by diverse liner mixed ratio of wheat canopy and soil background on the remote-sensing's pixel scale. The results show that: (1)wheat NDVIp of farmland increases along with the increment of vegetation coverage under the same type of soil background, and vice versa; (2)wheat NDVIp of farmland varies greatly with different soil background types, and the difference decreases while the vegetation coverage exceeds 25%; (3) NDVIp sensitivity also shows a quite difference to vegetation coverage under the diverse soil background types. The influence of soil background on NDVIp sensitivity is the lowest when the vegetation coverage ranges from 25% to 35%.

Gap fraction (GF) and clumping index (CI) play key roles in plant light interception, and therefore they have strong impacts on plant growth and canopy radiative transfer processes. On the one hand, in the field of remote sensing, leaves were often assumed to be randomly distributed in tree crowns in previous researches. While this assumption is not correct for many forest canopies, especially for coniferous forests because trees have obvious hierarchical characteristics in an individual crown; one the other hand, reconstruction of detailed canopy architecture in a computer using methods of three-dimensional (3-D) architectural simulations is very labor-intensive and time-consuming at present. In this paper, structures within crown were described using simple probability models based on plant growing rules instead of 3-D simulations. Gap fraction (GF) and clumping index (CI) of foliage were calculated at branch level within crown. The results show that: (1) GF and CI are closely related to the number of nwb (the number of branches in a branch whorl): GF decrease and CI increase with increasing of nwb; (2) the assumption that branches or leaves are randomly distributed within crowns is not true, especially for larger value of nwb.

As one of the major crops in China, the winter wheat production is directly related to the national economic development, the social stabilization and the food security. Normalized difference vegetation index (NDVI) time series has been widely used in crop identification, while most of the present researches about NDVI time series focused on moderate or low resolution remote sensing images, which affecting the accuracy of winter wheat extraction. With the successful launch of the first satellite GF-1 of China High-resolution Earth Observation System, more possibilities have been provided for construction of NDVI time series with high time resolution and high spatial resolution. Considering the peculiarity of winter wheat, which curvilinear integral of the NDVI time series different from other crops, this can be used for identification of winter wheat. So the paper present a high precision winter wheat identification method, Curvilinear integral method, which makes full use of phenology characteristics of winter wheat based on curvilinear integral of NDVI time series. This method is technologically easy and apparently effective.

Black carbon, organic matter, and mineral dust are the main absorption components in atmospheric aerosols and their fractions determine the wavelength dependence of absorption. We analyze the spectral measurements of imaginary refractive indices (k) from 440 to 1020 nm obtained from ground-based AErosol RObotic NETwork sun-sky radiometer located in Beijing. The results show that the k spectra in Beijing have a wavelength dependent characteristic, with k at 440 nm significantly higher than that at the other three bands. Hence, the ratio of k(440 nm) to k(670 nm) is established to express the k spectral behavior. A significant seasonal variations of k(440 nm)/k(670 nm) is found in Beijing, with maximum value of about 3.5 appeared in March, and minimum value of about 0.9 in August. This is consistent with the variation trends of absorption angstrom exponent, indicating that k(440 nm)/k(670 nm) is also a useful tool in identifying aerosol composition.

Leaf area index (LAI) is an important structure parameter of vegetation system. The quantitative remote sensing can offer two dimensional distribution of LAI. The variation of background, atmospheric condition and canopy anisotropic reflectance were the three factors that can restrain the retrieved accuracy of LAI. Along with the emergence of hyperspectral remote sensor, such as Hyperion, it's possible to calculate LAI using the second derivative method in spectral dimension. The second derivative can reduce the influence of background and improve the accuracy of LAI inversion. In order to integrate the second derivative into physical model and eliminate the influence of canopy reflectance anisotropy, we propose a new directional spectral second derivative method. Firstly a new hybrid canopy model was used, and then the directional spectral second derivative was deduced from the hybrid model, so the effects of anisotropy of canopy reflectance and background were removed in theory. Numerical and field tests show the noise can greatly impact the directional second derivative method. We put forward an innovative noise filtering approach in spectral and space domains, the directional second derivative worked well on the LAI retrieval by Hyperion image.