The reconstruction from three to dozens of spectral bands, known as spectral super resolution (SSR) has achieved remarkable progress with the continuous development of deep learning. However, the reconstructed hyperspectral images (HSIs) still suffer from the spatial degeneration due to the insufficient retention of high-frequency (HF) information during the SSR process. To remedy this issue, a novel Wavelet-based Hybrid Asymmetric Network (WHANet) is proposed to establish a RGB-to-HSI translation in wavelet domain, thus reserving and emphasizing the HF features in hyperspectral space. Basically, the backbone is designed in a hybrid asymmetric structure that learns the exact representations of decomposed wavelet coefficients in hyperspectral domain in a parallel way. Innovatively, a CNN-based HF reconstruction module (HFRM) and a transformer-based low frequency (LF) reconstruction module (LFRM) are delicately devised to perform the SSR process individually, which are able to process the discriminative wavelet coefficients contrapuntally. Furthermore, a hybrid loss function incorporated with the Fast Fourier loss (FFL) is proposed to directly regularize and emphasis the missing HF components. Eventually, experimental results over three benchmark datasets and one remote sensing dataset demonstrate that our WHANet is able to reach the state-of-the-art performance quantitatively and qualitatively.

Dozens of spectral bands of hyperspectral images (HSIs) have been successfully reconstructed from only three color band images using deep neural networks according to their powerful nonlinear mapping capability. However, the existing deep-learning-based approaches tend to directly reconstruct HSIs from RGB inputs without emphasizing the discriminative intrinsic properties of different materials, resulting in certain distortion in reconstructed spectra. In this article, an intrinsic image decomposition (IID)-based spectral super-resolution (SSR) framework is proposed to reconstruct spectra of pixels from their reflectance feature and shading feature separately, by which the intrinsic properties can be emphasized during spectral reconstruction. Specifically, a dual hierarchical regression network (DHRNet) is designed for the proposed IID-based SSR task, in which a shading feature extraction module (SFEM) based on dense structure and a reflectance feature extraction module (RFEM) with attention mechanism are first, respectively, designed to reconstruct spectral information from reflectance feature and shading feature, and a feature enhancement module (FEM) is consequently devised to further improve the coarse combined estimation. Ultimately, a novel hybrid loss combining smooth$\boldsymbol {l}_{1}$loss, spectral angel mapper (SAM), and gradient prior is also presented to restrain the spectral distortion while enhancing the sharpness of the reconstructed HSI. Experimental results over three datasets demonstrate the superiority of our proposed framework.

Feature representation is crucial for hyperspectral image (HSI) classification. However, existing convolutional neural network (CNN)-based methods are limited by the convolution kernel and only focus on local features, which causes it to ignore the global properties of HSIs. Transformer-based networks can make up for the limitations of CNNs because they emphasize the global features of HSIs. How to combine the advantages of these two networks in feature extraction is of great importance in improving classification accuracy. Therefore, a cross-attention fusion network bridging CNN and Transformer (CAF-Former) is proposed, which can fully utilize the advantages of CNN in local features and Transformer’s long time-dependent feature learning for hyperspectral classification. In order to fully explore the local and global information within an HSI, a Dynamic-CNN branch is proposed to effectively encode local features of pixels, while a Gaussian Transformer branch is constructed to accurately model the global features and long-range dependencies. Moreover, in order to fully interact with local and global features, a cross-attention fusion (CAF) module is proposed as a bridge to fuse the features extracted by the two branches. Experiments over several benchmark datasets demonstrate that the proposed CAF-Former significantly outperforms both CNN-based and Transformer-based state-of-the-art networks for HSI classification.

Aerial person detection (APD) is vital for enhancing search and rescue (SaR) operations, particularly when locating victims in remote, poorly-lit areas. Despite advancements in detection technologies, achieving a balance between detection speed and accuracy on mobile devices in “edge AI” continues to pose challenges. In this article, a lightweight distillation network (APDNet) is proposed for edge deployment of APD, which enables real-time inference as well as minimizes accuracy loss during model transfer. The proposed APDNet employs a distillation network between varying-depth backbones and integrates an 8-bit quantized optimizer to reduce the floating-point operations of network parameters. Specifically, in the teach-assistant distillation (TAD) stage, small student models using random weight initialization are trained with pseudo-labels generated by deeper teacher models, facilitating consistent learning for a more accurate, lighter model. Moreover, a low-precision quantization (LPQ) stage incorporates an offline, quantization-aware training strategy that dynamically adjusts the ranges of weight and activation function float-point values, reducing computational complexity. In order to compensate for the potential accuracy decline, a pluggable tracker updates the position and feature information of persons frame-by-frame, with tracking results integrated with detection outputs to enhance accuracy. Extensive experiments on the Heridal, Manipal-UAV, and VTSaR datasets confirm the effectiveness of APDNet, demonstrating its superior performance in edge-based APD.

Spectral super-resolution (SR), which reconstructs high spatial-resolution hyperspectral images (HSIs) from RGB inputs, has been demonstrated to be one of the effective computational imaging techniques to acquire HSIs. Though deep neural networks have shown their superiority in such a complex mapping problem, existing networks generally involve a very complex structure with huge amounts of parameters, resulting in giant memory occupation. In this article, a lightweight multiresolution feature fusion network (MRFN) is proposed, which adopts a multiresolution feature extraction and fusion framework to fully explore RGB inputs in different scales of resolution. Specifically, a lightweight feature extraction module (LFEM), which adopts cheap convolution and attention mechanisms, is constructed to explore different scales of features under a lightweight structure. Moreover, a hybrid loss function is proposed by encountering not only pixel-value level reconstruction error but also spectral continuity and fidelity. Experiments over three benchmark datasets, i.e., CAVE, Interdisciplinary Computational Vision Laboratory (ICVL), and NTIRE2022 datasets, have demonstrated that the proposed MRFN can reconstruct HSIs from RGB inputs in higher quality with fewer parameters and computational floating-point operations (FLOPs) compared with several state-of-the-art networks.

Spectral super-resolution (SR), which generally reconstructs hyperspectral images (HSIs) from RGB inputs, has attracted lots of attention recently. In this paper, a spectral SR algorithm based on intrinsic image decomposition (IID) is proposed, in which RGB images are decomposed into reflectance images and shading images to fully explore RGB features for HSI reconstruction. Considering that features of the reflectance image are only related to the material of objects, the sparsity of material reflectivity is used to reconstruct the reflectance image of HSI. Moreover, an convonlutional neural network (CNN) is constructed to reconstruct shading parts of HSI. Finally, these two reconstructed results are fused to generate the high spectral resolution HSI and an enhancement network is also designed to further improve the recontruction performance. Experimental results with two benchmark datasets, ICVL and CAVE, demonstrate that the performance of the proposed algorithm is superior to several state-of-the-art spectral SR algorithms.