Reflections on the water surface hinder the extraction of valuable information from water surface images. To remove reflections from water surface images, we construct a synthetic dataset and propose a multi-task network for water surface reflection detection and removal. Specifically, we first use a U-Net-based reflection detection module to generate a reflection mask, followed by a GAN-based network to remove the reflection. To extract multi-level features from the images, we design a color feature extraction network and a detail feature extraction network. Finally, to enhance the model's ability to remove large-area reflections, we pre-train the reflection removal network on an image restoration dataset. Experimental results on the proposed synthetic dataset and real water surface reflection images from the Internet show that our method significantly outperforms other methods in water surface reflection detection and removal.

With the rapid advancements in object detection, oriented object detection has gained increasing attention. However, challenges such as boundary discontinuity and square-like problems in oriented object detection persist, as most existing methods directly regress the rotation angle, leading to instability in boundary angle prediction. To address these challenges, this paper introduces a novel rotation angle encoding method called the Composite Trigonometric Function Coder (CTFC), which transforms discrete angles into continuous curves. Leveraging the smooth characteristics of trigonometric functions, CTFC eliminates abrupt curvature changes, thereby avoiding the sudden angle steps inherent in conventional methods and simplifying the optimization process. Experiments conducted on three datasets validate the effectiveness of the proposed method. Additionally, the performance of CTFC has been further analyzed using three different detector heads. Experimental results and data analysis demonstrate that CTFC is efficient and effective in oriented object detection.

Video anomaly detection (VAD) methods are mostly CNN-based or Transformer-based, achieving impressive results, but the focus on detection accuracy often comes at the expense of inference speed. The emergence of state space models in computer vision, exemplified by the Mamba model, demonstrates improved computational efficiency through selective scans and showcases the great potential for long-range modeling. Our study pioneers the application of Mamba to VAD, dubbed VADMamba, which is based on multi-task learning for frame prediction and optical flow reconstruction. Specifically, we propose the VQ-Mamba Unet (VQ-MaU) framework, which incorporates a Vector Quantization (VQ) layer and Mamba-based Non-negative Visual State Space (NVSS) block. Furthermore, two individual VQ-MaU networks separately predict frames and reconstruct corresponding optical flows, further boosting accuracy through a clip-level fusion evaluation strategy. Experimental results validate the efficacy of the proposed VADMamba across three benchmark datasets, demonstrating superior performance in inference speed compared to previous work. Code is available at https://github.com/jLooo/VADMamba.

Hyperspectral anomaly detection (HAD) aims to identify spectrally distinct pixels within a hyperspectral image (HSI). This task necessitates capturing both local spectral information and spatial smoothness, posing a significant challenge for traditional methods. This letter proposes a novel autoencoder framework that leverages a Gaussian-inspired attention mechanism to address this challenge effectively. Specifically, we introduce a novel Gaussian attention layer embedded within the encoder. This layer utilizes a learnable Gaussian kernel to prioritize the local neighborhood of each pixel. This approach effectively captures fine-grained features crucial for background reconstruction. The learned representations are then passed through a deep autoencoder architecture to reconstruct anomaly-free data. Pixels with significant reconstruction errors are subsequently flagged as anomalies. Experiments on several datasets demonstrate the effectiveness of the proposed approach. Compared to existing methods, our framework achieves superior performance in terms of detection accuracy. This finding highlights the potential of Gaussian-inspired attention mechanisms for enhancing HAD. The code is released at:https://github.com/rk-rkk/Gaussian-Inspired-Attention-Mechanism-for-Hyperspectral-Anomaly-Detection.

Benefiting from the high spectral resolution, hyperspectral image (HSI) owns the property of discriminating material. However, the spatial resolution of HSIs is limited by the hardware and, thus, makes HSI super-resolution (SR) a necessary and hot topic. Existed HSI SR methods rarely consider the high-frequency edge information, resulting in unsatisfactory quality of spatial reconstruction. To address the problem mentioned above, we propose a novel method for HSI SR named attention-driven dual feature guidance net (AD-DFGNet), which makes full use of the spatial–spectral information. Specifically, AD-DFGNet mainly consists of three modules, which are shallow feature extraction, DFG blocks, and upsampling fusion module. First, the three adjacent bands of an HSI cube are selected sequentially as one of the inputs to the proposed AD-DFGNet. Their feature dimension is upgraded through shallow feature extraction. Second, middle band of three adjacent bands is used as the other input to DFG block, and it makes the network focus on spatial feature extraction by DFG, which includes feature aggregation guidance (FAG) and gradient–texture attention (GTA) guidance. Then, the output features of DFG blocks are upsampled and fused, in turn, to obtain the single-band SR result. Finally, the SR results of the whole HSI are obtained from the sequential concatenation of each single band. In addition, the proposed method reduces the size of the generated model and makes it possible to apply on different datasets. The efficiency of the AD-DFGNet is validated on three publicly available hyperspectral datasets and yields the state-of-the-art performance.

Abstract Rain can severely hamper the visibility of scene objects. Although existing deep learning methods have reported promising performance, they often fail to obtain satisfactory results in many practical situations, especially when the input image contains both rain streaks and haze‐like degradation. In this paper, a new two‐stage method based on attention smoothed dilated network (SDN) is proposed. Unlike most fully‐supervised methods, the mixture of rain streaks and haze‐like effects is considered in the model. The proposed method consists of two stages. First, a generative adversarial network guided by the rain‐streak attention map is proposed to remove rain streaks, where a multi‐stage attention module is used to accurately locate rain streaks in the generator. Second, haze‐like effects are further removed through SDN with the same structure as the generator. Extensive experiments on multiple datasets show that the method outperforms the state‐of‐the‐art in both objective evaluation and visual quality.

A novel hyperspectral anomaly detection method is proposed in this letter. This method is motivated by two important observations. First, there are hundreds of bands in a hyperspectral image (HSI), among which redundant bands and some noisy bands could be eliminated to increase the discriminability from the spectral domain. Meanwhile, considering the small amount of the anomalies, they may be neglected in the spatial clustering process, and the anomalies can be highlighted by eliminating the clustered result from the original HSI. In this way, the spatial-spectral extraction is proposed to detect the anomalies in the HSI. First, an iterative optimal neighborhood reconstruction (IONR) method is utilized to select the bands while preserving their divergence. Second, the spatial clustering is achieved by minimizing the intradistance in any given category. Finally, the spatial clustered HSI is eliminated from the band selected HSI and acts as the direct input for the detection process. Experimental results and data analysis have demonstrated the effectiveness of the proposed method.

In this paper, a novel hyperspectral image (HSI) anomaly detection method is proposed. This method is inspired by three ideas. First, the spatial resolution of the HSIs is sacrificed for their spectral information. Structural information of the HSIs tends to be smooth and distorts from that of the real scene. Second, with a loose false alarm rate, it is not difficult to pick out all the anomalies. Third, gradients of these probable anomalies can be transformed to enhance the spatial information of the HSI. Meanwhile, it is desirable that the enhanced HSI could be detected more precisely. Three modules are designed with respect to these three ideas, which are locating the probable pixels, local gradient guidance, and the anomaly detection for the enhanced HSI. Specifically, some probable anomalies are firstly selected. Secondly, the gradients of these selected pixels are transformed and utilized to guide the spatial enhancement for the HSI locally. Finally, the final detection is implemented on the enhanced HSI. Experimental results obtained on four real HSIs demonstrate the effectiveness of the proposed method.

Abstract Single image de‐raining based on convolutional neural network (CNN) has made considerable progress in recent years. However, usually the de‐rained result has dark artifacts and image textures tend to be over‐smoothed. In this paper, a pyramid non‐local enhanced residual dense network is proposed to reduce such distortion. Firstly, the down‐sampled images are input into the Laplacian pyramid, which can extract the overall and partial texture clues, and subsequently a set of images of different scales are produced. Secondly, these images are fed into a non‐local enhanced residual dense block, which can not only capture long‐distance dependencies of feature maps, but also fully utilizes the hierarchical features in every dense block, leading to high accuracy of rain streaks extraction and better preservation of image edge detail. Finally, the de‐rained image is gradually restored by Gaussian reconstruction pyramid. Experimental results on both synthetic data and real‐world data show that the artifacts distortion is obviously reduced by the proposed network. And the quality of de‐rained image is significantly improved compared with the state‐of‐the‐art methods.

Abstract Salient target detection in hyperspectral image is a significant task in image segmentation, target tracking, image classification and so on. Many existing saliency detection algorithms for hyperspectral image detection cannot present the boundary of the salient target well and the description of the target is not enough. A method based on visual attention to detect the salient target of hyperspectral image is proposed in this paper. In this method, frequency‐tuned (FT) salient detection model is combined with spectral salient to detect target in hyperspectral image. FT model is used to get target with clear border, and spectral information is made full use of to improve the accuracy of target detection. Firstly, FT is used to detect saliency of hyperspectral image and the saliency map is generated. Then, spectral information of the hyperspectral image is measured by similarity, and the spectral saliency is obtained by calculating spectral angle distance between the spectral vectors. Finally, the FT's saliency map and the spectral saliency map are combined to form the final saliency target maps. Experimental results show that our method is superior to other methods in saliency target detection of hyperspectral image, and the precision‐recall curve and F‐measure are better as well.

Hyperspectral image (HSI) super-resolution is currently attracting great interest in remote sensing, since it allows the generation of high spatial resolution HSIs and circumventing the main limitation of the imagery sensors. This paper proposes a novel deep intra fusion network (IFN) for the HSI super-resolution, in which both the spatial and the spectral information have been fully and automatically exploited. Specifically, parallel convolutions are applied to two adjacent bands and their difference band, and obtain the high-dimensional features. Meanwhile, an automatically aggregation module is applied in the IFN to achieve the intra-fusion between these features. In this way, both the spatial information of the current band and the spectral information between neighboring bands are utilized in the super-resolving process. Experimental results and data analysis suggest the effectiveness of the proposed method.

This article presents an intrafusion network (IFN) for hyperspectral image (HSI) super-resolution (SR). Given that the HSI is a 3-D data cube with both the spatial information and the spectral information, the key challenge to construct HSI SR is how to efficiently exploit the spectral information among consecutive low-resolution (LR) bands, besides the spatial information. The proposed IFN consists of three modules, including the spectral difference module, the parallel convolution module, and the intrafusion module, which directly utilizes both the spatial information and the spectral information for reconstructing the high-resolution HSI. Different from most of the existed methods that tackle the spatial and spectral information separately, the proposed spatial-spectral utilization is achieved in one integrated network, which opens up a new way for HSI SR. Meanwhile, applications of this three modules strategy (first spectral difference, then parallel convolution, and finally, intrafusion) on both the conventional convolutional neural network and the residual network with deeper depth have shown the generalization capacity of this proposal. Experimental results and data analysis demonstrate the effectiveness of the proposed method using three hyperspectral data sets.

Limited by existing electromagnetic sensors, the hyperspectral image (HSI) is characterized by having a high spectral resolution but a low spatial resolution. The super-resolution (SR) technique, which aims at enhancing the spatial resolution of the input image, is a hot topic in computer vision. This paper presents a rapid HSI SR method based on a deep information distillation network (IDN) and an intra-fusion operation to fully utilize the spatial-spectral information. Specifically, some bands are firstly selected and super-resolved by utilizing their spatial information through IDN. Non-selected bands are super-resolved by spectral interpolation. Moreover, to take a full advantage of the information these non-selected bands conveys, intra-fusion is operated on the input HSI and the spectrally-interpolated high resolution HSI. Contrary to most existed fusion methods which require multiple observations of the same scene, this intra-fusion is more flexible, and makes further utilization of the information the input HSI conveys simultaneously. In addition, this method requires less computation and is more suitable for practical applications. Experimental data and comparative analysis have demonstrated the effectiveness this method.

With success of convolutional neural networks (CNNs) in computer vision, the CNN has attracted great attention in hyperspectral classification. Many deep learning-based algorithms have been focused on deep feature extraction for classification improvement. In this letter, a novel deep learning framework for hyperspectral classification based on a fully CNN is proposed. Through convolution, deconvolution, and pooling layers, the deep features of hyperspectral data are enhanced. After feature enhancement, the optimized extreme learning machine (ELM) is utilized for classification. The proposed framework outperforms the existing CNN and other traditional classification algorithms by including deconvolution layers and an optimized ELM. Experimental results demonstrate that it can achieve outstanding hyperspectral classification performance.

The image super-resolution (SR) technique, which aims at reconstructing a high-resolution (HR) image from a single low-resolution (LR) image, is a classical problem in computer vision. Limited by the imaging hardware, the spatial resolution of a hyperspectral images (HSI) is usually very coarse. Meanwhile, the spectral information of the HSI is extremely important for its applications and cannot be severely distorted. This paper presents a spatial constraint (SCT) strategy with combination of a deep learning method for HSI SR. The SCT strategy restraints the LR HSI generated by the reconstructed HR HSI should be spatially close to the input LR HSI. The deep learning method learns an end-to-end mapping between the spectral difference of the LR HSI and that of the HR HSI. The mapping is represented as a deep convolutional neural network (CNN). The CNN learned spectral difference is utilized to super-resolve the LR HSI while preserve the important spectral information of the desired HR HSI. Experiments have been conducted on three databases that contains both indoor scenes and outdoor scenes. Comparative analyses have verified the effectiveness of the overall method.

A hyperspectral image (HSI) super-resolution (SR) is a highly attractive topic in computer vision. However, most existed methods require an auxiliary high-resolution (HR) image with respect to the input low-resolution (LR) HSI. This limits the practicability of these HSI SR methods. Moreover, these methods often destroy the important spectral information. This letter presents a deep spectral difference convolutional neural network (SDCNN) with the combination of a spatial-error-correction (SEC) model for HSI SR. This method allows for full exploration of the spectral and spatial correlations, which achieves a good spatial information enhancement and spectral information preservation. In the proposed method, the key band is automatically selected and super-resolved with the boundary bands. Meanwhile, spectral difference mapping between the LR and HR HSIs can be learned by the SDCNN, and then be transformed according to the SEC model, which aims at correcting the spatial error while preserving the spectral information. The rest nonkey bands will be super-resolved under the guidance of the transformed spectral difference. Experimental results on synthesized and real-scenario HSIs suggest that the proposed method: (1) achieves comparable performance without requiring any auxiliary images of the same scene and (2) requires less computation time than the state-of-the-art methods.

In High Efficiency Video Coding (HEVC), intra coding plays an important role, but also involves huge computational complexity due to a flexible coding unit (CU) structure and a large number of prediction modes. This paper presents a fast algorithm based on the sole- and multi-depth measurements to reduce the complexity from CU and prediction mode decisions. For the CU decision, evaluation results with sole and multiple depths are utilized to judge if the CU is a heterogeneous, homogeneous, or depth prominent one, where fast CU decisions are made. For the prediction mode decision, the tendencies for different CU sizes are detected based on multiple depths. The number of searching modes is decreased adaptively for the depth with fewer tendencies. Experimental results show the proposed algorithm reduces 61.49% computational complexity, with 0.75% bit-rate increasing, which is more efficient than state-of-the-arts.