Fig. 1. Parameters vs. PSNR vs. FLOPs on Manga109 dataset (×4).Fig. 1. Parameters vs. PSNR vs. FLOPs on Manga109 dataset (×4).

Existing normal instance segmentation networks have achieved promising performance in clean scenarios. However, these methods often fail to work well in rainy environments as the appearance of rain causes some important and detailed content to be missing. To solve this challenge, we propose an instance segmentation method in the rain, containing three key elements: pixel enhancement module, feature aggregation block, and rain adaptive learning. These components effectively reduce rain degradation both pixel-level and feature-level, enabling the model to adaptively learn rain-invariant features and extract rich multi-scale context information. Furthermore, to mitigate the scarcity of annotated rainy image instance segmentation datasets, we specially generate a realistic rainy dataset based on a widely used rain synthetic pipeline. Experimental results show that the proposed method obviously outperforms existing state-of-the-art algorithms on rainy scenes, while improving the performance in clear weather.

Remote sensing scene classification (RSSC) seeks to allocate correct semantic labels to remote sensing images. Recently, numerous algorithms have made significant contributions to enhancing the accuracy of RSSC. However, models with high parameters and computational complexity still dominate. To address this issue, we propose a lightweight network architecture, namely Two-Stage TripletNet. In this proposed algorithm, we employ a two-stage optimizing strategy involving label optimization and loss function optimization. First, a KD-tree generated by remote sensing image features is utilized to produce visual labels. Secondly, we establish the triplet sampling method based on the visual and semantic labels of the images. Finally, the triplet loss and cross-entropy loss are jointly applied to train our model. Experimental results on mainstream datasets demonstrate the effectiveness of our proposed framework. Meanwhile, the two-stage optimizing strategy renders our model more competitive compared to other state-of-the-art algorithms.

Remote sensing scene classification aims to assign correct semantic labels to remote sensing images. Many state-of-the-art algorithms have made significant contributions to improving model accuracy. However, these algorithms often involve a large number of parameters and floating-point computations. To address this, we propose a lightweight network architecture named RandommaskFormer. This network initially utilizes random position filtering to reduce model complexity. Next, effective feature interactions are achieved through feature covariance analysis. Finally, we implement a two-stage optimization strategy, incorporating both label optimization and loss function optimization. Among the above strategy, we first perform clustering analysis on remote sensing image features using a KD tree to assign new visual labels. Then, we combine triplet loss with cross-entropy loss to guide model training. Experimental results on three mainstream datasets demonstrate the effectiveness of RandommaskFormer. Additionally, the deployment of the two-stage optimization strategy further improves the model's performance.

Noise interference often occurs, during the collection of point cloud data, which can significantly affect the original color characteristics of the data. Currently, point cloud color denoising methods are usually based on graph structures or filters, whose denoising effects are generally influenced by the construction of the graph or the choice of filter. Different from them, this paper proposes a deep learning network for point cloud color denoising that combines an implicit neural autoencoder with a multi-frame sparse convolutional learning. We propose a novel sparse feature extraction module based on sparse convolution. The use of sparse convolution allows only processing the features at the effective positions of the point cloud, significantly improving computational efficiency. Moreover, considering that valuable information can gradually be diluted through sparse convolution, a pretrained implicit neural autoencoder is employed to map the input point cloud features to a more informative high-dimensional space, allowing for more thorough network learning. Furthermore, we propose a new method for processing point cloud data that compacts the point cloud while retaining all its color features. Finally, to further utilize the spatiotemporal information of the point cloud, a multi-frame fusion algorithm is also proposed. Experimental results show that compared with state-of-the-art algorithms, our proposed method achieves better denoising performance where the average PSNR of denoised point cloud is improved up to 1.14 dB.

Rain streaks bring complicated pixel intensity changes and additional gradients, greatly obstructing the extraction of image features from background. This causes serious performance degradation in feature-based applications. Thus, it is critical to remove rain streaks from a single rainy image to recover image features. Recently, many excellent image deraining methods have made remarkable progress. However, these human visual system-driven approaches mainly focus on improving image quality with pixel recovery as loss function, and neglect how to enhance image feature recovery ability. To address this issue, we propose a task-driven image deraining algorithm to strengthen image feature supply for subsequent feature-based applications. Due to the extensive use and strong practicability of Scale-Invariant Feature Transform (SIFT), we first propose two separate networks using distinct losses and modules to achieve two goals, respectively. One is difference of Gaussian (DoG) pyramid recovery network (DPRNet) for SIFT detection, and the other gradients of Gaussian images recovery network (GGIRNet) for SIFT description. Second, in the DPRNet we propose an alternative interest point loss that directly penalizes scale response extrema to recover the DoG pyramid. Third, we advance a gradient attention module in the GGIRNet to recover those gradients of Gaussian images. Finally, with the recovered DoG pyramid and gradients, we can regain SIFT key points. This divide-and-conquer scheme to set different objectives for SIFT detection and description leads to good robustness. Compared with state-of-the-art methods, experimental results demonstrate that our proposed algorithm achieves better performance in both the number of recovered SIFT key points and their accuracy.

Image deletion refers to removing images from a compressed image set in cloud servers, which has always received much attention. However, in some cases images are not successfully deleted, and coding performance still remains to rise. In this paper, we propose a low-complexity and high-coding-efficiency image deletion algorithm. First, all the images are classified into to-be-deleted images, images unneeded to be processed, and images needed to be processed further divided into images needed to be only decoded and images needed to be re-encoded. Then, we also propose a depth- and subtree-constrained minimum spanning tree (DSCMST) heuristics to produce the DSCMST of images needed to be processed. Third, every image unneeded to be processed is added to the just obtained DSCMST as the child of the vertex that is still its parent in the compressed image set. Finally, after the encoding of images needed to be re-encoded, a new compressed image set is constructed, implying the completion of image deletion. Experimental results show that under various circumstances our proposed algorithm can effectively remove any images, including root vertex, internal vertices, and leaf vertices. Moreover, compared with state-of-the-art methods, the proposed algorithm achieves higher coding efficiency while having the minimum complexity.

Image deletion and image insertion, the current commonly used two kinds of image set management, refer to removing compressed images from and adding new photos to a compressed image set, respectively. However, they do not deal with well the problem of the combination of images selected from multiple compressed image sets. To address this issue, in this paper we first propose an image subset union algorithm for compressed image sets. Image subset union aims to integrate image subsets derived from multiple compressed image sets into a new compressed image set. First, we put forward a way to identify the root vertex candidate of the new compressed image set. Second, we classify all the images of image subsets into three categories: images needed to be re-encoded, images needed to be only decoded, and images unneeded to be re-encoded or decoded. Third, we also employ minimum spanning tree production, a proposed vertex layer candidate assignment method, depth- and subtree-constrained minimum spanning tree generation, as well as image re-encoding to build the new compressed image set. Experimental results show that our proposed algorithm can effectively accomplish image subset union with low complexity.

Due to the adverse effects of far more complex rain streaks on the performance of outdoor vision systems, it is critical to accurately eliminate rain streaks from a single rainy image. Recently, many algorithms have made significant progress in rain streak removal. However, these approaches do not give enough consideration to the established fact that rain streaks reflected in different wavelet subbands may have distinct spatial and intensity distributions, resulting in residual degraded components in subbands if not addressed properly. To deal with this problem, we propose a subband differentiated learning network (SDLNet) for rain streak removal. First, we put forward a wavelet subband mask module (WSMM) to produce a wavelet mask, in which different concentrations of rain streak features are provided for different subbands. Second, we advance a WSMM-driven repetitive learning resblock (WSMM-driven RLR) with a novel repetitive learning mechanism (RLM). This RLM enables the WSMM-driven RLR to further probe rain streak features. Third, we propose an adaptive wavelet subband loss (AWSL) function, with the same loss structure but a different loss weight for each subband, exploiting unique error priors in subbands. Finally, the proposed SDLNet employs both a UNet-like architecture with the novel WSMM-driven RLR and the AWSL function, generating a derained image. Compared with state-of-the-art methods, experimental results show that our proposed SDLNet achieves better rain streak elimination performance, where the average PSNR of derained images is improved up to 1.86 dB, while recovering more texture details of clean images with better subjective quality in recovered images.

Scale-Invariant Feature Transform (SIFT) plays a significant role in vision applications. Indeed, even with the advent of deep learning, SIFT-based object re-identification has been found to be competitive in a variety of object and scene recognition challenges. This is a relatively robust method. But under severe imaging conditions like hazy, the haze greatly hinders SIFT detection, causing matching performance degradation. To solve the problem, in this paper we propose a deep learning SIFT recovery algorithm from a single hazy image with a two-task framework, namely, Difference of Gaussian (DoG) Pyramid and Gradient Image Recovery (DPGIR). One task is to recover the dehazed DoG pyramid of a hazy image, and the other is to recover its dehazed Gaussian gradients. This scheme to set different objectives for SIFT detection and description leads to very robust performance. Compared with state-of-the-art methods, experimental results demonstrate that our proposed algorithm recovers more SIFT key points.

Scale-Invariant Feature Transform (SIFT) is one of the most well-known image matching methods, which has been widely applied in various visual fields. Because of the adoption of a difference of Gaussian (DoG) pyramid and Gaussian gradient information for extrema detection and description, respectively, SIFT achieves accurate key points and thus has shown excellent matching results but except under adverse weather conditions like rain. To address the issue, in the paper we propose a divide-and-conquer SIFT key points recovery algorithm from a single rainy image. In the proposed algorithm, we do not aim to improve quality for a derained image, but divide the key point recovery problem from a rainy image into two sub-problems, one being how to recover the DoG pyramid for the derained image and the other being how to recover the gradients of derained Gaussian images at multiple scales. We also propose two separate deep learning networks with different losses and structures to recover them, respectively. This divide-and-conquer scheme to set different objectives for SIFT extrema detection and description leads to very robust performance. Experimental results show that our proposed algorithm achieves state-of-the-art performances on widely used image datasets in both quantitative and qualitative tests.