Recently, state space models (SSM) with efficient design, i.e., Mamba, have shown great potential in modeling long-range dependencies with linear complexity. However, the pure SSM-based model yields sub-optimal underwater enhancement performance due to insufficient local details. Given the superiority of convolution in local perception, we propose a hybrid network, named UWMamba, which combines SSM and convolution for underwater image enhancement. We introduce a conv mamba layer (CML) as the foundation layer to combine the visual state space block (VSSB) with convolution. The convolution is used to capture local detailed features, while the VSSB is employed to capture long-range global features, which complement each other. Furthermore, considering underwater images suffer from severe and uneven degradation of spatial regions and color channels, we propose a Mamba Attention Fusion Module (MAFM), which fuses VSSB with an attention mechanism for better perception of channels and spatial regions. Extensive experiments on real-world underwater image datasets demonstrate the promising performance of our method in both objective metrics and subjective comparisons.

Replacing CCD and CMOS image sensors in conventional cameras with digital micromirror devices (DMD), single-pixel cameras low-costly shot images by capturing compressed measurements and computation. However, the compressed measurements lack explicit spatial information, causing difficulties for high-level tasks such as salient object detection (SOD) that are usually designed to have visual inputs. To address the issue, we propose a single-pixel imaging-based SOD network called SPISODNet that enables predicting saliency maps directly from compressed measurements with high accuracy. Specifically, we first design an underlying feature inversion module (UFIM) to capture the underlying scene information, and then develop a context-aware flow (CAF) consisting of a feature focus module (FFM), three bidirectional attention modules (BAMs), and a spatial information-induced attention module (SIAM) to acquire and polish saliency predictions. Extensive experiments demonstrate that our method achieves superior performance for single-pixel imaging-based SOD.

Due to the excellent semantics extraction capabilities, deep learning methods have significantly progressed in salient object detection (SOD). However, these methods often require time-consuming pre-training and large training datasets with ground truth. To address these issues, by referring to the framework known as “deep image prior (DIP),” we propose a SOD method called deep label prior network (DLPNet), which consists of$\mathcal A$-stream and$\mathcal B$-stream. The$\mathcal A$-stream includes two cascaded UNets and a simple CNNs module to extract the initial saliency map, while the$\mathcal B$-stream contains only two cascaded UNets, which refines the extracted initial saliency map. Unlike most of the current deep learning methods, DLPNet views the SOD task as a conditional image generation problem, relying on only the internal prior of the input itself to generate the saliency map. Hence, our DLPNet does not require pre-training or large annotated / unannotated datasets. Furthermore, we propose a morphology operation scheme, which creates rich pseudo-labels for facilitating the updating of network weights. Extensive experiments demonstrate that our method outperforms state-of-the-art unsupervised techniques and is even comparable to state-of-the-art supervised and weakly supervised methods on different evaluation metrics.

The performance of nighttime semantic segmentation is restricted by the poor illumination and a lack of pixel-wise annotation, which severely limit its application in autonomous driving. Existing works, e.g., using the twilight as the intermediate target domain to perform the adaptation from daytime to nighttime, may fail to cope with the inherent difference between datasets caused by the camera equipment and the urban style. Faced with these two types of domain shifts, i.e., the illumination and the inherent difference of the datasets, we propose a novel domain adaptation framework via cross-domain correlation distillation, called CCDistill. The invariance of illumination or inherent difference between two images is fully explored so as to make up for the lack of labels for nighttime images. Specifically, we extract the content and style knowledge contained in features, calculate the degree of inherent or illumination difference between two images. The domain adaptation is achieved using the invariance of the same kind of difference. Extensive experiments on Dark Zurich and ACDC demon-strate that CCDistill achieves the state-of-the-art performance for nighttime semantic segmentation. Notably, our method is a one-stage domain adaptation network which can avoid affecting the inference time. Our implementation is available at https://github.com/ghuan99/CCDistill.

Real-time semantic segmentation plays a significant role in many real-world applications. However, existing methods usually neglect the importance of aggregating global scene clues and multi-level semantics due to computational limits of mobile devices. To address the above challenges and maintain higher accuracy, we propose an efficient attention-augmented network, namely EANet. Specifically, we first leverage an extremely lightweight attention module called sparse strip attention module (SSAM) to retain global contextual information while greatly reducing computation cost. Moreover, the meticulously designed joint attention fusion module (JAFM) follows an attention strategy to efficiently integrate semantics and details from multi-level features. On Cityscapes test set, our network achieves 74.6% mIoU at 35.4 FPS on a single GTX1080Ti GPU with a 1024×2048-pixel image. Extensive experiments show that our EANet achieves promising results on Cityscapes dataset.

Underwater object detection plays an important role in a variety of marine applications. However, the complexity of the underwater environment (e.g. complex background) and the quality degradation problems (e.g. color deviation) significantly affect the performance of the deep learning-based detector. Many previous works tried to improve the underwater image quality by overcoming the degradation of underwater or designing stronger network structures to enhance the detector feature extraction ability to achieve a higher performance in underwater object detection. However, the former usually inhibits the performance of underwater object detection while the latter does not consider the gap between open-air and underwater domains. This paper presents a novel framework to combine underwater object detection with image reconstruction through a shared backbone and Feature Pyramid Network (FPN). The loss between the reconstructed image and the original image in the reconstruction task is used to make the shared structure have better generalization capability and adaptability to the underwater domain, which can improve the performance of underwater object detection. Moreover, to combine different level features more effectively, UNet-based FPN (UFPN) is proposed to integrate better semantic and texture information obtained from deep and shallow layers, respectively. Extensive experiments and comprehensive evaluation on the URPC2020 dataset show that our approach can lead to 1.4% mAP and 1.0% mAP absolute improvements on RetinaNet and Faster R-CNN baseline with negligible extra overhead. The code is available at https://github.com/BIGWangYuDong/uwtoolbox.

Abstract Unmanned aerial vehicles (UAVs) are widely used in wireless communication networks due to their rapid deployment and high mobility. However, in practical scenarios, the existence of obstacles and eavesdroppers will seriously interfere with the communication quality of the UAV network and produce a security risk. Thus, this paper combines reconfigurable intelligent surface (RIS) technology with UAVs to build a secure UAV communication network. Normally, a rotary‐wing UAV (labeled as UAV‐S) acting as a base station sends information signals to a legitimate user on the ground with RIS equipment. However, there is a passive eavesdropper on the ground who can steal the information. Therefore, a friendly UAV jammer (labeled as UAV‐J) with a fixed location is introduced to send jamming signals to confuse the eavesdropper. The goal of this paper is to maximize the average secrecy rate of the communication network by jointly optimizing the flight trajectory, transmit power of the UAV‐S and UAV‐J, and phase shifter of the RIS. Since the constructed problem is highly nonconvex, an alternating optimization algorithm based on successive convex approximation techniques is proposed to solve the problem. Simulation results show that the proposed algorithm can achieve a higher secrecy rate in comparison with other schemes.

The paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network. Our method trains a lightweight deep network, DCE-Net, to estimate pixel-wise and high-order curves for dynamic range adjustment of a given image. The curve estimation is specially designed, considering pixel value range, monotonicity, and differentiability. Zero-DCE is appealing in its relaxed assumption on reference images, i.e., it does not require any paired or unpaired data during training. This is achieved through a set of carefully formulated non-reference loss functions, which implicitly measure the enhancement quality and drive the learning of the network. Our method is efficient as image enhancement can be achieved by an intuitive and simple nonlinear curve mapping. Despite its simplicity, we show that it generalizes well to diverse lighting conditions. Extensive experiments on various benchmarks demonstrate the advantages of our method over state-of-the-art methods qualitatively and quantitatively. Furthermore, the potential benefits of our Zero-DCE to face detection in the dark are discussed.

During recent years, we have witnessed a rapid development of wireless network technologies which have revolutionized the way people take and share multimedia content. However, images captured in the outdoor scenes usually suffer from limited visibility due to suspended atmospheric particles, which directly affects the quality of photos. Despite the recent progress of image dehazing methods, the visual quality of dehazed results still needs further improvement. In this paper, we propose a deep convolutional neural network (CNN) for single image dehazing called PDR-Net, which includes a perception-inspired haze removal subnetwork that reconstructs the latent dehazed image and a refinement subnetwork that further enhances the contrast and color properties of the dehazed result by joint multi-term loss optimization. Compared to the previous methods, our method combines the advantages of existing indoor and outdoor image dehazing training data, which makes the proposed PDR-Net generalized to various hazy images and effective for improving the visual quality of the dehazed results. Extensive experiments demonstrate that the proposed method achieves comparable and even better performance on both real and synthetic images in qualitative and quantitative metrics. Additionally, the potential usage of our method in high-level vision tasks is discussed.

Recently, text images deblurring has achieved advanced development. Unlike previous methods based on hand‐crafted priors or assume specific kernel, the authors recognise the text deblurring problem as a semantic generation task, which can be achieved by a generative adversarial network. The structure is an essential property of text images; thus, they propose a structural loss function and a detailed loss function to regularise the recovery of text images. Furthermore, they learn from the coarse‐to‐fine strategy and present a multi‐scale generator, which is utilised for sharpening the generated text images. The model has a robust capability of generating realistic latent images with photo‐quality effect. Extensive experiments on the synthetic and real‐world blurry images have shown that the proposed network is comparable to the state‐of‐the‐art methods.

Zero-shot learning (ZSL) is typically achieved by resorting to a class semantic embedding space to transfer the knowledge from the seen classes to unseen ones. Capturing the common semantic characteristics between the visual modality and the class semantic modality (e.g., attributes or word vector) is a key to the success of ZSL. In this paper, we propose a novel encoder-decoder approach, namely latent space encoding (LSE), to connect the semantic relations of different modalities. Instead of requiring a projection function to transfer information across different modalities like most previous work, LSE performs the interactions of different modalities via a feature aware latent space, which is learned in an implicit way. Specifically, different modalities are modeled separately but optimized jointly. For each modality, an encoder-decoder framework is performed to learn a feature aware latent space via jointly maximizing the recoverability of the original space from the latent space and the predictability of the latent space from the original space. To relate different modalities together, their features referring to the same concept are enforced to share the same latent codings. In this way, the common semantic characteristics of different modalities are generalized with the latent representations. Another property of the proposed approach is that it is easily extended to more modalities. Extensive experimental results on four benchmark datasets [animal with attribute, Caltech UCSD birds, aPY, and ImageNet] clearly demonstrate the superiority of the proposed approach on several ZSL tasks, including traditional ZSL, generalized ZSL, and zero-shot retrieval.

Rapid development of affordable and portable consumer depth cameras facilitates the use of depth information in many computer vision tasks such as intelligent vehicles and 3D reconstruction. However, depth map captured by low-cost depth sensors (e.g., Kinect) usually suffers from low spatial resolution, which limits its potential applications. In this paper, we propose a novel deep network for depth map super-resolution (SR), called DepthSR-Net. The proposed DepthSR-Net automatically infers a high resolution (HR) depth map from its low resolution (LR) version by hierarchical features driven residual learning. Specifically, DepthSR-Net is built on a residual U-Net deep network architecture. Given LR depth map, we first obtain the desired HR by bicubic interpolation upsampling, and then construct an input pyramid to achieve multiple level receptive fields. Next, we extract hierarchical features from the input pyramid, intensity image, and encoder-decoder structure of UNet. Finally, we learn the residual between the interpolated depth map and the corresponding HR one using the rich hierarchical features. The final HR depth map is achieved by adding the learned residual to the interpolated depth map. We conduct an ablation study to demonstrate the effectiveness of each component in the proposed network. Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art methods. Additionally, the potential usage of the proposed network in other low-level vision problems is discussed.

Zero-Shot Learning (ZSL) is generally achieved via aligning the semantic relationships between the visual features and the corresponding class semantic descriptions. However, using the global features to represent fine-grained images may lead to sub-optimal results since they neglect the discriminative differences of local regions. Besides, different regions contain distinct discriminative information. The important regions should contribute more to the prediction. To this end, we propose a novel stacked semantics-guided attention (S2GA) model to obtain semantic relevant features by using individual class semantic features to progressively guide the visual features to generate an attention map for weighting the importance of different local regions. Feeding both the integrated visual features and the class semantic features into a multi-class classification architecture, the proposed framework can be trained end-to-end. Extensive experimental results on CUB and NABird datasets show that the proposed approach has a consistent improvement on both fine-grained zero-shot classification and retrieval tasks.

Underwater vision suffers from severe effects due to selective attenuation and scattering when light propagates through water. Such degradation not only affects the quality of underwater images, but limits the ability of vision tasks. Different from existing methods that either ignore the wavelength dependence on the attenuation or assume a specific spectral profile, we tackle color distortion problem of underwater images from a new view. In this letter, we propose a weakly supervised color transfer method to correct color distortion. The proposed method relaxes the need for paired underwater images for training and allows the underwater images being taken in unknown locations. Inspired by cycle-consistent adversarial networks, we design a multiterm loss function including adversarial loss, cycle consistency loss, and structural similarity index measure loss, which makes the content and structure of the outputs same as the inputs, meanwhile the color is similar to the images that were taken without the water. Experiments on underwater images captured under diverse scenes show that our method produces visually pleasing results, even outperforms the state-of-the-art methods. Besides, our method can improve the performance of vision tasks.

As an important and challenging problem in computer vision, zero-shot learning (ZSL) aims at automatically recognizing the instances from unseen object classes without training data. To address this problem, ZSL is usually carried out in the following two aspects: 1) capturing the domain distribution connections between seen classes data and unseen classes data and 2) modeling the semantic interactions between the image feature space and the label embedding space. Motivated by these observations, we propose a bidirectional mapping-based semantic relationship modeling scheme that seeks for cross-modal knowledge transfer by simultaneously projecting the image features and label embeddings into a common latent space. Namely, we have a bidirectional connection relationship that takes place from the image feature space to the latent space as well as from the label embedding space to the latent space. To deal with the domain shift problem, we further present a transductive learning approach that formulates the class prediction problem in an iterative refining process, where the object classification capacity is progressively reinforced through bootstrapping-based model updating over highly reliable instances. Experimental results on four benchmark datasets (animal with attribute, Caltech-UCSD Bird2011, aPascal-aYahoo, and SUN) demonstrate the effectiveness of the proposed approach against the state-of-the-art approaches.

Zero-shot learning (ZSL) endows the computer vision system with the inferential capability to recognize new categories that have never seen before. Two fundamental challenges in it are visual-semantic embedding and domain adaptation in cross-modality learning and unseen class prediction steps, respectively. This paper presents two corresponding methods named Adaptive STructural Embedding (ASTE) and Self-PAced Selective Strategy (SPASS) for both challenges. Specifically, ASTE formulates the visual-semantic interactions in a latent structural support vector machine framework by adaptively adjusting the slack variables to embody different reliablenesses among training instances. To alleviate the domain shift problem in ZSL, SPASS borrows the idea from self-paced learning by iteratively selecting the unseen instances from reliable to less reliable to gradually adapt the knowledge from the seen domain to the unseen domain. Consequently, by combining SPASS and ASTE, we present a self-paced Transductive ASTE (TASTE) method to progressively reinforce the classification capacity. Extensive experiments on three benchmark data sets (i.e., AwA, CUB, and aPY) demonstrate the superiorities of ASTE and TASTE. Furthermore, we also propose a fast training (FT) strategy to improve the efficiency of most existing ZSL methods. The FT strategy is surprisingly simple and general enough, which speeds up the training time of most existing ZSL methods by 4~300 times while holding the previous performance.

For the large number of the image data produced by the sensor nodes in wireless multimedia sensor networks (WMSNs), the reduction of the sensor data and energy efficient transmission of this data are the most challenging problem. Compressed sensing based image compression provides the dramatic reduction of image sampling rates, energy consumption for WMSNs data collection and transmission. For better suitable for real-time sensing of image and reduce the sensing matrix size, block compressed sensing method acquire and process image in a block-by-block manner, by the same operator. Recently, nonlocal sparsity has been evidenced to improve the reconstruction of image details in various compressed sensing studies. In this paper, based on block compressed sensing, both the local sparsity of the whole image and nonlocal sparsity of similar patches are integrated into a compressed sensing recovery framework. And the nonlocal sparsity of images is characterized by the low-rank approximation. For better approximation of the rank function, the non-convex low-rank regularization namely Schatten p-norm minimization is applied for block compressed sensing recovery. The experiments show that the proposed algorithm can reduce the reconstruction error and improve the quality of the reconstruction image. Moreover, the proposed algorithm is suitable for WMSNs with few memory and low transmission bandwidth.

Restoring underwater image from a single image is known to be an ill-posed problem. Some assumptions made in previous methods are not suitable in many situations. In this paper, an effective method is proposed to restore underwater images. Using the quad-tree subdivision and graph-based segmentation, the global background light can be robustly estimated. The medium transmission map is estimated based on minimum information loss principle and optical properties of underwater imaging. Qualitative experiments show that our results are characterized by relatively genuine color, natural appearance, and improved contrast and visibility. Quantitative comparisons demonstrate that the proposed method can achieve better quality of underwater images when compared with several other methods.

Images captured under water are usually degraded due to the effects of absorption and scattering. Degraded underwater images show some limitations when they are used for display and analysis. For example, underwater images with low contrast and color cast decrease the accuracy rate of underwater object detection and marine biology recognition. To overcome those limitations, a systematic underwater image enhancement method, which includes an underwater image dehazing algorithm and a contrast enhancement algorithm, is proposed. Built on a minimum information loss principle, an effective underwater image dehazing algorithm is proposed to restore the visibility, color, and natural appearance of underwater images. A simple yet effective contrast enhancement algorithm is proposed based on a kind of histogram distribution prior, which increases the contrast and brightness of underwater images. The proposed method can yield two versions of enhanced output. One version with relatively genuine color and natural appearance is suitable for display. The other version with high contrast and brightness can be used for extracting more valuable information and unveiling more details. Simulation experiment, qualitative and quantitative comparisons, as well as color accuracy and application tests are conducted to evaluate the performance of the proposed method. Extensive experiments demonstrate that the proposed method achieves better visual quality, more valuable information, and more accurate color restoration than several state-of-the-art methods, even for underwater images taken under several challenging scenes.

In this paper, the work inis analyzed. An error in its theoretical description part is pointed out and illustrated by a simple example. A modification suggestion is proposed to make the theoretical description of the workmore deliberate and thus being used appropriately.