Single Image Dehazing is a challenging ill-posed problem. While Transformers excel in long-range modeling, the low-pass characteristic of their self-attention further obscures high-frequency scene details attenuated by scattering. Furthermore, haze-degraded encoder features exhibit Mean-Elevation and Variance-Compression, leading to a significant statistical discrepancy with clear decoder features that hinders effective feature fusion. To address these limitations, this paper proposes the Frequency-Domain Stepwise and Moment Adjustment Network (FDSMA-Net). First, we design the High-Frequency Disentangled Attention (HFDA) mechanism, which explicitly decouples high-frequency components via attention inversion and utilizes a Dual Dynamic Frequency Modulation mechanism to calibrate and enhance details, effectively overcoming the spectral bias of standard attention. Second, the Moment-Driven Channel Fusion Attention (MDCFA) mechanism is introduced in the skip connections of the U-Net architecture. By explicitly aligning the first-order moment and second-order moment, MDCFA dynamically corrects the distribution shift caused by haze, ensuring statistically aligned feature fusion. Enabled by frequency disentanglement and moment alignment, FDSMA-Net excels on synthetic and real benchmarks, effectively recovering scene details and enhancing visual clarity. The source code of our proposed FDSMA-Net will be released at https://github.com/SWU-CS-MediaLab/FDSMA-Net.

Depth completion reconstructs dense depth maps from sparse LiDAR measurements and synchronized RGB images. Existing methods mainly face two challenges: the texture-geometry ambiguity and the topological rigidity propagation. To overcome these limitations, we propose a unified framework of YCbCr color space and the Dual-path Attention Network (YDANet), which treats the depth completion pipeline as a Disentanglement-Regression-Propagation paradigm. In the YDANet, we first introduce the YCbCr-Guided Adaptive Fusion (YGAF) module, which explicitly disentangles structural luminance from texture-rich chrominance and exploits intrinsic illumination-depth correlations to mitigate the texture-copying artifacts common in RGB-based methods. Then, to bridge the semantic gap between the backbone and refinement heads, we propose the Dual-Path Attention (DPA) module. Functioning as a lightweight Guidance-Refiner, DPA employs Spatial Reduction Attention (SRA) for global context and CNN-CBAM for local fidelity, utilizing a channel-wise fusion mechanism to distill clean affinity priors. Additionally, we incorporate the Gated Deformable Spatial Propagation Network (GD-SPN) to overcome the topological rigidity propagation. By employing a competitive gating mechanism to ensemble heterogeneous propagation kernels and sharing learned geometric parameters across iterations, GD-SPN achieves efficient non-rigid refinement. Extensive experiments demonstrate that YDANet achieves state-of-the-art performance among all 2D methods on the NYU Depth v2 dataset and ranks first on the KITTI depth completion online leaderboard among real-time approaches (< 80 ms) at the time of submission. The source code of our proposed YDANet will be released at https://github.com/SWU-CS-MediaLab/YDANet.

Continuous Learning (CL) enables deep neural networks to adapt to evolving data while preserving performance on previously learned tasks, without access to all historical data. A significant challenge in Continuous Learning is the risk of over-fitting to limited new-class data and catastrophic forgetting of prior knowledge. Although recent methods have demonstrated that pre-trained Vision Transformers (ViTs) with prompt tuning can mitigate these issues, these methods rely on fixed, learnable prompts, which limit knowledge transfer and lead to sub-optimal performance. To address these limitations, we propose a ViT-Enhanced Prompts (VEP) framework for the tasks of continuous learning based on the pre-trained ViTs. Our VEP extracts highly distinctive global and local feature representations from the Multi-Head Self-Attention (MSA) and Multi-Layer Perceptron (MLP) modules of ViT, integrating them with dynamic prompts to enhance knowledge learning and transferring. Additionally, we introduce a diversity loss mechanism to reduce redundant information accumulation in the learned prompts and enhance their discrimination, aiming to mitigate catastrophic forgetting effectively. Extensive experiments on benchmark datasets, including ImageNet-R, CUB200, and CIFAR100, validate the contribution of each component in VEP and demonstrate that VEP achieves state-of-the-art performance on the task of Continuous Learning while maintaining low parameter overhead. The source code of our designed VEP is at https://github.com/SWU-CSMediaLab/VEP.

Most existing Visible-Infrared Person Re-Identification (VI-ReID) methods primarily rely on fixed datasets for training, which struggle to accommodate continuously evolving cross-domain data, thus significantly limiting the adaptation in real-world dynamic scenarios. The task of Lifelong Visible-Infrared Person Re-Identification (LVI-ReID) emerged and is required to overcome the challenge of the semantic gap caused by both cross-modality and cross-domain data. Drawing inspiration from complementary learning systems, we propose a prompt-based dynamic learning framework to address the challenges inherent in LVI-ReID. Specifically, we design a Prompt Pool (PP) module to encapsulate shared knowledge across tasks or domains. In addition, we propose an instance-level prompt generator (IPG) to further enhance the model's ability to capture domain-specific knowledge, overcoming the limitations of a fixed-size prompt pool. For task-agnostic inference during the LVI-ReID phase, we develop a query-key mechanism that adaptively selects the most relevant prompt by evaluating the similarity between query tokens and keys, thereby addressing the nuanced requirements of varying tasks. Extensive experimental evaluations demonstrate the superiority of our proposed prompt learning-based PP-IPG framework over state-of-the-art methods in both lifelong learnings, lifelong person re-identification (LReID), and LVI-ReID settings. These results underscore the efficacy and practicality of our framework for advancing LVI-ReID across dynamic cross-modality and cross-domains. The source code of our designed PP-IPG method is at https://github.com/SWU-CSMediaLab/PP-IPG.

The existing arbitrary style transfer methods mainly suffer two challenges. One is content integrity, as most methods focus too much on style, resulting in incomplete content information and missing details. The other is style consistency, which requires more exploration of style information to alleviate pattern and texture inconsistencies. Thus, a framework of Content Integrity and Style Consistency preserving arbitrary style transfer (CISC-ST) is proposed, which consists of a Dual Style Attention (DSA) mechanism and Frequency Domain Enhancement (FDE) architecture. The DSA aims to explore more style information and generate finer-grained style details. The FDE aims to enhance the overall content information to improve content preservation. Qualitative and quantitative experiments show that our method can produce stylized results that are more detailed in content and visually harmonious. The source code of our CISC-ST is available at https://github.com/SWU-CS-MediaLab/CISC-ST.

Since the diverse geometric appearances and densities of rain streaks, local-global information is equally essential for single image deraining. Balancing local-global information becomes a challenge. Thus, a Multi-Scale Fusion of Gated Neighborhood Attention Transformers (MSF-GNAT) for single image deraining is proposed in this paper. Firstly, a Gated Neighborhood Attention Transformer (GNAT) block is designed to achieve complex condition deraining by complementarily fusing global and local information. Secondly, a Multi-Scale Fusion (MSF) block is developed to fuse multi-scale information for richer representation. Moreover, a Simplified Gated Feed-forward Network (SGFN) is proposed to ensure that the convolutional results primarily attend to valid pixels. Evaluation experiments on synthetic and real-world datasets demonstrate the superiority of our MSF-GNAT over state-of-the-art methods. The source code is available at https://github.com/SWU-CS-MediaLab/MSF-GNAT.

Visible-infrared person re-identification has been extensively explored, but it typically relies on stationary datasets for training. However, data is collected in a streaming manner in practical scenarios, necessitating the model's ability to continually learn without forgetting previous tasks. Existing methods focus simply on the lifelong single-modality person re-identification, but the visible images are sometimes unavailable, e.g., at night. To this end, this paper introduces a more challenging yet practical problem: Lifelong Visible-Infrared Person Re-identification (LVI-ReID). Inspired by the complementary learning systems, we propose a Tri-Token transformer with a Query-Key mechanism (TTQK) to tackle the LVI-ReID. Firstly, a general token is designed to capture robust domain-general features, shared across different domains, aiming to enhance the generalization capability. Subsequently, recognizing that different domains possess unique features like illumination and scenes, we allocate a specific token for each domain to extract significant domain-specific features, aiming to enhance the adaptability across domains. Furthermore, to prevent using the task identifier in the inference stage of LVI-ReID, we design a query-key mechanism to adaptively select the appropriate specific token based on the similarity between the query token and keys. Extensive experiments demonstrate that our method outperforms other lifelong learning and LReID methods. The source code of our designed LVI-ReID method is at https://github.com/SWU-CS-MediaLab/TTQK.

Few-Shot Class-Incremental Learning (FSCIL) aims to keep recognizing novel classes from a limited number of samples after training on abundant data from base classes while maintaining the performance of the old classes. The challenge, however, is that limited data from new classes not only leads to the issue of overfitting but also catastrophic forgetting. To address these two issues, we propose a causal inference strategy in the mainstream FSCIL framework, which encourages the model to learn significant knowledge in the base training session and enhance the model's ability to extract features to cope with the emergence of unseen classes in the incremental session, by improving the learning of causal relationships between features and predictions for perturbed samples. In addition, to improve the effectiveness of learning new tasks in the incremental sessions while preventing the model from overfitting to the novel class data, we freeze the feature extractor while adding a Fourier transform after the feature extractor in the incremental session. It can denoise the features, strengthen the features of the novel classes, and suppress the error in extracting the features of the limited number of samples directly from the feature extractor. Extensive experiments on CIFAR100, Caltech-USCD Birds-200-2011, and miniImageNet datasets show that our proposed framework achieves state-of-the-art performance on FSCIL. The source code of our designed framework is at https://github.com/SWU-CS-MediaLab/CIFSCIL.

The previous deep CNN-based single-image dehazing methods are devoted to improving the performance by increasing the network’s depth and width. In this paper, a novel Large Kernel Convolution Dehaze Network (LKD-Net) is proposed to enhance the network’s performance by increasing the size of the convolutional kernel. The main module in LKD-Net is the designed Large Kernel Convolution Dehaze Block (LKD Block), which consists of the Decomposition deep-wise Large Kernel Convolution Block (DLKCB) and the Channel Enhanced Feed-forward Network (CEFN). DLKCB is designed to reduce the massive amount of computational overhead and parameters of the large kernel by splitting the deep-wise large kernel convolution into a smaller depth-wise convolution and a depth-wise dilated convolution. Meanwhile, CEFN is designed to enhance the robustness of the Feed-forward Network by exploiting significant channels. The extensive experiments on RESIDE dataset demonstrate that our LKD-Net outperforms the state-of-the-art with far less computational overhead and parameters.

It is challenging to generate 3D human motions automatically from the text. In an ideal scenario, the generated motions should explore the text-grounded motion space while accurately depicting the content in the prescribed text descriptions. Text2length and text2motion training have been used to address this problem in previous research. There is, however, a lack of knowledge about the relationship between motion length and text. In this work, context-aware retrieval-based approaches are proposed for predicting motion lengths and generating proper 3D motions (C-MO). Specifically, we train a context-aware encoder-decoder model that leverages the previous output of the decoder or the embedding vector of a ground truth motion as the context so that the model becomes increasingly aware of previous alignments. Retrieving the most similar motions from the training set is based on the trained model given a text. Finally, we use the retrieval motion to guide the probability distribution for the final generated motions. Our method combines the advantages of both information retrieval and neural machine translation. C-MO is evaluated on a large-scale dataset, KIT, and its experimental results show that it achieves great improvements over the state-of-the-art.

Through an analysis of the Faster R-CNN [1] and SSD [2] algorithm models and their applications in image recognition, various advantages and limitations were discovered. Simultaneously, a novel image recognition technique, based on the fusion of Faster R-CNN and SSD, was proposed for application in medical image recognition. Furthermore, the principles and apparatus for medical virus diagnosis using this technique were designed. Validation experiments were conducted to achieve efficient recognition and diagnosis of white blood cells.

Deep neural networks-based scene text detection has obtained increasing attention in recent years. However, the existing scene text detection methods cannot effectively solve the problem of unclear text features. In this paper, a Multi-scale Residual Orthogonal-Channel Attention Network (MS-ROCANet) is proposed to improve the recall and accuracy of scene text detection. Specifically, a Detail-aware FPN is designed to capture more detailed information. Then, a Shared Composite Attention Head (SCAH) consists of a Residual Orthogonal Attention Module (ROAM), and a Residual Channel Attention Block (RCAB) is proposed. It can enhance textual region features at a multi-scale level. Finally, a global context extraction module is proposed to obtain global contextual information after referring to the core idea of Transformer. Extensive experiments demonstrated that our MS-ROCANet achieved competitive results on a variety of baselines for text detection. The codes and models are available at https://github.com/ASentry/MS-ROCANet

Most existing image enhancement methods require paired images, and rarely consider the aesthetic quality. This paper proposes a bi-directional normalization and color attention-guided generative adversarial network (BNCAGAN) for unsupervised image enhancement. An auxiliary attention classifier (AAC) and a bi-directional normalization residual (BNR) module are designed to assist the generator in flexibly controlling the local details with the constraint from both the low/high-quality domain. Moreover, a color attention module (CAM) is proposed to preserve the color fidelity in the discriminator. The qualitative and quantitative experimental results demonstrate that our BNCAGAN is superior to the existing methods with distinctively improved authenticity and naturalness of the enhanced images. The source code is available at https://github.com/SWU-CS-MediaLab/BNCAGAN.

The development of social media has provided an ideal platform for sharing information, but it has also become a hotbed for rumor posting and spreading.Existing rumor detection methods mainly look for clues from the textual content, static propagation structures.However, existing studies do not make full use of user information while ignoring the dynamic change of the communication structure.Therefore, this paper proposes a rumor detection model based on dynamic propagation structure called Recurrent Graph Convolutional Network (Re-GCN).In terms of rumor content representation, both the textual content of rumors and user information are considered.In terms of propagation structure, the dynamic propagation structure of rumors is considered.A dynamically changing propagation structure representation is constructed by dividing rumor propagation into multiple stages, and the dynamically changing propagation features are captured using a Bi-directional Recurrent Neural Network (Bi-RNN) and Graph Convolutional Networks (GCN).Finally, the attention mechanism is introduced to fuse the information of each stage to classify rumors.Experimental results on two real-world datasets show that the proposed method has significant improvement compared to the state-of-the-art benchmark methods.

Due to its effectivity and efficiency, deep hashing approaches are widely used for large-scale visual search. However, it is still challenging to produce compact and discriminative hash codes for images associated with multiple semantics for two main reasons, 1) similarity constraints designed in most of the existing methods are based upon an oversimplified similarity assignment(i.e., 0 for instance pairs sharing no label, 1 for instance pairs sharing at least 1 label), 2) the exploration in multi-semantic relevance are insufficient or even neglected in many of the existing methods. These problems significantly limit the discrimination of generated hash codes. In this paper, we propose a novel Deep Hashing with Self-Supervised Asymmetric Semantic Excavation and Margin-Scalable Constraint(SADH) approach to cope with these problems. SADH implements a self-supervised network to sufficiently preserve semantic information in a semantic feature dictionary and a semantic code dictionary for the semantics of the given dataset, which efficiently and precisely guides a feature learning network to preserve multi-label semantic information using an asymmetric learning strategy. By further exploiting semantic dictionaries, a new margin-scalable constraint is employed for both precise similarity searching and robust hash code generation. Extensive empirical research on four popular benchmarks validates the proposed method and shows it outperforms several state-of-the-art approaches. The source codes URL of our SADH is: http://github.com/SWU-CS-MediaLab/SADH .

With the fast progress of deep neural networks and the quick search efficiency of hashing, deep cross-modal hashing (CMH) methods have attracted more and more attention.Generally speaking, the existing CMH methods simultaneously learn hash functions and hash codes in an end-to-end architecture.However, they primarily focus on the hash codes generation stage neglected the losing of rich semantic information in the hash representations learning stage.Besides, the single-label supervision information is leveraged, while most instances are labeled by multiple categories.Thus, we propose a novel Deep Semantic Preserving and Attention-based Hashing (DSPAH) for cross-modal retrieval.In the DSPAH, we first use a crosslevel attention block to emphasize significant parts of hash representations and oversee unnecessary ones.Moreover, a Fine-Grained Similarity Criterion (FGSC) is proposed to explore the multiple semantic of image or text instances, helping to learn robust and optimal hash codes.Extensive experiment results on two large-scale public datasets have shown the competition of our proposed DSPAH. Index Terms-Deep cross-modal

Person re-identification (Re-id) aims at matching images of the same person across disjoint camera views. The most challenging aspect lies to design robust features and similarity models that reduce the impact of appearance variations. To address this problem, we develop a joint learning deep un-supervised model that aggregates the representation power of different level features for preserving the invariant knowledge of the same person in different domains. Specifically, we formulate a novel unified Re-id architecture called Unsupervised Joint Attention-Attribute Network (UJ-AAN). Our model is used for joint learning of attention selection and high-level semantic attributes to minimize the distance between different viewpoints of the same person by designing a bilinear feature aggregation module. Furthermore, we introduce attribute-related learning scheme to ensure the domain adaptability of UJ-AAN model in unlabeled domain. Extensive comparative evaluations demonstrate the advantages of the proposed UJ-AAN model over a wide range of state-of-the-art Re-id methods on two auto-detected Re-id benchmarks DukeMTMC-ReID and Market-1501.

Facial Expression Recognition (FER) has attracted considerable attention due to its potential applications in computer vision. Recently, convolutional neural network (CNN) has shown excellent performance on FER. However, most established deeper, wider and more complex network structures trained by small facial expression training dataset have a risk of overfitting. Moreover, most existing CNN models utilize the softmax loss as a supervision signal to penalize the deviation of classification, which enhances inter-class separation, yet intra-class compactness is not taken into consideration. In this paper, we propose a novel multi-scale CNN integrated with an attention-based learning layer (AMSCNN) for robust facial expression recognition. The attention-based learning layer is designed to automatically learn the importance of different receptive fields in the face during training. Moreover, the multi-scale CNN is optimized by the proposed Regularized Center Loss (RCL). Regularized center loss learns a center for deep features of each class and penalizes the distance between deep features and corresponding center, aiming to strengthen the discriminability of different facial expression. Extensive experiments conducted on two popular human FER benchmarks (CK+ and Oulu-CASIA dataset) demonstrated the effective of our proposed AMSCNN, and it obtained competitive results compared to the state-of-the-art.

In the well-known Bag-of-Words model, local features, such as the SIFT descriptor, are extracted and quantized into visual words. Then, an index is created to reduce computational burden. However, local clues serve as low-level representations that can not represent high-level semantic concepts. Recently, the success of deep features extracted from convolutional neural networks(CNN) has shown promising results toward bridging the semantic gap. Inspired by this, we attempt to introduce deep features into inverted index based image retrieval and thus propose the DeepIndex framework. Moreover, considering the compensation of different deep features, we incorporate multiple deep features from different fully connected layers, resulting in the multiple DeepIndex. We find the optimal integration of one midlevel deep feature and one high-level deep feature, from two different CNN architectures separately. This can be treated as an attempt to further reduce the semantic gap. Extensive experiments on three benchmark datasets demonstrate that, the proposed DeepIndex method is competitive with the state-of-the-art on Holidays(85:65% mAP), Paris(81:24% mAP), and UKB(3:76 score). In addition, our method is efficient in terms of both memory and time cost.

Image copy detection is a major challenge with regard to computational efficiency, memory requirements and accuracy. A popular approach from the literature is to use visual words (or Bag-of-Words) constructed from real value (SIFT and SURF) and binary string salient point descriptors (BRIEF, ORB, BRISK and FREAK). To accommodate large scale data sets, we used the approximate nearest neighbor (ANN) based cluster approach. Our results on several well-known test sets reveal that some of the recent binary string approaches outperformed notable descriptors such as SIFT and SURF.

In this paper, a method of browsing a large collection of images in a 3D environment is presented. Planetary bodies are used as a metaphor for browsing the image collection, where each planet represents a cluster of related images from the dataset. Salient point descriptors are used to cluster the imagery into planets. For each planet, maximum salient overlap is used to select a representative image. The user can move in 3D space through the planetary solar system. From a large distance, representative images painted on the planets are displayed. From a short distance, the images within the cluster are shown on the planet.

We present the Retina-inspired Invariant Fast Feature, RIFF, which was designed for invariance to scale, rotation, and affine image deformations. The feature descriptor is based on pair-wise comparisons over a sampling pattern loosely based on the human retina and introduces a method for improving accuracy by maximizing the discriminatory power of the point set. A performance evaluation with regard to bag of words based image retrieval on several well-known international datasets demonstrates that the RIFF descriptor has competitive performance to the state-of-the-art descriptors (e.g. SIFT, SURF, BRISK, and FREAK).

Processing visual content in images and videos is a challenging task associated with the development of modern computer vision. Because salient point approaches can represent distinctive and affine invariant points in images, many approaches have been proposed over the past decade. Each method has particular advantages and limitations and may be appropriate in different contexts. In this paper we evaluate the performance of a wide set of salient point detectors and descriptors. We begin by comparing diverse salient point algorithms (SIFT, SURF, BRIEF, ORB, FREAK, BRISK, STAR, GFTT and FAST) with regard to repeatability, recall and precision and then move to accuracy and stability in real-time video tracking.