Multi-modal image fusion, particularly between infrared (IR) and visible (VIS) images, integrates complementary information from diverse imaging sources to enhance perception in applications like autonomous driving and surveillance. While IR images capture thermal radiation and VIS images provide rich texture details, existing fusion methods face challenges in preserving infrared thermal signatures under high illumination, where overexposed VIS regions dominate fusion outputs. To address the above problems, we propose XOR-Fuse, a novel logical operation-driven fusion framework that explicitly captures complementary pixel-level discrepancies between IR and VIS modalities. The XOR-Fuse defines the pixel-wise analogy for the logical XOR operation, which rectifies the suppression of IR features caused by conventional maximum-intensity fusion rules under high illumination. To further reinforce IR feature preservation, we integrate multi-scale Gabor wavelet filtering and wavelet decomposition for illumination-invariant texture extraction and VGG-based semantic constraints, ensuring structural congruence between IR and VIS details. Experiments on MSRS, RoadScene, and TNO datasets demonstrate significant improvements over four baseline models (DeepFuse, SDNet, U2Fusion, and DATFuse). For instance, the enhanced DeepFuse achieves SD=48.27 and VIF=0.62 on MSRS, outperforming the original model (SD=33.79, VIF=0.42). Qualitative results under variable illumination confirm the recovery of suppressed IR details while retaining VIS textures.

Skin cancer is one of the most prevalent cancer types. Training a convolutional neural network (CNN) for skin cancer diagnosis usually requires a large amount of labeled data to yield good performance. However, obtaining high-quality labels is laborious and expensive, as accurately annotating medical images demands expertise knowledge of the clinicians. In this paper, we present a novel semi-supervised framework for skin cancer diagnosis, SSCD-Net combines curriculum learning strategy, disease relation and the guidance of clinical information. Curriculum learning (CL) strategy is designed to help train from easy instances and then gradually handles harder ones to improve the generalization capacity and convergence rate. The whole training set is split into a number of subsets ranked from an easy one to a more complex one, in an unsupervised manner. The SSCD-Net is driven by disease relation apart from conventional individual consistency, which explicitly enforces the consistency of disease relation among different samples under perturbations, encouraging the model to explore extra semantic information from unlabeled data. At the end of the classification pipeline, a multimodal information fusion (MIF) module is designed to fuse the image features and clinical features, further introducing the guidance of clinical information. Comprehensive experiments are conducted on four publicly accessible dataset, i.e., PADUEFC-20, ISIC 2018, ISIC 2019 and ISIC-Archive. Simulations show that the SSCD-Net has better or comparable performance against that of the other state-of-the-art semi-supervised models.

For medical image segmentation, contrastive learning recently becomes the dominant method to get the class-separated visual representations by pulling augmented views of the same samples closer in a representation space, and pushing apart augmented views of different samples. However, most current contrastive learning methods may suffer from the issue of class collision because those images (pixels) that from the same class are forcefully contrasted due to the pretext task simply takes another augmentation of the same image as positive pairs and all other images are view as negative samples. Moreover, the problem looks more serious in pixel-level contrastive learning in that massive pixels from the same class are viewed as negative pairs to differ is intolerable. Therefore, to alleviate this, we propose a novel semi-supervised learning framework based on hierarchical data augmentation in contrastive learning for medical image segmentation. Specifically, We first propose a novel contrastive loss function based on class-aware patch by using the pseudo labels and source image effectively. With the novel loss, we learn better intra-class compactness and inter-class separability in the feature space. We then introduce a patch-based hierarchical data augmentation module to learn the hierarchical invariance by applying different strength of augment to different probability of patches, thus making the patches with a high probability of belong to a certain class more alike and avoiding the issue of class collision in contrastive learning. Experiments on several public accessible datasets from multiple domains reveals the superiority of our proposed method as compared to the state-of-the-art semi-supervised and contrastive learning methods.

Traditional shadow detectors often identify all shadow regions of static images or video sequences. This work presents the Referring Video Shadow Detection (RVSD), which is an innovative task that rejuvenates the classic paradigm by facilitating the segmentation of particular shadows in videos based on descriptive natural language prompts. This novel RVSD not only achieves segmentation of arbitrary shadow areas of interest based on descriptions (flexibility) but also allows users to interact with visual content more directly and naturally by using natural language prompts (interactivity), paving the way for abundant applications ranging from advanced video editing to virtual reality experiences. To pioneer the RVSD research, we curated a well-annotated RVSD dataset, which encompasses 86 videos and a rich set of 15,011 paired textual descriptions with corresponding shadows. To the best of our knowledge, this dataset is the first one for addressing RVSD. Based on this dataset, we propose a Referring Shadow-Track Memory Network (RSM-Net) for addressing the RVSD task. In our RSM-Net, we devise a Twin-Track Synergistic Memory (TSM) to store intra-clip memory features and hierarchical inter-clip memory features, and then pass these memory features into a memory read module to refine features of the current video frame for referring shadow detection. We also develop a Mixed-Prior Shadow Attention (MSA) to utilize physical priors to obtain a coarse shadow map for learning more visual features by weighting it with the input video frame. Experimental results show that our RSM-Net achieves state-of-the-art performance for RVSD with a notable Overall IOU increase of 4.4%. Our code and dataset are available at https://github.com/whq-xxh/RVSD.

With the rapid advancement of autonomous driving technology, there is an increasingly urgent demand for faster and more accurate object detection frameworks. Recently, numerous deep learning-based object detectors have shown impressive performance in real-time driving applications. However, the detection of small objects such as various traffic signs remains challenging due to the complex nature of these objects. This paper proposes a transformer-based detector TS-DETR to improve the accuracy of small object detection in autonomous driving systems. We introduce a constrained decoder structure to focus the model's attention on the predicted boxes. Additionally, a content-aware deformable cross-attention mechanism is proposed to obtain more comprehensive attention weights. Experimental results on challenging public datasets such as TT100K and CCTSDB2021 demonstrate that our method demonstrates substantial performance enhancements while introducing only a slight increment in parameter count compared to current algorithms.

Current mainstream multi-object tracking (MOT) algorithms aim to maintain the identities of targets by data-association. However, the accuracy of multi-object tracking would be affected by the unreliability results of detectors. Simultaneously, the separation of motion module and appearance module cannot fully exploit affinity features that will increase computational complexity. To address these issues, we propose a multi-task tracking framework including the United Siamese Network (USN) and Candidate-Refreshing (CR) model. The USN integrates motion affinity and appearance affinity into an end-to-end network. Such design can pay attention to joint learning of similarities between motion patterns and appearance features and enhance the ability to distinguish similar targets in complex environments. The CR model aims to combine detection candidates with tracking candidates, and compensate for unreliable detection results by scoring and regressing the bounding boxes. In addition, we equip our framework with a tracklet confidence function, which is based on the spatialtemporal information. This function can determine the vitality of unmatched tracklets and further alleviate the influence of detectors. Experiments on the MOT16 and MOT17 benchmark datasets show that our framework has achieved outstanding performance on online MOT algorithms.