With the rapid development of intelligent surveillance technology, the massive amount of multimodal data (e.g., videos, images, and text) has imposed higher demands on efficient information retrieval and security. Traditional single-modal retrieval methods struggle to meet practical requirements, making multimodal image-text retrieval a research hotspot in this field. Existing approaches, however, still face challenges in fine-grained semantic alignment and suffer from rigid matching mechanisms. To address these issues, this paper introduces SeaNcr, a novel framework that integrates cross-modal semantic entity alignment with non-correspondence reasoning. Our method constructs class-level entity representations enhanced by saliency-guided masking to capture discriminative semantic features. A pseudo-frozen asynchronous optimization strategy is introduced to maintain semantic consistency across modalities by associating stable entity representations with dynamically updated encoder features. Moreover, to overcome rigid matching, we design a non-correspondence reasoning module that jointly leverages intra-modal similarity and cross-modal mutual nearest neighbor constraints, optimizing matching flexibility and generalization. Extensive experiments validate that SeaNcr significantly enhances cross-modal feature representation and retrieval robustness, achieving state-of-the-art performance on multiple person re-identification benchmarks.

Medical image segmentation is critical for clinical diagnosis, but the scarcity of annotated data limits robust model training, making few-shot learning indispensable. Existing methods often suffer from two issues—performance degradation due to significant inter-class variations in pathological structures, and overreliance on attention mechanisms with high computational complexity ( O ( n 2 )), which hinders the efficient modeling of long-range dependencies. In contrast, the state space model (SSM) offers linear complexity ( O ( n )) and superior efficiency, making it a key solution. To address these challenges, we propose PPFFR (parallel prototype filter and feature refinement) for few-shot medical image segmentation. The proposed framework comprises three key modules. First, we propose the prototype refinement (PR) module to construct refined class subgraphs from encoder-extracted features of both support and query images, which generates support prototypes with minimized inter-class variation. We then propose the parallel prototype filter (PPF) module to suppress background interference and enhance the correlation between support and query prototypes. Finally, we implement the feature refinement (FR) module to further enhance segmentation accuracy and accelerate model convergence with SSM’s robust long-range dependency modeling capability, integrated with multi-head attention (MHA) to preserve spatial details. Experimental results on the Abd-MRI dataset demonstrate that FR with MHA outperforms FR alone in segmenting the left kidney, right kidney, liver, and spleen, and in terms of mean accuracy, confirming MHA’ s role in improving precision. In extensive experiments conducted on three public datasets under the 1-way 1-shot setting, PPFFR achieves Dice scores of 87.62%, 86.74%, and 79.71% separately, consistently surpassing state-of-the-art few-shot medical image segmentation methods. As the critical component, SSM ensures that PPFFR balances performance with efficiency. Ablation studies validate the effectiveness of the PR, PPF, and FR modules. The results indicate that explicit inter-class variation reduction and SSM-based feature refinement can enhance accuracy without heavy computational overhead. In conclusion, PPFFR effectively enhances inter-class consistency and computational efficiency for few-shot medical image segmentation. This work provides insights for few-shot learning in medical imaging and inspires lightweight architecture designs for clinical deployment.

Person re-identification (re-ID) is a research hotspot in the field of intelligent monitoring and security. Domain generalizable (DG) person re-identification transfers the trained model directly to the unseen target domain for testing, which is closer to the practical application than supervised or unsupervised person re-ID. Meta-learning strategy is an effective way to solve the DG problem, nevertheless, existing meta-learning-based DG re-ID methods mainly simulates the test process in a single aspect such as identity or style, while ignoring the completely different person identities and styles in the unseen target domain. As to this problem, we consider a double disentangling from two levels of training strategy and feature learning, and propose a novel dualistic disentangled meta-learning (D$^{\mathbf {2}}$ML) model. D$^{\mathbf {2}}$ML is composed of two disentangling stages, one is for learning strategy, which spreads one-stage meta-test into two-stage, including an identity meta-test stage and a style meta-test stage. The other is for feature representation, which decouples the shallow layer features into identity-related features and style-related features. Specifically, we first conduct identity meta-test stage on different person identities of the images, and then employ a feature-level style perturbation module (SPM) based on Fourier spectrum transformation to conduct the style meta-test stage on the image with diversified styles. With these two stages, abundant changes in the unseen domain can be simulated during the meta-test phase. Besides, to learn more identity-related features, a feature disentangling module (FDM) is inserted at each stage of meta-learning and a disentangled triplet loss is developed. Through constraining the relationship between identity-related features and style-related features, the generalization ability of the model can be further improved. Experimental results on four public datasets show that our D$^{\mathbf {2}}$ML model achieves superior generalization performance compared to the state-of-the-art methods.

Infrared and visible image fusion aims to generate one image with comprehensive information. It can maintain rich texture characteristics and thermal information. However, for existing image fusion methods, the fused images either sacrifice the salience of thermal targets and the richness of textures or introduce the interference of useless information like artifacts. To alleviate these problems, an effective cross-modal coordinate attention network for infrared and visible image fusion called CCAFusion is proposed in this paper. To fully integrate complementary features, the cross-modal image fusion strategy based on coordinate attention is designed, which consists of the feature-awareness fusion module and the feature-enhancement fusion module. Moreover, a multiscale skip connection-based network is employed to obtain multiscale features in the infrared image and the visible image, which can fully utilize the multi-level information in the fusion process. To reduce the discrepancy between the fused image and the input images, a multiple constrained loss function including the base loss and the auxiliary loss is developed to adjust the gray-level distribution and ensure the harmonious coexistence of structure and intensity in fused images, thereby preventing the pollution of useless information like artifacts. Extensive experiments conducted on widely used datasets demonstrate that our CCAFusion achieves superior performance over state-of-the-art image fusion methods in both qualitative evaluation and quantitative measurement. Furthermore, the application to salient object detection reveals the potential of our CCAFusion for high-level vision tasks, which can effectively boost the detection performance.

Abstract As a basic component in the field of computer vision, the pedestrian detection plays an essential role in several real‐world applications such as video surveillance. The promising performance has been achieved in pedestrian detection relying on deep learning, but large‐scale variance and small‐scale pedestrian detection remain inherently hard as before. In order to deal with the aforementioned problems, this paper proposes a multi‐scale pedestrian detection method with global–local attention and multi‐scale receptive field context (MRFC). To make the network focus on small‐scale pedestrians, we add a high‐resolution detection branch on the original detector. To better integrate the incongruous semantic feature, the global–local attention module is embedded to highlight the feature representation of pedestrians so as to implement the feature fusion effectively. In order to adapt the receptive field of the network to achieve scale‐variance detection, the MRFC is applied. Based on integrating the above structures, the proposed method achieves competitive results on Caltech and CityPersons datasets. The source code is released in https://github.com/xiaopan999/yolov5‐pedestrian_detection .

Abstract Automated breast ultrasound (ABUS) imaging system is a practical technique to automatically scan the whole breast. Automatic tumor detection plays a significant role in the clinic. However, training deep convolutional neural networks (CNNs) for tumor detection needs a large quantity of labeled data. It is time‐consuming and expensive to manually annotate tumor positions in ABUS images. In this paper, a novel semi‐supervised learning EfficientDet (SSL‐E) model is proposed for ABUS tumor detection. Our SSL‐E model solves the tumor detection problem from high similarity and serious unbalance between tumors and backgrounds. Considering the image contrast variation and tumor scale variations in ABUS images, color transformation and geometric transformation are employed for data augmentations. Then the consistency between image and its augmented version is developed, thus the robustness of the detector can be improved. Aiming at the problem of serious unbalance between tumors and backgrounds, a novel copy‐paste synthesis strategy is designed, which can generate more tumor samples and enhance tumor diversity. This method is tested on 68 tumor volumes and 68 normal volumes, including 43,248 slices (1683 tumor slices and 41,565 normal slices). It obtains a promising result with sensitivity of 90.2% and false positives per image (FPs/I) at 0.15.

Multimodal magnetic resonance imaging (MRI) contains complementary information in anatomical and functional images that help the accurate diagnosis and treatment evaluation of lung cancers. However, effectively exploiting the complementary information in chest MRI images remains challenging due to the lack of rigorous registration. In this paper, a novel method is proposed that can effectively exploit the complementary information in weakly paired images for accurate tumor segmentation, namely coco-attention mechanism. Coco-attention module consists of two parts: the multi-modal co-attention (MultiCo-attn) and the multi-level coordinate attention (MultiCord-attn). The former aims to obtain tumor-aware deep features for accurate tumor localization, and the latter aims to highlight tumor area for more precise segmentation. Specifically, the MultiCo-attn extracts complementary information from multimodal high-dimensional semantic features using a bidirectional algorithm to generate attention maps focused on tumor region, and then uses the attention maps to enhance the feature representations. The MultiCord-attn leverages multi-level feature information to highlight tumor regions by adjusting the weight of each point in the feature. We evaluate the proposed method on lung tumor segmentation with a clinical dataset of 90 chest MRI scans of non-small cell lung cancer (NSCLC). The results show that the proposed method is effective for tumor segmentation in weakly paired images and achieves significant improvement (p < 0.005) over several commonly used multimodal segmentation methods. Furthermore, the ablation experiment results confirm the effectiveness and interpretability of the proposed coco-attention module.

Person re-identification (re-ID) aims to match images of the same person in different camera views. The general convolutional neural network has the problem of insufficient ability to discriminate specific targets, resulting in limited feature representation when directly applied in person re-ID task. In this paper, a person re-ID baseline model based on attention block neural architecture search is proposed. Since the attention mechanism is helpful to improve the feature expression ability of the network, attention blocks are automatically searched and added to ResNet-50 model, which aims to find the most suitable model structure for person re-ID task. Besides, in order to integrate information between the samples of same identity, an intra-class self-distillation loss is introduced according to the idea of knowledge integration. Experiments on two popular datasets confirm the effectiveness of our baseline. The code has been released in https://github.com/Nicholasxin/Attention-NASReID.

Tumor segmentation in 3D automated breast ultrasound (ABUS) plays an important role in breast disease diagnosis and surgical planning. However, automatic segmentation of tumors in 3D ABUS images is still challenging, due to the large tumor shape and size variations, and uncertain tumor locations among patients. In this paper, we develop a novel cross-model attention-guided tumor segmentation network with a hybrid loss for 3D ABUS images. Specifically, we incorporate the tumor location into a segmentation network by combining an improved 3D Mask R-CNN head into V-Net as an end-to-end architecture. Furthermore, we introduce a cross-model attention mechanism that is able to aggregate the segmentation probability map from the improved 3D Mask R-CNN to each feature extraction level in the V-Net. Then, we design a hybrid loss to balance the contribution of each part in the proposed cross-model segmentation network. We conduct extensive experiments on 170 3D ABUS from 107 patients. Experimental results show that our method outperforms other state-of-the-art methods, by achieving the Dice similarity coefficient (DSC) of 64.57%, Jaccard coefficient (JC) of 53.39%, recall (REC) of 64.43%, precision (PRE) of 74.51%, 95th Hausdorff distance (95HD) of 11.91 mm, and average surface distance (ASD) of 4.63 mm. Our code will be available online (https://github.com/zhouyuegithub/CMVNet).

Unsupervised cross domain (UCD) person re-identification (re-ID) aims to apply a model trained on a labeled source domain to an unlabeled target domain. It faces huge challenges as the identities have no overlap between these two domains. At present, most UCD person re-ID methods perform "supervised learning" by assigning pseudo labels to the target domain, which leads to poor re-ID performance due to the pseudo label noise. To address this problem, a multi-loss optimization learning (MLOL) model is proposed for UCD person re-ID. In addition to using the information of clustering pseudo labels from the perspective of supervised learning, two losses are designed from the view of similarity exploration and adversarial learning to optimize the model. Specifically, in order to alleviate the erroneous guidance brought by the clustering error to the model, a ranking-average-based triplet loss learning and a neighbor-consistency-based loss learning are developed. Combining these losses to optimize the model results in a deep exploration of the intra-domain relation within the target domain. The proposed model is evaluated on three popular person re-ID datasets, Market-1501, DukeMTMC-reID, and MSMT17. Experimental results show that our model outperforms the state-of-the-art UCD re-ID methods with a clear advantage.

Image registration of lung dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) is challenging because the rapid changes in intensity lead to non-realistic deformations of intensity-based registration methods. To address this problem, we propose a novel landmark-based registration framework by incorporating landmark information into a group-wise registration. Robust principal component analysis is used to separate motion from intensity changes caused by a contrast agent. Landmark pairs are detected on the resulting motion components and then incorporated into an intensity-based registration through a constraint term. To reduce the negative effect of inaccurate landmark pairs on registration, an adaptive weighting landmark constraint is proposed. The method for calculating landmark weights is based on an assumption that the displacement of a good matched landmark is consistent with those of its neighbors. The proposed method was tested on 20 clinical lung DCE-MRI image series. Both visual inspection and quantitative assessment are used for the evaluation. Experimental results show that the proposed method effectively reduces the non-realistic deformations in registration and improves the registration performance compared with several state-of-the-art registration methods.

Accurate breast mass segmentation of automated breast ultrasound (ABUS) images plays a crucial role in 3D breast reconstruction which can assist radiologists in surgery planning. Although the convolutional neural network has great potential for breast mass segmentation due to the remarkable progress of deep learning, the lack of annotated data limits the performance of deep CNNs. In this article, we present an uncertainty aware temporal ensembling (UATE) model for semi-supervised ABUS mass segmentation. Specifically, a temporal ensembling segmentation (TEs) model is designed to segment breast mass using a few labeled images and a large number of unlabeled images. Considering the network output contains correct predictions and unreliable predictions, equally treating each prediction in pseudo label update and loss calculation may degrade the network performance. To alleviate this problem, the uncertainty map is estimated for each image. Then an adaptive ensembling momentum map and an uncertainty aware unsupervised loss are designed and integrated with TEs model. The effectiveness of the proposed UATE model is mainly verified on an ABUS dataset of 107 patients with 170 volumes, including 13382 2D labeled slices. The Jaccard index (JI), Dice similarity coefficient (DSC), pixel-wise accuracy (AC) and Hausdorff distance (HD) of the proposed method on testing set are 63.65%, 74.25%, 99.21% and 3.81mm respectively. Experimental results demonstrate that our semi-supervised method outperforms the fully supervised method, and get a promising result compared with existing semi-supervised methods.

In this paper, a bilateral mass detection method is proposed for mammogram combining self-supervised learning network and Siamese-Faster-RCNN. The breast region is first identified by threshold segmentation and morphological filter. Then self-supervised learning network is built to learn the spatial transformation between the bilateral breast regions. Following bilateral mammograms are registered, a Siamese-Faster-RCNN consisting of the Region Proposal Network (RPN) and a Siamese fully connected (Siamese-FC) network is designed and employed for mass detection. The proposed method is estimated on two datasets (publicly available dataset INbreast and private dataset BCPKUPH). Experimental results show that the proposed method performs better than the previous state of art methods, which demonstrates the promise of the proposed method.

Mammography is one of the broadly used imaging modality for breast cancer screening and detection. Locating mass from the whole breast is an important work in computer-aided detection. Traditionally, handcrafted features are employed to capture the difference between a mass region and a normal region. Recently convolution neural network (CNN) which automatically discovers features from the images shows promising results in many pattern recognition tasks. In this paper, three mass detection schemes based on CNN are evaluated. First, a suspicious region locating method based on heuristic knowledge is employed. Then three different CNN schemes are designed to classify the suspicious region as mass or normal. The proposed schemes are evaluated on a dataset of 352 mammograms. Compared with several handcrafted features, CNN-based methods shows better mass detection performance in terms of free receiver operating characteristic (FROC) curve.