Cross-view geo-localization (CVGL) matches query images (e.g., drone) to geographically corresponding opposite-view imagery (e.g., satellite). While supervised methods achieve strong performance, their reliance on extensive pairwise annotations limits scalability. Unsupervised alternatives avoid annotation costs but suffer from noisy pseudo-labels due to intrinsic cross-view domain gaps. To address these limitations, we propose UniABG, a novel dual-stage unsupervised cross-view geo-localization framework integrating adversarial view bridging with graph-based correspondence calibration. Our approach first employs View-Aware Adversarial Bridging (VAAB) to model view-invariant features and enhance pseudo-label robustness. Subsequently, Heterogeneous Graph Filtering Calibration (HGFC) refines cross-view associations by constructing dual inter-view structure graphs, achieving reliable view correspondence. Extensive experiments demonstrate state-of-the-art unsupervised performance, showing that UniABG improves Satellite → Drone AP by +10.63% on University-1652 and +16.73% on SUES-200, even surpassing supervised baselines.

Cross-view object-level image geo-localization (CVOIGL) aims to locate ground/drone-view query objects in satellite imagery. This task confronts two obstacles, namely view differences from imaging platform viewpoint changes and detection ambiguities from similar objects in large-scale satellite views. Existing methods typically employ uniform feature processing across objects while overlooking query-reference cross-view differences, leading to compromised localization precision when handling structurally analogous objects with scale variations. In this paper, we propose HiSymGeo, a Hierarchical Context Symbiosis framework with dual cooperative learning, achieving cross-view representation alignment and structural ambiguity resolution. Specifically, to mitigate cross-view differences, the Diversified View Enhancer (DiVE) first incorporates context-aware query enhancement for ground/drone-view representation while constructing scale-agnostic reference enhancement in satellite views to handle scale variations. These view-specific features then undergo contrastive learning via semantic-aware matching to align query and reference representations. Furthermore, the Query-Gated Multi-Expert View Fusion (QG-MEVF) introduces dynamic expert routing via multi-scale pyramidal representations, in which a Mixture-of-Experts (MoE) inspired architecture employs query-driven gating to adaptively select scale-specific fusion expert. This differentiable routing mechanism boosts structural discrimination against analogous objects, enabling precise object localization. Extensive ablation experiments demonstrate HiSymGeo's superiority, achieving state-of-the-art effectiveness while ensuring high cross-dataset generalization. We have released our code at https://github.com/chenqi142/HiSymGeo.

Sketch-based Person Retrieval (SBPR) aims to identify and retrieve a target individual across non-overlapping camera views using professional sketches as queries. In practice, sketches drawn by different artists often present diverse painting styles unpredictably. The substantial style variations among sketches pose significant challenges to the stability and generalizability of SBPR models. Prior works attempt to mitigate style variations through style manipulation methods, which inevitably undermine the inherent structural relations among multiple sketch features. This leads to overfitting on existing training styles and struggles with generalizing to new, unseen sketch styles. In this paper, we introduce FreeStyle, an innovative style-inclusive framework for SBPR, built upon the foundational CLIP architecture. FreeStyle explicitly models the relations across diverse sketch styles via style consistency enhancement, enabling dynamic adaptation to both seen and unseen style variations. Specifically, Diverse Style Semantic Unification is first devised to enhance the style consistency of each identity at the semantic level by introducing objective attribute-level semantic constraints. Meanwhile, Diverse Style Feature Squeezing tackles unclear feature boundaries among identities by concentrating the intra-identity space and separating the inter-identity space, thereby strengthening style consistency at the feature representation level. Additionally, considering the feature distribution discrepancy between sketches and photos, an identity-centric cross-modal prototype alignment mechanism is introduced to facilitate identity-aware cross-modal associations and promote a compact joint embedding space. Extensive experiments validate that FreeStyle not only achieves stable performance under seen style variations but also demonstrates strong generalization to unseen sketch styles.

Sketch-Based Person Re-Identification (Sketch ReID) aims to retrieve a person of interest across disjoint cameras using hand-drawn sketches as queries. A significant issue is the limited structural clues of sketch queries, which fail to convey high-level semantic retrieval intentions, such as colors and genders. Existing works typically combine sketches and texts for multi-modal retrieval, which inevitably introduces modality interference and relies heavily on expensive tri-modal datasets. In this paper, we propose, for the first time, an interactive and flexible sketch-based person retrieval framework that incorporates user feedback to refine the sketch person retrieval ranking without text training. A lightweight vision-to-text converting network is proposed to represent sketches with equivalent pseudo-word tokens, which aims to provide context assistance for interactive retrieval. Then, the sketch token can be seamlessly integrated with text feedback tokens within CLIP’s textual space for explicit sketch-text compositionality, thus achieving feedback-guided ranking refinement. Extensive experiments underscore the superiority of our InteractReID. Code will be available at https://github.com/littlexinyi/InteractReID.

Text-based Person Retrieval (TPR) plays a pivotal role in video surveillance systems for safeguarding public safety. As a fine-grained retrieval task, TPR faces the significant challenge of precisely capturing highly discriminative features across image and text modalities. Existing methods primarily focus on establishing modality-shared feature spaces to bridge cross-modal discrepancies. However, these methods are prone to disturbances from irrelevant information, such as background noises in the visual modality, and often over-emphasize specific local regions while neglecting the capture of diverse discriminative modal features, thereby limiting the robustness of cross-modal matching. In this paper, we introduce a novel framework, termed the Diverse Co-saliency Feature Learning Network (DCFL), which mines the co-saliency information between image and text modalities and enhances the diversity of cross-modal discriminative features while mitigating the interference of noise. Specifically, to construct cross-modal co-saliency features, we devise the Intra-modal Saliency Feature Learning (ISFL) and Cross-modal Saliency Feature Matching (CSFM) modules. ISFL employs a weighted mask mechanism to guide the model in reducing the impact of noise information in both modalities. Complementing ISFL, CSFM establishes consistent relationships between saliency features across modalities, leveraging text descriptions to align pedestrian-relevant visual regions. Furthermore, we propose the Diverse Co-saliency Feature Mining (DCFM) to bolster the diversity of discriminative co-saliency features across both image and text modalities. This module integrates a diversity regularization term, enabling the extraction of varied visual cues and capturing comprehensive features of the target individual. Extensive benchmark experiments demonstrate a substantial superiority of our approach over the state-of-the-art methods. The code will be released publicly.

Weakly-Supervised Change Detection (WSCD) aims to distinguish specific object changes (e.g., objects appearing or disappearing) from background variations (e.g., environmental changes due to light, weather, or seasonal shifts) in paired satellite images, relying only on paired image (i.e., image-level) classification labels. This technique significantly reduces the need for dense annotations required in fully-supervised change detection. However, as image-level supervision only indicates whether objects have changed in a scene, WSCD methods often misclassify background variations as object changes, especially in complex remote-sensing scenarios. In this work, we propose an Adversarial Class Prompting (AdvCP) method to address this co-occurring noise problem, including two phases: a) Adversarial Prompt Mining: After each training iteration, we introduce adversarial prompting perturbations, using incorrect one-hot image-level labels to activate erroneous feature mappings. This process reveals co-occurring adversarial samples under weak supervision, namely background variation features that are likely to be misclassified as object changes. b) Adversarial Sample Rectification: We integrate these adversarially prompt-activated pixel samples into training by constructing an online global prototype. This prototype is built from an exponentially weighted moving average of the current batch and all historical training data. Serving as an unbiased anchor, the global prototype guides the rectification of adversarial pixel samples. Our AdvCP can be seamlessly integrated into current WSCD methods without adding additional inference cost. Experiments on ConvNet, Transformer, and Segment Anything Model (SAM)-based baselines demonstrate significant performance enhancements, achieving up to 7.37%, 7.46%, and 6.56% IoU improvements on the WHU-CD, LEVIR-CD, and DSIFN-CD datasets. Furthermore, we demonstrate the generalizability of AdvCP to other multi-class weakly-supervised dense prediction scenarios. Code is available at https://github.com/zhenghuizhao/AdvCP.

Matching hand-drawn sketches with photos (a.k.a sketch-photo recognition or re-identification) faces the information asymmetry challenge due to the abstract nature of the sketch modality. Existing works tend to learn shared embedding spaces with CNN models by discarding the appearance cues for photo images or introducing GAN for sketch-photo synthesis. The former unavoidably loses discriminability, while the latter contains ineffaceable generation noise. In this paper, we start the first attempt to design an information-aligned sketch transformer (Sketch Trans+) viacross-modal disentangled prototype learning, while the transformer has shown great promise for discriminative visual modelling. Specifically, we design an asymmetric disentanglement scheme with a dynamic updatable auxiliary sketch (A-sketch) to align the modality representations without sacrificing information. The asymmetric disentanglement decomposes the photo representations into sketch-relevant and sketch-irrelevant cues, transferring sketch-irrelevant knowledge into the sketch modality to compensate for the missing information. Moreover, considering the feature discrepancy between the two modalities, we present a modality-aware prototype contrastive learning method that mines representative modality-sharing information using the modality-aware prototypes rather than the original feature representations. Extensive experiments on categoryand instance-level sketch-based datasets validate the superiority of our proposed method under various metrics. Code is available athttps://github.com/ccq195/SketchTrans

This paper introduces a simple yet powerful channel augmentation for visible-infrared re-identification. Most existing augmentation operations designed for single-modality visible images do not fully consider the imagery properties in visible to infrared matching. Our basic idea is to homogeneously generate color-irrelevant images by randomly exchanging the color channels. It can be seamlessly integrated into existing augmentation operations, consistently improving the robustness against color variations. For cross-modality metric learning, we design an enhanced channel-mixed learning strategy to simultaneously handle the intra- and cross-modality variations with squared difference for stronger discriminability. Besides, a weak-and-strong augmentation joint learning strategy is further developed to explicitly optimize the outputs of augmented images, which mutually integrates the channel augmented images (strong) and the general augmentation operations (weak) with consistency regularization. Furthermore, by conducting the label association between the channel augmented images and infrared modalities with modality-specific clustering, a simple yet effective unsupervised learning baseline is designed, which significantly outperforms existing unsupervised single-modality solutions. Extensive experiments with insightful analysis on two visible-infrared recognition tasks show that the proposed strategies consistently improve the accuracy. Without auxiliary information, the Rank-1/mAP achieves 71.48%/68.15% on the large-scale SYSU-MM01 dataset.

Unsupervised visible-infrared person re-identification (US-VI-ReID) aims at learning a cross-modality matching model under unsupervised conditions, which is an extremely important task for practical nighttime surveillance to retrieve a specific identity. Previous advanced US-VI-ReID works mainly focus on associating the positive cross-modality identities to optimize the feature extractor by off-line manners, inevitably resulting in error accumulation of incorrect off-line cross-modality associations in each training epoch due to the intra-modality and inter-modality discrepancies. They ignore the direct cross-modality feature interaction in the training process, i.e., the on-line representation learning and updating. Worse still, existing interaction methods are also susceptible to inter-modality differences, leading to unreliable heterogeneous neighborhood learning. To address the above issues, we propose a dual consistency-constrained learning framework (DCCL) simultaneously incorporating off-line cross-modality label refinement and on-line feature interaction learning. The basic idea is that the relations between cross-modality instance-instance and instance-identity should be consistent. More specifically, DCCL constructs an instance memory, an identity memory, and a domain memory for each modality. At the beginning of each training epoch, DCCL explores the off-line consistency of cross-modality instance-instance and instance-identity similarities to refine the reliable cross-modality identities. During the training, DCCL finds credible homogeneous and heterogeneous neighborhoods with on-line consistency between query-instance similarity and query-instance domain probability similarities for feature interaction in one batch, enhancing the robustness against intra-modality and inter-modality variations. Extensive experiments validate that our method significantly outperforms existing works, and even surpasses some supervised counterparts. The source code is available athttps://github.com/yangbincv/DCCL.

Person re-identification (ReID) with descriptive query (text or sketch) provides an important supplement for general image-image paradigms, which is usually studied in a single cross-modality matching manner, e.g., text-to-image or sketch-to-photo. However, without a camera-captured photo query, it is uncertain whether the text or sketch is available or not in practical scenarios. This motivates us to study a new and challenging modality-agnostic person re-ideruification problem. Towards this goal, we propose a unified person re-identification (UNIReID) architecture that can effectively adapt to cross-modality and multi-modality tasks. Specifically, UNIReID incorporates a simple dual-encoder with task-specific modality learning to mine and fuse visual and textual modality information. To deal with the imbalanced training problem of different tasks in UNIReID, we propose a task-aware dynamic training strategy in terms of task difficulty, adaptively adjusting the training focus. Besides, we construct three multi-modal ReID datasets by collecting the corresponding sketches from photos to support this challenging study. The experimental results on three multi-modal ReID datasets show that our UNIReID greatly improves the retrieval accuracy and generalization ability on different tasks and unseen scenarios.

Sketch-photo recognition is a cross-modal matching problem whose query sets are sketch images drawn by artists or amateurs. Due to the significant modality difference between the two modalities, it is challenging to extract discriminative modality-shared feature representations. Existing works focus on exploring modality-invariant features to discover shared embedding space. However, they discard modality-specific cues, resulting in information loss and diminished discriminatory power of features. This paper proposes a novel asymmetrical disentanglement and dynamic synthesis learning method in the transformer framework (SketchTrans) to handle modality discrepancy by combining modality-shared information with modality-specific information. Specifically, an asymmetrical disentanglement scheme is introduced to decompose the photo features into sketch-relevant and sketch-irrelevant cues while preserving the original sketch structure. Using the sketch-irrelevant cues, we further translate the sketch modality component to photo representation through knowledge transfer, obtaining cross-modality representations with information symmetry. Moreover, we propose a dynamic updatable auxiliary sketch (A-sketch) modality generated from the photo modality to guide the asymmetrical disentanglement in a single framework. Under a multi-modality joint learning framework, this auxiliary modality increases the diversity of training samples and narrows the cross-modality gap. We conduct extensive experiments on three fine-grained sketch-based retrieval datasets, i.e., PKU-Sketch, QMUL-ChairV2, and QMUL-ShoeV2, outperforming the state-of-the-arts under various metrics.

Video-based person re-identification (ReID) matches the same people across the video sequences with rich spatial and temporal information in complex scenes. It is highly challenging to capture discriminative information when occlusions and pose variations exist between frames. A key solution to this problem rests on extracting the temporal invariant features of video sequences. In this paper, we propose a novel method for discovering temporal coherence by designing a region-level saliency and granularity mining network (SGMN). Firstly, to address the varying noisy frame problem, we design a temporal spatial-relation module (TSRM) to locate frame-level salient regions, adaptively modeling the temporal relations on spatial dimension through a probe-buffer mechanism. It avoids the information redundancy between frames and captures the informative cues of each frame. Secondly, a temporal channel-relation module (TCRM) is proposed to further mine the small granularity information of each frame, which is complementary to TSRM by concentrating on discriminative small-scale regions. TCRM exploits a one-and-rest difference relation on channel dimension to enhance the granularity features, leading to stronger robustness against misalignments. Finally, we evaluate our SGMN with four representative video-based datasets, including iLIDS-VID, MARS, DukeMTMC-VideoReID, and LS-VID, and the results indicate the effectiveness of the proposed method.

Matching the daytime visible and nighttime infrared person images, namely visible infrared person re-identification (VI-ReID), is a challenging cross-modality retrieval problem. Due to the difficulty of data collection and annotation in nighttime surveillance, VI-ReID usually suffers from noise problems, making it challenging to directly learn part discriminative features. In order to improve the discriminability and enhance the robustness against noisy images, this paper proposes a novel dynamic tri-level relation mining (DTRM) framework by simultaneously exploring channel-level, part-level intra-modality, and graph-level cross-modality relation cues. To address the misalignment within the person images, we design an intra-modality weighted-part attention (IWPA) to construct part-aggregated representation. It adaptively integrates the body part relation into the local feature learning with a residual batch normalization (RBN) connection scheme. Besides, a cross-modality graph structured attention (CGSA) is incorporated to improve the global feature learning by utilizing the contextual relation between images from two modalities. This module reduces the negative effects of noisy images. To seamlessly integrate two components, a parameter-free dynamic aggregation strategy is designed in a progressive joint learning manner. To further improve the performance, we additionally design a simple yet effective channel-level learning strategy by exploiting the rich channel information of visible images, which significantly reinforces the performance without modifying the network structure or changing the training process. Extensive experiments on two visible infrared re-identification datasets have verified the effectiveness under various settings. Code is available at:https://github.com/mangye16/DDAG

Visible-infrared person re-identification (VI-ReID) is a cross-modality retrieval problem, which aims at matching the same pedestrian between the visible and infrared cameras. Due to the existence of pose variation, occlusion, and huge visual differences between the two modalities, previous studies mainly focus on learning image-level shared features. Since they usually learn a global representation or extract uniformly divided part features, these methods are sensitive to misalignments. In this paper, we propose a structure-aware positional transformer (SPOT) network to learn semantic-aware sharable modality features by utilizing the structural and positional information. It consists of two main components: attended structure representation (ASR) and transformer-based part interaction (TPI). Specifically, ASR models the modality-invariant structure feature for each modality and dynamically selects the discriminative appearance regions under the guidance of the structure information. TPI mines the part-level appearance and position relations with a transformer to learn discriminative part-level modality features. With a weighted combination of ASR and TPI, the proposed SPOT explores the rich contextual and structural information, effectively reducing cross-modality difference and enhancing the robustness against misalignments. Extensive experiments indicate that SPOT is superior to the state-of-the-art methods on two cross-modal datasets. Notably, the Rank-1/mAP value on the SYSU-MM01 dataset has improved by 8.43%/6.80%.

Existing approaches usually form the tracking task as an appearance matching procedure. However, the discrimination ability of appearance features is insufficient in these trackers, which is caused by their weak feature supervision constraints and inadequate exploitation of spatial contexts. To tackle this issue, this article proposes a novel appearance matching tracking (AMT) method to strengthen the feature restraints and capture discriminative spatial representations. Specifically, we first utilize a triplet structural loss function, which improves the learning capability of features by applying a structural similarity constraint with a triplet metric format on the features. It leverages feature statistics to capture the complex interactions of visual parts. Second, we put forward an adaptive matching module that exploits the dual spatial enhancement module to reinforce target feature discrimination. This not only boosts the representation ability of spatial context but also realizes spatially dynamic feature selection by attending to target deformation information. Moreover, this model introduces a simple but effective matching unit to intuitively evaluate the relative appearance differences between the target and the proposals. In addition, with the obtained discriminative features, AMT is capable of providing precise localization for the target. Therefore, the impact of spatial suppression imposed by window functions can be alleviated, allowing for effective tracking of high-speed moving objects. Extensive experiments prove that AMT outperforms state-of-the-art methods on six public datasets and demonstrate the effectiveness of each component in AMT.

The RGB-D cross-modal person re-identification (re-id) task aims to identify the person of interest across the RGB and depth image modes. The tremendous discrepancy between these two modalities makes this task difficult to tackle. Few researchers pay attention to this task, and the deep networks of existing methods still cannot be trained in an end-to-end manner. Therefore, this article proposes an end-to-end module for RGB-D cross-modal person re-id. This network introduces a cross-modal relational branch to narrow the gaps between two heterogeneous images. It models the abundant correlations between any cross-modal sample pairs, which are constrained by heterogeneous interactive learning. The proposed network also exploits a dual-modal local branch, which aims to capture the common spatial contexts in two modalities. This branch adopts shared attentive pooling and mutual contextual graph networks to extract the spatial attention within each local region and the spatial relations between distinct local parts, respectively. Experimental results on two public benchmark datasets, that is, the BIWI and RobotPKU datasets, demonstrate that our method is superior to the state-of-the-art. In addition, we perform thorough experiments to prove the effectiveness of each component in the proposed method.