In this study, a novel self-supervised contrastive learning framework that integrates geographic knowledge into the analysis of street view imagery for representing urban space is proposed. Traditional methods that rely solely on deep learning often struggle to capture the complex spatial characteristics of urban environments. To address this issue, in the proposed framework, we first extracted visual knowledge (VK) from street view imagery using semantic segmentation and then constructed contrastive samples through VK–imagery pairs. Finally, we introduced distance-weighted temperatures into the contrastive loss function to adjust the similarities of urban features encoded in the representations on the basis of geographical proximity, thereby integrating semantic knowledge among geographical locations (GSK). The method was validated through two case studies: urban village classification in Guangzhou and Foshan and urban-mobility pattern prediction in Shenzhen. The results showed significant improvements in classification accuracy (OA: 0.967) and reduced prediction error (MAE: 28.069) compared with conventional approaches. This research demonstrates the effectiveness of integrating geographic knowledge (including VK and GSK) into contrastive learning. The approach enhances model interpretability and generalizability for urban studies and provides a tool for analyzing urban development patterns and mobility needs that will be useful for urban planning and policymaking.

Integrating language and images for Composed Image Retrieval (CIR) allows for a flexible representation of search intent, making it a focal point in multi-modal research. Traditional CIR methods train models on complex triplet datasets. In contrast, Zero-Shot Composed Image Retrieval (ZS-CIR) eliminates the need for constructing datasets and training models for each specific task, attracting significant attention from researchers. CIReVL is a training-free method that translates visual content into textual representations, employing large language models (LLMs) to perform robust text inference and visual language models (VLMs) to ensure multi-modal alignment during retrieval. Although this approach achieves high performance without training and offers strong interpretability, relying solely on single descriptive texts for retrieval has limitations in handling complex search demands. To address this issue, we propose a DualCIR framework, which utilizes an LLM to generate refined dual-directional descriptions-both positive and negative-alongside generating holistic target descriptions. We score retrieval data separately via these descriptions, merge the scores, perform ranking, and ultimately achieve retrieval. Extensive experiments across three datasets in the natural image and fashion domains show that our approach maintains low inference costs while noticeably enhancing retrieval effectiveness. Compared to existing training-free methods, our DualCIR achieves state-of-the-art performance and is even competitive with the training-based methods.

Urbanization has been a driving force for economic growth, yet it has also caused the emergence of informal urban settlements such as urban villages (UVs), which are characterized by issues such as arbitrary land use, high-density construction, and insufficient infrastructure. In previous studies on UV detection, the semantic imbalance and feature interaction among cross-modal data have not been comprehensively considered, impacting the accuracy and interpretability of the results. In this work, a cross-modal fusion framework is proposed that integrates high-resolution remote sensing and street view images for UV detection. First, convolutional neural networks (ResNet-50) are used for feature extraction from both remote sensing and street view images. Then, an inner product channel attention module is used to dynamically adjust weights while considering multiangle views of street view images. A cross-modal feature fusion module that incorporates dilation convolution and a global-based feature fusion block is used to enhance feature interaction and fusion. The method has an overall accuracy (OA) of 0.975 for UV classification in a case study of the Guangzhou–Foshan metropolitan area in China, outperforming a set of baseline methods. The integration of remote sensing and street view images improves the OA value by approximately 2%. This work enhances the understanding of the distribution of UVs via both top-down and ground-level view data in an automatic and efficient way, providing urban planners with valuable insights to accurately identify UVs and support targeted, sustainable urban renewal aligned with the SDGs for inclusive, resilient cities.