Image collection summarization techniques aim to present a compact representation of an image gallery through a carefully selected subset of images that captures its semantic content. When it comes to web content, however, the ideal selection can vary based on the user's specific intentions and preferences. This is particularly relevant at Booking.com, where presenting properties and their visual summaries that align with users' expectations is crucial. To address this challenge, in this work, we consider user intentions in the summarization of property visuals by analyzing property reviews and extracting the most significant aspects mentioned by users. By incorporating the insights from reviews in our visual summaries, we enhance the summaries by presenting the relevant content to a user. Moreover, we achieve it without the need for costly annotations. Our experiments, including human perceptual studies, demonstrate the superiority of our cross-modal approach, which we coin as CrossSummarizer over the no-personalization and image-based clustering baselines.

Multi-label image classification is a foundational topic in various domains. Multimodal learning approaches have recently achieved outstanding results in image representation and single-label image classification. For instance, Contrastive Language-Image Pretraining (CLIP) demonstrates impressive image-text representation learning abilities and is robust to natural distribution shifts. This success inspires us to leverage multimodal learning for multi-label classification tasks, and benefit from contrastively learnt pretrained models. We propose the Multimodal Multi-label Image Classification (MuMIC) framework, which utilizes a hardness-aware tempered sigmoid based Binary Cross Entropy loss function, thus enables the optimization on multi-label objectives and transfer learning on CLIP. MuMIC is capable of providing high classification performance, handling real-world noisy data, supporting zero-shot predictions, and producing domain-specific image embeddings. In this study, a total of 120 image classes are defined, and more than 140K positive annotations are collected on approximately 60K Booking.com images. The final MuMIC model is deployed on Booking.com Content Intelligence Platform, and it outperforms other state-of-the-art models with 85.6% GAP@10 and 83.8% GAP on all 120 classes, as well as a 90.1% macro mAP score across 32 majority classes. We summarize the modelling choices which are extensively tested through ablation studies. To the best of our knowledge, we are the first to adapt contrastively learnt multimodal pretraining for real-world multi-label image classification problems, and the innovation can be transferred to other domains.

Many land cover change detection (LCCD) approaches applied on very high resolution (VHR) remote sensing images utilize spatial information by using a regular window or strict mathematical model. However, regular shape or strict models cannot fit the various shapes and sizes of the ground targets. In this article, a novel LCCD approach without the parameter is proposed to detect land cover change with VHR remote sensing images. First, an adaptive spatial-context extraction algorithm is applied to explore contextual information around a pixel. Second, the change magnitude between pairwise pixels is quantitatively measured by computing the band-to-band distance which is defined by the pairwise adaptive regions around the corresponding pixels. Finally, after the generation of a change magnitude image (CMI), a binary threshold method called double-window flexible pace search (DFPS) is adopted to divide CMI into a binary change detection map. The performance of the proposed approach is verified by comparing it with five state-of-the-art methods with three pairs of VHR images. The comparisons demonstrated that the proposed approach achieved the improved detected results comparing with state-of-the-art LCCD methods. The code of the proposed approach is available athttps://github.com/TongfeiLiu/ASEA-CD.

Land cover change detection (LCCD) using remote sensing images (RSIs) plays an important role in natural disaster evaluation, forest deformation monitoring, and wildfire destruction detection. However, bitemporal images are usually acquired at different atmospheric conditions, such as sun height and soil moisture, which usually cause pseudo and noise change into the change detection map. Changed areas on the ground also generally have various shapes and sizes, consequently making the utilization of spatial contextual information a challenging task. In this paper, we design a novel neural network with spatial-spectral attention mechanism and multi-scale dilation convolution modules. This work is based on the previously demonstrated promising performance of convolutional neural network for LCCD with RSIs and attempts to capture more positive changes and further enhance the detection accuracies. The learning of the proposed neural network is guided with a change magnitude image. The performance and feasibility of the proposed network are validated with four pairs of RSIs that depict real land cover change events on the Earth’s surface. Comparison of the performance of the proposed approach with that of five state-of-art methods indicates the superiority of the proposed network in terms of 10 quantitative evaluation metrics and visual performance. Such as, the proposed network achieved an improvement about 0.08%~14.87% in terms of OA for Dataset-A.

Landslide inventory mapping (LIM) is an important application in remote sensing for assisting in the relief of landslide geohazards. However, while conducting LIM tasks performing change detection analysis using bi-temporal very high-resolution (VHR) remote sensing images, due to landslide usually occurred in a mountain area, the phenological difference and outcrop rock may bring pseudo-changes to LIM results. In this paper, a novel change detection approach based on Adaptive Region Shape Similarity (ARSS) is proposed for LIM with VHR remote sensing images to improve detection performance. First, an adaptive region around each pixel is extended to explore the contextual information. Then, direction lines within an adaptive region are defined to describe the shape of the adaptive region. Finally, the pixels located on each direction line are taken into account to build the corresponding histogram. The shape similarity between the pairwise histogram curves is measured by using the Discrete Frchet Distance (DFD). Once the bi-temporal images are processed by using the abovementioned steps, a change magnitude image (CMI) is generated, while a threshold is then used to obtain a final binary change map. The proposed approach is applied to three pairs of landslide sites images acquired with aerial plane and one land use change dataset acquired by Quick Bird Satellite. Compared with ten state-of-the-art methods, the proposed approach achieved LIMs and detection results with higher accuracies and better performance.