LLM-powered search agents are increasingly being used for multi-step information seeking tasks, yet the IR community lacks empirical understanding of how agentic search sessions unfold and how retrieved evidence is reflected in later queries. This paper presents a large-scale log analysis of agentic search based on 14.44M search requests (3.97M sessions) collected from DeepResearchGym, i.e., an open-source search API accessed by external agentic clients. We sessionize the logs, assign session-level intents and step-wise query-reformulation labels using LLM-based annotation, and propose Context-driven Term Adoption Rate (CTAR) to quantify whether newly introduced query terms are lexically traceable to previously retrieved evidence. Our analyses reveal distinctive behavioral patterns. First, over 90\% of multi-turn sessions contain at most ten steps, and 89\% of inter-step intervals fall under one minute. Second, behavior varies by intent. Fact-seeking sessions exhibit high repetition that increases over time, while sessions requiring reasoning sustain broader exploration. Third, query reformulations are often traceable to retrieved evidence across steps. On average, 54\% of newly introduced query terms appear in the accumulated evidence context, with additional traceability to earlier steps beyond the most recent retrieval. These findings provide candidate signals for repetition-aware stopping, intent-adaptive retrieval budgeting, and explicit cross-step context tracking. We released the anonymized logs, making them available at a public HuggingFace~\chref{https://huggingface.co/datasets/cx-cmu/deepresearchgym-agentic-search-logs}{repository}.

Neural retrieval models excel in Web search, but their training requires substantial amounts of labeled query-document pairs, which are costly to obtain. With the widespread availability of Web document collections like ClueWeb22, synthetic queries generated by large language models offer a scalable alternative. Still, synthetic training queries often vary in quality, which leads to suboptimal downstream retrieval performance. Existing methods typically filter out noisy query-document pairs based on signals from an external re-ranker. In contrast, we propose a framework that leverages Direct Preference Optimization (DPO) to integrate ranking signals into the query generation process, aiming to directly optimize the model towards generating high-quality queries that maximize downstream retrieval effectiveness. Experiments show higher ranker-assessed relevance between query-document pairs after DPO, leading to stronger downstream performance on the MS~MARCO benchmark when compared to baseline models trained with synthetic data.

Methods based on Contrastive Language-Image Pre-training (CLIP) are nowadays extensively used in support of vision-and-language tasks involving remote sensing data, such as cross-modal retrieval. The adaptation of CLIP to this specific domain has relied on model fine-tuning with the standard contrastive objective, using existing human-labeled image-caption datasets, or using synthetic data corresponding to image-caption pairs derived from other annotations over remote sensing images (e.g., object classes). The use of different pre-training mechanisms has received less attention, and only a few exceptions have considered multilingual inputs. This work proposes a novel vision-and-language model for the remote sensing domain, exploring the fine-tuning of a multilingual CLIP model and testing the use of a self-supervised method based on aligning local and global representations from individual input images, together with the standard CLIP objective. Model training relied on assembling pre-existing datasets of remote sensing images paired with English captions, followed by the use of automated machine translation into nine additional languages. We show that translated data is indeed helpful, e.g. improving performance also on English. Our resulting model, which we named Remote Sensing Multilingual CLIP (RS-M-CLIP), obtains state-of-the-art results in a variety of vision-and-language tasks, including cross-modal and multilingual image-text retrieval, or zero-shot image classification.

Tip-of-the-Tongue (ToT) retrieval is challenging for search engines because the queries are usually natural-language, verbose, and contain uncertain and inaccurate information. This paper studies the generalization capabilities of existing retrieval methods with ToT queries in multiple domains. We curate a multi-domain dataset and evaluate the effectiveness of recall-oriented first-stage retrieval methods across the different domains, considering in-domain, out-of-domain, and multi-domain training settings. We further explore the use of a Large Language Model (LLM), i.e. GPT-4, for zero-shot re-ranking in various ToT domains, relying solely on the item titles. Results show that multi-domain training enhances recall, and that LLMs are strong zero-shot re-rankers, especially for popular items, outperforming direct GPT-4 prompting without first-stage retrieval. Datasets and code can be found on GitHub https://github.com/LuisPB7/TipTongue

Socio-demographic information is usually only accessible at relatively coarse spatial resolutions. However, its availability at thinner granularities is of substantial interest for several stakeholders, since it enhances the formulation of informed hypotheses on the distribution of population indicators. Spatial disaggregation methods aim to compute these fine-grained estimates, often using regression algorithms that employ ancillary data to re-distribute the aggregated information. However, since disaggregation tasks are ill-posed, and given that examples of disaggregated data at the target geospatial resolution are seldom available, model training is particularly challenging. We propose to address this problem through a self-supervision framework that iteratively refines initial estimates from seminal disaggregation heuristics. Specifically, we propose to co-train two different models, using the results from one model to train/refine the other. By doing so, we are able to explore complementary views from the data. We assessed the use of co-training with a fast regressor based on random forests that takes individual raster cells as input, together with a more expressive model, based on a fully-convolutional neural network, that takes raster patches as input. We also compared co-training against the use of self-training with a single model. In experiments involving the disaggregation of a socio-demographic variable collected for Continental Portugal, the results show that our co-training approach outperforms alternative disaggregation approaches, including methods based on self-training or co-training with two similar fully-convolutional models. Co-training is effective at exploring the characteristics of both regression algorithms, leading to a consistent improvement in different types of error metrics.

People often ask questions about places, and this is reflected on the frequency of geo-spatial queries made to information retrieval and question answering systems. Recent developments associated to these two types of systems rely on deep neural networks, specifically on methods for passage retrieval based on Transformer models, trained on large datasets like MS-MARCO. Despite significant progress in approaches for retrieving (or re-ranking) passages from a document collection according to their relevance to an input query, few studies have specifically looked at geo-spatial queries (i.e., where-questions directly concerning locations, and also questions covering other informational needs relating to places, their types, and affordances). In this work, we explore neural retrieval models in the context of geo-spatial queries, using a subset of MS-MARCO with questions and passages containing place-names. After characterizing the subset of MS-MARCO, we analyzed a re-ranking strategy based on geographic distance, which we argue to be useful for selecting hard negative examples for model training. Then, we fine-tuned neural ranking models, following bi-encoder or cross-encoder strategies, using the MS-MARCO subset together with a geographically-aware negative sampling procedure. Experimental results show that the fine-tuned models can indeed achieve a superior performance. We also describe a simple knowledge distillation procedure to further improve the computationally more efficient bi-encoder models, using the results of the cross-encoder.

Remote sensing image captioning involves generating a concise textual description for an input aerial image. Most previous methods are based on neural encoder-decoder models trained to generate a sequence of discrete outputs with the standard cross-entropy token-level loss. This paper explores an alternative method based on continuous outputs, generating sequences of embedding vectors instead of directly predicting discrete word tokens. We argue that continuous outputs can facilitate the optimization of semantic similarity, as opposed to exact word-by-word matches. It also facilitates the use of loss functions that compare different views of the data. This includes comparing representations for individual tokens and for the entire captions, and also comparing captions against intermediate image representations. We experimentally compared discrete versus continuous output methods over the RSICD dataset, extensively used in the area. Results show that continuous outputs can indeed lead to better results, and our approach performs competitively with the state-of-the-art model in the area.

We present a novel deep learning approach for spatio-temporal forecasting with remote sensing data, extending a previous model named Spatio-Temporal Convolutional Sequence to Sequence Network (STConvS2S) in several directions. Experiments using datasets from previous studies show that the proposed approaches outperform the original STConvS2S and other baseline models on tasks related to predicting future time-steps. In tests related to predicting a missing time-step, some of the proposed extensions also lead to improvements over the original STConvS2S architecture, although simpler models seem to be beneficial in this scenario.

Scholars in the humanities have a long tradition in the investigation of spatial theory and methods, as well as on the analysis of space and place within its multiple disciplines. Although technolo...

Toponym matching, i.e. pairing strings that represent the same real-world location, is a fundamental problemfor several practical applications. The current state-of-the-art relies on string similarity metrics, either specifically developed for matching place names or integrated within methods that combine multiple metrics. However, these methods all rely on common sub-strings in order to establish similarity, and they do not effectively capture the character replacements involved in toponym changes due to transliterations or to changes in language and culture over time. In this article, we present a novel matching approach, leveraging a deep neural network to classify pairs of toponyms as either matching or nonmatching. The proposed network architecture uses recurrent nodes to build representations from the sequences of bytes that correspond to the strings that are to be matched. These representations are then combined and passed to feed-forward nodes, finally leading to a classification decision. We present the results of a wide-ranging evaluation on the performance of the proposed method, using a large dataset collected from the GeoNames gazetteer. These results show that the proposed method can significantly outperform individual similarity metrics from previous studies, as well as previous methods based on supervised machine learning for combining multiple metrics.

Following the increasing adoption of online social networks, the users of those services can now easily collect and transmit various types of data instantaneously from real-world locations (e.g., geo-referenced pictures, videos, messages, etc.). This paper argues that one can take advantage of the information that is spread throughout social networks like Twitter, in order to provide a realistic solution for situational awareness that can be useful for decision makers during particular events. Through the generation of English captions assigned to geo-referenced photographs extracted from tweets, and by using these captions to infer important features of those photographs, social monitoring approaches based on the images can produce prompt contextual information that helps in disaster monitoring and response. We specifically describe the use of the Show and Tell model for this purpose. We managed to obtain 100% of precision, 94.42% of accuracy, and 93.93% of recall, in the task of classifying geo-referenced Twitter images depicting the accumulation of snow, useful for applications related to snow removal.

This paper describes Triplex-ST, a novel information extraction system for collecting spatio-temporal information from textual resources. Triplex-ST is based on a distantly supervised approach, which leverages rich linguistic annotations together with information in existing knowledge bases. In particular, we leverage triples associated with temporal and/or spatial contexts, e.g., as available from the YAGO knowledge base, so as to infer templates that capture new facts from previously unseen sentences.

The LocWeb 2014 workshop continues a successful workshop series at the intersection of geospatial search, information management, and Web architecture with a focus towards location-aware information access. The workshop reflects a multitude of fields that demand and utilize location features, featuring presentations that look at the topic of location on the Web from an interdisciplinary perspective, including new approaches dealing with or utilizing geospatial information.

This article presents an automated method for defining the boundaries of imprecise geographic regions, based on publicly available data. The method uses one-class support vector machines (SVMs) for interpolating from a set of point locations, which are assumed to lie in the region whose boundaries are to be defined, and leverages also on a combination of multiple Gaussian kernels, within the formalism of SVMs, to improve accuracy. The points that are used for model training correspond to geospatial coordinates associated with Flickr photos that are tagged with the name of the vague region to be defined. Besides considering latitude and longitude coordinates from Flickr photos, as done in a previous related work, each point location is also associated with a set of descriptive features, obtained from textual annotations and from publicly available raster datasets encoding population counts, terrain elevation, and/or land coverage information. The overall approach is evaluated by means of statistical classification measures, using regions whose boundaries are well defined (i.e., the official boundaries for several European countries). Besides this formal evaluation, we also illustrate our results for several vague regions. Results show that our method performs better than a previous state-of-the-art approach (i.e., we measured an improvement of 5.5% in terms of the F1 metric), which was based solely on interpolating from the geospatial coordinates of known points.

With the prevalence of mobile computing systems and location based services, the research interest on spatiotemporal data has significantly increased, as evidenced by the collection of huge amounts of movement data. Consequently, this type of data raises several issues, namely in the research area of geographic information visualization. Despite the existence of several visual analysis techniques for the exploration of movement data, it is still unclear how usable and useful these techniques are, how can they be improved, and for which situations are these techniques most suitable. In this paper, we present current open challenges on the visual analysis of movement data, and the Ph.D work in progress aiming to address these problems. Our work will explore several factors that may affect the users' performance, and, based on those factors we will propose a taxonomy and an evaluation framework covering different tasks and techniques.

The association of illustrative photos to textual contents is a challenging cross-media retrieval problem with many practical applications. We have, for instance, that the association of photos to specific parts of travelogues, i.e. textual descriptions for travel experiences, may lead to a better usage of these documents. Despite the huge number of high quality photos in websites like Flickr, these photos are currently not being properly explored in cross-media retrieval tasks.

Geo-temporal information is pervasive over textual documents, since most of them contain references to particular locations, calendar dates, clock times or duration periods. An important text analytics problem is therefore related to resolving the place names and the temporal expressions referenced in the texts, i.e. linking the character strings in the documents that correspond to either locations or temporal instances, to the specific geospatial coordinates or the time intervals that they refer to. However, geo-temporal reference resolution presents several non-trivial problems to the area of text mining, due to the inherent ambiguity and contextual assumptions of natural language discourse.

This paper presents an application of PageRank for assigning documents with a corresponding geographical scope. We describe the technique in detail, together with its theoretical formulation. Experimental results are promising, comparing favorably with previous proposals.

Automated methods for resource annotation are a clear necessity, as the success of the semantic Web depends on the availability of Web resources with meta data conforming to known standards and ontologies. This paper describes the WebCAT framework for automatically generating RDF descriptions of Web pages. We present a general view of the system and the algorithms involved, giving an emphasis to typical issues in processing Web data.