Web search engines have long served as indispensable tools for information retrieval; user behavior and query formulation strategies have been well studied. The introduction of search engines powered by large language models (LLMs) suggested more conversational search and new types of query strategies. In this paper, we compare traditional and LLM-based search for the task of image geolocation, i.e., determining the location where an image was captured. Our work examines user interactions, with a particular focus on query formulation strategies. In our study, 60 participants were assigned either traditional or LLM-based search engines as assistants for geolocation. Participants using traditional search more accurately predicted the location of the image compared to those using the LLM-based search. Distinct strategies emerged between users depending on the type of assistant. Participants using the LLM-based search issued longer, more natural language queries, but had shorter search sessions. When reformulating their search queries, traditional search participants tended to add more terms to their initial queries, whereas participants using the LLM-based search consistently rephrased their initial queries.

Annotated images are required for numerous computer vision tasks; however, the annotation process can be time-consuming for crowdworkers and experts. Previous work has investigated novel interaction techniques and task reformulation to speed up this process; however, there remains a gap in optimizing more complex annotation tasks, such as object matching. In this paper, we explore the impact of varying the amount of context provided to annotators. We hypothesize that reducing the context around the object being matched will improve speed without sacrificing the accuracy of the annotation task. To test this hypothesis, we developed a semi-automated annotation pipeline that pre-processes images to adjust the amount of context shown around an object of interest. We conducted two studies (n = 130, n = 10) to assess the effects of context quantitatively and qualitatively. We found that while the accuracy remained the same, the time spent on the task was significantly reduced when there was less context surrounding the object. However, our qualitative findings revealed multiple scenarios in which context served as a means of guiding the object matching task, and many others in which the distinctiveness of the object guided the matching task and additional context was not needed.

Amassing large-scale datasets used to train machine learning algorithms often includes crowd-sourcing or web scraping. The data resulting from these approaches can carry undesired societal biases that are reflected in the predictions of the learning system. Recently, researchers have proposed mitigation strategies targeting either the learning algorithms or the data used for training. In this paper, we evaluate a simple data augmentation strategy for the task of automated image captioning, namely substituting gendered terms for gender-neutral equivalents. We evaluated this approach on multiple recent image captioning models using both objective and subjective analysis. We found that human raters did not find a difference in the quality of the image captions and, in some cases, the model was able to generate more accurate captions with additional detail when trained with gender-neutral data.