User experience and trust in AI-assisted technologies are key factors in controlling their adoption. We investigate these aspects in an Agentic AI platform that integrates routine productivity services and exhibits different levels of autonomy: a manual baseline that lacks AI-driven automation, an Agentic AI with medium autonomy that requires user confirmation before acting, and an Agentic AI with high autonomy that acts proactively for low-stakes tasks. The study, involving 230 participants with heterogeneous professional backgrounds, examines how autonomy of the system affects user activity, user workload, perceived support, and trust. We found that both Agentic AI systems outperformed the baseline in user productivity. In task execution, they achieved a precision of over 82%, higher than the baseline’s 65%. The recall of the Agentic AI system with high autonomy was 63%. This denotes much higher throughput than the system without AI-driven automation (14%). The Agentic AI systems outperformed the baseline in workload reduction (NASA-TLX Aggregate score) with a statistically significant difference. Both AI-driven systems received equivalent or slightly higher trust than the baseline. However, the system with medium autonomy was the best at balancing productivity gains and user preferences for control. Specifically, the correlations between individual user characteristics (Desirability of Control and Propensity to Trust) and the resulting trust in the systems suggest that the influence of personal traits on system evaluation is least pronounced when automation is combined with explicit user intervention. These results encourage the adoption of user-controllable Agentic AI architectures in multitasking support.

Reproducibility lies at the basis of the empirical method: a novel approach will be widely adopted if its experimental results can be validated and reproduced by the community. Previous work on reproducibility in Information Retrieval (IR) has mainly addressed the reproducibility and replicability of offline experiments, with a few exceptions that replicate user studies. To the best of our knowledge, no previous work has investigated how reproducibility affects real users. In this paper, we do that by evaluating and comparing the reproducibility of an IR system both offline and online. We consider a reference system and generate a constellation of reproduced systems with varying parameters. We select 6 systems with different degrees of offline reproducibility. We then run a between-subjects online experiment with 280 participants and collect clicks to evaluate online reproducibility. Results show that real users do not perceive moderate variations of the reproducibility degree of systems, while they become relevant when the difference with the original system increases. Furthermore, we trained a click model to evaluate online reproducibility with simulated clicks. Results are not consistent with those from the user study, suggesting that better click models are needed to evaluate online reproducibility. Our data and source code are publicly available: https://github.com/angelogeninatti/reproducibilityLogs .

In the rapidly evolving landscape of technology and sustainability, leveraging Recommender Systems has emerged as a powerful tool for driving positive change. With a foundation in AI and data analytics, Recommender Systems can be effective in various domains, from e-commerce to energy management, inclusion and well-being. By harnessing the power of recommendation algorithms under a multi-stakeholder perspective, organizations and researchers can guide users towards more sustainable choices and behaviors, contributing to broader environmental and social goals. With this aim, our workshop provides a unique platform for researchers, practitioners, and platform owners to explore the integration of sustainability principles into Recommender Systems. Through presentations, discussions, and panels, participants can explore the theoretical foundations, practical implementations, and ethical and environmental considerations of sustainable Recommender Systems. By fostering collaboration and knowledge exchange, the workshop aims to catalyze innovation and inspire collective action towards a more sustainable future.

Autism Spectrum Disorder (ASD) affects sensory perception, making spatial exploration difficult. Recommender systems can assist ASD users by suggesting Points of Interest (POIs) aligned with their sensory preferences. However, demographic constraints, difficulties in engaging ASD users, and the complexity of obtaining sensory data position POI recommendation for ASD people as a low-resource problem. In this paper, we identify key challenges in developing such systems and present our ongoing efforts. Using a local ASD center as a use case, we are developing a structured user involvement protocol. From the limited data, we are deriving knowledge graphs (KGs) to model preferences and sensory aspects. We are then exploring KG-based techniques to generate paths from users to POIs to suggest. With psychologists, we are refining the paths structure to match varying complexity levels and translate them into natural language accessible for people with ASD.

The Workshop on Information Retrieval for Understudied Users (IR4U2) serves as a platform to highlight information retrieval (IR) research that directly impacts often understudied user groups. The second (IR4U2) workshop focuses on a user-centred AI perspective, which is vital for informing the design, development, and assessment of information retrieval systems that thoughtfully address the diverse needs of understudied populations, ensuring genuine accessibility and inclusivity. The objectives of IR4U2 are: (1) to build community and awareness by sharing AI and IR developments that serve underrepresented user groups in this research area; (2) to identify challenges and open issues along with lessons learned and challenges inherent to this area of research; and (3) to spark discussions that establish common frameworks for future research.

Recommender systems are designed to help users in situations of information overload. In recent years we observed increased interest in session-based recommendation scenarios, where the problem is to make item suggestions to users based only on interactions observed in an ongoing session, e.g., on an e-commerce site. However, in cases where interactions from previous user sessions are also available, the recommendations can be personalized according to the users’ long-term preferences, a process called session-aware recommendation. Today, research in this area is scattered, and many works only compare a newly proposed session-aware with existing session-based models. This makes it challenging to understand what represents the state-of-the-art. To close this research gap, we benchmarked recent session-aware algorithms against each other and against a number of session-based recommendation algorithms along with heuristic extensions thereof. Our comparison, to some surprise, revealed that (i) simple techniques based on nearest neighbors consistently outperform recent neural techniques and that (ii) session-aware models were mostly not better than approaches that do not use long-term preference information. Our work therefore points to potential methodological issues where new methods are compared to weak baselines, and it also indicates that there remains a huge potential for more sophisticated session-aware recommendation algorithms.

The benefits of neural approaches are undisputed in many application areas. However, today's research practice in applied machine learning---where researchers often use a variety of baselines, datasets, and evaluation procedures---can make it difficult to understand how much progress is actually achieved through novel technical approaches. In this work, we focus on the fast-developing area of session-based recommendation and aim to contribute to a better understanding of what represents the state-of-the-art.

The Information Retrieval research has used semantics to provide accurate search results, but the analysis of conceptual abstraction has mainly focused on information integration. We consider session-based query expansion in Geographical Information Retrieval, and investigate the impact of semantic granularity (i.e., specificity of concepts representation) on the suggestion of relevant types of information to search for. We study how different levels of detail in knowledge representation influence the capability of guiding the user in the exploration of a complex information space. A comparative analysis of the performance of a query expansion model, using three spatial ontologies defined at different semantic granularity levels, reveals that a fine-grained representation enhances recall. However, precision depends on how closely the ontologies match the way people conceptualize and verbally describe the geographic space.

Textual queries are largely employed in information retrieval to let users specify search goals in a natural way. However, differences in user and system terminologies can challenge the identification of the user's information needs, and thus the generation of relevant results. We argue that the explicit management of ontological knowledge, and of the meaning of concepts (by integrating linguistic and encyclopaedic knowledge in the system ontology), can improve the analysis of search queries, because it enables a flexible identification of the topics the user is searching for, regardless of the adopted vocabulary. This paper proposes an information retrieval support model based on semantic concept identification. Starting from the recognition of the ontology concepts that the search query refers to, this model exploits the qualifiers specified in the query to select information items on the basis of possibly fine-grained features. Moreover, it supports query expansion and reformulation by suggesting the exploration of semantically similar concepts, as well as of concepts related to those referred in the query through thematic relations. A test on a data-set collected using the OnToMap Participatory GIS has shown that this approach provides accurate results.