We present the Robot Bookworm, a multi-session intervention co-designed with children and educators to foster reading motivation through personalized book discussions. The robot assigned each child a personally fitting book and engaged them in pedagogically structured discussions, with personalized book-aligned dialogic content selectively generated offline by a language model and moderated by people to ensure safety. We compared a personalized book discussion condition with a book-neutral control in a four-session, large-scale user study in two primary schools (N = 101, 8-11 y.o.). The intervention significantly increased reader-book relatedness and reading enjoyment, particularly for children with below-ceiling baseline enjoyment, but had no effect on intrinsic motivation. At a one-year follow-up, the quantitative effects were not sustained. However, children reported perceived positive shifts in attitudes towards reading, which they attributed to the Robot Bookworm.

Retrieving specific symbolic elements within paintings, such as a pomegranate representing fertility, requires fine-grained cross-modal understanding beyond whole-image-level matching. Existing art datasets lack region-level annotations that link localized objects to their iconographic descriptions, limiting semantic search to object labels. We present RichArt, a benchmark dataset of 7,087 paintings with 20,882 region-level annotations pairing visual elements with their symbolic narratives extracted from museum catalogs using a scalable, semi-automated LLM-based pipeline, followed by human validation. To support bidirectional retrieval (text to region and region to text), we introduce MARGE-GD (Multi-modal Alignment of RichArt Grounding Embeddings via Grounding DINO). Building on Grounding DINO’s architecture, MARGE-GD projects region and text representations into a shared embedding space via MLP heads trained with contrastive loss, enabling retrieval while preserving visual grounding capability. On RichArt, MARGE-GD achieves an MRR of 0.75 (text to region) and 0.61 (region to text), outperforming CLIP by 2.59 × and 2.77 × respectively, demonstrating significant improvements in fine-grained semantic retrieval.

Conversational agents in art exhibitions can enhance user engagement and understanding of artworks by providing contextual information, especially through voice interactions. However, creating a deeper personal connection with art - which often requires direct aesthetic and visual experiences - remains a challenge. This paper examines how integrating visual perception into conversational agents can enhance alignment with visitors' artistic interpretations, thereby fostering deeper engagement with interactive art exhibitions. We introduce a voice-based conversational agent enhanced with visual capabilities via a multimodal large language model (MLLM), allowing the agent to perceive, interpret and discuss artworks in real-time with visitors. The system utilizes a simplified Retrieval-Augmented Generation (RAG) architecture, which collects voice inputs, retrieves relevant information from a domain knowledge graph, and uses the LLM to generate conversational responses, which are then converted into voice outputs. A user study with 36 participants, divided into two groups, was conducted to compare the enhanced system with a baseline system that lacked visual input. Our results show that the visually enhanced system significantly improved visitor engagement and perception. Content analysis of the conversational transcripts further revealed a wider range of conversational topics, deeper visitor perceptions, and the agent's ability to provide more nuanced, visually-related discussions.

Understanding how users engage with socially interactive agents in immersive cultural environments is crucial for designing adaptive and contextually appropriate virtual experiences.This study explores how individual user traits-including familiarity with virtual reality, museum visitation habits, prior knowledge, interest in specific artworks, and interaction preferences-relate to gaze behavior and conversational engagement in a VR art exhibition.Fifty-two participants explored a virtual museum featuring five paintings and interacted with agents designed to share information and facilitate discussion.Data were collected through questionnaires, eye-tracking metrics (e.g., fixation duration, scanpath length), and conversational transcripts (e.g., response length, number of user turns).A multimodal analysis aligned gaze and speech data to examine how visual attention and verbal interactions co-occur in relation to user traits.Results indicate that traits such as prior knowledge, interest, and personalization preferences significantly influence both gaze patterns and conversational behaviors.While some effects varied in strength or consistency, the findings offer valuable insights into how personal characteristics shape user-agent interactions.These insights inform the design of adaptive virtual agents and support broader efforts to model multimodal user bahevior for real-time personalization in immersive environment.

We propose a novel type of low distortion radial embedding which focuses on one specific entity and its closest neighbors. Our embedding preserves near-exact distances to the focus entity and aims to minimize distortion between the other entities. We present an interactive exploration tool SolarView which places the focus entity at the center of a "solar system" and embeds its neighbors guided by concentric circles. SolarView provides an implementation of our novel embedding and several state-of-the-art dimensionality reduction and embedding techniques, which we adapted to our setting in various ways. We experimentally evaluated our embedding and compared it to these state-of-the-art techniques. The results show that our embedding competes with these techniques and achieves low distortion in practice. Our method performs particularly well when the visualization, and hence the embedding, adheres to the solar system design principle of our application. Nonetheless-as with all dimensionality reduction techniques-the distortion may be high. We leverage interaction techniques to give clear visual cues that allow users to accurately judge distortion. We illustrate the use of SolarView by exploring the high-dimensional metric space of bibliographic entity similarities.

Artificial intelligence has a long history of learning from domain problems ranging from chess to jeopardy. In this work, we look at a problem stemming from social science, namely, how do social relationships influence communication content and vice versa. The tools used to study communication content (content analysis) have rarely been combined with those used to study social relationships (social network analysis). Furthermore, there is even less work addressing the longitudinal characteristics of such a combination. This paper presents a general framework for measuring the dynamic bi-directional influence between communication content and social networks. The framework leverages the idea that knowledge about both kinds of networks can be represented using the same knowledge representation. In particular, through the use of Semantic Web standards, the extraction of networks is made easier. The framework is applied to two use-cases: online forum discussions and conference publications. The results provide a new perspective over the dynamics involving both social networks and communication content.

We consider continuous quantities that are used to describe the physical world, such as colour, shape, sound, texture and spatial and temporal arrangements. Natural languages are not adept at describing these quantities, nor are they easily incorporated into ontologies in the form of discrete terms. In this article, we analyse the way that natural languages handle continuous quantities, propose a general semantics based on metric spaces, and describe how to treat semantic values computationally, so that we may automate the processing of texts which describe continuous quantities allowing, for example, query evaluation and the integration of multiple texts. This provides a basis for incorporating these quantities into ontologies and combining their semantics with automated reasoning tools. We run a series of experiments to evaluate the semantics, the general framework, and the computational system we have developed.

Information integration and retrieval have been important problems for many information systems-it is hard to combine multidimensional and parallel information and make them available for application queries. In our previous work [12], we have shown how to use ontologies to facilitate integrating and querying parallel but single dimensional information. In this paper, we further investigate how to take advantage of ontologies to facilitate integrating parallel information and querying over multiple quantities.