News outlets are currently adopting AI to summarize news stories and experiment with conversational agents to convey news. In this paper, we explore this emerging practice, in which an AI-powered agent retells news in dialogue with the user rather than presenting them with a fixed narrative. In a co-speculation workshop with industry professionals and through a field trial of an LLM-based conversational news agent probe, we explore the design space of this conversational news format and examine the experiences, insights, and concerns reported by the participants. Our contribution is twofold. First, we identify five dimensions that shape how news can be retold by a conversational agent. Second, we provide a detailed empirical account of how users experience conversational news as clear and easy to follow, enabling them to probe and question stories in new ways, and address how these interactions are marked by tensions around trust, accuracy, and transparency.

Traditionally, sports commentators provide viewers with diverse information, encompassing in-game developments and player performances. Yet young adult football viewers increasingly use mobile devices for deeper insights during football matches. Such insights into players on the pitch and performance statistics support viewers’ understanding of game stakes, creating a more engaging viewing experience. Inspired by commentators’ traditional roles and to incorporate information into a single platform, we developed AiCommentator, a Multimodal Conversational Agent (MCA) for embedded visualization and conversational interactions in football broadcast video. AiCommentator integrates embedded visualization, either with an automated non-interactive or with a responsive interactive commentary mode. Our system builds upon multimodal techniques, integrating computer vision and large language models, to demonstrate ways for designing tailored, interactive sports-viewing content. AiCommentator’s event system infers game states based on a multi-object tracking algorithm and computer vision backend, facilitating automated responsive commentary. We address three key topics: evaluating young adults’ satisfaction and immersion across the two viewing modes, enhancing viewer understanding of in-game events and players on the pitch, and devising methods to present this information in a usable manner. In a mixed-method evaluation (n=16) of AiCommentator, we found that the participants appreciated aspects of both system modes but preferred the interactive mode, expressing a higher degree of engagement and satisfaction. Our paper reports on our development of AiCommentator and presents the results from our user study, demonstrating the promise of interactive MCA for a more engaging sports viewing experience. Systems like AiCommentator could be pivotal in transforming the interactivity and accessibility of sports content, revolutionizing how sports viewers engage with video content.

Advancements in Natural Language Processing (NLP) and Computer Vision (CV) are revolutionizing how we experience sports broadcasting. Traditionally, sports commentary has played a crucial role in enhancing viewer understanding and engagement with live games. Yet, the prospects of automated commentary, especially in light of these technological advancements and their impact on viewers’ experience, remain largely unexplored. This paper elaborates upon an innovative automated commentary system that integrates NLP and CV to provide a multimodal experience, combining auditory feedback through text-to-speech and visual cues, known as italicizing, for real-time in-game commentary. The system supports color commentary, which aims to inform the viewer of information surrounding the game by pulling additional content from a database. Moreover, it also supports play-by-play commentary covering in-game developments derived from an event system based on CV. As the system reinvents the role of commentary in sports video, we must consider the design and implications of multimodal artificial commentators. A focused user study with eight participants aimed at understanding the design implications of such multimodal artificial commentators reveals critical insights. Key findings emphasize the importance of language precision, content relevance, and delivery style in automated commentary, underscoring the necessity for personalization to meet diverse viewer preferences. Our results validate the potential value and effectiveness of multimodal feedback and derive design considerations, particularly in personalizing content to revolutionize the role of commentary in sports broadcasts.

Recovering from serious illness involves a bodily and psychosocial reorientation in everyday life. Survivors of gynecological cancer often experience bodily changes, fear of cancer recurrence, and changes in sexual health. This paper explores how we can use audiovisual narratives based on experiences of gynecological cancer survival in the design of an online intervention. From a typology of cancer survival, we designed three audiovisual narratives in an experience-centered design process involving gynecological cancer survivors. The narratives were evaluated by 10 participants formerly treated for gynecological cancer. In a thematic analysis, we explore how these narratives set the stage for identification and reflection by being experienced as relatable, provoking, and realistic. Finally, we discuss how the survivors’ experience of the narratives can be construed as meaningful, and how accounts of experiences can be included in a design process to create narrative content for online interventions.