The practice of interactive documentary can provide participatory opportunities thanks to its capacity for making space for audiences to leave their marks in a film. However, participants’ inclusion has often been limited to an a-posteriori contribution of materials rather than a structural involvement in the film design. What happens when we treat participants as authors and let them imagine and design their own interactive film? This paper explores how design processes from participatory filmmaking can be adapted to achieve this goal by presenting the design process that led to the production of an interactive participatory film on mental health, Stepping Through Interactive. Five participants with lived experience of mental health problems explored, designed, and produced a non-linear film form to effectively represent their personal accounts of mental health. We review the challenges faced and the strategies deployed in the design process in view of supporting similar forms of production in other contexts.

Esport is a large and still growing industry with vast audiences. Multiplayer Online Battle Arenas (MOBAs), a sub-genre of esports, possess a very complex environment, which often leads to experts missing important coverage while broadcasting live competitions. One common game event that holds significant importance for broadcasting is referred to as a team fight engagement. Professional player's own knowledge and understanding of the game may provide a solution to this problem. This paper suggests a model that predicts and detects ongoing team fights in a live scenario. This approach outlines a novel technique of deriving representations of a complex game environment by relying on player knowledge. This is done by analysing the positions of the in-game characters and their associated cameras, utilising this data to train a neural network. The proposed model is able to both assist in the production of live esport coverage as well as provide a live, expert-derived, analysis of the game without the need of relying on outside sources.

Work in AI-based explanation systems has uncovered an interesting contradiction: people prefer and learn best from why explanations but expert esports commentators primarily answer what questions when explaining complex behavior in real-time strategy games. Three possible explanations for this contradiction are: 1.) broadcast audiences are well-informed and do not need why explanations; 2.) consuming why explanations in real-time is too cognitively demanding for audiences; or 3.) producing live why explanations is too difficult for commentators. We answer this open question by investigating the effects of explanation types and presentation modalities on audience recall and cognitive load in the context of an esports broadcast. We recruit 111 Dota 2 players and split them into three groups: the first group views a Dota 2 broadcast, the second group has the addition of an interactive map that provides what explanations, and the final group receives the interactive map with detailed why explanations. We find that participants who receive short interactive text prompts that provide what explanations outperform the no explanation group on a multiple-choice recall task. We also find that participants who receive detailed why explanations submit reports of cognitive load that are higher than the no explanation group. Our evidence supports the conclusion that informed audiences benefit from explanations but do not have the cognitive resources to process why answers in real-time. It also supports the conclusion that stacked explanation interventions across different modalities, like audio, interactivity, and text, can aid real-time comprehension when attention resources are limited. Together, our results indicate that interactive multimedia interfaces can be leveraged to quickly guide attention and provide low-cost explanations to improve intelligibility when time is too scarce for cognitively demanding why explanations.

Brooke Leave Home is a personalized film designed to engage a non-expert audience with open data about the support young adults receive when leaving the care system in England. The film draws upon a range of video-based data storytelling techniques to present each viewer with a personalized perspective on the topic based on data from their own local area. We present the film's design and describe how its storytelling techniques were developed to support viewers in understanding, and fostering empathic connections with, the data sources featured and the implications they have for care leavers. We also present a study with 47 viewers, which explores how these techniques were experienced and how effective they were in aiding engagement with the data included and its meaning.

Esports (competitive videogames) have grown into a global phenomenon with over 450m viewers and a 1.5bn USD market. Esports broadcasts follow a similar structure to traditional sports. However, due to their virtual nature, a large and detailed amount data is available about in-game actions not currently accessible in traditional sport. This provides an opportunity to incorporate novel insights about complex aspects of gameplay into the audience experience – enabling more in-depth coverage for experienced viewers, and increased accessibility for newcomers. Previous research has only explored a limited range of ways data could be incorporated into esports viewing (e.g. data visualizations post-match) and only a few studies have investigated how the presentation of statistics impacts spectators’ experiences and viewing behaviors. We present Weavr, a companion app that allows audiences to consume data-driven insights during and around esports broadcasts. We report on deployments at two major tournaments, that provide ecologically valid findings about how the app’s features were experienced by audiences and their impact on viewing behavior. We discuss implications for the design of second-screen apps for live esports events, and for traditional sports as similar data becomes available for them via improved tracking technologies.

This paper introduces a generic framework for OBM storytelling. Aiming to function as a complete end-to-end reference for authoring OBM narrative content - from conception to realization - it proposes an integrated model that includes the three essential levels: conceptual, technological and aesthetic. At the conceptual level, we introduce a set of abstractions which provide a unified reference for thinking, describing and analysing interactive narrative structures of OBM content. Their recursive nature make our model stand out in terms of its power of expression. These abstractions have direct one-to-one operational counterparts implemented in our production-independent authoring toolkit – Cutting Room. This ensures that any specific story designs conceived within the proposed conceptual model are directly realisable as OBM productions. This isomorphic relationship between the abstract concepts and their operationalisation is another distinguishing aspect of our overall proposition. We have validated the model at the aesthetic level through the production of the interactive film What is Love?, experienced by over 900 people at the media art festival Mediale 2018 in York, UK, and evaluated through a dedicated questionnaire by 94 of them. As the foundations of OBM storytelling have not yet been established, we trust this paper constitutes a significant milestone in its development.

We examine the concept and characteristics of “Augmented Reality Television” (ARTV) using a four-step investigation method consisting of (1) an analysis of commonly-accepted perspectives on Augmented and Mixed Reality systems, (2) a literature survey of previous work on ARTV, (3) relevant connections with other areas of scientific investigation from TVX/IMX, such as Ambient Media, Interactive TV, and 3-D TV, and (4) by proposing a conceptual framework for ARTV called the “Augmented Reality Television Continuum.” Our work comes at a moment when the excitement and hype about the potential of AR for home entertainment has overlooked rigorous analysis and clear-cut examinations of the concepts involved, which should be the hallmark of any exact science. With this work, our goal is to draw the community’s attention toward fundamentals and first principles of ARTV and to tease out its salient qualities on solid foundations.

Giving presents can have a positive impact upon the giver's mental well-being [1-4]. We wonder whether giving a present to an AI agent could have the same effect. We investigate this question in the context of video games with non-player characters (NPCs), because the rapid expansion of the game industry makes NPCs one of the most common ways in which we experience AI agents. We seek to understand whether players benefit mentally by giving gifts to NPCs in video games. We intend to observe the behaviour of players and measure their psychological changes through a specially designed video game providing them with mechanisms for gifting to NPCs. We have taken an understanding of gifting from the real world and applied this understanding within a game. The game development has been an iterative process of design-build-playtest. Such a process helps us better understand the research space and to better describe the questions. This article reports the latest findings from the game development after playtesting with 12 participants.

This paper reports on an experimental production of The Tempest that was developed in collaboration with Miracle Theatre Company realised as a distributed performance from two separate stages through a dynamically configured telepresence system. The production allowed an exploration of the way a range of technologies, including consumer grade broadband, cameras and projection technologies could affect the development and delivery of live theatre by regional touring company. The architecture of the communication platform used to deliver the performance is introduced as are two novel software tools that are used to describe and control the way the play should be captured and represented.

In this technical demonstration, we invite visitors to join a group videoconferencing session with remote participants across Europe, using the Vconect system. This research prototype automatically takes decisions about the presentation of available video streams by processing the audio stream from each participant. It observes patterns in turn-taking behavior and dynamically computes conversation metrics to automatically decide, for each conversation node's screen, how to compose (in layout) and mix (in time) the streams originating from the other nodes. The demo invites to discuss future possibilities in video mediated group communication beyond the currently implemented functionality of the demo system.

This paper reports research into video-mediated synchronous communication within social groups. The ultimate aim of the research is to create a more natural medium for interaction, aware of the context in which it operates, able to continuously adapt itself to the communication needs and optimise the way in which it captures and transmits aspects of the communication. This, is hypothesised, can be achieved by equipping each of the various locations involved in the communication with multiple controllable video cameras and microphones, and mixing the resulting content through techniques similar to those used in television?a process referred to as "orchestration". Through orchestration, each location should be able to receive the appropriate perspectives and levels of detail, thus generating experiences in which the spatial separation between participants is minimised. The paper defines the concept of orchestration and presents two major evaluation experiments that provide supporting evidence for the main assumption and motivate further research, in richer interaction contexts, into this concept.

In this paper we present an approach to the reasoning required to support multi-location, multi-camera group-to-group video communication, which we call orchestration. Orchestration is akin to virtual directing: it has to ensure that each location displays the most adequate shots from all the other available sources. Its input is low-level cues extracted automatically from the AV streams. They are processed to detect higher-level events that determine the state of the communication. Directorial decisions are then inferred, reflecting social communication as well as stylistic criteria. Finally, they are transformed into camera and editing commands, directly executable by the AV infrastructure. Here, we present the architecture of the Orchestrator and sketch our rule-based approach to reasoning.

We report on the development of a video communication medium through which groups of people situated in different physical locations can naturally talk to each other, see and hear each other, and engage in social entertaining activities. Participants are free to move within their space and behave in a manner closer to collocated experiences. Essentially, this is implemented as a multi-location, multi-camera, hands-free video conferencing system between groups, with integrated support for entertaining activities. In this paper we focus on automatic orchestration, the reasoning process that applies screen grammar to best support the communication. We present a formal model for representing orchestration rules and discuss initial evaluation results.

This article is motivated by the question whether television should do more than simply offer interactive services alongside (and separately from) traditional linear programs, in the context of its dominance being seriously challenged and threatened by interactive forms of screen media entertainment. It suggests: yes. Interactive narrativity , that is, the ability to interact with (and influence) stories whilst they are being told, represents one clear development path for interactive television. The capabilities of computing technology are ripe for exploring this new form of storytelling, from creation to commercial distribution. The article starts by looking at the relationship between narrativity and interactivity in the current context of screen media, and identifies clear signs of interest from certain European public broadcasters in interactive TV narratives. It then presents in detail four recent experimental interactive TV productions in the genres of drama, news, and documentary, developed in collaboration with public broadcasters, which illustrate the potential and richness of this new form of storytelling, but also highlight new technological capabilities necessary for such productions. A number of essential technological requirements are then discussed in more detail in the final part. The article suggests that the ShapeShifting Media Technology, employed in the implementation of the four productions, has made significant advances both at the technological and the creative ends in supporting the development of interactive TV narrativity, but, however, that further developments are required before being able to answer questions such as “Would end users want such a form of screen media entertainment?” and “Would it be effective for both end users and producers?”

This paper is part of an investigation into formal and operational models of the notion of compliance between a design and a body of regulatory design knowledge. Design compliance is defined here on the basis of database integrity. The paper provides a formal model of design compliance, based on the assumption that designs are representable as Horn clauses with negation in the body and design rules are representable as denials in first order logic (FOL). This model has been implemented in aiDA, a prototype of a critiquing intelligent design assistant developed for this research. Unfortunately, FOL does not provide adequate representational support for all the statement structures that occur in real bodies of regulatory design knowledge. Some statements require different logic forms and, for these, the definition of compliance has to be reconsidered. This paper presents a motivation for formalising design regulations, an overview of what has so far been done by using first order logic, and a new extension of the mechanism to duty and contrary-to-duty statements. The results draw on results in database systems and normative reasoning.