As immersive technologies advance, Mixed Reality Performances (MRP) increasingly integrate them but often face technical challenges that must balance artistic vision and practical constraints.These constraints sometimes lead to unavoidable limitations.The diversity of technologies and artistic goals in MRP prevents a onesize-fits-all solution to such dilemmas.This paper presents a model unifying strategies for addressing compromises between artistic intent and technological feasibility.Using reflexive thematic analysis of a performance-led case study, interviews, and independent case studies, we identify recurring strategies in Mixed Reality experiences with varying constraints.These strategies fall on an axis based on the audience's awareness of limitations and are categorized into five approaches: Avoid, Disguise, Tolerate, Integrate, or Leverage.We argue that this framework helps designers better navigate the limitations inherent in creating MRPs, offering practical pathways to align technological capabilities with creative objectives.

Recent artistic explorations in VR environments have explored how users engage with virtual spaces, sounds, and bodies whether, as artists, coders, or spectators. While many applications focus primarily on musical interaction or motion capture, few allow users to actively compose spatial and temporal relationships within a multimodal collaborative creation context merging altogether different artistic modalities. In this project, we present a Virtual Reality application that enables novel forms of collaboration between dancers and musicians. The application allows dancers to record full-body performances, which musicians can then use as the basis for sonic composition by spatially mapping sound triggers onto the dancer’s recorded movement. Rather than relying on live capture or real-time gesture tracking, our approach treats movement as a timeline for interaction blending choreography with sound design in an asynchronous workflow. We employ an iterative design process to ensure usability among experts. This paper details the first implementation and study involving 10 participants recruited from professional and amateur artists with electronic music backgrounds, highlighting positive reception of the application’s creative potential and usability.

While 3D user interfaces are often designed with efficiency and accuracy in mind, artistic performances have their own very specific constraints and criteria for a successful interaction in mixed or virtual reality, which have yet to be fully understood. In this paper, we study the design of 3D interactions for a Spatial Augmented-Reality display in the context of a cyber-opera. We perform an analysis of design decisions taken during the multiple residencies and lab sessions, and we conduct a reflexive thematic analysis of interviews with the director and actors, which highlight how they integrated the story, the actors' performance, the audience experience and the technology. We identify four main criteria for designing 3D interactions in performing arts, namely their efficiency, expressiveness, contextualisation and legibility, and we derive guidelines for future research and creation.

When several people are co-located and immersed in a mixed reality environment, they may feel like they share the virtual environment or not. This feeling of copresence, along with its parent dimensions of social presence and presence, has been mostly studied by relying on subjective measures gathered through questionnaires. As a way to address the drawbacks of this approach, we introduce a protocol to gather behavioral measures in the context of co-located mixed reality. As a pair of participants avoid obstacles moving towards them, their errors, gaze, interpersonal distance, and timing are measured. By combining subjective measures gathered through a questionnaire drawing from previous studies on social presence with behavioral measures, we demonstrate new ways to assess how users experience copresence. We illustrate this protocol by evaluating the effect of visual feedback on collaborators' activity. The results of this experiment suggest the capability of our protocol by revealing the effect of visual feedback on both objective and subjective measures.

In this paper, we present a class of mixed-reality displays which allow for the 3D exploration of content in public exhibitions. The shared experience of the exhibition and the preservation of artworks are two very important aspects of these contexts, in particular for museum exhibits. The use of display cases as a protection tool is substantially accepted. It decreases the risks of damages to artworks and cultural materials hosted in museums. In addition, the transparent panels create a reflection of the visitors inside the display case. This reflection can be used to augment and interact in 3D with the exhibited content, by coupling Spatial Augmented-Reality and Optical Combiners. We call such a combination a Revealable Volume Display (RVD). It allows visitors to reveal information placed freely inside or around protected artefacts, visible by all, using their reflection in the panel. However, it may also suffer from unfamiliar gestures and disrupted depth perception cues, making 3D exploration of content difficult. In this paper, we first discuss the implementation of RVDs, providing both projector-based and mobile versions. We then present a design space that describes the interaction possibilities that it offers. Drawing on insights from a field study during a first exhibition, we finally propose and evaluate techniques for facilitating 3D exploration with RVDs.

When the collaboration between humans and machines happens in public, the audience can face difficulties in distinguishing the actual human contribution from the contribution of autonomous processes. In music concerts involving digital interfaces doubts about the performer's contribution can drastically hinder the audience interest. The disappearing of the direct physical link between actions and effects is one of the reasons of this confusion. Consequently both artists and researchers have explored techniques to augment the experience of spectators. However their respective impact on the multiple aspects of audience experience has not yet been formally compared. In this controlled study, we compare two techniques : pre-performance explanations and visual augmentations. Despite contradictory results on comprehension tasks, we show that contrary to pre-performance explanations, visual augmentations improve the audience experience, increase their subjective comprehension and restore the trust in performers by reversing the doubt in their favour.

Despite the development of touchscreens, many expert systems for working with digital multimedia content, such as in music composition and performance, video editing or visual performance, still rely on control surfaces. This can be due to the accuracy and appropriateness of their sensors, the haptic feedback that they offer, and most importantly the way they can be adapted to the specific subset of gestures and tasks that users need to perform. On the other hand, visual feedback on controllers remains limited and/or fixed, preventing similar personalizing. In this paper, we propose ControllAR, a novel system that facilitates the appropriation of rich visual feedback on control surfaces through remixing of graphical user interfaces and augmented reality display. We then use our system to study current and potential appropriation of visual feedback in the case of digital musical instruments and derive guidelines for designers and developers.

Despite the development of touchscreens, many expert systems for working with digital multimedia content, such as in music composition and performance, video editing or visual performance, still rely on control surfaces. This can be due to the accuracy and appropriateness of their sensors, the haptic feedback that they offer, and most importantly the way they can be adapted to the specific subset of gestures and tasks that users need to perform. On the other hand, visual feedback on controllers remains limited and/or fixed, preventing similar personalizing. Here we present ControllAR, a novel system that facilitates the appropriation of rich visual feedback on control surfaces through remixing of graphical user interfaces and augmented reality display.

Reflective optical combiners like beam splitters and two way mirrors are used in AR to overlap digital contents on the users' hands or bodies. Augmentations are usually unidirectional, either reflecting virtual contents on the user's body (Situated Augmented Reality) or augmenting user's reflections with digital contents (AR mirrors). But many other novel possibilities remain unexplored. For example, users' hands, reflected inside a museum AR cabinet, can allow visitors to interact with the artifacts exhibited. Projecting on the user's hands as their reflection cuts through the objects can be used to reveal objects' internals. Augmentations from both sides are blended by the combiner, so they are consistently seen by any number of users, independently of their location or, even, the side of the combiner through which they are looking. This paper explores the potential of optical combiners to merge the space in front and behind them. We present this design space, identify novel augmentations/interaction opportunities and explore the design space using three prototypes.