As immersive technologies advance, Mixed Reality Performances (MRP) increasingly integrate them but often face technical challenges that must balance artistic vision and practical constraints.These constraints sometimes lead to unavoidable limitations.The diversity of technologies and artistic goals in MRP prevents a onesize-fits-all solution to such dilemmas.This paper presents a model unifying strategies for addressing compromises between artistic intent and technological feasibility.Using reflexive thematic analysis of a performance-led case study, interviews, and independent case studies, we identify recurring strategies in Mixed Reality experiences with varying constraints.These strategies fall on an axis based on the audience's awareness of limitations and are categorized into five approaches: Avoid, Disguise, Tolerate, Integrate, or Leverage.We argue that this framework helps designers better navigate the limitations inherent in creating MRPs, offering practical pathways to align technological capabilities with creative objectives.

When several people are co-located and immersed in a mixed reality environment, they may feel like they share the virtual environment or not. This feeling of copresence, along with its parent dimensions of social presence and presence, has been mostly studied by relying on subjective measures gathered through questionnaires. As a way to address the drawbacks of this approach, we introduce a protocol to gather behavioral measures in the context of co-located mixed reality. As a pair of participants avoid obstacles moving towards them, their errors, gaze, interpersonal distance, and timing are measured. By combining subjective measures gathered through a questionnaire drawing from previous studies on social presence with behavioral measures, we demonstrate new ways to assess how users experience copresence. We illustrate this protocol by evaluating the effect of visual feedback on collaborators' activity. The results of this experiment suggest the capability of our protocol by revealing the effect of visual feedback on both objective and subjective measures.

Much progress has been made on interactive behavior development tools for expert programmers. However, little effort has been made in investigating how these tools support creative communities who typically struggle with technical development. This is the case, for instance, of media artists and composers working with interactive environments. To address this problem, we introduce ZenStates: a new specification model for creative interactive environments that combines Hierarchical Finite-States Machines, expressions, off-the-shelf components called Tasks, and a global communication system called the Blackboard. Our evaluation is three-folded: (a) implementing our model in a direct manipulation-based software interface; (b) probing ZenStates' expressive power through 90 exploratory scenarios; and (c) performing a user study to investigate the understandability of ZenStates' model. Results support ZenStates viability, its expressivity, and suggest that ZenStates is easier to understand-in terms of decision time and decision accuracy-compared to two popular alternatives.

Acquisition of instrumental gestures in musical performances is a field of increasing interest with applications in different areas ranging from acoustics and sound synthesis to motor learning or artistic performances. Direct acquisition approaches are based on measurements with sensors attached on the instrument or the performer, a process that usually involves the use of expensive sensing systems and complex setups that are generally intrusive in practice. An alternative is the indirect acquisition without sensors based on analysis of the audio signal. This paper reports a novel indirect acquisition method for the estimation of continuous violin controls from audio-signal analysis based on the training of statistical models with a database of previously recorded violin performances. The database contains synchronized streams of audio features and instrumental controls. Audio signal was captured with a vibration transducer built into the violin bridge, and instrumental controls were measured with sensors. The controls include bowing parameters (played string, bowing velocity, bowing force, and bowing distance to the bridge) as well as fingering position. Once the model is trained for a specific violin, we can perform indirect acquisition from analysis of the signal captured with its embedded transducer without the need for the sensors any more. The statistical methods used are Hidden Markov Models (HMM) with observation distributions parameterized as Multivariate Gaussian Mixtures (GM). HMMs provide a means for note recognition and following and parameter prediction is based on GM regression. Results show that the presented method constitutes an accurate, non-intrusive and low-cost alternative for instrumental control acquisition of a previously calibrated violin and recording device.

Force-feedback interaction with a bowed string model can suffer critically from noise in the velocity signal derived from differentiating position measurements. To address this problem, we present a model for bowed string interaction based on a position-constraint friction. We validate the proposed model by comparing to previous work using off-line simulations, and show measurements of interaction on haptic hardware. This noise-free excitation signal leads to cleaner string motion than previous models, thereby improving the quality of force and audio synthesis in the presence of noise.

This paper describes DIMPLE, a software application for haptic force-feedback controllers which allows easy creation of interactive rigid-body simulations. DIMPLE makes extensive use of an established standard for control-rate transmission of audio control commands, which can be used to drive many simultaneous parameters of a given audio/visual synthesis engine. Because it is used with a high-level, visual multimedia programming language, DIMPLE allows fast and uncomplicated development of responsive, haptically-enabled virtual environments useful for fast prototyping of applications in fields where lower level programming skills may not be widespread. Examples of specific scenes constructed using DIMPLE are given, with applications to perception, HCI research, music, and multimedia. A pilot evaluation study was performed comparing DIMPLE to another implementation of a specific scene, which showed comparable results between subjects’ overall impressions of the simulation.

This paper provides a review of gestural control of sound synthesis in the context of the design and evaluation of digital musical instruments. It discusses research in various areas related to this field and equally focuses on four main topics: analysis of music performers' gestures, gestural capture technologies, real-time sound synthesis methods, and strategies for mapping gesture variables to sound synthesis input parameters. Finally, this approach is illustrated by presenting an application of this research to the control of digital audio effects.

This article presents ESCHER, a sound synthesis environment based on Ircam's real-time audio environment jMax. ESCHER is a modular system providing synthesis-independent prototyping of gesturally-controlled instruments by means of parameter interpolation. The system divides into two components: gestural controller and synthesis engine. Mapping between components takes place on two independent levels, coupled by an intermediate abstract parameter layer. This separation allows a flexible choice of controllers and/or sound synthesis methods.