As immersive technologies advance, Mixed Reality Performances (MRP) increasingly integrate them but often face technical challenges that must balance artistic vision and practical constraints.These constraints sometimes lead to unavoidable limitations.The diversity of technologies and artistic goals in MRP prevents a onesize-fits-all solution to such dilemmas.This paper presents a model unifying strategies for addressing compromises between artistic intent and technological feasibility.Using reflexive thematic analysis of a performance-led case study, interviews, and independent case studies, we identify recurring strategies in Mixed Reality experiences with varying constraints.These strategies fall on an axis based on the audience's awareness of limitations and are categorized into five approaches: Avoid, Disguise, Tolerate, Integrate, or Leverage.We argue that this framework helps designers better navigate the limitations inherent in creating MRPs, offering practical pathways to align technological capabilities with creative objectives.

International audience

In this paper, we compare eyes-free input gestures to gestures produced in the presence or the absence of a visual feedback for a single-handed mobile device interaction. We consider two gesture sets ((i) freeform gestures (operands, rationally invariant and mnemonic gestures) and (ii) mark-based gestures (on-axis rectilinear segments)) and two gesture beginnings (free: gesture can start from anywhere on the screen and bezel: gesture should start from one of the edge). Our findings indicate that gestures made in an eyes-free interaction were geometrically different from gestures generated in the presence or absence of visual feedback; they generated more directional movements around the y axis and had a lower recognition rate, which in itself was quite good (95.53%). Our findings also indicate that bezel gestures generated more phone movement and were less appreciated than those initialized freely. Freeform gestures were also preferred over mark ones. Based on those results, we propose five guidelines for eyes-free stroke gesture design, ergonomics, and recognizer development.

Touch knobs usually require visual attention when performing a targeting task. This can draw the user’s visual focus to the device, which is not desirable if the visual attention is needed elsewhere. In this work, we investigated the use of haptic feedback for touch knobs when performing another primary task that saturates the attention. In an experiment, we compare three potential tactile feedback designs to a no-feedback variant for a circular targeting task, while performing an attention saturating primary task. Our findings indicate that two investigated designs, namely, gradual+hapticObject and adaptive+hapticObject, providing the user with both haptic feedback on the objects and continuous haptic feedback through the trajectory, can serve as a reliable and effective mechanism for increasing the speed and reducing the mental demand for the secondary task while reducing visual distraction for the primary task.

In this paper, we present a class of mixed-reality displays which allow for the 3D exploration of content in public exhibitions. The shared experience of the exhibition and the preservation of artworks are two very important aspects of these contexts, in particular for museum exhibits. The use of display cases as a protection tool is substantially accepted. It decreases the risks of damages to artworks and cultural materials hosted in museums. In addition, the transparent panels create a reflection of the visitors inside the display case. This reflection can be used to augment and interact in 3D with the exhibited content, by coupling Spatial Augmented-Reality and Optical Combiners. We call such a combination a Revealable Volume Display (RVD). It allows visitors to reveal information placed freely inside or around protected artefacts, visible by all, using their reflection in the panel. However, it may also suffer from unfamiliar gestures and disrupted depth perception cues, making 3D exploration of content difficult. In this paper, we first discuss the implementation of RVDs, providing both projector-based and mobile versions. We then present a design space that describes the interaction possibilities that it offers. Drawing on insights from a field study during a first exhibition, we finally propose and evaluate techniques for facilitating 3D exploration with RVDs.

Alongside vision and sound, hardware systems can be readily designed to support various forms of tactile feedback; however, while a significant body of work has explored enriching visual and auditory communication with interactive systems, tactile information has not received the same level of attention. In this work, we explore increasing the expressivity of tactile feedback by allowing the user to dynamically select between several channels of tactile feedback using variations in finger speed. In a controlled experiment, we show that a user can learn the dynamics of eyes-free tactile channel selection among different channels, and can reliable discriminate between different tactile patterns during multi-channel selection with an accuracy up to 90% when using two finger speed levels. We discuss the implications of this work for richer, more interactive tactile interfaces.

When the collaboration between humans and machines happens in public, the audience can face difficulties in distinguishing the actual human contribution from the contribution of autonomous processes. In music concerts involving digital interfaces doubts about the performer's contribution can drastically hinder the audience interest. The disappearing of the direct physical link between actions and effects is one of the reasons of this confusion. Consequently both artists and researchers have explored techniques to augment the experience of spectators. However their respective impact on the multiple aspects of audience experience has not yet been formally compared. In this controlled study, we compare two techniques : pre-performance explanations and visual augmentations. Despite contradictory results on comprehension tasks, we show that contrary to pre-performance explanations, visual augmentations improve the audience experience, increase their subjective comprehension and restore the trust in performers by reversing the doubt in their favour.

Previous research demonstrated the ability for users to accurately recognize tactile textures on mobile surface. However, the experiments were only run in a lab setting and the ability for users to recognize tactile texture in a real-world environment remains unclear. In this paper, we investigate the effects of physical challenging activities on tactile textures recognition. We consider five conditions: (1) seated on an office, (2) standing in an office, (3) seated in the tramway, (4) standing in the tramway and (5) walking in the street. Our findings indicate that when walking, performances deteriorated compared to the remainder conditions. However, despite this deterioration, the recognition rate stay higher than 82% suggesting that tactile texture could be effectively recognized and used by users in different physical challenges activities including walking.

Relative pointing through using tactile mobile device (such as tablets of phones) on a large display is a viable interaction technique (that we call Pad in this paper) which permits accurate pointing. However, limited device size has consequences on interaction. Such systems are known to often require clutching, which degrades performances. We present E-Pad, an indirect relative pointing interaction technique which takes benefit of the mobile tactile surface combined with its surrounding space. A user can perform continuous relative pointing starting on the pad then continuing in the free space around the pad, within arm's reach. As a first step toward E-Pad, we first introduce extended continuous relative pointing gestures and conduct a preliminary study to determine how people move their hand around the mobile device. We then conduct an experiment that compares the performance of E-Pad and Pad. Our findings indicate that E-Pad is faster than Pad and decreases the number of clutches without compromising accuracy. Our findings also suggest an overwhelming preference for E-Pad.

Many touch based interactions provide limited opportunities for direct tactile feedback; examples include multi-user touch displays, augmented reality based projections on passive surfaces, and mid-air input. In this paper, we consider distal feedback, through vibrotactile stimulation on a smart-watch placed on the user's non-dominant wrist, as an alternative feedback mechanism to interaction location vibrotactile feedback, under the user's finger. We compare the effectiveness of interaction location feedback vs. distal feedback through a Fitts's Law task completed on a smartphone. Results show that distal and interaction location feedback both reduce errors in target acquisition and exhibit statistically comparable performance, suggesting that distal vibrotactile feedback is a suitable alternative when interaction location feedback is not readily available.

In mobile touchscreebn interaction, an important challenge is to find solutions to balance the size of individual widgets against the number of widgets needed during interaction. In this work, to address display space limitations, we explore the design of invisible off-screen toolbars (ether-toolbars) that leverage computer vision to expand application features by placing widgets adjacent to the display screen. We show how simple computer vision algorithms can be combined with a natural human ability to estimate physical placing to support highly accurate targeting. Our ether-toolbar design promises targeting accuracy approximating on-screen widget accuracy while significantly expanding the interaction space of mobile devices. Through two experiments, we examine off-screen content placement metaphors and off-screen precision of participants accessing these toolbars. From the data of the second experiment, we provide a basic model that reflects how users perceive mobile surroundings for ether-widgets and validate it. We also demonstrate a prototype system consisting of an inexpensive 3D printed mount for a mirror that supports ether-toolbar implementations. Finally, we discuss the implications of our work and potential design extensions that can increase the usability and the utility of ether-toolbars.

We investigate the relevance of surface haptic rendering techniques for tactile devices. We focus on the two major existing techniques and show that they have complementary benefits. The first one, called textsc{S}urface textsc{H}aptic textsc{O}bject (textsc{SHO}), which is based on finger position, is shown to be more suitable to render sparse textures; while the second one, called textsc{S}urface textsc{H}aptic textsc{T}exture (textsc{SHT}), which is based on finger velocity, is shown to be more suitable for dense textures and fast finger movements. We hence propose a new rendering technique, called textsc{L}ocalized textsc{H}aptic textsc{T}exture (textsc{LHT}), which is based on the concept of textit{taxel} considered as an elementary tactile information that is rendered on the screen. By using a grid of taxels to encode a texture, textsc{LHT} is shown to provide a consistent tactile rendering across different velocities for high density textures, and is found to reduce user textit{error rate} by up to 77.68% compared to textsc{SHO}.

We study users' perception of simultaneous tactile textures in ultrasonic devices. We investigate how relevant is providing the user with different complementary and simultaneous textures with respect to the different fingers that can be used to touch the surface. We show through a controlled experiment that users are able to distinguish the number of different textures independently of using fingers from one or two hands. However, our findings indicate that users are not able to differentiate between two different textures, that is to correctly identify each of them, when using fingers from the same hand. Based on our findings, we are then able to outline three relevant guidelines to assist multi-finger tactile feedback ergonomic and devices design.

Haptic rendering of soft bodies is essential in medical simulations of procedures such as surgery or palpation. The most commonly used approach is to recreate the sense of touch using a specific design and control of a robotic arm. In this paper, we propose a new approach, based on soft-robotics technology. We create a tangible deformable device that allows users to “touch” soft tissues and perceive mechanical material properties, in a realistic manner. The device is able to dynamically provide user touch with different stiffness perceptions, thanks to actuators placed at the boundaries. We introduce a control algorithm, based on inverse Finite Element Analysis, which controls the actuators in order to recreate a desired stiffness that corresponds to the contact with soft tissues in the virtual environment. The approach uses antagonistic actuation principle to create a wide range of stiffness. We validate our algorithm and demonstrate the method using prototypes based on simple mechanisms.

The CMIS workshop proposes to bring together researchers who are interested in improving collaborative experiences through the combination of multiple interaction surfaces with diverse sizes and formats, ranging from large-scale walls, to tables, mobiles, and wearables. The opportunities for innovation exist, but the ISS, CHI, CSCW, and other HCI communities have not yet thoroughly addressed the problem of bringing effective collaboration activities together using multiple interactive surfaces, especially in complex work domains. Of particular interest is the potential synergy that one can obtain by effectively combining different-sized surfaces and sharing information between devices.

Tactile devices with ultrasonic vibrations (based on squeeze film effect) using piezoelectric actuators are one of the existing haptic feedback technologies. In this study we have performed two psychophysical experiments on an ultrasonic haptic tablet, in order to find the minimum size of a tactile element on which all the users are able to perfectly identify different types of textures. Our results show that the spatial resolution of the tactile element on haptic touchscreen actually varies, depending on the number and types of tactile feedback information. A first experiment exhibits three different tactile textures, chosen as being easily recognized by users. We use these textures in a second experiment, and evaluate minimal spatial area on which the chosen set of textures can be recognized. Among other, we find the minimal size depends on the texture nature.

We present an interaction technique combining tactile actions and midair pointing to access out-of-reach content on large displays without the need to walk across the display. Users can start through a touch gesture on the display surface and finish midair by pointing to push content away or inversely to retrieve a content. The technique takes advantage of well-known semantics of pointing in human-to-human interaction. These, coupled with the semantics of proximal relations and deictic proxemics make the proposed technique very powerful as it leverages on well-understood human-human interaction modalities. Experimental results show this technique to outperform direct tactile interaction on dragging tasks. From our experience we derive four guidelines for interaction with large-scale displays.

We present a simple, two-step technique for recognizing multi-touch gesture input that is invariant to how users articulate gestures, i.e., by using one or two hands, one or multiple fingers, one or multiple strokes, synchronous or asynchronous stroke input. We introduce, for the first time in the gesture literature, a preprocessing step that is specific to multi-touch gestures (Match-Up) that clusters together similar strokes produced by different fingers, before running a gesture recognizer (Conquer). We report gains in recognition accuracy up to 10% leveraged by our new preprocessing step, which manages to construct a more adequate representation of multi-touch gestures in terms of key strokes. It is our hope that the Match-Up technique will add to the practitioners' toolkit of gesture preprocessing techniques, as a first step toward filling today's lack of algorithmic knowledge to process multi-touch input and leading toward the design of more efficient and accurate recognizers for touch surfaces.

We show that users are consistent in their assessments of the articulation difficulty of multi-touch gestures, even under the many degrees of freedom afforded by multi-touch input, such as (1) various number of fingers touching the surface, (2) various number of strokes that structure the gesture shape, and (3) single-handed and bimanual input. To understand more about perceived difficulty, we characterize gesture articulations captured under these conditions with geometric and kinematic descriptors computed on a dataset of 7,200 samples of 30 distinct gesture types collected from 18 participants. We correlate the values of the objective descriptors with users' subjective assessments of articulation difficulty and report path length, production time, and gesture size as the highest correlators (max Pearson's r=.95). We also report new findings about multi-touch gesture input, e.g., gestures produced with more fingers are larger in size and take more time to produce than single-touch gestures; bimanual articulations are not only faster than single-handed input, but they are also longer in path length, present more strokes, and result in gesture shapes that are deformed horizontally by 35% in average. We use our findings to outline a number of 14 guidelines to assist multi-touch gesture set design, recognizer development, and inform gesture-to-function mappings through the prism of the user-perceived difficulty of gesture articulation.

Pervasive computing is a vision that has been an inspiring long-term target for many years now. Interaction techniques that allow one user to efficiently control many screens, or that allow several users to collaborate on one distant screen, are still hot topics, and are often considered as two different questions. Standard approaches require a strong coupling between the physical location of input device, and users. We propose to consider these two questions through the same basic concept, that uncouples physical location and user input, using a mid-air approach. We present the concept of mimetic interaction spaces (MIS), a dynamic user-definition of an imaginary input space thanks to an iconic gesture, that can be used to define mid-air interaction techniques. We describe a participative design user-study, that shows this technique has interesting acceptability and elicit some definition and deletion gestures. We finally describe a design space for MIS-based interaction, and show how such concept may be used for multi-screen control, as well as screen sharing in pervasive environments.

We show that large consensus exists among users in the way they articulate stroke gestures at various scales (i.e., small, medium, and large), and formulate a simple rule that estimates the user-intended scale of input gestures with 87% accuracy. Our estimator can enhance current gestural interfaces by leveraging scale as a natural parameter for gesture input, reflective of user perception (i.e., no training required). Gesture scale can simplify gesture set design, improve gesture-to-function mappings, and reduce the need for users to learn and for recognizers to discriminate unnecessary symbols.

Multitouch displays represent a promising technology for the display and manipulation of data. While the manipulation of 2D data has been widely explored, 3D manipulation with multitouch displays remains largely unexplored. Based on an analysis of the integration and separation of degrees of freedom, we propose a taxonomy for 3D manipulation techniques with multitouch displays. Using that taxonomy, we introduce Depth-Separated Screen-Space (DS3), a new 3D manipulation technique based on the separation of translation and rotation. In a controlled experiment, we compared DS3 with Sticky Tools and Screen-Space. Results show that separating the control of translation and rotation significantly affects performance for 3D manipulation, with DS3 performing faster than the two other techniques.

Clicking is a key feature any interaction input system needs to provide. In the case of 3D input devices, such a feature is often difficult to provide (e.g. vision-based, or tracking systems for free-hand interaction do not natively provide any button). In this work, we show that it is actually possible to build an application that provides two classical interaction tasks (selection, and pick-release), without any button-like feature. Our method is based on trajectory and kinematic gesture analysis. In a preliminary study we exhibit the principle of the method. Then, we detail an algorithm to discriminate selection, pick and release tasks using kinematic criteria. We present a controlled experiment that validates our method with an average success rate equal to 90.1% across all conditions.

Multi-touch displays represent a promising technology for the display and manipulation of data. While the manipulation of 2D data has been widely explored, 3D manipulation with multi-touch displays remains largely uncovered. Based on an analysis of the integration and separation of degrees of freedom, we propose a taxonomy for 3D manipulation techniques with multi-touch displays. Using that taxonomy, we introduce DS3 (Depth-Separated Screen Space), a new 3D manipulation technique based on the separation of translation and rotation. In a controlled experiment, we compare DS3 with Sticky Tools and Screen-Space. Results show that separating the control of translation and rotation significantly affects performance for 3D manipulation, with DS3 being at least 22% faster.

Multi-touch displays represent a promising technology for the display and manipulation of 3D data. To fully exploit their capabilities, appropriate interaction techniques must be designed. In this paper, we explore the design of free 3D positioning techniques for multi-touch displays to exploit the additional degrees of freedom provided by this technology. We present a first interaction technique to extend the standard four viewports technique found in commercial CAD applications and a second technique designed to allow free 3D positioning with a single view of the scene.

In this work, we propose interactive and physically based animation of one-dimensional deformable models using geometrically exact energy formulation. The proposed mechanical model has a high level of accuracy: it is based on continuous spline support and continuum mechanics media equations. We also detail a new efficient solving scheme, that can automatically switch between dynamic and static during simulation. With this scheme, we want to raise inconsistencies that could show up when human user is interacting with a physical simulation. We finally present a practical example in which the proposed model provides high-quality interaction.

Abstract This article presents a model for handling multi‐resolution animation of complex deformable models. An octree‐based animation and algorithmic tools are introduced to provide real‐time interaction on complex objects. Some efficient collision detection scheme is also described for deformable models. A method is also presented for time‐guaranteed mechanical simulation: It allows an automatic tuning of the octree resolution to match some external performance constraints, such as a pre‐defined CPU consumption limit. Copyright © 2005 John Wiley & Sons, Ltd.

This research work is aimed toward the development of a VR-based trainer for colon cancer removal. It enables the surgeons to interactively view and manipulate the concerned virtual organs as during a real surgery. First, we present a method for animating the small intestine and the mesentery (the tissue that connects it to the main vessels) in real-time, thus enabling user interaction through virtual surgical tools during the simulation. We present a stochastic approach for fast collision detection in highly deformable, self-colliding objects. A simple and efficient response to collisions is also introduced in order to reduce the overall animation complexity. Second, we describe a new method based on generalized cylinders for fast rendering of the intestine. An efficient curvature detection method, along with an adaptive sampling algorithm, is presented. This approach, while providing improved tessellation without the classical self-intersection problem, also allows for high-performance rendering thanks to the new 3D skinning feature available in recent GPUs. The rendering algorithm is also designed to ensure a guaranteed frame rate. Finally, we present the quantitative results of the simulations and describe the qualitative feedback obtained from the surgeons.

This paper studies the problem of real-time rendering of highly deformable generalized cylinders. Some efficient schemes for high axis curvature detection are presented, as well as an incremental non-uniform sampling process. We also show how the recent 3D card "skinning" feature, classical in character animation, can be derived in order to allow for very high frame-rate when rendering such generalized cylinders. Finally, an algorithm is presented, that permits the object to dynamically adapt its display process, for guaranteed frame-rate purposes. This algorithm dynamically modifies the different sampling parameters in order to achieve optimal quality visualization for a given pre-imposed frame-rate.

This paper proposes to study a spline model, called HB‐splines, that is in fact a B‐spline representation of Hermite splines, combined with some restriction on the differential values at segment boundaries. Although this model does not appear able to offer something new to the computer graphics community, we think that HB‐splines deserve to be considered for themselves because they embed many interesting features. First, they include all the classical properties required in a geometric modeling environment (convex hull, local control, arbitrary orders of parametric or geometric continuity). Second, they have a nice aptitude for direct manipulation (i.e. manipulation without using control points). For this purpose, we propose a new graphic widget, called control sails, that offers the user an intuitive way to specify local properties (position, tangent, curvature) of a curve or a surface. Finally, they provide an elegant formulation of a biorthogonal wavelet family, that permits multiresolution manipulations of the resulting curves or surfaces, in a very efficient way.