International audience

We study Congruent Indirect Touch (CIT) interaction in a desktop context. CIT only differs from direct touch in that the motor and display spaces are separated: touch occurs on the horizontal desk; while users’ visual focus is on a vertical display where small pointers show the position of the fingers. We introduce an accurate fingertip tracking approach based on optical tracking and fingertip modeling as a sphere. This allows updating the pointer when the finger is hovering above the surface and implementing an efficient CIT interaction. This interaction was evaluated in a longitudinal user study. Six participants with no experience with CIT performed target acquisitions on eight different days. Throughput was measured with CIT and mouse. In the last session, two participants had similar throughput with both interactions; the four others were notably more efficient with CIT. Averaged across all participants, throughput improved by 14%. This study promotes the study of CIT as a potential efficient replacement for the mouse on the desktop.

Virtual and Augmented Reality Environments have long been seen as having strong potential for educational applications. However, research showing actual evidences of their benefits is sparse. Indeed, some recent studies point to unnoticeable benefits, or even a detrimental effect due to an increase of cognitive demand for the students when using these environments. In this work, we question if a clear benefit of AR and VR can be robustly measured for a specific education-related task: learning a 3D object.

Handheld Perspective-Corrected Displays (HPCDs) are physical objects that have a notable volume and that display a virtual 3D scene on their entire surface. Being handheld, they create the illusion of holding the scene in a physical container (the display). This has strong benefits for the intuitiveness of 3D interaction: manipulating objects of the virtual scene amounts to physical manipulations of the display. HPCDs have been limited so far to technical demonstrators and experimental tools to assess their merits. However, they show great potential as interactive systems for actual 3D applications. This requires that novel interactions be created to go beyond object manipulation and to offer general-purpose services such as menu command selection and continuous parameter control. Working with a two-handed spherical HPCD, we report on the design and informal evaluations of various interaction techniques for distant object selection, scene scaling, menu interaction and continuous parameter control. In particular, our design leverages the efficient two-handed control of the rotations of the display. We demonstrate how some of these techniques can be assemble in a self-contained anatomy learning application. Novice participants used the application in a qualitative user experiment. Most participants used the application effortlessly without any training or explanations.

The feeling of presence of virtual entities is an important objective in virtual reality, teleconferencing, augmented reality, exposure therapy and video games. Presence creates emotional involvement and supports intuitive and efficient interactions. As a feeling, presence is mostly measured via subjective questionnaire, but its validity is disputed. We introduce a new method to measure the contribution of several technical parameters toward presence. Its robustness stems from asking participant to rank contrasts rather than asking absolute values, and from the statistical analysis of repeated answers. We implemented this method in a user study where virtual entities were created with a handheld perspective corrected display. We evaluated the impact on two virtual entities' presence of four important parameters of digital visual stimuli: resolution, latency, frame rate and jitter. Results suggest that jitter and frame rate are critical for presence but not latency, and resolution depends on the explored entity.

Handheld Perspective Corrected Displays (HPCDs) can create the feeling of holding a virtual 3D object. They offer a direct interaction that is isomorphic to the manipulation of physical objects. This illusion depends on the ability to provide a natural visuomotor coupling. High performances systems are thus required to evaluate the fundamental merits of HPCDs. We built a spherical HPCD using external projection. The system offers a lightweight wireless seamless display with head-coupled stereo, robust tracking, and low latency. We compared users' performances with this HPCD and two other interactions that used a fixed planar display and either a touchpad or the spherical display as an indirect input. The task involved the inspection of complex virtual 3D puzzles. Physical puzzles were also tested as references. Contrary to expectations, all virtual interactions were found to be more efficient than a more "natural" physical puzzle. The HPCD yielded lower performances than the touchpad. This study indicates that the object examination task did not benefit from the accurate and precise rotations offered by the HPCD, but benefited from the high C/D gain of the touchpad.

Touch latency has been shown to deteriorate users' performances at levels as low as 25 ms, but this was tested only in short experimental sessions. Real life usage of touchscreens covers much longer periods. It provides training which could lead to reduce the impact of latency.

Six degrees of freedom docking is one of the most fundamental tasks when interacting with 3D virtual worlds. We investigated docking performances with isomorphic interactions that directly relate the 6-dof pose of the input device to that of the object controlled. In particular, we studied a Handheld Perspective-Coupled Display (HPCD); which is a novel form of interactive system where the display itself is handheld and used as the input device. It was compared to an opaque HMD and to a standard indirect flat display used with either a sphere or an articulated arm as the input device. A novel computation of an Index of Difficulty was introduced to measure the efficiency of each interaction. We observed superior performances with the HPCD compared with the other interactions by a large margin (17% better than the closest interaction).

Touch latency has been shown to reduce users' performances but most studies focus on one-handed elementary tasks such as pointing or tracking a single object. The everyday use of touch devices is made, however, of more complex "composite" tasks combining several objects with potential bimanual interaction. Such a composite task may increase users' cognitive load which makes latency less perceivable. We thus expected that the impact of latency on users' performances should be smaller in composite tasks than in elementary tasks. We tested this hypothesis by comparing the degradation effect of latency on users' performances in an elementary vs. a composite task. The elementary task consisted in positioning a single object. The composite task involved sorting and positioning objects with a two-handed interaction, inducing more complex planning and motor strategies that could be seen as an additional cognitive load. Contrary to expectations, the degradation effect was comparable in the two tasks. This study indicates that the substantial hindrance of latency, demonstrated on elementary tasks, also exists in more complex tasks that better represent the every day use of touch devices. This strengthens the motivation to question the interaction between the task properties and latency effect and to adapt commercial devices and applications accordingly.

Human-computer interaction should be natural. However, the notion of natural is questioned due to a lack of theoretical background and methods to objectively measure the naturalness of a HCI. A frequently cited aspect of natural HCIs is their ability to benefit from knowledge and skills that users develop in their interaction with the real (non-digital) world. Among these skills, sensory-motor abilities are essential to operate many HCIs. This suggests that the transfer of these abilities between physical and digital interactions could be used as an experimental tool to assess the sensory-motor similarity between interactions, and could be considered as an objective measurement of the sensory-motor grounding of naturalness. In this framework, we introduce a new experimental paradigm inspired by motor learning research to assess sensory-motor similarity, as revealed by the transfer of learning. We tested this paradigm in an empirical study to question the naturalness of three HCIs: direct-touch, mouse pointing and absolute indirect-touch. The study revealed how skill learning transfers from these three digital interactions towards an equivalent physical interaction. We observed strong transfer of skill between direct-touch and physical interaction, but no transfer from the other two interactions. This work provides a first objective assessment of the sensory-motor basis of direct-touch naturalness, and a new empirical path to question HCI similarity and naturalness.

Now that 3D interaction is available on tablets and smart phones, it becomes critical to provide efficient 3D interaction techniques for novice users. This paper investigates interaction techniques for 3D rotation with two fingers of a single hand, on multitouch mobile devices.

Despite great promise of vision-based user interfaces, commercial employment of such systems remains marginal. Most vision-based interactive systems are one-time, "proof of concept" prototypes that demonstrate the interest of a particular image treatment applied to interaction. In general, vision systems require parameter tuning, both during setup and at runtime, and are thus difficult to handle by nonexperts in computer vision. In this paper, we present a pragmatic, developer-centric, service-oriented framework for the construction of vision-based interactive systems. Our framework is designed to allow developers unfamiliar with vision to use computer vision as an interaction modality. To achieve this goal, we address specific developer- and interaction-centric requirements during the design of our system. We validate our approach with an implementation of standard GUI widgets (buttons and sliders) based on computer vision.

Visual tracking of bare fingers allows more direct manipulation of digital objects, multiple simultaneous users interacting with their two hands, and permits the interaction on large surfaces, using only commodity hardware. After presenting related work, we detail our implementation. Its design is based on our modeling of two classes of algorithms that are key to the tracker: Image Differencing Segmentation (IDS) and Fast Rejection Filters (FRF). We introduce a new chromatic distance for IDS and a FRF that is independent to finger rotation. The system runs at full frame rate (25 Hz) with an average total system latency of 80 ms, independently of the number of tracked fingers. When used in a controlled environment such as a meeting room, its robustness is satisfying for everyday use.

Visual processes to detect and track faces for video compression and transmission. The system is based on an architecture in which a supervisor selects and activates visual processes in cyclic manner. Control of visual processes is made possible by a confidence factor which accompanies each observation. Fusion of results into a unified estimation for tracking is made possible by estimating a covariance matrix with each observation. Visual processes for face tracking are described using blink detection, normalised color histogram matching, and cross correlation (SSD and NCC). Ensembles of visual processes are organised into processing states so as to provide robust tracking. Transition between states is determined by events detected by processes. The result of face detection is fed into recursive estimator (Kalman filter). The output from the estimator drives a PD controller for a pan/tilt/zoom camera. The resulting system provides robust and precise tracking which operates continuously at approximately 20 images per second on a 150 megahertz computer workstation.

This paper describes an active-camera real-time system for tracking, shape description, and classification of the human face and mouth using only an SGI Indy computer. The system is based on use of 2-D blob features, which are spatially-compact clusters of pixels that are similar in terms of low-level image properties. Patterns of behavior (e.g., facial expressions and head movements) can be classified in real-time using Hidden Markov Model (HMM) methods. The system has been tested on hundreds of users and has demonstrated extremely reliable and accurate performance. Typical classification accuracies are near 100%.

In Computer Mediated Communication such as desktop video conferencing, static video cameras provide a restricted field of view of remote sites. The effective field of view can be enlarged, while maintaining the user's freedom of movement, by slaving a remote controlled camera to movements of the user's head. This paper concerns techniques for tracking of faces. We demonstrate that robustness and reliability can be increased by combining multiple perceptual processes such as eye blink detection, skin color histogram and cross correlation, that adapt to a variety of operating conditions. We illustrate our technique with CoMedi, a media-space currently under development.