The seamless transition of control between drivers and autonomous systems remains a critical challenge in automated driving, affecting both safety outcomes and overall user experience. To address this challenge, our study examines the effectiveness of two distinct haptic feedback approaches—pneumatic and vibrotactile—when implemented as intelligent interface components for takeover requests (TORs) during these transition periods. We specifically investigate how these haptic modalities can effectively signal drivers when human intervention is required, facilitating smoother control transitions from automated to manual driving. We designed a comprehensive experimental setup integrating these haptic modalities with audio and visual cues and evaluated their performance across nine interaction tasks to understand how multi-modal feedback influences driver responsiveness during takeover scenarios. Our findings reveal that multi-modal approaches incorporating either pneumatic or vibrotactile feedback, combined with standard visual cues, substantially outperform audio-only alerts in both response time and accuracy metrics for takeover requests (TORs). Notably, pneumatic feedback offered more natural sensation and smoother transitions than vibrotactile feedback, with pneumatic systems excelling in comfort while vibrotactile feedback better serves urgent takeovers. This first systematic comparison provides valuable insights for developing interfaces that balance effectiveness with comfort in human-machine systems.

Source confusion occurs when individuals attribute a memory to the wrong source (e.g., confusing a picture with an experienced event). Virtual Reality (VR) represents a new source of memories particularly prone to being confused with reality. While previous research identified causes of source confusion between reality and other sources (e.g., imagination, pictures), there is currently no understanding of what characteristics specific to VR (e.g., immersion, presence) could influence source confusion. Through a laboratory study (n=29), we 1) confirm the existence of VR source confusion with current technology, and 2) present a quantitative and qualitative exploration of factors influencing VR source confusion. Building on the Source Monitoring Framework, we identify VR characteristics and assumptions about VR capabilities (e.g., poor rendering) that are used to distinguish virtual from real memories. From these insights, we reflect on how the increasing realism of VR could leave users vulnerable to memory errors and perceptual manipulations.

Virtual Reality (VR) technologies become ubiquitous, allowing people to employ immersive experiences in their homes. Since VR participants are visually disconnected from their real-world environment, commercial products propose safety boundaries to prevent colliding with their surroundings. However, there is a lack of empirical knowledge on how people perceive and interact with safety boundaries in everyday VR usage. This paper investigates this research gap with two mixed-method empirical studies. Study 1 reports an online survey (n=48) collecting data about attitudes towards safety boundaries, behavior while interacting with them, and reasons for breakout. Our analysis with open coding reveals that some VR participants ignored safety boundaries intentionally, even breaking out of them and continuing their actions. Study 2 investigates how and why VR participants intentionally break out when interacting close to safety boundaries and obstacles by replicating breakouts in a lab study (n=12). Our interview and breakout data discover three strategies, revealing VR participants sometimes break out of boundaries based on their real-world spatial information. Finally, we discuss improving future VR safety mechanisms by supporting participants’ real-world spatial mental models using landmarks.

Whole-body movements enhance the presence and enjoyment of Virtual Reality (VR) experiences. However, using large gestures is often uncomfortable and impossible in confined spaces (e.g., public transport). We introduce FingerMapper, mapping small-scale finger motions onto virtual arms and hands to enable whole-body virtual movements in VR. In a first target selection study (n=13) comparing FingerMapper to hand tracking and ray-casting, we found that FingerMapper can significantly reduce physical motions and fatigue while having a similar degree of precision. In a consecutive study (n=13), we compared FingerMapper to hand tracking inside a confined space (the front passenger seat of a car). The results showed participants had significantly higher perceived safety and fewer collisions with FingerMapper while preserving a similar degree of presence and enjoyment as hand tracking. Finally, we present three example applications demonstrating how FingerMapper could be applied for locomotion and interaction for VR in confined spaces.

Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR.

Smartphones’ screens are touch-sensitive and offer rich visual output capabilities, but do not allow users to control a wide range of pressure input nor provide rich pressure sensations. We propose In-Flat, an input/output pressure-sensitive overlay for smartphones. In-Flat consists of a transparent inflatable skin-like silicon layer that can be placed on the top or the back of a smartphone. As an output device, In-Flat offers tangible affordances and dynamic pressure feedback coupled with visual display. As an input device, In-Flat enables users to continuously perform a wide range of input gestures, notably press and pinch-and-pull gestures. Thus, In-Flat can be used to finely manipulate visual objects in mobile interaction or mediate interpersonal touch communications. In contrast to previous studies that mostly focus on press, we investigated the performance of pinch-and-pull and compared it with press. Our experiment (N=12) showed that participants could perform pinch-and-pull (83.8%) as well as press (84.7%), but felt having more control when performing pinch-and-pull. We explored the use of In-Flat to enable multimodal interaction that couples visual display and touch input/output. Participants appreciated this coupling as well as the touch sensation supplied by the In-Flat device.

“Virtual-Physical Perceptual Manipulations” (VPPMs) such as redirected walking and haptics expand the user’s capacity to interact with Virtual Reality (VR) beyond what would ordinarily physically be possible. VPPMs leverage knowledge of the limits of human perception to effect changes in the user’s physical movements, becoming able to (perceptibly and imperceptibly) nudge their physical actions to enhance interactivity in VR. We explore the risks posed by the malicious use of VPPMs. First, we define, conceptualize and demonstrate the existence of VPPMs. Next, using speculative design workshops, we explore and characterize the threats/risks posed, proposing mitigations and preventative recommendations against the malicious use of VPPMs. Finally, we implement two sample applications to demonstrate how existing VPPMs could be trivially subverted to create the potential for physical harm. This paper aims to raise awareness that the current way we apply and publish VPPMs can lead to malicious exploits of our perceptual vulnerabilities.

Humans have the ability to convey an array of emotions through complex and rich touch gestures. However, it is not clear how these touch gestures can be reproduced through interactive systems and devices in a remote mediated communication context. In this article, we explore the design space of device-initiated touch for conveying emotions with an interactive system reproducing a collection of human touch characteristics. For this purpose, we control a robotic arm to touch the forearm of participants with different force, velocity and amplitude characteristics to simulate human touch. In view of adding touch as an emotional modality in human-machine interaction, we have conducted two studies. After designing the touch device, we explore touch in a context-free setup and then in a controlled context defined by textual scenarios and emotional facial expressions of a virtual agent. Our results suggest that certain combinations of touch characteristics are associated with the perception of different degrees of valence and of arousal. Moreover, in the case of non-congruent mixed signals (touch, facial expression, textual scenario) not conveying a priori the same emotion, the message conveyed by touch seems to prevail over the ones displayed by the visual and textual signals.

Social distancing may force people to restrict social touch practices. Our survey (N=136) highlighted substantial social touch breakdowns during the COVID-19 pandemic for semi-intimate relationships (e.g., friends, colleagues), with handshakes being the most reduced, and frustrations at having to re-establish social touch habits. We then designed SansTouch, a multi-modal hand sleeve used together along with a smartphone to enable mediated hand-to-hand interactions such as handshakes or holding hands. To invoke the mediated touch, users synchronously mimic the hand position as in real life while holding SansTouch. Users can feel the touch sensation in real time without touching. Participants from our observational study (N=12) quickly adopted the hand-to-hand interactions of SansTouch for exchanging greetings face-to-face with colleagues and reported stronger preferences towards using SansTouch as opposed to mid-air gestures (e.g., waving). We discuss design implications, including the trade-offs of multi-modality for touch devices in face-to-face communication.

Supporting many gestures on small surfaces allows users to interact remotely with complex environments such as smart homes, large remote displays, or virtual reality environments, and switching between them (e.g., AR setup in a smart home). Providing eyes-free gestures in these contexts is important as this avoids disrupting the user's visual attention. However, very few techniques enable large sets of commands on small wearable devices supporting the user's mobility and even less provide eyes-free interaction. We present Side-Crossing Menus (SCM), a gestural technique enabling large sets of gestures on %small interactive surfaces like a smartwatch. Contrary to most gestural techniques, SCM relies on broad and shallow menus that favor small and rapid gestures. We demonstrate with a first experiment that users can efficiently perform these gestures eyes-free aided with tactile cues; 95% accuracy after training 20 minutes on a representative set of 30 gestures among 172. In a second experiment, we focus on the learning of SCM gestures and do not observe significant differences with conventional Multi-stroke Marking Menus in gesture accuracy and recall rate. As both techniques utilize contrasting menu structures, our results indicate that SCM is a compelling alternative for enhancing the input capabilities of small surfaces.

As one of the most important non-verbal communication channel, touch plays an essential role in interpersonal affective communication. Although some researchers have started exploring the possibility of using wearable devices for conveying emotional information, most of the existing devices still lack the capability to support affective and dynamic touch in interaction. In this paper, we explore the effect of dynamic visual cues on the emotional perception of vibrotactile signals. For this purpose, we developed VisualTouch, a haptic sleeve consisting of a haptic layer and a visual layer. We hypothesized that visual cues would enhance the interpretation of tactile cues when both types of cues are congruent. We first carried out an experiment and selected 4 stimuli producing substantially different responses. Based on that, a second experiment was conducted with 12 participants rating the valence and arousal of 36 stimuli using SAM scales.

We propose a paradigm called Skin-On interfaces, in which interactive devices have their own (artificial) skin, thus enabling new forms of input gestures for end-users (e.g. twist, scratch). Our work explores the design space of Skin-On interfaces by following a bio-driven approach: (1) From a sensory point of view, we study how to reproduce the look and feel of the human skin through three user studies;(2) From a gestural point of view, we explore how gestures naturally performed on skin can be transposed to Skin-On interfaces; (3) From a technical point of view, we explore and discuss different ways of fabricating interfaces that mimic human skin sensitivity and can recognize the gestures observed in the previous study; (4) We assemble the insights of our three exploratory facets to implement a series of Skin-On interfaces and we also contribute by providing a toolkit that enables easy reproduction and fabrication.

Previous studies have shown that spatial memory and semantic aids can help users learn and remember gestural commands. Using the body as a support to combine both dimensions has therefore been proposed, but no formal evaluations have yet been reported. In this paper, we compare an on-body interaction technique (BodyLoci) to mid-air Marking menus in a virtual reality context. We consider three levels of semantic aids: no aid, story-making, and story-making with background images. Our results show important improvement when story-making is used, especially for Marking menus (28.5% better retention). Both techniques performed similarly without semantic aids, but Marking menus outperformed BodyLoci when using them (17.3% better retention). While our study does not show a benefit in using body support, it suggests that inducing users to leverage simple learning techniques, such as story-making, can substantially improve recall, and thus make it easier to master gestural techniques. We also analyze the strategies used by the participants for creating mnemonics to provide guidelines for future work.

Various techniques have been proposed to faster command selection. Many of them either rely on directional gestures (e.g. Marking menus) or pointing gestures using a spatially-stable arrangement of items (e.g. FastTap). Both types of techniques are known to leverage memorization, but not necessarily for the same reasons. In this paper, we investigate whether using directions or positions affects gesture learning. Our study shows that, while recall rates are not significantly different, participants used the novice mode more often and spent more time while learning commands with directional gestures, and they also reported more physical and mental efforts. Moreover, this study also highlights the importance of semantic relationships between gestural commands and reports on the memorization strategies that were elaborated by the participants.

In this paper, we explore the interaction space of MobiLimb, a small 5-DOF serial robotic manipulator attached to a mobile device. It (1) overcomes some limitations of mobile devices (static, passive, motionless); (2) preserves their form factor and I/O capabilities; (3) can be easily attached to or removed from the device; (4) offers additional I/O capabilities such as physical deformation and (5) can support various modular elements such as sensors, lights or shells. We illustrate its potential through three classes of applications: As a tool, MobiLimb offers tangible affordances and an expressive controller that can be manipulated to control virtual and physical objects. As a partner, it reacts expressively to users' actions to foster curiosity and engagement or assist users. As a medium, it provides rich haptic feedback such as strokes, pat and other tactile stimuli on the hand or the wrist to convey emotions during mediated multimodal communications.

We present MarkPad, a novel interaction technique taking advantage of the touchpad. MarkPad allows creating a large number of size-dependent gestural shortcuts that can be spatially organized as desired by the user. It relies on the idea of using visual or tactile marks on the touchpad or a combination of them. Gestures start from a mark on the border and end on another mark anywhere. MarkPad does not conflict with standard interactions and provides a novice mode that acts as a rehearsal of the expert mode. A first study showed that an accuracy of 95% could be achieved for a dense configuration of tactile and/or visual marks allowing many gestures. Performance was 5% lower in a second study where the marks were only on the borders. A last study showed that borders are rarely used, even when the users are unaware of the technique. Finally, we present a working prototype and briefly report on how it was used by two users for a few months.

Multi-touch wall-sized displays afford collaborative exploration of large datasets and re-organization of digital content. However, standard touch interactions, such as dragging to move content, do not scale well to large surfaces and were not designed to support collaboration, such as passing an object around. This paper introduces CoReach, a set of collaborative gestures that combine input from multiple users in order to manipulate content, facilitate data exchange and support communication. We conducted an observational study to inform the design of CoReach, and a controlled study showing that it reduced physical fatigue and facilitated collaboration when compared with traditional multi-touch gestures. A final study assessed the value of also allowing input through a handheld tablet to manipulate content from a distance.

Wall-sized displays support small groups of users working together on large amounts of data. Observational studies of such settings have shown that users adopt a range of collaboration styles, from loosely to closely coupled. Shared interaction techniques, in which multiple users perform a command collaboratively, have also been introduced to support co-located collaborative work. In this paper, we operationalize five collaborative situations with increasing levels of coupling, and test the effects of providing shared interaction support for a data manipulation task in each situation. The results show the benefits of shared interaction for close collaboration: it encourages collaborative manipulation, it is more efficient and preferred by users, and it reduces physical navigation and fatigue. We also identify the time costs caused by disruption and communication in loose collaboration and analyze the trade-offs between parallelization and close collaboration. These findings inform the design of shared interaction techniques to support collaboration on wall-sized displays.

System schematics, such as those used for electrical or hydraulic systems, can be large and complex. Fisheye techniques can help navigate such large documents by maintaining the context around a focus region, but the distortion introduced by traditional fisheye techniques can impair the readability of the diagram. We present SchemeLens, a vector-based, topology-aware fisheye technique which aims to maintain the readability of the diagram. Vector-based scaling reduces distortion to components, but distorts layout. We present several strategies to reduce this distortion by using the structure of the topology, including orthogonality and alignment, and a model of user intention to foster smooth and predictable navigation. We evaluate this approach through two user studies: Results show that (1) SchemeLens is 16-27% faster than both round and rectangular flat-top fisheye lenses at finding and identifying a targ et alng one or several paths in a network diagram; (2) augmenting SchemeLens with a model of user intentions aids in learning the network topology.

Physical Loci, a technique based on an ancient memory technique, allows users to quickly learn a large command set by leveraging spatial, object and verbal/semantic memory to create a cognitive link between individual commands and nearby physical objects in a room (called loci). We first report on an experiment that showed that for learning 25 items Physical Loci outperformed a mid-air Marking Menu baseline. A long-term retention experiment with 48 items then showed that recall was nearly perfect one week later and, surprisingly, independent of whether the command/locus mapping was one's own choice or somebody else's. A final study suggested that recall performance is robust to alterations of the learned mapping, whether systematic or random.

The advent of ultra-high resolution wall-size displays and their use for complex tasks require a more systematic analysis and deeper understanding of their advantages and drawbacks compared with desktop monitors. While previous work has mostly addressed search, visualization and sense-making tasks, we have designed an abstract classification task that involves explicit data manipulation. Based on our observations of real uses of a wall display, this task represents a large category of applications. We report on a controlled experiment that uses this task to compare physical navigation in front of a wall-size display with virtual navigation using pan-and-zoom on the desktop. Our main finding is a robust interaction effect between display type and task difficulty: while the desktop can be faster than the wall for simple tasks, the wall gains a sizable advantage as the task becomes more difficult. A follow-up study shows that other desktop techniques (overview+detail, lens) do not perform better than pan-and-zoom and are therefore slower than the wall for difficult tasks.

Despite the demonstrated benefits of multi-finger input, todays gesture vocabularies offer a limited number of postures and gestures. Previous research designed several posture sets, but does not address the limited human capacity of retaining them. We present a multi-finger chord vocabulary, introduce a novel hand-centric approach to detect the identity of fingers on off-the-shelf hand-held tablets, and report on the detection accuracy. A between-subjects experiment comparing "random" to a "categorized" chord-command mapping found that users retained categorized mappings more accurately over one week than random ones. In response to the logical posture-language structure, people adapted to logical memorization strategies, such as 'exclusion', 'order', and 'category', to minimize the amount of information to retain. We conclude that structured chord-command mappings support learning, short-, and long-term retention of chord- command mappings.

We present WatchIt, a prototype device that extends interaction beyond the watch surface to the wristband, and two interaction techniques for command selection and execution. Because the small screen of wristwatch computers suffers from visual occlusion and the fat finger problem, we investigated the use of the wristband as an available interaction resource. Not only does WatchIt use a cheap, energy efficient and invisible technology, but it involves simple, basic gestures that allow good performance after little training, as suggested by the results of a pilot study. We propose a novel gesture technique and an adaptation of an existing menu technique suitable for wristband interaction. In a user study, we investigated their usage in eyes-free contexts, finding that they perform well. Finally, we present techniques where the bracelet is used in addition to the screen to provide precise continuous control over list scrolling. We also report on a preliminary survey of traditional and digital jewelry that points to the high frequency of watches and bracelets in both genders and gives a sense of the tasks people feel like performing on such devices.

We propose Augmented Letters, a new technique aimed at augmenting gesture-based techniques such as Marking Menus [9] by giving them natural, mnemonic associations. Augmented Letters gestures consist of the initial of command names, sketched by hand in the Unistroke style, and affixed with a straight tail. We designed a tentative touch device interaction technique that supports fast interactions with large sets of commands, is easily discoverable, improves user's recall at no speed cost, and supports fluid transition from novice to expert mode. An experiment suggests that Augmented Letters outperform Marking Menu in terms of user recall.

We present Bezel-Tap Gestures, a novel family of interaction techniques for immediate interaction on handheld tablets regardless of whether the device is alive or in sleep mode. The technique rests on the close succession of two input events: first a bezel tap, whose detection by accelerometers will awake an idle tablet almost instantly, then a screen contact. Field studies confirmed that the probability of this input sequence occurring by chance is very low, excluding the accidental activation concern. One experiment examined the optimal size of the vocabulary of commands for all four regions of the bezel (top, bottom, left, right). Another experiment evaluated two variants of the technique which both allow two-level selection in a hierarchy of commands, the initial bezel tap being followed by either two screen taps or a screen slide. The data suggests that Bezel-Tap Gestures may serve to design large vocabularies of micro-interactions with a sleeping tablet.

This paper presents S-Notebook, a tool that makes it possible to "extend" mobile devices with augmented paper. Paper is used to overcome the physical limitations of mobile devices by offering additional space to annotate digital files and to easily create relationships between them. S-Notebook allows users to link paper annotations or drawings to anchors in digital files without having to learn pre-defined pen gestures. The systems stores meta data such as spatial or temporal location of anchors in the document as well as the zoom level of the view. Tapping on notes with the digital pen make appear the corresponding documents as displayed when the notes were taken. A given piece of augmented paper can contain notes associated to several documents, possibliy at several locations. The annotation space can thus serve as a simple way to relate various pieces of one or several digital documents between them. When the user shares his notes, the piece of paper becomes a tangible token that virtually contains digital information.

This paper introduces JerkTilts, quick back-and-forth gestures that combine device pitch and roll. JerkTilts may serve as gestural self-delimited shortcuts for activating commands. Because they only depend on device acceleration and rely on a parallel and independent input channel, these gestures do not interfere with finger activity on the touch screen. Our experimental data suggest that recognition rates in an eight-choice selection task are as high with JerkTilts as with thumb slides on the touch screen. We also report data confirming that JerkTilts can be combined successfully with simple touch-screen operation. Data from a field study suggest that inadvertent JerkTilts are unlikely to occur in real-life contexts. We describe three illustrative implementations of JerkTilts, which show how the technique helps to simplify and shorten the sequence of actions to reach frequently used commands.

Interaction with TV sets, set-top boxes or media centers strongly differs from interaction with personal computers: not only does a typical remote control suffer strong form factor limitations but the user may well be slouching in a sofa. In the face of more and more data, features, and services made available on interactive televisions, we propose to exploit the new capabilities provided by gesture-aware remote controls. We report the data of three user studies that suggest some guidelines for the design of a gestural vocabulary and we propose five novel interaction techniques. Study 1 reports that users spontaneously perform pitch and yaw gestures as the first modality when interacting with a remote control. Study 2 indicates that users can accurately select up to 5 items with eyes-free roll gestures. Capitalizing on our findings, we designed five interaction techniques that use either device motion, or button-based interaction, or both. They all favor the transition from novice to expert usage for selecting favorites. Study 3 experimentally compares these techniques. It reveals that motion of the device in 3D space, associated with finger presses at the surface of the device, is achievable, fast and accurate. Finally, we discuss the integration of these techniques into a coherent multimedia menu system.

This paper presents the design and evaluation of the Wavelet menu and its implementation on the iPhone. The Wavelet menu consists of a concentric hierarchical Marking menu using simple gestures. The novice mode, i.e. when the menu is displayed, is well adapted to the limited screen space of handheld devices because the representation of the menu hierarchy is inverted, the deeper submenu being always displayed at the center of the screen. The visual design is based on a stacking metaphor to reinforce the perception of the hierarchy and to help users to quickly understand how the technique works. The menu also supports submenu previsualization, a key property to navigate efficiently in a hierarchy of commands. The quantitative evaluation shows that the Wavelet menu provides an intuitive way for supporting efficient gesture-based navigation. The expert mode, i.e. gesture without waiting for the menu to pop-up, is another key property of the Wavelet menu: By providing stroke shortcuts, the Wavelet favors the selection of frequent commands in expert mode and makes eyes-free selection possible. A user experiment shows that participants are able to select commands, eyes-free, while walking.

We propose Radial-Stroke and Finger-Count Shortcuts, two techniques aimed at augmenting the menubar on multi-touch surfaces. We designed these multi-finger two-handed interaction techniques in an attempt to overcome the limitations of direct pointing on interactive surfaces, while maintaining compatibility with traditional interaction techniques. While Radial-Stroke Shortcuts exploit the well-known advantages of Radial Strokes, Finger-Count Shortcuts exploit multi-touch by simply counting the number of fingers of each hand in contact with the surface. We report the results of an experimental evaluation of our technique, focusing on expert-mode performance. Finger-Count Shortcuts outperformed Radial-Stroke Shortcuts in terms of both easiness of learning and performance speed.

International audience

The input vocabulary for touch-screen interaction on handhelds is dramatically limited, especially when the thumb must be used. To enrich that vocabulary we propose to discriminate, among thumb gestures, those we call MicroRolls, characterized by zero tangential velocity of the skin relative to the screen surface. Combining four categories of thumb gestures, Drags, Swipes, Rubbings and MicroRolls, with other classification dimensions, we show that at least 16 elemental gestures can be automatically recognized. We also report the results of two experiments showing that the roll vs. slide distinction facilitates thumb input in a realistic copy and paste task, relative to existing interaction techniques.

Exploration and navigation in multimedia data hierarchies (e.g., photos, music) are frequent tasks on mobile devices. However, visualization and interaction are impoverished due to the limited size of the screen and the lack of precise input devices. As a result, menus on mobile devices do not provide efficient navigation as compared to many innovative menu techniques proposed for Desktop platforms. In this paper, we present Wavelet, the adaptation of the Wave menu for the navigation in multimedia data on iPhone. Its layout, based on an inverted representation of the hierarchy, is particularly well adapted to mobile devices. Indeed, it guarantees that submenus are always displayed on the screen and it supports efficient navigation by providing previsualization of the submenus.

This paper presents Flower menu, a new type of Marking menu that does not only support straight, but also curved gestures for any of the 8 usual orientations. Flower menus make it possible to put many commands at each menu level and thus to create as large a hierarchy as needed for common applications. Indeed our informal analysis of menu breadth in popular applications shows that a quarter of them have more than 16 items. Flower menus can easily contain 20 items and even more (theoretical maximum of 56 items). Flower menus also support within groups as well as hierarchical groups. They can thus favor breadth organization (within groups) or depth organization (hierarchical groups): as a result, the designers can lay out items in a very flexible way in order to reveal meaningful item groupings. We also investigate the learning performance of the expert mode of Flower menus. A user experiment is presented that compares linear menus (baseline condition), Flower menus and Polygon menus, a variant of Marking menus that supports a breadth of 16 items. Our experiment shows that Flower menus are more efficient than both Polygon and Linear menus for memorizing command activation in expert mode.

International audience

This paper presents TactiMote, a remote control with tactile feedback designed for navigating in long lists and catalogues. TactiMote integrates a joystick that allows 2D interaction with the thumb and a Braille cell that provides tactile feedback. This feedback is intended to help the selection task in novice mode and to allow for fast eyes-free navigation among favorite items in expert mode. The paper describes the design of the TactiMote prototype for TV channel selection and reports a preliminary experiment that shows the feasibility of the approach.

We present an off-line cursive word recognition system based completely on neural networks: reading models and models of early visual processing. The first stage (normalization) preprocesses the input image in order to reduce letter position uncertainty; the second stage (feature extraction) is based on the feedforward model of orientation selectivity; the third stage (letter pre-recognition) is based on a convolutional neural network, and the last stage (word recognition) is based on the interactive activation model.

Treemaps provide an interesting solution for representing hierarchical data. However, most studies have mainly focused on layout algorithms and paid limited attention to the interaction with treemaps. This makes it difficult to explore large data sets and to get access to details, especially to those related to the leaves of the trees. We propose the notion of zoomable treemaps (ZTMs), an hybridization between treemaps and zoomable user interfaces that facilitates the navigation in large hierarchical data sets. By providing a consistent set of interaction techniques, ZTMs make it possible for users to browse through very large data sets (e.g., 700,000 nodes dispatched amongst 13 levels). These techniques use the structure of the displayed data to guide the interaction and provide a way to improve interactive navigation in treemaps.

This paper presents a new GUI architecture for creating advanced interfaces. This model is based on a limited set of general principles that improve flexibility and provide capabilities for implementing information visualization techniques such as magic lenses, transparent tools or semantic zooming. This architecture also makes it possible to create multiple views and application-sharing systems (by sharing views on multiple computer screens) in a simple and uniform way and to handle bimanual interaction and multiple pointers. An experimental toolkit called Ubit was implemented to test the feasibility of this approach. It is based on a pseudo-declarative C++ API that tries to simplify GUI programming by providing a higher level of abstraction.

Zoomable User Interfaces (ZUIs) are difficult to use on large information spaces in part because they provide insufficient context. Even after a short period of navigation users no longer know where they are in the information space nor where to find the information they are looking for. We propose a temporary in-place context aid that helps users position themselves in ZUIs. This context layer is a transparent view of the context that is drawn over the users' focus of attention. A second temporary in-place aid is proposed that can be used to view already visited regions of the information space. This history layer is an overlapping transparent layer that adds a history mechanism to ZUIs. We complete these orientation aids with an additional window, a hierarchy tree, that shows users the structure of the information space and their current position within it. Context layers show users their position, history layers show them how they got there, and hierarchy trees show what information is available and where it is.

In this paper, we present a system for the recognition of cursive handwriting that utilizes the Hough transform and a neural network. The Hough transform is a line detection technique which has the ability of tolerating deformation, disconnections and noise. Instead of searching for linear strokes in the image, we compute global directional information at each pixel of the image. This information is stored into several feature maps. Thus we avoid assigning to each pixel a single orientation in order to preserve useful information. Each feature map is then processed by zones in order to estimate the local orientation of the strokes. Finally, we recognize the image by means of a neural network classifier. We have tested the system for the recognition of segmented cursive characters, cursive words and the first letter of cursive words. The results obtained are encouraging and compare well with respect to other results.

This paper presents a new interactive UIDE that is based on visual programming and constrained sketch drawing. At the early stages of the iterative conception process, GUIs are interactively designed by drawing a "rough sketch" that acts as a first draft of the final description. This drawing is interpreted in real time by the system in order to produce the corresponding widget view (i.e. the actual visible GUI) and a graph of abstract objects that represents the GUI structure. This graph can then be easily modified by mixing visual and textual programming in a fully iterative and incremental way. This system is also based on the use of generic objects which are dynamically instantiated into actual widgets according to their structural and functional context. This scheme makes it possible to define very generic GUIs that can then be deeply refined in an efficient way at any stage of the conception process.

Presents a model for reading cursive scripts which has an architecture inspired by a reading model and which is based on perceptual concepts. We limit the scope of our study to the off-line recognition of isolated cursive words. First of all, we justify why we chose McClelland & Rumelhart's (1981) reading model as the inspiration for our system. A brief resume/spl acute/ of the method's behavior is presented and the main originalities of our model are underlined. After this, we focus on the new updates added to the original system: a new baseline extraction module, a new feature extraction module and a new generation, validation and hypothesis insertion process. After implementation of our method, new results have been obtained on real images from a training set of 184 images, and a testing set of 100 images, and are discussed. We are concentrating now on validating the model using a larger database.

Presents the first achievements of HERS (Hypermedia Edit and Read Station), which is devoted to the browsing and editing of hypermedia documents from literary material including document images. Concerning the editing, our purpose as twofold. First, capabilities are offered to transcribe manuscripts. Transcribing the text consists of coupling lines typed on the keyboard with their corresponding text lines in the manuscript images. A collaborative system, based on computer-human interaction and document analysis, is proposed for performing this task. Second, interactive tools are offered to organize the electronic document and establish hypermedia links between its different components (image areas, transcribed words or lines, or other kinds of heterogeneous data). Concerning the browsing, we developed an approach based on information visualization in order to provide users with an idea of the overall organization of the hyperdocument and so help them to navigate through it.

This paper presents XXL, a new interactive development system for building user interfaces which is based on the concept of textual and visual equivalence.XXL includes an interactive builder and a "small" C compatible special-purpose language that is both interpretable and compilable.The visual builder is able to establish the reverse correspondence between the dynamic objects that it manipulates and their textual descriptions in the original source code.Interactive modifications performed by using the builder result in incremental modifications of the original text.Lastly, XXL not only allows users to specify the widget part of the interface but can also be used to manage various behaviors and to create distributed interfaces,

Character segmentation has long been a critical area of the OCR process. The higher recognition rates for isolated characters vs. those obtained for words and connected character strings well illustrate this fact. A good part of recent progress in reading unconstrained printed and written text may be ascribed to more insightful handling of segmentation. This paper provides a review of these advances. The aim is to provide an appreciation for the range of techniques that have been developed, rather than to simply list sources. Segmentation methods are listed under four main headings. What may be termed the "classical" approach consists of methods that partition the input image into subimages, which are then classified. The operation of attempting to decompose the image into classifiable units is called "dissection." The second class of methods avoids dissection, and segments the image either explicitly, by classification of prespecified windows, or implicitly by classification of subsets of spatial features collected from the image as a whole. The third strategy is a hybrid of the first two, employing dissection together with recombination rules to define potential segments, but using classification to select from the range of admissible segmentation possibilities offered by these subimages. Finally, holistic approaches that avoid segmentation by recognizing entire character strings as units are described.

This paper provides a review of advances in character segmentation. Segmentation methods are listed under four main headings. The operation of attempting to decompose the image into classifiable units on the basis of general image features is called "dissection". The second class of methods avoids dissection, and segments the image either explicitly, by classification of specified windows, or implicitly by classification of subsets of spatial features collected from the image as a whole. The third strategy is a hybrid of the first two, employing dissection together with recombination rules to define potential segments, but using classification to select from the range of admissible segmentation possibilities offered by these subimages. Finally, holistic approaches that avoid segmentation by recognizing entire character strings as units are described.

This paper presents a new perception based model for reading cursive script. We describe the organization of our pseudo-neuronal system and show the role of activation mechanism in perceiving and reading cursive script. We have introduced into our model some characteristics specific to cursive script. First, we use more appropriate features such as ascenders and descenders. Second, we deal with the ambiguity of letter location by introducing the concept of the fuzzy position. The location as well as the missing letters are deduced from the context (i.e. the word-letter lexicon). After implementation of our method, preliminary qualitative results have been obtained and are discussed. We are concentrating now on further formalizing and generalizing the proposed model on a larger data base.

A recognition scheme for reading handwritten cursive words using three word recognition techniques is described. The focus is on the implementation used to combine the three techniques based on a comparative study of different strategies. The first holistic recognition technique derives a global encoding of the word. The other techniques both rely on the segmentation of the word into letters, but differ in the character classifier they use. The former runs a statistical linear classifier, and the latter runs a neural network with a different representation of the input data. The testing, comparison, and combination studies have been performed on word images from mail provided by the USPS. The top choice recognition rates achieved so far correspond to 88%, 76%, 65% with respect to lexicon sizes of 10, 100, and 1000 words.>

The automatic reading of unconstrained cursive city names written on real mail is considered. An analytical approach is described which first achieves an explicit segmentation of cursive words. The results after crosschecking of the zip code and the city name are a 61.7% efficiency rate and a 1.3 confusion rate, which allow operational implementation.>