This article tackles two challenges of the empirical evaluation of interaction techniques that rely on user memory, such as hotkeys, here coined Recall-based interaction techniques (RBITs): (1) the lack of guidance to design the associated study protocols, and (2) the difficulty of comparing evaluations performed with different protocols. To address these challenges, we propose a model-based evaluation of RBITs. This approach relies on a computational model of human memory to (1) predict the informativeness of a particular protocol through the variance of the estimated parameters (Fisher Information) (2) compare RBITs recall performance based on the inferred parameters rather than behavioral statistics, which has the advantage of being independent of the study protocol. We also release a Python library implementing our approach to aid researchers in producing more robust and meaningful comparisons of RBITs.

Trust between humans and AI in the context of decision-making has acquired an important role in public policy, research and industry. In this context, Human-AI Trust has often been tackled from the lens of cognitive science and psychology, but lacks insights from the stakeholders involved. In this paper, we conducted semi-structured interviews with 7 AI practitioners and 7 decision subjects from various decision domains. We found that 1) interviewees identified the prerequisites for the existence of trust and distinguish trust from trustworthiness, reliance, and compliance; 2) trust in AI-integrated systems is strongly influenced by other human actors, more than the system’s features; 3) the role of Human-AI trust factors is stakeholder-dependent. These results provide clues for the design of Human-AI interactions in which trust plays a major role, as well as outline new research directions in Human-AI Trust.

Despite the benefits of expert interaction techniques, many users do not learn them and continue to use novice ones. This article aims at better understanding if, when and how users decide to learn and ultimately adopt expert interaction techniques. This dynamic learning process is a complex skill-acquisition and decision-making problem. We first present and compare three generic benchmark models, inspired by the neuroscience literature, to explain and predict the learning process for shortcut adoption. Results show that they do not account for the complexity of users’ behavior. We then introduce a dedicated model, Transition, combining five cognitive mechanisms: implicit and explicit learning, decay, planning and perseveration. Results show that our model outperforms the three benchmark models both in terms of model fitting and model simulation. Finally, a post-analysis shows that each of the five mechanisms contribute to goodness-of-fit, but the role of perseveration is unclear regarding model simulation.

Hand redirection is a visuo-haptic illusion offering rich haptic feedback in Virtual Reality while using a limited number of physical objects. This technique relies on a redirection function which determines the virtual hand position depending on the physical hand position. In this paper, we extend the design space of the Hand-redirection technique by generalizing this redirection function. A user study compares 6 redirection functions. Our results suggest that the redirection function does not deteriorate the illusion offering more flexibility and control to designer over the hand trajectory and avoid collisions.

Implicit Statistical Learning (ISL) studies how exposing individuals to repeated statistical patterns can help develop skills in the absence of conscious awareness, such as learning a language or detecting familiar shapes. This paper transposes ISL in the context of menu design learnability. Our analysis of menu patterns in various applications from the 80s to today reveals a consistent linear pattern with command names on the left and keyboard shortcut cues aligned on the right. We then develop a design space of menu patterns by manipulating two factors of ISL theory, spatial proximity (distance) and relative positioning between commands and shortcut cues. We empirically compare four menu patterns of this design space on whether they can improve keyboard shortcut adoption through two controlled experiments. Results did not capture clear effects among the menu patterns, suggesting that ISL in the context of HCI might involve more complex factors than initially anticipated, such as the time the users are exposed to the menu pattern. We reflect on the challenges in applying theories from cognitive science to HCI and hope that our systematic methodology and experiment designs will serve as a basis for encouraging more studies in the area.

Encountered-type of Haptic devices (ETHD) are robotic interfaces physically overlaying virtual counterparts prior to a user interaction in Virtual Reality. They theoretically reliably provide haptics in Virtual environments, yet they raise several intrinsic design challenges to properly display rich haptic feedback and interactions in VR applications. In this paper, we use a Failure Mode and Effects Analysis (FMEA) approach to identify, organise and analyse the failure modes and their causes in the different stages of an ETHD scenario and highlight appropriate solutions from the literature to mitigate them. We help justify these interfaces’ lack of deployment, to ultimately identify guidelines for future ETHD designers.

Capturing content of fabrication activities is the first step for producing knowledge resources and an integral part of maker culture. As a secondary task, it conflicts though with the fabrication activity, and thus it is often forgotten and knowledge resources end up incomplete. In this article, we investigate different dimensions of content capture for knowledge resources in fabrication workshops. Based on past work in this area, we first propose a framework through which we identify two research directions to investigate. From these, we derive three dimensions to explore in more depth: The number of capturing devices, their feature variety and the degree of automation of each feature. We then explore the design space resulting from these three dimensions with the help of a design concept and an online survey study (N=66). Results show (1) a variety of needs and preferences justifying feature variety and multiplicity, (2) challenges in defining the right degree of manual and automatic control, and (3) the socio-technical impact of cameras in a shared space regarding privacy and ethics. We conclude with discussions on the benefits and vulnerabilities of equipping fabrication workshop with distributed camera-based capturing devices and offer opportunities for design.

Humans have the ability to convey an array of emotions through complex and rich touch gestures. However, it is not clear how these touch gestures can be reproduced through interactive systems and devices in a remote mediated communication context. In this article, we explore the design space of device-initiated touch for conveying emotions with an interactive system reproducing a collection of human touch characteristics. For this purpose, we control a robotic arm to touch the forearm of participants with different force, velocity and amplitude characteristics to simulate human touch. In view of adding touch as an emotional modality in human-machine interaction, we have conducted two studies. After designing the touch device, we explore touch in a context-free setup and then in a controlled context defined by textual scenarios and emotional facial expressions of a virtual agent. Our results suggest that certain combinations of touch characteristics are associated with the perception of different degrees of valence and of arousal. Moreover, in the case of non-congruent mixed signals (touch, facial expression, textual scenario) not conveying a priori the same emotion, the message conveyed by touch seems to prevail over the ones displayed by the visual and textual signals.

Disability professionals could use digital fabrication tools to provide customised assistive technologies or accessible media beneficial to the education of Blind or visually impaired youth. However, there is little documentation of long-term practices with these tools by professionals in this field, limiting our ability to support their work. We report on such practices in a French organisation, providing disability educational services and using digital fabrication since 2013, for six years. We trace how professionals defined how digital fabrication could and should be used through a range of projects, based on pedagogical uses and the constraints in creation, production and maintenance. We outline new research perspectives going beyond 3D printers and its promises of automation to embrace hybrid approaches currently supported by laser cutters, the learning and documentation process, and the production of accessible tactile media at a regional or national scale.

Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user – for example, due to surprise or relearning effort – or “trap” the process to a suboptimal design immaturely. However, effects on users are hard to predict as they depend on factors that are latent and evolve over the course of interaction. We propose a novel approach for adaptive user interfaces that yields a conservative adaptation policy: It finds beneficial changes when there are such and avoids changes when there are none. Our model-based reinforcement learning method plans sequences of adaptations and consults predictive HCI models to estimate their effects. We present empirical and simulation results from the case of adaptive menus, showing that the method outperforms both a non-adaptive and a frequency-based policy.

The spread of AI-embedded systems involved in human decision making makes studying human trust in these systems critical. However, empirically investigating trust is challenging. One reason is the lack of standard protocols to design trust experiments. In this paper, we present a survey of existing methods to empirically investigate trust in AI-assisted decision making and analyse the corpus along the constitutive elements of an experimental protocol. We find that the definition of trust is not commonly integrated in experimental protocols, which can lead to findings that are overclaimed or are hard to interpret and compare across studies. Drawing from empirical practices in social and cognitive studies on human-human trust, we provide practical guidelines to improve the methodology of studying Human-AI trust in decision-making contexts. In addition, we bring forward research opportunities of two types: one focusing on further investigation regarding trust methodologies and the other on factors that impact Human-AI trust.

Bedside robotic endoscopes render surgeons autonomous from assistants, potentially improving surgical outcome and decreasing costs. Why then have they not been widely adopted? We take a step back and first characterize classic (non-robotic) endoscope use through observations, literature and a domain expert interview. We review the literature on bedside robotic endoscopes and find that existing controls, individually, do not have the power to support both intended and appropriated endoscope uses. We thus explore combining controls to support this diversity of uses. Through an iterative cycle, we design and implement a multimodal and mixed-initiative technique that combines two user controls and one system control. Our evaluations confirm that individual controls do not satisfy the diversity of endoscope uses, and also that our technique indeed does so. Our work highlights the relevance of HCI research in the medical domain through robotic systems.

We investigated retroactive transfer when users alternate between different interfaces. Retroactive transfer is the influence of a newly learned interface on users' performance with a previously learned interface. In an interview study, participants described their experiences when alternating between different interfaces, e.g. different operating systems, devices or techniques. Negative retroactive transfer related to text entry was the most frequently reported incident. We then reported a laboratory experiment that investigated the impact of similarity between two abstract keyboard layouts, and the number of alternations between them, on retroactive interference. Results indicated that even small changes in the interference interface produced a significant performance drop for the entire previously learned interface. The amplitude of this performance drop decreases with the number of alternations. We suggest that retroactive transfer should receive more attention in HCI, as the ubiquitous nature of interactions across applications and systems requires users to increasingly alternate between similar interfaces.

We present CoVR, a novel robotic interface providing strong kinesthetic feedback (100 N) in a room-scale VR arena. It consists of a physical column mounted on a 2D Cartesian ceiling robot (XY displacements) with the capacity of (1) resisting to body-scaled users' actions such as pushing or leaning; (2) acting on the users by pulling or transporting them as well as (3) carrying multiple potentially heavy objects (up to 80kg) that users can freely manipulate or make interact with each other. We describe its implementation and define a trajectory generation algorithm based on a novel user intention model to support non-deterministic scenarios, where the users are free to interact with any virtual object of interest with no regards to the scenarios' progress. A technical evaluation and a user study demonstrate the feasibility and usability of CoVR, as well as the relevance of whole-body interactions involving strong forces, such as being pulled through or transported.

Robotic-assisted Minimally Invasive Surgery (MIS) is adopted more and more as it overcomes the shortcomings of classic MIS for surgeons while keeping the benefits of small incisions for patients. However, introducing new technology oftentimes affects the work of skilled practitioners. Our goals are to investigate the impacts of telemanipulated surgical robots on the work practices of surgical teams and to understand their cause. We conducted a field study observing 21 surgeries, conducting 12 interviews and performing 3 data validation sessions with surgeons. Using Thematic Analysis, we find that physically separating surgeons from their teams makes them more autonomous, shifts their use of perceptual senses, and turns the surgeon's assistant into the robot's assistant. We open design opportunities for the HCI field by questioning the telemanipulated approach and discussing alternatives that keep surgeons on the surgical field.

This paper presents SAM, a modular and extensible JavaScript framework for self-adapting menus on webpages. SAM allows control of two elementary aspects for adapting web menus: (1) the target policy, which assigns scores to menu items for adaptation, and (2) the adaptation style, which specifies how they are adapted on display. By decoupling them, SAM enables the exploration of different combinations independently. Several policies from literature are readily implemented, and paired with adaptation styles such as reordering and highlighting. The process---including user data logging---is local, offering privacy benefits and eliminating the need for server-side modifications. Researchers can use SAM to experiment adaptation policies and styles, and benchmark techniques in an ecological setting with real webpages. Practitioners can make websites self-adapting, and end-users can dynamically personalise typically static web menus.

We propose a paradigm called Skin-On interfaces, in which interactive devices have their own (artificial) skin, thus enabling new forms of input gestures for end-users (e.g. twist, scratch). Our work explores the design space of Skin-On interfaces by following a bio-driven approach: (1) From a sensory point of view, we study how to reproduce the look and feel of the human skin through three user studies;(2) From a gestural point of view, we explore how gestures naturally performed on skin can be transposed to Skin-On interfaces; (3) From a technical point of view, we explore and discuss different ways of fabricating interfaces that mimic human skin sensitivity and can recognize the gestures observed in the previous study; (4) We assemble the insights of our three exploratory facets to implement a series of Skin-On interfaces and we also contribute by providing a toolkit that enables easy reproduction and fabrication.

Human decisions are prone to biases, and this is no less true for decisions made within data visualizations. Bias mitigation strategies often focus on the person, by educating people about their biases, typically with little success. We focus instead on the system, presenting the first evidence that altering the design of an interactive visualization tool can mitigate a strong bias - the attraction effect. Participants viewed 2D scatterplots where choices between superior alternatives were affected by the placement of other suboptimal points. We found that highlighting the superior alternatives weakened the bias, but did not eliminate it. We then tested an interactive approach where participants completely removed locally dominated points from the view, inspired by the elimination by aspects strategy in the decision-making literature. This approach strongly decreased the bias, leading to a counterintuitive suggestion: tools that allow removing inappropriately salient or distracting data from a view may help lead users to make more rational decisions.

This paper argues for designing geo-technologies supporting non-visual sensory knowledge. Sensory knowledge refers to the implicit and explicit knowledge guiding our uses of our senses to understand the world. To support our argument, we build on an 18 months field-study on geography classes for primary school children with visual impairments. Our findings show (1) a paradox in the use of non-visual sensory knowledge: described as fundamental to the geography curriculum, it is mostly kept out of school; (2) that accessible geo-technologies in the literature mainly focus on substituting vision with another modality, rather than enabling teachers to build on children's experiences; (3) the importance of the hearing sense in learning about space. We then introduce a probe, a wrist-worn device enabling children to record audio cues during field-trips. By giving importance to children's hearing skills, it modified existing practices and actors' opinions on non-visual sensory knowledge. We conclude by reflecting on design implications, and the role of technologies in valuing diverse ways of understanding the world.

Predicting human performance in interaction tasks allows designers or developers to understand the expected performance of a target interface without actually testing it with real users. In this work, we present a deep neural net to model and predict human performance in performing a sequence of UI tasks. In particular, we focus on a dominant class of tasks, i.e., target selection from a vertical list or menu. We experimented with our deep neural net using a public dataset collected from a desktop laboratory environment and a dataset collected from hundreds of touchscreen smartphone users via crowdsourcing. Our model significantly outperformed previous methods on these datasets. Importantly, our method, as a deep model, can easily incorporate additional UI attributes such as visual appearance and content semantics without changing model architectures. By understanding about how a deep learning model learns from human behaviors, our approach can be seen as a vehicle to discover new patterns about human behaviors to advance analytical modeling.

In this paper, we explore the interaction space of MobiLimb, a small 5-DOF serial robotic manipulator attached to a mobile device. It (1) overcomes some limitations of mobile devices (static, passive, motionless); (2) preserves their form factor and I/O capabilities; (3) can be easily attached to or removed from the device; (4) offers additional I/O capabilities such as physical deformation and (5) can support various modular elements such as sensors, lights or shells. We illustrate its potential through three classes of applications: As a tool, MobiLimb offers tangible affordances and an expressive controller that can be manipulated to control virtual and physical objects. As a partner, it reacts expressively to users' actions to foster curiosity and engagement or assist users. As a medium, it provides rich haptic feedback such as strokes, pat and other tactile stimuli on the hand or the wrist to convey emotions during mediated multimodal communications.

We propose a novel perspective on the design of toolbar buttons that aims to increase keyboard shortcut accessibility. IconHK implements this perspective by blending visual cues that convey keyboard shortcut information into toolbar buttons without denaturing the pictorial representation of their command. We introduce three design strategies to embed the hotkey, a visual encoding to convey the modifiers, and a magnification factor that determines the blending ratio between the pictogram of the button and the visual representation of the keyboard shortcut. Two studies examine the benefits of IconHK for end-users and provide insights from professional designers on the practicality of our approach for creating iconsets. Building on these insights, we develop a tool to assist designers in applying the IconHK design principle.

TouchTokens are passive tokens that can be recognized on any capacitive surface based on the spatial configuration of the fingers that hold them. However, interaction with these tokens is confined to the basic two-state model of touch interaction as the system only knows the tokens' position and cannot detect tokens that are not touched. We increase the expressive power of TouchTokens by introducing laser-cut lattice hinges in their design, so as to make them flexible. A new recognizer, that analyzes the micro-movements of the fingers that hold the tokens, enables the system to detect when a token is left on the surface rather than taken off it. It can also detect bend events that can be mapped to command triggers, and a squeezed state that can be used for quasi-modal interaction.

While it is well known that menu usage follows a Zipfian distribution, there has been little interest in the impact of menu item frequency distribution on user's behavior. In this note, we explore the effects of frequency distribution on average menu performance as well as individual item performance. We compare three frequency distributions of menu item usage: Uniform; Zipfian with s=1 and Zipfian with s=2. The results show that (1) user's behavior is sensitive to different frequency distributions at both menu and item level; (2) individual item selection time depends on, not only its frequency, but also the frequency of other items in the menu. Finally, we discuss how these findings might have impacts on menu design, empirical studies and menu modeling.

We investigate the potential benefits of actuated devices for the desktop workstation which remains the most used environment for daily office works. A formative study reveals that the desktop workstation is not a fixed environment because users manually change the position and the orientation of their devices. Based on these findings, we present the LivingDesktop, an augmented desktop workstation with devices (mouse, keyboard, monitor) capable of moving autonomously. We describe interaction techniques and applications illustrating how actuated desktop workstations can improve ergonomics, foster collaboration, leverage context and reinforce physicality. Finally, the findings of a scenario evaluation are (1) the perceived usefulness of ergonomics and collaboration applications; (2) how the LivingDesktop inspired our participants to elaborate novel accessibility and social applications; (3) the location and user practices should be considered when designed actuated desktop devices.

We report on the design process leading to the creation of MapSense, a multi-sensory interactive map for visually impaired children. We conducted a formative study in a specialized institute to understand children's educational needs, their context of care and their preferences regarding interactive technologies. The findings (1) outline the needs for tools and methods to help children to acquire spatial skills and (2) provide four design guidelines for educational assistive technologies. Based on these findings and an iterative process, we designed and deployed MapSense in the institute during two days. It enables collaborations between children with a broad range of impairments, proposes reflective and ludic scenarios and allows caretakers to customize it as they wish. A field experiment reveals that both children and caretakers considered the system successful and empowering.

TouchTokens make it possible to easily build interfaces that combine tangible and gestural input using passive tokens and a regular multi-touch surface. The tokens constrain users' grasp, and thus, the relative spatial configuration of fingers on the surface, theoretically making it possible to design algorithms that can recognize the resulting touch patterns. We performed a formative user study to collect and analyze touch patterns with tokens of varying shape and size. The analysis of this pattern collection showed that individual users have a consistent grasp for each token, but that this grasp is user-dependent and that different grasp strategies can lead to confounding patterns. We thus designed a second set of tokens featuring notches that constrain users' grasp. Our recognition algorithm can classify the resulting patterns with a high level of accuracy (>95%) without any training, enabling application designers to associate rich touch input vocabularies with command triggers and parameter controls.

System schematics, such as those used for electrical or hydraulic systems, can be large and complex. Fisheye techniques can help navigate such large documents by maintaining the context around a focus region, but the distortion introduced by traditional fisheye techniques can impair the readability of the diagram. We present SchemeLens, a vector-based, topology-aware fisheye technique which aims to maintain the readability of the diagram. Vector-based scaling reduces distortion to components, but distorts layout. We present several strategies to reduce this distortion by using the structure of the topology, including orthogonality and alignment, and a model of user intention to foster smooth and predictable navigation. We evaluate this approach through two user studies: Results show that (1) SchemeLens is 16-27% faster than both round and rectangular flat-top fisheye lenses at finding and identifying a targ et alng one or several paths in a network diagram; (2) augmenting SchemeLens with a model of user intentions aids in learning the network topology.

One reason that human interaction with technology is difficult to understand is because the way in which people perform interactive tasks is highly adaptive. One such interactive task is menu search. In the current article we test the hypothesis that menu search is rationally adapted to (1) the ecological structure of interaction, (2) cognitive and perceptual limits, and (3) the goal to maximise the trade-off between speed and accuracy. Unlike in previous models, no assumptions are made about the strategies available to or adopted by users, rather the menu search problem is specified as a reinforcement learning problem and behaviour emerges by finding the optimal Markov Decision Process (MDP). The model is tested against existing empirical findings concerning the effect of menu organisation and menu length. The model predicts the effect of these variables on task completion time and eye movements. The discussion considers the pros and cons of the modelling approach relative to other well-known mod- elling approaches.

We propose iSkin, a novel class of skin-worn sensors for touch input on the body. iSkin is a very thin sensor overlay, made of biocompatible materials, and is flexible and stretchable. It can be produced in different shapes and sizes to suit various locations of the body such as the finger, forearm, or ear. Integrating capacitive and resistive touch sensing, the sensor is capable of detecting touch input with two levels of pressure, even when stretched by 30% or when bent with a radius of 0.5cm. Furthermore, iSkin supports single or multiple touch areas of custom shape and arrangement, as well as more complex widgets, such as sliders and click wheels. Recognizing the social importance of skin, we show visual design patterns to customize functional touch sensors and allow for a visually aesthetic appearance. Taken together, these contributions enable new types of on-body devices. This includes finger-worn devices, extensions to conventional wearable devices, and touch input stickers, all fostering direct, quick, and discreet input for mobile computing.

This paper presents a novel mathematical model for visual search and selection time in linear menus. Assuming two visual search strategies, serial and directed, and a pointing sub-task, it captures the change of performance with five fac- tors: 1) menu length, 2) menu organization, 3) target position, 4) absence/presence of target, and 5) practice. The novel aspect is that the model is expressed as probability density distribution of gaze, which allows for deriving total selection time. We present novel data that replicates and extends the Nielsen menu selection paradigm and uses eye-tracking and mouse tracking to confirm model predictions. The same parametrization yielded a high fit to both menu selection time and gaze distributions. The model has the potential to improve menu designs by helping designers identify more effective solutions without conducting empirical studies.

In this paper we present a series of three field studies on the integration of multiple modalities (touch and mid-air gestures) in a public display. We analyze our field studies using Conversion Diagrams, an approach to model and evaluate usage of multimodal public displays. Conversion diagrams highlight the transitions inherent in a multimodal system and provide a systematic approach to investigate which factors affect them and how. We present a semi-automatic annotation technique to obtain Conversion Diagrams. We use Conversion Diagrams to evaluate interaction in the three field studies. We found that 1) clear affordances for touch were necessary when mid-air gestures were present. A call-to-action caused significantly more users to touch than a button (+200%), 2) the order of modality usage was different from what we designed for, and the location impacted which modality was used first, and 3) small variations in the application did lead to considerable user increase (+290%).

Most of today's public displays only show predefined contents that passers-by are not able to change. We argue that interactive public displays would benefit from immediately usable mid-air techniques for choosing options, expressing opinions or more generally selecting one among several items. We propose a design space for hand-gesture based mid-air selection techniques on interactive public displays, along with four specific techniques that we evaluated at three different locations in the the field. Our findings include: 1) if no hint is provided, people successfully use Point+Dwell for selecting items, 2) the user representation could be switched from Mirror to Cursor after registration without causing confusion, 3) people tend to explore items before confirming one, 4) in a public context, people frequently interact inadvertently (without looking at the screen). We conclude by providing recommendations for designers of interactive public displays to support immediate usability for mid-air selection.

Hotkeys are an efficient method of selecting commands on a keyboard. However, these shortcuts are often underused by users. We present Métamorphe, a novel keyboard with keys that can be individually raised and lowered to promote hotkeys usage. Métamorphe augments the output of traditional keyboards with haptic and visual feedback, and offers a novel design space for user input on raised keys (e.g., gestures such as squeezing or pushing the sides of a key). We detail the implementation of Métamorphe and discuss design factors. We also report two user studies. The first is a user-defined interface study that shows that the new input vocabulary is usable and useful, and provides insights into the mental models that users associate with raised keys. The second user study shows improved eyes-free selection performance for raised keys as well as the surrounding unraised keys.

Keyboard shortcuts allow fast interaction, but they are known to be infrequently used, with most users relying heavily on traditional pointer-based selection for most commands. We describe the goals, design, and evaluation of ExposeHK, a new interface mechanism that aims to increase hotkey use. ExposeHK's four key design goals are: 1) enable users to browse hotkeys; 2) allow non-expert users to issue hotkey commands as a physical rehearsal of expert performance; 3) exploit spatial memory to assist non-expert users in identifying hotkeys; and 4) maximise expert performance by using consistent shortcuts in a flat command hierarchy. ExposeHK supports these objectives by displaying hotkeys overlaid on their associated commands when a modifier key is pressed. We evaluated ExposeHK in three empirical studies using toolbars, menus, and a tabbed \'18ribbon' toolbar. Results show that participants used more hotkeys, and used them more often, with ExposeHK than with other techniques; they were faster with ExposeHK than with either pointing or other hotkey methods; and they strongly preferred ExposeHK. Our research shows that ExposeHK can substantially improve the user's transition from a \'18beginner mode' of interaction to a higher level of expertise.

We propose and validate a model of the perception area of content on public displays in order to predict from where users can read. From this model, we derive Screenfinity, a technique to rotate, translate, and zoom content in order to enable reading while passing by very large displays. Screenfinity is comfortable to read when close, supports different content for different users, does not waste screen real estate and allows expert passers-by to read content while walking. A laboratory study shows that expert users are able to perceive content when it moves. A field study evaluates the effect of Screenfinity on novice users in an ecologically valid setting. We find 1) first time users can read content without slowing down or stopping; 2) Passers-by stopping did so to explore the technology. Users explore the interaction, the limits of the system, manipulate the technology, and look behind the screen.

We investigate how to reveal an initial mid-air gesture on interactive public displays. This initial gesture can serve as gesture registration for advanced operations. We propose three strategies to reveal the initial gesture: spatial division, temporal division and integration. Spatial division permanently shows the gesture on a dedicated screen area. Temporal division interrupts the application to reveal the gesture. Integration embeds gesture hints directly in the application. We also propose a novel initial gesture called Teapot to illustrate our strategies. We report on a laboratory and field study. Our main findings are: A large percentage of all users execute the gesture, especially with spatial division (56%). Users intuitively discover a gesture vocabulary by exploring variations of the Teapot gesture by themselves, as well as by imitating and extending other users' variations.

Menu systems are challenging to design because design spaces are immense, and several human factors affect user behavior. This paper contributes to the design of menus with the goal of interactively assisting designers with an optimizer in the loop. To reach this goal, 1) we extend a predictive model of user performance to account for expectations as to item groupings; 2) we adapt an ant colony optimizer that has been proven efficient for this class of problems; and 3) we present MenuOptimizer, a set of inter-actions integrated into a real interface design tool (QtDesigner). MenuOptimizer supports designers' abilities to cope with uncertainty and recognize good solutions. It allows designers to delegate combinatorial problems to the optimizer, which should solve them quickly enough without disrupting the design process. We show evidence that satisfactory menu designs can be produced for complex problems in minutes.

Rotating 3D objects is a difficult task on mobile devices, because the task requires 3 degrees of freedom and (multi-)touch input only allows for an indirect mapping. We propose a novel style of mobile interaction based on mid-air gestures in proximity of the device to increase the number of DOFs and alleviate the limitations of touch interaction with mobile devices. While one hand holds the device, the other hand performs mid-air gestures in proximity of the device to control 3D objects on the mobile device's screen. A flat hand pose defines a virtual surface which we refer to as the PalmSpace for precise and intuitive 3D rotations. We constructed several hardware prototypes to test our interface and to simulate possible future mobile devices equipped with depth cameras. We conducted a user study to compare 3D rotation tasks using the most promising two designs for the hand location during interaction -- behind and beside the device -- with the virtual trackball, which is the current state-of-art technique for orientation manipulation on touch-screens. Our results show that both variants of PalmSpace have significantly lower task completion times in comparison to the virtual trackball.

When the user is engaged with a real-world task it can be inappropriate or difficult to use a smartphone. To address this concern, we developed ShoeSense, a wearable system consisting in part of a shoe-mounted depth sensor pointing upward at the wearer. ShoeSense recognizes relaxed and discreet as well as large and demonstrative hand gestures. In particular, we designed three gesture sets (Triangle, Radial, and Finger-Count) for this setup, which can be performed without visual attention. The advantages of ShoeSense are illustrated in five scenarios: (1) quickly performing frequent operations without reaching for the phone, (2) discreetly performing operations without disturbing others, (3) enhancing operations on mobile devices, (4) supporting accessibility, and (5) artistic performances. We present a proof-of-concept, wearable implementation based on a depth camera and report on a lab study comparing social acceptability, physical and mental demand, and user preference. A second study demonstrates a 94-99% recognition rate of our recognizers.

Most interactive public displays currently rely on flat screens. This form factor impacts how users (1) notice the public display (2) develop motivation and (3) (socially) interact with the public display. In this paper, we present Chained Displays, a combination of several screens to create different form factors for interactive public displays. We also present a design space based on two complementary concepts, Focus and Nimbus, to describe and compare chained display configurations. Finally, we performed a field study comparing three chained displays: Flat, Concave, and Hexagonal. Results show that Flat triggers the strongest honeypot effect, Hexagonal causes low social learning, and Concave triggers the smallest amount of simultaneously interacting users among other findings.

In this paper we present our findings from a lab and a field study investigating how passers-by notice the interactivity of public displays. We designed an interactive installation that uses visual feedback to the incidental movements of passers-by to communicate its interactivity. The lab study reveals: (1) Mirrored user silhouettes and images are more effective than avatar-like representations. (2) It takes time to notice the interactivity (approx. 1.2s). In the field study, three displays were installed during three weeks in shop windows, and data about 502 interaction sessions were collected. Our observations show: (1) Significantly more passers-by interact when immediately showing the mirrored user image (+90%) or silhouette (+47%) compared to a traditional attract sequence with call-to-action. (2) Passers-by often notice interactivity late and have to walk back to interact (the landing effect). (3) If somebody is already interacting, others begin interaction behind the ones already interacting, forming multiple rows (the honeypot effect). Our findings can be used to design public display applications and shop windows that more effectively communicate interactivity to passers-by.

Interaction with TV sets, set-top boxes or media centers strongly differs from interaction with personal computers: not only does a typical remote control suffer strong form factor limitations but the user may well be slouching in a sofa. In the face of more and more data, features, and services made available on interactive televisions, we propose to exploit the new capabilities provided by gesture-aware remote controls. We report the data of three user studies that suggest some guidelines for the design of a gestural vocabulary and we propose five novel interaction techniques. Study 1 reports that users spontaneously perform pitch and yaw gestures as the first modality when interacting with a remote control. Study 2 indicates that users can accurately select up to 5 items with eyes-free roll gestures. Capitalizing on our findings, we designed five interaction techniques that use either device motion, or button-based interaction, or both. They all favor the transition from novice to expert usage for selecting favorites. Study 3 experimentally compares these techniques. It reveals that motion of the device in 3D space, associated with finger presses at the surface of the device, is achievable, fast and accurate. Finally, we discuss the integration of these techniques into a coherent multimedia menu system.

This paper presents the design and evaluation of the Wavelet menu and its implementation on the iPhone. The Wavelet menu consists of a concentric hierarchical Marking menu using simple gestures. The novice mode, i.e. when the menu is displayed, is well adapted to the limited screen space of handheld devices because the representation of the menu hierarchy is inverted, the deeper submenu being always displayed at the center of the screen. The visual design is based on a stacking metaphor to reinforce the perception of the hierarchy and to help users to quickly understand how the technique works. The menu also supports submenu previsualization, a key property to navigate efficiently in a hierarchy of commands. The quantitative evaluation shows that the Wavelet menu provides an intuitive way for supporting efficient gesture-based navigation. The expert mode, i.e. gesture without waiting for the menu to pop-up, is another key property of the Wavelet menu: By providing stroke shortcuts, the Wavelet favors the selection of frequent commands in expert mode and makes eyes-free selection possible. A user experiment shows that participants are able to select commands, eyes-free, while walking.

We propose Radial-Stroke and Finger-Count Shortcuts, two techniques aimed at augmenting the menubar on multi-touch surfaces. We designed these multi-finger two-handed interaction techniques in an attempt to overcome the limitations of direct pointing on interactive surfaces, while maintaining compatibility with traditional interaction techniques. While Radial-Stroke Shortcuts exploit the well-known advantages of Radial Strokes, Finger-Count Shortcuts exploit multi-touch by simply counting the number of fingers of each hand in contact with the surface. We report the results of an experimental evaluation of our technique, focusing on expert-mode performance. Finger-Count Shortcuts outperformed Radial-Stroke Shortcuts in terms of both easiness of learning and performance speed.

Exploration and navigation in multimedia data hierarchies (e.g., photos, music) are frequent tasks on mobile devices. However, visualization and interaction are impoverished due to the limited size of the screen and the lack of precise input devices. As a result, menus on mobile devices do not provide efficient navigation as compared to many innovative menu techniques proposed for Desktop platforms. In this paper, we present Wavelet, the adaptation of the Wave menu for the navigation in multimedia data on iPhone. Its layout, based on an inverted representation of the hierarchy, is particularly well adapted to mobile devices. Indeed, it guarantees that submenus are always displayed on the screen and it supports efficient navigation by providing previsualization of the submenus.

This paper presents Flower menu, a new type of Marking menu that does not only support straight, but also curved gestures for any of the 8 usual orientations. Flower menus make it possible to put many commands at each menu level and thus to create as large a hierarchy as needed for common applications. Indeed our informal analysis of menu breadth in popular applications shows that a quarter of them have more than 16 items. Flower menus can easily contain 20 items and even more (theoretical maximum of 56 items). Flower menus also support within groups as well as hierarchical groups. They can thus favor breadth organization (within groups) or depth organization (hierarchical groups): as a result, the designers can lay out items in a very flexible way in order to reveal meaningful item groupings. We also investigate the learning performance of the expert mode of Flower menus. A user experiment is presented that compares linear menus (baseline condition), Flower menus and Polygon menus, a variant of Marking menus that supports a breadth of 16 items. Our experiment shows that Flower menus are more efficient than both Polygon and Linear menus for memorizing command activation in expert mode.

This paper presents TactiMote, a remote control with tactile feedback designed for navigating in long lists and catalogues. TactiMote integrates a joystick that allows 2D interaction with the thumb and a Braille cell that provides tactile feedback. This feedback is intended to help the selection task in novice mode and to allow for fast eyes-free navigation among favorite items in expert mode. The paper describes the design of the TactiMote prototype for TV channel selection and reports a preliminary experiment that shows the feasibility of the approach.

MOTIVATION: In high-throughput projects aiming to identify rare positives using a binary assay, smart-pooling constitutes an appealing strategy liable of significantly reducing the number of tests while correcting for experimental noise. In order to perform simulations for choosing an appropriate set of pools, and later to interpret the experimental results, the pool outcomes must be 'decoded'. The intuitive aim is clearly to identify the positives that gave rise to an observation, whether real or simulated. However, this goal is not well-formalized and has been the focus of very few studies. RESULTS: We first provide a clear combinatorial formalization of the 'decoding problem'. We then present interpool, an exact algorithm to solve this problem. An efficient implementation is freely available. Its usefulness is illustrated in the context of yeast-two-hybrid interactome mapping with the Shifted Transversal Design. AVAILABILITY: The implementation, licensed under the GNU GPL, can be downloaded from http://www-timc.imag.fr/Nicolas.Thierry-Mieg/.

In this paper we describe NAVRNA, an interactive system that enables biologists or researchers in bioinformatics to visualize, explore and edit RNA molecules. The key characteristics of NAVRNA are (1) to exploit multiple display surfaces (2) to enable the manipulation of both the 2D view of RNA called secondary structure, as well as the 3D view of RNA called tertiary structure while maintaining consistency between the two views, (3) to enable co-located synchronous collaborative manipulation of the RNA structures and (4) to provide two-handed interaction techniques for navigating and editing RNA structures and in particular a two-handed technique for bending the structure.

Ribonucleic Acid (RNA) is an important molecule which performs a wide range of functions in biological systems. We present a method for visualizing, exploring and editing RNA molecules. The method relies on the visualization of the 3D tertiary structure of RNA as well as on the automatic extraction and visualization of the 2D secondary structure. The 2D representation is used to navigate in the tertiary structure. The method has been implemented in a multi-surface interactive system, NAVRNA, that allows collaborative analysis of RNA.