Haptic perception on touchscreens varies across fingers, yet little is known about how finger identity and multi-finger use shape tactile discrimination and user experience. We conducted two experiments with four haptic feedback. In Experiment 1, right-handed participants explored each of the ten fingers individually under stationary and moving conditions. Experiment 2 examined two-finger sequences with same participants. Results showed that moving exploration enhanced accuracy, confidence, and enjoyment, while stationary touch increased cognitive and physical load, especially for weaker fingers such as the left ring and pinky. The right thumb and index consistently performed best. In dual-finger trials, moving exploration improved second-finger performance, and adjacent same-hand pairs (e.g., Left Index–Left Thumb, Right Thumb–Right Index) yielded higher synergy. These findings highlight the role of finger anatomy, motion, and coordination, and provide concrete guidelines on which fingers (or combinations) and exploration modes to assign for haptic surfaces that optimize accuracy, comfort, and engagement.

International audience

In this paper, we compare eyes-free input gestures to gestures produced in the presence or the absence of a visual feedback for a single-handed mobile device interaction. We consider two gesture sets ((i) freeform gestures (operands, rationally invariant and mnemonic gestures) and (ii) mark-based gestures (on-axis rectilinear segments)) and two gesture beginnings (free: gesture can start from anywhere on the screen and bezel: gesture should start from one of the edge). Our findings indicate that gestures made in an eyes-free interaction were geometrically different from gestures generated in the presence or absence of visual feedback; they generated more directional movements around the y axis and had a lower recognition rate, which in itself was quite good (95.53%). Our findings also indicate that bezel gestures generated more phone movement and were less appreciated than those initialized freely. Freeform gestures were also preferred over mark ones. Based on those results, we propose five guidelines for eyes-free stroke gesture design, ergonomics, and recognizer development.

Touch knobs usually require visual attention when performing a targeting task. This can draw the user’s visual focus to the device, which is not desirable if the visual attention is needed elsewhere. In this work, we investigated the use of haptic feedback for touch knobs when performing another primary task that saturates the attention. In an experiment, we compare three potential tactile feedback designs to a no-feedback variant for a circular targeting task, while performing an attention saturating primary task. Our findings indicate that two investigated designs, namely, gradual+hapticObject and adaptive+hapticObject, providing the user with both haptic feedback on the objects and continuous haptic feedback through the trajectory, can serve as a reliable and effective mechanism for increasing the speed and reducing the mental demand for the secondary task while reducing visual distraction for the primary task.

While many smartphones now include a waterfall (or curved edge) display, this edge is rarely exploited for input. In this paper, we introduce EdgeMark, a thumb-operated, one-handed, gestural menu to provide rapid and subtle access to commands. In a first study, we assess the movement range of the user during uni-manual input to define a spatial range for our EdgeMark menu. In a second study, we compare three potential EdgeMark designs to a marking menu variant to assess accuracy and performance. Our findings indicate that EdgeMark menus can serve as a reliable, effective mechanism for subtle, one-hand command invocation in modern, waterfall-edged smartphones.

Alongside vision and sound, hardware systems can be readily designed to support various forms of tactile feedback; however, while a significant body of work has explored enriching visual and auditory communication with interactive systems, tactile information has not received the same level of attention. In this work, we explore increasing the expressivity of tactile feedback by allowing the user to dynamically select between several channels of tactile feedback using variations in finger speed. In a controlled experiment, we show that a user can learn the dynamics of eyes-free tactile channel selection among different channels, and can reliable discriminate between different tactile patterns during multi-channel selection with an accuracy up to 90% when using two finger speed levels. We discuss the implications of this work for richer, more interactive tactile interfaces.

Previous research demonstrated the ability for users to accurately recognize tactile textures on mobile surface. However, the experiments were only run in a lab setting and the ability for users to recognize tactile texture in a real-world environment remains unclear. In this paper, we investigate the effects of physical challenging activities on tactile textures recognition. We consider five conditions: (1) seated on an office, (2) standing in an office, (3) seated in the tramway, (4) standing in the tramway and (5) walking in the street. Our findings indicate that when walking, performances deteriorated compared to the remainder conditions. However, despite this deterioration, the recognition rate stay higher than 82% suggesting that tactile texture could be effectively recognized and used by users in different physical challenges activities including walking.

Relative pointing through using tactile mobile device (such as tablets of phones) on a large display is a viable interaction technique (that we call Pad in this paper) which permits accurate pointing. However, limited device size has consequences on interaction. Such systems are known to often require clutching, which degrades performances. We present E-Pad, an indirect relative pointing interaction technique which takes benefit of the mobile tactile surface combined with its surrounding space. A user can perform continuous relative pointing starting on the pad then continuing in the free space around the pad, within arm's reach. As a first step toward E-Pad, we first introduce extended continuous relative pointing gestures and conduct a preliminary study to determine how people move their hand around the mobile device. We then conduct an experiment that compares the performance of E-Pad and Pad. Our findings indicate that E-Pad is faster than Pad and decreases the number of clutches without compromising accuracy. Our findings also suggest an overwhelming preference for E-Pad.

Bimanual input is frequently used on touch and tangible interaction on tabletop surfaces. Considering a composite task, such as moving a set of objects, attention, decision making and fine motor control have to be phased with the coordination of the two hands. However, attention demand is an important factor to design easy to learn and recall interaction techniques. This, determining what interaction modality demands less attention and which one performs better in these conditions is important to improve design. In this work, we present the first empirical results on this matter. We report that users are consistent in their assessments of the attention demand for both touch and tangible modalities, even under different hands synchronicity, and different population sizes and densities. Our findings indicate that the one hand condition and small populations demand less attention comparing to respectively, two hands conditions and bigger populations. Also, we show that tangible modality reduces significantly the attention when using two hands synchronous movements or when moving the sparse populations, decreases the movement time over touch modality, without compromising the traveled distance. We use our findings to outline a set of guidelines to assist touch and tangible design.

In mobile touchscreebn interaction, an important challenge is to find solutions to balance the size of individual widgets against the number of widgets needed during interaction. In this work, to address display space limitations, we explore the design of invisible off-screen toolbars (ether-toolbars) that leverage computer vision to expand application features by placing widgets adjacent to the display screen. We show how simple computer vision algorithms can be combined with a natural human ability to estimate physical placing to support highly accurate targeting. Our ether-toolbar design promises targeting accuracy approximating on-screen widget accuracy while significantly expanding the interaction space of mobile devices. Through two experiments, we examine off-screen content placement metaphors and off-screen precision of participants accessing these toolbars. From the data of the second experiment, we provide a basic model that reflects how users perceive mobile surroundings for ether-widgets and validate it. We also demonstrate a prototype system consisting of an inexpensive 3D printed mount for a mirror that supports ether-toolbar implementations. Finally, we discuss the implications of our work and potential design extensions that can increase the usability and the utility of ether-toolbars.

We investigate the relevance of surface haptic rendering techniques for tactile devices. We focus on the two major existing techniques and show that they have complementary benefits. The first one, called textsc{S}urface textsc{H}aptic textsc{O}bject (textsc{SHO}), which is based on finger position, is shown to be more suitable to render sparse textures; while the second one, called textsc{S}urface textsc{H}aptic textsc{T}exture (textsc{SHT}), which is based on finger velocity, is shown to be more suitable for dense textures and fast finger movements. We hence propose a new rendering technique, called textsc{L}ocalized textsc{H}aptic textsc{T}exture (textsc{LHT}), which is based on the concept of textit{taxel} considered as an elementary tactile information that is rendered on the screen. By using a grid of taxels to encode a texture, textsc{LHT} is shown to provide a consistent tactile rendering across different velocities for high density textures, and is found to reduce user textit{error rate} by up to 77.68% compared to textsc{SHO}.

We study users' perception of simultaneous tactile textures in ultrasonic devices. We investigate how relevant is providing the user with different complementary and simultaneous textures with respect to the different fingers that can be used to touch the surface. We show through a controlled experiment that users are able to distinguish the number of different textures independently of using fingers from one or two hands. However, our findings indicate that users are not able to differentiate between two different textures, that is to correctly identify each of them, when using fingers from the same hand. Based on our findings, we are then able to outline three relevant guidelines to assist multi-finger tactile feedback ergonomic and devices design.

Tactile devices with ultrasonic vibrations (based on squeeze film effect) using piezoelectric actuators are one of the existing haptic feedback technologies. In this study we have performed two psychophysical experiments on an ultrasonic haptic tablet, in order to find the minimum size of a tactile element on which all the users are able to perfectly identify different types of textures. Our results show that the spatial resolution of the tactile element on haptic touchscreen actually varies, depending on the number and types of tactile feedback information. A first experiment exhibits three different tactile textures, chosen as being easily recognized by users. We use these textures in a second experiment, and evaluate minimal spatial area on which the chosen set of textures can be recognized. Among other, we find the minimal size depends on the texture nature.

We present an interaction technique combining tactile actions and midair pointing to access out-of-reach content on large displays without the need to walk across the display. Users can start through a touch gesture on the display surface and finish midair by pointing to push content away or inversely to retrieve a content. The technique takes advantage of well-known semantics of pointing in human-to-human interaction. These, coupled with the semantics of proximal relations and deictic proxemics make the proposed technique very powerful as it leverages on well-understood human-human interaction modalities. Experimental results show this technique to outperform direct tactile interaction on dragging tasks. From our experience we derive four guidelines for interaction with large-scale displays.

We present a simple, two-step technique for recognizing multi-touch gesture input that is invariant to how users articulate gestures, i.e., by using one or two hands, one or multiple fingers, one or multiple strokes, synchronous or asynchronous stroke input. We introduce, for the first time in the gesture literature, a preprocessing step that is specific to multi-touch gestures (Match-Up) that clusters together similar strokes produced by different fingers, before running a gesture recognizer (Conquer). We report gains in recognition accuracy up to 10% leveraged by our new preprocessing step, which manages to construct a more adequate representation of multi-touch gestures in terms of key strokes. It is our hope that the Match-Up technique will add to the practitioners' toolkit of gesture preprocessing techniques, as a first step toward filling today's lack of algorithmic knowledge to process multi-touch input and leading toward the design of more efficient and accurate recognizers for touch surfaces.

We show that users are consistent in their assessments of the articulation difficulty of multi-touch gestures, even under the many degrees of freedom afforded by multi-touch input, such as (1) various number of fingers touching the surface, (2) various number of strokes that structure the gesture shape, and (3) single-handed and bimanual input. To understand more about perceived difficulty, we characterize gesture articulations captured under these conditions with geometric and kinematic descriptors computed on a dataset of 7,200 samples of 30 distinct gesture types collected from 18 participants. We correlate the values of the objective descriptors with users' subjective assessments of articulation difficulty and report path length, production time, and gesture size as the highest correlators (max Pearson's r=.95). We also report new findings about multi-touch gesture input, e.g., gestures produced with more fingers are larger in size and take more time to produce than single-touch gestures; bimanual articulations are not only faster than single-handed input, but they are also longer in path length, present more strokes, and result in gesture shapes that are deformed horizontally by 35% in average. We use our findings to outline a number of 14 guidelines to assist multi-touch gesture set design, recognizer development, and inform gesture-to-function mappings through the prism of the user-perceived difficulty of gesture articulation.