Interacting with desktop GUIs often requires users to perform concurrent operations, such as scrolling through a document while selecting text or switching applications during file transfer. Although wheel mice support these tasks with a scroll wheel, touchpads lack this feature. In this paper, we introduce ThumbSwipe, an input technique for touchpads that enables concurrent operations by leveraging the thumb’s dexterity. We conducted two experiments to evaluate its performance for continuous and discrete input tasks. The first experiment demonstrated that ThumbSwipe achieves input throughputs comparable to those of cursor operations using the index or middle finger on a touchpad. The second experiment confirmed ThumbSwipe’s efficiency for discrete input tasks. Additionally, a third experiment assessed ThumbSwipe’s advantages in typical unimanual concurrent operation scenarios, showing improvements in touchpad user experience regarding task completion time, task load, and subjective usability ratings.

The paradigm of bare-hand interaction has become increasingly prevalent in Augmented Reality (AR) and Virtual Reality (VR) environments, propelled by advancements in hand tracking technology. However, a significant challenge arises in delivering haptic feedback to users’ hands, due to the necessity for the hands to remain bare. In response to this challenge, recent research has proposed an indirect solution of providing haptic feedback to the forearm. In this work, we present QuadStretcher, a skin stretch display featuring four independently controlled stretching units surrounding the forearm. While achieving rich haptic expression, our device also eliminates the need for a grounding base on the forearm by using a pair of counteracting tactors, thereby reducing bulkiness. To assess the effectiveness of QuadStretcher in facilitating immersive bare-hand experiences, we conducted a comparative user evaluation (n = 20) with a baseline solution, Squeezer. The results confirmed that QuadStretcher outperformed Squeezer in terms of expressing force direction and heightening the sense of realism, particularly in 3-DoF VR interactions such as pulling a rubber band, hooking a fishing rod, and swinging a tennis racket. We further discuss the design insights gained from qualitative user interviews, presenting key takeaways for future forearm-haptic systems aimed at advancing AR/VR bare-hand experiences.

We present ExpanStick, a novel thumbstick that augments video game interaction using the force that the user applies at the physical limit of the gamepad. Gamers frequently manipulate a thumbstick leaning on the limit, or physical boundary around it, either to apply maximum input or to secure physical support, which makes the contact force an easily accessible input channel. The ExpanStick preserves the form factor and usability of the thumbstick, while expanding its input space. We built the ExpanStick and inserted it in a commercial gamepad as a proof-of-concept prototype. Using the prototype, we conducted a design workshop with console gamers to design the ExpanStick game interaction. Through demonstration of the interactions, we found the early use experience with the ExpanStick is positive and well accepted by the game players.

Virtual reality (VR) environments are expected to become future workspaces. An effective bimanual 3D manipulation technique would be essential to support this vision. A ball-shaped tangible input device that can be rolled in a hand is known to be useful for 3D object manipulation because such devices allow users to utilize their finger dexterity. In this study, we further explored the potential of a rollable in-hand controller. First, we evaluated the effects of its form factor on user behavior and performance. Although the size and shape of a rollable controller are expected to influence user behavior and performance, their effects have not been empirically explored in prior works. Next, we evaluated a rollable controller on bimanual 3D assembly tasks. A rollable controller may incur a high mental load as it requires users to use finger dexterity; therefore, the benefit of using such a device in each hand is not obvious. We found that a 5 cm-diameter ball-shaped controller was the most effective among the sizes and forms that we considered, and that a pair of in-hand rollable controllers showed significantly faster completion time than a pair of VR controllers for complex bimanual assembly tasks involving frequent rotations.

Beyond a simple notification of incoming calls or messages, more complex information such as alphabets and digits can be delivered through spatiotemporal tactile patterns (STPs) on a wrist-worn tactile display (WTD) with multiple tactors. However, owing to the limited skin area and spatial acuity of the wrist, frequent confusions occur between closely located tactors, resulting in a low recognition accuracy. Furthermore, the accuracies reported in previous studies have mostly been measured for a specific posture and could further decrease with free arm postures in real life. Herein, we present Heterogeneous Stroke, a design concept for improving the recognition accuracy of STPs on a WTD. By assigning unique vibrotactile stimuli to each tactor, the confusion between tactors can be reduced. Through our implementation of Heterogeneous Stroke, the alphanumeric characters could be delivered with high accuracy (93.8% for 26 alphabets and 92.4% for 10 digits) across different arm postures.

Smartwatches are a convenient substitute for smartphones in hand-busy situations, but there is limited interactivity to one-handed use of wrist-worn devices. In this paper, we propose WristDial, a one-handed integer input technique which uses quantization of the angular distance of the wrist rotation with nonvisual feedback. Using tactile feedback, we present four techniques based on counting of tactile stimuli. With one of them, 94.44% average accuracy and 2.81 s average completion time were acquired in the task of entering a number from the range 1 to 10. We also evaluate the quantitative performance of the combination of interruptible speech and non-speech auditory feedback. Our results show that when speech feedback exists, adding a continuous change of pitch or clicking sound at the boundaries does not have a significant effect on the performance. Using only speech feedback, 95.47% accuracy and 1.45 s completion time were acquired.

Force feedback has not been fully explored in modern gaming environments where a gamepad is the main interface. We developed various game interaction scenarios where force feedback through the thumbstick of the gamepad can be effective, and categorized them into five themes. We built a haptic device and control system that can support all presented interactions. The resulting device, FS-Pad, has sufficient fidelity to be used as a haptic game interaction design tool. To verify the presented interactions and effectiveness of the FS-Pad, we conducted a user study with game players, developers, and designers. The subjects used an FS-Pad while playing a demo game and were then interviewed. Their feedback revealed the actual needs for the presented interactions as well as insight into the potential design of game interactions when applying FS-Pad.

A watch-back tactile display (WBTD) is an attractive output option due to its always-available nature. However, employing commonly-used vibration modality on a WBTD may result in a low efficiency since its stimulation area is relatively wide compared with the small contact area of a watch-back. We considered using a more localized tactile stimulus, a poke, to improve the efficiency of a WBTD. We built a WBTD consisting of overlapping 3×3 poke and vibrotactile tactor arrays so that it may be used either as a poke display or as a vibrotactile display. An experiment was conducted to optimize the parameters of poke stimuli, and its results revealed that four directional patterns were best recognized when poking depth was deepest (3 mm) and sensory saltation was exploited. In the next two experiments, we compared the information transfer capacities of the poke and vibrotactile displays. The information transfer capacity of the poke display (1.55 bits) was shown to be higher than that of the vibrotactile display (1.32 bits) in a simulated environment with the mental load of a primary task. This result confirmed our expectation that using a more localized tactile stimulus would improve the efficiency of a WBTD.

A tactile display on the back of a smartwatch is an attractive output option; however, its channel capacity is limited owing to the small contact area. In order to expand the channel capacity, we considered using two perceptually distinct types of stimuli, wind and vibration, together on the same skin area. The result is a multimodal tactile display that combines wind and vibration to create "colored" tactile sensations on the wrist. As a first step toward this goal, we conducted in this study four user experiments with a wind-vibration tactile display to examine different ways of combining wind and vibration: Individual, Sequential, and Simultaneous. The results revealed the sequential combination of wind and vibration to exhibit the highest potential, with an information transfer capacity of 3.29 bits. In particular, the transition of tactile modality was perceived at an accuracy of 98.52%. The current results confirm the feasibility and potential of a multimodal tactile display combining wind and vibration.

Tap-typing with a pen on a virtual keyboard is often used for intermittent text entry on a pen-equipped tablet; however, this has usability issues owing to the large size of the virtual keyboard. We speculated that tap-typing with a pen on a tablet may not require such a large keyboard, but there was no empirical study to help us to determine the best virtual keyboard size for it. Therefore, we conducted two experiments. In the first experiment, we compared virtual keyboards with different key sizes (3, 4, 5, 6, 8, 10, and 13 mm) using a text entry task. The results showed that keyboards with 6-, 8-, and 10-mm keys were the best when text entry speed, task load, and preference were considered. In the second experiment, a keyboard with 6-mm keys was compared with other conventional pen text entry options using a presentation slide editing task with a pen. The results showed that the keyboard with 6-mm keys performed better than the others in terms of preference, System Usability Scale (SUS) scores, and ease-of-learning scores in a Usefulness, Satisfaction, and Ease of use (USE) questionnaire.