Interacting with desktop GUIs often requires users to perform concurrent operations, such as scrolling through a document while selecting text or switching applications during file transfer. Although wheel mice support these tasks with a scroll wheel, touchpads lack this feature. In this paper, we introduce ThumbSwipe, an input technique for touchpads that enables concurrent operations by leveraging the thumb’s dexterity. We conducted two experiments to evaluate its performance for continuous and discrete input tasks. The first experiment demonstrated that ThumbSwipe achieves input throughputs comparable to those of cursor operations using the index or middle finger on a touchpad. The second experiment confirmed ThumbSwipe’s efficiency for discrete input tasks. Additionally, a third experiment assessed ThumbSwipe’s advantages in typical unimanual concurrent operation scenarios, showing improvements in touchpad user experience regarding task completion time, task load, and subjective usability ratings.

We present ExpanStick, a novel thumbstick that augments video game interaction using the force that the user applies at the physical limit of the gamepad. Gamers frequently manipulate a thumbstick leaning on the limit, or physical boundary around it, either to apply maximum input or to secure physical support, which makes the contact force an easily accessible input channel. The ExpanStick preserves the form factor and usability of the thumbstick, while expanding its input space. We built the ExpanStick and inserted it in a commercial gamepad as a proof-of-concept prototype. Using the prototype, we conducted a design workshop with console gamers to design the ExpanStick game interaction. Through demonstration of the interactions, we found the early use experience with the ExpanStick is positive and well accepted by the game players.

Users with visual impairments find it difficult to enjoy real-time 2D interactive applications on the touchscreen. Touchscreen applications such as sports games often require simultaneous recognition of and interaction with multiple moving targets through vision. To mitigate this issue, we propose ThroughHand, a novel tactile interaction that enables users with visual impairments to interact with multiple dynamic objects in real time. We designed the ThroughHand interaction to utilize the potential of the human tactile sense that spatially registers both sides of the hand with respect to each other. ThroughHand allows interaction with multiple objects by enabling users to perceive the objects using the palm while providing a touch input space on the back of the same hand. A user study verified that ThroughHand enables users to locate stimuli on the palm with a margin of error of approximately 13 mm and effectively provides a real-time 2D interaction experience for users with visual impairments.

In first-person shooters (FPS), professional players (a.k.a., Gosu) outperform amateur players. The secrets behind the performance of professional FPS players have been debated in online communities with many conjectures; however, attempts of scientific verification have been limited. We addressed this conundrum through a data-collection study of the gameplay of eight professional and eight amateur players in the commercial FPS Counter-Strike: Global Offensive. The collected data cover behavioral data from six sensors (motion capture, eye tracker, mouse, keyboard, electromyography armband, and pulse sensor) and in-game data (player data and event logs). We examined conjectures in four categories: aiming, character movement, physicality, and device and settings. Only 6 out of 13 conjectures were supported with statistically sufficient evidence.

Smartwatches are a convenient substitute for smartphones in hand-busy situations, but there is limited interactivity to one-handed use of wrist-worn devices. In this paper, we propose WristDial, a one-handed integer input technique which uses quantization of the angular distance of the wrist rotation with nonvisual feedback. Using tactile feedback, we present four techniques based on counting of tactile stimuli. With one of them, 94.44% average accuracy and 2.81 s average completion time were acquired in the task of entering a number from the range 1 to 10. We also evaluate the quantitative performance of the combination of interruptible speech and non-speech auditory feedback. Our results show that when speech feedback exists, adding a continuous change of pitch or clicking sound at the boundaries does not have a significant effect on the performance. Using only speech feedback, 95.47% accuracy and 1.45 s completion time were acquired.

Promisingly, driving is adapting to a Voice User Interface (VUI) that lets drivers utilize diverse applications with little effort. However, the VUI has innate usability issues, such as a turn-taking problem, a short-term memory workload, inefficient controls, and difficulty correcting errors. To overcome these weaknesses, we explored supplementing the VUI with tactile interaction. As an early result, we present the Voice+Tactile interactions that augment the VUI via multi-touch inputs and high-resolution tactile outputs. We designed various Voice+Tactile interactions to support different VUI interaction stages and derived four Voice+Tactile interaction themes: Status Feedback, Input Adjustment, Output Control, and Finger Feedforward. A user study showed that the Voice+Tactile interactions improved the VUI efficiency and its user experiences without incurring significant additional distraction overhead on driving. We hope these early results open new research questions to improve in-vehicle VUI with a tactile channel.

There are needs for pen interaction on a laptop, and the market sees many pen-enabled laptop products. Many of these laptops can be transformed into tablets, when pen interaction is needed. In a real situation, however, a workflow often requires both keyboard and pen interactions, and such a convertible feature may not be effective. In this study, we introduce MirrorPad, a novel interface device contained in a laptop for direct pen interaction. It is both a normal touchpad and a viewport for pen interaction with a mirrored region on the screen. We report findings and decisions obtained from the design iterations that we conducted with users to refine MirrorPad toward the final design. In the user study, MirrorPad showed the same performance as that of the laptop configuration during keyboard interaction and a performance similar to that of the tablet configuration during pen interaction. The user study results confirmed that MirrorPad effectively supports a workflow, which requires interspersed keyboard and pen interactions, thereby achieving its initial goal.

Force feedback has not been fully explored in modern gaming environments where a gamepad is the main interface. We developed various game interaction scenarios where force feedback through the thumbstick of the gamepad can be effective, and categorized them into five themes. We built a haptic device and control system that can support all presented interactions. The resulting device, FS-Pad, has sufficient fidelity to be used as a haptic game interaction design tool. To verify the presented interactions and effectiveness of the FS-Pad, we conducted a user study with game players, developers, and designers. The subjects used an FS-Pad while playing a demo game and were then interviewed. Their feedback revealed the actual needs for the presented interactions as well as insight into the potential design of game interactions when applying FS-Pad.

In countries where languages with non-Latin characters are prevalent, people use a keyboard with two language modes namely, the native language and English, and often experience mode errors. To diagnose the mode error problem, we conducted a field study and observed that 78% of the mode errors occurred immediately after application switching. We implemented four methods (Auto-switch, Preview, Smart-toggle, and Preview & Smart-toggle) based on three strategies to deal with the mode error problem and conducted field studies to verify their effectiveness. In the studies considering Korean-English dual input, Auto-switch was ineffective. On the contrary, Preview significantly reduced the mode errors from 75.1% to 41.3%, and Smart-toggle saved typing cost for recovering from mode errors. In Preview & Smart-toggle, Preview reduced mode errors and Smart-toggle handled 86.2% of the mode errors that slipped past Preview. These results suggest that Preview & Smart-toggle is a promising method for preventing mode errors for the Korean-English dual-input environment.

There are needs for pen interaction on a laptop, and the market sees many pen-enabled laptop products. Many of them can be transformed into tablets when pen interaction is needed. In a real situation, however, users use multiple applications simultaneously, and such a convertible feature may not be useful. We introduce MirrorPad, a novel interface device contained in a laptop for direct pen interaction. It is both a normal touchpad and a viewport for pen interaction with a mirrored region on the screen. MirrorPad allows users to perform pen annotation work and keyboard work interspersed in the same workflow while the device remains in the laptop-like configuration.

In film directing, a bias towards the representation of a particular gender can cause the audience to form a distorted stereotype of the gender role. The Bechdel test has been widely used to objectively judge the existence of such bias in films. However, because its analysis is based solely on the script of a film, the Bechdel test is incapable of considering the broad spectrum of bias that films can have as a visual medium. This study proposes a more comprehensive analysis system that quantifies the degree of bias in the visual representations of female and male characters in commercial films. By analyzing the image frames of a movie using the latest image analysis techniques, a total of 40 films were analyzed based on 8 quantitative indices. The result demonstrates that there exists a statistically significant difference in the visual representation of female and male characters. Specifically, female characters showed lower values in emotional diversity, spatial occupancy, and temporal occupancy compared to male characters in commercial films. Further, female characters were less likely to wear eyeglasses and also appeared more in static scenes, such as indoors.

As smartwatch functions expand, target selection among many items will probably become a common task. List search interfaces (LSIs) for a smartwatch use a prefix matching query to search for items, and need two modes because of the small screen size: query input mode and list navigation mode. Despite the modes, LSIs may be more efficient than list interfaces (LIs), which involve no text querying, for large pools. Actually, we could show that the LSI outperformed the LI for pool sizes over 60. However, we also found that the LSI users experience overhead when deciding whether to switch the modes or not. To reduce the overhead, we designed two auto-switching LSIs: input-length-based auto-switching LSI (IA-LSI) and list-size-based auto-switching LSI (LA-LSI). We could show both auto-switching LSIs outperformed the conventional LSI for pool sizes over 60. We also conducted experiments with the auto-switching LSIs for various pool sizes, and provided their results and guidelines for the optimal switching criteria for the LSIs.