The paradigm of bare-hand interaction has become increasingly prevalent in Augmented Reality (AR) and Virtual Reality (VR) environments, propelled by advancements in hand tracking technology. However, a significant challenge arises in delivering haptic feedback to users’ hands, due to the necessity for the hands to remain bare. In response to this challenge, recent research has proposed an indirect solution of providing haptic feedback to the forearm. In this work, we present QuadStretcher, a skin stretch display featuring four independently controlled stretching units surrounding the forearm. While achieving rich haptic expression, our device also eliminates the need for a grounding base on the forearm by using a pair of counteracting tactors, thereby reducing bulkiness. To assess the effectiveness of QuadStretcher in facilitating immersive bare-hand experiences, we conducted a comparative user evaluation (n = 20) with a baseline solution, Squeezer. The results confirmed that QuadStretcher outperformed Squeezer in terms of expressing force direction and heightening the sense of realism, particularly in 3-DoF VR interactions such as pulling a rubber band, hooking a fishing rod, and swinging a tennis racket. We further discuss the design insights gained from qualitative user interviews, presenting key takeaways for future forearm-haptic systems aimed at advancing AR/VR bare-hand experiences.

The palmrest area of laptops has the potential as an additional input space, considering its consistent palm contact during keyboard interaction. We propose Palmrest+, leveraging shear force exerted on the palmrest area. We suggest two input techniques: Palmrest Shortcut, for instant shortcut execution, and Palmrest Joystick, for continuous value input. These allow seamless and subtle input amidst keyboard typing. Evaluation of Palmrest Shortcut against conventional keyboard shortcuts revealed faster performance for applying shear force in unimanual and bimanual-manner with a significant reduction in gaze shifting. Additionally, the assessment of Palmrest Joystick against the laptop touchpad demonstrated comparable performance in selecting one- and two- dimensional targets with low-precision pointing, i.e., for short distances and large target sizes. The maximal hand displacement significantly decreased for both Palmrest Shortcut and Palmrest Joystick compared to conventional methods. These findings verify the feasibility and effectiveness of leveraging the palmrest area as an additional input space on laptops, offering promising enhanced typing-related user interaction experiences.

The ball-shaped device is considered an intuitive means for 3D interaction, but its effectiveness in object manipulation has not been fully explored yet. In this study, we explored the use of an in-hand ball-shaped controller for object manipulation in virtual reality. We developed a ball-shaped controller with pressure-sensing capabilities featuring specifically designed interactions for object manipulation, including selection, translation, rotation, and scaling. Evaluations are conducted on 6-DOF docking tasks involving translation and rotation, and on 7-DOF tasks that additionally include scaling. These tasks involved manipulating objects at both close and distant ranges. The results indicated that, while the performance of the ball-shaped controller for direct object manipulation was comparable to other methods, it significantly improved completion times and reduced task load for distant object manipulation. Furthermore, the ball-shaped controller effectively reduced wrist and arm movements and is the most preferred method.

Virtual reality (VR) environments are expected to become future workspaces. An effective bimanual 3D manipulation technique would be essential to support this vision. A ball-shaped tangible input device that can be rolled in a hand is known to be useful for 3D object manipulation because such devices allow users to utilize their finger dexterity. In this study, we further explored the potential of a rollable in-hand controller. First, we evaluated the effects of its form factor on user behavior and performance. Although the size and shape of a rollable controller are expected to influence user behavior and performance, their effects have not been empirically explored in prior works. Next, we evaluated a rollable controller on bimanual 3D assembly tasks. A rollable controller may incur a high mental load as it requires users to use finger dexterity; therefore, the benefit of using such a device in each hand is not obvious. We found that a 5 cm-diameter ball-shaped controller was the most effective among the sizes and forms that we considered, and that a pair of in-hand rollable controllers showed significantly faster completion time than a pair of VR controllers for complex bimanual assembly tasks involving frequent rotations.

Smartwatches are a convenient substitute for smartphones in hand-busy situations, but there is limited interactivity to one-handed use of wrist-worn devices. In this paper, we propose WristDial, a one-handed integer input technique which uses quantization of the angular distance of the wrist rotation with nonvisual feedback. Using tactile feedback, we present four techniques based on counting of tactile stimuli. With one of them, 94.44% average accuracy and 2.81 s average completion time were acquired in the task of entering a number from the range 1 to 10. We also evaluate the quantitative performance of the combination of interruptible speech and non-speech auditory feedback. Our results show that when speech feedback exists, adding a continuous change of pitch or clicking sound at the boundaries does not have a significant effect on the performance. Using only speech feedback, 95.47% accuracy and 1.45 s completion time were acquired.

Near-surface multi-finger tracking (NMFT) technology expands the input space of touchscreens by enabling novel interactions such as mid-air and finger-aware interactions. We present DeepFisheye, a practical NMFT solution for mobile devices, that utilizes a fisheye camera attached at the bottom of a touchscreen. DeepFisheye acquires the image of an interacting hand positioned above the touchscreen using the camera and employs deep learning to estimate the 3D position of each fingertip. We created two new hand pose datasets comprising fisheye images, on which our network was trained. We evaluated DeepFisheye's performance for three device sizes. DeepFisheye showed average errors with approximate value of 20 mm for fingertip tracking across the different device sizes. Additionally, we created simple rule-based classifiers that estimate the contact finger and hand posture from DeepFisheye's output. The contact finger and hand posture classifiers showed accuracy of approximately 83 and 90%, respectively, across the device sizes.

Tap-typing with a pen on a virtual keyboard is often used for intermittent text entry on a pen-equipped tablet; however, this has usability issues owing to the large size of the virtual keyboard. We speculated that tap-typing with a pen on a tablet may not require such a large keyboard, but there was no empirical study to help us to determine the best virtual keyboard size for it. Therefore, we conducted two experiments. In the first experiment, we compared virtual keyboards with different key sizes (3, 4, 5, 6, 8, 10, and 13 mm) using a text entry task. The results showed that keyboards with 6-, 8-, and 10-mm keys were the best when text entry speed, task load, and preference were considered. In the second experiment, a keyboard with 6-mm keys was compared with other conventional pen text entry options using a presentation slide editing task with a pen. The results showed that the keyboard with 6-mm keys performed better than the others in terms of preference, System Usability Scale (SUS) scores, and ease-of-learning scores in a Usefulness, Satisfaction, and Ease of use (USE) questionnaire.

The QWERTY keyboard has a wide form factor, whereas smartwatches frequently have round or square form factors. Even if extra space for an input field is considered, the aspect ratio of the QWERTY keyboard may not be optimal for a round or square smartwatch. We surmised that a narrower keyboard would be better able to utilize the screen area of a round or square smartwatch, thereby enabling larger keys and improved performance. Larger keys, however, may not necessarily result in better performance because of the reduced familiarity of a modified keyboard layout. To investigate this, we designed DiaQwerty keyboards, which are QWERTY variants with an aspect ratio of 10:7, and compared them with a plain QWERTY keyboard and a SplitBoard on round and square smartwatches. For a 33 mm round watch, DiaQwerty was comparable with QWERTY and was faster than SplitBoard. For a 24 mm x 30 mm square watch, DiaQwerty was faster than both QWERTY and SplitBoard. In the latter case, DiaQwerty achieved a 15% improvement over QWERTY. We concluded that the advantages of the enlarged keys outweighed the disadvantages of the reduced familiarity of the modified layout on a square watch.