We present Lattice Menu, a gaze-based marking menu utilizing a lattice of visual anchors that helps perform accurate gaze pointing for menu item selection. Users who know the location of the desired item can leverage target-assisted gaze gestures for multilevel item selection by looking at visual anchors over the gaze trajectories. Our evaluation showed that Lattice Menu exhibits a considerably low error rate (~1%) and a quick menu selection time (1.3-1.6 s) for expert usage across various menu structures (4 × 4 × 4 and 6 × 6 × 6) and sizes (8, 10 and 12°). In comparison with a traditional gaze-based marking menu that does not utilize visual targets, Lattice Menu showed remarkably (~5 times) fewer menu selection errors for expert usage. In a post-interview, all 12 subjects preferred Lattice Menu, and most subjects (8 out of 12) commented that the provisioning of visual targets facilitated more stable menu selections with reduced eye fatigue.

This work explores the design of marking menus for gaze-based AR/VR menu selection by expert and novice users. It first identifies and explains the challenges inherent in ocular motor control and current eye tracking hardware, including overshooting, incorrect selections, and false activations. Through three empirical studies, we optimized and validated design parameters to mitigate these errors while reducing completion time, task load, and eye fatigue. Based on the findings from these studies, we derived a set of design guidelines to support gaze-based marking menus in AR/VR. To overcome the overshoot errors found with eye-based expert marking menu behaviour, we developed StickyPie, a marking menu technique that enables scale-independent marking input by estimating saccade landing positions. An evaluation of StickyPie revealed that StickyPie was easier to learn than the traditional technique (i.e., RegularPie) and was 10% more efficient after 3 sessions.

Text entry is expected to be a common task for smart glass users, which is generally performed using a touchpad on the temple or by a promising approach using eye tracking. However, each approach has its own limitations. For more efficient text entry, we present the concept of gaze-assisted typing (GAT), which uses both a touchpad and eye tracking. We initially examined GAT with a minimal eye input load, and demonstrated that the GAT technology was 51% faster than a two-step touch input typing method (i.e.,M-SwipeBoard: 5.85 words per minute (wpm) and GAT: 8.87 wpm). We also compared GAT methods with varying numbers of touch gestures. The results showed that a GAT requiring five different touch gestures was the most preferred, although all GAT techniques were equally efficient. Finally, we compared GAT with touch-only typing (SwipeZone) and eye-only typing (adjustable dwell) using an eye-trackable head-worn display. The results demonstrate that the most preferred technique, GAT, was 25.4% faster than the eye-only typing and 29.4% faster than the touch-only typing (GAT: 11.04 wpm, eye-only typing: 8.81 wpm, and touch-only typing: 8.53 wpm).

As smartwatch functions expand, target selection among many items will probably become a common task. List search interfaces (LSIs) for a smartwatch use a prefix matching query to search for items, and need two modes because of the small screen size: query input mode and list navigation mode. Despite the modes, LSIs may be more efficient than list interfaces (LIs), which involve no text querying, for large pools. Actually, we could show that the LSI outperformed the LI for pool sizes over 60. However, we also found that the LSI users experience overhead when deciding whether to switch the modes or not. To reduce the overhead, we designed two auto-switching LSIs: input-length-based auto-switching LSI (IA-LSI) and list-size-based auto-switching LSI (LA-LSI). We could show both auto-switching LSIs outperformed the conventional LSI for pool sizes over 60. We also conducted experiments with the auto-switching LSIs for various pool sizes, and provided their results and guidelines for the optimal switching criteria for the LSIs.

The QWERTY keyboard has a wide form factor, whereas smartwatches frequently have round or square form factors. Even if extra space for an input field is considered, the aspect ratio of the QWERTY keyboard may not be optimal for a round or square smartwatch. We surmised that a narrower keyboard would be better able to utilize the screen area of a round or square smartwatch, thereby enabling larger keys and improved performance. Larger keys, however, may not necessarily result in better performance because of the reduced familiarity of a modified keyboard layout. To investigate this, we designed DiaQwerty keyboards, which are QWERTY variants with an aspect ratio of 10:7, and compared them with a plain QWERTY keyboard and a SplitBoard on round and square smartwatches. For a 33 mm round watch, DiaQwerty was comparable with QWERTY and was faster than SplitBoard. For a 24 mm x 30 mm square watch, DiaQwerty was faster than both QWERTY and SplitBoard. In the latter case, DiaQwerty achieved a 15% improvement over QWERTY. We concluded that the advantages of the enlarged keys outweighed the disadvantages of the reduced familiarity of the modified layout on a square watch.

While smart glasses make information more accessible in mobile scenarios, entering text on these devices is still difficult. In this paper, we suggest using a smartwatch as an indirect input device for smart glasses text entry. With the watch-glasses combination, users do not need to lift the arm to touch the glasses nor need to carry a special external input device. To prove the feasibility of the suggested combination, we implemented two text entry methods: a modified version of SwipeBoard, which we adapted for the suggested combination, and HoldBoard, which we newly designed and implemented specifically for the suggested combination. We evaluated the performances of the two text entry methods through two user studies, and could show that they are faster than prior art for smart glasses text entry in a seated condition. A further study showed that they are competitive with the prior art also in a walking condition.

Small touchscreen interfaces such as a smartwatch have usability problems due to the small screen. One solution to these problems is to utilize the space around the smartwatch as continuous input space for the touchscreen interface. We defined four steps for a gesture that starts on the touchscreen and continues in the air. The goal of this definition was to bring the experience of large touchscreen devices into a smartwatch usage. We compared design options for the four steps and made decisions for the options based on the results of four user experiments. We expect that gestures designed based on these decisions will be both easy to learn and robust.