Foldable phones naturally support displaying two applications side by side on adjacent screens. However, one-handed interaction with this dual-screen configuration faces two challenges: first, users must change grip to reach each screen; second, menus displayed on each screen take valuable screen real estate and can be difficult to reach. To address this, we investigated how the hinge of foldable phones can enrich tactile interaction and provide access to hinge menus, unifying interaction across both screens. In a first experiment, we examined how users hold a foldable phone at different fold angles and identified the thumb-accessible screen areas, validating that a hinge-based grip enhances reachability. In two subsequent experiments, we evaluated the feasibility and performance of hinge gestures, defined as touch inputs (tap, tap-tap or swipe) executed fully or partially on the hinge. Building on these findings, we designed hinge menus that combine hinge gestures for menu activation and item selection. Our final experiment identifies different hinge menus that outperform linear menus on adjacent screens. Our findings provide practical guidelines that can immediately inform and improve interaction on current foldable phones.

Embedding data into the physical environment using augmented reality (AR) is a practical approach for data visualization as it offers a large and flexible display space on or around the physical referent, i.e., the physical object to which the data is related. Yet, current interaction in such context is often performed using cumbersome dedicated devices, tiring mid-air gestures, or awkward on-body input. In this article, we investigate the use of the physical referent itself as a support for input interaction with an embedded space-time cube (STC) representation. Hence, we first identify the most promising mappings between the physical features of the referent (edges, faces, corners) and the STC dimensions. Then, we design three data selection techniques using the physical referent and compare them to mid-air gestures when performing selection tasks on the STC. Overall, our work demonstrates that using the physical referent to support input interaction with embedded data representations is an efficient and comfortable approach for data selection in standing and sitting situations.

See-through Head-Mounted Displays (HMDs) offer interesting opportunities to augment the interaction space around screens, especially around horizontal tabletops. In such context, HMDs can display surrounding vertical virtual windows to complement the tabletop content with data displayed in close vicinity. However, the effects of such combination on the visual acquisition of targets in the resulting combined display space have scarcely been explored. In this paper we conduct a study to explore visual acquisitions in such contexts, with a specific focus on the analysis of visual transitions between the horizontal tabletop display and the vertical virtual displays (in front and on the side of the tabletop). To further study the possible visual perception of the tabletop content out of the HMD and its impact on visual interaction, we distinguished two solutions for displaying information on the horizontal tabletop: using the see-through HMD to display virtual content over the tabletop surface (virtual overlay), i.e. the content is only visible inside the HMD’s FoV, or using the tabletop itself (tabletop screen). 12 participants performed visual acquisition tasks involving the horizontal and vertical displays. We measured the time to perform the task, the head movements, the portions of the displays visible in the HMD’s field of view, the physical fatigue and the user’s preference. Our results show that it is faster to acquire virtual targets in the front display than on the side. Results reveal that the use of the virtual overlay on the tabletop slows down the visual acquisition compared to the use of the tabletop screen, showing that users exploit the visual perception of the tabletop content on the peripheral visual space. We were also able to quantify when and to which extent targets on the tabletop can be acquired without being visible within the HMD's field of view when using the tabletop screen, i.e. by looking under the HMD. These results lead to design recommendations for more efficient, comfortable and integrated interfaces combining tabletop and surrounding vertical virtual displays.

Numerous interaction devices are designed through device composition. However, there is no conceptual support for this process and designers are left out to explore the space of combinations in an ad-hoc manner. In this paper we propose a design space for device composition, DECO, which focuses on the physical aspects of the composition. This design space is built around two main axes, namely Physical arrangement, which describes how elements are physically combined, and Physical manipulation, which describes how users manipulate each element. We first classify existing devices using our design space and then compare four of them to illustrate their similarities and differences. Using DECO, we design a new compound device: RPM2. This device is based on the combination of a regular mouse with the Roly-Poly Mouse. We describe in detail the user-centered iterative design process that leads to the final prototype. Finally we propose a set of design guidelines to assist the use of DECO for device composition.

We present the design and evaluation of the Roly-Poly Mouse (RPM), a rolling input device that combines the advantages of the mouse (position displacement) and of 3D devices (roll and rotation) to unify 2D and 3D interaction. Our first study explores RPM gesture amplitude and stability for different upper shapes (Hemispherical, Convex) and hand postures. 8 roll directions can be performed precisely and their amplitude is larger on Hemispherical RPM. As minor rolls affect translation, we propose a roll correction algorithm to support stable 2D pointing with RPM. We propose the use of compound gestures for 3D pointing and docking, and evaluate them against a commercial 3D device, the SpaceMouse. Our studies reveal that RPM performs 31% faster than the SpaceMouse for 3D pointing and equivalently for 3D rotation. Finally, we present a proof-of-concept integrated RPM prototype along with discussion on the various technical challenges to overcome to build a final integrated version of RPM.

As public displays integrate 3D content, Overview+Detail (O+D) interfaces on mobile devices will allow for a personal 3D exploration of the public display. In this paper we study the properties of mobile-based interaction with O+D interfaces on 3D public displays. We evaluate three types of existing interaction techniques for the 3D translation of the Detail view: touchscreen input, mid-air movement of the mobile device (Mid-Air Phone) and mid-air movement of the hand around the device (Mid-Air Hand). In a first experiment, we compare the performance and user preference of these three types of techniques with previous training. In a second experiment, we study how well the two mid-air techniques perform with no training or human help to imitate usual conditions in public context. Results reveal that Mid-Air Phone and Hand perform best with training. However, without training or human help Mid-Air Phone is more intuitive and performs better on the first trial. Interestingly, on both experiments users preferred Mid-Air Hand. We conclude with a discussion on the use of mobile devices to interact with public O+D interfaces.