Surveys, especially Likert scale questionnaires, are widely used in HCI to capture users’ attitudes and experiences, but numeric ratings alone provide little insight into the rationales behind those ratings. While adding open-ended text fields or post-questionnaire interviews can elicit richer explanations, they often impose extra effort, leading to survey fatigue or recall bias. To address this gap, we proposed Rationalizer, an LLM-supported questionnaire system that generates contextualized rationales to support participants articulate their explanations alongside each Likert item and rating. In a user evaluation comparing it with the traditional questionnaire that included open-ended text fields, Rationalizer increased the percentage of Likert items with rationales, sustained participants’ willingness to provide self-input rationales, and supported them in articulating longer explanations within comparable writing durations as the study progressed. Quality analyses further showed that Rationalizer yielded higher-quality rationales (i.e., justification and relevance) than the traditional questionnaire. These findings highlight the potential of LLM-supported questionnaires to enrich Likert ratings with contextualized, richer explanations.

We present the Eye2Head Gesture (E2H), a gaze-based technique that integrates saccade gestures with eye-head coordination for target selection in virtual reality. It involves a back-and-forth saccadic gesture, aligning the gaze with the head forward direction and then returning to the target to complete the selection. The first study identified the parameters for implementing the E2H technique, including visualization of the relay zone (i.e., the area anchor to the head’s forward direction), relay zone size, and saccade gesture duration. Four variants of the E2H were developed, each differing in the shape and timing of the relay zone. The second study evaluated four E2H variants against Dwell and Eye&Head techniques, revealing that Region-shaped variant of E2H outperformed others in speed, error, and fatigue. This work contributes a gaze-based technique that harnesses eye-head coordination, using saccadic gestures to enhance the efficiency of target selection, particularly for those at large angular distances.

We present VisionPort, an enhanced Point-and-Teleport technique designed for navigating building-scale indoor virtual environments, specifically addressing the challenges posed by obstacles including walls, ceilings, and floors. VisionPort is available in two versions: VisionPort-essential and VisionPort-full. VisionPort-essential removes only the necessary portion of an obstacle targeted by the pointer, revealing the landing position behind it. In contrast, VisionPort-full allows for the complete removal of obstacles. Both versions enable users to seamlessly pass through barriers during the Point-and-Teleport locomotion process, while preserving the natural flow of navigation within the virtual building. Our evaluation, conducted in a multi-floor building setting, demonstrates that VisionPort improves navigation by reducing the time, head movement, and distance required to reach destinations. While VisionPort-full enhances efficiency, VisionPort-essential provides users with a greater sense of control, reflecting diverse preferences among participants.

We introduce JettingPointer , a skin-to-pointer interaction technique that enables accurate near-surface 2D touch input on minimal wearable devices, such as smart glasses. The core component is an airflow jet, embedded in the glasses frame, that functions as a haptic pointer by providing localized feedback to the finger skin during touch interactions performed above the frame. Users activate functions by aligning their finger phalanx with the airflow stream, guided by proprioception and a distinct point sensation. We optimized the airflow using fluid dynamics principles and characterized the required flow rate for stable tactile perception. In Study 1, we validated its perceptual clarity, confirming that a perceptible point sensation could be reliably achieved within 20 mm of the nozzle. In Study 2, participants performed eyes-free touch tasks with nearly three times greater accuracy when supported by haptic feedback (7.49 mm vs. 21.85 mm error). These findings demonstrate the potential of JettingPointer as a practical method for enabling proprioception-guided, near-surface interaction on compact wearables, with implications for expanding dense input in space-constrained form factors.

We present Surrogate Avatar, an adaptive telepresence method that enhances user mobility and situated co-presence in symmetric avatar-mediated communication. The system enables a remote user’s avatar to autonomously position itself in socially and environmentally appropriate locations within the local user’s space—based on spatial affordances, interactional norms, and environmental constraints—supporting fluid interaction without requiring a shared environmental context. Through a formative study, we derived key adaptation objectives and implemented them using a distributed optimization framework based on the AUIT system. The framework distributes adaptation tasks across server and client to balance responsiveness and computational efficiency. A user study involving both stationary and nomadic scenarios demonstrated consistently high usability and presence, with some limitations observed under walking conditions. An additional exploratory field study in a semi-structured public setting demonstrated the system’s viability beyond controlled lab conditions. These findings motivate future designs of mobile telepresence systems that dynamically adapt to spatial and conversational context while mitigating misunderstandings that can arise from asymmetric environmental awareness and supporting privacy-sensitive interaction.

The Point-and-Teleport locomotion in VR allows for instantaneous movement and helps minimize motion sickness. However, precise targeting can be difficult when the intended destination is obscured by an obstruction. This work presents Preview Teleport, an enhanced point-and-teleport technique that incorporates a preview window, enabling users to visualize the landing site behind obstructions in a seamless, single-step process. In a study of 28 participants, we investigated how window position, arc type, camera pose - defined as the three critical design aspects of the preview window - affect user performance. Additionally, we explored the impact of environment familiarity, simulated through a see-through effect. Our findings suggest Preview Teleport boosts efficiency, enhances spatial awareness and reduces fatigue, though it slightly compromises accuracy relative to methods without previews. This drop in accuracy is due to Preview Teleport's use of a preview window for directly targeting distant, obscured areas. In contrast, no-preview methods typically involve users first teleporting to a nearby visible location, then making a final adjustment to the obscured destination, allowing for more precise targeting. Within preview teleport designs, the window attachment to the controller, the camera pose at user-height, and the use of the standard parabolic-arc pointer are recommended. Notably, the rotated-align arc pointer demonstrates superior accuracy in targeting within familiar environments.

We introduce Seated-WIP, a footstep-based locomotion technique tailored for users seated in confined spaces such as on an airplane. It emulates real-world walking using forefoot or rearfoot in-place stepping, enhancing embodiment while reducing fatigue for prolonged interactions. Our footstep-locomotion maps users’ footstep motions to four locomotion actions: walking forward, turning-in-place, walking backward, and sidestepping. Our first study examined embodiment and fatigue levels across various sitting positions using forefoot, rearfoot, and fullfoot stepping methods. While all these methods effectively replicated walking, users favored the forefoot and rearfoot methods due to reduced fatigue. In our second study, we compared the footstep-locomotion to leaning- and controller-locomotion on a multitasking navigation task. Results indicate that footstep locomotion offers the best embodied sense of walking and has comparable fatigue levels to controller-locomotion, albeit with slightly reduced efficiency than controller-locomotion. In seated VR environments, footstep locomotion offers a harmonious blend of embodiment, fatigue mitigation, and efficiency.

Computational methods can potentially facilitate user interface design by complementing designer intuition, prior experience, and personal preference. Framing a user interface design task as a multi-objective optimization problem can help with operationalizing and structuring this process at the expense of designer agency and experience. While offering a systematic means of exploring the design space, the optimization process cannot typically leverage the designer’s expertise in quickly identifying that a given “bad” design is not worth evaluating. We here examine a cooperative approach where both the designer and optimization process share a common goal and work in partnership by establishing a shared understanding of the design space. We tackle the research question: How can we foster cooperation between the designer and a systematic optimization process in order to best leverage their combined strength? We introduce and present an evaluation of a cooperative approach that allows the user to express their design insight and work in concert with a multi-objective design process. We find that the cooperative approach successfully encourages designers to explore more widely in the design space than when they are working without assistance from an optimization process. The cooperative approach also delivers design outcomes that are comparable to an optimization process run without any direct designer input but achieves this with greater efficiency and substantially higher designer engagement levels.

The similarity effect refers to the tendency for people to be more easily influenced by others who resemble them in appearance. This phenomenon has been found to have positive impacts, including on the building of trust, that enrich the quality of communication (e.g., fluency or collaboration performance). While research has shown that the similarity effect occurs in screen-based communication platforms, it remains unclear how this phenomenon impacts user perceptions, especially of others' persuasiveness, in immersive environments such as virtual reality (VR). In this study, we adopted a mixed-methods approach to exploring how interaction with avatars of similar appearance to one's own self-representation influences conversations. Such similarity was operationalized as having three levels: identicality, moderate similarity, and dissimilarity. The study found that avatars of moderate similarity have the greatest persuasiveness; however, in both identicality and moderate similarity conditions, participants felt it was easier to communicate with and lower eeriness rating to avatars than in the dissimilarity condition. Multiple linear regression further revealed that users who had relatively low self-esteem and/or were relatively conscientious were more susceptible to the positive effect of appearance similarity on persuasiveness. We conclude that the similarity effect, especially when the similarity in question is moderate, could be leveraged to support persuasiveness in VR-based communication.

Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. In HCI, design optimization problems are often exceedingly complex, involving multiple objectives and expensive empirical evaluations. Model-based computational design algorithms assist designers by generating design examples during design, however they assume a model of the interaction domain. Black box methods for assistance, on the other hand, can work with any design problem. However, virtually all empirical studies of this human-in-the-loop approach have been carried out by either researchers or end-users. The question stands out if such methods can help designers in realistic tasks. In this paper, we study Bayesian optimization as an algorithmic method to guide the design optimization process. It operates by proposing to a designer which design candidate to try next, given previous observations. We report observations from a comparative study with 40 novice designers who were tasked to optimize a complex 3D touch interaction technique. The optimizer helped designers explore larger proportions of the design space and arrive at a better solution, however they reported lower agency and expressiveness. Designers guided by an optimizer reported lower mental effort but also felt less creative and less in charge of the progress. We conclude that human-in-the-loop optimization can support novice designers in cases where agency is not critical.

Virtual reality (VR) has increasingly been used in many areas, and the need to deliver notifications in VR is also expected to increase accordingly. However, untimely interruptions could largely impact the experience in VR. Identifying opportune times to deliver notifications to users allows for notifications to be scheduled in a way that minimizes disruption. We conducted a study to investigate the use of sensor data available on an off-the-shelf VR device and additional contextual information, including current activity and engagement of users, to predict opportune moments for sending notifications using deep learning models. Our analysis shows that using mainly sensor features could achieve 72% recall, 71% precision and 0.86 area under receiver operating characteristic (AUROC); performance can be further improved to 81% recall, 82% precision, and 0.93 AUROC if information about activity and summarized user engagement is included.

This paper explores directional kinesthetic force feedback on feet for enhancing foot-related experiences in virtual reality. We present Push-Ups, a pair of pneumatic shoes capable of outputting motions to users’ feet soles via the shape-forming shoe bases, to enhance users’ kinesthetic experience in a virtual ski. Each Push-Ups shoe comprises four pneumatic muscles in the base, allowing to present ten shape patterns and thus ten different motion outputs. Our first study validated the effectiveness of the motion output through a recognition study. The second study evaluated the user experience of the system on a virtual ski experience, which demonstrated that users perceived higher enjoyment and realism with the kinesthetic force feedback provided by Push-Ups.

Research has enabled virtual reality (VR) users to interact with the physical world by blending the physical world view into the virtual environment. However, current solutions are designed for specific use cases and hence are not capable of covering users’ varying needs for accessing information about the physical world. This work presents RealityLens, a user interface that allows users to peep into the physical world in VR with the reality lenses they deployed for their needs. For this purpose, we first conducted a preliminary study with experienced VR users to identify users’ needs for interacting with the physical world, which led to a set of features for customizing the scale, placement, and activation method of a reality lens. We evaluated the design in a user study (n=12) and collected the feedback of participants engaged in two VR applications while encountering a range of interventions from the physical world. The results show that users’ VR presence tends to be better preserved when interacting with the physical world with the support of the RealityLens interface.

Body-controlled avatars provide a more intuitive method to real-time control virtual avatars but require larger environment space and more user effort. In contrast, hand-controlled avatars give more dexterous and fewer fatigue manipulations within a close-range space for avatar control but provide fewer sensory cues than the body-based method. This paper investigates the differences between the two manipulations and explores the possibility of a combination. We first performed a formative study to understand when and how users prefer manipulating hands and bodies to represent avatars’ actions in current popular video games. Based on the top video games survey, we decided to represent human avatars’ motions. Besides, we found that players used their bodies to represent avatar actions but changed to using hands when they were too unrealistic and exaggerated to mimic by bodies (e.g., flying in the sky, rolling over quickly). Hand gestures also provide an alternative to lower-body motions when players want to sit during gaming and do not want extensive effort to move their avatars. Hence, we focused on the design of hand gestures and upper-body postures. We present Puppeteer, an input prototype system that allows players directly control their avatars through intuitive hand gestures and upper-body postures. We selected 17 avatar actions discovered in the formative study and conducted a gesture elicitation study to invite 12 participants to design best representing hand gestures and upper-body postures for each action. Then we implemented a prototype system using the MediaPipe framework to detect keypoints and a self-trained model to recognize 17 hand gestures and 17 upper-body postures. Finally, three applications demonstrate the interactions enabled by Puppeteer.

Smartwatches are a convenient substitute for smartphones in hand-busy situations, but there is limited interactivity to one-handed use of wrist-worn devices. In this paper, we propose WristDial, a one-handed integer input technique which uses quantization of the angular distance of the wrist rotation with nonvisual feedback. Using tactile feedback, we present four techniques based on counting of tactile stimuli. With one of them, 94.44% average accuracy and 2.81 s average completion time were acquired in the task of entering a number from the range 1 to 10. We also evaluate the quantitative performance of the combination of interruptible speech and non-speech auditory feedback. Our results show that when speech feedback exists, adding a continuous change of pitch or clicking sound at the boundaries does not have a significant effect on the performance. Using only speech feedback, 95.47% accuracy and 1.45 s completion time were acquired.

In this paper, we present a novel collaboration tool, OmniGlobeVR, which is an asymmetric system that supports communication and collaboration between a VR user (occupant) and multiple non-VR users (designers) across the virtual and physical platform. OmniGlobeVR allows designer(s) to explore the VR space from any point of view using two view modes: a 360° first-person mode and a third-person mode. In addition, a shared gaze awareness cue is provided to further enhance communication between the occupant and the designer(s). Finally, the system has a face window feature that allows designer(s) to share their facial expressions and upper body view with the occupant for exchanging and expressing information using nonverbal cues. We conducted a user study to evaluate the OmniGlobeVR, comparing three conditions: (1) first-person mode with the face window, (2) first-person mode with a solid window, and (3) third-person mode with the face window. We found that the first-person mode with the face window required significantly less mental effort, and provided better spatial presence, usability, and understanding of the partner's focus. We discuss the design implications of these results and directions for future research.

We present the Skin-Stroke Display, a system mounted on the lens inside the head-mounted display, which exerts subtle yet recognizable tactile feedback on the eye-ring using a motorized air jet. To inform our design of noticeable air-jet haptic feedback, we conducted a user study to identify absolute detection thresholds. Our results show that the tactile sensation had different sensitivity around the eyes, and we determined a standard intensity (8 mbar) to prevent turbulent airflow blowing into the eyes. In the second study, we asked participants to adjust the intensity around the eye for equal sensation based on standard intensity. Next, we investigated the recognition of point and stroke stimuli with or without inducing cognitive load on eight directions on the eye-ring. Our longStroke stimulus can achieve an accuracy of 82.6% without cognitive load and 80.6% with cognitive load simulated by the Stroop test. Finally, we demonstrate example applications using the skin-stroke display as the off-screen indicator, tactile I/O progress display, and tactile display.

This work introduces the concept of camera handling for a 360 camera and proposes Director-360, a 360 camera enhanced with two novel handling techniques. Pointer and field-of-view (FoV) are designed to explicitly and implicitly capture the 360 photographer’s subject of interest within the 360 media at capture time. Pointer lets users specify a subject of interest about the scene by directly pointing the 360 camera as if using the camera as a flashlight, while FoV captures the user’s subject of interest within the 360 scene by mapping the user’s face direction to the 360 media. We described an implementation using deep-learning algorithms. We also presented the Director-360 Editor, which incorporates the handling data to streamline the post-editing process. To understand how Director-360 helps to compose 360 media, a pilot study was carried out to create video storytelling in three target scenarios. The results and user feedback from a pilot study were reported.

This work presents Slice of Light, a visualization design created to enhance transparency and integrative transition between realities of Head-Mounted Display (HMD) users sharing the same physical environment. Targeted at reality-guests, Slice of Light's design enables guests to view other HMD users' interactions contextualized in their own virtual environments while allowing the guests to navigate among these virtual environments. In this paper, we detail our visualization design and the implementation. We demonstrate Slice of Light with a block-world construction scenario that involves a multi-HMD-user environment. VR developer and HCI expert participants were recruited to evaluate the scenario, and responded positively to Slice of Light. We discuss their feedback, our design insights, and the limitations of this work.

We present HMD Light, a proof-of-concept Head-Mounted Display (HMD) implementation that reveals the Virtual Reality (VR) user's experience in the physical environment to facilitate communication between VR and external users in a mobile VR context. While previous work externalized the VR user's experience through an on-HMD display, HMD Light places the display into the physical environment to enable larger display and interaction area. This work explores the interaction design space of HMD Light and presents four applications to demonstrate its versatility. Our exploratory user study observed participant pairs experience applications with HMD Light and evaluated usability, accessibility and social presence between users. From the results, we distill design insights for HMD Light and asymmetric VR collaboration.

Smartwatches enable the wrist to be used as an ideal location to provide always-available haptic notifications as they are constantly worn with direct contact with the skin. With the wrist straps, the haptic feedback can be extended to the full space around the wrist to provide more spatial and enriched feedback. With ThermalBracelet, we investigate thermal feedback as a haptic feedback modality around the wrist. We present three studies that lead to the development of a smartwatch-integratable thermal bracelet that stimulates six locations around the wrist. Our initial evaluation reports on the selection of the thermal module configurations. Secondly, with the selected six-module configuration, we explore its usability in a real-world scenarios such as walking and reading. Thirdly, we investigate its capability of providing spatio temporal feedback while engaged in distracting tasks. Finally we present application scenarios that demonstrates its usability.

Many crucial applications in the fields of filmmaking, game design, education, and cultural preservation - among others - involve the modeling, authoring, or editing of 3D objects and scenes. The two major methods of creating 3D models are 1) modeling, using computer software, and 2) reconstruction, generally using high-quality 3D scanners. Scanners of sufficient quality to support the latter method remain unaffordable to the general public. Since the emergence of consumer-grade RGBD cameras, there has been a growing interest in 3D reconstruction systems using depth cameras. However, most such systems are not user-friendly, and require intense efforts and practice if good reconstruction results are to be obtained. In this paper, we propose to increase the accessibility of depth-camera-based 3D reconstruction by assisting its users with augmented reality (AR) technology. Specifically, the proposed approach will allow users to rotate/move a target object freely with their hands and see the object being overlapped with its reconstructing model during the reconstruction process. As well as being more instinctual than conventional reconstruction systems, our proposed system will provide useful hints on complete 3D reconstruction of an object, including the best capturing range; reminder of moving and rotating the object at a steady speed; and which model regions are complex enough to require zooming-in. We evaluated our system via a user study that compared its performance against those of three other stateof- the-art approaches, and found our system outperforms the other approaches. Specifically, the participants rated it highest in usability, understandability, and model satisfaction.

Rendering haptic feedback in virtual reality is a common approach to enhancing the immersion of virtual reality content. However, current editing tools allow developers to access the haptic feedback only through physical contact with the actuators, making it difficult to fast iterate haptic interaction designs. This paper introduces SeeingHaptics, an authoring tool which visualizes haptic properties in 3D scenes. The active area of certain feedback is simulated with mesh shapes, while the 2D icons allow for indicating the type of haptic sensation. Our evaluation showed that SeeingHaptics helps developers rapidly create haptic feedbacks after a short training session.

We present TilePoP, a new type of pneumatically-actuated interface deployed as floor tiles which dynamically pop up by inflating into large shapes constructing proxy objects for whole-body interactions in Virtual Reality. TilePoP consists of a 2D array of stacked cube-shaped airbags designed with specific folding structures, enabling each airbag to be inflated into a physical proxy and then deflated down back to its original tile shape when not in use. TilePoP is capable of providing haptic feedback for the whole body and can even support human body weight. Thus, it allows new interaction possibilities in VR. Herein, the design and implementation of TilePoP are described in detail along with demonstrations of its applications and the results of a preliminary user evaluation conducted to understand the users' experience with TilePoP.

We present FaceWidgets, a device integrated with the backside of a head-mounted display (HMD) that enables tangible interactions using physical controls. To allow for near range-to-eye interactions, our first study suggested displaying the virtual widgets at 20 cm from the eye positions, which is 9 cm from the HMD backside. We propose two novel interactions, widget canvas and palm-facing gesture, that can help users avoid double vision and allow them to access the interface as needed. Our second study showed that displaying a hand reference improved performance of face widgets interactions. We developed two applications of FaceWidgets, a fixed-layout 360 video player and a contextual input for smart home control. Finally, we compared four hand visualizations against the two applications in an exploratory study. Participants considered the transparent hand as the most suitable and responded positively to our system.

We present Pull-Ups, a suspension kit that can suggest a range of body postures and thus enables various exercise styles of users perceiving the kinesthetic force feedback by suspending their weight with arm exertion during the interaction. Pull-Ups actuates the user's body to move up to 15 cm by pulling his or her hands using a pair of pneumatic artificial muscle groups. Our studies informed the discernible kinesthetic force feedbacks that were then exploited for the design of kinesthetic force feedback in three physical activities: kitesurfing, paragliding, and space invader. Our final study on user experiences suggested that a passive suspension kit alone added substantially to users' perceptions of realism and enjoyment (all above neutral) with passive physical support, while sufficient active feedback can further level them up. In addition, we found that both passive and active feedback of the suspension kit significantly reduced motion sickness in simulated kitesurfing and paragliding compared to when no suspension kit (thus no feedback) was provided. This work suggests that a passive suspension kit is cost-effective as a home exercise kit, while active feedback can further level up user experience, though at the cost of the installation (e.g., an air compressor in our prototype).

Virtual Reality (VR) sickness occurs when exposure to a virtual environment causes symptoms that are similar to motion sickness, and has been one of the major user experience barriers of VR. To reduce VR sickness, prior work has explored dynamic field-of-view modification and galvanic vestibular stimulation (GVS) that recou-ples the visual and vestibular systems. We propose a new approach to reduce VR sickness, called PhantomLegs, that applies alternating haptic cues that are synchronized to users' footsteps in VR. Our prototype consists of two servos with padded swing arms, one set on each side of the head, that lightly taps the head as users walk in VR. We conducted a three-session, multi-day user study with 30 participants to evaluate its effects as users navigate through a VR environment while physically being seated. Results show that our approach significantly reduces VR sickness during the initial exposure while remaining comfortable to users.

This work presents HapticSphere, a wearable spherical surface enabled by bridging a finger and the head-mounted display (HMD) with a passive string. Users perceive a physical support on a finger attached to a string, when extending their arm and reaching out to the string's maximum extension. This physical support assists users in precise touch interaction in the context of stationary and walking virtual or mixed-reality experiences. We propose three methods of attachment of the haptic string (directly on the head or on the body), and illustrate a novel single-step calibration algorithm that supports these configurations by estimating a grand haptic sphere, once a head-coordinated touch interaction is established. Two user studies were conducted to validate our approach and to compare the touch performance with physical support in sitting and walking conditions in the context of mobile mixed-reality scenarios. The results show that, in the walking condition, touch interaction with physical support significantly outperformed the visual-only condition.

Smart-rings are ideal for subtle and always-available haptic notifications due to their direct contact with the skin. Previous researchers have highlighted the feasibility of haptic technology in smart-rings and their promise in delivering noticeable stimulations by poking a limited set of planar locations on the finger. However, the full potential of poking as a mechanism to deliver richer and more expressive information on the finger is overlooked. With three studies and a total of 76 participants, we informed the design of PokeRing, a smart-ring capable of delivering information via stimulating eight different locations around the index finger's proximal phalanx. We report our evaluation of the performance of PokeRing in semi-realistic wearable conditions, (standing and walking), and its effective usage for information transfer with twenty-one spatio-temporal patterns designed by six interaction designers in a workshop. Finally, we present three applications that exploit PokeRing's notification usages.

We develop a Deep Learning-based Wearable Vision-system with Vibrotactile-feedback (DLWV2)to guide Blind and Visually Impaired (BVI)people to reach objects. The system achieves high accuracy in object detection and tracking in 3-D using an extended deep learning-based 2.5-D detector and a 3-D object tracker with the ability to track 3-D object locations even outside the camera field-of-view. We train our detector with a large number of images with 2.5-D object ground-truth (i.e., 2-D object bounding boxes and distance from the camera to objects). A novel combination of HTC Vive Tracker with our system enables us to automatically obtain the ground-truth labels for training while requiring very little human effort to set up the system. Moreover, our system processes frames in real-time through a client-server computing platform such that BVI people can receive realtime vibrotactile guidance. We conduct a thorough user study on 12 BVI people in new environments with object instances which are unseen during training. Our system outperforms the non-assistive guiding strategy with statistic significance in both time and the number of contacting irrelevant objects. Finally, the interview with BVI users confirms that our system with distance-based vibrotactile feedback is mostly preferred, especially for objects requiring gentle manipulation such as a bottle with water inside.

Navigation and mobility mechanics for virtual environments aim to be realistic or fun, but rarely prioritize the accuracy of movement. We propose PinchMove, a highly accurate navigation mechanic utilizing pinch gestures and manipulation of the viewport for confined environments that prefers accurate movement. We ran a pilot study to first determine the degree of simulator sickness caused by this mechanic, and a comprehensive user study to evaluate its accuracy in a virtual environment. We found that utilizing an 80° tunneling effect at a maximum speed of 15.18° per second was deemed suitable for PinchMove in reducing motion sickness. We also found our system to be at average, more accurate in enclosed virtual environments when compared to conventional methods. This paper makes the following three contributions: 1) We propose a navigation solution in near-field virtual environments for accurate movement, 2) we determined the appropriate tunneling effect for our method to minimize motion sickness, and 3) We validated our proposed solution by comparing it with conventional navigation solutions in terms of accuracy of movement. We also propose several use- case scenarios where accuracy in movement is desirable and further discuss the effectiveness of PinchMove.

This paper presents FacePush, a Head-Mounted Display (HMD) integrated with a pulley system to generate normal forces on a user's face in virtual reality (VR). The mechanism of FacePush is obtained by shifting torques provided by two motors that press upon a user's face via utilization of a pulley system. FacePush can generate normal forces of varying strengths and apply those to the surface of the face. To inform our design of FacePush for noticeable and discernible normal forces in VR applications, we conducted two studies to iden- tify the absolute detection threshold and the discrimination threshold for users' perception. After further consideration in regard to user comfort, we determined that two levels of force, 2.7 kPa and 3.375 kPa, are ideal for the development of the FacePush experience via implementation with three applications which demonstrate use of discrete and continuous normal force for the actions of boxing, diving, and 360 guidance in virtual reality. In addition, with regards to a virtual boxing application, we conducted a user study evaluating the user experience in terms of enjoyment and realism and collected the user's feedback.

The sensation of being able to feel the shape of an object when grasping it in Virtual Reality (VR) enhances a sense of presence and the ease of object manipulation. Though most prior works focus on force feedback on fingers, the haptic emulation of grasping a 3D shape requires the sensation of touch using the entire hand. Hence, we present Pop-up Prop on Palm (PuPoP), a light-weight pneumatic shape-proxy interface worn on the palm that pops several airbags up with predefined primitive shapes for grasping. When a user's hand encounters a virtual object, an airbag of appropriate shape, ready for grasping, is inflated by way of the use of air pumps; the airbag then deflates when the object is no longer in play. Since PuPoP is a physical prop, it can provide the full sensation of touch to enhance the sense of realism for VR object manipulation. For this paper, we first explored the design and implementation of PuPoP with multiple shape structures. We then conducted two user studies to further understand its applicability. The first study shows that, when in conflict, visual sensation tends to dominate over touch sensation, allowing a prop with a fixed size to represent multiple virtual objects with similar sizes. The second study compares PuPoP with controllers and free-hand manipulation in two VR applications. The results suggest that utilization of dynamically-changing PuPoP, when grasped by users in line with the shapes of virtual objects, enhances enjoyment and realism. We believe that PuPoP is a simple yet effective way to convey haptic shapes in VR.

Currently, "walkable" virtual reality (VR) is achieved by dedicating a room-sized space for VR activities, which is not shared with non-HMD users engaged in their own activities. To achieve the goal of allowing shared use of space for all users while overcoming the obvious difficulty of integrating use with those immersed in a VR experience, we present ShareSpace, a system that allows external users to communicate their needs for physical space to those wearing an HMD and immersed in their VR experience. ShareSpace works by allowing external users to place "shields" in the virtual environment by using a set of physical shield tools. A pad visualizer helps this process by allowing external users to examine the arrangement of virtual shields. We also discuss interaction techniques that minimize the interference between the respective activities of the HMD wearers and the other users of the same physical space. To evaluate our design, a user study was conducted to collect user feedback from participants in four trial scenarios. The results indicate that our ShareSpace system allows users to perform their respective activities with improved engagement and safety. In addition, this study shows that while the HMD users did perceive a considerable degree of interference due to the internal visual indications from the ShareSpace system, they were still more engaged in their VR experience than when interrupted by direct external physical interference initiated by external users.

One-handed input for smartwatches is crucial when users’ hands are occupied. A rotational motion is leveraged, which can be detected by built-in motion sensors in most smartwatches, as input to provide item selection. Users control a cursor by rotating the smartwatch into different orientations using the proposed item selection gestures. To understand human wrist dexterity and the ability to perform rotational motion input along each degree of freedom on smartwatches, a human-factor study was performed. Except dexterity, users’ consensus and agreement of user-defined rotational motion input gestures for selection and commitment on smartwatches in an exploratory user study were also observed. Based on both the studies, gestures roll and tilt are proposed as selection gestures, and shake and nod are used as commitment gestures. The item selection performance using the proposed gestures in 2D layouts in a user study was further evaluated. The study results showed that tilt-nod and tilt-shake are suitable for item numbers (4 × 3) and (5 × 5), respectively. Furthermore, due to high error rate for tilt-shake in (2 × 2) caused by false positive in shake detection, roll-shake with better social acceptance can be used as an alternative for smartwatches without a built-in front camera in (2 × 2).

Smart rings are an emerging wearable technology particularly suitable for discrete notifications based on haptic cues. Previous work mostly focused on tactile actuators that stimulate only specific skin receptors on the finger, resulting in limited information expressiveness. We propose tactoRing, a novel tactile display that, by dragging a small tactor on the skin around the finger, excites multiple skin areas resulting in more accurate cue recognition. In this paper, we present the hardware and a perception study to understand the ability of users to recognize eight distinct points around the finger. Moreover, we show two different techniques to encode information through skin-dragging motion with accuracy up to 94%. We finally showcase a set of applications that, by combining sequences of tactile stimuli, achieve higher expressiveness than prior methods.

Head Mounted Displays (HMDs) provide a promising opportunity for providing haptic feedback on the head for an enhanced immersive experience. In ThermoVR, we integrated five thermal feedback modules on the HMD to provide thermal feedback directly onto the user's face. We conducted evaluations with 15 participants using two approaches: Firstly, we provided simultaneously actuated thermal stimulations (hot and cold) as directional cues and evaluated the accuracy of recognition; secondly, we evaluated the overall immersive thermal experience that the users experience when provided with thermal feedback on the face. Results indicated that the recognition accuracy for cold stimuli were of approx. 89.5% accuracy while the accuracy for hot stimuli were 68.6%. Also, participants reported that they felt a higher level of immersion on the face when all modules were simultaneously stimulated (hot and cold). The presented applications demonstrate the ThermoVR's directional cueing and immersive experience.

A head-mounted display (HMD) immerses users in a virtual world, but separates them from outsiders in the real world. We present FrontFace, which is a novel HMD that combines an eye-tracker with a front-facing screen, to lower the communication barrier between HMD users and outsiders. The front-facing screen reveals user attention (e.g., the users eye motions) and user presence in the virtual or real world by displaying the scene in the virtual world or a skin background respectively, enabling eye-contact interactions between the HMD user and the outsiders. FrontFace has the following benefits. Firstly, it communicates the presence of the HMD user to outsiders; secondly, it reveals the player's visual attention by introducing the HMD users originally occluded eye motions, enabling outsiders to make sense of the HMD user's reaction in the virtual world or the real world. Three interactive techniques for the outsiders to initiate communication to HMD users are proposed: they are tap-trigger, hand-gesture trigger, and voice-trigger interactions. A small focus group provided feedback.

ForceClicks is a novel touch button input technique for consecutive clicking which incorporates touch force sensors. From force data of a single continuous touch over time, ForceClicks detects peaks and generates discrete clicks. Compared to typical button interaction, this is effective in a sense that consecutive clicks do not require finger positional movements. Additionally, stable force over a certain time threshold can trigger an alternate state, long press, and can be mapped to other actions. The usability of ForceClicks has been evaluated in terms of a) scattering level and b) efficiency. Results suggest higher stability than typical touch, especially when the task requires visual engagement on remote content. The relatively scatter-free characteristic of ForceClicks allows it to be applied on rapid clicking while gaming, and reduce of visual dedication allows easier control of external devices, and two applications, a shooting game and a number picker, are presented for demonstration.

We present Dwell+, a method that boosts the effectiveness of typical dwell selection by augmenting the passive dwell duration with active haptic ticks which promptly drives rapid switches of modes forward through the user's skin sensations. In this way, Dwell+ enables multi-level dwell selection using rapid haptic ticks. To select a mode from a button, users dwell-touch the button until the mode of selection is haptically prompted. Our haptic stimulation design consists of a short 10ms vibrotacile feedback that indicates a mode arriving and a break that separates consecutive modes. We first tested the effectiveness of 170ms, 150ms, 130ms, and 110ms intervals between modes for a 10-level selection. The results reveal that 3-beats-per-chunk rhythm design, e.g., displaying longer 25ms vibrations initially for all three modes, could potentially achieve higher accuracy. The second study reveals significant improvement wherein a 94.5% accuracy was achieved for a 10-level Dwell+ selection using the 170ms interval with 3-beats-per-chunk design, and a 93.82% rate of accuracy using the more frequent 150ms interval with similar chunks for 5-level selection. The performance of conducting touch and receiving vibration from disparate hands was investigated for our final study to provide a wider range of usage. Our applications demonstrated implementing Dwell+ across interfaces, such as text input on a smartwatch, enhancing touch space for HMDs, boosting modalities of stylus-based tool selection, and extending the input vocabulary of physical interfaces.

360-degree video contains a full field of environmental content. However, browsing these videos, either on screens or through head-mounted displays (HMDs), users consume only a subset of the full field of view per a natural viewing experience. This causes a search problem when a region-of-interest (ROI) in a video is outside of the current field of view (FOV) on the screen, or users may search for non-existing ROIs. We propose Outside-In, a visualization technique which re-introduces off-screen regions-of-interest (ROIs) into the main screen as spatial picture-in-picture (PIP) previews. The geometry of the preview windows further encodes a ROI's relative location vis-à-vis the main screen view, allowing for effective navigation. In an 18-participant study, we compare Outside-In with traditional arrow-based guidance within three types of 360-degree video. Results show that Outside-In outperforms in regard to understanding spatial relationship, the storyline of the content and overall preference. Two applications are demonstrated for use with Outside-In in 360-degree video navigation with touchscreens, and live telepresence.

Thumb-to-fingers interfaces augment touch widgets on fingers, which are manipulated by the thumb. Such interfaces are ideal for one-handed eyes-free input since touch widgets on the fingers enable easy access by the stylus thumb. This study presents DigitSpace, a thumb-to-fingers interface that addresses two ergonomic factors: hand anatomy and touch precision. Hand anatomy restricts possible movements of a thumb, which further influences the physical comfort during the interactions. Touch precision is a human factor that determines how precisely users can manipulate touch widgets set on fingers, which determines effective layouts of the widgets. Buttons and touchpads were considered in our studies to enable discrete and continuous input in an eyes-free manner. The first study explores the regions of fingers where the interactions can be comfortably performed. According to the comfort regions, the second and third studies explore effective layouts for button and touchpad widgets. The experimental results indicate that participants could discriminate at least 16 buttons on their fingers. For touchpad, participants were asked to perform unistrokes. Our results revealed that since individual participant performed a coherent writing behavior, personalized $1 recognizers could offer 92% accuracy on a cross-finger touchpad. A series of design guidelines are proposed for designers, and a DigitSpace prototype that uses magnetic-tracking methods is demonstrated.

This paper presents EdgeVib, a system of spatiotemporal vibration patterns for delivering alphanumeric characters on wrist-worn vibrotactile displays. We first investigated spatiotemporal pattern delivery through a watch-back tactile display by performing a series of user studies. The results reveal that employing a 2×2 vibrotactile array is more effective than employing a 3×3 one, because the lower-resolution array creates clearer tactile sensations in less time consumption. We then deployed EdgeWrite patterns on a 2×2 vibrotactile array to determine any difficulties of delivering alphanumerical characters, and then modified the unistroke patterns into multistroke EdgeVib ones on the basis of the findings. The results of a 24-participant user study reveal that the recognition rates of the modified multistroke patterns were significantly higher than the original unistroke ones in both alphabet (85.9% vs. 70.7%) and digits (88.6% vs. 78.5%) delivery, and a further study indicated that the techniques can be generalized to deliver two-character compound messages with recognition rates higher than 83.3%. The guidelines derived from our study can be used for designing watch-back tactile displays for alphanumeric character output.

This paper presents Cyclops, a single-piece wearable device that sees its user's whole body postures through an ego-centric view of the user that is obtained through a fisheye lens at the center of the user's body, allowing it to see only the user's limbs and interpret body postures effectively. Unlike currently available body gesture input systems that depend on external cameras or distributed motion sensors across the user's body, Cyclops is a single-piece wearable device that is worn as a pendant or a badge. The main idea proposed in this paper is the observation of limbs from a central location of the body. Owing to the ego-centric view, Cyclops turns posture recognition into a highly controllable computer vision problem. This paper demonstrates a proof-of-concept device, and an algorithm for recognizing static and moving bodily gestures based on motion history images (MHI) and a random decision forest (RDF). Four example applications of interactive bodily workout, a mobile racing game that involves hands and feet, a full-body virtual reality system, and interaction with a tangible toy are presented. The experiment on the bodily workout demonstrates that, from a database of 20 body workout gestures that were collected from 20 participants, Cyclops achieved a recognition rate of 79% using MHI and simple template matching, which increased to 92% with the more advanced machine learning approach of RDF.

This work presents WonderLens, a system of optical lenses and mirrors for enabling tangible interactions on printed paper. When users perform spatial operations on the optical components, they deform the visual content that is printed on paper, and thereby provide dynamic visual feedback on user interactions without any display devices. The magnetic unit that is embedded in each lens and mirror allows the unit to be identified and tracked using an analog Hall-sensor grid that is placed behind the paper, so the system provides additional auditory and visual feedback through different levels of embodiment, further enhancing the interactivity with the printed content on the physical paper.

This work presents LEaD, a helmet-based visual guidance system utilizing light movement in scooter drivers' peripheral vision for turn-by-turn navigation. A linear light strip mounted on a helmet navigates for scooter drivers using simple 1D light movement, which can be easily acquired and identified by peripheral vision with the on-going foveal vision task. User studies suggest that this novel system can effectively direct scooter drivers without introducing visual distractions in route-guided experiences.

In this paper, we explored eyes-free gesture interactions on palms, which enables users to interact with devices by drawing stroke gestures on palms without looking at palms. We conducted a 24-person user study to understand how users draw gestures on the palm with varying characteristics including regions, orientation and starting points. Based on the findings, we proposed two new interaction techniques for palm-based gesture interface. To explore and demonstrate the feasibility of the interaction, we implemented EyeWrist, a wrist-mounted prototype which detects gestures on palms by using an IR camera and laser-line projector. The preliminary evaluation revealed that EyeWrist enabled users to draw graffiti letter and multi-stroke gestures with above 90% accuracy and that both the concept of using palms as gesture interfaces for eyes-free input and the proposed two interaction techniques were appealing to users.

This paper presents CyclopsRing, a ring-style fisheye imaging wearable device that can be worn on hand webbings to en- able whole-hand and context-aware interactions. Observing from a central position of the hand through a fisheye perspective, CyclopsRing sees not only the operating hand, but also the environmental contexts that involve with the hand-based interactions. Since CyclopsRing is a finger-worn device, it also allows users to fully preserve skin feedback of the hands. This paper demonstrates a proof-of-concept device, reports the performance in hand-gesture recognition using random decision forest (RDF) method, and, upon the gesture recognizer, presents a set of interaction techniques including on-finger pinch-and-slide input, in-air pinch-and-motion input, palm-writing input, and their interactions with the environ- mental contexts. The experiment obtained an 84.75% recognition rate of hand gesture input from a database of seven hand gestures collected from 15 participants. To our knowledge, CyclopsRing is the first ring-wearable device that supports whole-hand and context-aware interactions.

This paper presents FlexiBend, an easily installable shape-sensing strip that enables interactivity of multi-part, deformable fabrications. The flexible sensor strip is composed of a dense linear array of strain gauges, therefore it has shape sensing capability. After installation, FlexiBend can simultaneously sense user inputs in different parts of a fabrication or even capture the geometry of a deformable fabrication.

We present TouchSense, which provides additional touchscreen input vocabulary by distinguishing the areas of users' finger pads contacting the touchscreen. It requires minimal touch input area and minimal movement, making it especially ideal for wearable devices such as smart watches and smart glasses. For example, users of a calculator application on a smart watch could tap normally to enter numbers, and tap with the right side of their fingers to enter the operators (e.g. , -, =). Results from two human-factor studies showed that users could tap a touchscreen with five or more distinct areas of their finger pads. Also, they were able to tap with more distinct areas closer to their fingertips. We developed a TouchSense smart watch prototype using inertial measurement sensors, and developed two example applications: a calculator and a text editor. We also collected user feedback via an explorative study.

This work describes a novel building block system for tangible interaction design, GaussBricks, which enables real-time constructive tangible interactions on portable displays. Given its simplicity, the mechanical design of the magnetic building blocks facilitates the construction of configurable forms. The form constructed by the magnetic building blocks, which are connected by the magnetic joints, allows users to stably manipulate with various elastic force feedback mechanisms. With an analog Hall-sensor grid mounted to its back, a portable display determines the geometrical configuration and detects various user interactions in real time. This work also introduce several methods to enable shape changing, multi-touch input, and display capabilities in the construction. The proposed building block system enriches how individuals interact with the portable displays physically.

This work presents GaussStones, a system of shielded magnetic tangibles design for supporting multi-token interactions on portable displays. Unlike prior works in sensing magnetic tangibles on portable displays, the proposed tangible design applies magnetic shielding by using an inexpensive galvanized steel case, which eliminates interference between magnetic tangibles. An analog Hall-sensor grid can recognize the identity of each shielded magnetic unit since each unit generates a magnetic field with a specific intensity distribution and/or polarization. Combining multiple units as a knob further allows for resolving additional identities and their orientations. Enabling these features improves support for applications involving multiple tokens. Thus, using prevalent portable displays provides generic platforms for tangible interaction design.

We present GaussBits, which is a system of the passive magnetic tangible designs that enables 3D tangible interactions in the near-surface space of portable displays. When a thin magnetic sensor grid is attached to the back of the display, the 3D position and partial 3D orientation of the GaussBits can be resolved by the proposed bi-polar magnetic field tracking technique. This portable platform can therefore enrich tangible interactions by extending the design space to the near-surface space. Since non-ferrous materials, such as the user's hand, do not occlude the magnetic field, interaction designers can freely incorporate a magnetic unit into an appropriately shaped non-ferrous object to exploit the metaphors of the real-world tasks, and users can freely manipulate the GaussBits by hands or using other non-ferrous tools without causing interference. The presented example applications and the collected feedback from an explorative workshop revealed that this new approach is widely applicable.

This work presents a novel and always-available nail mounted display known as NailDisplay. The proposed display augments the use of a finger by allowing for always-available visual feedback owing to its fast accessibility and binding user controls with the display, i.e. what you control is what you see (through the display). Potential benefits of NailDisplay are demonstrated in three applications: from displaying to combining it with user controls. In the first application, NailDisplay can reveal what is occluded under a finger touch, making it a solution to operate small UI elements. In the second application, NailDisplay is complementary to an imaginary interface, helping users to learn an imaginary interface (e.g., on the users' arms) and allowing them to reassure the interface when their memory of it becomes unclear. In the third application, NailDisplay is integrated with rich finger interactions, such as swiping in the air. We also report users' feedbacks gathered from an explorative user study.

We present FingerPad, a nail-mounted device that turns the tip of the index finger into a touchpad, allowing private and subtle interaction while on the move. FingerPad enables touch input using magnetic tracking, by adding a Hall sensor grid on the index fingernail, and a magnet on the thumbnail. Since it permits input through the pinch gesture, FingerPad is suitable for private use because the movements of the fingers in a pinch are subtle and are naturally hidden by the hand. Functionally, FingerPad resembles a touchpad, and also allows for eyes-free use. Additionally, since the necessary devices are attached to the nails, FingerPad preserves natural haptic feedback without affecting the native function of the fingertips. Through user study, we analyze the three design factors, namely posture, commitment method and target size, to assess the design of the FingerPad. Though the results show some trade-off among the factors, generally participants achieve 93% accuracy for very small targets (1.2mm-width) in the seated condition, and 92% accuracy for 2.5mm-width targets in the walking condition.

Recent research proposes augmenting capacitive touch pads with tangible objects, enabling a new generation of mobile applications enhanced with tangible objects, such as game pieces and tangible controllers. In this paper, we extend the concept to capacitive tangibles consisting of multiple parts, such as stackable gaming pieces and tangible widgets with moving parts. We achieve this using a system of wires and connectors inside each block that causes the capacitance of the bottom-most block to reflect the entire assembly. We demonstrate three types of tangibles, called CapStones, Zebra Dials and Zebra Sliders that work with current consumer hardware and investigate what designs may become possible as touchscreen hardware evolves.

We present TUIC, a technology that enables tangible interaction on capacitive multi-touch devices, such as iPad, iPhone, and 3M's multi-touch displays, without requiring any hardware modifications. TUIC simulates finger touches on capacitive displays using passive materials and active modulation circuits embedded inside tangible objects, and can be used with multi-touch gestures simultaneously. TUIC consists of three approaches to sense and track objects: spatial, frequency, and hybrid (spatial plus frequency). The spatial approach, also known as 2D markers, uses geometric, multi-point touch patterns to encode object IDs. Spatial tags are straightforward to construct and are easily tracked when moved, but require sufficient spacing between the multiple touch points. The frequency approach uses modulation circuits to generate high-frequency touches to encode object IDs in the time domain. It requires fewer touch points and allows smaller tags to be built. The hybrid approach combines both spatial and frequency tags to construct small tags that can be reliably tracked when moved and rotated. We show three applications demonstrating the above approaches on iPads and 3M's multi-touch displays.

This paper proposes convenient and useful approaches to automatically calibrate the projectors of a multi-resolution display. The proposed approaches estimate both the keystone effect and misalignment of the projections with an assistance of a color camera. Structured light patterns are employed to construct the geometric relationship between projectors and the projection surface, and then pre-warp the images so that they appear undistorted as a result. Experimental results demonstrate that the proposed approaches successfully reduce the human-effort and lower the calibration time of multi-resolution display calibration task.

In this paper, we explore the challenges in applying and investigate methodologies to improve direct-touch interaction on intangible displays. Direct-touch interaction simplifies object manipulation, because it combines the input and display into a single integrated interface. While traditional tangible display-based direct-touch technology is commonplace, similar direct-touch interaction within an intangible display paradigm presents many challenges. Given the lack of tactile feedback, direct-touch interaction on an intangible display may show poor performance even on the simplest of target acquisition tasks. In order to study this problem, we have created a prototype of an intangible display. In the initial study, we collected user discrepancy data corresponding to the interpretation of 3D location of targets shown on our intangible display. The result showed that participants performed poorly in determining the z-coordinate of the targets and were imprecise in their execution of screen touches within the system. Thirty percent of positioning operations showed errors larger than 30mm from the actual surface. This finding triggered our interest to design a second study, in which we quantified task time in the presence of visual and audio feedback. The pseudo-shadow visual feedback was shown to be helpful both in improving user performance and satisfaction.

This paper presents a new 3D model-based gesture tracking system for controlling multimedia player in an intuitive way. The motivation of this paper is to make home appliance aware of user's intention. This 3D model-based gesture tracking system adopts a Bayesian framework to track the user's 3D hand position and to recognize meaning of these postures for controlling 3D player interactively. To avoid the high dimensionality of the whole 3D upper body model, which may complicate the gesture tracking problem, our system applies a novel hierarchical tracking algorithm to improve the system performance. Moreover, this system applies multiple cues for improving the accuracy of tracking results. Based on the above idea, we have implemented a 3D hand gesture interface for controlling multimedia players. Experimental results have shown that the proposed system robustly tracks the 3D position of the hand and has high potential for controlling the multimedia player.

Many map systems are created to help the user finding a place or define a route to follow. Google Map extends the concept of "surfing the map" by adding a street view that allows the user to explore a place from real pictures, creating the same feeling of walking through the streets. The horizontal 2D map and vertical panoramic street view, however, cause usability problems, while operating with traditional computer mouse and keyboards and presenting by single vertical or horizontal display. This paper presents a new table system composed of a horizontal tabletop screen and a vertical screen. The map view and the street view are displayed on the horizontal and vertical displays of our system respectively. Users can place the tangible pawn on the 2D map to have direct access of the street view from the pawn's point of view. In the user study, we compare our system with a standard computer system in the navigation task. The results reported that our system improves the intuitiveness of use, efficiency of city exploring and ease of remembrance on spaces that are not familiar beforehand. We also discuss limitations of using tangible objects for map navigation.

This paper presents a programmable infrared (IR) technique that utilizes invisible, programmable markers to support interaction beyond the surface of a diffused-illumination (DI) multi-touch system. We combine an IR projector and a standard color projector to simultaneously project visible content and invisible markers. Mobile devices outfitted with IR cameras can compute their 3D positions based on the markers perceived. Markers are selectively turned off to support multi-touch and direct on-surface tangible input. The proposed techniques enable a collaborative multi-display multi-touch tabletop system. We also present three interactive tools: i-m-View, i-m-Lamp, and i-m-Flashlight, which consist of a mobile tablet and projectors that users can freely interact with beyond the main display surface. Early user feedback shows that these interactive devices, combined with a large interactive display, allow more intuitive navigation and are reportedly enjoyable to use.

Previous studies have outlined the advantages of multi-resolution large-area displays over their fixed-resolution counterparts, however the mobility of the focus region has up until the present time received little attention. To study this phenomenon further, we have developed a multi-resolution tabletop display system with a steerable high resolution focus region to compare the performance between steerable and fixed focus region systems under different working scenarios. We have classified these scenarios according to region of interest (ROI) with analogies to different eye movement types (fixed, saccadic, and pursuit ROI). Empirical data gathered during the course of a multi-faceted user study demonstrates that the steerable focus region system significantly outperforms the fixed focus region system. The former is shown to provide enhanced display manipulation and proves especially advantageous in cases where the user must maintain spatial awareness of the display content as is the case in which, within a single session, several regions of the display are to be visited.

In this technical demonstration, we showcase an interactive content presentation (ICP) system that integrates media-expressed-emotion-based composition, user-perceived preference feedback, and interactive digital art creation. ICP harmonizes the browsing of multimedia contents by presenting them in the form of music videos (photos, blog articles with accompanied music) based on their expressed emotion similarity. ICP facilitates content browsing by automatically and dynamically selecting the media to be played next in real time, responding to user's preference feedback measured from physiological signals. In addition, ICP enhances the enjoyments of content browsing by incorporating interactive digital art creation. ICP achieves these goals by properly integrating recent researches on media-expressed emotion classification,cross-media composition, and physiological signal processing.

This article presents "i-m-Top", a rear-projection interactive multi-resolution tabletop display system. This table allows the user to perform regular and long-term works such as reading and writing of documents. The design of the tabletop system well meets the following concerns: (1) provide movable high-resolution perception of the tabletop, (2) avoid the shadowing effect by rear projection, (3) reserve space for the user to place their legs with comfort and (4) allow interactions on the tabletop by bare hands and a pen.

Crystal balls are generally considered as media to perform divination or fortune-telling. These imaginations are mainly from some fantasy films and fiction, in which an augur can see into the past, the present, or the future through a crystal ball. With the distinct impressions, crystal ball has revealed itself as a perfect interface for the users to access and to manipulate visual media in an intuitive, imaginative and playful manner. We developed an interactive visual display system named Magic Crystal Ball (MaC Ball). MaC Ball is a spherical display system, which allows the users to see a virtual object/scene appearing inside a transparent sphere, and to manipulate the displayed content with barehanded interactions. Interacting with MaC Ball makes the users feeling acting with magic power. With MaC Ball, user can manipulate the display with touch and hover interactions. For instance, the user waves hands above the ball, causing clouds blowing from bottom of the ball, or slides fingers on the ball to rotate the displayed object. In addition, the user can press single finger to select an object or to issue a button. MaC Ball takes advantages on the impressions of crystal balls, allowing the users acting with visual media following their imaginations. For applications, MaC Ball has high potential to be used for advertising and demonstration in museums, product launches, and other venues.