Voice assistants (VAs) are typically evaluated through task performance metrics and self-report questionnaires, but people’s voices themselves carry rich paralinguistic cues that reveal affect, effort, and interaction breakdowns. We present a within-subjects study (N=49) that systematically compared three VA personas across three usage scenarios to investigate whether speech-derived audio features can serve as a proxy for user experience (UX). Participants’ speech was analyzed for temporal, spectral, and linguistic markers, alongside standardized UX measures, brief mood and stress ratings, and a post-study questionnaire. We found correlations between specific speech features and self-reported satisfaction and experience. Furthermore, a machine learning model trained on speech features achieved promising accuracy in classifying UX levels, indicating that this might be a reasonable alternative to self-report instruments. Our findings establish speech as a viable, real-time signal for implicitly measuring UX and point toward adaptive VUIs that respond dynamically to emotional and usability-related vocal cues.

What if future dining involved eating robots? We explore this question through a playful and poetic experiential dinner theater: a tangible design fiction staged as a 2052 Paris restaurant where diners consume a biohybrid flying robot in place of the banned delicacy of ortolan bunting. Moving beyond textual or visual speculation, our “dinner-in-the-drama” combined performance, ritual, and multisensory immersion to provoke reflection on sustainability, ethics, and cultural identity. Six participants from creative industries engaged as diners and role-players, responding with curiosity, discomfort, and philosophical debate. They imagined biohybrids as both plausible and unsettling—raising questions of sentience, symbolism, and technology adoption that extend beyond conventional sustainability framings of synthetic meat. Our contributions to HCI are threefold: (i) a speculative artifact that stages robots as food, (ii) empirical insights into how people negotiate cultural and ethical boundaries in post-natural eating, and (iii) a methodological advance in embodied, multisensory design fiction.

Conversational agents (CAs) increasingly detect users’ emotions, yet deciding how to respond, especially to negative affect, remains a central design challenge. We conducted a role-switching study in which participants reply as the CAs to simulated users expressing anger, sadness, or fear. Results reveal systematic, gender-linked patterns: most male participants favored a neutral, affect-balanced stance and prioritized clarification or task progress, whereas most female participants produced a wider range of non-neutral responses, more often using explicit empathy, reassurance, and reflective listening. We also observe differences in de-escalation phrasing, validation timing, and follow-up questioning across scenarios. These findings indicate that strategies for handling negative emotions vary with user characteristics and context. Based on these findings, we argue for adaptive CA response policies that calibrate first-turn acknowledgment and information-gathering, tailoring prosody and wording to emotional context in order to support de-escalation, perceived understanding, and user trust.

Cross-lingual meetings have become essential for global collaboration, yet current translation technologies often strip away vocal identity — the unique speaker characteristics that convey nuance and social presence. While generic text-to-speech (TTS) provides basic intelligibility, it creates a disconnect between speakers and their translated voices, potentially undermining engagement and comprehension. This paper investigates whether voice cloning technology can bridge this gap by preserving speaker identity in real-time translation. We present a controlled study comparing four voice conditions in meeting interpretation: original speech, gender-neutral TTS, gender-matched TTS, and voice cloning. Through a within-subjects experiment with 45 participants, we demonstrate that voice cloning significantly reduces mental workload (p <.001) and enhances user experience across pragmatic quality (p <.001), hedonic quality (p <.001), and overall satisfaction (p <.001) compared to traditional TTS. While original speech maintained advantages in naturalness, voice cloning achieved superior intelligibility, social impression, and user preference. Qualitative analysis revealed that participants valued voice cloning for preserving speaker identity and improving conversation tracking in multi-speaker scenarios. Our findings suggest that identity-preserving translation represents a significant advancement for cross-lingual communication systems, offering both cognitive and social benefits. We conclude with design implications for integrating voice cloning into meeting platforms while addressing ethical considerations around consent and transparency.

Smartphone usage in public spaces can raise privacy concerns, in terms of shoulder surfing and unintended camera capture. In real-world public space settings, we investigated the impact of tangible privacy-enhancing tools (here: screen filter and camera slider) on smartphone users’ reported privacy perception, behavioral adaptations, usability and social dynamics. We conducted a mixed-method, in-the-wild study (N = 22) using off-the-shelf smartphone privacy tools. We investigated subjective behavioral transition by combining questionnaires with semi-structured interviews. Participants used the screen filter and the camera slider for two weeks; they reported changes in attitude and behavior after using a screen filter including screen visibility and comfort when using phones publicly. They explained decreased privacy-protective behaviors, such as actively covering their screens, suggesting a shift in perceived risk. Qualitative findings about the camera slider suggested underlying psychological mechanisms, including privacy awareness and concerns about social perception, while also offering insights regarding the tools’ effectiveness.

Climate change data is complex and difficult to interpret, which may hinder public participation in mitigation efforts. This study examines whether gaze cueing can enhance the accessibility of this information by directing visual attention in the climate information-based collaboration task. Twenty participants collaborated in groups of four on a climate-related task while wearing mobile eye trackers. In the experimental condition, participants received gaze cues in an instructional video before performing the task, the purpose of which was to choose the best strategy to mitigate climate change. Statistical analysis using the Levenshtein distance on scanpaths showed that gaze cues fostered attention synchronization. The findings suggest that gaze cues in groups effectively guide attention through complex climate information, which is the first step to its comprehension.

This paper presents a novel approach to visual attention training for novice tram drivers by combining a tram driving simulator with eye-tracking-enabled Augmented Reality (AR) glasses. Based on previous eye-tracking studies and interviews with tram driving instructors, we identified issues related to the attention distribution of novice drivers. That is, we found that driver field-of-view areas critical to tram-driving safety, are often neglected by novices. For instance, such areas enable drivers to check the right-side mirror before turning right. The training system proposed here aims to direct novice visual attention to critical areas by highlighting overlooked regions, displaying directional arrows, or providing auditory cues from the left or right side. Integration of real-time eye tracking into AR glasses supports continuous monitoring of trainee gaze patterns, offering instructors valuable information on trainee visual behavior. Such real-time data allows for dynamic adaptation of training cues, ensuring that visual attention guidance is both relevant and personalized. The proposed system has the potential to enhance situational awareness among tram drivers.

What will people experience when drones become common in home environments? How will their functions and distances impact human experiences? To explore the potential usage of indoor drones, we conducted a mixed-methods study (N=60) on the reported perceptions of a small flying robot. We employed a factorial experimental design, involving four intended drone functions (camera, education, pet, unknown) at two distances (near, far). Our findings suggest that intended functions significantly influence participants’ perceptions. Among the functions examined, participants found the camera useful but annoying, and the pet useless but pleasant. The education emerged as the most favored function, while the unknown function was the least preferred one. Based on these findings, we discuss implications for designing positive interactions between humans and indoor drones, considering aspects such as context, transparency, privacy, technical factors, and personalization.

Traditionally, sports commentators provide viewers with diverse information, encompassing in-game developments and player performances. Yet young adult football viewers increasingly use mobile devices for deeper insights during football matches. Such insights into players on the pitch and performance statistics support viewers’ understanding of game stakes, creating a more engaging viewing experience. Inspired by commentators’ traditional roles and to incorporate information into a single platform, we developed AiCommentator, a Multimodal Conversational Agent (MCA) for embedded visualization and conversational interactions in football broadcast video. AiCommentator integrates embedded visualization, either with an automated non-interactive or with a responsive interactive commentary mode. Our system builds upon multimodal techniques, integrating computer vision and large language models, to demonstrate ways for designing tailored, interactive sports-viewing content. AiCommentator’s event system infers game states based on a multi-object tracking algorithm and computer vision backend, facilitating automated responsive commentary. We address three key topics: evaluating young adults’ satisfaction and immersion across the two viewing modes, enhancing viewer understanding of in-game events and players on the pitch, and devising methods to present this information in a usable manner. In a mixed-method evaluation (n=16) of AiCommentator, we found that the participants appreciated aspects of both system modes but preferred the interactive mode, expressing a higher degree of engagement and satisfaction. Our paper reports on our development of AiCommentator and presents the results from our user study, demonstrating the promise of interactive MCA for a more engaging sports viewing experience. Systems like AiCommentator could be pivotal in transforming the interactivity and accessibility of sports content, revolutionizing how sports viewers engage with video content.

Advancements in Natural Language Processing (NLP) and Computer Vision (CV) are revolutionizing how we experience sports broadcasting. Traditionally, sports commentary has played a crucial role in enhancing viewer understanding and engagement with live games. Yet, the prospects of automated commentary, especially in light of these technological advancements and their impact on viewers’ experience, remain largely unexplored. This paper elaborates upon an innovative automated commentary system that integrates NLP and CV to provide a multimodal experience, combining auditory feedback through text-to-speech and visual cues, known as italicizing, for real-time in-game commentary. The system supports color commentary, which aims to inform the viewer of information surrounding the game by pulling additional content from a database. Moreover, it also supports play-by-play commentary covering in-game developments derived from an event system based on CV. As the system reinvents the role of commentary in sports video, we must consider the design and implications of multimodal artificial commentators. A focused user study with eight participants aimed at understanding the design implications of such multimodal artificial commentators reveals critical insights. Key findings emphasize the importance of language precision, content relevance, and delivery style in automated commentary, underscoring the necessity for personalization to meet diverse viewer preferences. Our results validate the potential value and effectiveness of multimodal feedback and derive design considerations, particularly in personalizing content to revolutionize the role of commentary in sports broadcasts.

Augmented Reality (AR) could improve maritime collaboration by facilitating human factors like situation awareness and decision-making.However, it is not clear in which ways AR can be beneficial and for which maritime operations.This paper addresses the need for a systematic literature review of state-of-the-art ways AR can facilitate maritime collaboration.Following the PRISMA statement, we searched for the constructs "AR," "maritime operation," and "collaboration" in the Web of Science and IEEE Xplore databases.Out of 691 search results, we retained 32 publications for analysis.Ten publications from other sources were included.Our results provide a review of AR applications for the maritime operations of ship navigation, personal navigation, maritime construction, maritime maintenance and inspection, and other.We suggest that research focuses on bringing promising AR applications to higher technology readiness levels, learning from other industries, applying robust evaluation methods, and grounding more on human factors like decision-making, situation awareness, and communication.

RedirectedDoors is a visuo-haptic door-opening redirection technique in VR, and it has shown promise in its ability to efficiently compress the physical space required for a room-scale VR experience. However, its previous implementation has only supported laboratory experiments with a single door opening at a fixed location. To significantly expand this technique for room-scale VR, we have developed RedirectedDoors+, a robot-based system that permits consecutive door-opening redirection with haptics. Specifically, our system is mainly achieved with the use of three components: (1) door robots, a small number of wheeled robots equipped with a doorknob-like prop, (2) a robot-positioning algorithm that arbitrarily positions the door robots to provide the user with just-in-time haptic feedback during door opening, and (3) a user-steering algorithm that determines the redirection gain for every instance of door opening to keep the user away from the boundary of the play area. Results of simulated VR exploration in six virtual environments reveal our system's performance relative to user walking speed, paths, and number of door robots, from which we derive its usage guidelines. We then conduct a user study ($N=12$) in which participants experience a walkthrough application using the actual system. The results demonstrate that the system is able to provide haptic feedback while redirecting the user within a limited play area.

Augmented reality (AR) is emerging in visual search tasks for increasingly immersive interactions with virtual objects. We propose an AR approach providing visual and audio hints along with gaze-assisted instant post-task feedback for search tasks based on mobile head-mounted display (HMD). The target case was a book-searching task, in which we aimed to explore the effect of the hints together with the task feedback with two hypotheses. H1: Since visual and audio hints can positively affect AR search tasks, the combination outperforms the individuals. H2: The gaze-assisted instant post-task feedback can positively affect AR search tasks. The proof-of-concept was demonstrated by an AR app in HMD and a comprehensive user study (n=96) consisting of two sub-studies, Study I (n=48) without task feedback and Study II (n=48) with task feedback. Following quantitative and qualitative analysis, our results partially verified H1 and completely verified H2, enabling us to conclude that the synthesis of visual and audio hints conditionally improves the AR visual search task efficiency when coupled with task feedback.

When flying robots are used in close-range interaction with humans, the noise they generate, also called consequential sound, is a critical parameter for user acceptance. We conjecture that there is a benefit in adding natural sounds to noisy domestic drones. To test our hypothesis experimentally, we carried out a mixed-methods research study (N = 56) on reported user perception of a sonified domestic flying robot with three sound conditions at three distances. The natural sounds studied were, respectively, added to the robot’s inherent noises during flying; namely, a bird song and a rain sound, plus a control condition of no added sound. The distances studied were set according to proxemics; namely, near , middle , and far . Our results show that adding bird song or rain sound affects the participants’ perceptions, and the proxemic distances play a nonnegligible role. For instance, we found that participants liked the bird condition the most when the drone was at far , while they disliked the same sound the most when at near . We also found that participants’ perceptions strongly depended on their associations and interpretations deriving from previous experience. We derived six concrete design recommendations.

This article examines the attentional mechanism of in-person collaboration by means of System Dynamics-based simulations using an eye tracking experiment. Three experimental conditions were tested: in-person collaboration, remote collaboration, and single user. We hypothesized that collaboration focuses users’ attention on key information facilitating decision-making. Collaborating participants dwelt longer on key elements of the simulation than single users. Moreover, in-person collaboration and single users yielded a strategy of decision-making similar to an optimal strategy. Finally, in-person collaboration was less cognitively demanding and of higher quality. The contribution of this article is a deeper understanding of how in-person collaboration on a large display can help users focus their visual attention on the most important areas. With this novel understanding, we believe collaborative systems designers will be better equipped to design more effective attention-guiding mechanisms in remote collaboration systems. The present work has the potential to advance the study of collaborative, interactive technologies.

As a cutting edge technique requiring high-precision equipment, augmented reality (AR) and its users are influenced by the ambient environment. With the tremendous effect brought by COVID-19 pandemic, most people have shifted from on-site working to remote working. In this study, we propose an initial solution to explore the impact of COVID-19 pandemic on AR users working in these two situations. We develop a prototype application facilitated with gamification process in which users are requested to play an AR game in headset both in on-site and remote working environments. This game, which is highly dependent on the ambient environment, enables people to memorize, distinguish, and place virtual objects when immersing themselves into different surroundings with distinct distractors. We envision to conduct more user studies investigating how COVID-19 affects AR users, which could lead to more in-depth studies in the future.

Augmented reality (AR) is today becoming more widely utilized as it allows for interacting with the virtual objects. In this study, we propose an head-mounted display (HMD) AR system supporting book searching with visual cues. The visual cue is represented as a light-green blob hinting users for task completion, which significantly strengthens the overall performance. The system is implemented by using Microsoft HoloLens 2. The proof-of-concept version of the proposed solution is demonstrated in a pilot user study (n=8) comprising an experimental group (with visual cues, n=4) and a control group (without visual cues, n=4), followed by quantitative analysis of task completion time (TCT) and NASA task load index (TLX). The results show that our proposed HMD AR solution improved the task performance and had cognitive benefits for book searching tasks.

We propose RedirectedDoors, a novel space-efficient technique for redirection in VR focused on door-opening behavior. This technique manipulates the user’s walking direction by rotating the entire virtual environment (VE) at a certain angular ratio of the door being opened. This ratio is called door rotation gain. At the same time, the virtual door’s position is kept unmanipulated so that a realistic door-opening user experience can be ensured. We designed and implemented the rotational manipulation algorithm and two types of door-opening interfaces; with and without a doorknob-type passive haptic prop. We then conducted a user study (N = 12) to investigate redirection performance and user feedback as we examined three independent variables: door rotation gain, door-opening interface, and door-opening direction (push/pull). From the results, the estimated detection thresholds generally showed a higher space efficiency of redirection with our technique. Our results also showed that providing the haptic feedback led to a higher noticeability of redirection, but at the same time supported a higher subjective sense of realism and less discomfort. Following our results, we discuss which combinations of gain and door-opening direction can jointly provide lower noticeability and higher acceptability.

Virtual Reality (VR) headsets can open opportunities for users to accomplish complex tasks on large virtual displays using compact and portable devices. However, interacting with such large virtual displays using existing interaction techniques might cause fatigue, especially for precise manipulation tasks, due to the lack of physical surfaces. To deal with this issue, we explored the design of VXSlate, an interaction technique that uses a large virtual display as an expansion of a tablet. We combined a user’s head movements as tracked by the VR headset, and touch interaction on the tablet. Using VXSlate, a user head movements positions a virtual representation of the tablet together with the user’s hand, on the large virtual display. This allows the user to perform fine-tuned multi-touch content manipulations. In a user study with seventeen participants, we investigated the effects of VXSlate on users in problem-solving tasks involving content manipulations at different levels of difficulty, such as translation, rotation, scaling, and sketching. As a baseline for comparison, off-the-shelf touch-controller interactions were used. Overall, VXSlate allowed participants to complete the task with completion times and accuracy that are comparable to touch-controller interactions. After an interval of use, VXSlate significantly reduced users’ time to perform scaling tasks in content manipulations, as well as reducing perceived effort. We reflected on the advantages and disadvantages of VXSlate in content manipulation on large virtual displays and explored further applications within the VXSlate design space.

Human-drone interaction is a growing topic of interest within HCI research. Researchers propose many innovative concepts for drone applications, but much of this research does not incorporate knowledge on existing applications already adopted by professionals. This limits the validity of said research. To address this limitation, we present our findings from an in-depth interview study with 10 professional drone pilots. Our participants were armed with significant experience and qualifications – pertinent to both drone operations and a set of applications covering diverse industries. Our findings have resulted in design recommendations that should inform both ends and means of human-drone interaction research. These include, but are not limited to: safety-related protocols, insights from domain-specific use cases, and relevant practices outside of hands-on flight.

It is our great pleasure to welcome you to this issue of the Proceedings of the ACM on Human-Computer Interaction, the second to focus on the contributions from the research community Interactive Surfaces and Spaces (ISS). Interactive Surfaces and Spaces increasingly pervade our everyday life, appearing in various sizes, shapes, and application contexts, offering a rich variety of ways to interact. This diverse research community explores the design, development, and use of new and emerging interactive surface technologies and interactive spaces. The call for articles for this issue on ISS attracted 77 submissions, from all over the world. This issue has 23 papers, 4 submitted in February 2021 and 19 submitted in July 2021. After the winter round, 4 (total of 19 articles, 21.1%) articles were accepted and 5 (26.3%) articles required major revisions. After the summer round, 19 (total of 58 articles, 32.8%) articles were accepted, and 18 (31,0%) articles required major revisions. The editorial committee worked hard over the two iterations of the review process, winter and summer rounds, to arrive at final decisions. In total, counting both the winter and the summer rounds, 23 articles (total of 77 articles, 29.9%) were accepted. All authors of the accepted articles are invited to present at the ISS conference from November 14--17, 2021. This issue exists because of the dedicated volunteer effort of 31 senior editors who served as Associate Chairs (ACs), 105 expert reviewers in the winter round, and 206 expert reviewers in the summer round to ensure high quality and insightful reviews for all articles. Reviewers and committee members were kept constant for papers that submitted to both rounds. The Editorial Board is presented here: https://iss.acm.org/2021/organization/editorial_board

Mobile vocabulary learning interfaces typically present material only in auditory and visual channels, underutilizing the haptic modality. We explored haptic-integrated learning by adding free-form digital annotation to mobile vocabulary learning interfaces. Through a series of pilot studies, we identified three design factors: annotation mode, presentation sequence, and vibrotactile feedback, that influence recall in haptic-integrated vocabulary interfaces. These factors were then evaluated in a within-subject comparative study using a digital flashcard interface as baseline. Results using a 84-item vocabulary showed that the 'whole word' annotation mode is highly effective, yielding a 24.21% increase in immediate recall scores and a 30.36% increase in the 7-day delayed scores. Effects of presentation sequence and vibrotactile feedback were more transient; they affected the results of immediate tests, but not the delayed tests. We discuss the implications of these factors for designing future mobile learning applications.

The First International Workshop on "Intelligence Cross-Data Analytics and Retrieval" (ICDAR'20) welcomes any theoretical and practical works on intelligence cross-data analytics and retrieval to bring the smart-sustainable society to human beings. We have witnessed the era of big data where almost any event that happens is recorded and stored either distributedly or centrally. The utmost requirement here is that data came from different sources, and various domains must be harmonically analyzed to get their insights immediately towards giving the ability to be retrieved thoroughly. These emerging requirements lead to the need for interdisciplinary and multidisciplinary contributions that address different aspects of the problem, such as data collection, storage, protection, processing, and transmission, as well as knowledge discovery, retrieval, and security and privacy. Hence, the goal of the workshop is to attract researchers and experts in the areas of multimedia information retrieval, machine learning, AI, data science, event-based processing and analysis, multimodal multimedia content analysis, lifelog data analysis, urban computing, environmental science, atmospheric science, and security and privacy to tackle the issues as mentioned earlier.

The design space of social drones, where autonomous flyers operate in close proximity to human users or bystanders, is distinct from use cases involving a remote human operator and/or an uninhabited environment; and warrants foregrounding human-centered design concerns. Recently, research on social drones has followed a trend of rapid growth. This paper consolidates the current state of the art in human-centered design knowledge about social drones through a review of relevant studies, scaffolded by a descriptive framework of design knowledge creation. Our analysis identified three high-level themes that sketch out knowledge clusters in the literature, and twelve design concerns which unpack how various dimensions of drone aesthetics and behavior relate to pertinent human responses. These results have the potential to inform and expedite future research and practice, by supporting readers in defining and situating their future contributions. The materials and results of our analysis are also published in an open online repository that intends to serve as a living hub for a community of researchers and designers working with social drones.

While facilitated brainstorming is a proven ideation method for professional teams, distributed teams are currently not able to enjoy its benefits. As workers are shifting towards collaborating in distributed settings, understanding how interactive systems can support facilitated brainstorming is becoming necessary to ensure that distributed teams remain creative. To address this challenge we designed, implemented, and evaluated DigiMetaplan---an interactive surface-based system for distributed facilitated brainstorming for co-located and remote users. The design of DigiMetaplan was inspired by a widely-used facilitated brainstorming method called Metaplan, where the brainstorming process of a group is coordinated by a facilitator. We evaluated the usability of DigiMetaplan with five hybrid teams consisting of a co-located facilitator and two team members connected with one remote participant. Results showed that the features used in DigiMetaplan on interactive surfaces effectively supported teams in performing facilitated collaborative brainstorming in partially distributed settings. We contribute knowledge on how a brainstorming environment translated into a multi-surface distributed system affects facilitated collaboration.

Inspired by its high performance in image-based medical analysis, this poster paper explores the use of advanced segmentation techniques for industrial Microwave Tomography (MWT). Our context is the visual analysis of moisture levels in porous foams undergoing microwave drying. We propose an automatic segmentation technique---MWT Segmentation based on K-means (MWTS-KM) and demonstrate its efficiency and accuracy for industrial use. MWTS-KM consists of three stages: image augmentation, grey-scale conversion, and K-means implementation. To estimate the performance of this technique, we empirically benchmark its efficiency and accuracy against two well-established alternatives: Otsu and K-means. To elicit performance data, three metrics (Jaccard index, Dice coefficient and false positive) are used. Based on our experiments, our results indicate that MWTS-KM outperforms the well-established Otsu and K-means.

Abstract As the range of handheld, mobile and desktop devices expands and worldwide demand for collaborative application tools increases, there is a growing need for higher speed impromptu cross-device application sharing to keep up with workplace requirements for on-site or remote collaborations. To address this, we have developed CamCutter, a cross-device interaction technique enabling a user to quickly select and share an application running on another screen using the camera of a handheld device. This technique can accurately identify the targeted application on a display using our adapted computer vision algorithm, system architecture and software implementation, allowing impromptu real-time and synchronized application sharing between devices. For desktop and meeting room set-ups, we performed a technical evaluation, measuring accuracy and speed of migration. For a single-user reading task and a collaborative composition task, we carried out a user study comparing our technique with commercial screen sharing applications. The results of this study showed both higher performance and preference for our system. Finally, we discuss CamCutter’s limitations and present insights for future vision-based cross-device application sharing.

This paper explores the contrasting notions of "permanance and disposability," "the digital and the physical," and "symbolism and function" in the context of interaction design. Drawing from diverse streams of knowledge, we describe a novel design direction for enduring computational heirlooms based on the marriage of decentralized, trustless software and durable mobile hardware. To justify this concept, we review prior research; attempt to redefine the notion of "material;" propose blockchain-based software as a particular digital material to serve as a substrate for computational heirlooms; and argue for the use of mobile artifacts, informed in terms of their materials and formgiving practices by mechanical wristwatches, as its physical embodiment and functional counterpart. This integration is meant to enable mobile and ubiquitous interactive systems for the storing, experiencing, and exchanging value throughout multiple human lifetimes; showcasing the feats of computational sciences and crafts; and enabling novel user experiences.

We present WristOrigami, an origami-inspired design concept and system extending the interaction with smartwatches through a foldable structure with multiple on-wrist displays. The current design provides extra affordances via folding, flipping, and elastic pulling actions on a multi-display smartwatch. To motivate the design of WristOrigami, we developed a taxonomy that could be useful for analyzing and characterizing the origami-inspired multi-display smartwatch interaction. Through a participatory-design study with a set of prototypes with different levels of fidelity, we investigated users' perception of WristOrigami in a wide range of applications with the presented features, and summarized a list of common shape configurations. We summarized our findings into seven design recommendations, to inform the future design of foldable smartwatch interactions. We further developed a set of application demonstrations as proofs-of-concept.

Wearables are increasingly used during training to quantify performance and provide valuable real-time information. However, interacting with these devices in motion may disrupt the movements of the activity. We propose a method of interaction involving tapping specific locations on the body, identify candidate locations for running and cycling, and compare them in a series of controlled experiments with athletes. A purpose-built prototype measures speed of interaction and gives feedback cues for athletes to report the physical effects on the activity itself. Our results suggest that specific locations are faster and have minimal disruption to movement, even under induced fatigue conditions. The overall method is fast - 1.31s for running and 1.65s for cycling. Preferred locations differ significantly across sports, with stable body parts ranking higher. We effectively demonstrated the use of a single hand for interaction during running with two distinct tap gestures. A set of guidelines inform the design of new sports technologies.

We present EDITalk, a novel voice-based, eyes-free word processing interface. We used a Wizard-of-Oz elicitation study to investigate the viability of eyes-free word processing in the mobile context and to elicit user requirements for such scenarios. Results showed that meta-level operations like highlight and comment, and core operations like insert, delete and replace are desired by users. However, users were challenged by the lack of visual feedback and the cognitive load of remembering text while editing it. We then studied a commercial-grade dictation application and discovered serious limitations that preclude comfortable speak-to-edit interactions. We address these limitations through EDITalk's closed-loop interaction design, enabling eyes-free operation of both meta-level and core word processing operations in the mobile context. Finally, we discuss implications for the design of future mobile, voice-based, eyes-free word processing interface.

Large interactive tabletops are beneficial for various tasks involving exploration and visualization, but regions of the screen far from users can be difficult to reach. We propose AdapTable; a concept and prototype of a flexible multi-display tabletop that can physically reconfigure its layout, allowing for interaction with difficult-to-reach regions. We conducted a design study where we found users preferred to change screen layouts for full-screen interaction, motivated by reduced physical demands and frustration. We then prototyped AdapTable using four actuated tabletop displays each propelled by a mobile robot, and a touch menu was chosen to control the layout. Finally, we conducted a user study to evaluate how well AdapTable addresses the reaching problem compared with a conventional panning technique. Our findings show that AdapTable provides a more efficient method for complex full-screen interaction.

In addition to simple form filling, there is an increasing need for crowdsourcing workers to perform freeform interactions directly on content in microtask crowdsourcing (e.g. proofreading articles or specifying object boundary in an image). Such microtasks are often organized within well-designed workflows to optimize task quality and workload distribution. However, designing and implementing the interface and workflow for such microtasks is challenging because it typically requires programming knowledge and tedious manual effort. We present ReTool, a web-based tool for requesters to design and publish interactive microtasks and workflows by demonstrating the microtasks for text and image content. We evaluated ReTool against a task-design tool from a popular crowdsourcing platform and showed the advantages of ReTool over the existing approach.

Recent research has focused on how to facilitate interaction between humans and robots, giving rise to the field of human robot interaction. A related research area is human-drone interaction (HDI), investigating how interaction between humans and drones can be expanded in novel and meaningful ways. In this work, we explore the use of drones as companions in a home environment. We present three consecutive studies addressing user requirements and design space of companion drones. Following a user-centered approach, the three stages include online questionnaire, design workshops, and simulated virtual reality (VR) home environment. Our results show that participants preferred the idea of a drone companion at home, particularly for tasks such as fetching items and cleaning. The participants were also positive towards a drone companion that featured anthropomorphic features.

We present a human-centered designed social drone aiming to be used in a human crowd environment. Based on design studies and focus groups, we created a prototype of a social drone with a social shape, face and voice for human interaction. We used the prototype for a proxemic study, comparing the required distance from the drone humans could comfortably accept compared with what they would require for a nonsocial drone. The social shaped design with greeting voice added decreased the acceptable distance markedly, as did present or previous pet ownership, and maleness. We also explored the proximity sphere around humans with a social shaped drone based on a validation study with variation of lateral distance and heights. Both lateral distance and the higher height of 1.8 m compared to the lower height of 1.2 m decreased the required comfortable distance as it approached.

We present ViewFinder, a cross-platform mobile application to support the installation and reconfiguration of marker-based motion capture systems with multiple cameras. ViewFinder addresses a common issue when installing or reconfiguring motion capture systems: that system components such as cameras and the host computer can be physically separate and/or difficult to reach, requiring personnel to maneuver between them frequently and laboriously. ViewFinder allows setup technicians or end-users to visualize the output of each camera in the system in a variety of ways in real time, on a smartphone or tablet, while also providing a means to make adjustments to system parameters such as exposure or marker thresholds on the fly. The app has been designed and evaluated through a process observing user-centered design principles, and effectively reduces the amount of work involved in installing and reconfiguring motion capture systems.

Direct and natural images of hand gestures have been shown to benefit remote collaboration on physical tasks in several settings, including ad-hoc ones. However, to capture such unmediated hand gestures within collaborative tasks, existing approaches require stationary hardware systems or heavily instrumented mobile devices, making them unfeasible for use at ad-hoc workplaces. We present MirrorTablet, a low-cost mirror-based system taking advantage of the built-in front-facing camera of a tablet to capture the user's unmediated hand interactions on and above the screen. This system requires minimal instrumentation of the tablet and can be easily (un-)mounted, making it suitable for mobile usage. A user study with ten pairs of participants on a helper-worker setup working on construction tasks showed that MirrorTablet improved task completion time and had positive effects on participants' perceived workload when working with unfamiliar tasks compared to using a common sketch-only interface. In addition, qualitative feedback yielded design considerations on hand visualization for mobile device remote collaboration on physical tasks.

Despite the proliferation of screens in everyday environments, providing values to remote displays for exploring complex data sets is still challenging. Enhanced input for remote screens can increase their utility and enable the construction of rich data-driven environments. Here, we investigate the opportunities provided by a variable movement resistance slider (VMRS), based on a motorized slide potentiometer. These devices are often used in professional soundboards as an effective way to provide discrete input. We designed, built and evaluated a remote input device using a VMRS that facilitates choosing a number on a discrete scale. By comparing our prototype to a traditional slide potentiometer and a software slider, we determined that for conditions where users are not looking at the slider, VMRS can offer significantly better performance and accuracy. Our findings contribute to the understanding of discrete input and enable building new interaction scenarios for large display environments.

While videoconferencing has been available for years, tools for distributed creative collaboration such as brainstorming have not yet reached professional use. Unlike regular conferencing, brainstorming relies on information exchange across multiple channels in shared task and communication spaces. If a participant joins a facilitated brainstorming session remotely through his/her mobile device, perceiving these spaces is challenging. By proposing an immersive environment for the remote participant, this tech note addresses how to provide him/her a stronger engagement. Offered as a client application for the remote participant's tablet device, it enables him/her to see all channels and select which one to interact with. Our proof-of-concept client allows us to examine how this immersive environment for distributed creative collaboration can provide the remote participant with an increased engagement.

Science fiction media has had a long lasting influence on the progression of interactive technology, however recently contradictions are emerging in the development of the two disciplines. Therefore, in this exploratory position paper we report on the insights attained through a day long workshop amongst scientists and researchers on how the collaboration between science fiction and Human Computer Interaction (HCI) can be advanced. Discussions in the workshop focused on detailing the relationship between HCI and science fiction. In conclusion, as our main contribution an action plan and agenda is presented for facilitating deeper influences amongst the two disciplines.

Most existing colorblind aids help their users to distinguish and recognize colors but not compare them. We present HaptiColor, an assistive wristband that encodes discrete color information into spatiotemporal vibrations to support colorblind users to recognize and compare colors. We ran three experiments: the first found the optimal number and placement of motors around the wrist-worn prototype, and the second tested the optimal way to represent discrete points between the vibration motors. Results suggested that using three vibration motors and pulses of varying duration to encode proximity information in spatiotemporal patterns is the optimal solution. Finally, we evaluated the HaptiColor prototype and encodings with six colorblind participants. Our results show that the participants were able to easily understand the encodings and perform color comparison tasks accurately (94.4% to 100%).

Synchronous colocated collaborative sensemaking requires that analysts share their information and insights with each other. The challenge is to know when is the right time to share what information without disrupting the present state of analysis. This is crucial in ad-hoc sensemaking sessions with mobile devices because small screen space limits information display. To address these tensions, we propose and evaluate RAMPARTS - a spatially aware sensemaking system for collaborative crime analysis that aims to support faster information sharing, clue-finding, and analysis. We compare RAMPARTS to an interactive tabletop and a paper-based method in a controlled laboratory study. We found that RAMPARTS significantly decreased task completion time compared to paper, without affecting cognitive load or task completion time adversely compared to an interactive tabletop. We conclude that designing for ad-hoc colocated sensemaking on mobile devices could benefit from spatial awareness. In particular, spatial awareness could be used to identify relevant information, support diverse alignment styles for visual comparison, and enable alternative rhythms of sensemaking.

Motion tracking systems are gaining popularity and have a number of applications in research, entertainment, and arts. These systems must be calibrated before use. This process requires extensive user effort to determine a 3D coordinate system with acceptable accuracy. Usually, this is achieved by rapidly manipulating a calibration device (e.g. a calibration wand) in a volume for a set amount of time. While this is a complex spatial input task, improving the user experience of calibration inspired little research. This paper presents the design, implementation, and evaluation of WAVI --- a prototype device mounted on a calibration wand to jointly provide visual and tactile feedback during the calibration process. We conducted a user study that showed that the device significantly increases calibration quality without increasing user effort. Based on our experiences with WAVI, we present new insights for improving motion tracking calibration and complex spatial input.

People frequently post their photos on social network sites (e.g. Facebook, Instagram) as a way to share memorable moments, emotions, or locations visited. While sharing photos with the same subjects as past photos could lead to user reminiscence and potentially create valuable benefits, existing products still cannot support this adequately. Based on a survey on user habits in sharing and revisiting photos on social network sites, we propose NowAndThen, a photo recommendation concept and tool that assists reminiscence when sharing photos on social network sites. By combining visual features of the photos and associated tags, a prototype we developed can recommend old photos that have common subjects with the user's current photos of interest. Our study with the prototype shows that this approach can help users positively revive past memories and connections with their friends. In addition, our results include various design insights and implications for future reminiscence-support systems.

This work aims at bringing empirical knowledge on Human-Robot Interaction obtained from user studies closer to being integrated into the capabilities of robots currently available on the market. The Takagi-Sugeno-Kang method and results of a user study conducted with thirty two participants were used to build a fuzzy data-based model for Human-Robot Proxemics. The experiment investigated the effect of robot approach distance and angle on perceived human comfort. The proposed model, consisting of a set of rules, fuzzy sets and their parameters, can be used by the robotics community thanks to their formal form. It can also be directly translated into natural language statements. Results of model cross-validation are reported.

In this article, we present a novel application domain for human computation, specifically for crowdsourcing, which can help in understanding particle-tracking problems. Through an interdisciplinary inquiry, we built a crowdsourcing system designed to detect tracer particles in industrial tomographic images, and applied it to the problem of bulk solid flow in silos. As images from silo-sensing systems cannot be adequately analyzed using the currently available computational methods, human intelligence is required. However, limited availability of experts, as well as their high cost, motivates employing additional nonexperts. We report on the results of a study that assesses the task completion time and accuracy of employing nonexpert workers to process large datasets of images in order to generate data for bulk flow research. We prove the feasibility of this approach by comparing results from a user study with data generated from a computational algorithm. The study shows that the crowd is more scalable and more economical than an automatic solution. The system can help analyze and understand the physics of flow phenomena to better inform the future design of silos, and is generalized enough to be applicable to other domains.

Previous works illustrate that one's palm can reliably recognize 10 or more spatiotemporal vibrotactile patterns. However, recognition of the same patterns on other body parts is unknown. In this paper, we investigate how users perceive spatiotemporal vibrotactile patterns on the arm, palm, thigh, and waist. Results of the first two experiments indicate that precise recognition of either position or orientation is difficult across multiple body parts. Nonetheless, users were able to distinguish whether two vibration pulses were from the same location when played in quick succession. Based on this finding, we designed eight spatiotemporal vibrotactile patterns and evaluated them in two additional experiments. The results demonstrate that these patterns can be reliably recognized (>80%) across the four tested body parts, both in the lab and in a more realistic context.

Motivation studies on running are often focused on how to convince non-runners to run, mainly through designing for extrinsic motivations such as health concerns or external reward systems. In contrast, we conducted a structured inquiry into understanding how to design technology for those whom are already committed to running and participate in organized races. Through interviews, focus groups, ethnographic observation, questionnaires, and design-based research over the course of two years, we investigated the needs of the advanced amateur runner community. An analysis of the gathered data led to five design themes -- Festival, Competition, Practicalities, Togetherness, and Support -- to inform future runner motivation technology. While flow theory appears to be a convenient tool to understand support during a race, we observed a number of other factors that need to be considered. Through combining the themes with previous research, we conclude by presenting nine guidelines for designing technology for this domain.

In this paper the nexus between fictional and real robots in Human Robot Interaction (HRI) is explored. We claim that design guidelines for HRI must not be borrowed from fiction blindly, as contradictions between the two may emerge with respect to what is desired by the human user. To understand human perception of robots appearing in movies we analyse viewing statistics and qualitative comments of a set of YouTube videos comprising of fictional and real robots. Analysis of the viewing statistics showed that real robots are more popular. Furthermore, analysis of the comments showed that two real robots (Nao and Shakey) generated significantly more positive comments and significantly more attributions of usage in human society as compared to the two fictional robots (AstroBoy and HAL9000). Based on the sample of robots considered in this research, our results reveal that contrary to expectation humans are more exposed to real robots and are more preferred, and we conclude by reasserting the contradiction that emerges between real and fictional robots.

In this paper we introduce and evaluate a concept of a drone agent that can support us to keep the environment clean. The drone has four main functions: (1) find a waste item, (2) persuade the user to pick up the item, (3) lead the user to the nearest trash bin, (4) communicate with the user when the job is done. We prototyped three types of persuasion effects with the drone (Visual, Audio, and Visual+Audio) and evaluated them through an online study with eighty two participants. The results reveal that interest in the drone technology varied significantly between developed and developing countries, with individuals in developing countries having higher ratings for drone related persuasion, compliance, pleasantness and sensibility; gender differences also emerged. Finally, evaluation results led to many design recommendations that will increase the functionality and attractiveness and usage potential of the proposed technology.

Advances in display technologies could soon make wearable mid-air displays---devices that present dynamic images floating in mid-air relative to a mobile user---available. Such devices may enable new input and output modalities compared to current mobile devices, and seamlessly offer information on the go. This paper presents a functional prototype for the purpose of understanding these modalities in more detail, including suitable applications and device placement. We first collected results from an online survey identified map navigation as one of the most desirable applications and suggested placement preferences. Based on these rankings, we built a wearable mid-air display mockup consisting of mobile phone, pico projector, and a holder frame, mountable in two alternative ways: wrist and chest. We then designed an experiment, asking participants to navigate different urban routes using map navigation displayed in mid-air. For map navigation, participants ranked wrist-mount safer than chest-mount. The experiment results validate the use of a wearable mid-air display for map navigation. Based on our online survey and experiment, we offer insights and recommendations for the design of wearable mid-air displays.

This paper explores remote cheering during amateur races through a formative design inquiry. Friends and family of advanced amateur runners are part of their running experience. Runners rely on support during the race day and it is usually provided in the form of co-located cheers. RUFUS -- a prototype remote ambient runner support system -- was developed. The system enables supporters to send three types of signals to runners during a race and runners can send signals back to supporters. Input from supporters is sent through a webpage and received by runners through a device designed to lower distraction. An in situ study was conducted to evaluate the prototype during an organized race. Results show that two-way communication between runners and supporters was achieved. We also found that our system reflected varying user needs correctly. Runners and supporters reported increased motivation and enhanced race experience through feeling connected.

While mobile phones affect our behavior and tend to separate us from our physical environment, this very environment could instead become a responsive part of the information domain. For navigation using a map while cycling in an urban environment, we studied two alternative solutions: smartphone display and projection on the road. This paper firstly demonstrates by proof-of-concept a GPS-based map navigation using a bike-mounted projector. Secondly, it implements a prototype using both a projector and a smartphone mounted on a bike, comparing them for use in a navigation system for nighttime cycling. Thirdly, it examines how visuo-spatial factors influence navigation. We believe that our findings will be useful for designing navigation systems for bikes and even for cars, helping cyclists and drivers be more attentive to their environment while navigating, and to provide useful information while moving.

While there is increasing interest in creating eyes-free interaction technologies, a solid analysis of why users need or desire eyes-free interaction has yet to be presented. To gain a better understanding of such user motivations, we conducted an exploratory study with four focus groups, and suggest a classification of motivations for eyes-free interaction under four categories (environmental, social, device features, and personal). Exploring and analyzing these categories, we present early insights pointing to design implications for future eyes-free interactions.

The aim of this work is to identify the properties that a universal multidimensional input device should have in order to optimally function with 3D applications, including Web browsers, window managers, digital content creation software, and computer games. Such applications have become increasingly effective due to the rapid development of computer graphics. This situation has generated a greater need for multidimensional input devices. It is conjectured that an ideal universal multidimensional input device should allow for 2D pointing as well as precise manipulation and navigation within 3D environments. Accordingly, the device should offer (a) six degrees of freedom, (b) a range of motion adapted to finger manipulation, (c) elastic suspension providing rich sensory feedback, and (d) position and rate control. The input device, SpaceCat, was designed to offer all these properties. Although such devices are known to benefit 3D navigation, this work demonstrates that they also benefit 3D object manipulation.

The ForceFeedback Slider (FFS) is a one-dimensional actuated slider using a motor to produce tangible interaction with position and force as input and output parameters. To create a new concept, we have built a mixing desk, placed six FFSs (two implemented here) into a partially realized SliderBox, and added a LED and two toggle buttons to each slider for additional interactivity. We have developed a tool called BounceSlider for improvising music. This application for real time music performance and composition uses a slider handle that can act as a ball. Users can lift and release the handle to set the ball in motion and produce a particular sound each time it bounces against the baseline. Based on physical characteristics, the user can create different sounds and loops by changing two settings: gravity (speed) and bounce type (ball physical characteristics). BounceSlider allows the user to create and save loops of Musical Instrument Digital Interface (Midi) with up to five sounds at a time.

Augmented Chemistry (AC) is an application that utilizes a tangible user interface (TUI) for organic chemistry education. The empirical evaluation described in this paper compares learning effectiveness and user acceptance of AC versus the more traditional ball-and-stick model (BSM). Learning effectiveness results were almost the same for both learning environments. User preference and rankings, using NASA-TLX and SUMI, showed more differences and it was therefore decided to focus mainly on improving these aspects in a re-design of the AC system. For enhanced interaction, keyboard-free system configuration, and internal/external database (DB) access, a graphical user interface (GUI) has been incorporated into the TUI. Three-dimensional (3D) rendering has also been improved using shadows and related effects, thereby enhancing depth perception. The re-designed AC system was then compared to the old system by means of a small qualitative user study. This user study showed an improvement in subjective opinions a out the system's ease of use and ease of learning.

Physics education often relies on visualization of theoretical laws. While Java animations are widespread, they mostly lack user interaction. We propose a haptic device inviting users to interact with the law of physics. Based on an in-house design - a generic force feedback slider (FFS) - we have realized a software application simulating a catapult. As users interact, they receive both tactile and visual feedback. By calling upon two perceptual channels at a time, here tactile and visual, we assume users may construct their mental model more easily. This paper presents our application, the underlying FFS technology, a user study, future uses, and a discussion.

This paper describes how a tangible user interface (TUI) that was designed to construct molecules was extended to also visualize electronegativity and dipole moment. Using interactive tools working with the system, elements can be chosen from a booklet menu and composed into three-dimensional (3D) molecular models. The system is used to teach aspects of organic chemistry such as the octet ride. Based on user experience and expert interviews, a new mode to visualize electronegativity and dipole moment was realized. Evaluation of usability and the learning effectiveness of the AC system is in progress, also examining the dipole tool presented here. We currently compare AC with a ball-stick tool for chemistry education. The paper is an example of joint disciplines: Tangible UI, educational media for chemistry education, and interactive learning.

In this work we compare an in-house designed tangible user interface (TUI) with three alternative single-user tools through an empirical investigation. These three alternative tools are a 3D physical, a 2D cardboard, and a mathematical tool. We expected the 3D physical to perform best, followed by the TUI, the 2D cardboard, and the mathematical tool. A pilot study was first carried out, the results of which were used to design a major experiment. Participants solved the same positioning problem, each using one of the four tools. The mathematical tool was not used in the experiment. In the experiment, trial time, number of user operations, learning effect in both, and user satisfaction were measured. The TUI significantly outperformed the 2D cardboard tool. However, there was no significant difference between the TUI and the 3D physical tool. This justifies the value of researching TUI systems and carrying out usability studies with such systems.

We report on some of the advantages tangible interaction can bring to chemistry education. We describe the realisation of a tangible user interface (TUI) called Augmented Chemistry (AC). A set of interactive tools work within this system. Using these tools, elements can be chosen from a booklet menu and composed into 3D molecular models. The tools are one way towards seamless integration of the physical and digital realms. Since multiple tools can be used concurrently, single and multiple users can use the system at a time. To use the system in an educational context, it was extended into an educational workbench drawing on haptic and aural augmentation. The design and implementation of our system required contributions from optics, mathematics, molecular chemistry, software engineering, and 3D programming, making it a truly interdisciplinary project. Future challenges lie in user acceptance, educational effect, and further system development.