Web agents, like OpenAI's Operator and Google's Project Mariner, are powerful agentic systems pushing the boundaries of Large Language Models (LLM). They can autonomously interact with the internet at the user's behest, such as navigating websites, filling search masks, and comparing price lists. Though web agent research is thriving, induced sustainability issues remain largely unexplored. To highlight the urgency of this issue, we provide an initial exploration of the energy and CO₂ cost associated with web agents from both a theoretical —via estimation— and an empirical perspective —by benchmarking. Our results show how different philosophies in web agent creation can severely impact the associated expended energy, and that more energy consumed does not necessarily equate to better results. We highlight a lack of transparency regarding disclosing model parameters and processes used for some web agents as a limiting factor when estimating energy consumption. Our work contributes towards a change in thinking of how we evaluate web agents, advocating for dedicated metrics measuring energy consumption in benchmarks.

Effective communication and teamwork are vital in high-stakes environments such as the operating room, where timely and accurate information exchange directly affects patient safety and surgical outcomes. Among intraoperative interactions, the collaboration between the surgeon and anesthesiologist is especially critical for maintaining smooth workflows and preventing adverse events. Despite its importance, little HCI research has explicitly examined the unique needs of this dyad or how AI-driven supportive systems might be designed to address them. In this work, we present a qualitative study of surgeon–anesthesiologist collaboration, drawing on focus groups with both specialties and in-situ observations of 45 surgeries spanning open, laparoscopic, and robotic procedures. Our findings uncover key challenges, unmet needs, and coordination breakdowns that shape this relationship. Based on these insights, we conceptualize a systems design to better support intraoperative collaboration.

Recent technological advances popularized the use of image generation among the general public. Crafting effective prompts can, however, be difficult for novice users. To tackle this challenge, we developed PromptMap, a new interaction style for text-to-image AI that allows users to freely explore a vast collection of synthetic prompts through a map-like view with semantic zoom. PromptMap groups images visually by their semantic similarity, allowing users to discover relevant examples. We evaluated PromptMap in a between-subject online study ($n=60$) and a qualitative within-subject study ($n=12$). We found that PromptMap supported users in crafting prompts by providing them with examples. We also demonstrated the feasibility of using LLMs to create vast example collections. Our work contributes a new interaction style that supports users unfamiliar with prompting in achieving a satisfactory image output.

Head-mounted displays (HMDs) are increasingly used in safety-critical fields such as surgery, aviation, and industrial manufacturing. As major manufacturers shift toward video-see through (VST) designs to deliver unified AR and VR experiences, they also replace direct visual access to the real world with a video feed. This design choice raises concerns about its impact on user performance. This study investigates the isolated impacts of VST and optical-see through HMDs on user real-world perceptual–motor performance by comparing two leading HMDs, Apple Vision Pro (AVP) and HoloLens 2 (MHL2) against unencumbered vision using the Purdue Pegboard Test (PPT), a standard assessment of manual dexterity. Twenty participants completed tasks across three conditions (AVP, MHL2, and Baseline), while we recorded dexterity scores, cognitive load, system usability, VR sickness, and subjective feedback. Movement data were also collected via Apple Watches. Study results with 20 participants revealed that dexterity scores significantly declined under the AVP condition across all subtests. This was accompanied by significantly higher cognitive load and a notable drop in RMS acceleration values (observed in the RMS analysis of a subset of 13 participants). The analysis on dexterity score yielded a significant difference between MHL2 and Baseline only for a single subtest of the PPT (Left Hand). Post-task interviews revealed greater discomfort, visual fatigue, and reduced task confidence with AVP. These findings suggest that current VST HMDs impose a hidden ergonomic cost undermining user performance in tasks where precision, and comfort are essential. For AR applications designed to enhance user performance, such as assistive tools, training systems, or task guidance interfaces, designers must account for and mitigate this performance degradation through counterbalancing strategies to offset the visual and cognitive burden introduced by VST HMDs.

The integration of Augmented Reality (AR) technology into surgical procedures offers significant potential to enhance clinical outcomes. While there are plenty of lab-proven prototypes, systems employed in actual clinical settings require specialized design and rigorous clinical evaluation of these AR-based solutions to meet the high demands of complex medical fields. Our research exposes these complex requirements emerging from clinical environments, such as operation theaters. To address the challenges, we introduce ARAS, an operational AR assistance system for live open pancreatic surgery. Employing a user-centric design methodology, we designed and refined ARAS through several iterations, ensuring its practical applicability and effectiveness in a real-world surgical setting during clinical trials. ARAS enables in situ and precise visualization of the patient’s 3D reconstructed vascular system and tumor during the surgical procedure. We evaluated ARAS through clinical trials (N=7) involving patients diagnosed with pancreatic cancer. Our interviews with the surgeons underscored the utility of ARAS for open pancreatic surgery, especially in critical and highly time-pressured phases of the surgery, as it proved to be exceptionally beneficial in aiding surgeons during in their decision-making process. In a post-surgery evaluation, the surgeons also certified the precise visualization accuracy of ARAS during those critical phases. Our findings showcased the heightened requirements for AR-based solutions in operational clinical use and proved that ARAS met the challenges emerging during live surgeries. Consequently, surgical AR assistance systems do have a transformative potential to revolutionize traditional practices, but their applicability is subject to high design constraints during critical medical procedures.

The increased presence of large language models (LLMs) in educational settings has ignited debates concerning negative repercussions, including overreliance and inadequate task reflection.Our work advocates moderated usage of such models, designed in a way that supports students and encourages critical thinking.We developed two moderated interaction methods with ChatGPT: hintbased assistance and presenting multiple answer choices.In a study with students (N=40) answering physics questions, we compared the effects of our moderated models against two baseline settings: unmoderated ChatGPT access and internet searches.We analyzed the interaction strategies and found that the moderated versions exhibited less unreflected usage (e.g., copy & paste) compared to the unmoderated condition.However, neither ChatGPT-supported condition could match the ratio of reflected usage present in internet searches.Our research highlights the potential benefits of

Distractions caused by digital devices are increasingly causing dangerous situations on the road, particularly for more vulnerable road users like cyclists. While researchers have been exploring ways to enable richer interaction scenarios on the bike, safety concerns are frequently neglected and compromised. In this work, we propose Head 'n Shoulder, a gesture-driven approach to bike interaction without affecting bike control, based on a wearable garment that allows hands- and eyes-free interaction with digital devices through integrated capacitive sensors. It achieves an average accuracy of 97% in the final iteration, evaluated on 14 participants. Head 'n Shoulder does not rely on direct pressure sensing, allowing users to wear their everyday garments on top or underneath, not affecting recognition accuracy. Our work introduces a promising research direction: easily deployable smart garments with a minimal set of gestures suited for most bike interaction scenarios, sustaining the rider's comfort and safety.

Allowing users of interactive systems to reflect on their task proficiency is often incidental. This is unfortunate, as communicating meaningful task-related proficiency feedback could improve users’ awareness of their abilities and their willingness to improve. To highlight the feasibility of this concept, we evaluated how different methods of readability feedback impacted users during a text production task. In general, our results showed that having access to readability feedback allowed participants to reflect on their task solving approach, facilitating the users’ understanding of their proficiency. Revision-based methods are less distracting for the user than continuous feedback methods, while still offering high efficacy. Further, feedback should be paired with a subtle form of gamification elements. We envision this reflection-oriented design to user proficiency to be applicable to a variety of interactive systems, allowing for an improved and engaging user experience.

Mastering skills which involve high dexterity, such as playing the piano, requires extensive guidance through personal teaching. Understanding how we can leverage data from sensor-based systems to improve the learning process, allows us to build interactive systems which effectively facilitate skill acquisition. To explore such possibilities, we developed EyePiano—a gaze-assisted tool for reflective piano playing. EyePiano guides the practice process of learning piano scores through analyzing the pianist’s gaze behavior. We based the design of EyePiano on requirements identified through interviews with piano teachers and a feasibility evaluation of gaze metrics. Our system illustrates that basic gaze metrics are sufficient to predict difficult regions for students. Thus, highlighting sections of the music piece which are particularly difficult for the pianist allows EyePiano to support piano rehearsals for students. Our work showcases the feasibility of using gaze data for reflective music education, enabling effective instrument practice.

Recent research has shown that users increasingly seek meaning in technologies and that eudaimonic user experience (UX) is part of everyday encounters with technology. Yet, to date, there is no validated means to assess eudaimonic properties in interactive artefacts. We conceptualised, developed and validated a six-item questionnaire for measuring eudaimonic properties of technologies—the Eudaimonic Technology Experience Scale (ETES). Our scale includes two factors, which describe what aspects of a eudaimonic experience can be supported by technology: eudaimonic goals and self-knowlege. We consulted work in Human-Computer Interaction (HCI), psychology and philosophy to gather an initial set of concepts that could contribute to eudaimonic UX. We then built the scale based on expert interviews and exploratory factor analysis and verified its quality in a number of tests (confirmatory factor analysis, reliability and validity checks). ETES provides a standardised tool for identifying eudaimonic qualities in interactive systems and allows for rapidly comparing prototypes.

Technologies have become an integral part of physical activity. Yet, the majority of popular programs do not focus on promoting a genuine understanding of how sport affects our bodies. As apps and trackers persuade users to exercise more, lack of body awareness can be detrimental to health. In this work, we propose and evaluate the concept of in-session reflective feedback as a means to support informed exercise routines by design. We designed and implemented REPLAY, a system which presents users with a visualization of physiological signals (heart rate, movement) from body-worn sensors during high-intensity interval training (HIIT). Our evaluation showed that participants gained a better understanding of how their body reacted to physical activity, allowing them to understand its effect and recognize own weaknesses. Further, our work demonstrates how the type of feedback can significantly moderate a user's perceived exhaustion. We highlight how in-session reflective feedback using bodily signals can promote healthy and effective workouts through creating a deeper understanding of one's own body physiology and limits.

Absent-minded smartphone use has become a catalyst for digital stress. It is often triggered by the mere presence of the device itself, inciting physical opt-out solutions with potentially severe drawbacks. Consequently, there is a need to explore the design dimensions and entailing limitations of physical opt-out methods, especially for everyday use. Informed through expert interviews and online questionnaires, we designed a proof of concept prototype that creates a physical barrier between users and their smartphones while maintaining ease of access. We evaluated our prototype concept in a two-week in-the-wild study, surfacing known benefits of physical opt-out strategies, such as encouraging reflection and supporting deliberate breaks. Although we reduced absent-minded use for users with a high predisposition, drawbacks such as Fear of Missing Out (FoMO) remain. Our work highlights that finding the appropriate amount of physical separation is essential when designing opt-out strategies for digital wellbeing.

Racket sports offer an enjoyable form of physical activity and are fertile ground for interactive technologies supporting new players. Yet, current research has neglected its potential to support not only active players but also coaches in their training methods. To investigate how interactive technologies can support skill acquisition in training, we designed an Electrical Muscle Stimulation (EMS) system that helps maintain the ready position in crossminton. We compared the system with a vibrotactile solution in a user study, interviewing novice players and experienced coaches about their perception of the system. The system allowed coaches to effectively and immediately guide players to the ready position. An EMS-based feedback system for coaches can potentially reduce delay (physical and cognitive) for trainees, as stated by coaches. Our work contributes insights into designing systems that facilitate learning sports techniques using interactive feedback.

Novel input methods for game design often excite users, especially if they extend the way one interacts with the system. Electromyography (EMG) has the inherent potential to provide an intuitive - yet challenging - input channel for interactive systems. While this difficulty in control often limits the scope of applications for EMG in most systems, we argue that these qualities are especially relevant for games and playful interaction. The inherently challenging qualities of EMG input make the modality a prime candidate for designing body-centric playful experiences. Yet, we still need to understand its limitations to create engaging rather than frustrating experiences for users. In this work, we investigate EMG's potential to support playful interaction through exploratory studies, deriving feasible game interactions based on EMG's technical constraints, and study their application in game design. Based on our findings, we highlight design implications and pitfalls to avoid when creating EMG-based entertainment systems.

Interactive technologies are getting closer to our bodies and permeate the infrastructure of our homes. While such technologies offer many benefits, they can also cause an initial feeling of unease in users. It is important for Human-Computer Interaction to manage first impressions and avoid designing technologies that appear creepy. To that end, we developed the Perceived Creepiness of Technology Scale (PCTS), which measures how creepy a technology appears to a user in an initial encounter with a new artefact. The scale was developed based on past work on creepiness and a set of ten focus groups conducted with users from diverse backgrounds. We followed a structured process of analytically developing and validating the scale. The PCTS is designed to enable designers and researchers to quickly compare interactive technologies and ensure that they do not design technologies that produce initial feelings of creepiness in users.

Physical exercises can benefit our health, but avoiding improper form and overexertion is essential. Facilitating bodily insights can encourage learning about exercise form, allowing users to gain a deeper understanding of their physiology. To investigate this, we conducted a lab experiment where amateur users performed bicep curls, and interviews with sports coaches. Participants were provided with FitBack—a system that monitors muscle activity during exercises via electromyography (EMG) and offers real-time biofeedback. Amateurs reported that they were successful in improving their exercise form and could acquire deeper bodily insights. Coaches reflected on how understanding muscle activity through EMG could be effectively used for increasing body awareness during coaching, highlighting that EMG-based biofeedback is beneficial for a diverse set of users. Our work contributes insights into using bodily sensing to help users understand their bodies. We contribute guidelines for designing systems that use EMG biofeedback effectively in physical activity.

While fitness trackers are increasingly popular among users, recent studies have shown that the health benefits of wearing a tracker are not apparent. The need to explicitly retrieve data can lead to limited benefits. Understanding how users can access, understand, and reflect on their data can lead to building systems that benefit our wellbeing. In this work, we explore the feasibility of using ubiquitous artifacts for unobtrusive feedback in health tracking. We evaluated a concept based on design dimensions for personal visualization on a smart mirror in a user study. Our design puts emphasis on the temporality of presented data. Participants found the visualizations comprehensive, rating cardiac and inertial data most useful as well as approved the different levels of temporal aggregation. Our work contributes findings on how to represent health-related data with ubiquitous artifacts to increase users’ awareness.

Body movements, from a short smile to a marathon run, are driven by muscle activity. Despite the fact that measuring muscle activity with electromyography (EMG) is technically well established, it is highly complex and its use in interfaces has been limited. Easy access to muscle sensing can offer new opportunities to Human-Computer Interaction (HCI) research. Off-the-shelf sensors often only provide low-level access, hence requiring expertise in signal processing and widening the gulf of execution for users without engineering skills. To address this challenge, we introduce EMBody, a data-centric toolkit for EMG-based interface prototyping and experimentation. EMBody offers multiple levels of prototyping fidelity for EMG sensing, signal processing, and data interpretation. Our data-centric toolkit encapsulates the different data representation stages, offering a wide range of customization opportunities to experts while also allowing non-technical designers to focus on creating new interaction techniques. EMBody features a lightweight form factor and wireless connectivity. Additionally, the system leverages an exploration-centered workflow by allowing rapid access to measurement data via the accompanying software. Users define a set of motions to be recognized and interactively provide example data points. The toolkit then handles signal processing and classification. The recognized movements are streamed on the local network, ready to be used by interactive applications. This paper reports on how to use EMBody and its implementation. We iteratively developed the toolkit in a series of workshops and example applications. Users who had none or very limited knowledge of EMG could rapidly create engaging functional prototypes, while experts appreciated the modularity of the software component allowing for a high degree of customization. We contribute the software and hardware components of EMBody as a tool for the research community to stimulate creative exploration of EMG systems.

Improvising on the piano keyboard requires extensive skill development, which may reduce the feeling of immersion and flow for amateur players. However, being able to add simple musical effects greatly boosts a player's ability to express their unique playing style. To simplify this process, we designed an electromyography-based (EMG) system which integrates seamlessly into normal play by allowing musicians to modulate sound pitch using their thumb. We conducted an exploratory user study where users played a predefined melody and improvised using our system and a standard pitch wheel. Interview responses and survey answers showed that the EMG-based system supported the players' musical flow. Additionally, interviews indicated the system's capabilities to foster player creativity, and that players enjoyed experimenting with the effect. Our work illustrates how EMG can support seamless integration into existing systems to extend the range of interactions provided by a given interface.

While Virtual Reality continues to increase in fidelity, it remains an open question how to effectively reflect the user's movements and provide congruent feedback in virtual environments. We present VRsneaky, a system for producing auditory movement feedback, which helps participants orient themselves in a virtual environment by providing footstep sounds. The system reacts to the user's specific gait features and adjusts the audio accordingly. In a user study with 28 participants, we found that VRsneaky increases users' sense of presence as well as awareness of their own posture and gait. Additionally, we find that increasing auditory realism significantly influences certain characteristics of participants' gait. Our work shows that gait-aware audio feedback is a means to increase presence in virtual environments. We discuss opportunities and design requirements for future scenarios where users walk through immersive virtual worlds.

Mastering fine motor tasks, such as playing the guitar, takes years of time-consuming practice. Commonly, expensive guidance by experts is essential for adjusting the training program to the student's proficiency. In our work, we showcase the suitability of Electromyography to detect fine-grained hand and finger postures in an exemplary guitar tutor scenario. We present EMGuitar, an interactive guitar tutoring system, that assists students by reporting on play correctness and adjusts playback tempi automatically. We report person-dependent classification utilizing a ring of electrodes around the forearm with an F1 score of up to 0.89 on recorded calibration data. Furthermore, our system was received well by neither diminishing ease of use nor being disruptive for the participants. Based on the received comments, we identified the need for detailed play accuracy feedback down to individual chords, for which we suggest an adapted visualization and an algorithmic approach.

We are often confronted with information interfaces designed in an unfamiliar language, especially in an increasingly globalized world, where the language barrier inhibits interaction with the system. In our work, we explore the design space for building interfaces that can detect the user's language proficiency. Specifically, we look at how a user's gaze properties can be used to detect whether the interface is presented in a language they understand. We report a study (N=21) where participants were presented with questions in multiple languages, whilst being recorded for gaze behavior. We identified fixation and blink durations to be effective indicators of the participants' language proficiencies. Based on these findings, we propose a classification scheme and technical guidelines for enabling language proficiency awareness on information displays using gaze data.

Interactive systems, such as online search interfaces, require appropriate input if they are to produce accurate information. Without this, they can be inaccurate if the user is uncertain about the keywords. Current systems do not have the means to detect uncertainty, which may lead to a negative user experience. We explore physiological and behavioral measurements as tools to implicitly detect users' uncertainty, and provide a method to integrate input variability in interactive systems. We conducted a laboratory study where participants answered questions of varying difficulty, recording physiological data via a key logger, an eye tracker, and a heart rate sensor. Our results show that participants spent significantly more time on difficult questions and looked longer at their answers before submitting them. Based on our results, we provide initial insights on how data from physiological sensors and logged user behavior can be utilized to enrich interactive systems and evaluate a user's uncertainty level.