Mark McGill

dblp:61/7370 · DBLP profile ↗
← Back
64ranked-venue papers
10as first author
52since 2021 · last 2026
0000-0002-8333-5687ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 53 · 9 first-author · 42 since 2021Graphics, computer vision, multimedia, augmented reality and games · 26 · 2 first-author · 23 since 2021Applied, interdisciplinary, general and emerging computing · 2 · 2 since 2021
YearPublicationVenuePosition
2026 The BrutARlist: Exploring How and Why Individuals Alter a Known Brutalist Building with Augmented Reality
abstract
Buildings, which people encounter continuously in their day-to-day life, are a compelling target for augmented reality (AR), being fixed, static structures often with plain, flat surfaces to use as potential canvases for AR content. We present a user study (n=15) where participants described how they would augment a Brutalist building they see/use often with AR content. Our results show a willingness to augment the building aesthetically and functionally We identify preliminary insights on where augmentations on buildings are preferred to be made, i.e. targeting flat, empty surfaces and the roof, avoiding windows and entrances.
Joseph O'Hagan, Graham A. Wilson, Thomas Goodge, Diego Drago, Mark McGill
AVI5
2026 Time WizARd: Exploring the Use of Augmented Reality for Supporting Multitask Timekeeping in Kitchen Cooking
abstract
Augmented reality (AR) can aid individuals during complex, time sensitive tasks like cooking. To explore AR’s use within the kitchen we conducted an interview study (N=17) and identified time management, due to the constraints of single-device timers such as smartphones, as a challenge whilst cooking which AR could address. We conducted a user study (N=14) comparing an AR spatial timer, which allows multiple instances of timers to be positioned within a culinary environment, with a single device timer for workload perception and time management efficiency. Our results show spatial-timers can assist users by making it easier to check remaining time for specific kitchen station, and can more effectively draw attention to stations, but overall workload did not improve.
Joseph O'Hagan, Xinuo Zhou, Katharina Margareta Theresa Pöhlmann, Mark McGill
AVI4
2026 MovieMetaViewer: Extending TV Movie Browsing Interfaces to Visualise and Filter Based on Hidden or Unlinked Metadata using Augmented Reality Overlays
abstract
Streaming services do not present options neutrally. Less metadata is presented today than at the advent of streaming video, e.g. with user and critic ratings routinely hidden, inhibiting informed user choices. Our work combines Augmented Reality and Diminished Reality to explore meta-interfaces that augment TV movie browsing with additional information and perceptual filtering. We present MovieMetaViewer an AR extension for movie streaming UIs which augments streaming-service browsing with independent metadata and optional perceptual filtering. We aim to determine if such meta-interfaces are helpful during media browsing, and whether diminished reality or other visual effects can improve the browsing experience by ensuring viewers only perceive movies meeting their filtering criteria.
Charlie Henderson, Joseph O'Hagan, Thomas Goodge, Mark McGill
IMX4
2026 Exploring the Social Challenges and Opportunities of Group Use of Augmented Reality Through Speculative Design Workshops
abstract
With augmented reality (AR) soon expected become part of our everyday lives there is a pertinent need to consider how it should be used within social settings. We explore individuals’ inclinations towards incorporating AR into small group social interactions across a diverse range of settings. Through six speculative design focus groups involving 21 participants, 100 ideas were generated across three scenarios: living room settings (public – sociopetal), public transport seating (semi-public – mixed proxemics), and crowded public spaces (public – sociofugal). We discuss the challenges being addressed through the proposed social AR designs, ranging from functional use cases around conveying shared information and supporting group cohesion, to addressing boredom, to facilitating shared imagination.
Daniel Gyorgy Szittya, Joseph O'Hagan, Katharina Margareta Theresa Pöhlmann, Graham F. Wilson, Mark McGill
IMX5
2026 Investigating How Brain Stimulation Mitigates Motion Sickness Using Spatiotemporal Nystagmus Parameters Captured in VR
abstract
Mitigating motion sickness with non-invasive cortical vestibular brain stimulation (vBS) is promising because it does not require redesigning VR content or altering the motion profile of a moving platform. Instead, it aims to enhance individuals' intrinsic resistance to motion sickness. However, the mechanism underlying its online efficacy remains unclear (Here, "online efficacy" means the observed motion sickness reduction during vBS not before or after vBS). Motivated by lower motion sickness susceptibility observed in congenital nystagmus patients, we hypothesized that reductions in motion sickness under vBS condition are associated with increased eye movement instability. Leveraging sham-controlled, sex-balanced eye-tracking data from an open-access database, this study presents the first evaluation of a 20 Hz-vBS approach targeting the vestibular cortices as a countermeasure to motion sickness, using spatiotemporal eye movement metrics derived from eye-tracking data collected in VR. Results show that reduced motion sickness under active vBS indeed increased eye movement instability temporarily and spatially, compared to sham vBS. We also found that the temporal metrics was more strongly associated with female participants, whereas the spatial metric was more strongly associated with male participants.
Gang Li 0011, Mark McGill, Alana Grant, Katherina Poehlmann, Rory Holden, Stephen A. Brewster, Frank E. Pollick
IEEE Trans. Vis. Comput. Graph.2
2025 "Abracadabra, Block the Bullies!": Child Perceptions of Manual, Semi-Automated and Automated Interventions against Group Harassment in Social VR
abstract
Child users of Social VR (SVR) face challenges dealing with group harassment. While mitigation tools exist, it is unclear a) how suitable they are for children experiencing group harassment, b) the extent to which children want control over interventions. We conducted two user studies involving 74 children (N1=40, N2=34) in which we compared strategies for dealing with group harassment. First, we compared three ways of manually blocking harassers (SelectBlock, MassBlock, PointBlock) and two intervention positions (ground level vs elevated). Second, we contrasted manual blocking with different levels of automation provided by an Automated Moderator (AM). We investigated how the initiation and decision of blocking, whether by the child or by an AM, impact children’s perceptions. Results show children value engaging interventions, PointBlock, and elevation. They seek control but appreciate assistance when involved in AM decision-making. We provide design considerations for creating safe, empowering experiences for children facing SVR group harassment.
Cristina Fiani, Robin Bretin, Mark McGill, Mohamed Khamis
IDC3
2025 "Create a Fear of Missing Out" - ChatGPT Implements Unsolicited Deceptive Designs in Generated Websites Without Warning
abstract
With the recent advancements in Large Language Models (LLMs), web developers increasingly apply their code-generation capabilities to website design. However, since these models are trained on existing designerly knowledge, they may inadvertently replicate bad or even illegal practices, especially deceptive designs (DD). This paper examines whether users can accidentally create DD for a fictitious webshop using GPT-4. We recruited 20 participants, asking them to use ChatGPT to generate functionalities (product overview or checkout) and then modify these using neutral prompts to meet a business goal (e.g., “increase the likelihood of us selling our product”). We found that all 20 generated websites contained at least one DD pattern (mean: 5, max: 9), with GPT-4 providing no warnings. When reflecting on the designs, only 4 participants expressed concerns, while most considered the outcomes satisfactory and not morally problematic, despite the potential ethical and legal implications for end-users and those adopting ChatGPT’s recommendations.
Veronika Krauß, Mark McGill, Thomas Kosch, Yolanda Thiel, Jan Gugenheimer
CHI2
2025 Everything to Gain: Combining Area Cursors with increased Control-Display Gain for Fast and Accurate Touchless Input
abstract
Touchless displays often use mid-air gestures to control on-screen cursors for pointer interactions. Area cursors can simplify touchless cursor input by implicitly targeting nearby widgets without the cursor entering the target. However, for displays with dense target layouts, the cursor still has to arrive close to the widget, meaning the benefits of area cursors for time-to-target and effort are diminished. Through two experiments, we demonstrate for the first time that fine-tuning the mapping between hand and cursor movements (control-display gain – CDG) can address the deficiencies of area cursors and improve the performance of touchless interaction. Across several display sizes and target densities (representative of myriad public displays used in retail, transport, museums, etc), our findings show that the forgiving nature of an area cursor compensates for the imprecision of a high CDG, helping users interact more effectively with smaller and more controlled hand/arm movements.
Kieran Waugh, Mark McGill, Euan Freeman
CHI2
2025 InterFACE: Establishing a Facial Action Unit Input Vocabulary for Hands-Free Extended Reality Interactions, From VR Gaming to AR Web Browsing
abstract
Extended Reality (XR) interactions often rely on spatial hand or controller inputs - necessitating dexterous wrist, hand and finger movements including pressing virtual buttons, pinching to select, and performing hand gestures. However, there are scenarios where such dependencies may render XR devices and apps inaccessible to users - from situational/temporary impairments such as encumbrance, to physical motor impairments. In this paper, we contribute to a growing literature considering facial input as an alternative. In a user study (N=20) we systematically evaluate the usability of 53 Facial Action Units in VR, deriving a set of optimal (comfort, effort, performance) FAUs for interaction. We then use these facial inputs to drive and evaluate (N=10) two demonstrator apps: VR locomotion, and AR web browsing, showcasing how close facial interaction can get to existing baselines, and demonstrating that FAUs offer a viable, generalizable input modality for XR devices.
Graham A. Wilson, Jamie McCready, Euan Freeman, Florian Mathis, Harvey Russell, Mark McGill
CHI6
2025 The Spin Doctor: Leveraging Insensitivity to Passive Rotational & Translational Gain For Unbounded Motion-Based VR Experiences
abstract
Rotational gain has been studied largely under active self-motion, where users control their own movement. In multiple VR scenarios, the user is under passive self-motion: their body is moved by a training simulator, a motorised chair, or a vehicle-based VR application. Users may be less sensitive to manipulation under passive motion - especially when engaged in a secondary task - meaning motion experiences could be expanded by high gains and even opposed virtual-physical motion. We identified both the perceptible and maximum comfortable thresholds of rotational gain when passively turned in a motorised chair, with and without a task, for the first time. We then applied those thresholds to an ’unbounded’ in-car VR game where the user experiences an entirely different route to their physical movement. We provide the first guidelines for creating enhanced passive motion experiences and open the design space to new applications not restricted by physical motion.
Graham A. Wilson, Katharina Margareta Theresa Pöhlmann, David Al Baiaty Suarez, Mark McGill, Stephen A. Brewster
CHI4
2025 "Everyone Knows You're Watching P*rn": Reflecting on First-Hand User Accounts on the Use and Public Perception of in-Transit Passenger XR
abstract
AR/VR use in-transit is increasing due to the advantages it provides over more limited mobile and seatback displays for work and play. However, despite the now common support for “travel mode” in consumer XR headsets such as the Meta Quest 3 and Apple Vision Pro, little is known about how this emergent technology has been used in-practice by passengers - from why and how they use it, to what real-world challenges they face during usage in these very public, shared contexts. To address this gap, we contribute a novel large scale analysis of Reddit posts discussing first-hand use of VR/AR in-transit. We captured 11,150 posts, producing 437 user accounts which were synthesized into themes in relation to: in-the-wild use of immersive devices, users' positive/negative sentiment about usage, challenges they faced, and social interactions and impediments that occurred. Our snapshot of the current reality of passenger XR provides real-world insight evidencing the need for more passenger-centric designs of spatial interfaces, interaction techniques, and reality awareness.
Katharina Margareta Theresa Pöhlmann, Graham A. Wilson, Laura Bajorunaite, Mark McGill, Joseph O'Hagan
ISMAR4
2025 Big Frog Vibes: Jumping Superhuman Distances Forward, Vertically and Sideways in VR Using Comfortably High Translational Gain
abstract
Jumping is an engaging physical activity bounded by individual physical capabilities. However in Virtual Reality (VR) we can apply perceptual manipulations like translational gain to real jumps, amplifying the user's apparent capabilities - jumping higher and further than is possible in reality. Prior work has focused on imperceptible gains, but this greatly limits the range of available experiences users can engage in, and designers can create. Research outside of jumping has shown that higher, clearly noticeable gains can still be comfortable, engaging and usable in VR. Therefore, in this paper, we evaluate ($\mathrm{N}=18$) the maximal comfortable gain applied to three jump types: forward, vertical, and sideways. We found, for the majority of participants, high gains ($>100 \mathrm{x}$) are not only comfortable but made users feel enhanced whilst retaining jump ownership, deriving guidelines on the extent to which we can amplify real jumps to feel superhuman in virtuality.
Graham A. Wilson, Harvey Russell, Joseph O'Hagan, Mark McGill
ISMAR4
2025 Winter is Coming... To Your Commute: Adapting Passenger ARTV Ornamentations to Match On-screen Visuals For Enhanced Immersion in Fantasy Drama TV Content
abstract
Augmented Reality Television (ARTV) affords new ways to experience media – from displaying auxiliary information to increasing immersion with additional 3D models. Many people consume video content when travelling or commuting; accordingly, the benefits of ARTV in this context merit investigation. We conducted a study (N=20) exploring the effects of using ARTV Ornamentations when watching Game of Thrones on a train. We used Virtual Reality to simulate use of AR in a train carriage and evaluated four different levels of ARTV Ornamentation, from a simple floating panel through to themed models and interior textures that change to match the on-screen visuals. Results showed that Ornamentations matching the show’s genre were sufficient to provide an engaging and immersive experience, but Ornamentations themed to the show which changed based on the scene further elevated the experience. This means it is possible for passengers to have an immersive experience while commuting without significantly reducing their awareness of their environment. We also reflect on the implications of our work for an existing ARTV framework and propose the inclusion of several new aspects to encourage designers to consider the unique challenges and affordances of ARTV Ornamentations.
Iain Christie, Graham A. Wilson, Mark McGill, Stephen A. Brewster
IMX3
2025 Corrigendum for 'Exploring the Perspectives of Social VR-Aware Non-Parent Adults and Parents on Children's Use of Social Virtual Reality'
abstract
This is a corrigendum for the paper ''Exploring the Perspectives of Social VR-Aware Non-Parent Adults and Parents on Children's Use of Social Virtual Reality'', Proceedings of the ACM on Human-Computer Interaction, Volume 8, Issue CSCW1, Article No. 54. We report 1) the error, 2) the changes made, and 3) the result based on those changes. These corrections do not alter the overall conclusions of the study but ensure that the reported findings are more accurate and reflective of the data.
Cristina Fiani, Pejman Saeghe, Mark McGill, Mohamed Khamis
Proc. ACM Hum. Comput. Interact.3
2025 Reduced Motion Sickness Using Vestibular EEG-Guided tACS Under Mismatched Physical Rotation and VR Visual Motion
abstract
The increasing use of virtual reality (VR) in public transportation enables travelers to engage in immersive entertainment or productive tasks, thereby enhancing the overall travel experience. However, mismatched VR visual motion and physical car motion can cause motion sickness (MS), leading to nausea, postural instability and reduced time for enjoyment or productive tasks. Thus, the benefits of using VR in transportation systems are currently limited to individuals who do not experience MS. Thus, effective MS mitigation is essential for improving travel safety, maximizing travel time, and expanding VR accessibility. Neuromodulation targeting the vestibular apparatus, such as bone-conducted vibration (BCV), has shown promise in reducing MS. However, it remains unclear whether neuromodulation directly targeting vestibular cortical regions, such as transcranial alternating current stimulation (tACS), is superior. This paper focuses on the design and validation of a novel vestibular cortical neuromodulation approach using tACS in mitigating MS in a novel simulated in-car VR environment. Eighty participants were recruited to evaluate the tACS approach. The results demonstrate that the proposed tACS approach effectively reduces nausea, enhances postural stability, and extends survival time in MS. Compared to BCV, tACS demonstrates a faster onset of action and longer mitigation effects. However, from an applied perspective, the effects of our tACS approach were relatively short-lived and accompanied by side effects such as tingling and itching.
Gang Li 0011, Mark McGill, Alana Grant, Katharina Margareta Theresa Pöhlmann, Rory Holden, Stephen A. Brewster, Frank E. Pollick
IEEE Trans. Intell. Transp. Syst.2
2025 The Perception of a Tap: Using Auditory Augmented Reality to Alter the Contact Properties of a Physical Object
abstract
Auditory Augmented Reality (AAR) is a way of augmenting our aural perception of reality. AAR could offer new ways of expressing information embedded in everyday objects by augmenting their auditory contact properties in real-time. Many acoustic augmentations (e.g., altering the sound, reverb) are available, but how they impact the perception of the physical object and its audio properties are unknown. In two experiments, we explore the effects of different audio feedback (surface materials, musical instruments) and reverb parameters (mixing, depth) on perceived simultaneity, unity, agency, and object properties (object size, material, room size), along with their recognition. We found that audio feedback had a significant but small effect on interaction and altered perceived object properties. Mixing in reverb reduced audio recognition but boosted reverb determination. We offer guidelines on combining audio feedback and reverb based on acoustic properties and reverb parameter adjustment to provide multi-level feedback for AAR.
Isna Alfi Bustoni, Mark McGill, Stephen A. Brewster
ACM Trans. Comput. Hum. Interact.2
2025 Beyond Mute and Block: Adoption and Effectiveness of Safety Tools in Social VR, from Ubiquitous Harassment to Social Sculpting
abstract
Harassment in Social Virtual Reality (SVR) is a growing concern. The current SVR landscape features inconsistent access to non-standardised safety features, with minimal empirical evidence on their real-world effectiveness, usage and impact. We examine the use and effectiveness of safety tools across 12 popular SVR platforms by surveying 100 users about their experiences of different types of harassment and their use of features like muting, blocking, personal spaces and safety gestures. While harassment remained common-including hate speech, virtual stalking, and physical harassment-many find safety features insufficient or inconsistently applied. Reactive tools like muting and blocking are widely used, largely driven by users' familiarity from other platforms. Safety tools are also used to proactively curate individual virtual experiences, protecting users from harassment, but inadvertently leading to fragmented social spaces. We advocate for standardising proactive, rather than reactive, anti-harassment tools across platforms, and present insights into future safety feature development.
Maheshya Weerasinghe, Shaun Alexander Macdonald, Cristina Fiani, Joseph O'Hagan, Mathieu Chollet, Mark McGill, Mohamed Khamis
IEEE Trans. Vis. Comput. Graph.6
2024 What You Experience is What We Collect: User Experience Based Fine-Grained Permissions for Everyday Augmented Reality
abstract
Everyday Augmented Reality (AR) headsets pose significant privacy risks, potentially allowing prolonged sensitive data collection of both users and bystanders (e.g. members of the public). While users control data access through permissions, current AR systems inherit smartphone permission prompts, which may be less appropriate for all-day AR. This constrains informed choices and risks over-privileged access to sensors. We propose (N=20) a novel AR permission control system that allows better-informed privacy decisions and evaluate it using five mock application contexts. Our system’s novelty lies in enabling users to experience the varying impacts of permission levels on not only a) privacy, but also b) application functionality. This empowers users to better understand what data an application depends on and how its functionalities are impacted by limiting said data. Participants found that our method allows for making better informed privacy decisions, and deemed it more transparent and trustworthy than state-of-the-art AR and smartphone permission systems taken from Android and iOS. Our results offer insights into new and necessary AR permission systems, improving user understanding and control over data access.
Melvin Abraham, Mark McGill, Mohamed Khamis
CHI2
2024 "Pikachu would electrocute people who are misbehaving": Expert, Guardian and Child Perspectives on Automated Embodied Moderators for Safeguarding Children in Social Virtual Reality
abstract
Automated embodied moderation has the potential to create safer spaces for children in social VR, providing a protective figure that takes action to mitigate harmful interactions. However, little is known about how such moderation should be employed in practice. Through interviews with 16 experts in online child safety and psychology, and workshops with 8 guardians and 13 children, we contribute a comprehensive overview of how Automated Embodied Moderators (AEMs) can safeguard children in social VR. We explore perceived concerns, benefits and preferences across the stakeholder groups and gather first-of-their-kind recommendations and reflections around AEM design. The results stress the need to adapt AEMs to children, whether victims or harassers, based on age and development, emphasising empowerment, psychological impact and humans/guardians-in-the-loop. Our work provokes new participatory design-led directions to consider in the development of AEMs for children in social VR taking child, guardian, and expert insights into account.
Cristina Fiani, Robin Bretin, Shaun Alexander Macdonald, Mohamed Khamis, Mark McGill
CHI5
2024 From Slow-Mo to Ludicrous Speed: Comfortably Manipulating the Perception of Linear In-Car VR Motion Through Vehicular Translational Gain and Attenuation
abstract
To prevent motion sickness, Virtual Reality (VR) experiences for vehicle passengers typically present “matched motion”: real vehicle movements are replicated 1:1 by movements in VR. This significantly limits virtual applications. We provide foundations for in-car VR experiences that break this constraint by manipulating the passenger’s visual perception of linear velocity through amplifying and reducing the virtual speed. In two on-the-road studies, we examined the application of Vehicular Translational Gain (1.5-9.5x) and Attenuation (0.66-0.14x) to real car speeds (~50km/h) across two VR tasks (reading and gaming), exploring journey perception, impact on motion sickness, travel experience and tasks. We found that vehicular gain/attenuation can be applied without significantly increasing motion sickness. Gain was more noticeable and affected perceived speed, distance, safety, relaxation and excitement, being well-suited to gaming, while attenuation was more suitable for productivity. Our work unlocks new ways that VR applications can enhance and alter the passenger experience through novel perceptual manipulations of vehicle velocity.
Katharina Margareta Theresa Pöhlmann, Graham A. Wilson, Gang Li 0011, Mark McGill, Stephen A. Brewster
CHI4
2024 Exploring the Alteration and Masking of Everyday Noise Sounds using Auditory Augmented Reality
abstract
While noise-cancelling headphones can block out or mask environmental noise with digital sound, this costs the user situational awareness and information. With the advancement of acoustically transparent personal audio devices (e.g. headphones, open-ear audio frames), Auditory Augmented Reality (AAR), and real-time audio processing, it is feasible to preserve user situational awareness and relevant information whilst diminishing the perception of the noise. Through an online survey (n=124), this research explored users’ attitudes and preferred AAR strategy (keep the noise, make the noise more pleasant, obscure the noise, reduce the noise, remove the noise, and replace the noise) toward different types of noises from a range of categories (living beings, mechanical, and environmental) and varying degrees of relevance. It was discovered that respondents’ degrees of annoyance varied according to the kind of noise and its relevance to them. Additionally, respondents had a strong tendency to reduce irrelevant noise and retain more relevant noise. Based on our findings, we discuss how AAR can assist users in coping with noise whilst retaining relevant information through selectively suppressing or altering the noise, as appropriate.
Isna Alfi Bustoni, Mark McGill, Stephen A. Brewster
ICMI2
2024 Don't Record My Private pARts: Understanding The Role of Sensitive Contexts and Privacy Perceptions in Influencing Attitudes Towards Everyday Augmented Reality Sensor Usage
abstract
Everyday Augmented Reality (AR) headsets come with an array of sensing capabilities. Users wearing these headsets for extended periods may prefer specific sensors to remain inactive in some contexts for privacy and sensitivity reasons. Currently, the contexts in which users wish to limit sensor data collection are unclear. To explore this, we conducted a survey ($\mathrm{N}=100$), collecting 552 scenarios to understand which situations users wish to restrict or completely block data collection by specific sensors or combinations on their AR headset. Our results show the sensitive contexts can be classified into seven categories: 1) presence of confidential information; 2) risk of data quantification; 3) expectation of solitude; 4) rules prohibiting data collection; 5) modesty and nudity; 6) home environments; and 7) outdoor public locations. Our results provide insights into privacy-invasive contexts when people want to limit and restrict their AR sensors, building towards automating permission configurations during the prolonged use of everyday AR headsets.
Melvin Abraham, Mohamed Khamis, Mark McGill
ISMAR3
2024 Detecting in-car VR Motion Sickness from Lower Face Action Units
abstract
This paper presents the first in-car VR motion sickness (VRMS) detection model based on lower face action units (LF-AUs). Initially developed in a simulated in-car environment with 78 participants, the model’s generalizability was later tested in realworld driving conditions. Motion sickness was induced using visual linear motion in the VR headset and physical horizontal rotation via a rotating chair. We used a convolutional neural network (MobileNetV3) to automatically extract LF-AUs from images of the users’ mouth region, captured by the VR headset’s built-in camera. These LF-AUs were then used to train a Support Vector Regression (SVR) model to estimate motion sickness scores. We compared the SVR model’s performance using LF-AUs, pupil diameters, and physiological features (individually and in combination) from the same VR headset. Results showed that both individual LF-AU (right dimple) and combined LF-AUs had significant Pearson correlations with self-reported motion sickness scores and achieved lower root mean squared error compared to pupil diameters. The best detection results were obtained by combining LF-AUs and pupil diameters, while physiological features alone did not yield significant results. The LF-AUs-based model demonstrated encouraging generalizability across different settings in the independent studies.
Gang Li 0011, Tanaya Guha, Ogechi Onuoha, Zhanyan Qiu, Alana Grant, Zejian Feng, Kathariana Pohlmann, Mark McGill, Stephen A. Brewster, Frank E. Pollick
ISMAR9
2024 From Redirected Navigation to Forced Attention: Uncovering Manipulative and Deceptive Designs in Augmented Reality through Retail Shopping
abstract
In the near future ubiquitous Augmented Reality (AR) will see virtual spatial content seamlessly integrated into our everyday lives through fashionable, wearable devices such as AR glasses. In doing so, we will unlock the capacity for multiple stakeholders to augment and personalize our view of reality - not always to the benefit of users. Using speculative design, we explore the risks and repercussions of third party-driven AR-enacted manipulation and deception through the lens of supermarket grocery shopping - an activity where consumers are routinely tracked and exposed to manipulation of attention and purchasing decisions. Through a scenario generation activity, 20 participants (mixing both existing XR users and frequent shoppers) co-created 58 scenarios reflecting on how AR-driven manipulation or deception of shoppers could be enacted. Our results show that (1) manipulation, rather than deception, is the primary means of affecting consumer behaviour, necessitating we consider AR Manipulative and Deceptive Designs (MDDs) more broadly. Moreover (2), we highlight a) how known MDDs can manifest in AR, and b) four novel MDDs that are specific to AR: Redirected Navigation, Directed & Forced Attention Shifts, Reality Interference and Delayed and Detained. We reflect on the stakeholders that would manipulate consumer perception of reality, the different manipulations enacted across the lifecycle of consumer shopping, and the AR elements exploited to deliver these ARMDDs, deriving insights into future harms, ethics and safeguarding around ARMDDs to minimize their impact.
Martina Ruocco, Pejman Saeghe, Frederic Kerber, Jan Gugenheimer, Mark McGill, Mohamed Khamis
ISMAR5
2024 Welcome to the Jungle: Using ARTV to Balance Media Immersion and Reality Awareness for Passengers on Public Transport
abstract
Head-mounted displays (HMDs) offer new possibilities for reclaiming and improving passenger travel time. Common Virtual Reality (VR) HMDs provide an immersive mobile media viewing experience, overcoming the screen size constraints of mobile and seat-back displays. However, they also reduce environmental awareness as they completely occlude reality. Conversely, Augmented Reality (AR) glasses afford environmental awareness by default, allowing unrestricted presentation of media through large virtual planar displays, but without the same immersion as VR. Our work transposes the immersive benefits of VR to AR, whilst preserving awareness of the environment. We present results from a user study (N=24) exploring the addition of plants and textures as augmentations of a simulated VR train environment to enhance passenger media immersion when watching a nature documentary. We evaluated two different fields of view (50°, 104°) and three different levels of ARTV ornamentation (from none to a fully textured jungle environment in the train). We found that on high FOV devices, ARTV ornamentation results in levels of media immersion that are not significantly lower than those achieved with full VR whilst retaining environmental awareness not possible with typical VR solutions. On low FOV devices, ARTV ornamentation loses effectiveness and distract users. Our results show that the immersiveness of AR can be significantly increased while still allowing the awareness of the environment that travellers need to manage their journeys.
Iain Christie, Mark McGill, Stephen A. Brewster
IMX2
2024 Exploring User Placement for VR Remote Collaboration in a Constrained Passenger Space
abstract
Extended Reality (XR) offers the potential to transform the passenger experience by allowing users to inhabit varied virtual spaces for entertainment, work or social interaction, whilst escaping the constrained transit environment. XR allows remote collaborators to feel like they are together and enables them to perform complex 3D tasks. However, the social and physical constraints of the passenger space pose unique challenges to productive and socially acceptable collaboration. Using a collaborative VR puzzle task, we examined the effects of five different f-formations of collaborator placement and orientation in an interactive workspace on social presence, task workload, and implications for social acceptability. Our quantitative and qualitative results showed that face-to-face formations were preferred for tasks with a high need for verbal communication but may lead to social collisions, such as inadvertently staring at a neighbouring passenger, or physical intrusions, such as gesturing in another passenger’s personal space. More restrictive f-formations, however, were preferred for passenger use as they caused fewer intrusions on other passengers’ visual and physical space.
Daniel Medeiros 0001, Graham A. Wilson, Maurício Sousa, Nadia Pantidi, Mark McGill, Diego Drago, Stephen A. Brewster
VRST5
2024 A Critical Review of Virtual and Extended Reality Immersive Police Training: Application Areas, Benefits & Vulnerabilities
abstract
Virtual and Extended Reality (VR/XR) headsets have promised to enhance police training through the delivery of immersive simulations able to be conducted anywhere, anytime. However, little consideration has been given to reviewing the evidenced benefits and potential issues posed by XR police training. In this paper, we summarise the evidenced usage and benefits of XR police training through a formative targeted literature review (n=41 publications). We then reflect on the prospective technical, security, social and legal issues posed by XR police training, identifying four areas where issues or vulnerabilities exist: training content, trainees and trainers, systems and devices, and state and institutional stakeholders. We highlight significant concerns around e.g. the validity of training; the psychological impact and risks of trauma; the safety and privacy risks posed to trainees and trainers; and the risks to policing institutions. We aim to encourage end-user communities (e.g. police forces) to more openly reflect on the risks of immersive training, so we can ultimately move towards transparent, validated, trusted training that is evidenced to improve policing outcomes.
Lena Podoletz, Mark McGill, David McIlhatton, Jill Marshall, Niamh Healy, Leonie Maria Tanczer
VRST2
2024 Exploring the Perspectives of Social VR-Aware Non-Parent Adults and Parents on Children's Use of Social Virtual Reality
abstract
Social Virtual Reality (VR), where people meet in virtual spaces via 3D avatars, is used by children and adults alike. Children experience new forms of harassment in social VR where it is often inaccessible to parental oversight. To date, there is limited understanding of how parents and non-parent adults within the child social VR ecosystem perceive the appropriateness of social VR for different age groups and the measures in place to safeguard children. We present results of a mixed-methods questionnaire (N=149 adults, including 79 parents) focusing on encounters with children in social VR and perspectives towards children's use of social VR. We draw novel insights on the frequency of social VR use by children under 13 and current use of, and future aspirations for, child protection interventions. Compared to non-parent adults, parents familiar with social VR propose lower minimum ages and are more likely to allow social VR without supervision. Adult users experience immaturity from children in social VR, while children face abuse, encounter age-inappropriate behaviours and self-disclose to adults. We present directions to enhance the safety of social VR through pre-planned controls, real-time oversight, post-event insight and the need for evidence-based guidelines to support parents and platforms around age-appropriate interventions.
Cristina Fiani, Pejman Saeghe, Mark McGill, Mohamed Khamis
Proc. ACM Hum. Comput. Interact.3
2024 What Makes XR Dark? Examining Emerging Dark Patterns in Augmented and Virtual Reality through Expert Co-Design
abstract
Dark patterns are deceptive designs that influence a user’s interactions with an interface to benefit someone other than the user. Prior work has identified dark patterns in windows, icons, menus, and pointer (WIMP) interfaces and ubicomp environments, but how dark patterns can manifest in Augmented and Virtual Reality (collectively XR) requires more attention. We therefore conducted 10 co-design workshops with 20 experts in XR and deceptive design. Our participants co-designed 42 scenarios containing dark patterns, based on application archetypes presented in recent HCI/XR literature. In the co-designed scenarios, we identified 10 novel dark patterns in addition to 39 existing ones, as well as 10 examples in which specific characteristics associated with XR potentially amplified the effect dark patterns could have on users. Based on our findings and prior work, we present a classification of XR-specific properties that facilitate dark patterns: perception, spatiality, physical/virtual barriers, and XR device sensing. We also present the experts’ assessments of the likelihood and severity of the co-designed scenarios and highlight key aspects they considered for this evaluation, for example, technological feasibility, ease of upscaling and distributing malicious implementations, and the application’s context of use. Finally, we discuss means to mitigate XR dark patterns and support regulatory bodies to reduce potential harms.
Veronika Krauß, Pejman Saeghe, Alexander Boden, Mohamed Khamis, Mark McGill, Jan Gugenheimer, Michael Nebeling
ACM Trans. Comput. Hum. Interact.5
2024 Is Video Gaming a Cure for Cybersickness? Gamers Experience Less Cybersickness Than Non-Gamers in a VR Self-Motion Task
abstract
Cybersickness remains a major drawback of Virtual Reality (VR) headsets, as a breadth of stationary experiences with visual self-motion can result in visually-induced motion sickness. However, not everybody experiences the same intensity or type of adverse symptoms. Here we propose that prior experience with virtual environments can predict ones degree of cybersickness. Video gaming can enhance visuospatial abilities, which in-turn relate negatively to cybersickness - meaning that consistently engaging in virtual environments can result in protective habituation effects. In a controlled stationary VR experiment, we found that 'VR-naive' video gamers experienced significantly less cybersickness in a virtual tunnel-travel task and outperformed 'VR-naive' non-video gamers on a visual attention task. These findings strongly motivate the use of non-VR games for training VR cybersickness resilience, with future research needed to further understand the mechanism(s) by which gamers become cybersickness resilient - potentially expanding access to VR for even the most susceptible participants.
Katharina Margareta Theresa Pöhlmann, Gang Li 0011, Graham A. Wilson, Mark McGill, Frank E. Pollick, Stephen A. Brewster
IEEE Trans. Vis. Comput. Graph.4
2023 Big Buddy: Exploring Child Reactions and Parental Perceptions towards a Simulated Embodied Moderating System for Social Virtual Reality
abstract
Children experience new forms of harassment in Social Virtual Reality (VR), often inaccessible to parental oversight. We aimed to understand how an artificial intelligent moderator safeguarding children from harassment in social VR is perceived by children and parents, by introducing “Big Buddy”, a Wizard-of-Oz embodied AI-moderator. 43 children (aged 8-16) played a tower-block-construction game in a simulated Social VR classroom where fictitious competitors disrupted their game and, in experimental conditions where present, Big Buddy intervened. We measured children’s perceptions after the disruptions, towards Big Buddy, and the moderation actions it took. Children felt significantly less sad and safer when Big Buddy suspended the saboteur. Parents (n=17) noted Big Buddy’s usefulness and felt reassured but would remain in the supervision loop. We present the first empirical research of a VR-embodied AI-moderator with children’s and parents’ perspectives, and propose design directions for embodied AI-moderators in Social VR.
Cristina Fiani, Robin Bretin, Mark McGill, Mohamed Khamis
IDC3
2023 Manipulating the Orientation of Planar 2D Content in VR as an Implicit Visual Cue for Mitigating Passenger Motion Sickness
abstract
The use of Virtual Reality technology in vehicles is poised to bring about a new era of transport experiences, but its use can cause or enhance motion sickness. However, with careful design, VR devices have the potential to mitigate motion sickness and provide immersive experiences while travelling. We propose a novel implicit motion cue for manipulating a virtual display presented in Virtual and Extended Reality. Our design mitigates motion sickness by providing awareness of physical orientation changes through changes in the visual orientation of the virtual planar content. We performed two experiments on a rotating chair, testing mitigation properties of different levels of rotational gain applied to the display. Our results showed that the technique significantly reduced motion sickness, without negatively affecting task performance. Our findings show that we can subtly interleave motion cues in existing spatial content like planar displays and this can contribute to lessening motion sickness experienced.
Zhanyan Qiu, Mark McGill, Katharina Margareta Theresa Pöhlmann, Stephen A. Brewster
AutomotiveUI2
2023 The Benefits of Passive Haptics and Perceptual Manipulation for Extended Reality Interactions in Constrained Passenger Spaces
abstract
Extended Reality (XR) technology brings exciting possibilities for aeroplane passengers, allowing them to escape their limited cabin space. Using nearby physical surfaces enables a connection with the real world while improving the XR experience through touch. However, available surfaces may be located in awkward positions, reducing comfort and input performance and thus limiting their long-term use. We explore the usability of passive haptic surfaces in different orientations, assessing their effects on input performance, user experience and comfort. We then overcome ergonomic issues caused by the confined space by using perceptual manipulation techniques that remap the position and rotation of physical surfaces and user movements, assessing their effects on task workload, comfort and presence. Our results show that the challenges posed by constrained seating environments can be overcome by a combination of passive haptics and remapping the workspace with moderate translation and rotation manipulations. These manipulations allow for good input performance, low workload and comfortable interaction, opening up XR use while in transit.
Daniel Medeiros 0001, Graham A. Wilson, Mark McGill, Stephen A. Brewster
CHI3
2023 Memory Manipulations in Extended Reality
abstract
Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR.
Elise Bonnail, Wen-Jie Tseng 0003, Mark McGill, Eric Lecolinet, Samuel Huron, Jan Gugenheimer
CHI3
2023 Re-Evaluating VR User Awareness Needs During Bystander Interactions
abstract
Virtual reality (VR) users are often around bystanders, i.e. people in the real world the VR user may want to interact with. To facilitate bystander-VR user interactions, technology-mediated awareness systems have been introduced to increase a user’s awareness of bystanders. However, while prior works have found effective means of facilitating bystander-VR user interactions, it is unclear when and why one awareness system should be used over another. We reviewed, and selected, a breadth of bystander awareness systems from the literature and investigated their usability, and how they could be holistically used together to support varying awareness needs across 14 bystander-VR user interactions. Our results demonstrate VR users do not manage bystander awareness based solely on the usability of awareness systems but rather on the demands of social context weighted against desired immersion in VR (something existing evaluations fail to capture) and show the need for socially intelligent bystander awareness systems.
Joseph O'Hagan, Julie R. Williamson, Florian Mathis, Mohamed Khamis, Mark McGill
CHI5
2023 You spin me right round, baby, right round: Examining the Impact of Multi-Sensory Self-Motion Cues on Motion Sickness During a VR Reading Task
abstract
Motion sickness is a problem for many in everyday travel and will become more prevalent with the rise of automated vehicles. Virtual Reality (VR) headsets have shown significant promise in-transit, enabling passengers to engage in immersive entertainment and productivity experiences. In a controlled multi-session motion sickness study using an actuated rotating chair, we examine the potential of multi-sensory visual and auditory motion cues, presented during a VR reading task, for mitigating motion sickness. We found that visual cues are most efficient in reducing symptoms, with auditory cues showing some beneficial effects when combined with the visual. Motion sickness had negative effects on presence as well as task performance, and despite the cognitive demand and multi-sensory cues, motion sickness still reached problematic levels. Our work emphasises the need for effective mitigations and the design of stronger multi-sensory motion cues if VR is to fulfil its potential for passengers.
Katharina Margareta Theresa Pöhlmann, Gang Li 0011, Mark McGill, Reuben Markoff, Stephen A. Brewster
CHI3
2023 Augmenting People, Places & Media: The Societal Harms Posed by Everyday Augmented Reality, and the Case for Perceptual Human Rights
abstract
Everyday Augmented Reality (AR) displays, with wearable, fashionable, all-day form factors, may one day supplant our reliance on physical displays, heralding new capabilities in augmented intelligence and perception, communication, productivity, and more. Such technology has the potential to become as fundamental to our daily lives as smartphones are today, empowering users, communities, business, governments, and others to alter, augment, diminish or otherwise mediate our perception of reality. For social good, this technology can enable augmenting expression of social identity to better represent our ‘authentic’ self, and virtually enhancing real-world social spaces to encourage greater community ownership and social cohesion. For social harm however, everyday AR could facilitate and amplify manipulation, information disorder (e.g. dis-information), censorship and coercion in our day-to-day experience of reality. In this essay, we consider some of the key societal changes (and ethical challenges) posed by the adoption of everyday AR, and argue that everyday AR will provoke the need for new human rights to be considered alongside proposed neurorights and existing and envisaged digital human rights, around: who can mediate reality (perceptual autonomy); what elements of reality are permissible to alter/augment (perceptual agency); and governing permissible intent regarding why we augment the user’s perception of reality, in particular considering tensions in cognitive autonomy (e.g. manipulation) and perceptual integrity (e.g. information disorder).
Joseph O'Hagan, Jan Gugenheimer, Jolie Bonner, Florian Mathis, Mark McGill
MUM5
2023 User Reviews as a Reporting Mechanism for Emergent Issues Within Social VR Communities
abstract
Social virtual reality (VR) will potentially change online communities, transforming how people meet, interact, and socialise online. However, issues surrounding the growing threat of harassment (from verbal to cyber-physical assault) posed within social VR platforms are increasingly being noted within research.Yet, while prior research has successfully captured singular qualitative insights (e.g. identifying individual types of harassment or target user groups) it is essential that systems be developed to monitor emergent threats to community cohesion, including harassment, in a platform independent manner. In this paper, we explore whether user reviews are a valid and useful source of data around harassment incidences in social VR, a crucial first step towards platform agnostic, large scale and longitudinal monitoring of emerging harassment in social VR communities through user reviews posted about social VR applications.. We analysed 1000 user reviews posted on the Meta Quest store page for the social VR platform Rec Room and found 114 reviews report experiencing and/or witnessing harassment during use. Our results establish the validity of analysing user reviews as a means of monitoring harassment in social VR communities and we close by discussing their use in large scale, longitudinal monitoring systems.
Joseph O'Hagan, Florian Mathis, Mark McGill
MUM3
2023 Exploring Neural Biomarkers in Young Adults Resistant to VR Motion Sickness: A Pilot Study of EEG
abstract
VR (Virtual Reality) Motion Sickness (VRMS) refers to purely visually-induced motion sickness. Not everyone is susceptible to VRMS, but if experienced, nausea will often lead users to withdraw from the ongoing VR applications. VRMS represents a serious challenge in the field of VR ergonomics and human factors. Like other neuro-ergonomics researchers did before, this paper considers VRMS as a brain state problem as various etiologies of VRMS support the claim that VRMS is caused by disagreement between the vestibular and visual sensory inputs. However, what sets this work apart from the existing literature is that it explores anti-VRMS brain patterns via electroencephalogram (EEG) in VRMS-resistant individuals. Based on existing datasets of a previous study, we found enhanced theta activity in the left parietal cortex in VRMS-resistant individuals (N= 10) compared to VRMS-susceptible individuals (N=10). Even though the sample size per se is not large, this finding achieved medium effect size. This finding offers new hypotheses regarding how to reduce VRMS by the enhancement of brain functions per se (e.g., via non-invasive transcranial electrostimulation techniques) without the need to redesign the existing VR content.
Gang Li 0011, Katharina Margareta Theresa Pöhlmann, Mark McGill, Chao Ping Chen, Stephen A. Brewster, Frank E. Pollick
VR3
2023 When Filters Escape the Smartphone: Exploring Acceptance and Concerns Regarding Augmented Expression of Social Identity for Everyday AR
abstract
Mass adoption of Everyday Augmented Reality (AR) glasses will enable pervasive augmentation of our expression of social identity through AR filters, transforming our perception of self and others. However, despite filters’ prominent and often problematic usage in social media, research has yet to reflect on the potential impact AR filters might have when brought into everyday life. Informed by our survey of 300 existing popular AR filters used on Snapchat, Instagram and Tiktok, we conducted an AR-in-VR user study where participants (N=24) were exposed to 18 filters across six categories. We evaluated the social acceptability of these augmentations around others and attitudes towards an individual’s augmented self.Our findings highlight 1) how users broadly respected another individual’s augmented self; 2) positive use cases, such as supporting the presentation of gender identity; and 3) tensions around applying AR filters to others (e.g. censorship, changing protected characteristics) and their impact on self-perception (e.g. perpetuating unrealistic beauty standards). We raise questions regarding the rights of individuals to augment and be augmented that provoke the need for further consideration of AR augmentations in society.
Jolie Bonner, Florian Mathis, Joseph O'Hagan, Mark McGill
VRST4
2023 A Lack of Restraint: Comparing Virtual Reality Interaction Techniques for Constrained Transport Seating
abstract
Standalone Virtual Reality (VR) headsets can be used when travelling in cars, trains and planes. However, the constrained spaces around transport seating can leave users with little physical space in which to interact using their hands or controllers, and can increase the risk of invading other passengers' personal space or hitting nearby objects and surfaces. This hinders transport VR users from using most commercial VR applications, which are designed for unobstructed 1-2m 360° home spaces. In this paper, we investigated whether three at-a-distance interaction techniques from the literature could be adapted to support common commercial VR movement inputs and so equalise the interaction capabilities of at-home and on-transport users: Linear Gain, Gaze-Supported Remote Hand, and AlphaCursor. First, we analysed commercial VR experiences to identify the most common movement inputs so that we could create gamified tasks based on them. We then investigated how well each technique could support these inputs from a constrained $50\mathrm{x}50\text{cm}$ space (representative of an economy plane seat) through a user study $(\mathrm{N}=16)$, where participants played all three games with each technique. We measured task performance, unsafe movements (play boundary violations, total arm movement) and subjective experience and compared results to a control 'at-home' condition (with unconstrained movement) to determine how similar performance and experience were. Results showed that Linear Gain was the best technique, with similar performance and user experience to the 'at-home' condition, albeit at the expense of a high number of boundary violations and large arm movements. In contrast, AlphaCursor kept users within bounds and minimised arm movement, but suffered from poorer performance and experience. Based on the results, we provide eight guidelines for the use of, and research into, at-a-distance techniques and constrained spaces.
Graham A. Wilson, Mark McGill, Daniel Medeiros 0001, Stephen A. Brewster
IEEE Trans. Vis. Comput. Graph.2
2022 Exploring Manipulating In-VR Audio To Facilitate Verbal Interactions Between VR Users And Bystanders
abstract
Despite recent work investigating how VR users can be made aware of bystanders, few have explored how bystander-VR user interactions may be facilitated by, for example, increasing the user’s auditory awareness so they can better converse with bystanders. Through a lab study (N=15) we investigated 4 approaches of manipulating in-VR audio to facilitate verbal interactions between a VR user and bystander: (1) dynamically reducing application volume, (2) removing background audio, (3) removing sound effects and (4) removing all audio. Our results show audio manipulations can be used to significantly improve a VR user’s auditory awareness at the cost of reducing sense of presence in VR. They also show most preferred increased awareness be balanced with decreased presence in VR, however, they also identify a subset of participants who prioritised increasing awareness no matter the cost to presence.
Joseph O'Hagan, Julie R. Williamson, Mohamed Khamis, Mark McGill
AVI4
2022 The Dark Side of Perceptual Manipulations in Virtual Reality
abstract
“Virtual-Physical Perceptual Manipulations” (VPPMs) such as redirected walking and haptics expand the user’s capacity to interact with Virtual Reality (VR) beyond what would ordinarily physically be possible. VPPMs leverage knowledge of the limits of human perception to effect changes in the user’s physical movements, becoming able to (perceptibly and imperceptibly) nudge their physical actions to enhance interactivity in VR. We explore the risks posed by the malicious use of VPPMs. First, we define, conceptualize and demonstrate the existence of VPPMs. Next, using speculative design workshops, we explore and characterize the threats/risks posed, proposing mitigations and preventative recommendations against the malicious use of VPPMs. Finally, we implement two sample applications to demonstrate how existing VPPMs could be trivially subverted to create the potential for physical harm. This paper aims to raise awareness that the current way we apply and publish VPPMs can lead to malicious exploits of our perceptual vulnerabilities.
Wen-Jie Tseng 0003, Elise Bonnail, Mark McGill, Mohamed Khamis, Eric Lecolinet, Samuel Huron, Jan Gugenheimer
CHI3
2022 Exploring Attitudes Towards Increasing User Awareness of Reality From Within Virtual Reality
abstract
The occlusive nature of VR headsets introduces significant barriers to a user’s awareness of their surrounding reality. While recent research has explored systems to facilitate a VR user’s interactions with nearby people, objects, etc, we lack a fundamental understanding of user attitudes towards and expectations of these systems. We present the results of a card sorting study (N=14) which investigated attitudes towards increasing a VR user’s reality awareness (awareness of people, objects, audio, pets, and systems to manage and moderate personal usage) whilst in VR. Our results confirm VR headsets should be equipped with systems to increase a user’s awareness of reality. However, opinions vary on how increased awareness should be achieved as our results also highlight differing expectations regarding: persistent vs temporary notification design, notification content and when, why and how awareness should be increased.
Joseph O'Hagan, Mohamed Khamis, Mark McGill, Julie R. Williamson
IMX3
2022 Evaluating and Updating a Design Space for Augmented Reality Television
abstract
As Augmented Reality Television (ARTV) transitions out of the feasibility phase, it is crucial to understand the impact of design decisions on the viewers’ ARTV experiences. In a previous study, six ARTV design dimensions were identified by relying on insights from existing prototypes. However, the set of possible dimensions is likely to be broader. Building on top of previous work, we create an ARTV design space and present it in a textual cheat sheet. We subsequently evaluate the cheat sheet in a between-subject study (n = 10), with participants with wide-ranging expertise. We identified six new dimensions (genre, broadcast mode, audience demographics, cartoonish vs. photoreal representation, modality, and privacy), and a new aspect (360°) for the display dimension. In light of our observations, we provide an updated ARTV design space and observe that asking participants to write ARTV scenarios can be an effective method for harvesting novel design dimensions.
Pejman Saeghe, Mark McGill, Bruce Weir, Sarah Clinch, Robert Stevens 0001
IMX2
2022 Augmenting a Nature Documentary with a Lifelike Hologram in Virtual Reality
abstract
While augmented reality television (ARTV) is being investigated in research labs, the high cost of AR headsets makes it difficult for audiences to benefit from the research. However, the relative affordability of virtual reality (VR) headsets provides ARTV researchers with opportunities to test their prototypes in VR. Additionally, as VR becomes an acceptable medium for watching conventional TV, augmenting such viewing experiences in VR creates new opportunities. We prototype a nature documentary ARTV experience in VR and conduct a remote user study (n = 10) to investigate six points on the visual display design dimension of presenting a lifelike programme-related hologram. We manipulated the starting point and the movement behaviour of the hologram to gain insight into viewer preferences. Our findings highlight the importance of personal preferences and that of the perceived role of a hologram in relation to the underlying TV content; suggesting there may not be a single way to augment a TV programme. Instead, creators may need to provide the audiences with capabilities to customise ARTV content.
Pejman Saeghe, Bruce Weir, Mark McGill, Sarah Clinch, Robert Stevens 0001
IMX3
2022 PassengXR: A Low Cost Platform for Any-Car, Multi-User, Motion-Based Passenger XR Experiences
abstract
We present PassengXR, an open-source toolkit for creating passenger eXtended Reality (XR) experiences in Unity. XR allows travellers to move beyond the physical limitations of in-vehicle displays, rendering immersive virtual content based on - or ignoring - vehicle motion. There are considerable technical challenges to using headsets in moving environments: maintaining the forward bearing of IMU-based headsets; conflicts between optical and inertial tracking of inside-out headsets; obtaining vehicle telemetry; and the high cost of design given the necessity of testing in-car. As a consequence, existing vehicular XR research typically relies on controlled, simple routes to compensate. PassengXR is a cost-effective open-source in-car passenger XR solution. We provide a reference set of COTS hardware that enables the broadcasting of vehicle telemetry to multiple headsets. Our software toolkit then provides support to correct vehicle-headset alignment, and then create a variety of passenger XR experiences, including: vehicle-locked content; motion- and location-based content; and co-located multi-passenger applications. PassengXR also supports the recording and playback of vehicle telemetry, assisting offline design without resorting to costly in-car testing. Through an evaluation-by-demonstration, we show how our platform can assist practitioners in producing novel, multi-user passenger XR experiences.
Mark McGill, Graham A. Wilson, Daniel Medeiros 0001, Stephen A. Brewster
UIST1
2022 Multimodal Biosensing for Vestibular Network-Based Cybersickness Detection
abstract
Virtual reality (VR) has the potential to induce cybersickness (CS), which impedes CS-susceptible VR users from the benefit of emerging VR applications. To better detect CS, the current study investigated whether/how the newly proposed human vestibular network (HVN) is involved in flagship consumer VR-induced CS by simultaneously recording autonomic physiological signals as well as neural signals generated in sensorimotor and cognitive domains. The VR stimuli were made up of one or two moderate CS-inducing entertaining task(s) as well as a mild CS-inducing cognitive task implemented before and after the moderate CS task(s). Results not only showed that CS impaired cognitive control ability, represented by the degree of attentional engagement, but also revealed that combined indicators from all three HVN domains could together establish the best regression relationship with CS ratings. More importantly, we found that every HVN domain had its unique advantage with the dynamic changes in CS severity and time. These results provide evidence for involvement of the HVN in CS and indicate the necessity of HVN-based CS detection.
Gang Li 0011, Mark McGill, Stephen A. Brewster, Chao Ping Chen, Joaquin A. Anguera, Adam Gazzaley, Frank E. Pollick
IEEE J. Biomed. Health Informatics2
2022 Creating and Augmenting Keyboards for Extended Reality with the Keyboard Augmentation Toolkit
abstract
This article discusses the Keyboard Augmentation Toolkit (KAT), which supports the creation of virtual keyboards that can be used both for standalone input (e.g., for mid-air text entry) and to augment physically tracked keyboards/surfaces in mixed reality. In a user study, we firstly examine the impact and pitfalls of visualising shortcuts on a tracked physical keyboard, exploring the utility of virtual per-keycap displays. Supported by this and other recent developments in XR keyboard research, we then describe the design, development, and evaluation-by-demonstration of KAT. KAT simplifies the creation of virtual keyboards (optionally bound to a tracked physical keyboard) that support enhanced display —2D/3D per-key content that conforms to the virtual key bounds; enhanced interactivity —supporting extensible per-key states such as tap, dwell, touch, swipe; flexible keyboard mappings that can encapsulate groups of interaction and display elements, e.g., enabling application-dependent interactions; and flexible layouts —allowing the virtual keyboard to merge with and augment a physical keyboard, or switch to an alternate layout (e.g., mid-air) based on need. Through these features, KAT will assist researchers in the prototyping, creation and replication of XR keyboard experiences, fundamentally altering the keyboard’s form and function.
Mark McGill, Stephen A. Brewster, Daniel Medeiros 0001, Sidney Bovet, Mario Gutierrez, Aidan Kehoe
ACM Trans. Comput. Hum. Interact.1
2022 From Shielding to Avoidance: Passenger Augmented Reality and the Layout of Virtual Displays for Productivity in Shared Transit
abstract
Passengers spend considerable periods of time in shared transit spaces, relying on smartphones and laptops for work. However, these displays are limited in size and ergonomics compared to typical multi-monitor setups used in the office, impairing productivity. Augmented Reality (AR) headsets could provide large, flexible virtual workspaces during travel, enabling passengers to work more efficiently. This paper investigates the factors affecting how passengers choose to layout virtual displays in car, train, subway and plane environments, studying the affordances of each mode of transport and the presence of others. Results from our experiment showed: significant usage of the physical environment to align displays; strong social effects meant avoiding placing displays over other passengers or their belongings; and use of displays for shielding oneself from others. Our findings show the unique challenges posed by the mode of transport and presence of others on the use of AR for mobile productivity in the future.
Daniel Medeiros 0001, Mark McGill, Alexander Ng, Robert McDermid, Nadia Pantidi, Julie R. Williamson, Stephen A. Brewster
IEEE Trans. Vis. Comput. Graph.2
2021 The Passenger Experience of Mixed Reality Virtual Display Layouts in Airplane Environments
abstract
Augmented / Mixed Reality headsets will in-time see adoption and use in a variety of mobility and transit contexts, allowing users to view and interact with virtual content and displays for productivity and entertainment. However, little is known regarding how multi-display virtual workspaces should be presented in a transit context, nor to what extent the unique affordances of transit environments (e.g. the social presence of others) might influence passenger perception of virtual display layouts. Using a simulated VR passenger airplane environment, we evaluated three different AR-driven virtual display configurations (Horizontal, Vertical, and Focus main display with smaller secondary windows) at two different depths, exploring their usability, user preferences, and the underlying factors that influenced those preferences. We found that the perception of invading other’s personal space significantly influenced preferred layouts in transit contexts. Based on our findings, we reflect on the unique challenges posed by passenger contexts, provide recommendations regarding virtual display layout in the confined airplane environment, and expand on the significant benefits that AR offers over physical displays in said environments.
Alexander Ng, Daniel Medeiros 0001, Mark McGill, Julie R. Williamson, Stephen A. Brewster
ISMAR3
2021 Safety, Power Imbalances, Ethics and Proxy Sex: Surveying In-The-Wild Interactions Between VR Users and Bystanders
abstract
VR users and bystanders must sometimes interact, but our understanding of these interactions - their purpose, how they are accomplished, attitudes toward them, and where they break down - is limited. This current gap inhibits research into managing or supporting these interactions, and preventing unwanted or abusive activity. We present the results of the first survey (N=100) that investigates stories of actual emergent in-the-wild interactions between VR users and bystanders. Our analysis indicates VR user and bystander interactions can be categorised into one of three categories: coexisting, demoing, and interrupting. We highlight common interaction patterns and impediments encountered during these interactions. Bystanders play an important role in moderating the VR user’s experience, for example intervening to save the VR user from potential harm. However, our stories also suggest that the occlusive nature of VR introduces the potential for bystanders to exploit the vulnerable state of the VR user; and for the VR user to exploit the bystander for enhanced immersion, introducing significant ethical concerns.
Joseph O'Hagan, Julie R. Williamson, Mark McGill, Mohamed Khamis
ISMAR3
2020 Acoustic Transparency and the Changing Soundscape of Auditory Mixed Reality
abstract
Auditory headsets capable of actively or passively intermixing both real and virtual sounds are in-part acoustically transparent. This paper explores the consequences of acoustic transparency, both on the perception of virtual audio content, given the presence of a real-world auditory backdrop, and more broadly in facilitating a wearable, personal, private, always-available soundspace. We experimentally compare passive acoustically transparent, and active noise cancelling, orientation-tracked auditory headsets across a range of content types, both indoors and outdoors for validity. Our results show differences in terms of presence, realness and externalization for select content types. Via interviews and a survey, we discuss attitudes toward acoustic transparency (e.g. being perceived as safer), the potential shifts in audio usage that might be precipitated by adoption, and reflect on how such headsets and experiences fit within the area of Mixed Reality.
Mark McGill, Stephen A. Brewster, David K. McGookin, Graham A. Wilson
CHI1
2020 Assessing Social Text Placement in Mixed Reality TV
abstract
TV experiences are often social, be it at-a-distance (through text) or in-person (through speech). Mixed Reality (MR) headsets offer new opportunities to enhance social communication during TV viewing by placing social artifacts (e.g. text) anywhere the viewer wishes, rather than being constrained to a smartphone or TV display. In this paper, we use VR as a test-bed to evaluate different text locations for MR TV specifically. We introduce the concepts of wall messages, below-screen messages, and egocentric messages in addition to state-of-the-art on-screen messages (i.e., subtitles) and controller messages (i.e., reading text messages on the mobile device) to convey messages to users during TV viewing experiences. Our results suggest that a) future MR systems that aim to improve viewers’ experience need to consider the integration of a communication channel that does not interfere with viewers’ primary task, that is watching TV, and b) independent of the location of text messages, users prefer to be in full control of them, especially when reading and responding to them. Our findings pave the way for further investigations towards social at-a-distance communication in Mixed Reality.
Florian Mathis, Xuesong Zhang 0002, Mark McGill, Adalberto L. Simeone, Mohamed Khamis
IMX3
2020 Augmenting TV Viewing using Acoustically Transparent Auditory Headsets
abstract
This paper explores how acoustically transparent auditory headsets can improve TV viewing by intermixing headset and TV audio, facilitating personal, private auditory enhancements and augmentations of TV content whilst minimizing occlusion of the sounds of reality. We evaluate the impact of synchronously mirroring select audio channels from the 5.1 mix (dialogue, environmental sounds, and the full mix), and selectively augmenting TV viewing with additional speech (e.g. Audio Description, Directors Commentary, and Alternate Language). For TV content, auditory headsets enable better spatialization and more immersive, enjoyable viewing; the intermixing of TV and headset audio creates unique listening experiences; and private augmentations offer new ways to (re)watch content with others. Finally, we reflect on how these headsets might facilitate more immersive augmented TV viewing experiences within reach of consumers.
Mark McGill, Florian Mathis, Mohamed Khamis, Julie R. Williamson
IMX1
2020 A Quest for Co-Located Mixed Reality: Aligning and Assessing SLAM Tracking for Same-Space Multi-User Experiences
abstract
Current solutions for creating co-located Mixed Reality (MR) experiences typically rely on platform-specific synchronisation of spatial anchors or Simultaneous Localisation and Mapping (SLAM) data across clients, often coupled to cloud services. This introduces significant costs (in development and deployment), constraints (with interoperability across platforms often limited), and privacy concerns. For practitioners, support is needed for creating platform-agnostic co-located MR experiences. This paper explores the utility of aligned SLAM solutions by 1) surveying approaches toward aligning disparate device coordinate spaces, formalizing their theoretical accuracy and limitations; 2) providing skeleton implementations for audience-based, small-scale and large-scale co-location using said alignment approaches; and 3) detailing how we can assess the accuracy and safety of 6DoF/SLAM tracking solutions for any arbitrary device and dynamic environment without the need for an expensive ground truth optical tracking, by using trilateration and a $30 laser distance meter. Through this, we hope to further democratise the creation of cross-platform co-located MR experiences.
Mark McGill, Jan Gugenheimer, Euan Freeman
VRST1
2020 Expanding the Bounds of Seated Virtual Workspaces
abstract
Mixed Reality (MR), Augmented Reality (AR) and Virtual Reality (VR) headsets can improve upon existing physical multi-display environments by rendering large, ergonomic virtual display spaces whenever and wherever they are needed. However, given the physical and ergonomic limitations of neck movement, users may need assistance to view these display spaces comfortably. Through two studies, we developed new ways of minimising the physical effort and discomfort of viewing such display spaces. We first explored how the mapping between gaze angle and display position could be manipulated, helping users view wider display spaces than currently possible within an acceptable and comfortable range of neck movement. We then compared our implicit control of display position based on head orientation against explicit user control, finding significant benefits in terms of user preference, workload and comfort for implicit control. Our novel techniques create new opportunities for productive work by leveraging MR headsets to create interactive wide virtual workspaces with improved comfort and usability. These workspaces are flexible and can be used on-the-go, e.g., to improve remote working or make better use of commuter journeys.
Mark McGill, Aidan Kehoe, Euan Freeman, Stephen A. Brewster
ACM Trans. Comput. Hum. Interact.1
2019 PlaneVR: Social Acceptability of Virtual Reality for Aeroplane Passengers
abstract
Virtual reality (VR) headsets allow wearers to escape their physical surroundings, immersing themselves in a virtual world. Although escape may not be realistic or acceptable in many everyday situations, air travel is one context where early adoption of VR could be very attractive. While travelling, passengers are seated in restricted spaces for long durations, reliant on limited seat-back displays or mobile devices. This paper explores the social acceptability and usability of VR for in-flight entertainment. In an initial survey, we captured respondents' attitudes towards the social acceptability of VR headsets during air travel. Based on the survey results, we developed a VR in-flight entertainment prototype and evaluated this in a focus group study. Our results discuss methods for improving the acceptability of VR in-flight, including using mixed reality to help users transition between virtual and physical environments and supporting interruption from other co-located people.
Julie R. Williamson, Mark McGill, Khari Outram
CHI2
2018 Object Manipulation in Virtual Reality Under Increasing Levels of Translational Gain
abstract
Room-scale Virtual Reality (VR) has become an affordable consumer reality, with applications ranging from entertainment to productivity. However, the limited physical space available for room-scale VR in the typical home or office environment poses a significant problem. To solve this, physical spaces can be extended by amplifying the mapping of physical to virtual movement (translational gain). Although amplified movement has been used since the earliest days of VR, little is known about how it influences reach-based interactions with virtual objects, now a standard feature of consumer VR. Consequently, this paper explores the picking and placing of virtual objects in VR for the first time, with translational gains of between 1x (a one-to-one mapping of a 3.5m*3.5m virtual space to the same sized physical space) and 3x (10.5m*10.5m virtual mapped to 3.5m*3.5m physical). Results show that reaching accuracy is maintained for up to 2x gain, however going beyond this diminishes accuracy and increases simulator sickness and perceived workload. We suggest gain levels of 1.5x to 1.75x can be utilized without compromising the usability of a VR task, significantly expanding the bounds of interactive room-scale VR.
Graham A. Wilson, Mark McGill, Matthew Jamieson, Julie R. Williamson, Stephen A. Brewster
CHI2
2017 I Am The Passenger: How Visual Motion Cues Can Influence Sickness For In-Car VR
abstract
This paper explores the use of VR Head Mounted Displays (HMDs) in-car and in-motion for the first time. Immersive HMDs are becoming everyday consumer items and, as they offer new possibilities for entertainment and productivity, people will want to use them during travel in, for example, autonomous cars. However, their use is confounded by motion sickness caused in-part by the restricted visual perception of motion conflicting with physically perceived vehicle motion (accelerations/rotations detected by the vestibular system). Whilst VR HMDs restrict visual perception of motion, they could also render it virtually, potentially alleviating sensory conflict. To study this problem, we conducted the first on-road and in motion study to systematically investigate the effects of various visual presentations of the real-world motion of a car on the sickness and immersion of VR HMD wearing passengers. We established new baselines for VR in-car motion sickness, and found that there is no one best presentation with respect to balancing sickness and immersion. Instead, user preferences suggest different solutions are required for differently susceptible users to provide usable VR in-car. This work provides formative insights for VR designers and an entry point for further research into enabling use of VR HMDs, and the rich experiences they offer, when travelling.
Mark McGill, Alexander Ng, Stephen A. Brewster
CHI1
2016 Examining The Role of Smart TVs and VR HMDs in Synchronous At-a-Distance Media Consumption
Mark McGill, John Williamson 0001, Stephen A. Brewster
ACM Trans. Comput. Hum. Interact.1
2015 A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays
abstract
We identify usability challenges facing consumers adopting Virtual Reality (VR) head-mounted displays (HMDs) in a survey of 108 VR HMD users. Users reported significant issues in interacting with, and being aware of their real-world context when using a HMD. Building upon existing work on blending real and virtual environments, we performed three design studies to address these usability concerns. In a typing study, we show that augmenting VR with a view of reality significantly corrected the performance impairment of typing in VR. We then investigated how much reality should be incorporated and when, so as to preserve users' sense of presence in VR. For interaction with objects and peripherals, we found that selectively presenting reality as users engaged with it was optimal in terms of performance and users' sense of presence. Finally, we investigated how this selective, engagement-dependent approach could be applied in social environments, to support the user's awareness of the proximity and presence of others.
Mark McGill, Daniel Boland, Roderick Murray-Smith, Stephen A. Brewster
CHI1
2015 A review of collocated multi-user TV - Examining the changing role of the TV in the multi-viewer, multi-screen home
Mark McGill, John Williamson 0001, Stephen A. Brewster
Pers. Ubiquitous Comput.1
2009 Head tilting for interaction in mobile contexts
abstract
Developing interfaces for mobile situations requires that devices are useable on the move. Here, we explore head tilting as an input technique to allow a user to interact with a mobile device 'hands free'. A Fitts' Law style evaluation is described where a user acquires targets, moving the cursor by head tilt. We explore d position and velocity control cursor mechanisms in both static and mobile situations to see which provided the best level of performance. Results show that participants could successfully acquire targets using head tilting. Position control was shown to be si gnificantly faster and more accurate in a static context, but exhi bited significantly poorer accuracy and longer target acquisition times when the user was on the move. We further demonstrate how analysis of user's gait shows consistent targeting biases at different stages in the gait cycle.
Andrew Crossan, Mark McGill, Stephen A. Brewster, Roderick Murray-Smith
Mobile HCI2