Prior research has highlighted numerous accessibility barriers within virtual reality software, with guidelines emerging to address the requirements of diverse audiences. However, an empirical understanding of industry practitioner implementation of accessible guidelines within mainstream commercial applications is currently lacking. This review addresses this gap by categorising all accessibility features presented at a software-level in 330 of the most used virtual reality applications released between 2016 to 2023 on the Steam, Meta, Oculus, Viveport, and SideQuest platforms. Results suggest a growing lack of interaction customisation, with the number of applications allowing for alternative inputs and physical posture flexibility decreasing. Meanwhile, display output settings, such as text font resizing and colourblind alterations, are almost completely absent. Our findings highlight the evolution in the implementation of accessible features in virtual reality software, contributing to a representative overview of practitioner decisions, and acting as a catalyst towards the establishment of industry-wide guidelines.

Exploration of virtual reality locomotion has a rich history, including in the creation of taxonomies categorising individual techniques. However, most existing research collects data from academic sources only, with both historic industry practitioner exploration and a state-of-the-art understanding of locomotion in commercial applications comparatively underexplored. This systematic software-level review of the complete locomotion options in 330 of the most used virtual reality applications released between 2016 and 2023 on the Steam, Meta, Oculus, Viveport, and SideQuest platforms highlights the trends and gaps that exist between industry and academic exploration. Results suggest a decline in the usage of teleportation, with the prevalence of titles containing at least one teleportation technique decreasing from 48% of those released in 2016 to 18% in 2023. Arm-tracked grabbing locomotion techniques such as climbing meanwhile are being increasingly adopted by practitioners, from almost unused in 3% of applications released in 2016 to over 30% in each year between 2020 and 2023. Additionally, although the tracking capabilities afforded by consumer-level head-mounted display hardware has resulted in a high exploration of room-scale tracking, the large academic focus on walking-based locomotion appears to not be shared by practitioners, where room-scale tracking instead is most often paired with conventional controller joystick sliding locomotion. Finally, temporal analysis results showing the growing number of locomotion techniques offered in an average application signifies the need for further accessibility-related locomotion research, particularly in areas beyond visual sickness mitigation. Our findings highlight the continuing evolution of locomotion in commercial virtual reality applications, with industry practitioner locomotion technique adoption rates displaying the divergent interests between industry and academia, in turn adding rigour to future locomotion selections across both domains.

Augmented and virtual reality experiences present significant barriers for disabled people, making it challenging to fully engage with immersive platforms. Whilst researchers have started to explore potential solutions addressing these accessibility issues, we currently lack a comprehensive understanding of research areas requiring further investigation to support the development of inclusive AR/VR systems. To address current gaps in knowledge, we led a series of multidisciplinary sandpits with relevant stakeholders (i.e., academic researchers, industry specialists, people with lived experience of disability, assistive technologists, and representatives from disability organisations, charities, and special needs educational institutions) to collaboratively explore research challenges, opportunities, and solutions. Based on insights shared by participants, we present a research agenda identifying key areas where further work is required in relation to specific forms of disability (i.e., across the spectrum of physical, visual, cognitive, and hearing impairments), including wider considerations associated with the development of more accessible immersive platforms.

Conventionally, the accessibility research community centers most of its efforts on designing assistive technologies and systems related to single categories of impairments. Although this approach has contributed to valuable progress and advancements in the field, there is a growing consensus among accessibility researchers that focusing on designing technologies for single impairments oversimplifies disability since this approach may fail to adequately address the real-world experiences of a significant population of users with complex needs. Despite challenges related to conducting research with users living with multiple, profound, or complex disabilities, it is essential in terms of future work that the accessibility research community adopts a more inclusive approach where users with lived experience of multiple disabilities are directly informing and shaping the design of assistive systems and accessible technologies. Therefore, we propose the 2nd International Workshop on Accessibility Research and Users with Multiple Disabilities or Complex Needs. The workshop will act as a forum for participants to share their perspectives related to challenges and opportunities in designing accessible systems that consider the multidimensional needs of users living with multiple disabilities. This workshop intends to challenge current paradigms in the accessibility field, share latest work and foster future collaborations.

Voice input holds significant potential to support people with physical impairments in producing creative visual design outputs, although it is unclear whether well-established interaction methods used for manipulating graphical assets within mainstream creative applications (typically operated via a mouse, keyboard, or touch input) also present benefits for speech interaction. We present three new voice controlled approaches utilizing interface snapping techniques for manipulating a graphical object's dimensions: NoSnap, UserSnap, and AutoSnap. A user evaluation with people who have physical impairments (N=25) found that each method enabled participants to successfully control a graphical object's size across a series of design tasks, although the automated snapping approach utilized within AutoSnap was found to be more efficient, accurate, and usable. Subjective feedback from participants also highlighted a strong preference for AutoSnap over the other techniques in terms of efficiency and ease of use.

Historically, the accessibility community has focused on designing assistive technologies and systems related to single categories of impairments. While this approach is justifiable and contributes to the advancement of the field, many researchers argue that designing technologies for one impairment at a time contributes to an oversimplification of disability and does not reflect real-world experiences of a significant number of users who live with multiple impairments. How can the accessibility field leverage theories and concepts from other disciplines (e.g. disability studies, medical humanities, social work, engineering, education) to support more appropriate and inclusive technologies for users with complex needs? How can future assistive technologies take into consideration a diverse combination of sensory, cognitive, and physical characteristics? In this workshop, we invite researchers and practitioners from different disciplines to share their perspectives related to challenges and opportunities in designing accessible systems that consider the multidimensional and often complex needs of users living with multiple impairments.

The ubiquity of smartphones and the widespread usage of text entry by soft keyboard in different instant messaging applications (e.g., WhatsApp, FB messenger) have opened the possibilities of inferring emotions from longitudinal typing data. To build this emotion inference engine, we apply machine learning models on features extracted from user’s typing patterns (not content). However, one major challenge encountered while developing the emotion inference model is the requirement of individual training data as typing patterns are often person-specific. In this paper, we investigate the possibility of combining typing pattern with emotion self-reporting to identify a group of similar users so that the training data among these users can be shared to fulfill the requirement of personalized dataset. We develop a framework AffectPro, which quantifies the typing interaction behavior (e.g., typing speed, error rate) and self-reporting pattern (e.g., emotion state transition probability) to construct the affective profiles of users. We evaluated AffectPro in a 6-week in-the-wild study involving 28 users, who used an Android application encompassing a custom keyboard to perform all their typing activities, and to report their instantaneous emotions. We extracted different typing signatures and self-report behavior details from the collected dataset (≈ 5000 typing sessions, ≈ 108 hours of typing data) to construct the affective profile of users. Our results demonstrate similarity across users in terms of typing signature, emotion self-reporting pattern, and a combination of both; which can be leveraged to share training data among similar users to overcome the challenges of personalized data collection.

Text selection is an essential activity in interactive systems, including virtual reality (VR) head-mounted displays (HMDs). It is useful for: sharing information across apps or platforms, highlighting and making notes while reading articles, and text editing tasks. Despite its usefulness, the space of text selection interaction is underexplored in VR HMDs. In this research, we performed a user study with 24 participants to investigate the performance and user preference of six text selection techniques (Controller+Dwell, Controller+Click, Head+Dwell, Head+Click, Hand+Dwell, Hand+Pinch). Results reveal that Head+Click is ranked first since it has excellent speedaccuracy performance (2nd fastest task completion speed with 3rd lowest total error rate), provides the best user experience, and produces a very low workload—followed by Controller+Click, which has the fastest speed and comparable experience with Head+Click, but much higher total error rate. Other methods can also be useful depending on the goals of the system or the users. As a first systematic evaluation of pointing $\times$ selection techniques for text selection in VR, the results of this work provide a strong foundation for further research in this area of growing importance to the future of VR to help it become a more ubiquitous and pervasive platform.

We propose a novel phone-case-shaped interface named TetraForce, which enables four types of force input consisting of two force directions (i.e., pressure force and shear force) and two force-applied surfaces (i.e., touch surface and back surface) in a single device. Force detection is achieved using the smartphone's built-in magnetometer (and supplementary accelerometer and gyro sensor) by estimating the displacement of a magnet attached to a 3-DoF passively movable panel at the back. We conducted a user study (N=12) to investigate the fundamental user performance by our interface and demonstrated that the input was detected as intended with a success rate of 97.4% on average for all four input types. We further conducted an ideation workshop with people who were involved in human-computer interaction (N=12) to explore possible applications of this interface, and we obtained 137 ideas for applications using individual input types and 51 possible scenarios using them in combination. Organizing these ideas reveals the advantages of each input type and suggests that our interface is useful for applications that require complex operations and that it can improve intuitiveness through elastic feedback.

Deaf and hard of hearing (DHH) people have limited access to auditory input, so they mainly receive visual information during online meetings. In recent years, the usability of a system that visualizes the ongoing topic in a conference has been confirmed, but it has not been verified in a remote conference that includes DHH people. One possible reason is that visual dispersion occurs when there are multiple sources of visual information. In this study, we introduce “Word Cloud for Meeting,” a system that generates a separate word cloud for each participant and displays it in the background of each participant’s video to visualize who is saying what. We conducted an experiment with seven DHH participants and obtained positive qualitative feedback on the ease of recognizing topic changes. However, when the topic changed in a sequence, it was found to be distracting. Additionally, we discuss the design implications for visualizing topics for DHH people in online meetings.

Current VR systems provide various text input methods that enable users to enter text efficiently with virtual keyboards. However, little attention has been paid to facilitate text revision during the VR text input process. We first summarized existing text revision solutions in current VR text input research and found that backspace is the only tool available for text revision with virtual keyboards with few mentioning designs for caret control. To systematically explore VR text revision designs, we presented a design space for VR text revision based on backspace and caret. With the proposed design space, we further analyzed the feasibility of the combined usage of backspace and caret by proposing and evaluating four VR text revision techniques. Outcomes of this research can provide a fundamental understanding of VR text revision solutions (with backspace and caret) and a comparable basis for evaluating future VR text revision techniques.

In the midst of the coronavirus disease 2019 pandemic, online meetings are rapidly increasing. Deaf or hard of hearing (DHH) people participating in an online meeting often face difficulties in capturing the affective states of other speakers. Recent studies have shown the effectiveness of emoji-based representation of spoken text to capture such affective states. Nevertheless, in voice-only online meetings, it is still not clear how emoji-based spoken texts can assist DHH people to understand the feelings of speakers without perceiving their facial expressions. We therefore conducted a preliminary experiment to understand the effect of emoji-based text representation during voice-only online meetings by leveraging an emoji-based captioning system. Our preliminary results demonstrate the necessity of designing an advanced system to help DHH people understanding the voice-only online meetings more meaningfully.

Text revision is an important task to ensure the accuracy of text content. Revising text on mobile devices is cumbersome and time-consuming due to the imprecise caret control and the repetitive use of the backspace. We present Swap, a novel replacement-based technique to facilitate text revision on mobile devices. We conducted two user studies to validate the feasibility and the effectiveness of Swap compared to traditional text revision techniques. Results showed that Swap reduced efforts in caret control and repetitive backspace pressing during the text revision process. Most participants preferred to use the replacement-based technique rather than backspace and caret. They also commented that the new technique is easy to learn, and it makes text revision rapid and intuitive.

We present the design and evaluation of Talk-and-Gaze (TaG), a method for selecting and correcting errors with voice and gaze. TaG uses eye gaze to overcome the inability of voice-only systems to provide spatial information. The user's point of gaze is used to select an erroneous word either by dwelling on the word for 800 ms (D-TaG) or by uttering a "select" voice command (V-TaG). A user study with 12 participants compared D-TaG, V-TaG, and a voice-only method for selecting and correcting words. Corrections were performed more than 20% faster with D-TaG compared to the V-TaG or voice-only methods. As well, D-TaG was observed to require 24% less selection effort than V-TaG and 11% less selection effort than voice-only error correction. D-TaG was well received in a subjective assessment with 66% of users choosing it as their preferred choice for error correction in voice-based text entry.

We present a computational model of visual search on graphical layouts. It assumes that the visual system is maximising expected utility when choosing where to fixate next. Three utility estimates are available for each visual search target: one by unguided perception only, and two, where perception is guided by long-term memory (location or visual feature). The system is adaptive, starting to rely more upon long-term memory when its estimates improve with experience. However, it needs to relapse back to perception-guided search if the layout changes. The model provides a tool for practitioners to evaluate how easy it is to find an item for a novice or an expert, and what happens if a layout is changed. The model suggests, for example, that (1) layouts that are visually homogeneous are harder to learn and more vulnerable to changes, (2) elements that are visually salient are easier to search and more robust to changes, and (3) moving a non-salient element far away from original location is particularly damaging. The model provided a good match with human data in a study with realistic graphical layouts.

Current gesture interaction paradigm mainly involves a one-to-one gesture-command mapping. This leads to memorability issues regarding (1) the mapping - as each new command requires a new gesture, and (2) the gestures specifics (e.g., motion paths) - that can be complex to leverage the recognition of several gestures. We explore the concept of combining 3D gestures when interacting in smart environments. We first propose a design space to characterize the temporal and spatial combination aspects, and the gesture types used by the combination. We then report results from three user studies in the context of smart TV interaction. The first study reveals that end-users can create gesture sets with combinations fully optimized to reuse gestures. The second study shows that combining gestures can lead to improved memorability compared to single gestures. The third study reveals that preferences for gestures combination appear when single gestures have an abstract gesture-command mapping.

Older adults have difficulty using and learning to use smart phones, in part because the displays are too small to provide effective interactive help. Our work explores the use of a large display to temporarily augment the small phone display to support older adults during learning episodes. We designed and implemented a learning system called Help Kiosk which contains unique features to scaffold the smart phone learning process for older adults. We conducted a mixed-methods user study with 16 older adults (55+) to understand the impact of this unique design approach, comparing it with the smart phone's official printed instruction manual. We found Help Kiosk gave participants more confidence that they were doing the tasks correctly, and helped minimize the need to switch their attention between the instructions and their phone.

Although the HCI community inevitably contributes to engagement via beauty according to the attention paid to known and yet to be discovered principles of aesthetics for digital interface design, it is lacking an epistemological corpus which should include the notion, human factors and the quantification of aesthetic aspects. The aim of the proposed workshop is to discuss these issues in order to strengthen aesthetic studies specifically for HCI and related fields. We want to create a forum for discussing, drafting and promoting the foundations for disciplined aesthetics design within the HCI community. We thus welcome contributions such as theories, methodologies, evaluation methods, and potential applications regarding effective aesthetics for HCI and related fields. Concretely, we aim to (i) map the present state-of-art of aesthetic research in HCI, (ii) build a multidisciplinary community of experts, and (iii) raise the profile of this aesthetics research area within HCI community.

Predicting how users learn new or changed interfaces is a long-standing objective in HCI research. This paper contributes to understanding of visual search and learning in text entry. With a goal of explaining variance in novices' typing performance that is attributable to visual search, a model was designed to predict how users learn to locate keys on a keyboard: initially relying on visual short-term memory but then transitioning to recall-based search. This allows predicting search times and visual search patterns for completely and partially new layouts. The model complements models of motor performance and learning in text entry by predicting change in visual search patterns over time. Practitioners can use it for estimating how long it takes to reach the desired level of performance with a given layout.

Mid-air gestures enable intuitive and natural interactions. However, few studies have investigated the use of mid-air gestures for blind people. TV interactions are one promising use of mid-air gestures for blind people, as "listening"' to TV is one of their most common activities. Thus, we investigated mid-air TV gestures for blind people through two studies. Study 1 used a user-elicitation approach where blind people were asked to define gestures given a set of commands. Then, we present a classification of gesture types and the frequency of body parts usage. Nevertheless, our participants had difficulty imagining gestures for some commands. Thus, we conducted Study 2 that used a choice-based elicitation approach where the participants selected their favorite gesture from a predefined list of choices. We found that providing choices help guide users to discover suitable gestures for unfamiliar commands. We discuss concrete design guidelines for mid-air TV gestures for blind people.

This paper presents an Electrovibration Pen (EV-Pen) which incorporates electrovibration technology into pen interactions. The EV-Pen has two unique characteristics: precise interaction and pen-on-paper feeling. We conducted four experiments for evaluating the EV-Pen. Experiment 1 was to determine preliminary characteristics of the EV-Pen. Experiments 2 and 3 were to evaluate precise interaction task performance through a steering task and a tracing task. We compared user performance between the EV-Pen, a mechanical-vibration pen, a normal pen without feedback and electrovibration-based finger interaction. Experimental results indicated that the EV-Pen outperformed the other devices in precise task interactions. Experiment 4 tested pen-on-paper feeling in drawing and handwriting tasks, and it was observed that the EV-Pen significantly enhances user experience. Based on the experimental results, we discuss implications and potential benefits for the design of the EV-Pen.

Of late there has been significant development in Information and Communication Technology (ICT), which offers interaction with computing systems in a large scale. Text input mechanisms in users’ own languages are necessary for bringing the ICT advantages to the English illiterates. QWERTY keyboard, however, which was designed for text entry in English, is not as suitable for text composition in other languages. As an alternative, researchers advocate virtual keyboards in users’ mother languages. This article proposes an approach to designing virtual keyboards suitable for text entry in Indian languages. Composition of texts in Indian languages using virtual keyboards needs special attention due to the presence of large character sets, complex characters, inflexions, and so on. First, we examine the suitability of existing design principles in developing virtual keyboards in Indian languages. Then we propose a virtual keyboard layout suitable for efficient text entry in Indian languages. We have tested our approach with the three most spoken languages in India, namely, Bengali, Hindi, and Telugu. Performance of the keyboards have been evaluated, and the evaluation substantiates that proposed design achieves on average higher text entry rather than with conventional virtual keyboards. The proposed approach, in fact, provides a solution to deal with complexity in Indian languages and can be extended to many other languages in the world.