Parsons problems have become a mainstay of computer science education. They are heavily used among students, especially in K-12 and provide a small puzzle-like experience for students to practice their skills. Today, while prior work has explored complex issues with accessibility and block languages in general, the 2024 changes to accessibility regulations by the U.S. Department of Justice includes new rules around mobile platforms. These rules are ill-defined and in need of evaluation. In this work, we make several contributions. First, we conducted an evaluation of existing blocks with respect to their regulatory compliance and discuss a new blocks technology that we developed that meets these new mobile guidelines. Second, we conducted three empirical studies using Parsons problems to evaluate the usability of the technology with teachers of the visually impaired (n = 32), high-school students with diverse disabilities (n = 28), and high-school students with blindness or low vision (n = 13).

When communicating over long distances via telerobotic platforms, the ability to perform semantic haptic interactions is often lacking, stifling feelings of presence for those involved. One such interaction is handshaking—a haptic gesture used in various social contexts. This paper presents a within-subjects user study ( \(N=18\) ), where participants felt 14 unimodal actuated profiles and 1 no haptic (control) profile. The actuation of these profiles was facilitated by the Handshaking Over Long Distances (HOLD) system which uses radial squeeze, skin-stretch via a rocker, and vibration to simulate a handshake in the context of a telerobotic exchange. Participants’ impressions of the profiles were assessed via questionnaires. The findings from this study suggest that the type of haptic feedback (lateral skin stretch, radial squeeze, or vibration) has a larger effect on user preference and presence compared to the differences in profiles within each feedback type. All three feedback types provided a more preferable handshaking experience for the pilot user and increased presence compared to the control. However, radial squeeze was the most effective. This work highlights the importance of haptics, especially radial squeeze, in the remote handshaking experience and exhibits wearable haptic devices’ potential to improve the effectiveness of remote communication tools.

Touchscreen-based rendering of graphics using vibrations, sonification, and text-to-speech is a promising approach for nonvisual access to graphical information, but extracting trends from complex data representations nonvisually is challenging. This work presents the design of a multimodal feedback scheme with integrated spatial audio for the exploration of histograms and scatter plots on touchscreens. We detail the hardware employed and the algorithms used to control vibrations and sonification adjustments through the change of pitch and directional stereo output. We conducted formative testing with 5 blind or visually impaired participants, and results illustrate that spatial audio has the potential to increase the identification of trends in the data, at the expense of a skewed mental representation of the graph. This design work and pilot study are critical to the iterative, human-centered approach of rendering multimodal graphics on touchscreens and contribute a new scheme for efficiently capturing data trends in complex data representations.

Mobile Remote Presence (MRP) robots have emerged out of the need for telepresence in various settings such as the workplace and hospitals. As with face-to-face experiences, these robot mediated encounters have social aspects that current commercially available MRP robots lack the capabilities to incorporate. In previous work, we integrated a manipulator onto a commercial telerobotic platform to enable expressive gestures and demonstrated that the gesturing capabilities enhanced the social connection between remote and local users. However, we also found that controlling the robot for complex interactions, such as a handshake, diminishes the remote user” s social experience. This paper presents the discovery of models for handshakes in different social contexts, which can be used in a shared-control architecture to reduce the effort on the remote user. Using a haptic measurement glove, force and inertia data was collected for human-human handshakes in various social contexts. By applying a k-nearest neighbor algorithm in combination with dynamic time warping and a support vector machine algorithm, two classification models are derived that predict the social context and can be used in an intelligent shared-control robot architecture.

Despite their technical advancements, commercially available telerobots are limited in social interaction capabilities for both pilot and local users, specifically in nonverbal communication. Our group hypothesizes that the introduction of expressive gesturing and tangible interaction capabilities (e.g., handshakes, fist bumps) will enhance telerobotic interactions and increase social connection between users. To investigate the affordances to social connection that gestures and tangible interactions provide in telerobot-mediated interactions, we designed and integrated a lightweight manipulator terminating in an anthropomorphic end effector onto a commercially available telerobot (Anybots QB 2.0). Through virtual reality tracking of the pilot user’s arm and hand, expressive gestures and social contact interactions are recreated via the manipulator, enabling a pilot user and a local user to engage in a tangible exchange. To assess the usability and effectiveness of the gesturing system, we present evaluations from both the local and pilot user perspectives. First, we present a validation study to assess usability of the control system by the pilot user. Our results demonstrate that pilot user interactions can be replicated with a greater than 80% pass rate and mean ease of use rating of \(7.08 \pm 1.32\) (out of 10) with brief training. Finally, we present a user study to assess the social impacts of (1) using the telerobot without the manipulator from both the pilot user and local user perspectives and (2) using the control system and telerobotic manipulator from both the pilot user and local user perspectives. Results demonstrate that the robot with the manipulator elicited a more positive social experience than the robot without the arm for local users but no significant difference in conditions for pilot users. Future work will focus on improving the pilot user experience to support social contact interactions.

Although research indicates that telepresence robots offer a more socially telepresent alternative to conventional forms of remote communication, the lack of touch-based interactions presents challenges for both remote and local users. In order to address these challenges, we have designed and implemented a robotic manipulator emulating a human arm. However, contact interactions like handshakes with a robotic manipulator may feel awkward and unnatural to local users. In this work, we present the design of a wearable haptic measurement glove (HMG) and use it to collect force and inertial data on handshakes in human-human and humanrobot interactions in the interest of developing intelligent shared control algorithms for natural, human-like contact interactions in human-robot interactions.

The network management community has explored and exploited light, copper, and several wireless spectra (including acoustics) as a media to transfer control or data traffic. Meanwhile, haptic technologies are being explored in end-user (wearable) devices, and Tactile Internet is being used merely as a metaphor. However, with rare exceptions and for smaller scoped projects, to our knowledge, vibration has been largely untouched as networking communication media.In this paper, we share the lessons learned while creating and optimizing a pilot testbed that serves as an inexpensive starting point for the exploration of vibration-defined networking. We demonstrate the feasibility of vibrations as a tool for resiliency, physical layer security, and an innovative method of teaching networking concepts to the Visually Impaired (VI) community.

Vibration plays a significant role in the way users interact with touchscreens. For many users, vibration affords tactile alerts and other enhancements. For eyes-free users and users with visual impairments, vibration can also serve a more primary role in the user interface, such as indicating streets on maps, conveying information about graphs, or even specifying basic graphics. However, vibration is rarely used in current user interfaces beyond basic cuing. Furthermore, designers and developers who do actually use vibration more extensively are often unable to determine the exact properties of the vibration signals they are implementing, due to out-of-the-box software and hardware limitations. We make two contributions in this work. First, we investigate the contextual properties of touchscreen vibrations and how vibrations can be used to effectively convey traditional, embossed elements, such as dashes and dots. To do so, we developed an open source, Android-based library to generate vibrations that are perceptually salient and intuitive, improving upon existing vibration libraries. Second, we conducted a user study with 26 blind or visually impaired users to evaluate and categorize the effects with respect to traditional tactile line profiles. We have established a range of vibration effects that can be reliably generated by our haptic library and are perceptible and distinguishable by users.

In this demonstration we illustrate how haptic communication can occur via a mobile application. In particular, we present an Android-based platform that allows connectivity among haptic devices and smartphones. Our app was designed specifically for educational applications, and we used the Hapkit as our first exemplary hardware platform. Our open-source implementation offers several haptic emulations designed for K-12 Physics courses; furthermore, our Android application provides instrumented metrics capable of analyzing latency and other network parameters among communicating hardware and software processes.

While text-to-speech software has largely made textual information accessible in the digital space, analogous access to graphics still remains an unsolved problem. Because of their portability and ubiquity, several studies have alluded to touchscreens as a potential platform for such access, yet there is still a gap in our understanding of multimodal information transfer in the context of graphics. The current research demonstrates feasibility for following lines, a fundamental graphical concept, via vibrations and sounds on commercial touchscreens. Two studies were run with 21 blind and visually impaired participants ( N = 12; N = 9). The first study examined the presentation of straight, linear lines using a multitude of line representations, such as vibration-only, auditory-only, vibration lines with auditory borders, and auditory lines with vibration borders. The results of this study demonstrated that both auditory and vibratory bordered lines were optimal for precise tracing, although both vibration- and auditory-only lines were also sufficient for following, with minimal deviations. The second study examined the presentation of curving, non-linear lines. Conditions differed on the number of auditory reference points presented at the inflection and deflection points. Participants showed minimal deviation from the lines during tracing, performing nearly equally in both 1- and 3-point conditions. From these studies, we demonstrate that line following via multimodal feedback is possible on touchscreens, and we present guidelines for the presentation of such non-visual graphical concepts.

Recent research on telepresence robots demonstrates that while they enable new heights of remote communication, there still exists challenges for both local and remote users in creating a connectedness one only encounters in face-to-face interactions. A large part of communication is beyond hearing and vision. Tangible interactions, expressive gestures, and physical referencing represent three of the primary social behaviors missing in the current telepresence experience. There is an inherent, subconscious quality to these physical actions that has been shown to allow more expressive and engaging communication. In this project we present the design, fabrication, and initial performance validation of a lightweight, ergonomic manipulator with a heavy, anthropomorphic end effector that enables gesturing capabilities for telepresence interactions.