The past several decades have seen significant advances in monitoring older adults' health and well-being. However, creating viable, practical monitoring systems for informing caregivers requires understanding which behaviors and signs to track and what approaches best present that information. To investigate how technology can be leveraged to better augment caregivers' workflows, we take a multi-stage, qualitative approach to gain insights into the needs of caregivers and the older adults receiving care. Specifically, we use a series of domain expert interviews, cognitive walkthroughs, and semi-structured interviews with residents, and we synthesize our takeaways using thematic analysis at each phase. Our results show that this type of monitoring technology has great potential to reduce the effort needed by caregivers to complete their responsibilities and communicate with their teams. Additionally, we found that older adults are receptive to the technology but their privacy and autonomy must be prioritized for the sake of their mental wellbeing. These insights will facilitate greater intelligent interface development for Person-Centered Care by identifying important design considerations and vital features that require system support.

Increased virtual meeting software usage has allowed people to meet remotely in a more seamless fashion. However, compared to in-person meetings, valuable interaction cues such as impromptu group polling are less optimally executed due to increased difficulty in gauging remote participants, while also requiring prior meeting setup for automated counting with built-in polling tools. We propose a novel intelligent user interface approach for virtual meeting software that supports impromptu polling interactions by leveraging real-time hand gesture recognition and video filter feedback. We conducted studies to design and evaluate this intuitive gesture-based polling system with visual feedback. Our results demonstrated that our system was able to recognize attendees’ gestures and poll responses with reasonable accuracy, and showed improvements in hosts’ task workload performance. From our findings, our interface informs hosts of valuable results while maintaining organic gestural interaction cues with attendees similar to in-person meetings.

introduction Share on Special Issue on Highlights of IUI 2021: Introduction Authors: Tracy Hammond Texas A&M University, College Station, Texas, United States Texas A&M University, College Station, Texas, United StatesView Profile , Bart Knijnenburg Clemson University, Clemson, South Carolina, United States Clemson University, Clemson, South Carolina, United StatesView Profile , John O’Donovan University of California, Santa Barbara, California, United States University of California, Santa Barbara, California, United StatesView Profile , Paul Taele Texas A&M University, College Station, Texas, United States Texas A&M University, College Station, Texas, United StatesView Profile Authors Info & Claims ACM Transactions on Interactive Intelligent SystemsVolume 12Issue 403 February 2023Article No.: 25pp 1–4https://doi.org/10.1145/3561516Published:03 February 2023Publication History 0citation47DownloadsMetricsTotal Citations0Total Downloads47Last 12 Months47Last 6 weeks9 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

With the onset of the COVID-19 pandemic, faculty are suddenly thrown into a world where they have to teach virtually, forcing them to innovate in their teaching practice without the ability to “chat with their colleagues” next door. Additionally, many faculty do not know how to create an inclusive classroom, which is crucial for students who have also lost their learning community and support structure. Many faculty are afraid to include issues of inclusion and diversity, feeling ill equipped and unsupported. As tensions rise across the US, there is a critical need for engineering students to be able to discuss issues involving inclusion in the classroom as well as apply their abilities as engineers to make a significant impact (both positively or negatively) on inclusion through their applications and innovations. Without the proper support for the faculty to innovate and foster inclusive classrooms, millions of students will suffer in their education. Additionally, without a trusting support group, most engineering faculty would not choose to assume the risk. In response, we developed a community of practice with six teaching fellows who shared, watched, commented on, and emulated each other's classroom recordings. The teaching fellows learned a significant amount from each other, courageously implemented activities focused on inclusion and awareness in their classrooms, observed greater awareness among their students on inclusion issues, and were able to mentor faculty and publish about the data driven best practices that they completed.

Kanji script writing is a skill that is often introduced to novice Japanese foreign language students for achieving Japanese writing mastery, but often poses difficulties to students with primarily English fluency due to their its vast differences with written English. Instructors often introduce various pedagogical methods—such as visual structure and written techniques—to assist students in kanji study, but may lack availability providing direct feedback on students' writing outside of class. Current educational applications are also limited due to lacking richer instructor-emulated feedback. We introduce Kanji Workbook, a writing-based intelligent tutoring system for students to receive intelligent assessment that emulates human instructor feedback. Our interface not only leverages students' computing devices for allowing them to learn, practice, and review the writing of prompted characters from their course's kanji script lessons, but also provides a diverse set of writing assessment metrics—derived from instructor interviews and classroom observation insights—through intelligent scoring and visual animations. We deployed our interface onto novice- and intermediate-level university courses over an entire academic year, and observed that interface users on average achieved higher course grades than their peers and also reacted positively to our interface's various features.

Virtual Reality (VR) sickness is common with symptoms such as headaches, nausea, and disorientation, and is a major barrier to using VR. We propose WalkingVibe, which applies unobtrusive vibrotactile feedback for VR walking experiences, and also reduces VR sickness and discomfort while improving realism. Feedback is delivered through two small vibration motors behind the ears at a frequency that strikes a balance in inducing vestibular response while minimizing annoyance. We conducted a 240-person study to explore how visual, audio, and various tactile feedback designs affect the locomotion experience of users walking passively in VR while seated statically in reality. Results showed timing and location for tactile feedback have significant effects on VR sickness and realism. With WalkingVibe, 2-sided step-synchronized design significantly reduces VR sickness and discomfort while significantly improving realism. Furthermore, its unobtrusiveness and ease of integration make WalkingVibe a practical approach for improving VR experiences with new and existing VR headsets.

In this workshop, the focus will be on providing a forum for researchers in the various interdisciplinary fields to engage in and explore research outcomes and discussions on sketching interactions, specifically for interfaces that promote creativity and cognition. Attendees of the workshop will actively experience sessions related to the presentation, discussion, and sharing of discoveries and innovations that target or leverage sketching interactions for interfaces in domains that include the learning sciences, education, artificial, and engineering. The workshop will serve as a platform to encourage opportunities of new collaborations and inspire growth for leveraging sketching interactions towards creativity and cognition.

Modern touchscreen devices have recently introduced customizable touchscreen settings to improve accessibility for users with motor impairments. For example, iOS 10 introduced the following four Touch Accommodation settings: 1) Hold Duration, 2) Ignore Repeat, 3) Tap Assistance, and 4) Tap Assistance Gesture Delay. These four independent settings lead to a total of more than 1 million possible configurations, making it impractical to manually determine the optimal settings. We present PersonalTouch, which collects and analyzes touchscreen gestures performed by individual users, and recommends personalized, optimal touchscreen accessibility settings. Results from our user study show that PersonalTouch significantly improves touch input success rate for users with motor impairments (20.2%, N=12, p=.00054) and for users without motor impairments (1.28%, N=12, p=.032).

Recent trends in computer-mediated communication (CMC) have not only led to expanded instant messaging through the use of images and videos but have also expanded traditional text messaging with richer content in the form of visual communication markers (VCMs) such as emoticons, emojis, and stickers. VCMs could prevent a potential loss of subtle emotional conversation in CMC, which is delivered by nonverbal cues that convey affective and emotional information. However, as the number of VCMs grows in the selection set, the problem of VCM entry needs to be addressed. Furthermore, conventional means of accessing VCMs continue to rely on input entry methods that are not directly and intimately tied to expressive nonverbal cues. In this work, we aim to address this issue by facilitating the use of an alternative form of VCM entry: hand gestures. To that end, we propose a user-defined hand gesture set that is highly representative of a number of VCMs and a two-stage hand gesture recognition system (trajectory-based, shape-based) that can identify these user-defined hand gestures with an accuracy of 82%. By developing such a system, we aim to allow people using low-bandwidth forms of CMCs to still enjoy their convenient and discreet properties while also allowing them to experience more of the intimacy and expressiveness of higher-bandwidth online communication.

Deaf and hard-of-hearing (DHH) individuals encounter difficulties when engaged in group conversations with hearing individuals, due to factors such as simultaneous utterances from multiple speakers and speakers whom may be potentially out of view. We interviewed and co-designed with eight DHH participants to address the following challenges: 1) associating utterances with speakers, 2) ordering utterances from different speakers, 3) displaying optimal content length, and 4) visualizing utterances from out-of-view speakers. We evaluated multiple designs for each of the four challenges through a user study with twelve DHH participants. Our study results showed that participants significantly preferred speechbubble visualizations over traditional captions. These design preferences guided our development of SpeechBubbles, a real-time speech recognition interface prototype on an augmented reality head-mounted display. From our evaluations, we further demonstrated that DHH participants preferred our prototype over traditional captions for group conversations.

Proper ergonomics improves productivity and reduces risks for injuries such as tendinosis, tension neck syndrome, and back injuries. Despite having ergonomics standards and guidelines for computer usage since the 1980s, injuries due to poor ergonomics remain widespread. We present ActiveErgo, the first active approach to improving ergonomics by combining sensing and actuation of motorized furniture. It provides automatic and personalized ergonomics of computer workspaces in accordance to the recommended ergonomics guidelines. Our prototype system uses a Microsoft Kinect sensor for skeletal sensing and monitoring to determine the ideal furniture positions for each user, then uses a combination of automatic adjustment and real-time feedback to adjust the computer monitor, desk, and chair positions. Results from our 12-person user study demonstrated that ActiveErgo significantly improves ergonomics compared to manual configuration in both speed and accuracy, and helps significantly more users to fully meet ergonomics guidelines.

Drones offer camera angles that are not possible with traditional cameras and are becoming increasingly popular for videography. However, flying a drone and controlling its camera simultaneously requires manipulating 5-6 degrees of freedom (DOF) that needs significant training. We present ARPilot, a direct-manipulation interface that lets users plan an aerial video by physically moving their mobile devices around a miniature 3D model of the scene, shown via Augmented Reality (AR). The mobile devices act as the viewfinder, making them intuitive to explore and frame the shots. We leveraged AR technology to explore three 6DOF video-shooting interfaces on mobile devices: AR keyframe, AR continuous, and AR hybrid, and compared against a traditional touch interface in a user study. The results show that AR hybrid is the most preferred by the participants and expends the least effort among all the techniques, while the users' feedback suggests that AR continuous empowers more creative shots. We discuss several distinct usage patterns and report insights for further design.

Learning music theory not only has practical benefits for musicians to write, perform, understand, and express music better, but also for both non-musicians to improve critical thinking, math analytical skills, and music appreciation. However, current external tools applicable for learning music theory through writing when human instruction is unavailable are either limited in feedback, lacking a written modality, or assuming already strong familiarity of music theory concepts. In this paper, we describe Maestoso, an educational tool for novice learners to learn music theory through sketching practice of quizzed music structures. Maestoso first automatically recognizes students’ sketched input of quizzed concepts, then relies on existing sketch and gesture recognition techniques to automatically recognize the input, and finally generates instructor-emulated feedback. From our evaluations, we demonstrate that Maestoso performs reasonably well on recognizing music structure elements and that novice students can comfortably grasp introductory music theory in a single session.

A national study by the Australian Transport Safety Bureau revealed that motorcyclist deaths were nearly thirty times more prevalent than that of drivers of other vehicles. These fatalities represent approximately 5% of all highway deaths each year, yet motorcycles account for only 2% of all registered vehicles in the USA. Motorcyclists are highly exposed on the road, so maintaining situational awareness at all times is crucial. Route guidance systems enable users to efficiently navigate between locations using dynamic visual maps and audio directions, and have been well tested with motorists, but remain unsafe for use by motorcyclists. Audio/visual routing systems decrease motorcyclists' situational awareness and vehicle control, and thus elevate chances of an accident. To enable motorcyclists to take advantage of route guidance while maintaining situational awareness, we created HaptiMoto, a wearable haptic route guidance system. HaptiMoto uses tactile signals to encode the distance and direction of approaching turns, thus avoiding interference with audio/visual awareness. Our evaluations demonstrate that HaptiMoto is both intuitive and a safer alternative for motorcyclists compared to existing solutions.

Military course-of-action (COA) diagrams are used to depict battle scenarios and include thousands of unique symbols, complete with additional textual and designator modifiers. We have created a real-time sketch recognition interface that recognizes 485 freely-drawn military course-of-action sym- bols. When the variations (not allowable by other systems) are factored in, our system is several orders of magnitude larger than the next biggest system. On 5,900 hand-drawn symbols, the system achieves an accuracy of 90% when con- sidering the top 3 interpretations and requiring every aspect of the shape (variations, text, symbol, location, orientation) to be correct.