In this lightning talk, I share our strategy for integrating AI into a CS1-level course, taking a conservative approach with a few modifications to the course. While academic honesty has been a major concern for instructors, we did not take any antagonistic approaches by detecting or circumventing AI use. Instead, we aimed to promote student ownership of their learning, balancing AI literacy with growth. Instead of redesigning the course to feature large portions of AI-assisted coding and embracing AI (a hug), we instead introduced students to AI as a partner (a handshake) and helped them use it as a tool and develop AI literacy.

Augmented Reality (AR) interactions feature users interacting with virtual objects registered in the physical world. With contemporary AR experiences increasingly featuring interactions at distances, we conceptualized The Force, a technique that allows users to clone distant objects and manipulate their replicas. An empirical evaluation was conducted, comparing it against two well-established techniques including controller-based ray-casting and a gaze-based pinching technique in a pick-and-place task. We employed a within-subjects design, collecting data on both objective performance and subjective user experience. Results suggest that The Force allows for higher levels of accuracy and efficiency in medium-field tasks that require precision and fine motor control. Furthermore, we discovered avenues towards iteratively refining this technique. We go on to discuss the implications of our findings in an effort to facilitate better interactions in augmented reality.

To design intuitive and effective context-aware task guidance systems, we must understand users’ thought processes and the obstacles they experience when they perform tasks. Though task guidance systems have proven beneficial in many domains for improving task performance and reducing user frustration, there is a lack of general guidelines and design principles for their development. Prior work has shown that recipe-based cooking is a strong medium for studying task planning and execution. In response, we conducted a contextual inquiry study in home kitchens, observing eight different participants’ cooking sessions. We used affinity diagramming of our notes and transcripts to identify common obstacles faced by participants and establish user needs in the areas of object interaction, safety, knowledge base, and task coordination. We discuss how these findings can inform the design of technology-driven solutions for task guidance systems beyond cooking.

Motivational agents are virtual agents that seek to motivate users by providing feedback and guidance. Prior work has shown how certain factors of an agent, such as the type of feedback given or the agent's appearance, can influence user motivation when completing tasks. However, it is not known how nonverbal mirroring affects an agent's ability to motivate users. Specifically, would an agent that mirrors be more motivating than an agent that does not? Would an agent trained on real human behaviors be better? We conducted a within-subjects study asking 30 participants to play a "find-the-hidden-object" game while interacting with a motivational agent that would provide hints and feedback on the user's performance. We created three agents: a Control agent that did not respond to the user's movements, a simple Mimic agent that mirrored the user's movements on a delay, and a Complex agent that used a machine-learned behavior model. We asked participants to complete a questionnaire asking them to rate their levels of motivation and perceptions of the agent and its feedback. Our results showed that the Mimic agent was more motivating than the Control agent and more helpful than the Complex agent. We also found that when participants became aware of the mimicking behavior, it can feel weird or creepy; therefore, it is important to consider the detection of mimicry when designing virtual agents.

An increasing amount of data are being stored and processed by data centers every day for various commercial and industrial applications. For such large-scale systems dealing with enormous volumes of data, utilization of Dynamic Random-Access Memories (DRAM), integrated into horizontal or vertical stacks, has been proven to be the ideal solution in terms of area vs performance. At-speed test and repair in these stacks of DRAM memories requires complex solutions given the various lifecycle stages the memories go through. In this paper an efficient Built-In Self-Test (BIST) concept is proposed, focusing on both the in-system and production aspects of the test solution. It is demonstrated for a real-life High-Performance Computing (HPC) application.

Virtual agents are systems that add a social dimension to computing, often featuring not only natural language input but also an embodiment or avatar. This allows them to take on a more social role and leverage the use of nonverbal communication (NVC). In humans, NVC is used for many purposes, including communicating intent, directing attention, and conveying emotion. As a result, researchers have developed agents that emulate these behaviors. However, challenges pervade the design and development of NVC in agents. Some articles reveal inconsistencies in the benefits of agent NVC; others show signs of difficulties in the process of analyzing and implementing behaviors. Thus, it is unclear what the specific outcomes and effects of incorporating NVC in agents and what outstanding challenges underlie development. This survey seeks to review the uses, outcomes, and development of NVC in virtual agents to identify challenges and themes to improve and motivate the design of future virtual agents.

Mode switching allows applications to support a wide range of operations (e.g. selection, manipulation, and navigation) using a limited input space. While the performance of different mode switching techniques has been extensively examined for pen- and touch-based interfaces, investigating mode switching in augmented reality (AR) is still relatively new. Prior work found that using non-preferred hand is an efficient mode switching technique in AR. However, it is unclear how the technique performs when increasing the number of modes, which is more indicative of real-world applications. Therefore, we examined the scalability of non-preferred hand mode switching in AR with two, four, six, and eight modes. We found that as the number of modes increase, performance plateaus after the four-mode condition. We also found that counting gestures have varying effects on mode switching performance in AR. Our findings suggest that modeling mode switching performance in AR is more complex than simply counting the number of available modes. Our work lays a foundation for understanding the costs associated with scaling interaction techniques in AR.

Fitts' law has accurately modeled both children's and adults' pointing movements, but it is not as precise for modeling movement to small targets. To address this issue, prior work presented FFitts' law, which is more exact than Fitts' law for modeling adults' finger input on touchscreens. Since children's touch interactions are more variable than adults, it is unclear if FFitts' law should be applied to children. We conducted a 2D target acquisition task with 54 children (ages 5-10) to examine if FFitts' law can accurately model children's touchscreen movement time. We found that Fitts' law using nominal target widths is more accurate, with a R2 value of 0.93, than FFitts' law for modeling children's finger input on touchscreens. Our work contributes new understanding of how to accurately predict children's finger touch performance on touchscreens.

Augmented Reality (AR) headsets are being employed in industrial settings (e.g., the oil industry); however, there has been little work on how information should be presented in these headsets, especially in the context of situational awareness. We present a study examining three different presentation styles (Display, Environment, Mixed Environment) for textual secondary information in AR headsets. We found that the Display and Environment presentation styles assisted in perception and comprehension. Our work contributes a first step to understanding how to design visual information in AR headsets to support situational awareness.

The future of smart environments is likely to involve both passive and active interactions on the part of users. Depending on what sensors are available in the space, users may make use of multimodal interaction modalities such as hand gestures or voice commands. There is a shortage of robust yet controlled multimodal interaction datasets for smart environment applications. One application domain of interest based on current state-of-the-art is authentication for sensitive or private tasks, such as banking and email. We present a novel, large multimodal dataset for authentication interactions in both gesture and voice, collected from 106 volunteers who each performed 10 examples of each of a set of hand gesture and spoken voice commands chosen from prior literature (10,600 gesture samples and 13,780 voice samples). We present the data collection method, raw data and common features extracted, and a case study illustrating how this dataset could be useful to researchers. Our goal is to provide a benchmark dataset for testing future multimodal authentication solutions, enabling comparison across approaches.

While the effects of virtual agents in terms of likeability, uncanniness, etc. are well explored, it is unclear how their appearance and the feedback they give affects people's reactions. Is critical feedback from an agent embodied as a mouse or a robot taken less serious than from a human agent? In an intercultural study with 120 participants from Germany and the US, participants had to find hidden objects in a game and received feedback on their performance by virtual agents with different appearances. As some levels were designed to be unsolvable, critical feedback was unavoidable. We hypothesized that feedback would be taken more serious, the more human the agent looked. Also, we expected the subjects from the US to react more sensitively to criticism. Surprisingly, our results showed that the agents' appearance did not significantly change the participants' perception. Also, while we found highly significant differences in inspirational and motivational effects as well as in perceived task load between the two cultures, the reactions to criticism were contrary to expectations based on established cultural models. This work improves our understanding on how affective virtual agents are to be designed, both with respect to culture and to dialogue strategies.

Prior work has shown that embodiment can benefit virtual agents, such as increasing rapport and conveying non-verbal information. However, it is unclear if users prefer an embodied to a speech-only agent for augmented reality (AR) headsets that are designed to assist users in completing real-world tasks. We conducted a study to examine users' perceptions and behaviors when interacting with virtual agents in AR. We asked 24 adults to wear the Microsoft HoloLens and find objects in a hidden object game while interacting with an agent that would offer assistance. We presented participants with four different agents: voice-only, non-human, full-size embodied, and a miniature embodied agent. Overall, users preferred the miniature embodied agent due to the novelty of his size and reduced uncanniness as opposed to the larger agent. From our results, we draw conclusions about how agent representation matters and derive guidelines on designing agents for AR headsets.

Video annotation is a vital part of research examining gestural and multimodal interaction as well as computer vision, machine learning, and interface design. However, annotation is a difficult, time-consuming task that requires high cognitive effort. Existing tools for labeling and annotation still require users to manually label most of the data, limiting the tools helpfulness. In this paper, we present the Easy Automatic Segmentation Event Labeler (EASEL), a tool supporting gesture analysis. EASEL streamlines the annotation process by introducing assisted annotation, using automatic gesture segmentation and recognition to automatically annotate gestures. To evaluate the efficacy of assisted annotation, we conducted a user study with 24 participants and found that assisted annotation decreased the time needed to annotate videos with no difference in accuracy compared with manual annotation. The results of our study demonstrate the benefit of adding computational intelligence to video and audio annotation tasks.

People communicate through words and gestures,but current voice-based computer interfaces such as Siri exploitonly words. This is a shame: human-computer interfaces wouldbe natural if they incorporated gestures as well as words. To support this goal, we present a new dataset of naturally occurring gestures made by people working collaboratively on blocks world tasks. The dataset, called EGGNOG, contains over 8 hours ofRGB video, depth video, and Kinect v2 body position data of 40subjects. The data has been semi-automatically segmented into 24,503 movements, each of which has been labeled according to (1) its physical motion and (2) the intent of the participant.We believe this dataset will stimulate research into natural and gestural human-computer interfaces.