An animated agent’s believability, often defined as “the illusion of life,” has many facets, including the agent’s appearance, movements, and user interactions. Considering that human facial expressions are often not symmetrical, evaluating how facial expression asymmetries affect believability in animated agents would provide valuable insights for researchers and artists. Our research focused on facial movements and examined the impact of three dynamic facial asymmetries on users’ perception of the agent’s believability, appeal, and naturalness. Our research involved three studies: 1) a preliminary study in which we determined the maximum values to base asymmetry levels, 2) a study in which we identified what level of asymmetry yields the most perceived believability for each asymmetry location, and 3) a study in which we determined what effects different combinations of asymmetries have on perceived believability, appeal, and naturalness. The findings confirmed the effect of dynamic facial asymmetry on the agents’ perceived believability, appeal, and naturalness. Furthermore, our research suggests that including one or a combination of asymmetric eyebrow, mouth, or eyelid motions increases the agents’ perceived believability, appeal, and naturalness.

We examined the effects of virtual characters’ body type and voice pitch on perceived audio-visual correspondence and believability. For our within-group study (N = 72), we developed nine experimental conditions using a 3 (body type: ectomorph vs. mesomorph vs. endomorph body types) × 3 (voice pitch: low vs. medium vs. high fundamental frequency [F0]) design. We found statistically significant main effects from voice pitch and statistically significant interaction effects between a virtual character’s body type and voice pitch on both the level of perceived audio-visual correspondence and believability of female and male virtual characters. For female virtual characters, we also observed an additional statistically significant main effect from body type and a statistically significant interaction effect between the participant’s biological sex and the virtual character’s voice pitch on both perceived audio-visual correspondence and believability. Moreover, the results show that perceived believability is highly correlated to perceived audio-visual correspondence. Our findings have important practical implications in applications where the virtual character is meant to be an emotional or informational guide that requires some level of perceived believability, as the findings suggest that it is possible to enhance the perceived believability of the virtual characters by generating appropriate voices through pitch manipulation of existing voices.

Abstract We explored user interactions with a holographic sign language interpreter in a mixed reality (MR) classroom for deaf and hard of hearing students. The developed MR application projects a holographic signing avatar that translates in real time the lecture while a speaking instructor is teaching. Our study explored user interaction with the MR system, intending to provide design guidelines for digital MR sign language interpreters. We recruited eight participants and conducted a usability test focused on avatar framing (full‐body vs. half‐body) and avatar manipulation (fixed position, scale, and orientation vs. user‐adjustable position, scale, and orientation) in the MR classroom. We used a mixed‐method approach to analyze quantitative and qualitative data through recordings, surveys, and interviews. The results show user preferences toward viewing holographic signing avatars in the MR environment and user acceptability toward such applications.

Abstract Previous research has shown that student‐instructor interaction is vital to motivating students to learn a second language. However, it is unclear whether learners' demographics affect in‐game immersion and interactions with virtual instructors. This study's purpose is to investigate whether the number of years learning Japanese (foreign language familiarity) influences students' immersion levels in serious games and their interactions with virtual instructors. We developed a 3D animated Japanese roleplaying game with a virtual in‐game instructor. Eighty‐four college students enrolled in 200‐ and 300‐level Japanese language courses voluntarily participated in the study. Participants played the game and then answered a questionnaire concerning virtual character appearance, attentiveness to the instructor, and immersion in the game. The findings indicated that gender and the number of years studying Japanese significantly impact multiple measurements.

The paper presents the development and initial validation of the Senescence character framework, a simulation tool that can be used for rigging muscle deformer-based humanoid characters, with support for age. The senescence framework allows the user to rig any bipedal 3D character and manipulate skeleton and muscle parameters in order to render the age of the character realistically. The framework was developed using Python, Maya Embedded Language and PyQt. A study with 100 subjects was conducted to determine whether participants were able to perceive the age of the characters simulated with the Senescence framework. Findings show that while subjects were able to perceive age differences, they were not able to identify the age of the simulated characters with high level of accuracy. Results also show that subjects with animation expertise were able to identify the age of the characters more accurately than subjects without animation experience.

In this paper we describe the design, development, formative evaluation and initial findings of one level of a serious game whose objective is to teach Information Assurance concepts to undergraduate students in introductory programming courses. The game level focuses on the concept of 'operator precedence'. The player travels through a multilevel 3- dimensional maze and at each junction in the maze he/she is required to solve a mathematical problem that involves the application of operator precedence rules. A correct answer allows the player to move closer to the maze exit, an incorrect solution moves the player farther from the end of the maze. Initial findings from a formative study with a group of 14 undergraduate students show that the game level is usable, engaging and useful for learning/reviewing the intended programming concept.

This paper presents ongoing research aimed at improving surveying education through the use of virtual environments. Teaching surveying presents several challenges such as low student competence in mathematics, geometry and trigonometry, limited student access to instruments, limited availability of terrains on which to practice, dependency on weather conditions, and more. Prior research suggests that surveying education can be significantly improved with the use of digital teaching aids and virtual instruments. The goal of the work reported in the paper is to enhance traditional surveying instruction methods with a unique approach: an interactive Virtual Learning Environment (VLE) that supports students' mathematical representational fluency and includes realistic virtual terrains and surveying instruments that look, operate, and produce results comparable to the physical ones. The VLE is aimed at undergraduate students enrolled in Civil Engineering, Architecture and Building Construction Management programs and includes 5 educational modules, in this paper we describe the design, development and initial evaluation of the "Triangulations and Coordinates Calculations" module.

A conventional pinhole camera captures only a small fraction of a 3-D scene due to occlusions. We introduce the graph camera, a non-pinhole with rays that circumvent occluders to create a single layer image that shows simultaneously several regions of interest in a 3-D scene. The graph camera image exhibits good continuity and little redundancy. The graph camera model is literally a graph of tens of planar pinhole cameras. A fast projection operation allows rendering in feed-forward fashion, at interactive rates, which provides support for dynamic scenes. The graph camera is an infrastructure level tool with many applications. We explore the graph camera benefits in the contexts of virtual 3-D scene exploration and summarization, and in the context of real-world 3-D scene visualization. The graph camera allows integrating multiple video feeds seamlessly, which enables monitoring complex real-world spaces with a single image.

We present SMILE™(Science and Math in an Immersive Learning Environment) an immersive game in which deaf and hearing children ages 5-10 learn math and science concepts and ASL (American Sign Language) terminology through interaction with animated 3D characters and objects. The application can be displayed in stationary, four-wall immersive devices (i.e., the FLEX), Fish Tank VR systems, and non-immersive standard computer systems. Users interact with the program choosing from a variety of input devices including a 6DOF wand, a pinch glove, a data glove, a dance platform, or standard mouse and keyboard.

We present a prototype system called ARpm, (augmented reality for polygonal modeling). Created as a front-end to a commercially available 3D animation program (3D Studio Max), it does not require it's modification through inaccessible software code. Users look through a head mounted display (HMD) and the object being modeled appears as a 3D augmentation in the real world. This object can be manipulated using a tangible interface of marker panels that correspond to the modeling tools in 3D Studio Max, and a wireless pointer device. Unlike virtual reality which brings the user into the virtual world, this system places the 3D model in the user's world while allowing the use of a large range of complex 3D polygonal modeling tools provided by 3D Studio Max.

The object of the paper is the development of a new human computer communication (HCC) interface for configuration and animation of a signing semantroid. The work presented in this paper builds on two recent research results: (1) the concepts of semantroid (=semantic android - an avatar limited to head and hands), and semantic intensity (=semantic content); and (2) the KUI (keyboard user interface) method, a keyboard-based method of configuration and animation of 3D hand gestures and facial expressions. In particular, the paper focuses on: (1) development of a new set of keyboard-controlled facial parameters of the semantroid for animation of American Sign Language (ASL) non-manuals, and (2) design of an efficient HCC interface for input and encoding recording of ASL handshapes and facial expressions.

The paper focuses on the development of a human-computer communication method which utilizes the user's typing skills to control the facial expression of a computer-generated 3D face. The method is based on the realization that the human face is a movable and deformable system with 26 degrees of freedom (the same number as the letters of the English alphabet). Therefore it is possible to create a parameterized graphical facial model confined to a set of 26 parameters each one controlled by a letter key. The method is an extension of the KUI technique (Adamo-Villani and Beni, 2004; Adamo-Villani and Beni, 2003) recently developed to encode hand gestures.

We present a new highly interactive program to teach American sign language (ASL) for K-3 mathematics by 3D animation. We aim at increasing: (1) the opportunity for deaf children to learn arithmetic via interactive media; (2) the effectiveness of (hearing) parents in teaching arithmetic to their deaf children; (3) the effectiveness of ASL K-8 mathematics teachers. Using state-of-the-art 3D computer animation technology we have created a learning tool to be used for K-8 classroom and home learning, and for closed captioning in interactive media and Internet.