Locomotion is central to human navigation and spatial learning, with natural walking offering the most effective movement in immersive virtual environments (IVEs) due to its rich proprioceptive and vestibular feedback. However, space constraints often limit natural walking in IVEs, prompting the use of omnidirectional treadmills. The Cyberith Virtualizer Elite 2 enables users to walk in place by sliding their feet on a low-friction, tiltable platform while secured in a harness. It allows control over both tilt angle and walking speed, offering researchers flexibility in adjusting sensorimotor cues. In this study, we examined how tilt angle, speed, and training duration affect distance perception and cybersickness. In Experiment 1, we tested three tilt angles (0°, ~8.5°, ~17°) and three speeds (0.8×, 1.0×, 1.2× treadmill speed) during blind walking. A slower speed (0.8×) aligned best with natural walking and minimized cybersickness; users preferred the ~8.5° tilt. Additional analysis identified ~0.7× speed with ~8.5° tilt as preferred. In Experiment 2, we varied training durations (3, 6, or 9 minutes) and found that, with the preferred setup, treadmill distance judgments were comparable to natural walking. At least 6 minutes of training was sufficient for adaptation. These findings offer practical guidance for configuring treadmill locomotion in virtual reality research.

The use of virtual reality (VR) has become widespread, and ensuring accessibility to VR is critical for the technology to be inclusive. However, most VR technology relies heavily on visual input for navigation, interaction, and feedback, which makes the VR less accessible for individuals with low vision, a prevalent global health condition. In this study, we developed a virtual, gaze-contingent simulation of central vision loss (CVL) and peripheral vision loss (PVL) in a head-mounted display. We examined how these two types of restricted visual input affected people’s perception of an affordance that is highly related to independent mobility, specifically the perceived ability to step over a virtual gap. In two experiments, participants judged whether they could step over virtual gaps under no vision loss and CVL (Exp 1) or PVL (Exp 2) conditions. Across both experiments, the vision loss conditions showed increases in decision-making time and reduced confidence in judgments compared to no vision loss. Results showed no significant difference in affordance judgments for stepping over for the CVL and no vision loss conditions, suggesting that residual peripheral vision may compensate for central deficits. But participants in the PVL condition underestimated their ability to step over gaps relative to no vision loss. This effect may have been due to an inability to integrate multiple views of the gap in order to make judgments. These findings demonstrate that different types of vision loss differentially impact perception-action coupling in virtual reality and underscore the importance of VR systems that consider sensory loss. Our approach offers a framework for understanding visual accessibility in immersive environments and highlights the potential of low vision simulation for future perception-action research.

Designing effective navigation systems in mixed reality (MR) is a critical challenge that requires a nuanced understanding of how virtual cues interact with user characteristics and the physical environment. To address this, we investigated the impact of two fundamental design choices: the frame of reference for displaying cues (world-fixed vs. screen-fixed) and the use of explicit landmark annotations. An urban environment with local and global landmarks was designed to understand how the visibility of environmental features impacts users’ visual attention and interferes with the formation of spatial knowledge. We assessed the navigation behavior, gaze patterns, and spatial knowledge acquisition of 81 participants (40 female, 41 male). We also analyzed the role of individual differences, including spatial ability and gender. Results show that world-fixed cues aligned better with natural gaze patterns and reduced unnecessary saccades, landmark cues improved learning, particularly for participants with lower spatial ability, and the frequency of utilizing global landmarks during navigation enhances spatial memory. We further observed gender-related differences in gaze–head coordination strategies. These findings advance understanding of how MR navigation cues influence real-time behavior and outcomes in spatial tasks. They provide design recommendations for adaptive MR systems that match cue placement to environmental conditions and user characteristics.

Multi-user virtual reality (VR) systems support multiple locomotion methods, but coordination may be challenging when partners use different methods. We studied dyads navigating a shared virtual maze using symmetric (same) or asymmetric (different) locomotion methods-specifically, steering (continuous movement with physical turning) and teleportation (point-and-teleport within 30 meters with physical turning)-and assessed survey knowledge using direction and straightline distance estimates between objects. We found that (1) dyads acquired better survey knowledge than individuals across locomotion conditions; (2) teleportation showed no significant difference from steering in distance error and yielded significantly lower angular error; (3) asymmetric locomotion methods did not significantly increase cybersickness compared to symmetric locomotion methods; and (4) symmetric and asymmetric dyads showed no significant differences in distance or angular errors. Overall, these results suggest that collaborative navigation improves spatial learning and that asymmetric locomotion can improve accessibility without significantly increasing cybersickness in multi-user VR applications.

Maintaining physical distance is a crucial aspect of social interactions in the real world. This paper investigates how people embodied in self-avatars, manage such spacing in immersive virtual environments. Specifically, we measured how interpersonal distance (IPD) and peripersonal space (PPS) were affected by a surrounding virtual crowd. We designed three crowd density conditions - no crowd, a medium crowd, and a dense crowd - where participants interacted with a virtual human. Additionally, based on the results of a questionnaire, participants were grouped into one of three groups depending on their self-reported comfort level with crowds: some found crowds relaxing, some were indifferent, and some found crowds stressful. Our findings revealed that IPD decreased as crowd density increased, regardless of the participants' groupings. However, PPS exhibited a more nuanced response, with those stressed by a crowd showing heightened sensitivity to crowd density, reflected by larger PPS boundaries, faster reaction times, and less confidence when estimating their PPS boundary. The study also identified significant interactions between crowd density and participants' comfort levels, particularly among the group stressed by a crowd, where heightened stress led to expanded personal space. These results show the critical role of social and environmental factors in shaping spatial perception and behavior in virtual environments.

Gaze-adaptive interfaces are increasingly employed in mixed reality (MR) to support attention-aware interaction. However, it remains unclear how the usability and effectiveness of such interfaces are influenced by individual differences in spatial ability, cognition, or prior experience. This paper addresses this gap by focusing on user spatial navigation ability and its role in modulating user interaction with gaze-adaptive cues during spatial navigation. We compare gaze-adaptive cues to proximity-triggered cues in an MR navigation task, and leverage eye tracking to examine how users' gaze patterns relate to their spatial ability and spatial knowledge acquisition under each cue type. This enables us to explore contentindependent gaze measures as a potential subjective approach to assess spatial ability and learning outcomes in MR. Our findings show that gaze-adaptive cueing impacts users differently based on their spatial abilities. In particular, when adding gaze-adaptive cues, individuals with higher spatial ability seem to engaged with cues more and exhibited longer fixations. Furthermore, higher spatial ability and better learning outcomes were linked to more effective gaze patterns defined by longer fixation durations, larger saccadic amplitudes, and reduced gaze transition entropy, with effects varying by cue type. Our findings highlight the potential of gaze transition entropy for assessing spatial ability disparities in MR spatial tasks.

How children perceive and interact in immersive virtual environments (IVEs) is an important and emerging topic as virtual reality devices proliferate among this age group. How children understand their opportunities for action in an IVE — the perceived affordances of the space — in comparison to adults has, in particular, been rarely studied. This paper addresses this gap by examining children and adults’ reaching affordances in IVEs. We compared how children and adults judge their ability to reach up and out to targets at different distances in an IVE and how feedback from their actions may influence their accuracy. We found that both children and adults tend to overestimate their reaching capabilities, particularly when reaching out. However, by allowing them to actually reach up or out to the targets (action-outcome feedback), both groups significantly re-calibrated their judgments to align their estimates more closely with their actual capabilities. Notably, children exhibited greater initial overestimation than adults but recalibrated at a similar rate. These findings provide novel insights into how children interact with IVEs and suggest that well-designed feedback mechanisms can enhance their accuracy in action-based tasks. This work has implications for the design of IVEs for children, particularly in educational and training contexts, where ensuring accurate perception of capabilities is critical for learning and safety. By investigating both horizontal and vertical reach, this paper contributes to the understanding of affordance perception across age groups in IVEs, marking an essential step in adapting virtual reality technology to younger users.

Most modern head-mounted displays (HMDs) do not support the full range of adult inter-pupillary distances (IPDs) (i.e., 45 – 80 mm) due to technological limitations. Prior work indicates that the mismatch between a user’s actual IPD and the IPD set in the HMD (“IPD mismatch”) can affect distance and size judgments in near space (0 – 2 m). Therefore, users with IPDs outside of the supported HMD IPD range may not perceive virtual environments (VEs) accurately. Across three experiments, we investigated whether IPD mismatch significantly affects peoples’ distance judgments at longer distances (4 – 7 m). In two of the experiments, we recruited participants with IPDs smaller than the minimum supported IPD of the HTC Vive Pro HMD. They estimated distances in action space using verbal estimation (Experiment 1) and blind walking (Experiment 2) measures in indoor VEs. We found that: (i) distances were underestimated in action space, and (ii) IPD mismatch had minimal to no effect on their distance judgments. In a third experiment, we investigated whether we could generalize our findings to participants with an IPD within the supported HMD IPD range. We were able to replicate our previous findings. Overall, our findings suggest that IPD mismatch in an HMD may not be a major factor in distance underestimation in action space in VEs.

Omnidirectional treadmills provide one solution for locomoting through large virtual environments in confined physical spaces. Through two experiments, this paper evaluated locomotion on an omnidirectional treadmill (Cyberith Virtualizer Elite 2) by comparing it to natural walking in an open physical space. In Experiment 1, participants judged distances and completed a path integration task using the treadmill and natural walking. Participants walked further on the treadmill but had larger angular errors during path integration, potentially due to increased cybersickness. Experiment 2 varied path lengths during path integration and found that longer paths led to higher cybersickness scores but did not affect performance. The paper offers interpretations and suggestions for using omnidirectional treadmills in virtual reality.

Virtual reality (VR) has become widely accessible through the development of more commercially available head-mounted displays (HMDs). This accessibility has particularly increased the use of VR in children. However, much of the previous research on understanding perception and action in virtual reality has only been conducted on adults, leaving many open questions about how children perceive and interact with virtual environments. In this paper, we examine whether there are age-related differences in judging the ability to step over a gap using immersive VR. Affordances are a useful measure for understanding objective perceptions of the actions that can be performed in an immersive virtual environment. Such measures are particularly well suited for children given they can easily respond yes or no as to whether they perceive that they can step over a gap. Further, manipulations of the size of virtual body parts could allow us to ascertain how much children rely on the perceived size of their bodies to make decisions about actions. In Experiment 1, adults and children saw motion-tracked virtual feet that were either larger or smaller than their actual foot size. They had to respond as to whether they could step over gaps that varied in width. They also gave perceptual estimates of the width of the gap in feet or meters. The results showed that adults who experience the smaller virtual feet underestimated their stepping ability more than adults with the larger feet. However, children had the opposite effect, such that seeing smaller virtual feet led to an overestimation of their stepping ability. To test whether this age-related difference in body scaling was due to misperception of foot size, adults and children matched virtual shoes to their actual feet size in Experiment 2. This matching task showed no perceptual differences between the age groups. Across our two experiments, we showed that children scale gap affordance judgments differently than adults and this difference cannot be explained by difference in perceptions of the size of virtual feet. The results suggest that children can effectively make perception and action judgments in virtual reality, but do not always do so in the same manner as adults. Such a finding has implications for the design of virtual reality games, educational tools, and training systems that are becoming increasingly common for children.

While deep learning has revolutionized computer-aided drug discovery, the AI community has predominantly focused on model innovation and placed less emphasis on establishing best benchmarking practices. We posit that without a sound model evaluation framework, the AI community's efforts cannot reach their full potential, thereby slowing the progress and transfer of innovation into real-world drug discovery.Thus, in this paper, we seek to establish a new gold standard for small molecule drug discovery benchmarking, WelQrate. Specifically, our contributions are threefold: WelQrate dataset collection - we introduce a meticulously curated collection of 9 datasets spanning 5 therapeutic target classes. Our hierarchical curation pipelines, designed by drug discovery experts, go beyond the primary high-throughput screen by leveraging additional confirmatory and counter screens along with rigorous domain-driven preprocessing, such as Pan-Assay Interference Compounds (PAINS) filtering, to ensure the high-quality data in the datasets; WelQrate Evaluation Framework - we propose a standardized model evaluation framework considering high-quality datasets, featurization, 3D conformation generation, evaluation metrics, and data splits, which provides a reliable benchmarking for drug discovery experts conducting real-world virtual screening; Benchmarking - we evaluate model performance through various research questions using the WelQrate dataset collection, exploring the effects of different models, dataset quality, featurization methods, and data splitting strategies on the results.In summary, we recommend adopting our proposed WelQrate as the gold standard in small molecule drug discovery benchmarking. The WelQrate dataset collection, along with the curation codes, and experimental scripts are all publicly available at www.WelQrate.org.

Accurate perception of one’s action capabilities, or affordance perception, is essential for successful interaction with both real and virtual environments. Affordance perception can potentially be improved by receiving feedback. It is unknown what specific types of feedback are needed for improvements in affordance perception to occur, particularly in virtual environments where cues may be impoverished. The current work studied perception of horizontal reachability in virtual and augmented reality (VR and AR), specifically whether it would improve with feedback, and if any improvement transferred to the real world. Multiple types of feedback were studied in VR or AR: exploratory behavior, static outcome, and action outcome feedback. Our results indicate that exploratory behavior is sufficient for improvement in perceived reachability in VR, but in AR, outcome feedback is necessary. In both VR and AR, outcome feedback was required for improvement in perceived reachability to transfer to the real world. These findings have practical implications for training in virtual environments. If virtual environments are used for training actions that ultimately need to be performed in the real world, outcome feedback should be provided.

In computer-aided drug discovery, quantitative structure activity relation models are trained to predict biological activity from chemical structure. Despite the recent success of applying graph neural network to this task, important chemical information such as molecular chirality is ignored. To fill this crucial gap, we propose Molecular-Kernel Graph NeuralNetwork (MolKGNN) for molecular representation learning, which features SE(3)-/conformation invariance, chirality-awareness, and interpretability. For our MolKGNN, we first design a molecular graph convolution to capture the chemical pattern by comparing the atom's similarity with the learnable molecular kernels. Furthermore, we propagate the similarity score to capture the higher-order chemical pattern. To assess the method, we conduct a comprehensive evaluation with nine well-curated datasets spanning numerous important drug targets that feature realistic high class imbalance and it demonstrates the superiority of MolKGNN over other graph neural networks in computer-aided drug discovery. Meanwhile, the learned kernels identify patterns that agree with domain knowledge, confirming the pragmatic interpretability of this approach. Our code and supplementary material are publicly available at https://github.com/meilerlab/MolKGNN.

Although distance perception to Augmented Reality (AR) objects has been studied for decades, little is known about absolute distance perception with the newest available AR displays. One significant distinction in categories of head-worn AR displays is whether they are optical see-through (OST) or video see-through (VST). These two types of devices have different methods of rendering that could affect the cues available for perceiving distance. Specifically, rendering cast shadows can be challenging, especially in OST displays that rely on additive light for rendering, and there may be alternative shadow shading methods that are equally as effective for conveying cues to depth. The current study tests absolute egocentric distance judgments to targets 3-6 meters away from an observer with two types of shadows, in two types of AR displays, the Hololens 2 (OST) and the Varjo XR-3 (VST). Shadows were realistic cast shadows or non-realistic shadows in the form of a stylized ring placed beneath the object. Participants verbally reported perceived distance to spherical virtual targets presented on or above the ground, viewed through the displays in a real world classroom. We found overall distance underestimation in both devices, but that estimations were more accurate with the Hololens 2 compared to the Varjo XR-3. There was little support for a difference in accuracy of estimations between shadow conditions or position on or above the ground (confirmed by a Bayesian analysis), suggesting that non-realistic shadows may be a good option for providing additional shading cues for depth in AR.

We are pleased to present the proceedings of the 30th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2023), held March 25–29, 2023, in Shanghai, China, in hybrid format. These proceedings contain 69 of the 130 papers presented at IEEE VR 2023, with the other 61 papers being published in the IEEE VR 2023 special issue of the Transactions on Visualization and Computer Graphics. IEEE VR 2023 had 612 submissions for an acceptance rate of 21%.

As applications for virtual reality (VR) and augmented reality (AR) technology increase, it will be important to understand how users perceive their action capabilities in virtual environments. Feedback about actions may help to calibrate perception for action opportunities (affordances) so that action judgments in VR and AR mirror actors’ real abilities. Previous work indicates that walking through a virtual doorway while wielding an object can calibrate the perception of one’s passability through feedback from collisions. In the current study, we aimed to replicate this calibration through feedback using a different paradigm in VR while also testing whether this calibration transfers to AR. Participants held a pole at 45°and made passability judgments in AR (pretest phase). Then, they made passability judgments in VR and received feedback on those judgments by walking through a virtual doorway while holding the pole (calibration phase). Participants then returned to AR to make posttest passability judgments. Results indicate that feedback calibrated participants’ judgments in VR. Moreover, this calibration transferred to the AR environment. In other words, after experiencing feedback in VR, passability judgments in VR and in AR became closer to an actor’s actual ability, which could make training applications in these technologies more effective.

In this special issue of IEEE Transactions on Visualization and Computer Graphics (TVCG), we are pleased to present the top papers from the 30th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2023), held March 25–29, 2023, in Shanghai, China, in hybrid format.

Daily travel usually demands navigation on foot across a variety of different application domains, including tasks like search and rescue or commuting. Head-mounted augmented reality (AR) displays provide a preview of future navigation systems on foot, but designing them is still an open problem. In this paper, we look at two choices that such AR systems can make for navigation: 1) whether to denote landmarks with AR cues and 2) how to convey navigation instructions. Specifically, instructions can be given via a head-referenced display (screen-fixed frame of reference) or by giving directions fixed to global positions in the world (world-fixed frame of reference). Given limitations with the tracking stability, field of view, and brightness of most currently available head-mounted AR displays for lengthy routes outdoors, we decided to simulate these conditions in virtual reality. In the current study, participants navigated an urban virtual environment and their spatial knowledge acquisition was assessed. We experimented with whether or not landmarks in the environment were cued, as well as how navigation instructions were displayed (i.e., via screen-fixed or world-fixed directions). We found that the world-fixed frame of reference resulted in better spatial learning when there were no landmarks cued; adding AR landmark cues marginally improved spatial learning in the screen-fixed condition. These benefits in learning were also correlated with participants' reported sense of direction. Our findings have implications for the design of future cognition-driven navigation systems.

Although it is commonly accepted that depth perception in augmented reality (AR) displays is distorted, we have yet to isolate which properties of AR affect people’s ability to correctly perceive virtual objects in real spaces. From prior research on depth perception in commercial virtual reality, it is likely that ergonomic properties and graphical limitations impact visual perception in head-mounted displays (HMDs). However, an insufficient amount of research has been conducted in augmented reality HMDs for us to begin isolating pertinent factors in this family of displays. To this end, in the current research, we evaluate absolute measures of distance perception in the Microsoft HoloLens 2, an optical see-through AR display, and the Varjo XR-3, a video see-through AR display. The current work is the first to evaluate either device using absolute distance perception as a measure. For each display, we asked participants to verbally report distance judgments to both grounded and floating targets that were rendered either with or without a cast shadow along the ground. Our findings suggest that currently available video see-through displays may induce more distance underestimation than their optical see-through counterparts. We also find that the vertical position of an object and the presence of a cast shadow influence depth perception.

Consumer level virtual experiences almost always occur when physical space is limited, either by the constraints of an indoor space or of a tracked area. This observation coupled with the need for movement through large virtual spaces has resulted in a proliferation of research into locomotion interfaces that decouples movement through the virtual environment from movement in the real world. While many locomotion interfaces support movement of some kind in the real world, some do not. This paper examines the effect of the amount of physical space used in the real world on one popular locomotion interface, resetting, when compared to a locomotion interface that requires minimal physical space, walking in place. The metric used to compare the two locomotion interfaces was navigation performance, specifically, the acquisition of survey knowledge. We find that, while there are trade-offs between the two methods, walking in place is preferable in small spaces.

Virtual objects in augmented reality (AR) often appear to float atop real world surfaces, which makes it difficult to determine where they are positioned in space. This is problematic as many applications for AR require accurate spatial perception. In the current study, we examine how the way we render cast shadows-which act as an important monocular depth cue for creating a sense of contact between an object and the surface beneath it-impacts spatial perception. Over two experiments, we evaluate people's sense of surface contact given both traditional and non-traditional shadow shading methods in optical see-through augmented reality (OST AR), video see-through augmented reality (VST AR), and virtual reality (VR) head-mounted displays. Our results provide evidence that nontraditional shading techniques for rendering shadows in AR displays may enhance the accuracy of one's perception of surface contact. This finding implies a possible tradeoff between photorealism and accuracy of depth perception, especially in OST AR displays. However, it also supports the use of more stylized graphics like non-traditional cast shadows to improve perception and interaction in AR applications.

In this paper, we examine how embodiment and manipulation of a self-avatar's dimensions - specifically the arm length - affect users' judgments of the personal space around them in an immersive virtual environment. In the real world, personal space is the immediate space around the body in which physical interactions are possible. Personal space is increasingly studied in virtual environments because of its importance to social interactions. Here, we specifically look at two components of personal space, interpersonal and peripersonal space, and how they are affected by embodiment and the sizing of a self-avatar. We manipulated embodiment, hypothesizing that higher levels of embodiment will result in larger measures of interpersonal space and smaller measures of peripersonal space. Likewise, we manipulated the arm length of a self-avatar, hypothesizing that while interpersonal space would change with changing arm length, peripersonal space would not. We found that the representation of both interpersonal and peripersonal space change when the user experiences differing levels of embodiment in accordance with our hypotheses, and that only interpersonal space was sensitive to changes in the dimensions of a self-avatar's arms. These findings provide increased understanding of the role of embodiment and self-avatars in the regulation of personal space, and provide foundations for improved design of social interaction in virtual environments.

Today, augmented reality (AR) is most easily experienced through a mobile device such as a modern smartphone. For AR to be useful for applications such as training, it is important to understand how people perceive interactions with virtual objects presented to them via mobile AR. In this paper, we investigated two judgments of action capabilities (affordances) with virtual objects presented through smartphones: passing through an aperture and stepping over a gap. Our goals were to 1) determine if people can reliably scale these judgments to their body dimensions or capabilities and 2) explore whether cues presented in the context of the action could change their judgments. Assessments of perceived action capabilities were made in a pre/post-test design in which observers judged their affordances towards virtual objects prior to seeing an AR cue denoting their body dimension/capability, while viewing the cue, and after seeing the cue. Different patterns of results were found for the two affordances. For passing through, estimates became closer to shoulder width in the post-cue compared to the pre-cue block. For gap stepping, estimates were closer to actual stepping capability while viewing the cue, but did not persist when the cue was no longer present. Overall, our findings show that mobile smartphones can be used to assess perceived action capabilities with virtual targets and that AR cues can influence the perception of action capabilities in these devices. Our work provides a foundation for future studies investigating perception with the use of mobile AR with smartphones.

Immersive virtual environments (VEs) are most useful for training and education when viewers perceive and act accurately within them. Judgments of action capabilities within a VE provide a good measure of perceptual fidelity - the notion of how closely perception and action in the VE match that in the real world - and can also assess how perception for action may be calibrated with visual feedback based on one's own actions. In the current study we tested judgments of action capabilities within a VE for two different reaching behaviors: reaching out and reaching up. Our goal was to assess whether feedback from actual reaching improves judgments and if any recalibration due to feedback differed across reaching behaviors. We first measured participants' actual reaching out and reaching up capabilities so that feedback trials could be scaled to their actual abilities. Participants then completed blocks of alternating perceptual adjustment and feedback trials. In adjustment trials, they adjusted a virtual target to a distance perceived to be just reachable. In feedback trials, they viewed targets that were farther or closer than their actual reach, decided whether the target was reachable, and then reached out to the target to receive visual feedback from a hand-held controller. The first feedback block manipulated the target distance to be 30% over or under actual reach and subsequent blocks decreased the deviation to 20%,10% and 5% of actual reach. We found that for both reaching behaviors, reach was initially overestimated, and then perceptual estimations decreased to become more accurate over feedback blocks. Accuracy in the feedback trials themselves showed that targets just beyond reach were more difficult to judge correctly. This study establishes a straightforward methodology that can be used for calibration of actions in VEs and has implications for applications that depend on accurate reaching within VEs.

Augmented reality ( AR ) is important for training complex tasks, such as navigation, assembly, and medical procedures. The effectiveness of such training may depend on accurate spatial localization of AR objects in the environment. This article presents two experiments that test egocentric distance perception in augmented reality within and at the boundaries of action space (up to 35 m) in comparison with distance perception in a matched real-world ( RW ) environment. Using the Microsoft HoloLens, in Experiment 1, participants in two different RW settings judged egocentric distances (ranging from 10 to 35 m) to an AR avatar or a real person using a visual matching measure. Distances to augmented targets were underestimated compared to real targets in the two indoor, RW contexts. Experiment 2 aimed to generalize the results to an absolute distance measure using verbal reports in one of the indoor environments. Similar to Experiment 1, distances to augmented targets were underestimated compared to real targets. We discuss these findings with respect to the importance of methodologies that directly compare performance in real and mediated environments, as well as the inherent differences present in mediated environments that are “matched” to the real world.

Mixed reality is increasingly being used for training, especially for tasks that are difficult for humans to perform such as far distance perception. Although there has been much prior work on the perception of distance in mixed reality, the majority of the work has focused on distances ranging up to about 30 meters. The little work that has investigated distance perception beyond 30 meters has been mixed in terms of whether people tend to over- or underestimate distances in this range. In the current study, we investigated the perception of distances ranging from 25-500 meters in a fully mediated outdoor environment using the ZED Mini and HTC Vive Pro. To our knowledge, no work has examined distance perception to this range in mixed reality. Participants in the study verbally estimated the distances to a variety of objects. Distances were overestimated from 25-200 meters but underestimated from 300-500 meters. Overall, far distance perception in mixed reality may be biased in different directions depending on the range of distances, suggesting a need for future research in this area to support training of far distance perception.

Commodity-level virtual reality equipment is now available to all ages. To better understand how cognitive development affects people’s spatial memory in virtual reality, we assess how adults (20-29 years old) and teenagers (14-17 years old) represent their spatial memory of objects in an immersive virtual environment (IVE) where height is encoded. Despite virtual reality being a favorable conduit for the study of egocentric spatial memory, prior studies have predominately looked at objects placed at similar heights. Within a stairwell environment, participants learned the positions of nine target objects. In one condition, all objects were placed near eye height. In another, they were placed at varying heights. Our results indicate that participants’ errors and latencies were similar in both environments, and across age groups. Our results have implications for the development of IVEs and the expansion of immersive technology to a more diverse, younger audience.

In building a new drug design mode for the popular citizen scientist game Foldit, we focus on creating an easy-to-use and intuitive interface to confer complex scientific concepts to citizen scientist players. We hypothesize that to be efficient in the hands of citizen scientists such an interface will look different from well-established drug-design software used by experts. We used the relaxed think-aloud method to compare citizen and expert scientists working with our prototype interface for Foldit Drug Design Mode (FDDM). First, we tested if the two groups are providing different feedback when it comes to the usability of the prototype interface. Second, we investigated how the difference between the two groups might inform a new game design. As expected, the results confirm that experienced scientists differ from citizen scientists in engaging their background knowledge when interacting with the game. We then provided a prioritization list of background knowledge employed by the expert scientists to derive design suggestions for FDDM.

Navigation and wayfinding are often accomplished collectively, with groups of people. Yet most studies of navigation and wayfinding behavior, and of the acquisition of spatial knowledge more generally, focus on the individual. In this paper we extend the investigation of these topics to dyads. In particular, we focus on how well straight-line distances and directions between objects (survey knowledge) were learned by individuals and by the same individuals when comprising a dyad. Our experiment was carried out in a shared virtual environment and we report on the technical issues in conducting such a collaborative experiment in a shared virtual environment, such as the choice of locomotion mode and the provision of full-body self-avatars. Our findings indicate that dyads outperform individuals in their acquisition of survey knowledge.

In this paper we examine the extent of functional reaching space, or peripersonal space, in immersive three-dimensional virtual reality. In the real world a person’s peripersonal space boundaries can be altered by factors in the environment and by social context. We completed two studies with visual and tactile stimuli to determine peripersonal space boundaries. These studies investigated whether peripersonal space boundaries in an immersive virtual environment are consistent with those in the real world, and could be altered by object and virtual agent interactions. We found that while peripersonal space boundaries were consistent with those in the real world, they were responsive to object and agent interactions. Moreover, while people’s reactions to the objects and agents varied, the peripersonal space boundaries remained consistent. These findings have potential implications for the design of virtual environments.

The effectiveness of Augmented Reality (AR) for spatial applications is likely influenced by the extent to which users perceive that they can act on virtual objects as if they are real. While there are some promising initial results that action capabilities in AR are perceived similarly to the real world, AR differs from completely real-world viewing both in the visual information available within a restricted field of view (FOV) and in the ways that perceptual-motor feedback can be provided for the outcome of actions. We addressed both of these differences in a study of judgments of passing through an AR aperture defined by two moveable virtual walls presented via the Microsoft HoloLens. First, we investigated the possible effects of viewing distance on affordance judgments, predicting greater error in judgments at a close distance because of the inability to see the entirety of the aperture within the FOV. Second, we explored the effect of feedback on affordance judgments, predicting that verbal feedback about whether a judgment was correct would be sufficient to see improvements in judgments over time. In contrast to our predictions for distance, we found that judgments for passing through were closer to actual shoulder width when viewed at a distance near the aperture (0.85 m) compared to a farther viewing point (3.20 m). Verbal feedback reduced error over trials, but only at the farther distance, possibly because the judgments at the near distance were close to the ceiling of performance. These findings have implications for tasks that require accurate perception of action capabilities in AR, such as training procedures.

Navigation, or the means by which people find their way in an environment, depends on the ability to combine information from multiple sources so that properties of an environment, such as the location of a goal, can be estimated. An important source of information for navigation are spatial cues generated by self-motion. Navigation based solely on body-based cues generated by self-motion is called path integration. In virtual reality and video games, many locomotion systems, that is, methods that move users through a virtual environment, can often distort or deprive users of important self-motion cues. There has been much study of this issue, and in this paper, we extend that study in novel directions by assessing the effect of four game-like locomotion interfaces on navigation performance using path integration. The salient features of our locomotion interfaces are that two are primarily continuous, i.e., more like a joystick, and two are primarily discrete, i.e., more like teleportation. Our main findings are that the perspective of path integration, people are able to use all methods, although continuous methods outperform discrete methods.

The anatomy of the ear and the bones surrounding it are intricate yet critical for medical professionals to know. Current best practices teach ear anatomy through two-dimensional representations, which poorly characterize the three-dimensional (3D), spatial nature of the anatomy and make it difficult to learn and visualize. In this work, we describe an immersive, stereoscopic visualization tool for the anatomy of the ear based on real patient data. We describe the interface and its construction. And we compare how well medical students learn ear anatomy in the simulation compared with more traditional learning methods. Our preliminary results suggest that virtual reality may be an effective tool for anatomy education in this context.

Walking in place is a standard method for moving through large virtual environments when physical space or positional tracking is limited. This technique has become increasingly prominent with the advent of mobile virtual reality in which external tracking may not be present. In this paper, we revisit walking in place algorithms to address some of their technical challenges. Namely, our solutions attend to improving starting, stopping, and speed control for individual users. From a hand-tuned threshold based algorithm, we provide a new, fast method for individualizing the walking in place algorithm based on biomechanic measures of step rate. In addition, we introduce a new walking in place model based on a convolutional neural network trained to differentiate walking and standing. Over two experiments we assess these methods against a traditional threshold based algorithm on two mobile virtual reality platforms. The assessments are based on controllability, scale, and presence. Our results suggest that an adequately trained convolutional neural network can be an effective way of implementing walking in place.

Augmented reality (AR) technologies have the potential to provide individuals with unique training and visualizations, but the effectiveness of these applications may be influenced by users' perceptions of the distance to AR objects. Perceived distances to AR objects may be biased if these objects do not appear to make contact with the ground plane. The current work compared distance judgments of AR targets presented on the ground versus off the ground when no additional AR depth cues, such as shadows, were available to denote ground contact. We predicted that without additional information for height off the ground, observers would perceive the off-ground objects as placed on the ground, but at farther distances. Furthermore, this bias should be exaggerated when targets were viewed with one eye rather than two. In our experiment, participants judged the absolute egocentric distance to various cubes presented on or off the ground with an action-based measure, blind walking. We found that observers walked farther for off-ground AR objects and that this effect was exaggerated when participants viewed off-ground objects with monocular vision compared to binocular vision. However, we also found that the restriction of binocular cues influenced participants' distance judgments for on-ground AR objects. Our results suggest that distances to off-ground AR objects are perceived differently than on-ground AR objects and that the elimination of binocular cues further influences how users perceive these distances.

It is an open question as to whether people perceive and act in augmented reality environments in the same way that they do in real environments. The current work investigated participants' judgments of whether or not they could act on an obstacle portrayed with augmented reality. Specifically, we presented gaps of varying widths and depths to participants in augmented reality using the Microsoft HoloLens. We asked users to indicate whether or not they believed that they could step across the virtual gaps given their widths and depths. Averaging across changes in width and depth, users generally underestimated their abilities to cross gaps. However, this underestimation was significantly greater when the gaps were deep. Thus, our findings suggest that users in augmented reality respond with more conservative judgments when presented with risky stimuli-a response that mimics real world behavior. Their altered reactions to deeper gaps may provide early evidence for augmented reality's capacity to evoke a sense of realism or immersion and its use in evaluating perception and action.

An essential question in understanding how to develop and build collaborative immersive virtual environments (IVEs) is recognizing how people perform actions together. Many actions in the real world require that people act without prior planning, and these actions are executed quite successfully. In this paper, we study the common action of two people passing through an aperture together in both the real world (Experiment 1) and in a distributed, collaborative IVE (Experiment 2). The aperture's width is varied from too narrow to be passable to so wide as to be easily passable by both participants together simultaneously. We do this in the real world for all possible gender-based pairings. In virtual reality, however, there is potential for the gender of the participant and the gender of the self-avatar to be different. We also investigate the joint action for all possible gender-based pairings in the distributed IVE. Results indicated that, in the real world, social dynamics between gendered pairings emerged; male-male pairings refused to concede to one another until absolutely necessary while other pairings did not. Male-female pairings were most likely to provide ample space to one another during passage. These behaviors seemed not to appear in the IVE, and avatar gender across all pairings generated no significant behavioral differences. In addition, participants tended to require wider gaps to allow for passage in the IVE. These findings establish base knowledge of social dynamics and affordance behaviors within multi-user IVEs.

Most virtual environments that people locomote through with head-mounted displays are flat to match the physical environment that people are actively walking on. In this paper we simulated stair climbing, and evaluated how well people could assess the distance they had climbed after several minutes of the activity under various conditions. We varied factors such as the presence of virtual feet (shoes), whether the stairwell was open or enclosed, the presence or absence of passive haptic markers, and whether a subject was ascending or descending. In general, the distance climbed or descended was overestimated, consistent with prior work on the perception of height. We find that subjects have significantly better ability to estimate their error with the presence of virtual shoes than without, and when the environment was open. Having shoes also resulted in significantly higher ratings of presence. We also find a significant tendency for females to show higher ratings of simulator sickness.

The utility of mediated environments increases when environmental scale (size and distance) is perceived accurately. We present the use of perceived affordances---judgments of action capabilities---as an objective way to assess space perception in an augmented reality (AR) environment. The current study extends the previous use of this methodology in virtual reality (VR) to AR. We tested two locomotion-based affordance tasks. In the first experiment, observers judged whether they could pass through a virtual aperture presented at different widths and distances, and also judged the distance to the aperture. In the second experiment, observers judged whether they could step over a virtual gap on the ground. In both experiments, the virtual objects were displayed with the HoloLens in a real laboratory environment. We demonstrate that affordances for passing through and perceived distance to the aperture are similar in AR to those measured in the real world, but that judgments of gap-crossing in AR were underestimated. These differences across two affordances may result from the different spatial characteristics of the virtual objects (on the ground versus extending off the ground).

Underestimation of egocentric distances in immersive virtual environments using various head-mounted displays (HMDs) has been a puzzling topic of research interest for several years. As more commodity-level systems become available to developers, it is important to test the variation of underestimation in each system since reasons for underestimation remain elusive. In this article, we examine several different systems in two experiments and comparatively evaluate how much users underestimate distances in each one. To observe distance estimation behavior, a standard indirect blind walking task was used. An Oculus Rift DK1, weighted Oculus Rift DK1, Oculus Rift DK1 with an artificially restricted field of view, Nvis SX60, Nvis SX111, Oculus Rift DK2, Oculus Rift consumer version (CV1), and HTC Vive were tested. The weighted and restricted field of view HMDs were evaluated to determine the effect of these factors on distance underestimation; the other systems were evaluated because they are popular systems that are widely available. We found that weight and field of view restrictions heightened underestimation in the Rift DK1. Results from these conditions were comparable to the Nvis SX60 and SX111. The Oculus Rift DK1 and CV1 possessed the least amount of distance underestimation, but in general, commodity-level HMDs provided more accurate estimates of distance than the prior generation of HMDs.

As virtual reality expands in popularity, an increasingly diverse audience is gaining exposure to immersive virtual environments (IVEs). A significant body of research has demonstrated how perception and action work in such environments, but most of this work has been done studying adults. Less is known about how physical and cognitive development affect perception and action in IVEs, particularly as applied to preteen and teenage children. Accordingly, in the current study we assess how preteens (children aged 8-12 years) and teenagers (children aged 15-18 years) respond to mismatches between their motor behavior and the visual information presented by an IVE. Over two experiments, we evaluate how these individuals recalibrate their actions across functionally distinct systems of movement. The first experiment analyzed forward walking recalibration after exposure to an IVE with either increased or decreased visual flow. Visual flow during normal bipedal locomotion was manipulated to be either twice or half as fast as the physical gait. The second experiment leveraged a prism throwing adaptation paradigm to test the effect of recalibration on throwing movement. In the first experiment, our results show no differences across age groups, although subjects generally experienced a post-exposure effect of shortened distance estimation after experiencing visually faster flow and longer distance estimation after experiencing visually slower flow. In the second experiment, subjects generally showed the typical prism adaptation behavior of a throwing after-effect error. The error lasted longer for preteens than older children. Our results have implications for the design of virtual systems with children as a target audience.

Locomotion in large virtual environments is currently unsupported in smartphone-powered virtual reality headsets, particularly within the confines of limited physical space. While motion controllers are a workaround for this issue, they exhibit known problems: they occupy the subject's hands, and they cause poor navigation performance. In this paper, we investigate three hands-free methods for navigating large virtual environments. The first method is resetting, a reorientation technique that allows for both translation and rotation body-based cues. The other two methods are walking in place techniques that use only rotation-based cues. In the first walking in place technique, we make use of the inertial measurement unit of the smartphone embedded in a Samsung Gear VR to detect when subjects are stepping. The second technique uses the Kinect's skeletal tracking for step detection. In this paper, we measure the survey component of spatial knowledge to assess three navigation conditions. Our metrics examine how well subjects gather and retain information from their environment, as well as how well they integrate it into a single model. We find that resetting leads to the strongest acquisition of survey knowledge, which we believe is due to the vestibular cues provided by this method.

The use of first-person self-avatars in immersive virtual environments (VEs) has grown over recent years. It is unknown, however, how visual feedback from a self-avatar influences a user's online actions and subsequent calibration of actions within an immersive VE. The current paper uses a prism throwing adaptation paradigm to test the role of a self-avatar arm or full body on action calibration in a VE. Participants' throwing accuracy to a target on the ground was measured first in a normal viewing environment, then with the visual field rotated clockwise about their vertical axis by 17° (prism simulation), and then again in the normal viewing environment with the prism distortion removed. Participants experienced either no-avatar, a first-person avatar arm and hand, or a first-person full body avatar during the entire experimental session, in a between-subjects manipulation. Results showed similar throwing error and adaptation during the prism exposure for all conditions, but a reduced aftereffect (displacement with respect to the target in the opposite direction of the prism-exposure) when the avatar arm or full body was present. The results are discussed in the context of how an avatar can provide a visual frame of reference to aid in action calibration.

Interesting virtual environments that permit free exploration are rarely small. A number of techniques have been developed to allow people to walk in larger virtual spaces than permitted by physical extent of the virtual reality hardware, and in this paper we compare three such methods in terms of how they affect presence and spatial awareness. In our first psychophysical study, we compared two methods of reorientation and one method of redirected walking on subjects' presence and spatial memory while navigating a pre-specified path. We further compared the two reorientation methods in a second psychophysical study involving free exploration and navigation in a large virtual environment. Our results provide criteria by which the choice of a locomotion method for navigating large virtual environments may be selected.

Vection is the illusion of self-motion, usually induced by a visual stimulus. It is important in virtual reality because inducing it in motion simulations can lead to improved experiences. We examine linear and circular vection in commodity level head-mounted displays. We compare the experience of circular vection induced through a real world stimulus, an optokinetic drum, with that experienced through a virtual stimulus. With virtual stimuli, we also compare circular vection with linear horizontal and linear vertical vection. Finally, we examine circular and linear vection in more naturalistic virtual environments. Linear vection was induced more rapidly than any other type, but circular vection occurs more rapidly with a real world stimulus than a virtual one. Our results have practical application and can inform virtual reality design that uses head-mounted display technology and wishes to establish vection. Circular vection can be induced easily and rapidly through visual mechanisms that have been known since Mach [1]. We have an optokinetic drum that is capable of inducing compelling circular vection that works in almost all people we have tested in it. The goal of this project is thus twofold: (1) to see how well and how reliably vection can be induced in virtual reality with commodity-level HMDs as compared to real-world stimuli, and (2) to see if compelling vection could be comparably induced when the stimuli are complex naturalistic scenes, not abstract optic flow patterns.

Animated digital self-representations of the user in an immersive virtual environment, a self-avatar, have been shown to aid in perceptual judgments in the virtual environment and to provide critical information for people deciding what actions they can and cannot take. In this paper we explore whether the form of the self-avatar is important in providing this information. In particular, we vary the form of a self-avatar between having no self-avatar, a simple line-based skeleton avatar, or a full-body, gender-matched self-avatar and examine whether the form of the self-avatar affects peoples judgments in whether they could or could not step off of a virtual ledge. Our results replicate prior work that shows that having a self-avatar provides critical information for this judgment, but finds no difference in the form of the self-avatar having an effect on the judgment.

Collaborative immersive virtual environments allow the behavior of one user to be observed by other users. In particular, behavior of users in such an environment is represented by each user possessing a self-avatar, a digital representation of themself. In this study we examined dyadic interactions in a collaborative immersive virtual environment when both users were present in the same physical space. This collocation in physical space allows for physical interaction between users as well as virtual interaction. In the context of a common physical gesture, high fiving, we examined the question of whether the form of the self-avatar was important, and whether collocation in the physical world provided benefits or not. We find that the form of the avatar is important but that physical collocation is not. These results reinforce the growing body of evidence that indicates that having a full-body avatar in a virtual environment provides benefits, and these results are significant because they demonstrate this in the context of a dyadic interaction.

People judge what they can and cannot do all the time when acting in the physical world. Can I step over that fence or do I need to duck under it? Can I step off of that ledge or do I need to climb off of it? These qualities of the environment that people perceive that allow them to act are called affordances. This article compares people’s judgments of affordances on two tasks in both the real world and in virtual environments presented with head-mounted displays. The two tasks were stepping over or ducking under a pole, and stepping straight off of a ledge. Comparisons between the real world and virtual environments are important because they allow us to evaluate the fidelity of virtual environments. Another reason is that virtual environment technologies enable precise control of the myriad perceptual cues at work in the physical world and deepen our understanding of how people use vision to decide how to act. In the experiments presented here, the presence or absence of a self-avatar—an animated graphical representation of a person embedded in the virtual environment—was a central factor. Another important factor was the presence or absence of action, that is, whether people performed the task or reported that they could or could not perform the task. The results show that animated self-avatars provide critical information for people deciding what they can and cannot do in virtual environments, and that action is significant in people’s affordance judgments.

A previous study, Young et al. [2014], explored comparisons between cost-differentiated head-mounted displays for the evaluation of various perception and action tasks. One of the comparisons in that work was egocentric distance estimation comparison using an Nvis SX 60 head-mounted display (HMD) and an Oculus Rift. This poster abstract supplements that study results with an additional HMD for evaluation of egocentric distance estimation task, an Nvis SX111. We find that performance of the Nvis SX111 is comparable to that of the Nvis SX60 in Young et al. [2014], which was significantly more compressed than the Oculus Rift.

When two eyes view different objects or scenes at the same time, stable binocular single vision gives way to alternations in perception. Called binocular rivalry, this beguiling phenomenon discloses the existence of inhibitory interactions between neural representations associated with the conflicting visual inputs. One strategy for learning details about this neural competition is to ascertain to what extent it is susceptible to top-down influences such as expectations engendered by one's own activity [Maruya et al. 2007]. Thanks to advances in virtual reality (VR) technology, this strategy can be implemented in the laboratory, which we have done. Specifically we are measuring the extent to which self-generated motion (walking) biases perception during binocular rivalry between two competing visual optic flow fields (one specifying forward locomotion and the other specifying backward locomotion). Our progress to date is reported in this poster.

The results of multiple studies conducted in the real world have demonstrated that human observers are usually very accurate in judging egocentric distance (the distance from the observer to a target) up to about 25 meters if the target is located on a continuous ground surface. Sinai et al. [1998] found that the presence of a texture discontinuity on the ground surface leads to distance underestimation. Two experiments were conducted to investigate the effect of a discontinuous ground surface on distance perception in virtual environments (VEs). The first experiment was the replication of Experiment 1 from Wu et al. [2007]. The second experiment involved the manipulation of the position of the texture boundary.

Recent advances in technology and the opportunity to obtain commodity-level components have made the development and use of three-dimensional virtual environments more available than ever before. How well such components work to generate realistic virtual environments, particularly environments suitable for perception and action studies, is an open question. In this paper we compare two virtual reality systems in a variety of tasks: distance estimation, virtual object interaction, a complex search task, and a simple viewing experiment. The virtual reality systems center around two different head-mounted displays, a low-cost Oculus Rift and a high-cost Nvis SX60, which differ in resolution, field-of-view, and inertial properties, among other factors. We measure outcomes of the individual tasks as well as assessing simulator sickness and presence. We find that the low-cost system consistently outperforms the high-cost system, but there is some qualitative evidence that some people are more subject to simulator sickness in the low-cost system.

We explore whether a gender-matched, calibrated self-avatar affects the perception of the affordance of stepping off of a ledge, or visual cliff, in an immersive virtual environment. Visual cliffs form demonstrations in many immersive virtual environments because they create compelling environments. Understanding the role that self-avatars contribute to making such environments compelling is an important problem. We conducted an experiment to find the threshold at which subjects on a ledge in an immersive virtual environment would report that they could step gracefully off of the ledge without losing their balance, and compared the threshold height at which their decision changed under the condition of having and not having a self-avatar. The results show that people unrealistically say they can step off a ledge that is approximately 50% of their eyeheight without a self-avatar, and realistically about 25% of their eyeheight with a self-avatar.

In this study, we examine subjects' spatial orientation in a head--mounted display (HMD) based virtual environment (VE) when objects are placed at different vertical viewing angles. More specifically, we examine spatial orientation when subjects memorize the location of objects on the ground, at eyelevel, and up in the air. Interestingly, we found that spatial orientation was the same across all conditions. This finding is important because it informs researchers who use similar spatial orientation tasks that variations in viewing neck angle to targets is not necessarily an issue.

This experiment investigates the perceived differences in the quality of animation generated using motion capture data using a Vicon motion capture system and a Microsoft Kinect sensor. The Kinect uses a depth map camera to determine the position of users and objects within the view of the camera. Using this information, the depth map can be processed to track the motion of a user over time, and for this experiment, this was done with PrimeSense's NITE algorithm[PrimeSense]. The final recording generated from the depth map camera is noisy and is clearly a lower quality recording when compared to the motion capture data created using a Vicon system. To overcome this problem, the motion capture data from the Kinect can be filtered to smooth the noise and create more usable data. This experiment focuses on using a Butterworth filter to smooth the Kinect data and attempts to identify how much the data can be smoothed so that the quality of the motion capture data is indistinguishable from the Vicon data for short video segments.

The purpose of this study was to learn if self-avatars influence people's perception and action in virtual environments. People viewed two situations in a virtual environment through a head-mounted display and were asked to decide how they would act. In one situation their task was to imagine walking across a room which was divided by a horizontal bar. The bar's height was varied and the task was for people to say whether they would need to step over the bar or duck under it. In the other situation the task was to imagine walking through a doorway. The doorway's height was varied and the task was for people to say whether they could walk straight through the doorway or would need to duck to pass through. One-half the participants viewed the situations with a self-avatar and the others viewed it without an avatar. The height of the avatar was varied, so that it either equaled the participant's height or it was 15% taller. The results showed statistically significant effects of the avatar in the horizontal bar situation: their step-over or duck-under decision points were about 12% higher when the self-avatar was rendered taller. In the doorway situation the effect of the avatar was statistically non-significant. The step-over step-under task is a promising method to study perception and action in virtual environments.

This experiment investigates the spatial memory and attention when human acts as supervisor of one or two groups of distributed robot teams in a large virtual environment (VE). The problem is similar to learning a new environment and interpreting its spatial structure, e.g., [Mou and McNamara 2002], but less is known when attention is divided or locomotion is involved. Our motivation arises in the context of humans and robots acting cooperatively together as a team. Such teaming is becoming increasingly important in many scenarios, such as disaster relief, and wilderness search and rescue [Humphrey and Adams 2009].

This paper presents a mixed reality system for combining real robots, humans, and virtual robots. The system tracks and controls physical robots in local physical space, and inserts them into a virtual environment (VE). The system allows a human to locomote in a VE larger than the physically tracked space of the laboratory through a form of redirected walking. An evaluation assessed the conditions under which subjects found the system to be the most immersive.

In this work, we present a method of “Walking In Place” (WIP) on the Nintendo Wii Fit Balance Board to explore a virtual environment. We directly compare our method to joystick locomotion and normal walking. The joystick proves inferior to physically walking and to WIP on the Wii Balance Board (WIP--Wii). Interestingly, we find that physically exploring an environment on foot is equivalent in terms of spatial orientation to exploring an environment using our WIP--Wii method. This implies that the WIP--Wii is a good inexpensive alternative to exploring a virtual environment and it may be well--suited for exploring large virtual environments.

No abstract available.

This article develops methods for determining visually appealing motion transitions using linear blending. Motion transitions are segues between two sequences of animation, and are important components for generating compelling animation streams in virtual environments and computer games. Methods involving linear blending are studied because of their efficiency, computational speed, and widespread use. Two methods of transition specification are detailed, center-aligned and start-end transitions. First, we compute a set of optimal weights for an underlying cost metric used to determine the transition points. We then evaluate the optimally weighted cost metric for generalizability, appeal, and robustness through a cross-validation and user study. Next, we develop methods for computing visually appealing blend lengths for two broad categories of motion. We empirically evaluate these results through user studies. Finally, we assess the importance of these techniques by determining the minimum sensitivity of viewers to transition durations, the just noticeable difference, for both center-aligned and start-end specifications.

This paper describes the use of desktop and immersive virtual environments to study judgments that pedestrians make when deciding to cross a street. In particular, we assess the ability of people to discriminate and estimate time-to-contact (TTC) for approaching vehicles under a variety of conditions. Four experiments observing TTC judgments under various conditions are described. We examine the effect of type of vehicle, viewpoint, presentation mode, and TTC value on TTC judgments. We find no significant effect of type of vehicle or of viewpoint, extending prior work to cover all views typically encountered by pedestrians. Discrimination of short values for TTC judgments is generally consistent with the literature, but performance degrades significantly for long TTC values. Finally, we find no significant difference between judgments made in a desktop environment versus a head-mounted display, indicating that tracking the approaching vehicle with one's head does not aid discrimination. In general, people appear to use strategies similar to those that pedestrians use to make real-world, street-crossing decisions.

This paper presents results that demonstrate functional similarities in subjects' access to spatial knowledge (or spatial representation ) between real and virtual environments. Such representations are important components of the transfer of reasoning ability and knowledge between these two environments. In particular, we present two experiments aimed at investigating similarities in spatial knowledge derived from exploring on foot both physical environments and virtual environments presented through a head-mounted display. In the first experiment, subjects were asked to learn the locations of target objects in the real or virtual environment and then rotate the perspective by either physically locomoting to a new facing direction or imagining moving. The latencies and errors were generally worse after imagining locomoting and for greater degrees of rotation in perspective; they did not differ significantly across knowledge derived from exploring the physical versus virtual environments. In the second experiment, subjects were asked to imagine simple rotations versus simple translations in perspective. The errors and latencies indicated that the to-be-imagined disparity was linearly related after learning the physical and virtual environment. These results demonstrate functional similarities in access to knowledge of new perspective when it is learned by exploring physical environments and virtual renderings of the same environment.

We apply simplified image-based lighting methods to reduce the equipment, cost, time, and specialized skills required for high-quality photographic lighting of desktop-sized static objects such as museum artifacts. We place the object and a computer-steered moving-head spotlight inside a simple foam-core enclosure and use a camera to record photos as the light scans the box interior. Optimization, guided by interactive user sketching, selects a small set of these photos whose weighted sum best matches the user-defined target sketch. Unlike previous image-based relighting efforts, our method requires only a single area light source, yet it can achieve high-resolution light positioning to avoid multiple sharp shadows. A reduced version uses only a handheld light and may be suitable for battery-powered field photography equipment that fits into a backpack.

The sensory ego-sphere (SES) is a short-term memory for a robot in the form of an egocentric, tessellated, spherical, sensory-motor map of the robot's locale. This paper reports on: (1) the mapping to the SES of images with much higher resolution than the SES itself, and (2) the processing of visual attention on the SES. Described is a procedure to store spatially-overlapping imagery in the SES database and to generate and update a spherical composite of its visual contents. The composite image serves both as a visual map of the locale and as a representation of the local contents of the underlying full-resolution imagery. Visual attention enables fast alignment of overlapping images without warping or position optimization, since an attentional point (AP) on the composite typically corresponds to one on each of the collocated regions in the images. Such alignment speeds analysis of the multiple images of the area. Compositing and attention were performed two ways and compared: (1) APs were computed directly on the composite and not on the full resolution images until the time of retrieval. (2) The attentional operator was applied to all incoming imagery. Then the locations and values of the APs on the composite were computed as a function of those points within the corresponding regions of the collocated images. It was found that although the second method was slower, it produced consistent and, thereby, more useful APs

This paper reports that the superposition of a small set of behaviors, learned via teleoperation, can lead to robust completion of an articulated reach-and-grasp task. The results support the hypothesis that a robot can learn to interact purposefully with its environment through a developmental acquisition of sensory-motor coordination. Teleoperation can bootstrap the process by enabling the robot to observe its own sensory responses to actions that lead to specific outcomes within an environment. It is shown that a reach-and-grasp task, learned by an articulated robot through a small number of teleoperated trials, can be performed autonomously with success in the face of significant variations in the environment and perturbations of the goal. In particular, teleoperation of the robot to reach and grasp an object at nine different locations in its workspace enabled robust autonomous performance of the task anywhere within the workspace. Superpositioning was performed using the Verbs and Adverbs algorithm that was developed originally for the graphical animation of articulated characters. The work was performed on Robonaut, the NASA space-capable humanoid at Johnson Space Center, Houston, TX.

Progressive meshes [Hoppe, H., August 1996] provide an attractive mechanism for transmitting 3D geometry over networks. Progressive meshes (PM) transmit a coarse initial mesh and refinements that can be applied to the initial mesh. However, these techniques assume a reliable network protocol such as TCP/IP is used for data transmission. When transmitting 3D geometry for graphical applications such as concurrent virtual environments distributed over wide area networks with some nodes potentially using wireless networks, many losses will occur. TCP/IP performance degrades in the presence of packet loss and multicast communication mechanisms typically do not provide reliable communication. These applications motivate the issue of transmitting geometric data over unreliable networks. In this paper, we discuss several errors that are caused when reconstructing PM geometries after some packets have been lost. We modify the PM data structures to improve robustness during packet loss. We use these modifications to improve a hybrid transmission technique that uses TCP to transmit the base mesh and portion of the initial mesh and then use UDP to transmit the remainder of the mesh to improve transmission performance.

This paper describes the interface components for a system called Bettys Brain, an intelligent agent we have developed for studying the learning by teaching paradigm. Our previous studies have shown that students gain better understanding of domain knowledge when they prepare to teach others versus when they prepare to take an exam. This finding has motivated us to develop computer agents that students teach using concept map representations with a visual interface. Betty is intelligent not because she learns on her own, but because she can apply qualitative-reasoning techniques to answer questions that are directly related to what she has been taught through the concept map. We evaluate the agents interfaces in terms of how well they support learning activities, using examples of their use by fifth grade students in an extensive study that we performed in a Nashville public school. A critical analysis of the outcome of our studies has led us to propose the next generation interfaces in a multi-agent paradigm that should be more effective in promoting constructivist learning and self-regulation in the learning by teaching framework

Bettys Brain [1] is a learning-by-teaching environment where students "teach" Betty by constructing a concept map that models relations between domain concepts. The relations can be causal, hierarchical, and property links between the entities that represent the domain. The goal is for students to understand and then teach Betty about interdependence and balance among entities in a river ecosystem. As a part of the teaching process, students can query and quiz Betty to assess her understanding based on what she has been taught.Students can query Betty by asking her two types of questions: (i) "What happens to when increase/decrease?" and (ii) "Tell me about ". Betty answers questions by employing a qualitative reasoning mechanism, and explains her answers verbally and by using animation. Bettys explanations, and some feedback on the correctness of the answers should prompt the students to think more deeply about the domain processes, and motivate them to learn better before they teach her againStudents can request external feedback by asking Betty to take quizzes that are administered by a teacher agent. The teacher agent uses an overlay model to provide hints about concepts and links missing from the concept map. The hint levels start from general (e.g., suggesting that the student read a particular resource) to specific (e.g. indicating that a link is missing between two specific concepts. By seeing the quiz questions, students become aware of which concepts are important to model domain phenomena. The feedback from the teacher agent, points the students to understanding interrelationships among conceptsResults from our most recent study indicate that the query feature appeared to be effective in helping students develop an understanding of the interrelationships of living and non-living things in an ecosystem. The quiz feature is effective in helping students decide the important domain concepts and types of relationships to teach Betty. However, our observations of students during the study suggest that students using the quiz feature may have been overly focused on "getting the quiz questions correct" rather than "making sure that Betty (and themselves) understood the information.To help students focus more on learning, the next versions of Betty and teacher agent will be more interactive and metacognitive. In addition, the teacher will provide feedback that is related more to the global issues of balance and interdependence instead of individual links.At the demonstration, we will have a working version of our teachable agent system, Bettys Brain. We will demonstrate its user interfaces for the student to create and modify their concept maps, the query interface, and Bettys response to queries, and the teacher agent interface that responds when the students ask for help. In addition, the system also contains online resources that students can refer to when they are creating and updating their concept maps.

This paper describes the application of space time constraints to creating transitions between segments of human body motion.The motion transition generation uses a combination of spacetime constraints and inverse kinematic constraints to generate seamless and dynamically plausible transitions between motion segments.We use a fast recursive dynamics formulation which makes it possible to use spacetime constraints on systems with many degrees of freedom, such as human figures.The system uses an interpreter of a motion expression language to allow the user to manipulate motion data, break it into pieces, and reassemble it into new, more complex, motions.We have successfully used the system to create basis motions, cyclic data, and seamless motion transitions on a human body model with 44 degrees of freedom.