We present a prototype virtual reality user interface for robot teleoperation that supports high-level specification of 3D object positions and orientations in remote assembly tasks. Users interact with virtual replicas of task objects. They asynchronously assign multiple goals in the form of 6DoF destination poses without needing to be familiar with specific robots and their capabilities, and manage and monitor the execution of these goals. The user interface employs two different spatiotemporal visualizations for assigned goals: one represents all goals within the user’s workspace (Aggregated View), while the other depicts each goal within a separate world in miniature (Timeline View). We conducted a user study of the interface without the robot system to compare how these visualizations affect user efficiency and task load. The results show that while the Aggregated View helped the participants finish the task faster, the participants preferred the Timeline View.

Augmented reality (AR) has been used to guide users in multi-step tasks, providing information about the current step (cueing) or future steps (precueing). However, existing work exploring cueing and precueing a series of rigid-body transformations requiring rotation has only examined one-degree-of-freedom (DoF) rotations alone or in conjunction with 3DoF translations. In contrast, we address sequential tasks involving 3DoF rotations and 3DoF translations. We built a testbed to compare two types of visualizations for cueing and precueing steps. In each step, a user picks up an object, rotates it in 3D while translating it in 3D, and deposits it in a target 6DoF pose. Action-based visualizations show the actions needed to carry out a step and goal-based visualizations show the desired end state of a step. We conducted a user study to evaluate these visualizations and the efficacy of precueing. Participants performed better with goal-based visualizations than with action-based visualizations, and most effectively with goal-based visualizations aligned with the Euler axis. However, only a few of our participants benefited from precues, most likely because of the cognitive load of 3D rotations.

Work on cueing performance in AR and VR has focused on sequential tasks in which each step must be completed in order before the user can proceed to the next. However, for unordered tasks such as putting books back on a library shelf, the user may be able to perform multiple steps concurrently without needing to follow a specific order. In such situations, giving the user multiple cues for potentially concurrent steps may improve performance time. To investigate this, we built a bimanual VR testbed in which the user needs to move objects to designated destinations, guided by different numbers of cues. The user can decide the order to perform the cued steps and, in some conditions, can affect which cues are shown.In a formal user study, we found that in most conditions, participants perform fastest with three cues. Dynamically updating the set of displayed cues based on hand proximity improves performance, and updating the set based on eye gaze improves performance even more. Finally, for both the hand-proximity and eye-gaze mechanisms, performance can be further improved by locking the cues for objects predicted to be moved next based on hand distance.

Augmented reality has been used to improve sequential-task performance by cueing information about a current task step and precueing information about future steps. Existing work has shown the benefits of precueing movement (translation) information. However, rotation is also a major component in many real-life tasks, such as turning knobs to adjust parameters on a console. We developed an AR testbed to investigate whether and how much precued rotation information can improve user performance. We consider two unimanual tasks: one requires a user to make sequential rotations of a single object, and the other requires the user to move their hand between multiple objects to rotate them in sequence.

When a user is performing a manual task, AR or VR can provide information about the current subtask (cueing) and upcoming subtasks (precueing) that makes them easier and faster to complete. Previous research on cueing and precueing in AR and VR has focused on path-following tasks requiring simple actions at each of a series of locations, such as pushing a button or just visiting. We consider a more complex task, whose subtasks involve moving to and picking up an item, moving that item to a designated place while rotating it to a specific angle, and depositing it. We conducted two user studies to examine how people accomplish this task while wearing an AR headset, guided by different visualizations that cue and precue movement and rotation. Participants performed best when given movement information for two successive subtasks and rotation information for a single subtask. In addition, participants performed best when the rotation visualization was split across the manipulated object and its destination.

Work on VR and AR task interaction and visualization paradigms has typically focused on providing information about the current step (a cue) immediately before or during its performance. Some research has also shown benefits to simultaneously providing information about the next step (a precue). We explore whether it would be possible to improve efficiency by precueing information about multiple upcoming steps before completing the current step. To accomplish this, we developed a remote VR user study comparing task completion time and subjective metrics for different levels and styles of precueing in a path-following task. Our visualizations vary the precueing level (number of steps precued in advance) and style (whether the path to a target is communicated through a line to the target, and whether the place of a target is communicated through graphics at the target). Participants in our study performed best when given two to three precues for visualizations using lines to show the path to targets. However, performance degraded when four precues were used. On the other hand, participants performed best with only one precue for visualizations without lines, showing only the places of targets, and performance degraded when a second precue was given. In addition, participants performed better using visualizations with lines than ones without lines.

By necessity as well as desire, people move and act in space. Our feet carry us from place to place along paths, just as our minds carry us from idea to idea along paths. The same brain structures that underlie places and paths underlie ideas and relations. We talk that way, too; I just did. Our hands place, raise, lower, put together, take apart, turn over push, pull, and throw objects. Our minds and our talk do the same with thoughts. Our minds allow us to take different perspectives on the paths in space, on the relations among ideas, on actions on objects. That turns out to be the key to creativity.

We demonstrate an interaction technique that allows a user to point at a world-in-miniature representation of a city-scale virtual environment and perform efficient and precise teleportation by pre-orienting an avatar. A preview of the post-teleport view of the full-scale virtual environment updates interactively as the user adjusts the position, yaw, and pitch of the avatar's head with a pair of 6DoF-tracked controllers. We describe design decisions and contrast with alternative approaches to virtual travel.

Assembly or repair tasks often require objects to be held in specific orientations to view or fit together. Research has addressed the use of AR to assist in these tasks, delivered as registered overlaid graphics on stereoscopic head-worn displays. In contrast, we are interested in using monoscopic head-worn displays, such as Google Glass. To accommodate their small monoscopic field of view, off center from the user's line of sight, we are exploring alternatives to registered overlays. We describe four interactive rotation guidance visualizations for tracked objects intended for these displays.

In many complex tasks, a remote subject-matter expert may need to assist a local user to guide actions on objects in the local user's environment. However, effective spatial referencing and action demonstration in a remote physical environment can be challenging. We introduce two approaches that use Virtual Reality (VR) or Augmented Reality (AR) for the remote expert, and AR for the local user, each wearing a stereo head-worn display. Both approaches allow the expert to create and manipulate virtual replicas of physical objects in the local environment to refer to parts of those physical objects and to indicate actions on them. This can be especially useful for parts that are occluded or difficult to access. In one approach, the expert points in 3D to portions of virtual replicas to annotate them. In another approach, the expert demonstrates actions in 3D by manipulating virtual replicas, supported by constraints and annotations. We performed a user study of a 6DOF alignment task, a key operation in many physical task domains, comparing both approaches to an approach in which the expert uses a 2D tablet-based drawing system similar to ones developed for prior work on remote assistance. The study showed the 3D demonstration approach to be faster than the others. In addition, the 3D pointing approach was faster than the 2D tablet in the case of a highly trained expert.

Many tasks in real or virtual environments require users to view a target object or location from one of a set of strategic viewpoints to see it in context, avoid occlusions, or view it at an appropriate angle or distance. We introduce ParaFrustum, a geometric construct that represents this set of strategic viewpoints and viewing directions. ParaFrustum is inspired by the look-from and look-at points of a computer graphics camera specification, which precisely delineate a location for the camera and a direction in which it looks. We generalize this approach by defining a ParaFrustum in terms of a look-from volume and a look-at volume, which establish constraints on a range of acceptable locations for the user's eyes and a range of acceptable angles in which the user's head can be oriented. Providing tolerance in the allowable viewing positions and directions avoids burdening the user with the need to assume a tightly constrained 6DoF pose when it is not required by the task. We describe two visualization techniques for virtual or augmented reality that guide a user to assume one of the poses defined by a ParaFrustum, and present the results of a user study measuring the performance of these techniques. The study shows that the constraints of a tightly constrained ParaFrustum (e.g., approximating a conventional camera frustum) require significantly more time to satisfy than those of a loosely constrained one. The study also reveals interesting differences in participant trajectories in response to the two techniques.

Magic-lens style augmented reality applications allow users to control camera pose easily by manipulating a portable hand-held device and provide immediate visual feedback. However, strategic vantage points must often be revisited repeatedly, adding time and error and taxing memory. We describe a new approach that allows users to take snapshots of augmented scenes that can be virtually revisited at later times. The system stores still images of scenes along with camera poses, so that augmentations remain dynamic and interactive. Users can manipulate virtual objects while viewing snapshots, instead of moving to real-world views. We present a study comparing performance in snapshot and live mode conditions in a task in which a virtual object must be aligned with two pairs of physical objects. Proper alignment requires sequentially visiting two viewpoints. Participants completed the alignment task significantly faster and more accurately using snapshots than when using the live mode. Moreover, participants preferred manipulating virtual objects using snapshots to the live mode.

Designing effective instructions for everyday products is challenging. One reason is that designers lack a set of design principles for producing visually comprehensible and accessible instructions. We describe an approach for identifying such design principles through experiments investigating the production, preference, and comprehension of assembly instructions for furniture. We instantiate these principles into an algorithm that automatically generates assembly instructions. Finally, we perform a user study comparing our computer-generated instructions to factory-provided and highly rated hand-designed instructions. Our results indicate that the computer-generated instructions informed by our cognitive design principles significantly reduce assembly time an average of 35% and error by 50%. Details of the experimental methodology and the implementation of the automated system are described.

We present design principles for creating effective assembly instructions and a system that is based on these principles. The principles are drawn from cognitive psychology research which investigated people's conceptual models of assembly and effective methods to visually communicate assembly information. Our system is inspired by earlier work in robotics on assembly planning and in visualization on automated presentation design. Although other systems have considered presentation and planning independently, we believe it is necessary to address the two problems simultaneously in order to create effective assembly instructions. We describe the algorithmic techniques used to produce assembly instructions given object geometry, orientation, and optional grouping and ordering constraints on the object's parts. Our results demonstrate that it is possible to produce aesthetically pleasing and easy to follow instructions for many everyday objects.

This paper reviews research in geographical cognition that provides part of the theoretical foundation of geographical information science. Freestanding research streams in cognitive science, behavioural geography, and cartography converged in the last decade or so with work on theoretical foundations for geographical information systems to produce a coherent research community that advances geographical information science, geographical information systems, and the contributing fields and disciplines. Then, we review three high-priority research areas that are the topics for research initiatives within the NCGIA's Project Varenius. Other topics consider but ranked less important at this time are also reviewed.

Rapid growth in 3-D rendering technologies has deluged us with glitzy graphical representations.In what contexts do people find 3-D graphs of 2-D data both attractive and useful?We examine students' preferences for graphical display formats under several use scenarios.Line graphs were preferred more for conveying trends than details, and more for promoting memorability than for immediate use; bar graphs showed the opposite pattern.3-D graphs were preferred more for depicting details than trends, more for memorability than immediate use, and more for showing others than oneself.The reverse held for 2-D graphs.

Spatial language is widely used, both literally to describe space and figuratively to express a broad range of ideas. This article reviews two projects studying the nature of mental representations of space induced entirely by language. The first project investigates perspective in descriptions of large-scale (e.g., convention center, town) space. People typically describe environments using a route or survey perspective, or a mixture of both. Route perspectives take a view from within the environment and describe the locations of landmarks with respect to a moving observer in terms of the observer's left, right, front, and back. Survey perspectives take a view from above the environment and describe locations of landmarks with respect to each other in terms of north, south, east, and west. Features of the environment, such as having a single or multiple path, affect choice of perspective. In comprehension, readers seem to form the same perspective-free mental representation irrespective of description perspective. They respond with equal speed and accuracy to inference questions from either perspective regardless of read perspective. The second project investigates mental representations of the objects located immediately around the body. Readers seem to form mental spatial frameworks, extensions of the three body axes, associating objects to the frameworks. The accessibility of the three axes depends on characteristics of the body, characteristics of the perceptual world, and posture of the observer. For example, for an upright observer, times to access objects along the head/feet axis are fastest because it is an asymmetric axis of the body and is correlated with the only asymmetric axis of the world, that created by gravity. Times to access objects along the front/back axis are next fastest as it is an asymmetric axis of the body, and times to the left/right axis are slowest as it is an axis with few asymmetries. Evidence for the spatial framework hypothesis was obtained in a variety of situations, varying posture, perspective, number of observers, and cause of reorientations. © 1994 John Wiley & Sons, Inc.