We have built a virtual-reality simulator to design quadrotor robots that increase the independence of elderly persons living alone. The simulator lets us measure under what conditions a robot makes someone feel intrigued, or alarmed, or unconcerned, extending to nonhumanoids the studies of emotion that have been done for robots with facial expressions. This better model of how the virtual robot is perceived lets us agilely improve its appearance and behavior, containing the combinatorial explosion of parameters to study in costlier real-world experiments. The residence and the robot are rendered with Unity, and viewed with an Oculus Rift. Audio corresponding to flight maneuvers is resynthesized from recordings of an actual quadrotor. Flight dynamics using L1 adaptive control run in MATLAB and Simulink. Through menus operated by a Leap Motion hand tracker, the human subject commands the quadrotor to fly to various rooms (Unity waypoints), along precomputed paths that avoid collisions. The flight path is varied through a mathematical mapping to Laban Motion Factors (e.g., directness, sustainedness, lightness, and boundness) to provoke a range of responses. The subject's emotion is estimated from heart rate, skin conductance, and head tilt from the Rift's IMU (which is a quick metric for discomfort level).

Browsing large audio archives is challenging because of the limitations of human audition and attention. However, this task becomes easier with a suitable visualization of the audio signal, such as a spectrogram transformed to make unusual audio events salient. This transformation maximizes the mutual information between an isolated event's spectrogram and an estimate of how salient the event appears in its surrounding context. When such spectrograms are computed and displayed with fluid zooming over many temporal orders of magnitude, sparse events in long audio recordings can be detected more quickly and more easily. In particular, in a 1/10-real-time acoustic event detection task, subjects who were shown saliency-maximized rather than conventional spectrograms performed significantly better. Saliency maximization also improves the mutual information between the ground truth of nonbackground sounds and visual saliency, more than other common enhancements to visualization.

We propose a saliency-maximized audio spectrogram as a representation that lets human analysts quickly search for and detect events in audio recordings. By rendering target events as visually salient patterns, this representation minimizes the time and effort needed to examine a recording. In particular, we propose a transformation of a conventional spectrogram that maximizes the mutual information between the spectrograms of isolated target events and the estimated saliency of the overall visual representation. When subjects are shown spectrograms that are saliency-maximized, they perform significantly better in a 1/10-real-time acoustic event detection task.

Abstract Distributed scene graphs are important in virtual reality, both in collaborative virtual environments and in cluster rendering. Modern scalable visualization systems have high local throughput, but collaborative virtual environments (VEs) over a wide‐area network (WAN) share data at much lower rates. This complicates the use of one scene graph across the whole application. Myriad is an extension of the Syzygy VR toolkit in which individual scene graphs form a peer‐to‐peer network. Myriad connections filter scene graph updates and create flexible relationships between nodes of the scene graph. Myriad's sharing is fine‐grained: the properties of individual scene graph nodes to share are dynamically specified (in C++ or Python). Myriad permits transient inconsistency, relaxing resource requirements in collaborative VEs. A test application, WorldWideCrowd, demonstrates collaborative prototyping of a 300‐avatar crowd animation viewed on two PC‐cluster displays and edited on low‐powered laptops, desktops, and over a WAN. We have further used our framework to facilitate collaborative educational experiences and as a vehicle for undergraduates to experiment with shared virtual worlds. Copyright © 2006 John Wiley & Sons, Ltd.

Distributed scene graphs are important in virtual reality, both in collaborative virtual environments and in cluster rendering. In Myriad, individual scene graphs form a peer-to-peer network whose connections filter scene graph updates and create flexible relationships between scene graph nodes in the various peers. Modern scalable visualization systems often feature high intracluster throughput, but collaborative virtual environments (VEs) over a WAN share data at much lower rates, complicating the use of one scene graph system across the whole application. To avoid these difficulties, Myriad uses fine-grained sharing, whereby sharing properties of individual scene graph nodes can be dynamically changed from C++ and Python, and transient inconsistency, which relaxes resource requirements in collaborative VEs. A test application, WorldWideCrowd, implements these methods to demonstrate collaborative prototyping of a 300-avatar crowd animation viewed on two PC-cluster displays and edited on low-powered laptops, desktops, and even over a WAN.

We describe a large audio-visual speech corpus recorded in a car environment, as well as the equipment and procedures used to build this corpus. Data are collected through a multi-sensory array consisting of eight microphones on the sun visor and four video cameras on the dashboard. The script for the corpus consists of four categories: isolated digits, isolated letters, phone numbers, and sentences, all in English. Speakers from various language backgrounds are included, 50 male and 50 female. In order to vary the signal-to-noise ratio, each script has five different noise conditions: idling, driving at 35 mph with windows open and closed, and driving at 55 mph with windows open and closed. The corpus is available through

The Syzygy software library consists of tools for programming VR applications on PC clusters. Since the PC cluster environment presents application development constraints, it is impossible to simultaneously optimize for efficiency, flexibility, and portability between the single-computer and cluster cases. Consequently Syzygy includes two application frameworks: a distributed scene graph framework for rendering a single application's graphics database on multiple rendering clients, and a master/slave framework for applications with multiple synchronized instances. Syzygy includes a simple distributed OS and supports networked input devices, sound renderers, and graphics renderers, all built on a robust networking layer.