Audio is presented ahead of video in some videoconferencing systems since audio requires less time to process. Audio could be delayed to synchronize with video to achieve lip synchronization; however, the overall audio latency might then become unacceptable. We built a videoconferencing system to achieve lip synchronization with minimal perceived audio latency. Instead of adding a fixed audio delay, our system time-stretches the audio at the beginning of each utterance until the audio is synchronized with the video. We conducted user studies and found that (1) audio could lead video by roughly 50 msec and still be perceived as synchronized; (2) audio could lead video by 300 msec and still be perceived as synchronized if the audio was time-stretched to synchronization within a short period; and (3) our algorithm appears to strike a favorable balance between minimizing audio latency and supporting lip synchronization.

Effective classroom teaching often requires an instructor to be acutely aware of every student. The instructor must rapidly look from student to student to catch fleeting gestures or facial expressions. To facilitate the tracking of communicative actions in a remote classroom, we built a multiparty videoconferencing system that automatically determine whether students are speaking, making gestures, or moving in their seats. These activity indicators are displayed over the video such that the instructor can see into the recent past. The activity indicators are also grouped into a visualization of the classroom interaction dynamics, thereby providing a measure of the pulse of the classroom.We conducted a user study where teachers used our system in a simulated class. The teachers found that the activity indicators to be a useful teaching aid during class; however, the indicators are most useful as a record of the class. In a student survey, we found that if audio, video, or activity indicators must be recorded, students overwhelmingly prefer activity indicators since the indicators mask the content of the communication and thus are less intrusive to the students' privacy.

Eye contact is a natural and often essential element in the language of visual communication. Unfortunately, perceiving eye contact is difficult in most video-conferencing systems and hence limits their effectiveness. We conducted experiments to determine how accurately people perceive eye contact. We discovered that the sensitivity to eye contact is asymmetric, in that we are an order of magnitude less sensitive to eye contact when people look below our eyes than when they look to the left, right, or above our eyes. Additional experiments support a theory that people are prone to perceive eye contact, that is, we will think that someone is making eye contact with us unless we are certain that the person is not looking into our eyes. These experimental results suggest parameters for the design of videoconferencing systems. As a demonstration, we were able to construct from commodity components a simple dyadic videoconferencing prototype that supports eye contact

Multiparty videoconferencing with even a small number of people is often infeasible due to the high network bandwidth required. Bandwidth can be significantly reduced if most of the advantages of using full-motion video can be achieved with low-frame-rate video; unfortunately, the impact of low-frame-rate video on communication is relatively unexplored. We implemented a multiparty videoconferencing system that supports full-motion video, low-frame-rate video where the video is updated only once every few seconds, and a hybrid scheme where full-motion video is transmitted when the system detects that a user is making a gesture and low-frame-rate video is transmitted at all other times. We studied people using our system for small-group discussions and found that low-frame-rate video limited people's ability to request to speak or judge when to stop speaking. The hybrid scheme, conversely, was as effective as full-motion video for floor control, resulting in a similar number of speaker changes, while using only ten percent of the bandwidth.

In this paper, we propose an interactive DTV design that converts non-interactive broadcast DTV streams into interactive ones for multiple simultaneous viewers. To enable viewing interactivity, we show that it is critical to organize data intelligently for improving IO resolution, reducing disk latency and minimizing storage cost. We propose three data placement schemes that offer different tradeoffs between IO resolution, disk latency and storage. By employing different schemes under different workload scenarios, an intelligent system can minimize memory use and hence the system cost.

We built a videoconference system called the Virtual Auditorium to support dialog-based distance learning. The instructor can see dozens of students on a tiled wall-sized display and establish eye contact with any student. Telephone-quality audio and television-quality video can be streamed using commodity codecs such as wavelet and MPEG-4. Support for stream migration allows a seamless user interface to span the multiple computers driving the display wall..We performed user studies on the auditorium parameters. We found that the optimal display wall size must balance two contradictory requirements: subjects prefer larger videos for seeing facial expressions and smaller videos for seeing everyone without head movement. Ideally, each video should have a field of view that spans 14 degrees, which corresponds to a slightly larger than life-size image. At the very least, each video should have a field of view of 6 degrees. We found that a video window should be less than 2.7 degrees horizontally and 9 degrees vertically from the camera in order to maintain the appearance of eye contact for the remote viewer. In addition, we describe a previously unreported gaze phenomenon: a person's expectation determines his perception of eye contact under ambiguous conditions.