We propose and demonstrate the event-based visual microphone (EBVM), a passive electro-optical technique for remotely capturing audio signals using an event-based camera without any use of a conventional microphone. The event-based camera records local angular deformations of a surface induced by the sound propagation by observing the changes in the specular reflections at each pixel. By interpreting the timings of the specular incidences deduced from the event stream as signal level-crossings, we reconstruct the audio signal by imposing short-time Fourier sparsity conditions. The recovered audio signal is qualitatively comparable to or better than the prior art (intensity-based visual microphone), while simultaneously expanding the field of view by approximately 25 times and reducing data volume by three orders of magnitude. The proposed EBVM was tested on speech signal reconstruction as well as novel event-based acousto-optical passive ranging.

The goal of this project is to use projection augmented reality to add design studio learning models to a classroom for an introductory Media Computation computer science class. Students do classwork using an enhanced version of Pythy, a web IDE for Python and Jython, that captures students' work and displays it around the room. We leverage the Microsoft RoomAlive Toolkit to construct a room-scale augmented reality using pairs of projectors and depth cameras. The system “pins” students' work to the walls, where teachers and students can see and discuss the work. We hope that by seeing each other's work, the system will foster collaboration and support the creation of STEM learning experiences that encourage creativity, innovation, and help build strong peer learning environments.