In this paper, we examine the implementation and usage scenarios for WeatherPlay, a web site that collects weather and travel TV clips, microblog entries, amateur videos, and outdoor data. In order to present a comprehensive picture of people's outdoor experiences, WeatherPlay geolocates this media and places it in both a map and video gallery context.

ContextController is a content augmentation system that enables the real time collation and presentation of contextual information for linear TV broadcasts. The goal of this project is to enable viewers to supplement their TV viewing experience with externally sourced and summarized knowledge. Through this viewers with varied background knowledge are provided the ability to discover new contextual information, which aids in the improved understanding of the original content. Although this system can be used with a variety of content types this work focuses on News related broadcasts.

In this paper, we propose CoCam, a framework for mobile phones that enables uncoordinated real-time image and video collaboration between different users sharing the same context, in the same physical location. CoCam addresses the complexity and difficulty introduced when users wish to collaborate, create, and share media contents at the same time. CoCam is based on a middleware that creates a self-organizing ad-hoc network within the context of a shared event in a proximal physical space (a common scene). This middleware automatically handles the context detection, as well as the network configuration and peer discovery. It also enables real-time content sharing while reducing the burden of complicated settings and configurations by the users. As a result of operation tests and user study with the prototype implementation, we have verified that CoCam is feasible and has the potential to enrich the users' experience when sharing contents in an event scenario.

We describe a mobile application for sharing user-authored photo content in realtime called CoCam. CoCam is a collaborative content sharing framework based on opportunistic P2P proximal networking. CoCam users who are located in the same physical space can automatically share the photos they create as well as receive photos from other users around them. Since CoCam is based on an opportunistic P2P network middleware, users are not required to know each other in advance. It is also not necessary for them to agree on the same service provider nor coordinate the network configuration, infrastructure and security settings. This middleware automatically discovers other peers and handles the organization of ad-hoc network connections. With CoCam, we demonstrate that users are able to share and enjoy shared photos and video streams without the effort of manual setup and cost associated with the 3G/4G network.

We present an 8-dimensional (8D) display that allows glasses-free viewing of 3D imagery, whist capturing and reacting to incident environmental and user controlled light sources. We demonstrate two interactive possibilities enabled by our lens-array-based hardware prototype, and realtime GPU-accelerated software pipeline. Additionally, we describe a path to deploying such displays in the future, using current Sensor-in-Pixel (SIP) LCD panels, which physically collocate sensing and display elements.

ShAir is a middleware infrastructure that allows mobile applications to share resources of their devices (e.g., data, storage, connectivity, computation) in a transparent way. The goals of ShAir are: (i) abstracting the creation and maintenance of opportunistic delay-tolerant peer-to-peer networks; (ii) being decoupled from the actual hardware and network platform; (iii) extensibility in terms of supported hardware, protocols, and on the type of resources that can be shared; (iv) being capable of self-adapting at run-time; (v) enabling the development of applications that are easier to design, test, and simulate. In this paper we discuss the design, extensibility, and maintainability of the ShAir middleware, and how to use it as a platform for collaborative resource-sharing applications. Finally we show our experience in designing and testing a file-sharing application.

Recently, several camera designs have been proposed for either making defocus blur invariant to scene depth or making motion blur invariant to object motion. The benefit of such invariant capture is that no depth or motion estimation is required to remove the resultant spatially uniform blur. So far, the techniques have been studied separately for defocus and motion blur, and object motion has been assumed 1D (e.g., horizontal). This article explores a more general capture method that makes both defocus blur and motion blur nearly invariant to scene depth and in-plane 2D object motion. We formulate the problem as capturing a time-varying light field through a time-varying light field modulator at the lens aperture, and perform 5D (4D light field + 1D time) analysis of all the existing computational cameras for defocus/motion-only deblurring and their hybrids. This leads to a surprising conclusion that focus sweep, previously known as a depth-invariant capture method that moves the plane of focus through a range of scene depth during exposure, is near-optimal both in terms of depth and 2D motion invariance and in terms of high-frequency preservation for certain combinations of depth and motion ranges. Using our prototype camera, we demonstrate joint defocus and motion deblurring for moving scenes with depth variation.

MatchMaker is an automated collaborative filtering system that recommends friends to people on Facebook by analyzing and matching people's online profile with the profiles of TV characters. The goal of MatchMaker is to produce friend recommendations with rich contextual information through collaborative filtering in the existing social network. Using relationships in TV programs as a parallel comparison matrix, MatchMaker projects these relationships into reality to help people find friends whose personality and characteristics have been voted to suit them well by their social network. MatchMaker also encourages more TV content viewing by using the social network context and connections to provoke people's curiosity of TV characters whom they have been matched with in their social network.

CommenTV enables users to share their comments in parallel with time-based content. CommenTV utilizes existing web technologies to enhance current TV or video player solutions with a new type of comment system. CommenTV is able to take and display texts, images, and related videos as social comments. In this way, the CommenTV users can experience audiovisual content that blends with their social networks by sharing comments. Ultimately, CommenTV can be a social bookmarking system for audiovisual content that informs content creators about how viewers perceive their content and lets viewers participate in creating a more social content.

WorldTV is a proposed TV app for the discovery of user-generated video from around the world. We describe a two-screen solution (a TV app and accompanying mobile app to control what appears on the television screen) would let viewers browse and play geotagged videos plotted on a 3D model of the earth. The content would include video from viewers' own social networks as well as user-generated news and event video filtered according to viewers' interests, preferences, and social connections. In a world in which user-generated online video content is being uploaded and consumed at an exponential rate, the idea of using a globe-based navigation metaphor has powerful appeal. It is instantly approachable and easy to use, especially for people with global networks of friends, family, and colleagues living in other countries. We also argue that the widespread availability of geotagged video provides an additional impetus for using a geo-focused UI over more traditional browsing and discovery mechanisms.

ATTN-SPAN is a digital content summarization and contextualization platform designed to compile video footage into personalized episodes. Viewers are able to associate opinions and externally sourced static content to moments within the video, creating a far richer viewing experience and supporting a community knowledge base around civic information. The pilot project utilizes C-SPAN footage to create daily episodes for each user of the system based on his or her political interests.

Social television, in essence, is a way to share and experience media with others. Rather than attempt to bridge physical distance by creating new social experiences, we have built a system called TakeoverTV that augments canonical social interactions by addressing three key questions: 1. How can we better support social gatherings around media in public and private places? 2. How can we enable conversation across our devices rather than forcing us to communicate through them? 3. How can we help people come to agreements about what media they want to watch together? Primarily aimed at public spaces like bars, TakeoverTV is a system that lets local users influence and interact with the movies and television programs shown on public displays. Our system collects the media preferences of all people physically present at a given location, lets users start a vote among all members of the space, and allows everyone to participate using tangible physical objects like tables in addition to smart devices. Over time, public establishments can evolve complex identities based on the preferences and votes of their patrons while generating valuable analytic data for patrons, establishment owners, content-providers, and advertisers alike.

If everyone says time is relative, why is it still so rigidly defined? There have been many attempts to address the issue of coordinating schedules, but each of these attempts runs into an issue of rigidity: in order to negotiate an event, a specific time must be designated in advance. This model is inherently poor at accommodating life's unpredictability. Kairoscope looks at time from a human perspective, focusing on time as made up of a series of events, rather than simply a series of events in time. By removing our reliance on a fixed time system, events are coordinated socially and on the fly, without worrying about precision. This paper explores the creation of Kairoscope, rooted in ideas of time perception and aiming to reduce time-related stress, optimize time usage, and increase social interaction. The result is a socially-coordinated, constantly adapting, and highly malleable guide through time.

Qooqle is a mobile-based system that frees people from the confines of search boxes by integrating digital information directly into their everyday activities. By allowing casual expressions through speech and gesture with mobile phones, Qooqle's multi-modal user interface enables people to engage with one another while accessing the information world more naturally. Through the analysis of social media data, Qooqle retrieves relevant information to the users based upon the physical world context they are situated in. Combining mobile, cloud computing, and social media, Qooqle draws people closer to computing and makes computers less visible. The Qooqle system uses Speech Recognition and Gesture Recognition to achieve its natural user interface. It uses Natural Language Processing, Machine Learning, and other Data Mining techniques to search through people's speech and gestures in the physical world as well as their online social media data for relevant information retrieval.

Most organizations have a wealth of knowledge about themselves available online, but little for a visitor to interact with on-site. At the MIT Media Lab, we have designed and deployed a novel intelligent signage system, the Glass Infrastructure (GI) that enables small groups of users to physically interact with this data and to discover the latent connections between people, projects, and ideas. The displays are built on an adaptive, unsupervised model of the organization developed using dimensionality reduction and common sense knowledge which automatically classifies and organizes the information.The GI is currently in daily use at the lab. We discuss the AI models development, the integration of AI into an HCI interface, and the use of the GI during the labs peak visitor periods. We show that the GI is used repeatedly by lab visitors and provides a window into the workings of the organization.

Video content has become available on an increasingly diverse set of devices and from an ever growing number of sources, creating a vast amount of choice for viewers. At the same time, the varying methods of viewing, interacting with, and sharing content have diverged. This paper introduces neXtream, a new approach to delivering video by integrating multiple devices, content sources, and social networks. This concept is developed following research in social television and converged applications, providing both personalization features and social interaction. NeXtream delivers video by dynamically generating streams of video customized to a viewer, while facilitating a common dialog between users around the content, creating both a user- and community-centric viewing experience. NeXtream integrates smartphones, PCs, and TVs to deliver video content to viewers. The paper presents the system concept, theory, and architecture, and describes the developed prototype.

We describe a practical vision of ubiquitous computing with tangible interfaces, that orbits around an individual and is mediated by his or her personal consumer electronic devices. We illustrate this with a software and hardware platform for creating a personal area network of information accessories, connecting people physically with network-accessible information.

While IPTV architecture and user interfaces have received a lot of interest in recent years much less focus has been given to the issues of channel surfing in the IPTV era. IPTV is rapidly transitioning from a Telco solution into a generic service over any broadband access network. Traffic issues that relate to channel change are moving to the mainstream and are the talk of a number of standards organizations. This paper proposes a new approach to channel change. It uses peering and viewing statistics to reduce channel change delays and improve the user quality of experience.

We transform an LCD into a display that supports both 2D multi-touch and unencumbered 3D gestures. Our BiDirectional (BiDi) screen, capable of both image capture and display, is inspired by emerging LCDs that use embedded optical sensors to detect multiple points of contact. Our key contribution is to exploit the spatial light modulation capability of LCDs to allow lensless imaging without interfering with display functionality. We switch between a display mode showing traditional graphics and a capture mode in which the backlight is disabled and the LCD displays a pinhole array or an equivalent tiled-broadband code. A large-format image sensor is placed slightly behind the liquid crystal layer. Together, the image sensor and LCD form a mask-based light field camera, capturing an array of images equivalent to that produced by a camera array spanning the display surface. The recovered multi-view orthographic imagery is used to passively estimate the depth of scene points. Two motivating applications are described: a hybrid touch plus gesture interaction and a light-gun mode for interacting with external light-emitting widgets. We show a working prototype that simulates the image sensor with a camera and diffuser, allowing interaction up to 50 cm in front of a modified 20.1 inch LCD.