Blogs, wikis, tagging, podcasts, and social networking websites such as MySpace, Facebook, Flickr and YouTube have radically changed user interactions on the World Wide Web from a static, one-way, consumption model to a dynamic, multi-way, participation model. Broad user power and flexibility have changed how people engage in and experience their interconnections, interests, and collaborations. Online social interactions will evolve in the next decade to address the growing needs of its user community and make entries into many aspects of our lives. This evolution may very well be among the most exciting ones of our times where the individual and collective power of people to contribute and share content, experiences, ideas, expertise etc. may be even more enhanced than it is today. The enhancement may be shaped through a better understanding of user needs and behavior and it may be enabled through the seamless convergence of multi-modal technologies, new applications, new domains, data mining and better navigational and search capabilities. Some of these changes will also permeate into the workplace and change the way we work. This panel will discuss how online social interactions may evolve in the next decade and what impact it may have on diverse dimensions in our world.

We describe a framework for automatically selecting a summary set of photographs from a large collection of geo-referenced photos. The summary algorithm is based on spatial patterns in photo sets, but can be expanded to support social, temporal, as well as textual-topical factors of the photo set. The summary set can be biased by the user, the content of the user's query, and the context in which the query is made. An initial evaluation on a set of geo-referenced photos shows that our algorithm performs well, producing results that are highly rated by users.

This article addresses the problem of processing the annotations of preexisting video productions to enable reuse and repurposing of metadata. We introduce the concept of automatic content-based editing of preexisting semantic home video metadata. We propose a formal representation and implementation techniques for reusing and repurposing semantic video metadata in concordance with the actual video editing operations. A novel representation for metadata editing is proposed and an implementation framework for editing the metadata in accordance with the video editing operations is demonstrated. Conflict resolution and regularization operations are defined and implemented in the context of the video metadata editing operations.

Though cameraphones are rapidly becoming the dominant platform for consumer digital photography, users still face difficulties in transferring, managing, and sharing photos captured with cameraphones. The Mobile Media Metadata 2 (MMM2) system removes the difficulty in transferring photos from the device by providing an automatic upload capability and uses metadata about the context in which a photo was captured to simplify photo management and streamline the sharing process. In our MMM2 system, we have leveraged collaborative filtering techniques to infer the likely sharing recipients for photos based on contextual metadata, which allows the system to accurately guess likely share recipients for a photo and present them to the photographer at the time of capture.

The number of people using cameraphones is growing by tens of millions every month. Yet the majority of cameraphone users have difficulty transferring photos off their phone and sharing them with others. PhotoRouter is a software application for cameraphones that makes the photo sharing process destination-centric by allowing users to focus on who the photo should go to, not how it needs to get there. Attempting to produce an application which meets user needs better than current, technology-centric cameraphone photo sharing applications, we designed PhotoRouter. In this paper we describe PhotoRouter's user interface innovations that we will show in our technical demonstration.

As cameraphones become the dominant platform for consumer multimedia capture worldwide, multimedia researchers are faced both with the challenge of how to help users manage the billions of photographs they are collectively producing and the opportunity to leverage cameraphones' ability to automatically capture temporal, spatial, and social contextual metadata to help manage consumer multimedia content. In our Mobile Media Metadata 2 (MMM2) prototype, we apply collaborative filtering techniques to automatically gathered contextual metadata to infer the likely sharing recipients for photos captured on cameraphones. We show that while current cameraphone sharing interfaces are fraught with difficulty, it is possible to use a context-aware approach to make the sharing of cameraphone photos simpler and more satisfying for users. Based on our analysis of the relative contributions of different cameraphone sensors to predicting the likely recipients for photos, we discover for our user population that the temporal context of photo capture proved highly predictive of photo sharing behavior.

In this paper, we focus on the use of context-aware, collaborative filtering, machine-learning techniques that leverage automatically sensed and inferred contextual metadata together with computer vision analysis of image content to make accurate predictions about the human subjects depicted in cameraphone photos. We apply Sparse-Factor Analysis (SFA) to both the contextual metadata gathered in the MMM2 system and the results of PCA (Principal Components Analysis) of the photo content to achieve a 60% face recognition accuracy of people depicted in our cameraphone photos, which is 40% better than media analysis alone. In short, we use context-aware media analysis to solve the face recognition problem for cameraphone photos.

The number of digital photographs is growing beyond the abilities of individuals to easily manage and understand their own photo collections. Photo LOI (Level of Interest) is a technique that filters, aggregates, and visualizes photographs taken by multiple users who shared temporal, spatial, and/or social context at the point of photo capture. Photo LOI enables groups of photographers to see and manipulate visualizations of their photographic activities over time and social space in order to help cluster and select photos, and enables researchers to study contextual patterns in the phototaking habits of different users and groups of users. In this paper we give a brief overview of Photo LOI's features and describe some of its applications.

Advanced systems for finding, using, sharing, and remixing video require high-level representations of video content. A number of researchers have taken top-down, analytic approaches to the specification of representation structures for video. The resulting schemes, while showing the potential of high-level representations for aiding the retrieval and resequencing of video, have generally proved too complex for mainstream use. In this paper, we propose a bottom-up, emergent approach to developing video representation structures by examining retrieval requests and annotations made by a community of video remixers. Our initial research has found a useful degree of convergence between user-generated indexing terms and query terms, with the salient exception of descriptions of characters' corporeal characteristics.

As human-computer interaction becomes more closely modeled on human-human interaction, new techniques and strategies for human-computer interaction are required. In response to the inevitable shortcomings of recognition technologies, researchers have studied mediation: interaction techniques by which users can resolve system ambiguity and error. In this paper we approach the human-computer dialogue from the other side, examining system-initiated direction and mediation of human action. We conducted contextual interviews with a variety of experts in fields involving human-human direction, including a film director, photographer, golf instructor, and 911 operator. Informed by these interviews and a review of prior work, we present strategies for directing physical human action and an associated design space for systems that perform such direction. We illustrate these concepts with excerpts from our interviews and with our implemented system for automated media capture or Active Capture, in which an unaided computer system uses techniques identified in our design space to act as a photographer, film director, and cinematographer.

Camera phones offer a new platform for digital imaging that integrates capture, programmable processing, networking, and rich user interaction capabilities. This platform can support software that leverages the spatio-temporal context and social community of image capture and (re)use to infer media content. Our mobile media metadata prototype (MMM) uses a small custom client on the camera phone and its XHTML browser along with a metadata server to enable annotation at the time of image capture, leverage contextual metadata and networked metadata resources, and use iterative metadata refinement on the mobile imaging device.

The domains of folk computing applications touch on areas of interest to people around the world but are of pressing need to those in the developing world who often lack access to basic services and rights: especially health care, education, nutrition, and protection of human rights. In this paper, we describe how a new paradigm for media capture, called active capture, and toolkit support for creating applications of this type, work toward supporting the development of multimedia applications and interfaces for folk computing

No abstract available.

In the 2003, more camera phones were sold worldwide than digital cameras. With this new platform, we can leverage regularities in the spatio-temporal context and social community of media capture and use to infer media content. We created and deployed a "Mobile Media Metadata" (MMM) prototype on Nokia 3650 camera phones with 55 users that uses "context-to-content" inferencing, a shared metadata ontology, and user interaction at the point of capture to effectively infer media content annotations, specifically the semantic description of the location of the subject of users' photos.

The recent popularity of mobile camera phones allows for new opportunities to gather important metadata at the point of capture. This paper describes a method for generating metadata for photos using spatial, temporal, and social context. We describe a system we implemented for inferring location information for pictures taken with camera phones and its performance evaluation. We propose that leveraging contextual metadata at the point of capture can address the problems of the semantic and sensory gaps. In particular, combining and sharing spatial, temporal, and social contextual metadata from a given user and across users allows us to make inferences about media content.

The amount of personal digital media is increasing, and managing it has become a pressing problem. Effective management of media content is not possible without content-related metadata. In this paper we describe a content metadata creation process for images taken with a mobile phone. The design goals were to automate the creation of image content metadata by leveraging automatically available contextual metadata on the mobile phone, to use similarity processing algorithms for reusing shared metadata and images on a remote server, and to interact with the mobile phone user during image capture to confirm and augment the system supplied metadata. We built a prototype system to evaluate the designed metadata creation process. The main findings were that the creation process could be implemented with current technology and it facilitated the creation of semantic metadata at the time of image capture.

While the devices for media capture have advanced from mechanical to computational since the invention of photography and motion pictures in the 19th century, their underlying user interaction paradigms have remained largely unchanged. Current interaction techniques for media capture do not leverage computation to solve key problems: the skill required to capture high quality media assets; the effort required to select useable assets from captured assets; and the lack of metadata describing the content and structure of media assets that could enable them to be retrieved and (re)used. We describe a new interaction and processing paradigm for media capture that redefines capture as a control process with feedback. By integrating human-computer interaction and computer vision and audition into an "active capture" process, we overcome the limitations of current media capture devices, algorithms, and interaction techniques. Active capture leverages media production knowledge to automate direction and cinematography and thus enables the automated production of annotated, high quality, reusable media assets.

Current consumer media production is laborious, tedious, and produces unsatisfying results. To address this problem, Active Capture leverages media production knowledge, computer vision and audition algorithms, and user interaction techniques to automate direction and cinematography and thus enables the automatic production of annotated, high quality, reusable media assets. Active Capture is part of a new computational media production paradigm that transforms media production from a manual mechanical process into an automated computational one that can produce mass customized and personalized media integrating video of non-actors. The implemented system automates the process of capturing a non-actor performing two simple reusable actions ("screaming" and "turning her head to look at the camera") and automatically integrates those shots into various commercials and movie trailers.

The Active Capture demonstration is part of a new computational media production paradigm that transforms media production from a manual mechanical process into an automated computational one that can produce mass customized and personalized media integrating video of non-actors. Active Capture leverages media production knowledge, computer vision and audition, and user interaction design to automate direction and cinematography and thus enables the automatic production of annotated, high quality, reusable media assets. The implemented system automates the process of capturing a non-actor performing two simple reusable actions ("screaming" and "turning her head to look at the camera") and automatically integrates those shots into various commercials and movie trailers.