Facebook has rapidly become for many the dominant means for sharing images, and the number of shared images accessible to any given Facebook user is easily in the tens of thousands. The sheer volume of pictures relegates most to obscurity, yet some of those pictures would be of great interest -- if a person could only find them. This research explores ways to harness latent semantic information associated with pictures and interpersonal relationships to enable a person to browse for potentially interesting and germane images shared by people in their social network. The possibilities for semantic analysis are endless, this work illustrates two possible approaches while also highlighting future potential applications of semantic understanding.

As consumers accumulate more and more personal imagery, searching for specific images has become increasingly difficult. Consumers typically provide little or no annotations, and automated classifiers and concept tagging tools are limited in their scope and vocabulary. This work addresses this sparsity of semantic information by leveraging domain-specific information provided by online photo-sharing communities. Such information enables improved search by allowing user-provided search terms to be expanded into a set of semantically related concepts, using relevant semantic relationships provided by millions of users. Our system first extracts metadata using a modest number of image and event-based semantic classifiers, as well as any meaningful file or folder names. When users pose text-based queries, our system retrieves images from their personal image collections by leveraging Flickr's tag dataset for concept expansion. This approach enables users to search their collections without having to manually annotate their pictures. We compare the retrieval performance of using a Flickr-based concept expander with the performance obtained without concept expansion and with using a WordNet-based concept expander. The results demonstrate that common sense knowledge gleaned from online photo sharing communities can enable meaningful image search on consumer image collections, searches that would be impossible using only the available image metadata.

In this paper, we describe a method for matching song lyrics with semantic annotations of picture collections in order to suggest songs that reflect picture content in lyrics or genre. Picture collections are first analyzed to extract a variety of semantic information including scene type, event type, and geospatial information. When aggregated over a picture collection, this semantic information forms a semantic signature of the collection. Typical picture collections in our scenario consist of photo subdirectories in which people store pictures of a place, activity, or event. Picture collections are expected to contain coherent semantic content describing in part or whole the event or activity they depict. The semantic signature of a picture collection is compared against song lyrics using a WordNet expansion based text matching to find songs relevant to the collection. We present interesting song suggestions, compare and contrast scenarios with human versus machine labels, and perform a user study to validate the usefulness of the proposed method. The proposed method will be a useful tool to support user media creation.

Benchmark and ground truth databases are critical for evaluating imaging- and event-based algorithms, but such databases can be time consuming to create, cannot be representative of all real-life consumer image collections, and do not necessarily provide feedback on the general utility of an imperfect algorithm applied to real-life data. An alternative solution is needed to complement evaluation by benchmark and ground truth databases. We designed and developed event analyzer as a tool to meet this need. Event analyzer combines data visualization of metadata with event and image retrieval results in a way that enables researchers to easily review data trends, quickly filter and select data, see relationships between metadata values, and visually scan retrieved results to determine correctness. In this paper, we describe the tool and discuss how it can and has been used to analyze real-life consumer collections in an event-centric manner.

The ability to automatically assess image characteristics is an important function for content management, building digital image albums, storytelling with images, and retrieval of specific visual content. This capability is needed to organize and sort large numbers of image and video assets. This paper proposes a novel approach to assess and rate images based on multidimensional characteristics including image quality, social relationships, aesthetic quality, important events, and usage. This new approach provides additional flexibility for end user applications that utilize different aspects of image characteristics. Specifically, we describe a method for assessing image quality based upon technical characteristics of the image, and for predicting the significance of an image based upon the people portrayed in the image. Experimental results indicate that the proposed multidimensional approach provides a promising framework for automated image value assessment and rating.

Unorganized media collections hinder consumers from fully experiencing and enjoying their visual media. User interfaces can mediate the results of automated indexing by presenting data and interactions that leverage the strengths of individual and combined algorithm results, supporting multifaceted browsing, and enabling user correction in a way that is not disruptive to the userspsila activities. We describe the semantic system demonstration framework (SSDF), a flexible and extensible framework for combining multiple semantic indexing algorithms for consumer photo and video clip collections into one integrated system. We also describe key features of Koi, an SSDF desktop client application with a user interface designed to mediate and leverage the intelligent indexing incorporated in the SSDF server. Together, Koi and SSDF empower users to experience their personal multimedia in novel and sophisticated ways.

The wide-spread use of image capturing and scanning devices such as digital cameras and low-cost scanners is rapidly resulting in the digital equivalent of the overstuffed shoebox full of snapshots.This paper describes a software system for automatic albuming of pictures to facilitate the creation of consumer albums and album pages.This software system provides consumers an efficient way of organizing and albuming their pictures digitized born different sources.The system includes advanced features such as image event clustering, dud detection, and duplicate detection necessary for automated albuming.The software architecture is based on a simple database structure that can efficiently handle metadata information.The system includes a substantial user interface component that enables the user to preview and edit the generated album.

Since remote procedure call (RPC) has become the method of choice for client-server communication in a distributed operating system, providing a fault-tolerant RPC mechanism is crucial to ensuring system reliability. The author presents a replicated RPC library for the Amoeba distributed operating system. The library's RPC protocol is presented in detail, including preliminary performance figures. The protocol is distinguished by its use of closed process groups in conjunction with the coordinator-cohort method of computation. The method presented is applicable to any system supporting closed process groups and totally ordered multicast.>