Videos are an increasingly utilized part of the experience of the billions of people that use Facebook. These videos must be uploaded and processed before they can be shared and downloaded. Uploading and processing videos at our scale, and across our many applications, brings three key requirements: low latency to support interactive applications; a flexible programming model for application developers that is simple to program, enables efficient processing, and improves reliability; and robustness to faults and overload. This paper describes the evolution from our initial monolithic encoding script (MES) system to our current Streaming Video Engine (SVE) that overcomes each of the challenges. SVE has been in production since the fall of 2015, provides lower latency than MES, supports many diverse video applications, and has proven to be reliable despite faults and overload.

This paper describes the exploitation of hierarchical data names to achieve information-utility maximizing data collection in social sensing applications. We describe a novel transport abstraction, called the information funnel. It encapsulates a data collection protocol for social sensing that maximizes a measure of delivered information utility, that is the minimized data redundancy, by diversifying the data objects to be collected. The abstraction leverages named-data networking, a communication paradigm where data objects are named instead of hosts. We argue that this paradigm is especially suited for utility-maximizing transport in resource constrained environments, because hierarchical data names give rise to a notion of distance between named objects that is a function of only the topology of the name tree. This distance, in turn, can expose similarities between named objects that can be leveraged for minimizing redundancy among objects transmitted over bottlenecks, thereby maximizing their aggregate utility. With a proper hierarchical name space design, our protocol prioritizes transmission of data objects over bottlenecks to maximize information utility, with very weak assumptions on the utility function. This prioritization is achieved merely by comparing data name prefixes, without knowing application-level name semantics, which makes it generalizable across a wide range of applications. Evaluation results show that the information funnel improves the utility of the collected data objects compared to other lossy protocols.

Over 800 million people around the world share their social interactions with friends on Facebook, providing a rich body of information referred to as the social graph. In this talk, I describe how we model and serve this graph. Our model uses typed nodes (fbobjects) and edges (associations) to express the relationships and actions that happen on Facebook. We access the graph via a simple API that provides queries over the set of same-typed associations leaving an object. We have found this API to be both sufficiently expressive and amenable to a scalable implementation. In the last segment of the talk I describe the design of TAO, our graph data store. TAO is a distributed implementation of the fbobject and association API that has been serving production traffic at Facebook for more than 2 years.

Clustering large data sets with high dimensionality is a challenging data-mining task. This paper presents a framework to perform such a task efficiently. It is based on the notion of data space reduction, which finds high density areas, or dense cells, in the given feature space. The dense cells store summarized information of the data. A designated partitioning or hierarchical clustering algorithm can be used as the second step to find clusters based on the data summaries. Using Kmeans as an example, this paper presents GARDEN-Kmeans, which performs data space reduction using Gamma Region DENsity partition, and utilizes Kmeans to cluster the summarized information. The experimental study shows that GARDEN-Kmeans executes several orders of magnitude faster than basic Kmeans and the recursive bisection Kmeans algorithm of CLUTO, while producing comparable clustering quality

This paper compares two separately developed systems for monitoring activities to business contracts, describes how we integrated them and exploits the lessons learned from this process to identify a core set of requirements for a Business Contract Language (BCL). Concepts in BCL needed for contract monitoring include: the expression of coordinated concurrent actions; obliged, permitted and prohibited actions; rich timeliness expressions such as sliding windows; delegations; policy violations; contract termination/renewal conditions and reference to external data/events such as change in interest rates. The aim of BCL is to provide sufficient expressive power to describe contracts, including conditions which specify real-time processing, yet be simple enough to retain a human-oriented style for expressing contracts.

Materialized views in data warehouses are maintained incrementally, for reasons of efficiency, to present the latest updates to the users. These views are used by many warehouse readers (users) to execute OLAP queries by running several reader sessions and these views are maintained periodically by maintenance transactions. Therefore, there is an inherent problem of maintaining these views while the reader sessions continue to receive consistent data from these views. In this paper, we discuss a method that allows warehouse maintenance transactions to run concurrently with the reader sessions. Concurrency allows the readers to read the data from the views while the maintenance transaction updates these views. In our proposed method we create additional versions of views dynamically that contain only the modified tuples of the views and provide a mechanism to collapse these versions into the views periodically when there are no reader sessions accessing the views. These versions allow the reader sessions to access the old and the new information. The collapsing of the views is done by a low-priority process executing periodically.