Data and code working together is fundamental to machine learning (ML), but the context around datasets and interactions between datasets and code are in general captured only rudimentarily. Context such as how the dataset was prepared and created, what source data were used, what code was used in processing, how the dataset evolved, and where it has been used and reused can provide much insight, but this information is often poorly documented. That is unfortunate since it makes datasets into black-boxes with potentially hidden characteristics that have downstream consequences. We argue that making dataset preparation more accessible and dataset usage easier to record and document would have significant benefits for the ML community: it would allow for greater diversity in datasets by inviting modification to published sources, simplify use of alternative datasets and, in doing so, make results more transparent and robust, while allowing for all contributions to be adequately credited. We present a platform, Renku, designed to support and encourage such sustainable development and use of data, datasets, and code, and we demonstrate its benefits through a few illustrative projects which span the spectrum from dataset creation to dataset consumption and showcasing.

Large-scale critical infrastructures such as transportation, energy, or water distribution networks are increasingly equipped with smart sensor technologies. Low-latency analytics on the resulting times series would open the door to many exciting opportunities to improve our grasp on complex urban systems. However, sensor-generated time series often turn out to be noisy, non-uniformly sampled, and misaligned in practice, making them ill-suited for traditional data processing. In this paper, we introduce TRISTAN (massive TRIckletS Time series ANalysis), a new data management system for efficient storage and real-time processing of fine-grained time series data. TRISTAN relies on a dedicated, compressed sparse representation of the time series using a dictionary. In contrast to previous approaches, TRISTAN is able to execute most analytics queries on the compressed data directly, and supports efficient and approximate query answering based on the most significant atoms of the dictionary only. We present the overall architecture of our system and discuss its performance on several smarter city datasets, showing that TRISTAN can achieve up to 20:1 compression ratios and 250x speedup compared to a state-of-the-art system.

Smart sensors are increasingly being used to manage and monitor critical urban infrastructures, e.g., for telecommunication, transport, water, or energy networks, as well as for healthcare or smart buildings. Sensor-based monitoring systems offer ways of continuously monitoring low frequency activities, and open the door to new analytic and predictive applications in Smarter Cities. Such sensors generate “tricklets”, i.e., noisy and continuous time series. Tricklets are typically misaligned, non-uniformly sampled, and comprise low frequency activities and recurring patterns. Storing and making sense of such data in a typical database management system is difficult, due to the impedance mismatch between classical (e.g., relational) data and tricklets. In this paper, we investigate the management of large amounts of tricklets from an architectural perspective, and propose MiSTRAL (MaSsive TRicklets anALysis), an architecture designed for executing low-latency analytics on time series warehouses. MiSTRAL uses a dictionary based representation for tricklets that allows queries to be run natively on compressed representations and thus to achieve the low-latency goal. The architecture of MiSTRAL is presented in detail in the following, along with early experimental results on several Smarter Cities datasets.

Network operators are continuously confronted with malicious events, such as port scans, denial-of-service attacks, and spreading of worms. Due to the detrimental effects caused by these anomalies, it is critical to detect them promptly and effectively. There have been numerous softwares, algorithms, or rules developed to conduct anomaly detection over traffic data. However, each of them only has limited descriptions of the anomalies, and thus suffers from high false positive/false negative rates. In contrast, the combination of multiple atomic detectors can provide a more powerful anomaly capturing capability when the base detectors complement each other. In this paper, we propose to infer a discriminative model by reaching consensus among multiple atomic anomaly detectors in an unsupervised manner when there are very few or even no known anomalous events for training. The proposed algorithm produces a perevent based non-trivial weighted combination of the atomic detectors by iteratively maximizing the probabilistic consensus among the output of the base detectors applied to different traffic records. The resulting model is different and not obtainable using Bayesian model averaging or weighted voting. Through experimental results on three network anomaly detection datasets, we show that the combined detector improves over the base detectors by 10% to 20% in accuracy.

In this paper we discuss our experience with the design of a public transport awareness application developed for the city of Dublin. The application is capable to ingest, analyze and visualize in real-time high volumes of traffic data coming from a variety of sources. We address the challenges encountered during the design of the application and propose novel solutions to tackle those challenges.

Multimedia stream mining applications require the identification of several different attributes in data content, and hence rely on a set of cascaded statistical classifiers to filter and process the data dynamically. In this paper, we introduce a novel methodology for configuring such cascaded classifier topologies, specifically binary classifier trees, in resource-constrained, distributed stream mining systems. Instead of traditional load shedding, our approach configures classifiers with optimized operating points after jointly considering the misclassification cost of each end-to-end class of interest in the tree, the resource constraints for every classifier, and the confidence level of each data object that is classified. The proposed approach allows for both intelligent load shedding as well as data replication based on available resources dynamically. We evaluate the algorithm on a sports video concept detection application and identify huge cost savings over load shedding alone. Additionally, we propose several distributed algorithms that enable each classifier in the tree to reconfigure itself based on local information exchange. We analyze the associated tradeoffs between convergence time, information overhead, and the cost efficiency of results achieved by each classifier for each of these algorithms.

The emergence of large-field radio telescopes has generated the need to process huge amounts of data “on the fly” in order to avoid substantial costs in storage and signal processing. In this paper, we describe a streaming software approach to radio astronomy, and in particular, show how the IBM-developed stream computing architecture System S enables seamless “on the fly” radio astronomy imaging. This approach overcomes pertinent issues of memory limitation and latencies related to data processing, and it allows autonomous processor allocation to tasks as the need arises.

State-of-the-art learning algorithms accept data in feature vector format as input. Examples belonging to different classes may not always be easy to separate in the original feature space. One may ask: can transformation of existing features into new space reveal significant discriminative information not obvious in the original space? Since there can be infinite number of ways to extend features, it is impractical to first enumerate and then perform feature selection. Second, evaluation of discriminative power on the complete dataset is not always optimal. This is because features highly discriminative on subset of examples may not necessarily be significant when evaluated on the entire dataset. Third, feature construction ought to be automated and general, such that, it doesn't require domain knowledge and its improved accuracy maintains over a large number of classification algorithms. In this paper, we propose a framework to address these problems through the following steps: (1) divide-conquer to avoid exhaustive enumeration; (2) local feature construction and evaluation within subspaces of examples where local error is still high and constructed features thus far still do not predict well; (3) weighting rules based search that is domain knowledge free and has provable performance guarantee. Empirical studies indicate that significant improvement (as much as 9% in accuracy and 28% in AUC) is achieved using the newly constructed features over a variety of inductive learners evaluated against a number of balanced, skewed and high-dimensional datasets. Software and datasets are available from the authors.

With the widespread adoption of location tracking technologies like GPS, the domain of intelligent transportation services has seen growing interest in the last few years. Services in this domain make use of real-time location-based data from a variety of sources, combine this data with static location-based data such as maps and points of interest databases, and provide useful information to end-users. Some of the major challenges in this domain include i) scalability, in terms of processing large volumes of real-time and static data; ii) extensibility, in terms of being able to add new kinds of analyses on the data rapidly, and iii) user interaction, in terms of being able to support different kinds of one-time and continuous queries from the end-user. In this paper, we demonstrate the use of IBM InfoSphere Streams, a scalable stream processing platform, for tackling these challenges. We describe a prototype system that generates dynamic, multi-faceted views of transportation information for the city of Stockholm, using real vehicle GPS and road-network data. The system also continuously derives current traffic statistics, and provides useful value-added information such as shortest-time routes from real-time observed and inferred traffic conditions. Our performance experiments illustrate the scalability of the system. For instance, our system can process over 120000 incoming GPS points per second, combine it with a map containing over 600,000 links, continuously generate different kinds of traffic statistics and answer user queries.

Abstract Stream processing applications are used to ingest, process, and analyze continuous data streams from heterogeneous sources of live and stored data, generating streams of output results. These applications are, in many cases, complex, large‐scale, low‐latency, and distributed in nature. In this paper, we describe the design principles and architectural underpinnings for stream processing applications. These principles are distilled from our experience in building real‐world applications both for internal use as well as with customers from several industrial and academic domains. We provide principles, guidelines, as well as appropriate implementation examples to highlight the different aspects of stream processing application design and development. Copyright © 2010 John Wiley & Sons, Ltd.

We consider the problem of estimating the direction of arrival (DOA) of multiple waves incident on a single tripole sensor. Using the physical properties of the electric and magnetic fields, we show that we can disambiguate the DOA of multiple simultaneously incident waves using a set of time sampled 3D measurements from the single sensor. This is different from the traditional approach that uses arrays of antennas to estimate DOA. We use the unitary matrix pencil method to estimate the frequencies of the waves, and use a least squares solver to estimate the amplitude and phase coefficients. We combine these to compute the DOA and evaluate the approach using simulations. We show that the method is very effective at estimating DOA for different numbers of incident waves, and different noise levels.

In this paper, we propose a distributed solution to the problem of configuring classifier trees in distributed stream mining systems. The configuration involves selecting appropriate false-alarm detection tradeoffs for each classifier to minimize end-to-end penalty in terms of misclassification cost. In the proposed solution, individual classifiers select their operating points (i.e., actions) to maximize a local utility function. The utility may be purely local to the current classifier, corresponding to a myopic strategy, or may include the impact of the classifier actions on successive classifiers in the tree, corresponding to a foresighted strategy. We analytically show that actions determined by the foresighted strategies can improve the end-to-end performance of the classifier tree and derive an associated probability bound. We then evaluate our solutions on an application for hierarchical sports scene classification. By comparing centralized, myopic and foresighted solutions, we show that foresighted strategies result in better performance than myopic strategies, and also asymptotically approach the centralized optimal solution.

We consider the problem of configuring classifier trees in distributed stream mining system. The configuration involves selecting appropriate false-alarm detection tradeoffs for each classifier to minimize end-to-end penalty in terms of misclassification cost. We model this as a tree configuration game and design solutions, where individual classifiers select their operating points to maximize a local utility. We derive appropriate misclassification cost coefficients for intermediate classifiers, and determine the information that needs to be exchanged across classifiers, in order to successfully design the game. We analytically show that there is a unique pure strategy Nash equilibrium in operating points, which guarantees a convergence of the proposed approach. We evaluate the performance of our algorithm on an application for sports scene classification, and compare against centralized solutions. We show that our algorithm results in better performance than the centralized solution on average. Moreover, the algorithm approaches the optimal solution asymptotically with increasing number of actions per classifier.

This work examines under what conditions compression methodologies can retain the outcome of clustering operations. We focus on the popular k-means clustering algorithm and we demonstrate how a properly constructed compression scheme based on post-clustering quantization is capable of maintaining the global cluster structure. Our analytical derivations indicate that a 1-bit moment preserving quantizer per cluster is sufficient to retain the original data clusters. Merits of the proposed compression technique include: a) reduced storage requirements with clustering guarantees, b) data privacy on the original values, and c) shape preservation for data visualization purposes. We evaluate quantization scheme on various high-dimensional datasets, including 1-dimensional and 2-dimensional time-series (shape datasets) and demonstrate the cluster preservation property. We also compare with previously proposed simplification techniques in the time-series area and show significant improvements both on the clustering and shape preservation of the compressed datasets.

Large-scale multimedia semantic concept detection requires realtime identification of a set of concepts in streaming video or large image datasets. The potentially high data volumes of multimedia content, and high complexity associated with individual concept detectors, have hindered the practical deployment of many current solutions. In this paper, we present a summary of our work in building systems and applications for resource adaptive semantic concept detection in multimedia using large-scale distributed stream mining systems. We construct such concept detection applications as a hierarchical topology of individual concept detectors, and deploy them on distributed processing infrastructure. We then focus on dynamically configuring individual concept detectors to meet system imposed resource constraints while minimizing a penalty defined in terms of the misclassification cost. We present multiple centralized and distributed algorithms for this configuration, and describe the implemented application and system. We also verify through simulations that significant improvement in terms of accuracy of classification can be achieved through our approach.

When labeled examples are limited and difficult to obtain, transfer learning employs knowledge from a source domain to improve learning accuracy in the target domain. However, the assumption made by existing approaches, that the marginal and conditional probabilities are directly related between source and target domains, has limited applicability in either the original space or its linear transformations. To solve this problem, we propose an adaptive kernel approach that maps the marginal distribution of targetdomain and source-domain data into a common kernel space, and utilize a sample selection strategy to draw conditional probabilities between the two domains closer. We formally show that under the kernel-mapping space, the difference in distributions between the two domains is bounded; and the prediction error of the proposed approach can also be bounded. Experimental results demonstrate that the proposed method outperforms both traditional inductive classifiers and the state-of-the-art boosting-based transfer algorithms on most domains, including text categorization and web page ratings. In particular, it can achieve around 10 % higher accuracy than other approaches for the text categorization problem. The source code and datasets are available from the authors.

Transferring knowledge from one domain to another is challenging due to a number of reasons. Since both conditional and marginal distribution of the training data and test data are non-identical, model trained in one domain, when directly applied to a different domain, is usually low in accuracy. For many applications with large feature sets, such as text document, sequence data, medical data, image data of different resolutions, etc. two domains usually do not contain exactly the same features, thus introducing large numbers of "missing values" when considered over the union of features from both domains. In other words, its marginal distributions are at most overlapping. In the same time, these problems are usually high dimensional, such as, several thousands of features. Thus, the combination of high dimensionality and missing values make the relationship in conditional probabilities between two domains hard to measure and model. To address these challenges, we propose a framework that first brings the marginal distributions of two domains closer by "filling up" those missing values of disjoint features. Afterwards, it looks for those comparable sub-structures in the "latent-space" as mapped from the expanded feature vector, where both marginal and conditional distribution are similar. With these sub-structures in latent space, the proposed approach then find common concepts that are transferable across domains with high probability. During prediction, unlabeled instances are treated as "queries", the mostly related labeled instances from out-domain are retrieved, and the classification is made by weighted voting using retrieved out-domain examples. We formally show that importing feature values across domains and latent semantic index can jointly make the distributions of two related domains easier to measure than in original feature space, the nearest neighbor method employed to retrieve related out domain examples is bounded in error when predicting in-domain examples. Software and datasets are available for download.

We address the problem of keyword spotting in continuous speech streams when training and testing conditions can be different. We propose a keyword spotting algorithm based on sparse representation of speech signals in a time-frequency feature space. The training speech elements are jointly represented in a common subspace built on simple basis functions. The subspace is trained in order to capture the common time-frequency structures from different occurrences of the keywords to be spotted. The keyword spotting algorithm then employs a sliding window mechanism on speech streams. It computes the contribution of successive speech segments in the subspace of interest and evaluates the similarity with the training data. Experimental results on the TIMIT database show the effectiveness and the noise resilience of the low complexity spotting algorithm.

Frequent patterns provide solutions to datasets that do not have well-structured feature vectors. However, frequent pattern mining is non-trivial since the number of unique patterns is exponential but many are non-discriminative and correlated. Currently, frequent pattern mining is performed in two sequential steps: enumerating a set of frequent patterns, followed by feature selection. Although many methods have been proposed in the past few years on how to perform each separate step efficiently, there is still limited success in eventually finding highly compact and discriminative patterns. The culprit is due to the inherent nature of this widely adopted two-step approach. This paper discusses these problems and proposes a new and different method. It builds a decision tree that partitions the data onto different nodes. Then at each node, it directly discovers a discriminative pattern to further divide its examples into purer subsets. Since the number of examples towards leaf level is relatively small, the new approach is able to examine patterns with extremely low global support that could not be enumerated on the whole dataset by the two-step method. The discovered feature vectors are more accurate on some of the most difficult graph as well as frequent itemset problems than most recently proposed algorithms but the total size is typically 50% or more smaller. Importantly, the minimum support of some discriminative patterns can be extremely low (e.g. 0.03%). In order to enumerate these low support patterns, state-of-the-art frequent pattern algorithm either cannot finish due to huge memory consumption or have to enumerate 101 to 103 times more patterns before they can even be found. Software and datasets are available by contacting the author.

Real-time multimedia semantic concept detection requires instant identification of a set of concepts in streaming video or images. However, the potentially high data volumes of multimedia content, and high complexity associated with individual concept detectors, have hindered its practical deployment. In this paper, we present a new online concept detection system deployed on top of a distributed stream mining system. It uses a tree-topology of classifiers that are constructed on a semantic hierarchy of concepts of interest. We introduce a novel methodology for configuring such cascaded classifier topologies under constraints on the available resources. In our approach, we configure individual classifiers with optimized operating points after jointly and explicitly considering the misclassification cost of each end-to-end class of interest in the tree, the system imposed resource constraints, and the confidence level of each object that is classified. We describe the implemented application, system, and optimization algorithms, and verify that significant improvement in terms of accuracy of classification can be achieved through our approach.

Stream mining applications require the identification of several different attributes in data content and hence rely on a distributed set of cascaded statistical classifiers to filter and process the data dynamically. In this letter, we introduce a novel methodology for configuring cascaded classifier topologies, specifically binary classifier trees, with optimized operating points after jointly considering the misclassification cost of each end-to-end class of interest in the tree, the resource constraints for every classifier, and the confidence level of each data object that is classified. By configuring multiple operating points per classifier, we enable not only intelligent load shedding when resources are scarce but also intelligent replication of low confidence data across multiple edges when excess resources are available. Using a classifier tree constructed from support vector machine-based sports image classifiers, we verify huge cost savings and discuss how different classifier placements and costs can influence the gains obtained by various algorithms.

We design algorithms for optimally configuring, in terms of model complexity as well as operating point, a cascade of exclusive classifiers on distributed systems, given underlying resource constraints. Under rate-dependent constraints we design a Viterbi-like strategy to determine the optimal solution with significantly lower computational complexity than an exhaustive search. We illustrate the performance of this strategy on a cascade of classifiers for a speaker verification task, and highlight performance gains over using a single classifier.

Distributed stream processing represents a novel computing paradigm where data, sensed externally and possibly preprocessed, is pushed asynchronously to various connected computing devices with heterogeneous capabilities for processing. It enables novel applications typically characterized by the need to process high-volume data streams in a timely and responsive fashion. Some example applications include sensor networks, location-tracking services, distributed speech recognition, and network management. Recent work in large-scale distributed stream processing tackle various research challenges in both the application domain as well as in the underlying system. The main focus of this paper is to highlight some of the signal processing challenges such a novel computing framework brings. We first briefly introduce the main concepts behind distributed stream processing. Then we define the notion of relevant information from two related information-theoretic approaches. Finally, we browse existing techniques for sensing and quantizing the information given the set of classification, detection and estimation tasks, which we refer to as task-driven signal processing. We also address some of the related unexplored research challenges

State-of-the-art speaker verification systems are built around the likelihood ratio test, using Gaussian mixture models (GMM) for likelihood functions, a universal background model (UBM) for alternative speaker representation, and a form of Bayesian adaptation to derive speaker models from the UBM. This work tackles optimal quantizer design of the speech cepstral features (MFCCs) for such systems. The problem is posed as the minimization of loss of log-likelihood ratio between the quantized and unquantized speech features. First we show that the conventional mean squared error (MSE) quantizer for the top-scoring UBM Gaussian is optimal under practical assumptions. Then we derive the optimal bit allocation strategy across the dimensions of the feature vectors. Finally we demonstrate the validity of the approach against various quantization and bit allocation schemes by running experiments on the appropriately modified IBM speaker verification system. Experimental results on the HUB4 corpora show negligible impact on verification performance for bit rates as low as less than 1 bit per dimension on average in contrast to 32 bits per dimension in the original system

Distributed stream processing systems offer a highly scalable and dynamically configurable platform for time-critical applications ranging from real-time, exploratory data mining to high performance transaction processing. Resource management for distributed stream processing systems is complicated by a number of factors processing elements are constrained by their producer-consumer relationships, data and processing rates can be highly bursty, and traditional measures of effectiveness, such as utilization, can be misleading. In this paper, we propose a novel distributed, adaptive control algorithm that maximizes weighted throughput while ensuring stable operation in the face of highly bursty workloads. Our algorithm is designed to meet the challenges of extreme-scale stream processing systems, where overprovisioning is not an option, by making the best use of resources even when the proffered load is greater than available resources. We have implemented our algorithm in a real-world distributed stream processing system and a simulation environment. Our results show that our algorithm is not only self-stabilizing and robust to errors, but also outperforms traditional approaches over a broad range of buffer sizes, processing graphs, and burstiness types and levels.

Networks of classifiers are capturing the attention of system and algorithmic researchers because they offer improved accuracy over single model classifiers, can be distributed over a network of servers for improved scalability, and can be adapted to available system resources. This work provides a principled approach for the optimized allocation of system resources across a networked chain of classifiers. We begin with an illustrative example of how complex classification tasks can be decomposed into a network of binary classifiers. We formally define a global performance metric by recursively collapsing the chain of classifiers into one combined classifier. The performance metric trades off the end-to-end probabilities of detection and false alarm, both of which depend on the resources allocated to each individual classifier. We formulate the optimization problem and present optimal resource allocation results for both simulated and state-of-the-art classifier chains operating on telephony data.

Technologies that use the Internet network to deliver voice communications have the potential to reduce costs and improve access to communications services around the world. However, these new technologies pose several challenges in terms of confidentiality of the conversations and anonymity of the conversing parties. Call authentication and encryption techniques provide a way to protect confidentiality, while anonymity is typically preserved by an anonymizing service (anonymous call). This work studies the feasibility of revealing pairs of anonymous and encrypted conversing parties (caller/callee pair of streams) by exploiting the vulnerabilities inherent to VoIP systems. In particular, by exploiting the aperiodic inter-departure time of VoIP packets, we can trivialize each VoIP stream into a binary time-series. We first define a simple yet intuitive metric to gauge the correlation between two VoIP binary streams. Then we propose an effective technique that progressively pairs conversing parties with high accuracy and in a limited amount of time. Our metric and method are justified analytically and validated by experiments on a very large standard corpus of conversational speech. We obtain impressively high pairing accuracy that reaches 97% after 5 minutes of voice conversations.

Systems designed to extract time-critical information from large volumes of unstructured data must include the ability, both from an architectural and algorithmic point of view, to filter out unimportant data that might otherwise overwhelm the available resources. This paper presents an approach for data filtering to reduce computation in the context of a distributed speech processing architecture designed to detect or identify speakers. Here, filtering means either dropping and ignoring data or passing it on for further processing. The goal of the paper is to show that when the filter is designed to select and pass on a subset of the input data that best preserves the ability to recognize a specific desired speaker, or group of speakers, a large percentage of the data can be ignored while being able to preserve most of the accuracy

This paper presents a distortion optimized streaming algorithm for on-demand streaming of multimedia. Given the pre-encoded packets of a multimedia stream, we propose a fast algorithm for selecting an appropriate subset of these packets such that the overall client distortion is minimized. This minimization is performed within the rate constraints imposed by the communication channel. In particular, at each transmission opportunity, the proposed approach uses a linear-time algorithm to select the best packet to transmit through the minimization of the expected client distortion. The time complexity of the algorithm is reduced through a factorization of the streaming policy into simpler terms and performing a greedy optimization to select the packet. Inevitably, this in itself leads to sub-optimal results. To alleviate the adverse impact of the greedy optimization, the cost function is penalized with the expected buffer occupancy at the end of the epoch of the optimization. We pose this problem as a Lagrangian minimization. We demonstrate the efficacy of the proposed approach through empirical evaluation.

In this paper, we propose a semantic routing and filtering framework for large-scale monitoring of video streams. Our goal is to build a distributed system that at any given time is capable of simultaneously monitoring the content of multiple video streams being transmitted over the Internet (or a proprietary network). A key design requirement of such a system is the ability to handle tens of gigabytes of multimedia data per second. Traditional techniques have an important limitation. Once a bottleneck in terms of CPU or storage is reached, data is dropped indiscriminately.. In this paper, we propose distributed real-time semantic filters to route and filter video data. We propose a mechanism to alter the accuracy of classification with the complexity of execution; thus avoiding system failure during periods of overload. We propose a set of novel video features that perform better than our previous semantic classifiers. This system is capable of classifying over a hundred concepts. Experiments on 190 hours of pre-stored and live video streams validate the effectiveness of the proposed system.

In this paper we present multimedia adaptation strategies for live video streams, at the streaming server, where we switch among several versions of the coded multimedia to match the available network bandwidth accurately, and meet client delay constraints. We estimate information about the available network bandwidth, at the server, by monitoring the application buffer and then decide to adaptively switch up or switch down the transmitted bit-rate. We use a piecewise linear model for the network bandwidth to estimate the current and future server buffer drain delay, and derive the transmission rate to minimize client buffer starvation. We implement these strategies in an enterprise streaming system and verify the performance, in terms of the network fidelity and received video quality, using both real as well as simulated network traces.

This work tackles low delay adaptive video streaming over error-prone networks. Our framework consists of an encoding station, an edge server and a set of clients with various access rates. The edge server is capable of performing simple error concealment operations on the incoming data before forwarding the adapted media to its clients. We study two encoding scenarios: versions (multiple encodings at various output rates) and layers. We develop a unified end-to-end distortion model, which we use to derive the optimal coding strategy for both scenarios. Finally we analyze the performance of MPEG-4 coded versions against MPEG-4 FGS-coded layers in rate-constrained lossy environments. Experiments show that versions perform better than layers when the constraint on the aggregate rate is somewhat relaxed, for low to medium packet loss ratios.

Service providers are increasingly turning to federated approaches to meet customer demand while limiting the risk of large-scale computing infrastructure investments. In such models, service providers could meet the demands of peak processing periods or specialized application environments by offloading to a partner provider. While a number of distributed systems initiatives, including Grid computing, rely on such cooperatives, models for effective capacity planning and settlement in such environments has largely been ignored. In this paper, we develop a model for settlement and capacity planning in a multi-provider computing cooperative. We introduce two pricing tariffs - linear and burstable pricing - that are commonly used in the industry, and compare their relative merits.

In this demonstration we present the ARMS system which enables secure and adaptive rich media streaming to a large-scale, heterogeneous client population. The ARMS system dynamically adapts streams to available bandwidth, client capabilities, packet loss, and administratively imposed policies - all while maintaining full content security. The ARMS system is completely standards compliant and to our knowledge is the first such end-to-end MPEG-4-based system.

This paper describes the ARMS system which enables secure and adaptive rich media streaming to a large-scale, heterogeneous client population. The secure streaming algorithms ensure end-to-end security while the content is adapted and streamed via intermediate, potentially untrusted servers. ARMS streaming is completely standards compliant and to our knowledge is the first such end-to-end MPEG-4-based system.

The widespread use of the Internet and the maturing of digital video technology have led to an increase in various streaming media applications. As broadband to the home becomes more prevalent, the bottleneck of delivering quality streaming media is shifting upstream to the backbone, peering links, and the best-effort Internet. In this paper, we address the problem of efficiently streaming video assets to the end clients over a distributed infrastructure consisting of origin servers and proxy caches. We build on earlier work and propose a unified mathematical framework under which various server scheduling and proxy cache management algorithms for video streaming can be analyzed. More precisely, we incorporate known server scheduling algorithms (batching/patching/batch-patching) and proxy caching algorithms (full/partial/no caching with or without caching patch bytes) in our framework and analyze the minimum backbone bandwidth consumption under the optimal joint scheduling and caching strategies. We start by studying the optimal policy for streaming a single video object and derive a simple gradient-descent-based cache allocation algorithm to enable management of multiple heterogeneous videos efficiently. We then show that the performance of our heuristic is close to that of the optimal scheme, under a wide range of parameters.

We consider the transmission of variable bit rate (VBR) video over a network offering a guaranteed service such as ATM VBR or the guaranteed service of the IETF. The guaranteed service requires that the flow accepted by the network has to be conforming with a traffic envelope /spl sigma/. In this context, the output of the video encoder is constrained by the traffic envelope defined at the network entry point, the playback delay budget and the decoding buffer size. In previous works, the constraints are satisfied either by smoothing a fixed coder output, or by modifying the encoding parameters. In this paper we take a combined approach. This shows us to find a joint source rate selection/smoothing solution which minimizes the total average distortion while satisfying constraints on the traffic envelope, playback delay and decoding buffer size. Our solution is based on a Viterbi-like algorithm. Our approach is. Made possible by the representation of the optimally smoothed output as the time inverse of a shaper output. Experimental results exhibit significant improvements in terms of total average distortion compared to the smoothing of a fixed coder output, under equivalent traffic parameters and decoding constraints.

We address a new error-resilient scheme for broadcast quality MPEG-2 video streams to be transmitted over lossy packet networks. A new scene-complexity adaptive mechanism, namely Adaptive MPEG-2 Information Structuring (AMIS) is introduced. AMIS modulates the number of resynchronization points (i.e., slice headers and intra-coded macroblocks) in order to maximize the perceived video quality, assuming that the encoder is aware of the underlying packetization scheme, the packet loss probability (PLR), and the error-concealment technique implemented at the decoding side. The end-to-end video quality depends both on the encoding quality and the degradation due to data loss. Therefore, AMIS constantly determines the best compromise between the rate allocated to encode pure video information and the rate aiming at reducing the sensitivity to packet loss. Experimental results show that AMIS dramatically outperforms existing structuring techniques, thanks to its efficient adaptivity. We then extend AMIS with a forward-error-correction (FEC)-based protection algorithm to become AMISP. AMISP triggers the insertion of FEC packets in the MPEG-2 video packet stream. Finally, the performances of the AMISP scheme in an MPEG-2 over RTP/UDP/IP scenario are evaluated.

This paper deals with the optimal allocation of MPEG-2 encoding and media-independent forward error correction (FEC) rates under a total given bandwidth. The optimality is defined in terms of minimum perceptual distortion given a set of video and network parameters. We first derive the set of equations leading to the residual loss process parameters. That is, the packet loss ratio (PLR) and the average burst length after FEC decoding. We then show that the perceptual source distortion decreases exponentially with the increasing MPEG-2 source rate. We also demonstrate that the perceptual distortion due to data loss is directly proportional to the number of lost macroblocks, and therefore decreases with the amount of channel protection. Finally, we derive the global set of equations that lead to the optimal dynamic rate allocation. The optimal distribution is shown to outperform classical FEC scheme, thanks to its adaptivity to the scene complexity, the available bandwidth and to the network performance. Furthermore, our approach holds for any standard video compression algorithms (i.e., MPEG-x, H.26x).

We consider the transmission of variable bit rate (VBR) video over a network offering a guaranteed service such as ATM VBR or the guaranteed service of the IETF. The guaranteed service requires that the flow accepted by the network has to be conforming with a traffic envelope /spl sigma/; in return, it receives a service guarantee expressed by a network service curve /spl beta/. Functions /spl alpha/ and /spl beta/ are derived from the parameters used for setting up the reservation, for example, from the T-SPEC and R-SPEC fields used with the resource reservation protocol (RSVP). In order to satisfy the traffic envelope constraint, the output of the encoder is fed to a smoother, possibly with some look-ahead. The resulting stream is transported by the network; at the destination, the decoder waits for an initial playback delay and reads the stream from the receive buffer. We consider the problem of whether there exists one optimal strategy at the smoother which minimizes the playback delay and the receive buffer size, given the traffic envelope /spl alpha/ and the service curve /spl beta/. We show that there does exist such an optimal smoothing, and give an explicit representation for it. We also obtain a simple expression for the smallest playback delay and playback buffer size which can be achieved over all possible smoothing and playback strategies. We show that the computation of optimal smoothing and minimum playback delay do not depend on the past. We show that separate delay equalization is optimal in the constant bit rate (CBR) case, but not otherwise. We also apply the theory to the analysis of which T-SPEC should be requested by a source-destination pair, given some playback delay and buffer constraint, and given the path characteristics advertised in RSVP PATH messages.

This paper describes an extension of a vision model for video that has been designed to estimate how specific features in moving pictures are rendered as a consequence of a compression process. The resulting model permits to evaluate the distortions on contours and textures in a sequence, as well as to estimate how strong the blocking effect (resulting from a block-DCT compression scheme) is. The model has been used to evaluate feature rendition in MPEG-2 compressed sequences.

This work addresses the optimization of video services by improving the bit allocation stage in a video encoder. We first introduce a new block-based video metric based on a vision model, termed the perceptual visibility predictor (PVP), which accounts for spatial as well as temporal perceptual activities. In other words, we propose a way to classify sequence areas in terms of their relevance to human perception. We then briefly show how this metric may be used within a one-pass casual adaptive quantization scheme applied to MPEG-2 video encoding.

In this paper we propose a novel bit allocation scheme for MPEG-2 constant bit rate (CBR) video coding using a model of the early stages of human vision. On the basis of this model we derive a measure of the macroblock activity different from the one proposed in the MPEG-2 test model 5 (TM5). Experiments, obtained by substituting the proposed perceptual activity measure to the one proposed by the MPEG-2 TM5, yield better results, among which are the improvement of the perceptual quality for a fixed bitrate and vice-versa.