App markets have evolved into highly competitive and dynamic environments for developers. While the traditional app life cycle involves incremental updates for feature enhancements and issue resolution, some apps deviate from this norm by undergoing significant transformations in their use cases or market positioning. We define this previously unstudied phenomenon as ‘app metamorphosis'. In this paper, we propose a novel and efficient multi-modal search methodology to identify apps undergoing metamorphosis and apply it to analyse two snapshots of the Google Play Store taken five years apart. Our methodology uncovers various metamorphosis scenarios, including re-births, re-branding, re-purposing, and others, enabling comprehensive characterisation. Although these transformations may register as successful for app developers based on our defined success score metric (e.g., re-branded apps performing approximately 11.3% better than an average top app), we shed light on the concealed security and privacy risks that lurk within, potentially impacting even tech-savvy end-users.

Abstract Numerous tools have been developed to aggressively block the execution of popular JavaScript programs in Web browsers. Such blocking also affects functionality of webpages and impairs user experience. As a consequence, many privacy preserving tools that have been developed to limit online tracking, often executed via JavaScript programs, may suffer from poor performance and limited uptake. A mechanism that can isolate JavaScript programs necessary for proper functioning of the website from tracking JavaScript programs would thus be useful. Through the use of a manually labelled dataset composed of 2,612 JavaScript programs, we show how current privacy preserving tools are ineffective in finding the right balance between blocking tracking JavaScript programs and allowing functional JavaScript code. To the best of our knowledge, this is the first study to assess the performance of current web privacy preserving tools in determining tracking vs. functional JavaScript programs. To improve this balance, we examine the two classes of JavaScript programs and hypothesize that tracking JavaScript programs share structural similarities that can be used to differentiate them from functional JavaScript programs. The rationale of our approach is that web developers often “borrow” and customize existing pieces of code in order to embed tracking (resp. functional) JavaScript programs into their webpages. We then propose one-class machine learning classifiers using syntactic and semantic features extracted from JavaScript programs. When trained only on samples of tracking JavaScript programs, our classifiers achieve accuracy of 99%, where the best of the privacy preserving tools achieve accuracy of 78%. The performance of our classifiers is comparable to that of traditional two-class SVM. One-class classification, where a training set of only tracking JavaScript programs is used for learning, has the advantage that it requires fewer labelled examples that can be obtained via manual inspection of public lists of well-known trackers. We further test our classifiers and several popular privacy preserving tools on a larger corpus of 4,084 websites with 135,656 JavaScript programs. The output of our best classifier on this data is between 20 to 64% different from the tools under study. We manually analyse a sample of the JavaScript programs for which our classifier is in disagreement with all other privacy preserving tools, and show that our approach is not only able to enhance user web experience by correctly classifying more functional JavaScript programs, but also discovers previously unknown tracking services.

The increased popularity of smartphones has attracted a large number of developers to offer various applications for the different smartphone platforms via the respective app markets. One consequence of this popularity is that the app markets are also becoming populated with spam apps. These spam apps reduce the users’ quality of experience and increase the workload of app market operators to identify these apps and remove them. Spam apps can come in many forms such as apps not having a specific functionality, those having unrelated app descriptions or unrelated keywords, or similar apps being made available several times and across diverse categories. Market operators maintain antispam policies and apps are removed through continuous monitoring. Through a systematic crawl of a popular app market and by identifying apps that were removed over a period of time, we propose a method to detect spam apps solely using app metadata available at the time of publication. We first propose a methodology to manually label a sample of removed apps, according to a set of checkpoint heuristics that reveal the reasons behind removal. This analysis suggests that approximately 35% of the apps being removed are very likely to be spam apps. We then map the identified heuristics to several quantifiable features and show how distinguishing these features are for spam apps. We build an Adaptive Boost classifier for early identification of spam apps using only the metadata of the apps. Our classifier achieves an accuracy of over 95% with precision varying between 85% and 95% and recall varying between 38% and 98%. We further show that a limited number of features, in the range of 10--30, generated from app metadata is sufficient to achieve a satisfactory level of performance. On a set of 180,627 apps that were present at the app market during our crawl, our classifier predicts 2.7% of the apps as potential spam. Finally, we perform additional manual verification and show that human reviewers agree with 82% of our classifier predictions.

Data centers are an indispensable part of today's IT infrastructure. To keep pace with modern computing needs, data centers continue to grow in scale and consume increasing amounts of power. While prior work on data centers has led to significant improvements in their energy-efficiency, detailed measurements from these facilities' operations are not widely available, as data center design is often considered part of a company's competitive advantage. However, such detailed measurements are critical to the research community in motivating and evaluating new energy-efficiency optimizations. In this paper, we present a detailed analysis of a state-of-the-art 15MW green multi-tenant data center that incorporates many of the technological advances used in commercial data centers. We analyze the data center's computing load and its impact on power, water, and carbon usage using standard effectiveness metrics, including PUE, WUE, and CUE. Our results reveal the benefits of optimizations, such as free cooling, and provide insights into how the various effectiveness metrics change with the seasons and increasing capacity usage. More broadly, our PUE, WUE, and CUE analysis validate the green design of this LEED Platinum data center.

The proliferation of online video content has triggered numerous works on its evolution and popularity, as well as on the effect of social sharing on content propagation. In this paper, we focus on the observable dependencies between the virality of video content on a micro-blogging social network (in this case, Twitter) and the popularity of such content on a video distribution service (YouTube). To this end, we collected and analysed a corpus of Twitter posts containing links to YouTube clips and the corresponding video meta-data from YouTube. Our analysis highlights the unique properties of content that is both popular and viral, which allows such content to attract high number of views on YouTube and achieve fast propagation on Twitter. With this in mind, we proceed to the predictions of popular-and-viral clips and propose a framework that can, with high degree of accuracy and low amount of training data, predict videos that are likely to be popular, viral, and both. The key contribution of our work is the focus on cross-system dynamics between YouTube and Twitter. We conjecture and validate that cross-system prediction of both popularity and virality of videos is feasible, and can be performed with a reasonably high degree of accuracy. One of our key findings is that YouTube features capturing user engagement, have strong virality prediction capabilities. This findings allows to solely rely on data extracted from a video sharing service to predict popularity and virality aspects of videos.

Third-party identity management services enable cross-site information sharing, making Web access seamless but also raise significant privacy implications for the users. Using a combination of manual analysis of identified third-party identity management relationships and targeted case studies we capture how the protocol usage and third-party selection is changing, profile what information is requested to be shared (and actions to be performed) between websites, and identify privacy issues and practical problems that occur when using multiple accounts (associated with these services). The study highlights differences in the privacy leakage risks associated with different classes of websites, and shows that the use of multiple third-party websites, in many cases, can cause the user to lose (at least) partial control over which information is shared/posted on their behalf.

This paper considers the problem of providing users playing one streaming video the option of instantaneous and seamless playback of alternative videos. Recommendation systems can easily provide a list of alternative videos, but there is little research on how to best eliminate the startup time for these alternative videos. The problem is motivated by services that want to retain increasingly impatient users, who frequently watch the beginning of multiple videos, before viewing a video to the end. We present the design, implementation, and evaluation of an HTTP-based Adaptive Streaming (HAS) solution that provides careful prefetching and buffer management. We also present the design and evaluation of three fundamental policy classes that provide different tradeoffs between how aggressively new alternative videos are prefetched versus the importance of ensuring high playback quality. We show that our solution allows us to reduce the startup times of alternative videos by an order of magnitude and effectively adapt the quality such as to ensure the highest possible playback quality of the video being viewed. By improving the channel utilization we also address the discrimination problem that HAS clients often suffer from, allowing us to in some cases simultaneously improve the playback quality of the video being viewed and provide the value-added service of allowing instantaneous playback of the prefetched alternative videos.

Increased popularity of smartphones has attracted a large number of developers to various smartphone platforms. As a result, app markets are also populated with spam apps, which reduce the users' quality of experience and increase the workload of app market operators. Apps can be "spammy" in multiple ways including not having a specific functionality, unrelated app description or unrelated keywords and publishing similar apps several times and across diverse categories. Market operators maintain anti-spam policies and apps are removed through continuous human intervention. Through a systematic crawl of a popular app market and by identifying a set of removed apps, we propose a method to detect spam apps solely using app metadata available at the time of publication. We first propose a methodology to manually label a sample of removed apps, according to a set of checkpoint heuristics that reveal the reasons behind removal. This analysis suggests that approximately 35% of the apps being removed are very likely to be spam apps. We then map the identified heuristics to several quantifiable features and show how distinguishing these features are for spam apps. Finally, we build an Adaptive Boost classifier for early identification of spam apps using only the metadata of the apps. Our classifier achieves an accuracy over 95% with precision varying between 85%-95% and recall varying between 38%-98%. By applying the classifier on a set of apps present at the app market during our crawl, we estimate that at least 2.7% of them are spam apps.

Interactive branched video that allows users to select their own paths through the video, provides creative content designers with great personalization opportunities; however, such video also introduces significant new challenges for the system developer. For example, without careful prefetching and buffer management, the use of multiple alternative playback paths can easily result in playback interruptions. In this paper, we present a full implementation of an interactive branched video player using HTTP-based Adaptive Streaming (HAS) that provides seamless playback even when the users defer their branch path choices to the last possible moment. Our design includes optimized prefetching policies that we derive under a simple optimization framework, effective buffer management of prefetched data, and the use of parallel TCP connections to achieve efficient buffer workahead. Through performance evaluation under a wide range of scenarios, we show that our optimized policies can effectively prefetch data of carefully selected qualities along multiple alternative paths such as to ensure seamless playback, offering users a pleasant viewing experience without playback interruptions.

With Online Social Networking (OSN) services coming to prominence as a major platform for the popularization of Web content, recent research has focused on using information gleaned from these services to improve the performance of Content Distribution Networks (CDN). In this paper we find that contrary to popular belief, the simple cache replacement policy LRU outperforms methods based on OSN input. To establish this we propose an OSN-assisted caching strategy which we evaluate using a large Twitter dataset and through simulation. Our results show that vanilla LRU mostly outperforms the OSN-aided mechanisms. We study the reasons for this and find that if at any time there is a large enough percentage of highly popular content, which becomes popular within a short enough time span, CDN can do well by just following an LRU principle. While we do not claim generalizability to other workloads, or other caching architectures on the Internet, our results are important as they apply to the common use-case and show that simple algorithms do quite well.

We present the first comprehensive characterization of the diffusion of ideas on Twitter, studying more than 5.96 million topics that include both popular and less popular topics. On a data set containing approximately 10 million users and a comprehensive scraping of 196 million tweets, we perform a rigorous temporal and spatial analysis, investigating the time-evolving properties of the subgraphs formed by the users discussing each topic. We focus on two different notions of the spatial: the network topology formed by follower-following links on Twitter, and the geospatial location of the users. We investigate the effect of initiators on the popularity of topics and find that users with a high number of followers have a strong impact on topic popularity. We deduce that topics become popular when disjoint clusters of users discussing them begin to merge and form one giant component that grows to cover a significant fraction of the network. Our geospatial analysis shows that highly popular topics are those that cross regional boundaries aggressively.

This paper presents new results on characterization and modeling of user-generated video popularity evolution, based on a recent complementary data collection for videos that were previously the subject of an eight month data collection campaign during 2008/09. In particular, during 2011, we collected two contiguous months of weekly view counts for videos in two separate 2008/09 datasets, namely the ``recently-uploaded'' and the ``keyword-search'' datasets. These datasets contain statistics for videos that were uploaded within 7 days of the start of data collection in 2008 and videos that were discovered using a keyword search algorithm in 2008, respectively. Our analysis shows that the average weekly view count for the recently-uploaded videos had not decreased by the time of the second measurement period, in comparison to the middle and later portions of the first measurement period. The new data is used to evaluate the accuracy of a previously proposed model for synthetic view count generation for time periods that are substantially longer than previously considered. We find that the model yielded distributions of total (lifetime) video view counts that match the empirical distributions, however, significant differences between the model and empirical data were observed with respect to other metrics. These differences appear to arise because of particular popularity characteristics that change over time rather than being week-invariant as assumed in the model.

HTTP-based Adaptive Streaming (HAS) has become a widely-used video delivery technology. Use of HTTP enables relatively easy firewall/NAT traversal and content caching. While caching is an important aspect of HAS, there is not much public research on the performance impact proxies and their policies have on HAS. In this paper we build an experimental framework using open source Squid proxies and the most recent Open Source Media Framework (OSMF). A range of content-aware policies can be implemented in the proxies and tested, while the player software can be instrumented to measure performance as seen at the client. Using this framework, the paper makes three main contributions. First, we present a scenario-based performance evaluation of the latest version of the OSMF player. Second, we quantify the benefits using different proxy-assisted solutions, including basic best effort policies and more advanced content quality aware prefetching policies. Finally, we present and evaluate a cooperative framework in which clients and proxies share information to improve performance. In general, the bottleneck location and network conditions play central roles in which policy choices are most advantageous, as they significantly impact the relative performance differences between policy classes. We conclude that careful design and policy selection is important when trying to enhance HAS performance using proxy assistance.

Web-based catch-up TV has revolutionised watching habits as it provides users the opportunity to watch programs at their preferred time and place, using a variety of devices. With the increasing offer of TV content, there is an emergent need for personalised recommendation solutions, which help users to select programs of interest. In this work, we study the watching patterns of users of an Australian nation-wide catch-up TV service provider and develop a suite of approaches for a catch-up recommendation scenario. We evaluate these approaches using a new large-scale dataset gathered by the Web-based catch-up portal deployed by the provider. The evaluation allows us to compare the performance of several recommenders that address the discovery of both TV programs already watched by users and new programs that users may find relevant.

Video dissemination through sites such as YouTube can have widespread impacts on opinions, thoughts, and cultures. Not all videos will reach the same popularity and have the same impact. Popularity differences arise not only because of differences in video content, but also because of other "content-agnostic" factors. The latter factors are of considerable interest but it has been difficult to accurately study them. For example, videos uploaded by users with large social networks may tend to be more popular because they tend to have more interesting content, not because social network size has a substantial direct impact on popularity. In this paper, we develop and apply a methodology that is able to accurately assess, both qualitatively and quantitatively, the impacts of various content-agnostic factors on video popularity. When controlling for video content, we observe a strong linear "rich-get-richer" behavior, with the total number of previous views as the most important factor except for very young videos. The second most important factor is found to be video age. We analyze a number of phenomena that may contribute to rich-get-richer, including the first-mover advantage, and search bias towards popular videos. For young videos we find that factors other than the total number of previous views, such as uploader characteristics and number of keywords, become relatively more important. Our findings also confirm that inaccurate conclusions can be reached when not controlling for content.

One highly-scalable approach to content delivery is to harness the upload bandwidth of the clients. Peer-assisted content delivery systems have been shown to effectively offload the servers of popular files, as the request rates of popular content enable the formation of self-sustaining torrents, where the entire content of the file is available among the peers themselves. However, for less popular files, these systems are less helpful in offloading servers. With a long tail of mildly popular content, with a high aggregate demand, a large fraction of the file requests must still be handled by servers. In this paper, we present the design, implementation, and evaluation of a dynamic file bundling system, where peers are requested to download content which they may not otherwise download in order to “inflate” the popularity of less popular files. Our system introduces the idea of a super bundle, which consists of a large catalogue of files. From this catalogue, smaller bundles, consisting of a small set of files, can dynamically be assigned to individual users. The system can dynamically adjust the number of downloaders of each file and thus enables the popularity inflation to be optimized according to current file popularities and the desired tradeoff between download times and server resource usage. The system is evaluated on PlanetLab.

With the proliferation of cloud services, cloud-based systems can become a cost-effective means of on-demand content delivery. In order to make best use of the available cloud bandwidth and storage resources, content distributors need to have a good understanding of the tradeoffs between various system design choices. In this work we consider a peer-assisted content delivery system that aims to provide guaranteed average download rate to its customers. We show that bandwidth demand peaks for contents with moderate popularity, and identify these contents as candidates for cloud-based service. We then consider dynamic content bundling (inflation) and cross-swarm seeding, which were recently proposed to improve download performance, and evaluate their impact on the optimal choice of cloud service use. We find that much of the benefits from peer seeding can be achieved with careful torrent inflation, and that hybrid policies that combine bundling and peer seeding often reduce the delivery costs by 20% relative to only using seeding. Furthermore, all these peer-assisted policies reduce the number of files that would need to be pushed to the cloud. Finally, we show that careful system design is needed if locality is an important criterion when choosing cloud-based service provisioning.

This paper develops analytical models that characterize the behavior of on-demand stored media content delivery using BitTorrent-like protocols. The models capture the effects of different piece selection policies, including Rarest-First, two variants of In-Order, and two probabilistic policies (Portion and Zipf). Our models provide insight into system behavior and help explain the sluggishness of the system with In-Order streaming. We use the models to compare different retrieval policies across a wide range of system parameters, including peer arrival rate, upload/download bandwidth, and seed residence time. We also provide quantitative results on the startup delays and retrieval times for streaming media delivery. Our results provide insights into the design tradeoffs for on-demand media streaming in peer-to-peer networks. Finally, the models are validated using simulations.

Peer-assisted content delivery systems can provide scalable download service for popular files. For mildly popular content, however, these systems are less helpful in offloading servers as the request rate for less popular files may not enable formation of self-sustaining torrents (where the entire content of the file is available among the peers themselves). As there typically is a long tail of mildly popular content, with a high aggregate demand, a large fraction of the file requests must still be handled by servers, and is not off-loadable to peers. Bundling approaches have been proposed where peers are requested to download content which they may not otherwise be interested in order to ``inflate'' the popularity of less popular files. We present the design and implementation of a dynamic bundling system, in which a large number of files may be bundled to form a super bundle. From this super bundle, smaller individual bundles, consisting of a small set of files, can dynamically be assigned to individual users. Our system has the capability to dynamically adjust the number of downloaders of each file, thus allowing popularity inflation to be optimized according to current file popularities.

Today’s Web provides many different functionalities, including communication, entertainment, social networking, and information retrieval. In this article, we analyze traces of HTTP activity from a large enterprise and from a large university to identify and characterize Web-based service usage. Our work provides an initial methodology for the analysis of Web-based services. While it is nontrivial to identify the classes, instances, and providers for each transaction, our results show that most of the traffic comes from a small subset of providers, which can be classified manually. Furthermore, we assess both qualitatively and quantitatively how the Web has evolved over the past decade, and discuss the implications of these changes.

Video sharing services that allow ordinary Web users to upload video clips of their choice and watch video clips uploaded by others have recently become very popular. This article identifies invariants in video sharing workloads, through comparison of the workload characteristics of four popular video sharing services. Our traces contain metadata on approximately 1.8 million videos which together have been viewed approximately 6 billion times. Using these traces, we study the similarities and differences in use of several Web 2.0 features such as ratings, comments, favorites, and propensity of uploading content. In general, we find that active contribution, such as video uploading and rating of videos, is much less prevalent than passive use. While uploaders in general are skewed with respect to the number of videos they upload, the fraction of multi-time uploaders is found to differ by a factor of two between two of the sites. The distributions of lifetime measures of video popularity are found to have heavy-tailed forms that are similar across the four sites. Finally, we consider implications for system design of the identified invariants. To gain further insight into caching in video sharing systems, and the relevance to caching of lifetime popularity measures, we gathered an additional dataset tracking views to a set of approximately 1.3 million videos from one of the services, over a twelve-week period. We find that lifetime popularity measures have some relevance for large cache (hot set) sizes (i.e., a hot set defined according to one of these measures is indeed relatively “hot”), but that this relevance substantially decreases as cache size decreases, owing to churn in video popularity.

In this paper, we demonstrate that anomalies in Internet traces can have a significant impact on semi-experiments that are designed to determine the causes of scaling behavior of traffic. A semi-experiment involves artificially modifying a specific aspect of a trace and studying the resulting change in scaling behavior. We demonstrate using MAWI traces that semi-experiments performed without addressing the presence of anomalies give insights that contradict widely accepted theories regarding Internet traffic scaling behavior. For example, a direct semi-experimental analysis seems to suggest that removing large flows does not result in the removal of LRD behavior and that the scaling behavior of MAWI traces is the same before and after the removal of the large flows. This observation hence challenges the well-known hypothesis that the heavy-tailed distribution of flow sizes is the primary factor causing correlation at large time-scales. To mitigate the impact of anomalies, we couple the semi-experiments with a recently proposed sketch-based procedure for robust estimation of scaling behavior. We term these ``robust semi-experiments". Our analysis shows that using a robust estimation procedure enables a meaningful semi-experimental analysis and that the conclusions drawn from the robust semi-experiments agree with well-established theories regarding Internet traffic scaling behavior.

Many factors such as the tendency of individuals to develop relationships based on mutual acquaintances, proximity, common interests, or combinations thereof, are known to contribute toward evolution of social networks. In this paper, we analyze an evolving online social aggregator FriendFeed, which collates content generated by participating individuals on a variety of Web 2.0 services and allows easy dissemination of the aggregated content to other participants of the aggregator. Analyzing data collected between September 2008 and May 2009, we find that although preferential attachment captures the evolution of the network, its influence varies significantly based on how long ago a user joined the service. In particular, preferential attachment does not appear to apply to new entrants of the FriendFeed service. Analysis suggests that proximity bias plays an important role in link formation. We study the influence of common foci and find that individuals have a greater affinity toward those with similar interests.

The immense popularity of new-age ¿Web 2.0¿ applications such as YouTube, Flickr, and Facebook, and non-Web applications such as peer-to-peer (P2P) file sharing, voice over IP, online games, and media streaming have significantly altered the composition of Internet traffic with respect to what it was a few years ago. In light of these changes, this paper revisits Internet traffic characteristics and models that were proposed when ¿traditional¿ Web traffic was the largest contributor to Internet traffic. Specifically, we study whether or not the following characteristics, namely: (1) traffic is self-similar and long-range dependent, and (2) traffic can be approximated by Poisson at smaller time scales, are still valid. Our experiments on recent traces show that these traffic characteristics continue to hold. We further argue that current Internet traffic can be viewed to have two key constituents, namely Web+ and P2P+; Web+ traffic consists of traffic from both Web 1.0 and Web 2.0 applications; P2P+ traffic consists largely of traffic from P2P applications and other non-Web applications excluding applications on well-known ports such as FTP and SMTP. We then show that both Web+ and P2P+ components exhibit self-similar behavior and can be approximated by Poisson at smaller time scales.

No abstract available.

Realizing on-demand media streaming in a peer-to-peer (P2P) fashion is more challenging than in the case of live media streaming, since only peers with close-by media play progresses may help each other in obtaining the media content. The situation is further complicated if we wish to pursue low aggregated link cost in the transmission. In this paper, we present a new algorithmic perspective toward on-demand P2P streaming protocol design. While previous approaches employ streaming trees or passive neighbor reconciliation for media content distribution, we instead coordinate the streaming session as an auction where each peer participates locally by bidding for and selling media flows encoded with network coding. We show that this auction approach is promising in achieving low-cost on-demand streaming in a scalable fashion. It is amenable to asynchronous, distributed, and lightweight implementations, and is flexible to provide support for random-seek and pause functionalities. Through extensive simulation studies, we verify the effectiveness and performance of the proposed auction approach, focusing on the optimality in overall streaming cost, the convergence speed, and the communication overhead.

This paper develops analytic models that characterize the behavior of on-demand stored media content delivery using BitTorrent-like protocols. The models capture the effects of different piece selection policies, including Rarest-First and two variants of In-Order. Our models provide insight into transient and steady-state system behavior, and help explain the sluggishness of the system with strict In-Order streaming. We use the models to compare different retrieval policies across a wide range of system parameters, including peer arrival rate, upload/download bandwidth, and seed residence time. We also provide quantitative results on the startup delays and retrieval times for streaming media delivery. Our results provide insights into the optimal design of peer-to-peer networks for on-demand media streaming.

Peer-to-Peer (P2P) applications continue to grow in popularity, and have reportedly overtaken Web applications as the single largest contributor to Internet traffic. Using traces collected from a large edge network, we conduct an extensive analysis of P2P traffic, compare P2P traffic with Web traffic, and discuss the implications of increased P2P traffic. In addition to studying the aggregate P2P traffic, we also analyze and compare the two main constituents of P2P traffic in our data, namely BitTorrent and Gnutella. The results presented in the paper may be used for generating synthetic workloads, gaining insights into the functioning of P2P applications, and developing network management strategies. For example, our results suggest that new models are necessary for Internet traffic. As a first step, we present flow-level distributional models for Web and P2P traffic that may be used in network simulation and emulation experiments.

Conventional video consists of a single sequence of video frames. During a client's playback period, frames are viewed sequentially from some specified starting point. The fixed frame ordering of conventional video enables efficient scheduled broadcast delivery, as well as efficient near on-demand delivery to large numbers of concurrent clients through use of periodic broadcast protocols in which the video file is segmented and transmitted on multiple channels. This paper considers the problem of devising scalable protocols for near on-demand delivery of "nonlinear" media files whose content may have a tree or graph, rather than linear, structure. Such media allows personalization of the media playback according to individual client preferences. We formulate a mathematical model for determination of the optimal periodic broadcast protocol for nonlinear media with piecewise-linear structures. Our objective function allows differing weights to be placed on the startup delays required for differing paths through the media. Studying a number of simple nonlinear structures we provide insight into the characteristics of the optimal solution. For cases in which the cost of solving the optimization model is prohibitive, we propose and evaluate an efficient approximation algorithm.

Periodic broadcast protocols enable efficient streaming of highly popular media files to large numbers of concurrent clients. Most previous periodic broadcast protocols, however, assume that all clients can receive at the same rate, and also assume that reception bandwidth is not time-varying. In this article, we first develop a new periodic broadcast protocol, Optimized Heterogeneous Periodic Broadcast (OHPB), that can be optimized for a given population of clients with heterogeneous reception bandwidths and quality-of-service requirements. The OHPB protocol utilizes an optimized segment size progression determined by solving a linear optimization model that takes as input the client population characteristics and an objective function such as mean client startup delay. We then develop a generalization of the OHPB linear optimization model that allows optimal server bandwidth allocation among multiple concurrent OHPB broadcasts, wherein each media file and its clients may have different characteristics. Finally, we propose complementary client protocols employing work-ahead buffering of data during playback, so as to enable more uniform playback quality when the reception bandwidth is time-varying.

This paper presents a traffic characterization study of the popular video sharing service, YouTube. Over a three month period we observed almost 25 million transactions between users on an edge network and YouTube, including more than 600,000 video downloads. We also monitored the globally popular videos over this period of time.

In this paper we study the use of TCP Vegas as a transport protocol for streaming media. We also consider TCP NewReno as a transport protocol for streaming media. We find that: 1) TCP is able to transport streaming media with good performance in a wide variety of scenarios; 2) TCP Vegas performs better than TCP NewReno in many cases; and 3) for viable media streams, both TCP variants need to achieve mean throughputs that are at least 1.5 times the encoding rate of the media objects being carried.

Wireless local area networks (WLANs) have become omnipresent: WLANs are available at airports, coffee shops, university campuses, corporate environments, and homes. This surge in the popularity of WLANs motivates the study of how these networks are used. Characterizing WLANs, however, is complicated by a number of factors including the geographic diversity of WLAN deployments and the need for capturing activity in the wireless environment instead of the wired environment. In this paper, we describe our experiences with the deployment and use of a remote passive wireless-side measurement infrastructure for monitoring usage of WLANs, and compare our results with a commonly used wired-side measurement technique.

No abstract available.

Traffic classification is the ability to identify and categorize network traffic by application type. In this paper, we consider the problem of traffic classification in the network core.Classification at the core is challenging because only partial information about the flows and their contributors is available. We address this problem by developing a framework that can classify a flow using only unidirectional flow information. We evaluated this approach using recent packet traces that we collected and pre-classified to establish a "base truth". From our evaluation, we find that flow statistics for the server-to-client direction of a TCP connection provide greater classification accuracy than the flow statistics for the client-to-server direction. Because collection of the server-to-client flow statistics may not always be feasible, we developed and validated an algorithm that can estimate the missing statistics froma unidirectional packet trace.

We apply an unsupervised machine learning approach for Internet traffic identification and compare the results with that of a previously applied supervised machine learning approach. Our unsupervised approach uses an expectation maximization (EM) based clustering algorithm and the supervised approach uses the naive Bayes classifier. We find the unsupervised clustering technique has an accuracy up to 91% and outperform the supervised technique by up to 9%. We also find that the unsupervised technique can be used to discover traffic from previously unknown applications and has the potential to become an excellent tool for exploring Internet traffic.

In this paper, we compare the throughputs of two different TCP NewReno variants, namely Slow-but-Steady and Impatient. We develop analytic throughput models of these variants as a function of round-trip time, loss event rate, and the burstiness of packet drops within a loss event. Our models build upon prior work on TCP Reno throughput modeling, but extend this work to provide an analytical characterization of the NewReno fast recovery algorithms. We validated our models using the ns-2 simulator. Our models accurately predict the steady-state NewReno throughput for a wide range of loss rates. Based on these models, we analytically determine the preferred operating regions for each TCP variant. Our results show that the Slow-but-Steady variant is comparable to or superior to the Impatient variant in all but the most extreme scenarios for network packet loss.

Periodic broadcast protocols enable the efficient streaming of highly popular media files to large numbers of concurrent clients. Most previous periodic broadcast protocols, however, assume that all clients can receive at the same rate, and also assume that available bandwidth is not time-varying. In this paper, we first develop a new periodic broadcast protocol, Optimized Heterogeneous Periodic Broadcast (OHPB), that can be optimized for a given population of clients with heterogeneous reception bandwidths and quality-of-service requirements. The OHPB protocol utilizes an optimized segment size progression determined by solving a linear optimization model that takes as input the client population characteristics and an objective function such as mean client startup delay. We then propose complementary client protocols employing work-ahead buffering of data during playback, so as to enable more uniform playback quality when the available bandwidth is time-varying.

Realizing on-demand media streaming in a Peer-to-Peer (P2P) fashion is more challenging than in the case of live media streaming, since only peers with close-by media play progresses may help each other in obtaining the media content. The situation is further complicated if we wish to pursue low link cost in the transmission. In this paper, we present a new algorithmic perspective towards on-demand P2P streaming protocol design. While previous approaches employ streaming trees or passive neighbour reconciliation for media content distribution, we instead coordinate the streaming session as an auction where each peer participates locally by bidding for and selling media flows encoded with network coding. We show that this auction approach is promising in achieving low-cost on-demand streaming in a scalable fashion. It is amenable to asynchronous, distributed, and light-weight implementations, and is flexible enough to provide support for random-seek and pause functionalities.

Previous scalable on-demand streaming protocols do not allow clients to recover from packet loss. This paper develops new protocols that: (1) have a tunably short latency for the client to begin playing the media; (2) allow heterogeneous clients to recover lost packets without jitter as long as each client's cumulative loss rate is within a tunable threshold; and (3) assume a tunable upper bound on the transmission rate to each client that can be as small as a fraction (e.g., 25%) greater than the media play rate. Models are developed to compute the minimum required server bandwidth for a given loss rate and playback latency. The results of the models are used to develop the new protocols and assess their performance. The new protocols, Reliable Periodic Broadcast and Reliable Bandwidth Skimming, are simple to implement and achieve nearly the best possible scalability and efficiency for a given set of client characteristics and desirable/feasible media quality. Furthermore, the results show that the new reliable protocols that transmit to each client at only twice the media play rate have similar performance to previous protocols that require clients to receive at many times the play rate.

Inspired by recent techniques for reliable bulk data distribution, this paper develops scalable protocols for reliable on-demand delivery of streaming media. Models are developed that quantify the best possible scalability for given client characteristics. The results of the models are used to guide the design and assess the performance of the proposed streaming techniques. The new protocols, RPB and RBS, are relatively simple to implement and achieve nearly the best possible scalability and efficiency for a given set of client characteristics and desirable/feasible media quality.