Packet corrupting devices can cause bitflips and data corruption, and, depending on exactly where the corruption occurs, can evade even strong link-level checks. Even though such instances are probabilistically rare, they are all too common at hyper-scale. At Meta, they have been causing end-to-end errors for years in our datacenter fleet, resulting in dozens of instances of performance impact to services, and rare instances of data corruption. Over the years, we have developed three tools that have progressively improved our ability to localize this problem to specific hosts, rack switches, links, and even specific ports on those switches. They include a fleet wide passive monitoring service to track Layer-4 checksum errors detected by hosts, an active monitoring and fault localization system to probe for corruption on devices and localize them to individual links at any layer of the topology; and a loopback test to reproduce and confirm packet corruption on specific ports on localized switches. Together, they comprise Flipperino, a system that detects and localizes packet corrupting devices, making it easy to identify them for remediation. We share our experience from investigations, and recent results from Flipperino, which has localized devices causing 97% of Layer-4 checksum errors. We have also reproduced corruption on 8 switches in the last 3 months, including 7 rack switches and a fabric switch.

In datacenters, common incast traffic patterns are challenging because they violate the basic premise of bandwidth stability on which TCP congestion control convergence is built, overwhelming shallow switch buffers and causing packet losses and high latency. To understand why these challenges remain despite decades of research on datacenter congestion control, we conduct an in-depth investigation into high-degree incasts both in production workloads at Meta and in simulation. In addition to characterizing the bursty nature of these incasts and their impacts on the network, our findings demonstrate the shortcomings of widely deployed window-based congestion control techniques used to address incast problems. Furthermore, we find that hosts associated with a specific application or service exhibit similar and predictable incast traffic properties across hours, pointing the way toward solutions that predict and prevent incast bursts, instead of reacting to them.

Managing the performance of thousands of services across millions of servers demands a networking stack that can dynamically adjust protocol settings to match diverse priorities and network characteristics. Moreover, given the constantly evolving nature of services and their requirements, the set of configurable protocols must remain adaptable. However, current host networking stacks lack the necessary flexibility and adaptability. Although eBPF shows promise in this regard, it lacks essential primitives for efficient development and safe deployment of multiple co-existing services.

Managing data center networks with low loss requires understanding traffic dynamics at short (millisecond) time-scales, especially the burstiness of traffic, and to what extent bursts contend for switch buffer resources. Yet, monitoring traffic over such intervals is a challenge at scale.

Weather is a leading threat to the stability of our vital infrastructure. Last-mile Internet is no exception. Yet, unlike other vital infrastructure, weather's effect on last-mile Internet outages is not well understood. This work is the first attempt to quantify the effect of weather on residential outages.

People use recommender systems to improve their decisions; for example, item recommender systems help them find films to watch or books to buy. Despite the ubiquity of item recommender systems, they can be improved by giving users greater transparency and control. This article develops and assesses interactive strategies for transparency and control, as applied to event sequence recommender systems, which provide guidance in critical life choices such as medical treatments, careers decisions, and educational course selections. This article’s main contribution is the use of both record attributes and temporal event information as features to identify similar records and provide appropriate recommendations. While traditional item recommendations are based on choices by people with similar attributes, such as those who looked at this product or watched this movie, our event sequence recommendation approach allows users to select records that share similar attribute values and start with a similar event sequence. Then users see how different choices of actions and the orders and times between them might lead to users’ desired outcomes. This paper applies a visual analytics approach to present and explain recommendations of event sequences. It presents a workflow for event sequence recommendation that is implemented in EventAction and reports on three case studies in two domains to illustrate the use of generating event sequence recommendations based on personal histories. It also offers design guidelines for the construction of user interfaces for event sequence recommendation and discusses ethical issues in dealing with personal histories. A demo video of EventAction is available at https://hcil.umd.edu/eventaction.

Recommendation applications can guide users in making important life choices by referring to the activities of similar peers. For example, students making academic plans may learn from the data of similar students, while patients and their physicians may explore data from similar patients to select the best treatment. Selecting an appropriate peer group has a strong impact on the value of the guidance that can result from analyzing the peer group data. In this article, we describe a visual interface that helps users review the similarity and differences between a seed record and a group of similar records and refine the selection. We introduce the LikeMeDonuts, Ranking Glyph, and History Heatmap visualizations. The interface was refined through three rounds of formative usability evaluation with 12 target users, and its usefulness was evaluated by a case study with a student review manager using real student data. We describe three analytic workflows observed during use and summarize how users’ input shaped the final design.

Internet anycast depends on inter-domain routing to direct clients to their "closest" sites. Using data collected from a root DNS server for over a year (400M+ queries/day from 100+ sites), we characterize the load balancing and latency performance of global anycast. Our analysis shows that site loads are often unbalanced, and that most queries travel longer than necessary, many by over 5000 km.

Engaging undergraduates in research has been shown to improve retention, increase students' sense of science identity, and increase the chances that they will continue to graduate school. Yet many undergraduates don't participate in research until very late in their undergraduate program, while most undergraduates don't participate in research at all. On the other hand, many faculty are eager and willing to do research with undergraduates, but are unsure how to mentor and supervise them, particularly early undergraduates who may have very little specific technical knowledge and skills. This workshop will provide participants with concrete skills and techniques for engaging early undergraduates (first and second-year students) in real research projects, and, if desired, for developing or growing a department-wide early undergraduate research program. Participants will engage in hands-on activities where they will learn how to develop appropriately scoped research projects, manage and mentor early undergraduates successfully, and teach core research skills like reading research papers and writing research proposals. The workshop will also cover how to mitigate specific challenges faced by students from groups underrepresented in computer science. The materials presented in this workshop are based on the successful NSF-funded Early Research Scholars Program at UC San Diego (ersp.ucsd.edu), which is in its fourth year, and engages 40 second-year students per year--the majority of whom are women and/or students from underrepresented racial or ethnic groups--in academic-year research apprenticeships.

People often seek examples of similar individuals to guide their own life choices. For example, students making academic plans refer to friends; patients refer to acquaintances with similar conditions, physicians mention past cases seen in their practice. How would they want to search for similar people in databases? We discuss the challenge of finding similar people to guide life choices and report on a need analysis based on 13 interviews. Our PeerFinder prototype enables users to find records that are similar to a seed record, using both record attributes and temporal events found in the records. A user study with 18 participants and four experts shows that users are more engaged and more confident about the value of the results to provide useful evidence to guide life choices when provided with more control over the search process and more context for the results, even at the cost of added complexity.

Reverse DNS (rDNS) names often contain information not only about core routers in some networks but about hosts on edge networks. Although the rDNS names of routers have been decoded and their naming patterns verified by researchers, the information extracted from edge host rDNS names has been limited to intuitive keywords. In this paper, we develop a methodology for identifying and analyzing rDNS names for broadband Internet end hosts without using any pre-defined keyword. With this methodology, we study whether ISPs use intuitive keywords in their rDNS names and whether rDNS names embed correct information (e.g., whether keyword "cable" indicates cable clients but not fiber or DSL clients).

Tor is a peer-to-peer overlay routing network that achieves unlinkable communication between source and destination.Unlike traditional mix-nets, Tor seeks to balance anonymity and performance, particularly with respect to providing low-latency communication. As a result, understanding the latencies between peers in the Tor network could be an extremely powerful tool in understanding and improving Tor's performance and anonymity properties. Unfortunately, there are no practical techniques for inferring accurate latencies between two arbitrary hosts on the Internet, and Tor clients are not instrumented to collect and report on these measurements. In this paper, we present Ting, a technique for measuring latencies between arbitrary Tor nodes from a single vantage point. Through a ground-truth validation, we show that Ting is accurate, even with few samples, and does not require modifications to existing clients. We also apply Ting to the live Tor network, and show that its measurements are stable over time. We demonstrate that the all-pairs latency datasets that Ting permits can be applied in disparate ways, including faster methods of deanonymizing Tor circuits and efficiently finding long circuits with low end-to-end latency.

Active probing techniques, such as ping, have been used to detect outages. When a previously responsive end host fails to respond to a probe, studies sometimes attempt to confirm the outage by retrying the ping or attempt to identify the location of the outage by using other tools such as traceroute. The latent problem, however, is, how long should one wait for a response to the ping? Too short a timeout risks confusing congestion or other delay with an outage. Too long a timeout may slow the process and prevent observing and diagnosing short-duration events, depending on the experiment's design.

There are several mechanisms by which users can gain insight into where their packets have gone, but no mechanisms allow users undeniable proof that their packets did not traverse certain parts of the world while on their way to or from another host. This paper introduces the problem of finding "proofs of avoidance": evidence that the paths taken by a packet and its response avoided a user-specified set of "forbidden" geographic regions. Proving that something did not happen is often intractable, but we demonstrate a low-overhead proof structure built around the idea of what we call "alibis": relays with particular timing constraints that, when upheld, would make it impossible to traverse both the relay and the forbidden regions.

The ability to revoke certificates is a fundamental feature of a public key infrastructure. However, certificate revocation systems are generally regarded as ineffective and potentially insecure: Some browsers bundle revocation updates with more general software updates, and may go hours, days, or indefinitely between updates; moreover, some operating systems make it difficult for users to demand recent revocation data. This paper argues that this sad state of affairs is an inexorable consequence of relying on unicast communication to distribute revocation information. We present RevCast, a broadcast system that disseminates revocation data in a timely and private manner. RevCast is not emulated broadcast over traditional Internet links, but rather a separate metropolitan-area wireless broadcast link; specifically, we have designed RevCast to operate over existing FM radio, although the principles apply to alternative implementations. We present the design, implementation, and initial deployment of RevCast on a 3 kW commercial radio station using the FM RDS protocol. With the use of two types of receivers (an RDS-to-LAN bridge that we have prototyped and an RDS-enabled smartphone), we show that, even at a low bitrate, RevCast is able to deliver complete and timely revocation information, anonymously, even for receivers who do not receive all packets all the time.

On January 3, 2013, the D-root DNS server hosted at the University of Maryland changed IP address. To avoid service disruption, the old address continues to answer queries. In this paper, we perform an initial investigation of the traffic at both the new and old addresses before, during, and since the flag day. The data we collected show non-obvious behavior: the overall query volume to the D-roots increases by roughly 50%, the old address continues to receive a high volume of queries months after the changeover, and far more queries to the old address succeed than those to the new one. Our analysis provides a window into how compliant resolvers change over and how non-standard and seemingly malicious resolvers react (or not) to the IP address change. We provide evidence that a relatively small number of implementation errors account for nearly all discrepancies that are not misconfigurations or attacks.

Residential Internet connections are susceptible to weather-caused outages: Lightning and wind cause local power failures, direct lightning strikes destroy equipment, and water in the atmosphere degrades satellite links. Outages caused by severe events such as fires and undersea cable cuts are often reported upon by operators and studied by researchers. In contrast, outages cause by ordinary weather are typically limited in scope, and because of their small scale, there has not been comparable effort to understand how weather affects everyday last-mile Internet connectivity. We design and deploy a measurement tool called ThunderPing that measures the connectivity of residential Inter- net hosts before, during, and after forecast periods of severe weather. ThunderPing uses weather alerts from the US National Weather Service to choose a set of residential host addresses to ping from several vantage points on the Internet. We then process this ping data to determine when hosts lose connectivity, completely or partially, and categorize whether these failures occur during periods of severe weather or when the skies are clear. In our preliminary results, we find that compared to clear weather, failures are four times as likely during thunderstorms and two times as likely during rain. We also find that the duration of weather induced outages is relatively small for a satellite provider we focused on.

Network-level redundancy elimination (RE) algorithms reduce traffic volume on bandwidth-constrained network paths by avoiding the transmission of repeated byte sequences. Previous work shows that RE can suppress the transmission of 20-50% bytes when deployed at ISP access links or between routers. In this paper, we focus on the challenges of deploying RE in cellular networks. The potential benefifit is substantial, since cellular networks have a growing subscriber base and network links, including wired backhaul, are often oversubscribed. Using three large traces captured at two North American and one European wireless network providers, we show that RE can reduce the bandwidth consumption of the majority of mobile users by at least 10%.

Cellular radios consume more power and suffer reduced data rate when the signal is weak. According to our measurements, the communication energy per bit can be as much as 6x higher when the signal is weak than when it is strong. To realize energy savings, applications must preferentially communicate when the signal is strong, either by deferring non-urgent communication or by advancing anticipated communication to coincide with periods of strong signal. Allowing applications to perform such scheduling requires predicting signal strength, so that opportunities for energy-efficient communication can be anticipated. Furthermore, such prediction must be performed at little energy cost.

Cellular radio communication is a significant contributor to battery energy drain on smartphones, in some cases inflating the energy cost by a factor of 5 or more compared to the energy cost of the base device. Stratus is a system to reduce this energy consumption by leveraging cloud resources to make data communication on smartphones more efficient. Using a cloud-based proxy, Stratus employs optimizations that adapt an application's incoming and outgoing traffic to better match the energy characteristics of the radio interface. The optimizations include (a) aggregation to bunch up sporadic transmissions, (b) asymmetric dictionary-based compression to reduce the number of bits transmitted over the air, and (c) opportunistic scheduling to avoid communication during periods of poor signal reception. These optimizations can be used individually, or in combination, subject to an application's delay tolerance. For example, using our Stratus prototype, the aggregation and compression optimizations together achieve up to 50% energy savings for web browsing, while the aggregation and scheduling optimizations together achieve up to 35% energy savings for a media streaming application.

All in-text\treferences\tunderlined\tin\tblue\tare\tlinked\tto\tpublications\ton\tResearchGate, letting you\taccess\tand\tread\tthem\timmediately.

In this paper, we present DHTBL, an anti-spam blacklist built upon a novel secure distributed hash table (DHT). We show how DHTBL can be used to replace existing DNS-based blacklists (DNSBLs) of IP addresses of mail relays that forward spam. Implementing a blacklist on a DHT improves resilience to DoS attacks and secures message delivery, when compared to DNSBLs. However, due to the sensitive nature of the blacklist, storing the data in a peer-to-peer DHT would invite attackers to infiltrate the system. Typical DHTs can withstand fail-stop failures, but malicious nodes may provide incorrect routing information, refuse to return published items, or simply ignore certain queries. The neighborhoodwatch DHT is resilient to malicious nodes and maintains the O(logiV) bounds on routing table size and expected lookup time. NeighborhoodWatch depends on two assumptions in order to make these guarantees: (1) the existence of an on-line trusted authority that periodically contacts and issues signed certificates to each node, and (2) for every sequence of k + 1 consecutive nodes in the ID space, at least one is alive and non-malicious. We show how NeighborhoodWatch maintains many of its security properties even when the second assumption is violated. Honest nodes in NeighborhoodWatch can detect malicious behavior and expel the responsible nodes from the DHT.

Online social networks (OSNs) are immensely popular, with some claiming over 200 million users. Users share private content, such as personal information or photographs, using OSN applications. Users must trust the OSN service to protect personal information even as the OSN provider benefits from examining and sharing that information. We present Persona, an OSN where users dictate who may access their information. Persona hides user data with attribute-based encryption (ABE), allowing users to apply fine-grained policies over who may view their data. Persona provides an effective means of creating applications in which users, not the OSN, define policy over access to private data. We demonstrate new cryptographic mechanisms that enhance the general applicability of ABE. We show how Persona provides the functionality of existing online social networks with additional privacy benefits. We describe an implementation of Persona that replicates Facebook applications and show that Persona provides acceptable performance when browsing privacy-enhanced web pages, even on mobile devices.

When analyzing thousands of event histories, analysts often want to see the events as an aggregate to detect insights and generate new hypotheses about the data. An analysis tool must emphasize both the prevalence and the temporal ordering of these events. Additionally, the analysis tool must also support flexible comparisons to allow analysts to gather visual evidence. In a previous work, we introduced align, rank, and filter (ARF) to accentuate temporal ordering. In this paper, we present temporal summaries, an interactive visualization technique that highlights the prevalence of event occurrences. Temporal summaries dynamically aggregate events in multiple granularities (year, month, week, day, hour, etc.) for the purpose of spotting trends over time and comparing several groups of records. They provide affordances for analysts to perform temporal range filters. We demonstrate the applicability of this approach in two extensive case studies with analysts who applied temporal summaries to search, filter, and look for patterns in electronic health records and academic records.

Internet coordinate systems have emerged as an efficient method to estimate the latency between pairs of nodes without any communication between them. However, most coordinate systems have been evaluated solely on data sets built by their authors from measurements gathered over large periods of time. Although they show good prediction results, it is unclear whether the accuracy is the result of the system design properties or is more connected to the characteristics of the data sets. In this paper, we revisit a simple question: how do the features of the embedding space and the inherent attributes of the data sets interact in producing good embeddings? We adapt the Vivaldi algorithm to use Hyperbolic space for embedding and evaluate both Euclidean and Hyperbolic Vivaldi on seven sets of real-world latencies. Our results show that node filtering and latency distributions can significantly influence the accuracy of the predictions. For example, although Euclidean Vivaldi performs well on data sets that were chosen, constructed and filtered by the designers of the algorithm, its performance and robustness decrease considerably when run on third party data sets that were not filtered a priori. Our results offer important insight into designing and building coordinate systems that are both robust and accurate in Internet-like environments.

Mapping the router topology is an important component of Internet measurement. Alias resolution, the process of mapping IP addresses to routers, is critical to accurate Internet mapping. Ally, a popular alias resolution tool, was developed to resolve aliases in individual ISPs, but its probabilistic accuracy and need to send O(n2) probes to infer aliases among n IP addresses make it unappealing for large-scale Internet mapping. In this paper, we present RadarGun, a tool that uses IP identifier velocity modeling to improve the accuracy and scalability of the Ally-based resolution technique. We provide analytical bounds on Ally's accuracy and validate our predicted aliases against Ally. Additionally, we show that velocity modeling requires only O(n) probes and thus scales to Internet-sized mapping efforts.

Incentives play a crucial role in BitTorrent, motivating users to upload to others to achieve fast download times for all peers. Though long believed to be robust to strategic manipulation, recent work has empirically shown that BitTorrent does not provide its users incentive to follow the protocol. We propose an auction-based model to study and improve upon BitTorrent's incentives. The insight behind our model is that BitTorrent uses, not tit-for-tat as widely believed, but an auction to decide which peers to serve. Our model not only captures known, performance-improving strategies, it shapes our thinking toward new, effective strategies. For example, our analysis demonstrates, counter-intuitively, that BitTorrent peers have incentive to intelligently under-report what pieces of the file they have to their neighbors. We implement and evaluate a modification to BitTorrent in which peers reward one another with proportional shares of bandwidth. Within our game-theoretic model, we prove that a proportional-share client is strategy-proof. With experiments on PlanetLab, a local cluster, and live downloads, we show that a proportional-share unchoker yields faster downloads against BitTorrent and BitTyrant clients, and that under-reporting pieces yields prolonged neighbor interest.

Several models have been recently proposed for predicting the latency of end to end Internet paths. These models treat the Internet as a black-box, ignoring its internal structure. While these models are simple, they can often fail systematically; for example, the most widely used models use metric embeddings that predict no benefit to detour routes even though half of all Internet routes can benefit from detours.In this paper, we adopt a structural approach that predicts path latency based on measurements of the Internet's routing topology, PoP connectivity, and routing policy. We find that our approach outperforms Vivaldi, the most widely used black-box model. Furthermore, unlike metric embeddings, our approach successfully predicts 65% of detour routes in the Internet. The number of measurements used in our approach is comparable with that required by black box techniques, but using traceroutes instead of pings.

An accurate router-level topology of the Internet would benefit many research areas, including network diagnosis, inter-domain traffic engineering, and overlay construction. We present TCP Sidecar and Passenger, two elements of a system for router-level Internet topology discovery. Sidecar transparently injects measurement probes into non-measurement TCP streams, while Passenger combines TTL-limited probes with the often-ignored IP record route option. The combined approach mitigates problems associated with traceroute-based topology discovery, including abuse reports, spurious edge inference from multi-path routing, unresolved IP aliases, long network timeouts, and link discovery behind NATs and firewalls. We believe that we are the first mapping project to measure MPLS use with ICMP extensions and record route behavior when the TTL is not decremented. We are able to discover NATs when monitoring TCP connections that tunnel through them. In this paper, we present preliminary results for TCP Sidecar and Passenger on PlanetLab. Our experiments inject measurement probes into traffic generated both from the CoDeeN Web proxy project and from a custom web crawler to 166,745 web sites.

To date, realistic ISP topologies have not been accessible to the research community, leaving work that depends on topology on an uncertain footing. In this paper, we present new Internet mapping techniques that have enabled us to measure router-level ISP topologies. Our techniques reduce the number of required traces compared to a brute-force, all-to-all approach by three orders of magnitude without a significant loss in accuracy. They include the use of BGP routing tables to focus the measurements, the elimination of redundant measurements by exploiting properties of IP routing, better alias resolution, and the use of DNS to divide each map into POPs and backbone. We collect maps from ten diverse ISPs using our techniques, and find that our maps are substantially more complete than those of earlier Internet mapping efforts. We also report on properties of these maps, including the size of POPs, distribution of router outdegree, and the interdomain peering structure. As part of this work, we release our maps to the community.

Researchers have shown that the Internet exhibits path inflation -- end-to-end paths can be significantly longer than necessary. We present a trace-driven study of 65 ISPs that characterizes the root causes of path inflation, namely topology and routing policy choices within an ISP, between pairs of ISPs, and across the global Internet. To do so, we develop and validate novel techniques to infer intra-domain and peering policies from end-to-end measurements. We provide the first measured characterization of ISP peering policies. In addition to "early-exit," we observe a significant degree of helpful non-early-exit, load-balancing, and other policies in use between peers. We find that traffic engineering (the explicit addition of policy constraints on top of topology constraints) is widespread in both intra- and inter-domain routing. However, intra-domain traffic engineering has minimal impact on path inflation, while peering policies and inter-domain routing lead to significant inflation. We argue that the underlying cause of inter-domain path inflation is the lack of BGP policy controls to provide convenient engineering of good paths across ISPs.

Diagnosing faults in the Internet is arduous and time-consuming, in part because the network is composed of diverse components spread across many administrative domains. We consider an extreme form of this problem: can end users, with no special privileges, identify and pinpoint faults inside the network that degrade the performance of their applications? To answer this question, we present both an architecture for user-level Internet path diagnosis and a practical tool to diagnose paths in the current Internet. Our architecture requires only a small amount of network support, yet it is nearly as complete as analyzing a packet trace collected at all routers along the path. Our tool, tulip, diagnoses reordering, loss and significant queuing events by leveraging well deployed but little exploited router features that approximate our architecture. Tulip can locate points of reordering and loss to within three hops and queuing to within four hops on most paths that we measured. This granularity is comparable to that of a hypothetical network tomography tool that uses 65 diverse hosts to localize faults on a given path. We conclude by proposing several simple changes to the Internet to further improve its diagnostic capabilities.

Ensembles of distributed, heterogeneous resources, also known as computational grids, have emerged as critical platforms for high-performance and resource-intensive applications. Such platforms provide the potential for applications to aggregate enormous bandwidth, computational power, memory, secondary storage, and other resources during a single execution. However, achieving this performance potential in dynamic, heterogeneous environments is challenging. Recent experience with distributed applications indicates that adaptivity is fundamental to achieving application performance in dynamic grid environments. The AppLeS (Application Level Scheduling) project provides a methodology, application software, and software environments for adaptively scheduling and deploying applications in heterogeneous, multiuser grid environments. We discuss the AppLeS project and outline our findings.

We describe a novel constraint-based approach to approximate ISP link weights using only end-to-end measurements. Common routing protocols such as OSPF and IS-IS choose least-cost paths using link weights, so inferred weights provide a simple, concise, and useful model of intradomain routing. Our approach extends router-level ISP maps, which include only connectivity, with link weights that are consistent with routing. Our inferred weights agree well with observed routing: while our inferred weights fully characterize the set of shortest paths between 84--99% of the router-pairs, alternative models based on hop count and latency do so for only 47--81% of the pairs.

To date, realistic ISP topologies have not been accessible to the research community, leaving work that depends on topology on an uncertain footing. In this paper, we present new Internet mapping techniques that have enabled us to directly measure router-level ISP topologies. Our techniques reduce the number of required traces compared to a brute-force, all-to-all approach by three orders of magnitude without a significant loss in accuracy. They include the use of BGP routing tables to focus the measurements, exploiting properties of IP routing to eliminate redundant measurements, better alias resolution, and the use of DNS to divide each map into POPs and backbone. We collect maps from ten diverse ISPs using our techniques, and find that our maps are substantially more complete than those of earlier Internet mapping efforts. We also report on properties of these maps, including the size of POPs, distribution of router outdegree, and the inter-domain peering structure. As part of this work, we release our maps to the community.

We present an improved explicit congestion notification (ECN) mechanism that enables a router to signal congestion to the sender without trusting the receiver or other network devices along the signaling path. Without our mechanism, ECN-based transports can be manipulated to undermine congestion control. Web clients seeking faster downloads, for example, can trivially conceal congestion signals from Web servers. A misbehaving connection would exceed its fair bandwidth share at the expense of competing traffic by as much as an order of magnitude in our simulations. Our improved mechanism is robust because it does not depend on correct implementation at locations other than the sender and marking router, and it is practical because it admits an efficient implementation that is backwards-compatible with prior ECN and TCP/IP mechanisms.

The Computational Grid has been proposed, for the implementation of high-performance applications using widely dispersed computational resources. The goal of a Computational Grid is to aggregate ensembles of shared, heterogeneous, and distributed resources (potentially controlled by separate organizations) to provide computational, "power" to an application program. We provide a toolkit for the development of globally deployable Grid applications. The toolkit, called EveryWare, enables an application to draw computational power transparently from the Grid. It consists of a portable set of processes and libraries that can be incorporated into an application so that a wide variety of dynamically changing distributed infrastructures and resources can be used together to achieve supercomputer-like performance. We provide our experiences gained while building the EveryWare toolkit prototype and an explanation of its use in implementing a large-scale Grid application.

In this paper, we describe a receiver-based congestion control policy that leverages TCP flow control mechanisms to prioritize mixed traffic loads across access links. We manage queueing at the access link to: (1) improve the response time of interactive network applications; (2) reduce congestion-related packet losses; while (3) maintaining high throughput for bulk-transfer applications. Our policy controls queue length by manipulating receive socket buffer sizes. We have implemented this solution in a dynamically loadable Linux kernel module, and tested it over low-bandwidth links. Our approach yields a 7-fold improvement in packet latency over an unmodified system while maintaining 94% link utilization. In the common case, congestion-related packet losses at the access link can be eliminated. Finally, by prioritizing short flows, we show that our system reduces the time to download a complex Web page during a large background transfer by a factor of two.

We present a technique for identifying repetitive information transfers and use it to analyze the redundancy of network traffic. Our insight is that dynamic content, streaming media and other traffic that is not caught by today's Web caches is nonetheless likely to derive from similar information. We have therefore adapted similarity detection techniques to the problem of designing a system to eliminate redundant transfers. We identify repeated byte ranges between packets to avoid retransmitting the redundant data.

Focuses on the problem of making short- and medium-term forecasts of CPU availability on time-shared Unix systems. We evaluate the accuracy with which availability can be measured using the Unix load average, the Unix utility "vmstat" and the Network Weather Service (NWS) CPU sensor that uses both. We also examine the autocorrelation between successive CPU measurements to determine their degree of self-similarity. While our observations show a long-range autocorrelation dependence, we demonstrate how this dependence manifests itself in the short- and medium-term predictability of the CPU resources in our study.

The Computational Grid [10] has recently been proposed for the implementation of high-performance applications using widely dispersed computational resources. The goal of a Computational Grid is to aggregate ensembles of shared, heterogeneous, and distributed resources (potentially controlled by separate organizations) to provide computational "power" to an application program. In this paper, we provide a toolkit for the development of Grid applications. The toolkit, called EveryWare, enables an application to draw computational power transparently from the Grid. The toolkit consists of a portable set of processes and libraries that can be incorporated into an application so that a wide variety of dynamically changing distributed infrastructures and resources can be used together to achieve supercomputer-like performance. We provide our experiences gained while building the EveryWare toolkit prototype and the first true Grid application. 1 Introduction Increasingly, the high-perform...

Article Application level scheduling of gene sequence comparison on metacomputers Share on Authors: Neil Spring View Profile , Rich Wolski View Profile Authors Info & Claims ICS '98: Proceedings of the 12th international conference on SupercomputingJuly 1998 Pages 141–148https://doi.org/10.1145/277830.277860Published:13 July 1998 41citation266DownloadsMetricsTotal Citations41Total Downloads266Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

In this paper we describe the design and implementation of a system called the Network Weather Service (NWS) that takes periodic measurements of deliverable resource performance from distributed networked resources, and uses numerical models to dynamically generate forecasts of future performance levels. These performance forecasts, along with measures of performance fluctuation (e.g the mean square prediction error) and forecast lifetime that the NWS generates, are made available to schedulers and other resource management mechanisms at runtime so that they may determine the quality-of-service that will be available from each resource. We describe the architecture of the NWS and implementations that we have developed and are currently deploying for the Legion [13] and Globus/Nexus [7] metacomputing infrastructures. We also detail NWS forecasts of resource performance using both the Legion and Globus/Nexus implementations. Our results show that simple forecasting techniques substantially outperform measurements of current conditions (commonly used to gauge resource availability and load) in terms of prediction accuracy. In addition, the techniques we have employed are almost as accurate as substantially more complex modeling methods. We compare our techniques to a sophisticated time-series analysis system in terms of forecasting accuracy and computational complexity.