Malware and Vulnerability analysis are integral components of Cybersecurity. Malware authors employ code obfuscation techniques such as Control-flow flattening and Virtualization to escape detection from antivirus tools. Also, vulnerable code analysis gets complicated when the code is optimized using compiler flags. This paper proposes a binary function similarity detection (BFSD) framework that combines traditional Graph Neural Networks and relatively more recent Large Language Models using ensemble techniques to break code obfuscation and code optimization problems. The framework projects different facets of a binary program, such as control-flow graphs and assembly codes, into different Neural Network architectures, such as GNN and LLM, synchronizes their training process by maintaining the same testing and training data, and finally combines the predictions using ensemble techniques. The diverse features employed by the machine learning models expose unique subsets of functions, and the ensemble takes advantage of this. Experiments show state-of-the-art accuracy in breaking both code obfuscation and code optimization. We use two transformations for code obfuscation: Control-flow Flattening and Virtualization. We use three compiler flags for code optimization: $O_{1}, O_{2}$, and $O_{3}$. We also evaluate the robustness of the framework by cascadingMixed-Boolean Arithmetic with Flatten.

Distributed data-parallel (DDP) training improves overall application throughput as multiple devices train on a subset of data and aggregate updates to produce a globally shared model. The periodic synchronization at each iteration incurs considerable overhead, exacerbated by the increasing size and complexity of state-of-the-art neural networks. Although many gradient compression techniques propose to reduce communication cost, the ideal compression factor that leads to maximum speedup or minimum data exchange remains an open-ended problem since it varies with the quality of compression, model size and structure, hardware, network topology and bandwidth. We propose GraVAC, a framework to dynamically adjust compression factor throughout training by evaluating model progress and assessing gradient information loss associated with compression. GraVAC works in an online, black-box manner without any prior assumptions about a model or its hyperparameters, while achieving the same or better accuracy than dense SGD (i.e., no compression) in the same number of iterations/epochs. As opposed to using a static compression factor, GraVAC reduces end-to-end training time for ResNet101, VGG16 and LSTM by 4.32×, 1.95× and 6.67× respectively. Compared to other adaptive schemes, our framework provides 1.94× to 5.63× overall speedup.

Gradient compression alleviates expensive communication in distributed deep learning by sending fewer values and its corresponding indices, typically via Allgather (AG). Training with high compression ratio (CR) achieves high accuracy like DenseSGD, but has lower parallel scaling due to high communication cost (i.e., parallel efficiency). Using lower CRs improves parallel efficiency by lowering synchronization cost, but degrades model accuracy as well (statistical efficiency). Further, speedup attained with different models and CRs also varies with network latency, effective bandwidth and collective op used for aggregation. In many cases, collectives like Allreduce (AR) have lower cost than AG to exchange the same amount of data. In this paper, we propose an AR-compatible Top k compressor that is bandwidth-optimal and thus performs better than AG in certain network configurations. We develop a flexible communication strategy that switches between AG and AR based on which collective is optimal in the current settings, and model the pareto-relationship between parallel and statistical efficiency as a multiobjective optimization (MOO) problem to dynamically adjust CR and accelerate training while still converging to high accuracy.

In distributed training, deep neural networks (DNNs) are launched over multiple workers concurrently and aggregate their local updates on each step in bulk-synchronous parallel (BSP) training. However, BSP does not linearly scale-out due to high communication cost of aggregation. To mitigate this overhead, alternatives like Federated Averaging (FedAvg) and Stale-Synchronous Parallel (SSP) either reduce synchronization frequency or eliminate it altogether, usually at the cost of lower final accuracy. In this paper, we present SelSync, a practical, low-overhead method for DNN training that dynamically chooses to incur or avoid communication at each step either by calling the aggregation op or applying local updates based on their significance. We propose various optimizations as part of SelSync to improve convergence in the context of semi-synchronous training. Our system converges to the same or better accuracy than BSP while reducing training time by up to 14×.

We propose a circuit-level backdoor attack, QTrojan, against Quantum Neural Networks (QNNs) in this paper. QTrojan is implemented by a few quantum gates inserted into the variational quantum circuit of the victim QNN. QTrojan is much stealthier than a prior Data-Poisoning-based Backdoor Attack (DPBA) since it does not embed any trigger in the inputs of the victim QNN or require access to original training datasets. Compared to a DPBA, QTrojan improves the clean data accuracy by 21% and the attack success rate by 19.9%.

In this work, we propose IQGAN, a quantum Generative Adversarial Network (GAN) framework for multiqubit image synthesis that can be efficiently implemented on Noisy Intermediate Scale Quantum (NISQ) devices. We investigate the reasons for the inferior generative performance of current quantum GANs in our preliminary study and conclude that an adjustable input encoder is the key to ensuring high-quality data synthesis. We then propose the IQGAN architecture featuring a trainable multiqubit quantum encoder that effectively embeds classical data into quantum states. Furthermore, we propose a compact quantum generator that significantly reduces the design cost and circuit depth on NISQ devices. Experimental results on both IBM quantum processors and quantum simulators demonstrated that IQGAN outperforms state-of-the-art quantum GANs in qualitative and quantitative evaluation of the generated samples, model convergence, and quantum computing cost.

Today's graphics processing unit (GPU) applications produce vast volumes of data, which are challenging to store and transfer efficiently. Thus, data compression is becoming a critical technique to mitigate the storage burden and communication cost. LZSS is the core algorithm in many widely used compressors, such as Deflate. However, existing GPU-based LZSS compressors suffer from low throughput due to the sequential nature of the LZSS algorithm. Moreover, many GPU applications produce multi-byte data (e.g., int16/int32 index, floating-point numbers), while the current LZSS compression only takes single-byte data as input. To this end, in this work, we propose gpuLZ, a highly efficient LZSS compression on modern GPUs for multi-byte data. The contribution of our work is fourfold: First, we perform an in-depth analysis of existing LZ compressors for GPUs and investigate their main issues. Then, we propose two main algorithm-level optimizations. Specifically, we (1) change prefix sum from one pass to two passes and fuse multiple kernels to reduce data movement between shared memory and global memory, and (2) optimize existing pattern-matching approach for multi-byte symbols to reduce computation complexity and explore longer repeated patterns. Third, we perform architectural performance optimizations, such as maximizing shared memory utilization by adapting data partitions to different GPU architectures. Finally, we evaluate gpuLZ on six datasets of various types with NVIDIA A100 and A4000 GPUs. Results show that gpuLZ achieves up to 272.1× speedup on A4000 and up to 1.4× higher compression ratio compared to state-of-the-art solutions.

Distributed deep learning (DDL) training systems are designed for cloud and data-center environments that assumes homogeneous compute resources, high network bandwidth, sufficient memory and storage, a s w ell a s independent and identically distributed (IID) data across all nodes. However, these assumptions don’t necessarily apply on the edge, especially when training neural networks on streaming data in an online manner. Computing on the edge suffers from both systems and statistical heterogeneity. Systems heterogeneity is attributed to differences in compute resources and bandwidth specific to each device, while statistical heterogeneity comes from unbalanced and skewed data on the edge. Different streaming-rates among devices can be another source of heterogeneity when dealing with streaming data. If the streaming rate is lower than training batch-size, device needs to wait until enough samples have streamed in before performing a single iteration of stochastic gradient descent (SGD). Thus, low-volume streams act like stragglers slowing down devices with high-volume streams in synchronous training. On the other hand, data can accumulate quickly in the buffer if the streaming rate is too high and the devices can’t train at line-rate. In this paper, we introduce ScaDLES to efficiently train on streaming data at the edge in an online fashion, while also addressing the challenges of limited bandwidth and training with non-IID data. We empirically show that ScaDLES converges up to 3.29× faster compared to conventional distributed SGD.

In this work we examine a specific case of wireless sensor networks (WSN) we call peer-to-peer WSN where source and destination are both dynamic and each is subject to constraints of low bandwidth, limited energy storage, and limited computational resources. Peer-to-peer WSN require the consideration of data computation time as a limiting constraint on information availability unlike a standard WSN that can rely on the unconstrained sink to perform the necessary computation of the raw sensor data into usable information. To effectively manage and improve upon peer-to-peer WSN routing, and WSN routing in general, we present a deep reinforcement learning algorithm known as distributed cooperative reinforcement for routing (DCRL-R) which uses a neural network and expanded state space parameters to learn routing policies for WSN. DCRL-R also incorporates an increased action space for determining when and where to perform in-network computation of the raw sensor data. We perform tests of DCRL-R on a physical network utilizing measured node state parametric data and show its viability in future WSN applications compared to a baseline routing algorithm using shortest path decisions with no computational offloading.

The bitstream inside a Field-Programmable Gate Array (FPGA) is often protected using an encryption key, acting as a root of trust and stored inside the FPGA, to defend against bitstream piracy, tampering, overproduction, and static-time reverse engineering. For cost savings and faster production, trusted system designers often rely on an untrusted system assembler to program the encryption key into the FPGA, focusing only the end-user-stage threats. However, providing the secret encryption key to an untrusted entity introduces additional threats, since access to this key can compromise the entire root of trust and breach the encrypted bitstream enabling a multitude of attacks including Trojan insertion, piracy and overproduction. To address this issue, we propose the Secure Remote FPGA Initialization (SeRFI) protocol to transmit the encryption key securely from a trusted system designer into an FPGA in physical possession of an untrusted system assembler. Our protocol eliminates direct key sharing with the untrusted system assembler as well as prevents against adversarial intention of extracting the encryption key during the programming phase where the assembler has physical access to the FPGA.

To take full advantage of modern high performance architectures, many large-scale data-driven applications require loosely-coupled, fine-grained asynchronous communication. Accordingly, efficient lightweight middleware based on state-of-the-art networking technology such as RDMA is becoming a necessity. The performance critical task of handling RDMA synchronization, for existing message passing runtimes are mostly coarse granular in nature and thus may be associated with various hidden costs. While low-level RDMA libraries expose fine-grained RDMA communication that can be efficiently controlled, the critical tasks of RDMA buffer management, synchronization and flow control are left to the userspace applications requiring tedious programming effort that may lead to sub-optimal performance. In this paper we present a user-space RDMA transport layer that allows RDMA-enabled memory to be managed internally while still exposing zero-copy completion event-based RDMA transfers for message passing. The integration of an RDMA transport layer enables the opportunity for parallel applications to utilize RDMA-managed buffers for accelerating communication while co-existing with high-level MPI, GASNet or similar middleware. We show a performance speedup of up to 8X in latency/bandwidth benchmarks and 5%-90% improvement in response time or messaging rate in three reference applications with regard to their MPI implementations.

Maintaining a scalable high-performance virtual global address space using distributed memory hardware has proven to be challenging. In this paper we evaluate a new approach for such an active global address space that leverages the capabilities of the network fabric to manage addressing, rather than software at the endpoint hosts. We describe our overall approach, design alternatives, and present initial experimental results that demonstrate the effectiveness and limitations of existing network hardware.

The quality of data exchange in cloud computing applications relies on the connection performance between user clients and their cloud storage providers, and is often dependent on the wide area network (WAN) properties among data centers. For certain classes of applications, it can be crucial to provide an end-to-end solution that accelerates large data transfers and improves overall user experience. The development and deployment of WAN optimization technology has been investigated for improving application performance in heterogeneous, multi-domain environments. WAN optimization devices and services implement a number of approaches for performance improvement, and one key insight is that in contrast to traditional end-to-end TCP connections, middleboxes that segment and optimize transport-layer connections can improve the performance of wide area data transfers. In the context of dynamic cloud computing environments, there is an obvious target for implementations of WAN optimization as Network Function Virtualization (NFV), where the flexibility of virtualized cloud environments can be exploited. This paper describes recent developments and experimentation of our Phoebus WAN accelerator framework. We introduce a software suite that includes new Phoebus clients that operate with the Phoebus Gateway network. We test and discuss virtualizing Phoebus Gateways to provide acceleration services in cloud data transfers. Use cases and performance evaluations are conducted on FutureGrid and Internet2 testbeds, and we demonstrate the effectiveness of a virtualized Phoebus deployment.

Network resource measurement is a key functionality for large scale network management. Intelligent, network-aware applications may benefit from access to detailed representations of network resources, including multi-layer topologies and real-time traffic measurement, and shared resources may obtain better overall utilization by identifying performance bottlenecks. In this study, we describe a network measurement framework, which includes a network topology analysis mechanism as well as agent tools for running active probes and collecting data from end hosts. The system includes a centralized coordinator, which abstracts network elements into annotated network graphs and applies scheduling algorithms to calculate conflict free measurement probes over shared links. Our evaluation integrated perfSONAR services into our framework and included deployment scenarios on research and education networks such as Internet2 and ESnet. The data presented in this study offers compelling evidence that supports a method by which to measure the performance of real world networks.

In this paper, we describe a novel technique to optimize longest common subsequence (LCS) algorithm for one-to-many matching problem on GPUs by transforming the computation into bit-wise operations and a post-processing step. The former can be highly optimized and achieves more than a trillion operations (cell updates) per second (CUPS)-a first for LCS algorithms. The latter is more efficiently done on CPUs, in a fraction of the bit-wise computation time. The bit-wise step promises to be a foundational step and a fundamentally new approach to developing algorithms for increasingly popular heterogeneous environments that could dramatically increase the applicability of hybrid CPU-GPU environments.

Data set sizes are growing exponentially, so it is important to use data movement protocols that are the most efficient available. Most data movement tools today rely on TCP over sockets, which limits flows to around 20Gbps on today's hardware. RDMA over Converged Ethernet (RoCE) is a promising new technology for high-performance network data movement with minimal CPU impact over circuit-based infrastructures. We compare the performance of TCP, UDP, UDT, and RoCE over high latency 10Gbps and 40Gbps network paths, and show that RoCE-based data transfers can fill a 40Gbps path using much less CPU than other protocols. We also show that the Linux zero-copy system calls can improve TCP performance considerably, especially on current Intel “Sandy Bridge”-based PCI Express 3.0 (Gen3) hosts.

This paper presents the eXtensible Session Protocol (XSP), which provides a control plane for driving Software Defined Networks (SDNs). The XSP model supports proactive, application-driven configuration of dynamic network resources with support for authentication and authorization, within an extensible protocol framework. We describe XSP application use cases in SDNs using OpenFlow enabled network devices as well as dynamic forwarding rule management that can be implemented on existing router platforms.

In this paper, we present an algorithm and provide design improvements needed to port the serial Lempel-Ziv-Storer-Szymanski (LZSS), lossless data compression algorithm, to a parallelized version suitable for general purpose graphic processor units (GPGPU), specifically for NVIDIA's CUDA Framework. The two main stages of the algorithm, substring matching and encoding, are studied in detail to fit into the GPU architecture. We conducted detailed analysis of our performance results and compared them to serial and parallel CPU implementations of LZSS algorithm. We also benchmarked our algorithm in comparison with well known, widely used programs, GZIP and ZLIB. We achieved up to 34x better throughput than the serial CPU implementation of LZSS algorithm and up to 2.21x better than the parallelized version.

Monitoring and managing multi-gigabit networks requires dynamic adaptation to end-to-end performance characteristics. This paper presents a measurement collection and analysis framework that automates the troubleshooting of end-to-end network bottlenecks. We integrate real-time host, application, and network measurements with a common representation (compatible with perfSONAR) within a flexible and scalable architecture. Our measurement architecture is supported by a light-weight eXtensible Session Protocol (XSP), which enables context-sensitive adaptive measurement collection. We evaluate the ability of our system to analyze and detect bottleneck conditions over a series of high-speed and I/O intensive bulk data transfer experiments and find that the overhead of the system is very low and that we are able to detect and understand a variety of bottlenecks.

Increasing needs in efficient storage management and better utilization of network bandwidth with less data transfer have led the computing community to consider data compression as a solution. However, compression introduces extra overhead and performance can suffer. The key elements in making the decision to use compression are execution time and compression ratio. Due to negative performance impact, compression is often neglected. General purpose computing on graphic processing units (GPUs) introduces new opportunities where parallelism is available. Our work targets the use of opportunities in GPU based systems by exploiting parallelism in compression algorithms. In this paper we present an implementation of the Lempel-Ziv-Storer-Szymanski (LZSS) loss less data compression algorithm by using NVIDIA GPUs Compute Unified Device Architecture (CUDA) Framework. Our implementation of the LZSS algorithm on GPUs significantly improves the performance of the compression process compared to CPU based implementation without any loss in compression ratio. This can support GPU based clusters in solving application bandwidth problems. Our system outperforms the serial CPU LZSS implementation by up to 18×, the parallel threaded version up to 3× and the BZIP2 program by up to 6× in terms of compression time, showing the promise of CUDA systems in loss less data compression. To give the programmers an easy to use tool, our work also provides an API for in memory compression without the need for reading from and writing to files, in addition to the version involving I/O.

Large scale scientific data transfers are central to scientific processes. Data from large experimental facilities have to be moved to local institutions for analysis or often data needs to be moved between local clusters and large supercomputing centers. In this paper, we propose and evaluate a network overlay architecture to enable high-throughput, on-demand, coordinated data transfers over wide-area networks. Our work leverages Phoebus and On-demand Secure Circuits and Advance Reservation System (OSCARS) to provide high performance wide-area network connections. OSCARS enables dynamic provisioning of network paths with guaranteed bandwidth and Phoebus enables the coordination and effective utilization of the OSCARS network paths. Our evaluation shows that this approach leads to improved end-to-end data transfer throughput with minimal overheads. The achieved throughput using our overlay was limited only by the ability of the end hosts to sink the data.

Message Passing Interface (MPI) messages are centered around transmitting instances of MPI data types. The data types represented in MPI terms are usually modeled after data types native to the application. If a user does not want to transmit a field from the native data type, the user will sometimes align the MPI data type such that there is a gap in the displacement where the omitted field would be. With the resulting MPI data type now being non-contiguous, cycles are spent making the data contiguous for transmission and then expanding the data back out on the receiving side. We show that by performing data type fission (the process of segregating the transmitted fields from the non-transmitted fields) and aligning the MPI data type accordingly, we can completely eliminate the need to copy data during the packing and unpacking process, which can significantly improve the performance of communication-heavy Single Program Multiple Data applications.

Computational science is considered the third pillar of science but while we continue to make significant strides in high-performance computing, the ability to transfer the vast amounts of data generated and consumed by scientists has fallen behind. This divergence has significantly constrained scientific progress. This paper describes MNEMONIC, an end-to-end cyber-infrastructure system designed to improve scientists' productivity by automatically and transparently optimizing and tuning data movement over the network. MNEMONIC integrates advanced technologies, i.e., network interfaces, protocols, and measurement toolkits, to seamlessly manage high-performance data movement. We have integrated the Java implementation of GridFTP in MNEMONIC to provide a cross-platform data transfer capability. Preliminary results show that MNEMONIC+GridFTP outperforms the TCP+GridFTP implementation by over a factor of 10x, while remaining easy to install and operate.

Several existing compiler transformations can help improve communication-computation overlap in MPI applications. However, traditional compilers treat calls to the MPI library as a black box with unknown side effects and thus miss potential optimizations. This paper's contributions enable the development of an MPI-aware optimizing compiler that can perform transformations exploiting knowledge of MPI call effects to increase communication-computa-tion overlap. We formulate a set of data flow equations and rules to describe the side effects of key MPI functions so an MPI-aware compiler can automatically assess the safety of transformations. After categorizing existing compiler transformations based on their effect on the application code, we present an optimization algorithm that specifies when and how to apply these optimizing transformations to achieve improved communication-computation overlap. By manually applying the optimization algorithm to kernels extracted from HYCOM and the NAS benchmarks, we show that even when transforming these highly optimized codes, execution time can be decreased by an average of over 30%.

Phoebus is an infrastructure for improving end-to-end throughput in high-bandwidth, long-distance networks by using a "session layer" protocol and "gateways" in the network. Phoebus has the ability to dynamically allocate network resources and to use segmentspecific transport protocols between gateways, as well as to apply other performance-improving techniques on behalf of the user.

Abstract Scientific applications are evolving rapidly and rely heavily on the network for data movement, communication, control and result collection. This reliance on the network has increased the breadth of consumers of network status information while the increasing complexity of networks has expanded the diversity of the producers of this information. To enable this wide variety of producers and consumers to effectively interoperate, relevant standards are paramount. In this paper, we present a general model used to represent both network measurements collected from performance tools as well as describing the physical and logical characteristics of the underlying network. This system is currently being standardized in the Open Grid Forum to enable other uses within the wider grid and distributed computing community (Open Grid Forum, http://www.ogf.org/ ). Copyright © 2009 John Wiley & Sons, Ltd.

A non-trivial challenge in high performance, cluster computing is the communication overhead introduced by the cluster interconnect. A common strategy for addressing this challenge is the use of communication-computation overlapping. In this paper, we introduce Gravel, a portable communication library designed to inter-operate with MPI for improving communication-computation overlapping. Selected MPI calls are semi-automatically replaced by Gravel calls only in key locations in an application where performance is critical. Gravel separates the data transfers from the handshake messages, enabling application developers to utilize Remote Data Memory Access (RDMA) directly, whether the data exchange scheme of their application is pure one-sided, or a more traditional two-sided. The Gravel API is much simpler than existing low level libraries and details (i.e., pointers) are hidden from the application layer so Gravel is usable in FORTRAN applications. This paper presents an overview of Gravel.

A widely-distributed network monitoring system requires a scalable discovery mechanism. The "lookup service" component of the perfSONAR framework is able to manage component registration, distill resource data into tractable units, and respond to queries regarding system and performance information. A model of organizing and distributing information is presented to support both dynamic environments where services frequently change as well where different administrative configuration requirements exist. These interactions are accomplished by forming "federated hierarchies" to share information amongst the various logical overlays.

This paper describes a source to source compilation tool for optimizing MPI-based parallel applications. This tool is able to automatically apply a "prepushing" transformation that causes MPI programs to aggressively send data as soon as it is available, thus improving communication-computation overlap and improving application performance. In this paper we present asphalt_transformer; the Open64-based component of our framework, ASPhALT, responsible for automatically performing the prepushing transformation. We also present an extensive study of the performance gains witnessed from automatically transformed codes. In particular, we demonstrate how different levels of aggregation affect the performance of parallel programs executing various computation kernels on different clusters. Furthermore, we discuss the differences in performance improvement between the hand-optimized and automatically optimized codes, as well as the effect of automation on time-to-solution.

Applications that execute on parallel clusters face scalability concerns due to the high communication overhead that is usually associated with such environments. Modern network technologies that support remote direct memory access (RDMA) can offer true zero copy communication and reduce communication overhead by overlapping it with computation. For this approach to be effective the parallel application using the cluster must be structured in a way that enables communication computation overlapping. Unfortunately, the trade-off between maintainability and performance often leads to a structure that prevents exploiting the potential for communication computation overlapping. This paper describes a source-to-source optimizing transformation that can be performed by an automatic (or semi-automatic) system in order to restructure MPI codes towards maximizing communication-computation overlapping.

Phoebus is a project whose goal is to allow legacy applications to utilize high bandwidth connections over Internet2's Hybrid Packet Optical Infrastructure (HOPI) backbone. Phoebus works by breaking a single end-to-end TCP connection into a series of transport layer connections. The end connections (from Internet2 edge nodes to the end hosts) still use TCP. However, the connection over the Internet2 backbone is handled by server software running on the edge nodes. These servers can then choose the best transport protocol and protocol settings for sending the data across Internet2. Using this service, end users are no longer required to modify a variety of TCP settings in order to achieve good performance over long distances. The end users only need to be concerned about ensuring adequate speed across the path between themselves and the Internet2 edge nodes. In many cases, the default TCP settings will prove adequate for achieving acceptable speeds.

This paper describes a simple scheduling procedure for use in multicast data distribution within a logistical networking infrastructure. The goal of our scheduler is to generate a distribution schedule that will exploit the best network paths by using historic network performance information. A "spanning tree" is constructed between available logistical depots to help reduce the overall time of data movement. Our hypothesis is that we can generate appropriate schedules from historical network measurements. In order to evaluate the scheduling procedure we have employed the multicast operation used in the Internet backplane protocol (IBP) middleware suite. Investigation into the merits of such a scheduling procedure involved a control group that performs a broadcast to a set of logistical depots and an experimental group that is configured to perform a multicast via schedules generated based on historical network data. All testing was conducted on PlanetLab, a distributed network service testbed.

This paper presents program transformations directed toward improving communication-computation overlap in parallel programs that use MPI’s collective operations. Our transformations target a wide variety of applications focusing on scientific codes with computation loops that exhibit limited dependence among iterations. We include guidance for developers for transforming an application code in order to exploit the communicationcomputation overlap available in the underlying cluster, as well as a discussion of the performance improvements achieved by our transformations. We present results from a detailed study of the effect of the problem and message size, level of communication-computation overlap, and amount of communication aggregation on runtime performance in a cluster environment based on an RDMA-enabled network. The targets of our study are two scientific codes written by domain scientists, but the applicability of our work extends far beyond the scope of these two applications.

This work describes a technique for improving network performance in Grid environments that we refer to as "logistics." We demonstrate that by using storage and cooperative forwarding "in" the network, we can improve end to end throughput in many cases. Our approach uses TCP connections in series and offers performance benefits for high-bandwidth, high-latency networks. First, we examine the underlying causes of the logistical effect. Next, we present a graph based scheduling approach that can be solved quickly and, within our assumptions, optimally. Finally, we present a large-scale empirical evaluation of the system in order to validate our scheduling approach for taking advantage of network logistics. This study demonstrates performance improvement in many situations and aggregate speedup results are presented.

In this work we present the Internet Backplane Protocol (IBP), a middleware created to allow the sharing of storage resources, implemented as part of the network fabric. IBP allows an application to control intermediate data staging operations explicitly. As IBP follows a very simple philosophy, very similar to the Internet Protocol, and the resulting semantic might be too weak for some applications, we introduce the exNode, a data structure that aggregates storage allocations on the Internet.

In this paper, we discuss requirements for integrating dynamic performance information from the Network Weather Service (NWS) into the Grid Information Service infrastructure (GIS). We describe the object model that NWS uses internally and provide some rationale for its structure. Finally, we present the NWS 's implementation of a caching LDAP daemon that integrates NWS information into the reference GIS -the Glob s MDS.

This paper describes a new technique in the Network Weather Service for producing multi-variate forecasts. The new technique uses the NWS’s univariate forecasters and emprically gathered Cumulative Distribution Functions (CDFs) to make predictions from correlated measurement streams. Experimental results are shown in which throughput is predicted for long TCP/IP transfers from short NWS network probes.

The Logistical Session Layer is a system to enable enhanced functionality to distributed programming systems. The term Logistical refers to the fact that we enhance the traditional client-server model to allow for intermediate systems which are neither. This system generalizes the notion of caches but represents a cleaner architecture in that it explicitly declares itself to be a session layer protocol.

Presents a strategy for optimizing end-to-end TCP/IP throughput over long-haul networks (i.e. those where the product of the bandwidth and the delay is high.) Our approach defines a logistical session layer (LSL) that uses intermediate process-level "depots" along the network route from source to sink to implement an end-to-end communication session. Despite the additional processing overhead resulting from TCP/IP protocol stack-Unix kernel boundary traversals at each depot, our experiments show that dramatic end-to-end bandwidth improvements are possible. We also describe a prototype implementation of LSL that does not require any Unix kernel modification or root access privilege, which we used to generate the results, and we discuss its utility in the context of extant TCP/IP tuning methodologies.