This article presents an Internet-of-Drones (IoD)-enabled system for real-time, high-resolution, in-situ water quality sensing, sampling, 2-D mapping, and 3-D depth profiling from discrete sensing. Unlike isolated mooring platforms and stationary sensor buoys, our Uncrewed Aerial Vehicle (UAV) platform features a modular IoD architecture with wireless communication for adaptive sampling, parameter mapping, and real-time monitoring. The system integrates pH, temperature, turbidity, total dissolved solids (TDS), and depth sensors within a single-actuator multi-cartridge vessel that collects samples via TDS, depth, and ML-based triggers into four 50mL tubes. The proposed embedded system incorporates LoRa/LoRaWAN for low-power telemetry and LTE for wireless data transmission, with SD card logging, GPS geo-tagging, and Real-time Clock (RTC) synchronization. A high-resolution 3-D interpolation framework is implemented that reconstructs water quality fields from UAV-based missions over a 0.8m× 0.8m× 1mvolume and a comprehensive 2-D mapping over a 180m× 180marea. The system incorporates a smart sampling logic based on predefined thresholds (e.g., TDS and depth triggers), along with a machine–learning–assisted mode for adaptive chlorophyll-adetection and real-time decision-making. Comprehensive field tests at the Lake Erie digital test bed validate system performance.

Modern Function-as-a-Service (FaaS) cloud platforms offer great potential for supporting event-driven scientific workflows. Nonetheless, there remain barriers to adoption by the scientific community in domains such as environmental sciences, where R is the focal language used for the development of applications and where users are typically not well-versed with FaaS APIs. This paper describes the design and implementation of FaaSr, a novel middleware system that supports event-driven scientific workflows in R. A key novelty in FaaSr is the ability to deploy workflows across FaaS providers without the need for any managed servers for coordination. With FaaSr: 1) functions are written in R; 2) the runtime environments for their execution are customizable containers; 3) functions access data in cloud storage (S3) with a familiar file-based abstraction supporting both full file put/get primitives and subsetting using the Parquet format; and 4) function invocation and workflow coordination only requires S3 cloud object storage, without relying on any dedicated, active workflow engine server or cloud-specific queues/databases. The paper reports on the functionality and performance of FaaSr for micro-benchmarks and two case studies: event-driven forecast and batch job workflows. These demonstrate the ability to deploy workflows across multiple platforms (GitHub Actions, Amazon Web Services Lambda, and the open-source OpenWhisk), without the need for dedicated coordination servers, across both cloud and edge resources. FaaSr is open-source and available as a CRAN package.

In the ever-evolving networking landscape, the demand for efficient and adaptable Virtual Private Network (VPN) solutions is growing. Software-Defined Networks (SDNs), particularly Peer-to-Peer (P2P) overlay VPNs, offer a practical approach for networks spanning various edge and cloud providers. However, existing decentralized VPNs, while resilient and scalable, typically utilize a single tunnel type and overlook data plan costs and link performance in their selection processes. This oversight can lead to cost and performance inefficiencies, especially in edge-to-cloud networks where diverse nodes have unique needs that generic solutions fail to meet effectively. Although SDN facilitates the integration of multiple link types in overlay VPNs, existing systems lack efficient policies for selecting favorable tunnels. To bridge this gap, we introduce PolyNet, a Multi-Criteria approach designed to make cost- and performance-aware policy decisions in hybrid-link overlay networks. PolyNet employs a dynamic link selection policy during runtime that evaluates latency using Vivaldi network coordinates and considers cost, and integrates with SDN-based P2P overlays to enhance link management capabilities and support multiple link types. This paper presents the design of PolyNet and evaluates its performance through simulations and prototype testing. Results demonstrate that PolyNet achieves up to a 19.1% cost reduction and a 14.1% latency improvement over traditional methods in Symphony P2P topologies. Additionally, tests with a software prototype confirm the advantages of hybrid links, showing that kernel-layer GENEVE tunnels can increase throughput by up to 8.9 times compared to user-layer Nebula and WebRTC tunnels in edge clusters.

Memory plays a critical role in storing almost all computation data for various applications, including those with sensitive data such as bank transactions and critical business management. As a result, protecting memory security from attackers with physical access is ultimately important. Various memory attacks have been proposed, among which “cold boot” and RowHammer are two leading examples. DRAM manufacturers have deployed a series of protection mechanisms to counter these attacks. Even with the latest protection techniques, DRAM may still be vulnerable to attackers with physical access. In this paper, we proposed a novel “warm boot” attack which utilizes external power supplies to bypass the existing protection mechanisms and steal the data from the modern SODIMM DDR4 memory. The proposed “warm boot” attack is applied to various DRAM chips from different brands. Based on our experiments, the “warm boot” attack can achieve as high as 94% data recovery rate from SODIMM DDR4 memory.

The advent of virtualization and cloud computing has fundamentally changed how distributed applications and services are deployed and managed. With the proliferation of IoT and mobile devices, virtualized systems akin to those offered by cloud providers are increasingly needed geographically near the network’s edge to perform processing tasks in proximity to the data sources and sinks. Latency-sensitive, bandwidth-intensive applications can be decomposed into workflows that leverage resources at the edge — a model referred to as fog computing. Not only is performance important, but a trustworthy network is fundamental to guaranteeing privacy and integrity at the network layer. This paper describes Bounded Flood, a novel technique that enables virtual private Ethernet networks that span edge and cloud resources — including those constrained by NAT and firewall middleboxes. Bounded Flood builds upon a scalable structured peer-to-peer overlay, and is novel in how it integrates overlay tunnels with SDN software switches to create a virtual network with dynamic membership — supporting unmodified Ethernet/IP stacks to facilitate the deployment of edge applications. Bounded Flood has been implemented as the core of the EdgeVPN open-source virtual private network software system for edge computing. Experiments with the software demonstrate its functionality and scalability — one of which includes Kubernetes with Flannel across Raspberry Pi 4 edge devices behind different NATs.

While the Internet is open by design, it is still the case that users can be subject to censorship by governments or enterprises in accessing Web services and data. In this paper we propose SEnD, a fully-distributed censorship circumvention system built upon an overlay, where users have peer-to-peer virtual private IP tunnels to proxies within their social network. With SEnD, users in an uncensored area can act as proxy servers for their social friends in a censored area, allowing them to bypass the censorship. SEnD is able to outperform the current censorship techniques, such as IP address blocking and active probing attacks. We assessed the effectiveness of SEnD through extensive simulations based on a synthetic dataset, as well as experiments based on a prototype implementation. We built our synthetic dataset based on parameters obtained from questionnaires administered both inside and outside China (we consider China as a case study of censorship area). The results show that SEnD is feasible, efficient and scalable. For example, when the proportion of concurrent active users is less than 60, 99.9 percent of these users are able to find proxy servers.

The management of drinking water quality is critical to public health and can benefit from techniques and technologies that support near real-time forecasting of lake and reservoir conditions. The cyberinfrastructure (CI) needed to support forecasting has to overcome multiple challenges, which include: 1) deploying sensors at the reservoir requires the CI to extend to the network’s edge and accommodate devices with constrained network and power; 2) different lakes need different sensor modalities, deployments, and calibrations; hence, the CI needs to be flexible and customizable to accommodate various deployments; and 3) the CI requires to be accessible and usable to various stakeholders (water managers, reservoir operators, and researchers) without barriers to entry. This paper describes the CI underlying FLARE (Forecasting Lake And Reservoir Ecosystems), a novel system co-designed in an interdisciplinary manner between CI and domain scientists to address the above challenges. FLARE integrates R packages that implement the core numerical forecasting (including lake process modeling and data assimilation) with containers, overlay virtual networks, object storage, versioned storage, and event-driven Function-as-a-Service (FaaS) serverless execution. It is a flexible forecasting system that can be deployed in different modalities, including the Manual Mode suitable for end-users’ personal computers and the Workflow Mode ideal for cloud deployment. The paper reports on experimental data and lessons learned from the operational deployment of FLARE in a drinking water supply (Falling Creek Reservoir in Vinton, Virginia, USA). Experiments with a FLARE deployment quantify its edge-to-cloud virtual network performance and serverless execution in OpenWhisk deployments on both XSEDE-Jetstream and the IBM Cloud Functions FaaS system.

This demonstration will showcase EdgeVPN.io, an open-source software-defined virtual private network (VPN) that enables the creation of scalable layer-2 virtual networks across multiple providers - including scenarios where devices are behind Network Address Translation (NAT) and firewall middleboxes. Its architecture combines a distributed software-defined networking (SDN) control plane and a scalable structured peer-to-peer overlay of Internet tunnels that form its datapath. EdgeVPN.io provides a foundation for the deployment of virtual networks that enable research and development in distributed computing. The demonstration will include a brief overview of the architecture, and will show step-by-step how a researcher can deploy EdgeVPN.io networks on devices including Raspberry Pis, Jetson Nanos, and VMs/Docker containers in the cloud. Attendees will be provided with trial resources to allow them to follow the demonstration hands-on if they so desire.

Edge and fog computing encompass a variety of technologies that are poised to enable new applications across the Internet that support data capture, storage, processing, and communication across the networking continuum. These environments pose new challenges to the design and implementation of networks-as membership can be dynamic and devices are heterogeneous, widely distributed geographically, and in proximity to end-users, as is the case with mobile and Internet-of-Things (IoT) devices. We present a demonstration of EdgeVPN.io (Evio for short), an open-source programmable, software-defined network that addresses challenges in the deployment of virtual networks spanning distributed edge and cloud resources, in particular highlighting its use in support of the Kubernetes container orchestration middleware. The demo highlights a deployment of unmodified Kubernetes middleware across a virtual cluster comprising virtual machines deployed both in cloud providers, and in distinct networks at the edge-where all nodes are assigned private IP addresses and subject to different NAT (Network Address Translation) middleboxes, connected through an Evio virtual network. The demo includes an overview of the configuration of Kubernetes and Evio nodes and the deployment of Docker-based container pods, highlighting the seamless connectivity for TCP/IP applications deployed on the pods.

In typical commercial cloud and edge providers data collection and processing nodes might be owned by a single trusted entity. In contrast, in a voluntary infrastructure, ownership is distributed. In such a setup, while individuals contributing data might demand constraints on how their data is accessed and used, volunteers of compute resources might want assurance that their resources are used only for intended purposes. These significant differences motivate SocialEdge, an edge orchestration framework that 1) leverages online social network for bootstrapping and permissioned block chain for information assurance, 2) exploits recursive containerization for enabling secure data access, ensuring only intended workloads are run and enforcing fine-grained resource control, and 3) leverages components from Kubernetes to instantiate and manage an ephemeral cluster composed of volunteered resources. In this paper we describe the design and prototype implementation of SocialEdge and model the problem of orchestrating the on-demand data processing infrastructure under data access constraints as a variant of the multi-commodity flow problem. We propose and evaluate mixed integer programming and greedy heuristic approaches, and characterize recursive containerized environment experimentally using benchmarks.

New techniques are actively being researched to enable processing of information at the network's edge, closer to where data is generated. A nascent application enabled by such fog/edge computing model is that of community-based collaboration, where members pool personal resources together to accomplish a task, and data moves across edge resources. In such an environment, nodes that produce, store, and process data are often ephemeral, owned by different individuals, and distributed across multiple local networks. This poses new challenges at the network layer: how to automatically configure a network connecting distributed personal devices of social peers end-to-end, while enforcing privacy in communication among edge devices? We present PerSoNet, a novel virtual private network (VPN) that 1) automatically creates and manages private, authenticated overlay links across personal devices of social network peers, and 2) automatically manages software-defined networking (SDN) rules in software switches for packet forwarding, name resolution and mapping (for IP addresses and DNS names), and device network access control. PerSoNet abstracts away complexities, self-organizing a VPN that exposes IP/Ethernet network semantics, thereby enabling existing applications and middleware to be used in community-based fog/edge systems. In this work we describe the PerSoNet design and a prototype implementation based on SDN/OpenFlow and an open-source overlay. A prototype smart-task overlay application has also been built on top of PerSoNet to demonstrate its applicability and to evaluate key performance metrics of the system.

Multi-tenant data centers for cloud computing require the deployment of virtual private networks for tenants in an on-demand manner, providing isolation and security between tenants. To address these requirements, network virtualization techniques such as encapsulation and tunneling have been widely used. However, these approaches inherently incur processing overhead on end-points (such as the host hypervisor), reducing the effective throughput for the tenant virtual network compared to the native network. This problem is exacerbated with increases in line rates, now exceeding 10Gbps. In this paper, we introduce PARES (PAcket REwriting on SDN), a novel technique which uses the packet rewriting feature of SDN switches to provide multi-tenancy in data center networks at edge switches, thereby reducing the load on end-point hypervisors and improving the throughput, compared to tunneling. Experiments in an SDN testbed show that our proposed data center arhictecture with PARES achieves near line-rate multi-tenancy virtualization with 10Gbps links (compared to 20% of line-rate for VXLAN tunneling), without incurring processing overhead at end-point hypervisors or guest servers. Additionally, the paper evaluates the scalability of PARES for ARP protocol handling and with respect to number of SDN flow entries.

Since the Internet is an aggregation of multiple ASes (Autonomous Systems), congestion control and utilization are not globally optimized. For example, it is not uncommon that a direct shortest route with low latency delivers less bandwidth than an alternative, long and roundabout route. Previous research has shown that geospatially distributed computing instances in commercial clouds offer users an opportunity to deploy relay points to detour potentially congested ASes, and as a means to diversify paths to increase overall bandwidth and reliability. Such opportunity comes with a cost, as cloud-routed paths incur cost of not only provisioning of computing resources, but also for additional traffic to/from Internet. Well-established protocols, such as TCP, were created based on assumption of single end-point to end-point transfer; nonetheless, current computing devices have multiple end-points, and the increasing availability of overlay networks allows multiplexing multiple virtual network flows into a single physical network interface. In this paper, we empirically evaluate the extent to which using cloud paths to transfer data in parallel with the default Internet path can improve the end-to-end bandwidth in bulk data transfers. In our evaluation, we consider single-stream and multi-stream TCP transfers across one or more paths. Moreover, we suggest an application level design pattern that takes advantage of this improved aggregate bandwidth to reduce data transfer times.

Efficient and secure device to device communication is a necessary enabler for realization of fog and edge computing models. With an ever-increasing number, diversity, and geographical distribution of heterogeneous devices, ranging from pervasive sensor networks to edge cloudlets, the interconnection and network management of these devices emerges as a major challenge. Peer-to-peer (P2P) network overlays provide a scalable, decentralized and autonomous solution for interconnecting devices at the edge, and from the edge to the core of the cloud. This paper considers a networking system for edge computing that leverages trust embedded in online social networks (OSNs) to establish P2P overlays that offer the opportunity of private, authenticated device to device communications within the scope of a virtual private network (VPN). Mapping the social graph of OSNs onto an overlay network's topology presents challenges, since every social link that is mapped to an overlay has a cost associated with it, in terms of energy and resources consumed on link-setup and maintenance (especially for power-constrained devices). This paper addresses one key challenge in this context: dealing with high-degree social nodes. We propose a distributed mechanism - Frugal - that allows devices bound to a social user to make co-operative, but independent decisions to select social links to be mapped to overlay links. The approach leverages existing heuristics on computing minimum cost degree constrained spanning trees for graphs, but is novel in how it applies costs to graph edges using only local neighborhood information at each node, and how it operates in a distributed fashion. Simulation studies consider Frugal and two other policies (Greedy, Random) for capping the number of links in high-degree social nodes. Our simulation study carried out on social datasets indicates that significant degree reduction for high degree nodes/devices can be achieved without incurring major degradation in connectivity or performance loss in resulting overlay networks.

Summary The Pacific Rim Application and Grid Middleware Assembly (PRAGMA) is an international community of researchers that actively collaborate to address problems and challenges of common interest in eScience. The PRAGMA Experimental Network Testbed (PRAGMA‐ENT) was established with the goal of constructing an international software‐defined network (SDN) testbed to offer the necessary networking support to the PRAGMA cyberinfrastructure. PRAGMA‐ENT is isolated, and PRAGMA researchers have complete freedom to access network resources to develop, experiment, and evaluate new ideas without the concerns of interfering with production networks. In the first phase, PRAGMA‐ENT focused on establishing an international L2 backbone. With support from the Florida Lambda Rail, Internet2, PacificWave, Japan Gigabit Network, and TaiWan Advanced Research and Education Network, PRAGMA‐ENT backbone connects openflow‐enabled switches at University of Florida, University of California, San Diego, Nara Institute of Science and Technology (Japan), Osaka University (Japan), National Institute of Advanced Industrial Science and Technology (Japan), and National Applied Research Laboratories (Taiwan). The second phase of PRAGMA‐ENT consisted of an evaluation of technologies for the control plane that enables multiple experiments (ie, OpenFlow controllers) to coexist. Preliminary experiments with FlowVisor revealed some limitations leading to the development of a new approach, called AutoVFlow. This paper describes our experience in the establishment of PRAGMA‐ENT backbone (with international L2 links), its current status, and plans for the control plane. Discussion of preliminary application ideas, including optimization of routing control; multipath routing control; extending the backbone using overlay network; and remote visualization are also discussed.

Summary The GLEON Research And PRAGMA Lake Expedition—GRAPLE—is a collaborative effort between computer science and lake ecology researchers. It aims to improve our understanding and predictive capacity of the threats to the water quality of our freshwater resources, including climate change. This paper presents GRAPLEr, a distributed computing system used to address the modeling needs of GRAPLE researchers. GRAPLEr integrates and applies overlay virtual network, high‐throughput computing, and WEB service technologies in a novel way. First, its user‐level IP‐over‐P2P overlay network allows compute and storage resources distributed across independently administered institutions (including private and public clouds) to be aggregated into a common virtual network, despite the presence of firewalls and network address translators. Second, resources aggregated by the IP‐over‐P2P virtual network run unmodified high‐throughput‐computing middleware to enable large numbers of model simulations to be executed concurrently across the distributed computing resources. Third, a WEB service interface allows end users to submit job requests to the system using client libraries that integrate with the R statistical computing environment. The paper presents the GRAPLEr architecture, describes its implementation and reports on its performance for batches of general lake model simulations across 3 cloud infrastructures (University of Florida, CloudLab, and Microsoft Azure).

Many techniques have been proposed to provide, transparently, the abstraction of a layer-2 virtual network environment within a provider, e.g. by leveraging Software-Defined Networking (SDN). However, cloud providers often constrain layer-2 communication across instances; furthermore, SDN integration and layer-2 messaging between distinct domains distributed across the Internet is not possible, hindering the ability for tenants to deploy their virtual networks across providers. In contrast, overlay networks provide a flexible foundation for inter-cloud virtual private networking (VPN), by tunneling virtual network traffic through private, authenticated end-to-end overlay links. However, overlays inherently incur network virtualization overheads, including header encapsulation and user/kernel boundary crossing. This paper proposes a novel system -- VIAS (VIrtualization Acceleration over SDN) -- that delivers the flexibility of overlays for inter-cloud virtual private networking, while transparently applying SDN techniques (available in existing OpenFlow hardware or software switches) to selectively bypass overlay tunneling and achieve near-native performance for TCP/UDP flows within a provider. Architecturally, VIAS is unique in how it integrates SDN and overlay controllers in a distributed fashion to coordinate the management of virtual network links and flows. The approach is self-organizing, whereby overlay nodes can detect that peer endpoints are in the same network and program bypass flows between OpenFlow switches. While generally applicable, VIAS in particular applies to nested VMs/containers across cloud providers, supporting seamless communication within and across providers. VIAS has been implemented as an extension to an existing virtual network overlay platform (IP-over-P2P, IPOP) by integrating OpenFlow controller functionality with distributed overlay controllers. We evaluate the performance of VIAS in realistic cloud environments using an implementation based on IPOP, the RYU SDN framework, Open vSwitch, and LXC containers across various cloud environment including Amazon, Google compute engine, and CloudLab.

Applications on cloud infrastructures acquire virtual machines (VMs) from providers when necessary. The current interface for acquiring VMs from most providers, however, is too limiting for the tenants, in terms of granularity in which VMs can be acquired (e.g., small, medium, large, etc.), while giving very limited control over their placement. The former leads to VM underutilization, and the latter has performance implications, both translating into higher costs for the tenants. In this work, we leverage nested virtualization and a networking overlay to tackle these problems. We present Kangaroo, an Open Stack-based virtual infrastructure provider, and IPOPsm, a virtual networking switch for communication between nested VMs over different infrastructure VMs. In addition, we design and implement Skippy, the realization of our proposed virtual infrastructure API for programming Kangaroo. Our benchmarks show that through careful mapping of nested VMs to infrastructure VMs, Kangaroo achieves up to an order of magnitude better performance, with only half the cost on Amazon EC2. Further, Kangaroo's unified Open Stack API allows us to migrate an entire application between Amazon EC2 and our local Open Nebula deployment within a few minutes, without any downtime or modification to the application code.

Peer-to-peer systems are resilient in the presence of churn and uncorrelated failures. However, their behavior in extreme scenarios where massive correlated failures occur is not well-studied. Yet, there have been examples of situations where a country-scale fraction of Internet users have been disconnected from the rest of the network-for instance, when a government cuts connectivity to the outside world as a mechanism for suppression of uprisings. In this paper, we consider the effect of such partitions on topology and routing of structured and social-based unstructured P2P overlays, including a novel social-aware overlay. In particular, we consider nodes within a relatively small fraction of the network (2.5% or fewer Internet users), and study whether users can communicate with their (n-hop away) social neighbors in a peer-to-peer fashion after the partition. We perform an extensive simulation-based analysis to assess the probability for these communications to be possible. In our analysis, we consider both real and synthetic datasets of online social networks. Our results show that structured P2P overlay routability is severely hampered by country-scale partition events. In addition, the proposed social-based unstructured overlay network provides improved routability while maintaining a smaller number of links.

The increasing use of online social networks (OSNs) in emergency situations shows us a promising future of human cooperation through OSNs. Despite this intense interest, a number of fundamental limitations still exist, such as lack of appropriate conceptual models and limitations on cooperation meth

A fundamental problem encountered in designing decentralized social applications is the issue of efficiently locating target nodes in social peer-to-peer networks based on local information only. The unique “small-world” phenomenon of social networks shows that a typical pair of nodes is connected by very short chains of intermediate friends, and individuals are able to collectively discover such short paths. In this paper, we propose a decentralized algorithm that exploits this “small-world” phenomenon to discover efficient routes to reach target nodes in unstructured social overlays. In order to test the performance of this algorithm, we simulate it on a real-world social graph dataset crawled from a large online social networking website (LiveJournal). Compared with other related works, the simulation results show that our proposed decentralized social routing algorithm not only shortens the median length of the search path, but also increases the success rate of finding such routes between arbitrary pair of nodes in the LiveJournal social graph.

In the past few years, many peer-to-peer microblogging solutions have been proposed and/or implemented utilizing various technologies such as DHTs, multicast trees, and/or gossip protocols. These previous works address the issue of privacy and performance in a variety of ways including the use of session keys for message encryption or direct connections for low latency communication. We propose a decentralized microblogging service which takes advantage of available peer-to-peer virtual private networking (P2PVPN) technologies which provide privacy and low-latency communication in the common case of P2P messaging among social peers. Leveraging the private IP connectivity of P2PVPNs, our design utilizes both IP multicasting and random walks to ensure that peers are able to publish messages with varying degree of scope (i.e. friends, friends of friends, and/or the public). We study the implications of our data dissemination mechanism for a decentralized microblogging service through simulation-based analysis based on synthetic social graphs. Overall, our experimental results show that peers can effectively follow each other's updates with acceptable overhead. Through the use of our pseudo-random-walk algorithm, we estimate that, in a 900K social graph, with a TTL of 100, a user can retreive updates from anyone in the social graph 55% of the time, but by increasing the TTL to 400 that hit rate increases to 95%.

Microblogging services such as Twitter have become an indispensable communication tool on the Internet. However, due to their centralized nature, microblogging services have been susceptible to blocking by governments and powerful groups. To address this issue, we present a peer-to-peer microblogging service that is resistant to government intrusions and censorship. The strength of our design lies in the fact that we leverage trusted, peer-to-peer connections for the dissemination of information. Our proposed design consists mainly of two key components: a microblogging service which uses UDP and IP multicasting to push and pull updates, and a peer-to-peer VPN (SocialVPN) which enable IP multicasting over the Internet and provide direct IP connectivity among social peers. We also implemented and deployed a prototype to show the feasibility of our approach.

A core mission of the Integrated Digitized Biocollections (iDigBio) project is the building and deployment of a cloud computing environment customized to support the digitization workflow and integration of data from all U.S. non-federal biocollections. iDigBio chose to use cloud computing technologies to deliver a cyber infrastructure that is flexible, agile, resilient, and scalable to meet the needs of the biodiversity community. In this context, this paper describes the integration of open source cloud middleware, applications, and third party services using standard formats, protocols, and services. In addition, this paper demonstrates the value of the digitized information from collections in a broader scenario involving multiple disciplines.

OpenCL has emerged as the open standard for parallel programming for heterogeneous platforms enabling a uniform framework to discover, program, and distribute parallel workloads to the diverse set of compute units in the hardware. For that reason, there have been efforts exploring the advantages of parallelism from the OpenCL framework by offloading GPGPU workloads within an HPC cluster environment. In this paper, we present an OpenCL-based remote offloading framework designed for mobile platforms by shifting the motivation and advantages of using the OpenCL framework for the HPC cluster environment into mobile cloud computing where OpenCL workloads can be exported from a mobile node to the cloud. Furthermore, our offloading framework handles service discovery, access control, and data privacy by building the framework on top of a social peer-to-peer virtual private network, Social VPN. We developed a prototype implementation and deployed it into local- and wide-area environments to evaluate the performance improvement and energy implications of the proposed offloading framework. Our results show that, depending on the complexity of the workload and the amount of data transfer, the proposed architecture can achieve more energy efficient performance by offloading than executing locally.

Due to the popularity and importance of Parallel File Systems (PFSs) in modern High Performance Computing (HPC) centers, PFS designs and I/O optimizations are active research topics. However, the research process is often time-consuming and faces cost and complexity challenges in deploying experiments in real HPC systems. This paper describes PFSsim, a trace-driven simulator of distributed storage systems that allows the evaluation of PFS designs, I/O schedulers, network structures, and workloads. PFSsim differentiates itself from related work in that it provides a powerful platform featuring a modular design with high flexibility in the modeling of subsystems including the network, clients, data servers and I/O schedulers. It does so by designing the simulator to capture abstractions found in common PFSs. PFSsim also exposes script-based interfaces for detailed configurations. Experiments and validation against real systems considering sub-modules and the entire simulator show that PFSsim is capable of simulating a representative PFS (PVFS2) and of modeling different I/O scheduler algorithms with good fidelity. In addition, the simulation speed is also shown to be acceptable.

MapReduce is a popular large-scale parallel data processing framework. In the context of MapReduce processing on volunteer computing environments, it is important to devise scheduling and data placement policies that account for characteristics of opportunistic resources. This paper investigates availability characteristics of opportunistic resources with analyses based on log traces from the SETI@Home project. Based on the analysis, the paper devises heuristics to leverage the uptime of each available session to detect possibly long-lasting resources. Our proposed session uptime-based resource availability prediction approach shows a two-fold reduction in the number of service disturbance compared to an availability-rate based model. The paper paper investigates a heuristic that differentiates stable nodes from unstable nodes while increasing the chance of leveraging existing data blocks.

High Throughput Computing (HTC) platforms aggregate heterogeneous resources to provide vast amounts of computing power over a long period of time. Typical HTC systems, such as Condor and BOINC, rely on central managers for resource discovery and scheduling. While this approach simplifies deployment, it requires careful system configuration and management to ensure high availability and scalability. In this paper, we present a novel approach that integrates a self-organizing P2P overlay for scalable and timely discovery of resources with unmodified client/server job scheduling middleware in order to create HTC virtual resource Pools on Demand (PonD). This approach decouples resource discovery and scheduling from job execution/monitoring - a job submission dynamically generates an HTC platform based upon resources discovered through match-making from a large "sea" of resources in the P2P overlay and forms a "PonD" capable of leveraging unmodified HTC middleware for job execution and monitoring. We show that job scheduling time of our approach scales with O(log N), where N is the number of resources in a pool, through first-order analytical models and large-scale simulation results. To verify the practicality of PonD, we have implemented a prototype using Condor (called C-PonD), a structured P2P overlay, and a PonD creation module. Experimental results with the prototype in two WAN environments (PlanetLab and the FutureGrid cloud computing testbed) demonstrates the utility of C-PonD as a HTC approach without relying on a central repository for maintaining all resource information. Though the prototype is based on Condor, the decoupled nature of the system components - decentralized resource discovery, PonD creation, job execution/monitoring - is generally applicable to other grid computing middleware systems.

Virtualized IT infrastructures combined with virtual machine migration technologies have a potential to support IT services that are resilient to partial physical infrastructure failures caused by extreme events. This paper experimentally evaluates the migration of multiple VMs across long geographical distances - an activity that is required to move virtualized IT systems from a disaster site to a safe location. Taking into account the resource availability parameters observed after the Great East Japan Earthquake, experimental results show that if (1) service downtime in the order of minutes is acceptable, (2) VMs can be kept with small storage footprint, and (3) power and network are available for tens of minutes, it is possible to migrate tens of VMs from damaged sites to a very distant stable location.

Existing parallel file systems are unable to differentiate I/Os requests from concurrent applications and meet per-application bandwidth requirements. This limitation prevents applications from meeting their desired Quality of Service (QoS) as high-performance computing (HPC) systems continue to scale up. This paper presents vPFS, a new solution to address this challenge through a bandwidth virtualization layer for parallel file systems. vPFS employs user-level parallel file system proxies to interpose requests between native clients and servers and to schedule parallel I/Os from different applications based on configurable bandwidth management policies. vPFS is designed to be generic enough to support various scheduling algorithms and parallel file systems. Its utility and performance are studied with a prototype which virtualizes PVFS2, a widely used parallel file system. Enhanced proportional sharing schedulers are enabled based on the unique characteristics (parallel striped I/Os) and requirement (high throughput) of parallel storage systems. The enhancements include new threshold- and layout-driven scheduling synchronization schemes which reduce global communication overhead while delivering total-service fairness. An experimental evaluation using typical HPC benchmarks (IOR, NPB BTIO) shows that the throughput overhead of vPFS is small (;96% of target sharing ratio) for competing applications with diverse I/O patterns.

There is an increasing need for fault tolerance capabilities in logic devices brought about by the scaling of transistors to ever smaller geometries. This paper presents a hypervisor-based replication approach that can be applied to commodity hardware to allow for virtually lockstepped execution. It offers many of the benefits of hardware-based lockstep while being cheaper and easier to implement and more flexible in the configurations supported. A novel form of processor state fingerprinting is also presented, which can significantly reduce the fault detection latency. This further improves reliability by triggering rollback recovery before errors are recorded to a checkpoint. The mechanisms are validated using a full prototype and the benchmarks considered indicate an average performance overhead of approximately 14 percent with the possibility for significant optimization. Finally, a unique method of using virtual lockstep for fault injection testing is presented and used to show that significant detection latency reduction is achievable by comparing only a small amount of data across replicas.

This paper proposes and evaluates Self-Organizing Latency-Aware Resource Ensemble (SOLARE), a peer-to-peer self-organizing and self-managing cluster system based upon network coordinates and utility functions. In contrast to previous works, SOLARE is a fully decentralized clustering algorithm without any central units such as servers, super peers, cluster heads or landmarks. Furthermore, SOLARE allows for adaptability to dynamic network changes by monitoring the utility of a cluster and migrating nodes to other higher-utility clusters when the utility of an existing cluster is low. Quantitative, simulation-driven evaluations show that SOLARE is able to satisfy user demands expressed by utility functions that integrate system parameters in terms of intra cluster latencies and the number of cluster members. Also, we verify the ability of SOLARE to adapt to dynamic network changes through simulation based experiments that consider the number of nodes which migrate into another cluster and average utility value as nodes join SOLARE.

"Give a man a fish, feed him for a day. Teach a man to fish, feed him for a lifetime" -- Lau Tzu Large-scale grid computing projects such as TeraGrid and Open Science Grid provide researchers vast amounts of compute resources but with requirements that could limit access, results delayed due to potentially long job queues, and environments and policies that might affect a user's work flow. In many scenarios and in particular with the advent of Infrastructure-as-a-Service (IaaS) cloud computing, individual users and communities can benefit from less restrictive, dynamic systems that include a combination of local resources and on-demand resources provisioned by one or more IaaS provider. These types of scenarios benefit from flexibility in deploying resources, remote access, and environment configuration.

Recent virtualization-driven CPU architectural extensions involve tagging the hardware-managed Translation Look aside Buffer (TLB) entries to avoid TLB flushes during context switches, thereby sharing the TLB among multiple address spaces. While tagged TLBs are expected to improve the performance of virtualized workloads, a systematic evaluation of this improvement, its dependence on TLB and workload related factors and the performance implications of the contention arising from TLB sharing are yet to be investigated. This paper undertakes these investigations using a simulation-driven approach. We develop a simulation model for the tagged TLB and integrate it into a full-system simulation framework. Using this model, we show that the performance impact of using tagged TLBs ranges from 1% to 25% and is highly dependent on the size of the TLB, the TLB miss penalty and the nature of the workload and the type of tag used. The performance of consolidated workloads is also simulated and the observations from these simulations are used to highlight the performance variation due to resource contention in the shared TLB. Isolating the TLB behavior of one application in a consolidated workload from these variations due to the TLB contention by means of a static TLB usage control scheme is also explored. Furthermore, we show that the performance improvement due to tagged TLBs can be further increased by 1.4X for selected high-priority applications, by restricting the TLB usage of other low-priority workloads, in a consolidated workload scenario.

The ability to define domain names for resources in a collaborative virtual organization is usually reserved to network administrators through centralized domain name servers. We propose SocialDNS, a decentralized, naming service that gives individual collaborators the power to choose the domain nam

Decentralized and P2P (peer-to-peer) VPNs (virtual private networks) have recently become quite popular for connecting users in small to medium collaborative environments, such as academia, businesses, and homes. In the realm of VPNs, there exist centralized, decentralized, and P2P solutions. Centra

Voluntary Computing systems or Desktop Grids (DGs) enable sharing of commodity computing resources across the globe and have gained tremendous popularity among scientific research communities. Data management is one of the major challenges of adopting the Voluntary Computing paradigm for large data-intensive applications. To date, middleware for supporting such applications either lacks an efficient cooperative data distribution scheme or cannot easily accommodate existing data-intensive applications due to the requirement for using middleware-specific APIs.To address this challenge, in this paper we introduce Gator-Share, a data management framework that offers a file system interface and an extensible architecture designed to support multiple data transfer protocols, including BitTorrent, based on which we implement a cooperative data distribution service for DGs. It eases the integration with Desktop Grids and enables high-throughput data management for unmodified data-intensive applications. To improve the performance of BitTorrent in Desktop Grids, we have enhanced BitTorrent by making it fully decentralized and capable of supporting partial file downloading in an on-demand fashion.To justify this approach we present a quantitative evaluation of the framework in terms of data distribution efficiency. Experimental results show that the framework significantly improves the data dissemination performance for unmodified data-intensive applications compared to a traditional client/server architecture.

Due to the rising importance of virtualization, extensive efforts have gone into determining and improving the performance of workloads on virtualized platforms. This has resulted in a series of modifications to the leading architecture used in virtualized system (x86) by adding hardware support for virtualization, the latest of which is the addition of tags and tag comparators to the x86 TLB. In this context, it is necessary to have a thorough understanding of the TLB behavior of virtualized workloads and understand the change in this behavior with TLB related architectural parameters. One way of obtaining this understanding is by conducting a simulation-based study of the interaction of various micro-architectural parameters and their effect on the TLB behavior. However, the lack of suitable simulation frameworks makes such a study daunting. In this paper, we present a full-system simulation framework which is suitable for conducting such studies. We first motivate the need for TLB modeling in virtualized systems. Then, we present the framework, develop and validate a timing model for the TLB and evaluate the simulation speed when this model is used. Using the timing model, the influence of the TLB on workload performance is examined for a variety of single and multi-domain workloads and compared with equivalent non-virtualized workloads. It is found that the performance of virtualized workloads, in terms of instructions per cycle (IPC), can vary by 1% to 35% due to the TLB and that this IPC variation can be as much as 9 times the variation in non-virtualized workloads.

Archer is a simulation environment and computing resource for research in the field of computer architecture. Archer facilitates the creation of an on-demand computing grid, the deployment of simulation tools on this grid and the batch scheduling of large-scale simulation jobs on this grid. These features enable the use of Archer for simulation-based research as well as dissemination of tools and results among multiple research groups. This paper overviews and reports our experience in the use of Archer for collaborative research and for education.

Peer-to-Peer (P2P) overlays provide a framework for building distributed applications consisting of few to many resources with features including self-configuration, scalability, and resilience to node failures. Such systems have been successfully adopted in large-scale Internet services for content delivery networks, file sharing, and data storage. In small-scale systems, they can be useful to address privacy concerns as well as support for network applications that lack dedicated servers. The bootstrap problem, finding an existing peer in the overlay, remains a challenge to enabling these services for small-scale P2P systems. In large networks, the solution to the bootstrap problem has been the use of dedicated services, though creating and maintaining these systems requires expertise and resources, which constrain their usefulness and make them unappealing for small-scale systems. This paper surveys and summarizes requirements that allow peers potentially constrained by network connectivity to bootstrap small-scale overlays through the use of existing public overlays. In order to support bootstrapping, a public overlay must support the following requirements: a method for reflection in order to obtain publicly reachable addresses, so peers behind network address translators and firewalls can receive incoming connection requests; communication relaying to share public addresses and communicate when direct communication is not feasible; and rendezvous for discovering remote peers, when the overlay lacks stable membership. After presenting a survey of various public overlays, we identify two overlays that match the requirements: XMPP overlays, such as Google Talk and Live Journal Talk, and Brunet, a structured overlay based upon Symphony. We present qualitative experiences with prototypes that demonstrate the ability to bootstrap small-scale private structured overlays from public Brunet or XMPP infrastructures.

The rise in multi-core architectures has led to the abundance of computing resources on a chip. Virtualization has emerged as a way to efficiently partition and share these resources. Thus, the emphasis in micro-architecture design, especially in ×86, has shifted towards providing hardware support for better performance of VMs on bare metal. One of the areas of focus for these efforts is the Translation Lookaside Buffer (TLB). Recent modifications in the TLB include the addition of tags as a part of the TLB entry and the incorporation of hardware primitives to perform tag comparison during TLB lookup. In this paper we present the Tag Manager Table (TMT), a low-latency management architecture for tagging the TLB entries using process-specific identifiers (based on the CR3 register in ×86), and thereby reducing the number of flushes and the miss rate in the TLB. Using a full system simulation approach, we investigate the performance benefit of these tags and explore how it varies with the size of the TMT, the TLB architecture and the workload characteristics. We also perform a sensitivity analysis and quantify the relative importance of all these factors in determining the benefit from CR3 tagging. While our focus is on virtualized platforms, this approach is equally applicable for non virtualized environments.

Virtual networks (VNs) provide methods that simplify resource management, deal with connectivity constraints, and support legacy applications in distributed systems, by enabling global addressability of VN-connected machines through either a common layer 2 Ethernet or a NAT-free layer 3 IP network. This paper presents a novel VN design that supports dynamic, seamless addition of new resources with emphasis on scalability in a unified private IP address space. Key features of this system are: (1) Scalable connectivity via a P2P overlay with the ability to bypass overlay routing in LAN communications, (2) support for static and dynamic address allocation in conjunction with virtual nameservers through a distributed data store, and (3) support for transparent migration of IP endpoints across widearea networks.

Deploying virtual organizations (VOs) is difficult for small- and medium-scale collaborations: the overheads in establishing and managing trust, and in deploying and managing computational resources distributed across multiple organizations are daunting to many potential users, presenting a barrier to entry that significantly hinders wider deployment of VOs. We advocate an approach where social networking and self-configuring overlay virtual networks are integrated in a novel way that allows simple deployment and management of ad-hoc infrastructures for VOs. There are three central principles in our approach: (1) user relationships which have been increasingly recorded in social networking systems provide the opportunity to bootstrap trust relationships; (2) connections established at a social networking layer can efficiently be mapped to the IP layer of virtual network overlays to support existing TCP/IP applications for collaboration and resource sharing while maintaining security against untrusted parties; and (3) systems integrating social and virtual networks can be self-configuring, enabling deployment of collaborative infrastructures by non-experts. We discuss motivations for this approach, describe a prototype implementation which integrates the Facebook social network and the IPOP overlay network, and discuss a use case scenario towards ad-hoc social cycle-sharing virtual Condor pools.

Self-configuring virtual networks rely on structured P2P routing to provide seamless connectivity among nodes through overlay routing of virtual IP packets, support decentralized hole-punching to establish bi-directional communication links among nodes behind network address translators, and dynamic configuration of virtual IP addresses. Our experiences with deployments of virtual networks in support of wide-area overlays of virtual workstations (WOWs) reveal that connectivity constraints imposed by symmetric NATs and by Internet route outages often hinder P2P overlay structure maintenance and routability, subsequently limiting the ability of WOWs to deliver high-throughput computing through aggregation of resources in different domains.

We describe and evaluate a thin client solution for desktop grid computing based on virtual machine appliances whose images are fetched on-demand and on a per-block basis over wide-area networks. The approach uses a distributed file system redirection mechanism which enables the use of unmodified NFS clients/servers and local buffering of file system modifications during the appliances lifetime. The file system redirection technique is achieved through user-level proxies, and can be integrated with virtual private network overlays to provide transparent access to image servers even if they are behind firewalls. We have implemented and evaluated a prototype system which allows thin client diskless appliances to boot over a proxy VM bringing on-demand only a small fraction of the appliance image (16 MB out of WOMB) and showing low runtime overhead for CPU-intensive applications. The paper also presents decentralized mechanisms to support seamless image version upgrades.

Research projects in many fields are increasingly reliant on the use of computer-based simulation and computing grids. Many projects have successfully leveraged voluntary computing infrastructures by developing and distributing "@home" applications using the BOINC framework. Through generous contributions from the general public, these systems now have a computing backbone on which to have their data processed or simulations run. A shortcoming of such systems is that most users are often limited to contributing resources and few users are capable of developing or porting their own applications in order to use these resources. While many users are satisfied with receiving points (an intangible good) in return for their contribution, the need to port applications presents a barrier to entry to many other users who can potentially benefit from using the voluntary resources. In this paper, we describe enhancements made to the "grid appliance", a virtual machine based system which enables an execution environment in which users are given the opportunity to voluntarily share (providing and using) resources and run unmodified x86/Linux applications. Voluntary grids introduce a host of issues to tackle, most importantly getting users involved quickly. With that in mind, the grid appliance provides many tools for making a user-friendly environment for users, developers, and administrators. This paper summarizes the challenges of getting users involved, reducing the overhead for administrators, and describes the solutions used in the grid appliance.

The thrust of this paper is to profile the execution phases of applications, which helps optimize the efficiency of the underlying resources. Here we present a novel system-level application-resource-demand phase analysis and prediction approach in support of on-demand resource provisioning. The process we follow is to explore large-scale behavior of applications’ resource consumption, followed by analysis using a set of algorithms based on clustering. The phase profile, which learns from historical runs, is used to classify and predict future phase behavior. This process takes into consideration applications’s resource consumption patterns, phase transition costs and penalties associated with Service-Level Agreements (SLA) violations. Our experimental results with WorldCup98 replay web access logs show that prediction accuracies around 84% or larger for ten-phase cases can be achieved for network performance traces.

Deploying a modern software service usually involves installing several software components, and configuring these components properly to realize the complex interdependencies between them. This process, which accounts for a significant portion of information technology (IT) cost, is complex and error-prone. In this paper, we propose delta configuration - an approach that reduces the cost of software deployment by eliminating a large number of choices on parameter values that administrators have to make during deployment. In delta configuration, the complex software stack of a distributed service is first installed and tested in a test environment. The resulting software images are then captured and used for deployment in production environments. To deploy a software service, we only need to copy these pre-configured software images into a production environment and modify them to account for the difference between the test environment and a production environment. We have implemented a prototype system that achieves software deployment using delta configuration of the configuration state captured inside virtual machines. We perform a case study to demonstrate that our scheme leads to substantial reduction in complexity for the customer, over the traditional software deployment method.

Wide-area overlays of virtual workstations (WOWs) have been shown to provide excellent infrastructure for deploying high throughput computing environments on commodity desktop machines by (1) offering scalability to a large number of nodes, (2) facilitating addition of new nodes even if they are behind NATs/firewalls and (3) supporting unmodified applications and middleware. However, deployment of WOWs from scratch still requires setting up a bootstrapping network and managing centralized DHCP servers for IP address management. In this paper we describe novel techniques that allow multiple users to create independent, isolated virtual IP namespaces for their WOWs without requiring a dedicated bootstrapping infrastructure, and to provision dynamic host configuration (e.g. IP addresses) to unmodified DHCP clients without requiring the setup and management of a central DHCP server. We give qualitative and quantitative arguments to establish the feasibility of our approach.

The integration of multiple predictors promises higher prediction accuracy than the accuracy that can be obtained with a single predictor. The challenge is how to select the best predictor at any given moment. Traditionally, multiple predictors are run in parallel and the one that generates the best result is selected for prediction. In this paper, we propose a novel approach for predictor integration based on the learning of historical predictions. It uses classification algorithms such as k-Nearest Neighbor (k-NN) based supervised learning to forecast the best predictor for the workload under study. Then only the forecasted best predictor is run for prediction. Our experimental results show that it achieved 20.18% higher best predictor forecasting accuracy than the cumulative MSB based predictor selection approach used in the popular network weather service system. In addition, it outperformed the observed most accurate single predictor in the pool for 44.23% of the performance traces.

This paper presents a flexible technique that can be applied to many-core architectures to exploit idle resources and ensure reliable system operation. A dynamic fault tolerance layer is interposed between the hardware and OS through the use of a hypervisor. The introduction of a single point of failure is avoided by incorporating the hypervisor into the sphere of replication. This approach simplifies implementation over specialized hardware- or OS-based techniques while offering flexibility in the level of protection provided ranging from duplex to Byzantine protection. The feasibility of the approach is considered for both near- and long-term computing platforms.

A grid-wide distributed file system provides convenient data access interfaces that facilitate fine-grained cross-domain data sharing and collaboration. However, existing widely-adopted distributed file systems do not meet the security requirements for grid systems. This paper presents a Secure Grid File System (SGFS) which supports GSI-based authentication and access control, end-to-end message privacy, and integrity. It employs user-level virtualization of NFS to provide transparent grid data access leveraging existing, unmodified clients and servers. It supports user and application-tailored security customization per SGFS session, and leverages secure management services to control and configure the sessions. The system conforms to the GSI grid security infrastructure and allows for seamless integration with other grid middleware. A SGFS prototype is evaluated with both file system benchmarks and typical applications, which demonstrates that it can achieve strong security with an acceptable overhead, and substantially outperform native NFS in wide-area environments by using disk caching.

Virtualization provides levels of execution isolation and service partition that are desirable in many usage scenarios, but its associated overheads are a major impediment for wide deployment of virtualized environments. While the virtualization cost depends heavily on workloads, it has been demonstrated that the overhead is much higher with I/O intensive workloads compared to those which are compute-intensive. Unfortunately, the architectural reasons behind the I/O performance overheads are not well understood. Early research in characterizing these penalties has shown that cache misses and TLB related overheads contribute to most of I/O virtualization cost. While most of these evaluations are done using measurements, in this paper we present an execution-driven simulation based analysis methodology with symbol annotation as a means of evaluating the performance of virtualized workloads. This methodology provides detailed information at the architectural level (with a focus on cache and TLB) and allows designers to evaluate potential hardware enhancements to reduce virtualization overhead. We apply this methodology to study the network I/O performance of Xen (as a case study) in a full system simulation environment, using detailed cache and TLB models to profile and characterize software and hardware hotspots. By applying symbol annotation to the instruction flow reported by the execution driven simulator we derive function level call flow information. We follow the anatomy of I/O processing in a virtualized platform for network transmit and receive scenarios and demonstrate the impact of cache scaling and TLB size scaling on performance.

Abstract Science gateways require the easy enabling of legacy scientific applications on computing Grids and the generation of user‐friendly interfaces that hide the complexity of the Grid from the user. This paper presents the In‐VIGO approach to the creation and management of science gateways. First, we discuss the virtualization of machines, networks and data to facilitate the dynamic creation of secure execution environments that meet application requirements. Then we discuss the virtualization of applications, i.e. the execution on shared resources of multiple isolated application instances with customized behavior, in the context of In‐VIGO. A Virtual Application Service (VAS) architecture for automatically generating, customizing, deploying, and using virtual applications as Grid services is then described. Starting with a grammar‐based description of the command‐line syntax, the automated process generates the VAS description and the VAS implementation (code for application encapsulation and data binding) that is deployed and made available through a Web interface. A VAS can be customized on a per‐user basis by restricting the capabilities of the original application or by adding to it features such as parameter sweeping. This is a scalable approach to the integration of scientific applications as services into Grids and can be applied to any tool with an arbitrarily complex command‐line syntax. Copyright © 2006 John Wiley & Sons, Ltd.

This paper presents a case study of a Grid-enabled implementation of light scattering spectroscopy (LSS). The LSS technique allows noninvasive detection of precancerous changes in human epithelium, differentiating from traditional biopsies by allowing in vivo diagnosis of tissue samples and quantitative analyses of parameters related to cancerous changes via numerical techniques. This paper describes the architecture of GridLSS and its integration with a Web-based Grid computing portal. GridLSS solves an optimization problem of determining the light scattering spectrum that best fits experimental spectral data among a large set of spectra computed analytically using rigorous Mie theory. The novel approach taken in this paper is based on the precomputation and storage of Mie theory spectra in lookup databases that are queried during the minimization process. The paper makes three important contributions: 1) it presents a novel parallel application for LSS analysis that delivers high performance in wide-area distributed computing environment; 2) it evaluates and analyzes the performance of this application in cluster-based high-performance computing environments that are typical of Grid deployments; and 3) it shows that the performance of GridLSS benefits significantly from the use of on-demand Grid data transfers based on virtualized distributed file systems and from user-level caches for remote file system data.

This paper describes WOW, a distributed system that combines virtual machine, overlay networking and peer-to-peer techniques to create scalable wide-area networks of virtual workstations for high-throughput computing. The system is architected to: facilitate the addition of nodes to a pool of resources through the use of system virtual machines (VMs) and self-organizing virtual network links; to maintain IP connectivity even if VMs migrate across network domains; and to present to end-users and applications an environment that is functionally identical to a local-area network or cluster of workstations. We describe a novel, extensible user-level decentralized technique to discover, establish and maintain overlay links to tunnel IP packets over different transports (including UDP and TCP) and across firewalls. We also report on several experiments conducted on a testbed WOW deployment with 118 P2P router nodes over PlanetLab and 33 VMware-based VM nodes distributed across six firewalled domains. Experiments show that the latency in joining a WOW network is of the order of seconds: in a set of 300 trials, 90% of the nodes self-configured P2P routes within 10 seconds, and more than 99% established direct connections to other nodes within 200 seconds. Experiments also show that the testbed delivers good performance for two unmodified, representative benchmarks drawn from the life-sciences domain. The testbed WOW achieves an overall throughput of 53 jobs/minute for PBS-scheduled executions of the MEME application (with average single-job sequential running time of 24.1s) and a parallel speedup of 13.5 for the PVM-based fastDNAml application. Experiments also demonstrate that the system is capable of seamlessly maintaining connectivity at the virtual IP layer for typical client/server applications (NFS, SSH, PBS) when VMs migrate across a WAN

The inability to perform optimizations based on application-specific information presents a hurdle to the deployment of pervasive LAN file systems across WAN environments. This paper proposes a novel approach addressing this problem through application-tailored caching and consistency in widearea file systems. It leverages widely available Network File System (NFS) deployments without any modifications to kernels nor applications, and employs middleware to dynamically establish Grid-wide Virtual File System (GVFS) sessions with application-tailored cache consistency. Two consistency models are discussed in this paper: a relaxed model based on invalidation polling, and a stronger model based on delegation and callback. Experimental evaluation based on microbenchmarks and scientific applications show that with application-tailored cache consistency, GVFS is able to both improve application runtimes and reduce server load significantly, compared to kernel-level NFS in WAN.

Peer-to-peer (P2P) networks have mostly focused on task oriented networking, where networks are constructed for single applications, i.e. file-sharing, DNS caching, etc. In this work, we introduce IPOP, a system for creating virtual IP networks on top of a P2P overlay. IPOP enables seamless access to grid resources spanning multiple domains by aggregating them into a virtual IP network that is completely isolated from the physical network. The virtual IP network provided by IPOP supports deployment of existing IP-based protocols over a robust, self-configuring P2P overlay. We present implementation details as well as experimental measurement results taken from LAN, WAN, and Planet-Lab tests

Application awareness is an important factor of efficient resource scheduling. This paper introduces a novel approach for application classification based on the principal component analysis (PCA) and the k-nearest neighbor (k-NN) classifier. This approach is used to assist scheduling in heterogeneous computing environments. It helps to reduce the dimensionality of the performance feature space and classify applications based on extracted features. The classification considers four dimensions: CPU-intensive, I/O and paging-intensive, network-intensive, and idle. Application class information and the statistical abstracts of the application behavior are learned over historical runs and used to assist multi-dimensional resource scheduling. This paper describes a prototype classifier for application-centric virtual machines. Experimental results show that scheduling decisions made with the assistance of the application class information, improved system throughput by 22.11% on average, for a set of three benchmark applications.

This poster describes current work on the application of network virtualization and peer-to-peer routing techniques that overlay IP traffic and provide seamless connectivity to virtual machines in grid computing. Such IP-over-P2P virtual network - IPOP - is an overlay network that uses a virtual IP address space and allows nodes that belong to a grid to be seamlessly pooled together. The overlay network is self-configured as nodes join/leave the virtualized grid, and IP-level bi-directional connectivity among peers is provided. The decoupling provided by the overlay enables grid applications to leverage a wealth of IP-based software typically available in local-area environments. Such virtual networks, when combined to complementary resource virtualization techniques provided by O/S virtual machines [Xen, Rarham et al. (2003), VMware, Sugerman et al., (2001), User Mode Linux, Dike, J. (2000)], provide a scalable framework for dealing with a fundamental goal of grid computing: sharing resources in a secure and flexible manner by Figueiredo, Dinda and Fortes (2003)

This poster briefly introduces two resource-virtualization techniques needed for the creation of virtual(ized) grids: virtual networks and virtual application services. The former provides bidirectional network connectivity even in the presence of firewalls, network address translation gateways and proxies by creating virtual routers and virtual IP space. The later allows automated creation and deployment of legacy applications into grids by generating a virtual application service that allows the execution in shared resources of multiple isolated application instances with customized behavior.

This paper presents novel service-based grid data management middleware that leverages standards defined by WSRF specifications to create and manage dynamic grid file system sessions. A unique aspect of the service is that the sessions it creates can be customized to address application data transfer needs. Application-tailored configurations enable selection of both performance-related features (block-based partial file transfers and/or whole-file transfers, cache parameters and consistency models) and reliability features (file system copy-on-write checkpointing to aid recovery of client-side failures; replication, autonomous failure detection and data access redirection for server-side failures). These enhancements, in addition to cross-domain user identity mapping and encrypted communication, are implemented via user level proxies managed by the service, requiring no changes to existing kernels. Sessions established using the service is mounted as distributed file systems and can be used transparently by unmodified binary applications. The paper analyzes the use of the service to support virtual machine based grid systems and workflow execution, and also reports on the performance and reliability of service managed wide-area file system sessions with experiments based on scientific applications (NanoMOS/Matlab, CHID, GAUSS and SPECseis).

The operational principles, components, and organization of a Grid-computing infrastructure called In-VIGO (standing for In Virtual Information Grid Organizations)are described. In-VIGO enables computational engineering and science in virtual information Grid organizations. Its distinctive feature is the extensive use of virtualization technologies to provide secure execution environments as needed by tools and users. This paper reviews and motivates the requirements of a cyber infrastructure for computational nanoelectronics. It then explains how such requirements are addressed by the In-VIGO middleware approach, which uses virtualized resources to build computational Grids. The architecture and key design aspects of its first deployed version-In-VIGO 1.0-are presented. It is operational and currently being used to enable the use of computational electronics tools over the Web. Aspects of the design and architecture of the next version of In-VIGO are also presented. It uses Web services standards and components, and lessons learned from In-VIGO 1.0.

This paper examines the architecture of present day systems and shows that they are not trustworthy enough to support certain DRM features/restrictions, even when the DRM delivery system exclusively utilizes signed and protected operating system components. This weakness was discovered while creating a technique for remote transfer of audio streams generated by a Virtual Machine Monitor (VMM), to achieve network transparency for audio devices. The technique is based on the implementation of hosted I/O VMMs that intercept device I/O instructions executed by a "guest" O/S and emulate them through system calls processed by device drivers of a "host" O/S. The design consists of a virtual audio device driver that forwards sound streams to a user-level network server. Because (1) the virtual device intercepts audio data in an unprotected format (WAV), regardless of which application and file format are in use by the guest O/S, (2) modern virtual machine-based systems already achieve performance levels that allow for real-time audio playback, the playback only model of service/restriction imposed by some content delivery businesses is rendered ineffective by this technique. It enables Fair Use of DRM enabled media by allowing the user to make a copy of legally purchased audio media and time-shifting of Internet Radio stations. Experiments have shown that audibly perfect copies of media played by a VM "guest" can be made in PCM/WAV format, even though DRM-enabling features are present in the "guest" O/S drivers and media players. This paper also draws attention to the fact that the VM should be considered while designing the security and DRM capabilities in future general-purpose systems since a device driver in between the VMM and the host O/S has the potential of being an eavesdropper and a malicious end user.

Summary form only given. Single sign-on (SSO) is an essential desired feature of computational grids. Its implementation is challenging because resources cross administrative domains and are managed by heterogeneous access schemes. We present an approach for single sign-on in a deployed functioning grid called In-VIGO. The approach relies on decoupling grid user accounts from local user accounts and making use of role-based access control lists. Role-based accesses via delegation mechanisms using short-lived user identities enable In-VIGO to handle interactive applications and application-specific authentication mechanisms. This capability is not present in existing grid architectures. SSO implementations for usage scenarios in In-VIGO are described to highlight the applicability of the proposed approach. In particular, access to interactive applications with their own security mechanisms, such as VNC, and access to remote data can be achieved using proxies that delegate In-VIGO user access via short-lived user identities.

Virtual machines provide flexible, powerful execution environments for Grid computing, offering isolation and security mechanisms complementary to operating systems, customization and encapsulation of entire application environments, and support for legacy applications. This paper describes a Grid service — VMPlant — that provides for automated configuration and creation of flexible VMs that, once configured to meet application needs, can then subsequently be copied ("cloned") and dynamically instantiated to provide homogeneous execution environments across distributed Grid resources. In combination with complementary middleware for user, data and resource management, the functionality enabled by VMPlant allows for problem-solving environments to deliver Grid applications to users with unprecedented flexibility. VMPlant supports a graph-based model for the definition of customized VM configuration actions; partial graph matching, VM state storage and "cloning" for efficient creation. This paper presents the VMPlant architecture, describes a prototype implementation of the service, and presents an analysis of its performance.

We advocate a novel approach to grid computing that is based on a combination of "classic" operating system level virtual machines (VMs) and middleware mechanisms to manage VMs in a distributed environment. The abstraction is that of dynamically instantiated and mobile VMs that are a combination of traditional OS processes (the VM monitors) and files (the VM state). We give qualitative arguments that justify our approach in terms of security, isolation, customization, legacy support and resource control, and we show quantitative results that demonstrate the feasibility of our approach front a performance perspective. Finally, we describe the middleware challenges implied by the approach and an architecture for grid computing using virtual machines.

Describes a virtual file system that allows data to be transferred on demand between storage and computational servers for the duration of a computing session. The solution works with unmodified applications (even commercial ones) running on standard operating systems and hardware. The virtual file system employs software proxies to broker transactions between standard NFS (Network File System) clients and servers; the proxies are dynamically configured and controlled by computational grid middleware. The approach has been implemented and extensively exercised in the context of PUNCH (Purdue University Network Computing Hubs), an operational computing portal that has more than 1,500 users across 24 countries. The results show that the virtual file system performs well in comparison to native NFS: performance analyses show that the proxy incurs mean overheads of 1% and 18% with respect to native NFS for a single-client execution of the Andrew benchmark in two representative computing environments, and that the average overhead for eight clients can be reduced to within 1% of native NFS with concurrent proxies.

Data dependence speculation allows a compiler to relax the constraint of data-independence to issue tasks in parallel, increasing the potential for automatic extraction of parallelism from sequential programs. The paper proposes hardware mechanisms to support a data-dependence speculative distributed shared-memory (DDSM) architecture that enable speculative parallelization of programs with irregular data structures and inherent coarse-grin parallelism. Efficient support for coarse-grain tasks requires large buffers for speculative data; DDSM leverages cache and directory structures to provide large buffers that are managed transparently from applications. The proposed cache and directory extensions provide support for distributed speculative versions of cache blocks, run-time detection of dependence violations, and program-order reconciliation of cache blocks. The paper describes the DDSM architecture and presents a simulation-based evaluation of its performance on five benchmarks chosen from the Spec95 and Olden suites. The proposed system yields simulated speedups of 3.8 to 12.5 in a 16-node configuration for programs with coarse-grain speculative windows (millions of instructions and hundreds of KBytes of speculative data).

This paper elaborates on mechanisms by which users, data, and applications can bedecoupledfrom individual computers and administrative domains. The mechanisms, which consist of logical user accounts and a virtual #le system, introduce a layer of abstraction between the physical computing infrastructure and the virtual computational grid perceived by users. This abstraction converts compute servers into interchangeable parts, allowing a computational grid to assemble computing systems at run time without being limited by the traditional constraints associated with user accounts, #le systems, and administrative domains. The described approach has already been deployedinthe

This paper explores area/parallelism tradeoffs in the design of distributed shared-memory (DSM) multiprocessors built out of large single-chip computing nodes. In this context, area-efficiency arguments motivate a heterogeneous organization consisting of few nodes with large caches designed for single-thread parallelism, and a larger number of nodes with smaller caches designed for multi-thread parallelism. Quantitative performance of such organization is reported for a set of homogeneous multiprocessor programs from the SPLASH-2 benchmark suite. These programs are mapped onto the heterogeneous processors without source code modifications via static thread assignment policies. Simulation-based analysis is used to compare the performance of heterogeneous and homogeneous DSMs that occupy the same silicon area. The analysis shows that a 4-node heterogeneous DSM with 21 processors outperforms its homogeneous counterpart with 4 processors by an average age of 36% for the studied multiprocessor workload, while having the same performance for sequential codes. A sensitivity analysis based on a factorial design experiment is used to study the implications of processor, memory, and network heterogeneity on overall cost and performance of a heterogeneous DSM. The studied benchmarks are affected, on average, primarily by heterogeneity in processor performance (59.3%), followed by cache sizes (18.2%), memory latency (14.6%), and network latency (5.6%).

We present a new parallel programming tool environment that is (1) accessible and executable "anytime, anywhere," through standard Web browsers and (2) integrated in that it provides tools that adhere to a common underlying methodology for parallel programming and performance tuning. The environment is based on a new network computing infrastructure, developed at Purdue University. We evaluate our environment qualitatively by comparing our tool access method with conventional schemes of software download and installation. We also quantitatively evaluate the efficiency of interactive tool access in our environment. We do this by measuring the response times of various functions of the URSA MINOR tool and compare them with those of a Java Applet-based "anytime, anywhere" tool access method. We found that our environment offers significant advantages in terms of tool accessibility, integration, and efficiency.

No abstract available.