This paper presents an ontology-based approach for data quality inference on streaming observation data originating from large-scale sensor networks. We evaluate this approach in the context of an existing river basin monitoring program called the Intelligent River®. Our current methods for data quality evaluation are compared with the ontology-based inference methods described in this paper. We present an architecture that incorporates semantic inference into a publish/subscribe messaging middleware, allowing data quality inference to occur on real-time data streams. Our preliminary benchmark results indicate delays of 100ms for basic data quality checks based on an existing semantic web software framework. We demonstrate how these results can be maintained under increasing sensor data traffic rates by allowing inference software agents to work in parallel. These results indicate that data quality inference using the semantic sensor network paradigm is viable solution for data intensive, large-scale sensor networks.

This paper presents the various mechanisms for virtual machine image distribution within a large batch farm and between sites that offer cloud computing services. The work is presented within the context of the Large Hadron Collider Computing Grid (LCG), it has two main goals. First it aims at presenting the CERN specific mechanisms that have been put in place to test the pre-staging of virtual machine images within a large cloud infrastructure of several hundred physical hosts. Second it introduces the basis of a policy for trusting and distributing virtual machine images between sites of the LCG. Finally experimental results are shown for the distribution of a 10 GB virtual machine image distributed to over 400 physical nodes using a binary tree and a Bit Torrent algorithm. Results show that images can be pre-staged within 30 minutes.

This paper presents the results of using the Kestrel workload management system to test operating a Virtual Organization Cluster (VOC) across multiple sites. A Many-Task Computing (MTC) framework based on the Extensible Messaging and Presence Protocol (XMPP), Kestrel presents a special purpose scheduler that can offer better VOC scalability under certain workload assumptions, namely CPU bound processes and bag-of-tasks jobs.

Virtual organization clusters are systems comprised of virtual machines that provide dedicated computing clusters for each individual virtual organization. The design of these clusters allows individual virtual machines to be independent of the underlying physical hardware, potentially allowing virtual clusters to span multiple grid sites. A major challenge in using virtual organization clusters as a grid computing abstraction arises from the need to schedule and provision physical resources to run the virtual machines. This paper describes a virtual cluster scheduler implementation based on the Condor high throughput computing system. By means of real-time monitoring of the Condor job queue, virtual machines that belong to individual virtual organizations are provisioned and booted. Jobs belonging to each virtual organization are then run on the organization-specific virtual machines, which form a cluster dedicated to the specific organization. Once the queued jobs have executed, the virtual machines are terminated, thereby allowing the physical resources to be re-claimed. Tests of this system were conducted using synthetic workloads, demonstrating that dynamic provisioning of virtual machines preserves system throughput for all but the shortest-running of grid jobs, without undue increase in scheduling latency.

Cloud services that provide virtualized computational clusters present a dichotomy of systems management challenges, as the virtual clusters may be owned and administered by one entity, while the underlying physical fabric may belong to a different entity. On the physical fabric, scalable tools that ldquopushrdquo configuration changes and software updates to the compute nodes are effective, since the physical system administrators have complete system access. However, virtual clusters executing atop federated Grid sites may not be directly reachable for management purposes, as network or policy limitations may prevent unsolicited connections to virtual compute nodes. For these systems, a distributed middleware solution could permit the compute nodes to ldquopullrdquo updates from a centralized server, thereby permitting the management of virtual compute nodes that are inaccessible to the system administrator. This paper compares both models of system administration and describes emerging software utilities for managing both the physical fabric and the virtual clusters.

Sharing traditional clusters based on multiprogramming systems among different virtual organizations (VOs) can lead to complex situations resulting from the differing software requirements of each VO. This complexity could be eliminated if each cluster computing system supported only a single VO, thereby permitting the VO to customize the operating system and software selection available on its private cluster. While dedicating entire physical clusters on the Grid to single VOs is not practical in terms of cost and scale, an equivalent separation of VOs may be accomplished by deploying clusters of virtual machines (VMs) in a manner that gives each VO its own virtual cluster. Such virtual organization clusters (VOCs) can have numerous benefits, including isolation of VOs from one another, independence of each VOC from the underlying hardware, allocation of physical resources on a per-VO basis, and clear separation of administrative responsibilities between the physical fabric provider and the VO itself. Initial results of implementing a complete system utilizing the proposed Virtual Organization Cluster Model confirm the administrative simplicity of isolating VO software from the physical system. End-user computational jobs submitted through the Grid are executed only on the virtual cluster supporting the respective VO, and each VO has substantial administrative flexibility in terms of software choice and system configuration. Performance tests using the Kernel-based virtual machine (KVM) hypervisor indicated a visualization overhead of under 10% for latency-tolerant scientific applications, such as those that would be submitted to a standard or vanilla Condor universe. Latency-sensitive applications, such as MPI, experience substantial performance degradation with visualization overheads on the order of 60%. These results suggest that VOCs are suitable for high- throughput computing (HTC) applications, where real-time network performance is not critical. VOCs might also be useful for high-performance computing (HPC) applications if virtual network performance can be sufficiently improved.

By federating resources from multiple domains, a shared infrastructure provides aggregated computation resources to a large number of users. With rapid advances in virtualization technologies, we propose the concept of virtual distributed environments as a new sharing paradigm for a multi-domain shared infrastructure. Such virtual environments provide users with confined, customized platforms to execute legacy parallel/distributed applications. Furthermore, we propose to support autonomic adaptation of virtual distributed environments, driven by both dynamic availability of infrastructure resources and dynamic application resource demand. We identify new research challenges and describe our on-going work and preliminary results. 1