An HPC cloud, a flexible and robust cloud computing service specially dedicated to high performance computing, is a promising future e-Science platform. In cloud computing, virtualization is widely used to achieve flexibility and security. Virtualization makes migration or checkpoint/restart of computing elements (virtual machines) easy, and such features are useful for realizing fault tolerance and server consolidations. However, in widely used virtualization schemes, I/O devices are also virtualized, and thus I/O performance is severely degraded. To cope with this problem, VMM-bypass I/O technologies, including PCI passthrough and SR-IOV, in which the I/O overhead can be significantly reduced, have been introduced. However, such VMM-bypass I/O technologies make it impossible to migrate or checkpoint/restart virtual machines, since virtual machines are directly attached to hardware devices. This paper proposes a novel and practical mechanism, called Symbiotic Virtualization (SymVirt), for enabling migration and checkpoint/restart on a virtualized cluster with VMM-bypass I/O devices, without the virtualization overhead during normal operations. SymVirt allows a VMM to cooperate with a message passing layer on the guest OS, then it realizes VM-level migration and checkpoint/restart by using a combination of a PCI hotplug and coordination of distributed VMMs. We have implemented the proposed mechanism on top of QEMU/KVM and the Open MPI system. All PCI devices, including Infiniband and Myrinet, are supported without implementing specific para-virtualized drivers; and it is not necessary to modify either of the MPI runtime and applications. Using the proposed mechanism, we demonstrate reactive and proactive FT mechanisms on a virtualized Infiniband cluster. We have confirmed the effectiveness using both a memory intensive micro benchmark and the NAS parallel benchmark. Moreover, we also show that postcopy live migration enables us to reduce the down time of an application as the memory footprint increases.

Intercloud is a promising technology for data intensive applications. However, an important issue for Intercloud applications is orchestration of various virtualized and performance-assured resources, not only computers, but also network and storage, provided from multiple domains. We have been developing an advance reservation-based resource management framework, called Grid ARS, which can integrate heterogeneous resources and construct a performance-assured virtual infrastructure over Intercloud environment. Grid ARS provides four services that address resource management, resource allocation planning, provisioning and monitoring of the constructed virtual infrastructure. Grid ARS has been developed using common Web services technologies and standards. In this paper, we present overview of Grid ARS and its service components and describe Grid ARS demonstration challenges, demonstration at GLIF2010 and SC10 and OGF NSI interoperation in 2011.

The understanding of H diffusion in materials is pivotal to designing suitable processes. Though a nudged elastic band (NEB)+molecular dynamics (MD)/quantum mechanics (QM) algorithm has been developed to simulate H diffusion in materials by our group, it is often not computationally feasible for large-scale models on a conventional single system. We thus gridify the NEB+MD/QM algorithm on the top of an integrated framework developed by our group. A two days simulation on H diffusion in alumina has been successfully carried out over a trans-pacific grid infrastructure consisting of supercomputers provided by TeraGrid and AIST. In this paper, we describe the NEB+MD/QM algorithm, briefly introduce the framework middleware, present the grid enablement work, and report the techniques to achieve fault-tolerance and load-balance for sustainable simulation. We believe our experience is of benefit to both middleware developers and application users.

Abstract Many production Grid and e‐Science infrastructures have begun to offer services to end‐users during the past several years with an increasing number of scientific applications that require access to a wide variety of resources and services in multiple Grids. Therefore, the Grid Interoperation Now—Community Group of the Open Grid Forum—organizes and manages interoperation efforts among those production Grid infrastructures to reach the goal of a world‐wide Grid vision on a technical level in the near future. This contribution highlights fundamental approaches of the group and discusses open standards in the context of production e‐Science infrastructures. Copyright © 2009 John Wiley & Sons, Ltd.

The authors have been leading the GEO Grid project since 2005. This project is primarily aiming at providing an e-Science infrastructure for the worldwide Earth Sciences community. The GEO Grid is designed to virtually integrate all relevant data as a VO-enabled service based on Grid technology, and as such, is accessible as a set of services. This kind of infrastructure is demanded not only by geoscience, but also by biology, high-energy physics, and astronomy. In particular, an integration of existing application-driven service specifications is needed by the application community. However, the specifications in the geoscience community, which are discussed in the open geospatial consortium (OGC), are usually implemented in GET and POST operations under HTTP. Thus, it is difficult to enforce the security framework of Grid technology to OGC services, because the Grid security framework require SOAP-based application protocols. Hence, we consider the architecture and an implementation method for enabling them. In this paper, we design a security framework of the GEO grid services based on the requirements from an example application scenario. These services comply with standardized specifications from both grid technologies and the geoscience community thereby achieving service harmonization. In order to achieve this framework, we implement a credential broker, called OGCProxy, which provides an access method that the user connect VO-enabled OGC services without any modifications of existing applications. We also implement a credential delegation for OGC services, which is impossible so far, to achieve complex chaining, such as cascading services, third party data transfer, applying high performance computing behind processing services, etc. As a result of this security framework, an application that uses a complex chaining of VO-enabled OGC services is available without any modifications.

To ensure large-scale and long-time (LSLT) applications to run smoothly in a dynamic and heterogeneous grid environment, we have designed and implemented a WSRF-based framework with which users can reserve resources and request on-demand computing resources. The framework can be architecturely divided into three tiers: the tier providing client-side reservation and allocation APIs, the tier for reservation brokerage and resource allocation, and the tier for backend services. The reservation API is implemented for making and releasing a reservation, as well as for showing available reservations. The allocation API is implemented to request, to check, and to release a resource in a convenient way. The middle tier is designed to hide the complexity of the underlying grid infrastructure, and implemented to provide several allocation algorithms. One of the main backend services is Maui-based reservation service at present. A portal to facilitate the resource management is also available. In this paper, we present the API specification, the architecture, and the implementation of this framework. We also show a detailed experimental example.

We propose a reservation-based sustainable adaptive Grid supercomputing paradigm to enable tightly coupled computations of considerable scale (involving over 1,000 processors) and duration (over tens of continuous days) on a Grid of geographically distributed parallel supercomputers. The paradigm is demonstrated for an adaptive multiscale simulation application, in which accurate but compute-intensive quantum mechanical (QM) simulations are embedded within a classical molecular dynamics (Md) simulation only when and where high fidelity is required. Key technical innovations include: 1) an embedded divide-and-conquer algorithmic framework to maximally expose data and computation localities for enhanced scalability; 2) a buffered-cluster hybridization scheme to adaptively adjust MD/QM boundaries to maintain the model accuracy; and 3) a hybrid Grid remote procedure call (GridRPC) + message passing interface (MPI) Grid application framework to combine flexibility (adaptive resource allocation and migration), fault tolerance (automated fault recovery), and efficiency (scalable management of large computing resources). We have achieved an automated execution of multiscale MD/QM simulation on a Grid consisting of 6 supercomputer centers in Japan and the US (in total of 150 thousand processor-hours) for the dynamic simulation of implanted oxygen atoms in a silicon substrate, in which the number of processors changes dynamically on demand and resources are allocated and migrated dynamically according to both reservations and unexpected faults. The simulation results reveal a strong dependence of the oxygen penetration depth on the incident oxygen-beam position, which is useful information to further advance SIMOX (separation by implanted oxygen) technique to fabricate high speed and low power-consumption semiconductor devices.

This paper presents middleware enabling mutual and equal transfer of computing power between individuals, as in the original idea behind P2P, while also supporting large-scale distributed computation utilizing heterogeneous PCs. This goal is strongly supported by a network overlay over which peers can communicate with each other directly and bidirectionally. We made use of a general-purpose P2P library, JXTA, supporting the common requirements of P2P software, including network overlay. Other features of the P2P library, such as ad-hoc self-organization, discovery and grouping of peers, also support our middleware efficiently. In this paper, we propose and evaluate an application of those P2P concepts to virtual resource transfer and parallel computation with aggregated resources. However, such a P2P library imposes a certain amount of overhead on the middleware in terms of communication performance. Measured communication performance and throughput of an application program shows the feasibility of the application of P2P concepts. The middleware achieves 100 /spl times/ 10/sup 6/ bps communication performance and over a 20 fold increase in speed with 32 computers, even though the granularity of workunits is as fine as less than a second.

Presents a performance evaluation of a wide-area cluster system based on a firewall-enabled Globus metacomputing toolkit. In order to establish communication links beyond the firewall, we have designed and implemented a resource manager called RMF (Resource Manager beyond the Firewall) and the Nexus Proxy, which relays TCP communication links beyond the firewall. In order to extend the Globus metacomputing toolkit to become firewall-enabled, we have built the Nexus Proxy into the Globus toolkit. We have built a firewall-enabled Globus-based wide-area cluster system in Japan and run some benchmarks on it. In this paper, we report various performance results, such as the communication bandwidth and latencies obtained, as well as application performance involving a tree search problem. In a wide-area environment, the communication latency through the Nexus Proxy is approximately six times larger when compared to that of direct communications. As the message size increases, however, the communication overhead caused by the Nexus Proxy can be negligible. We have developed a tree search problem using MPICH-G. We used a self-scheduling algorithm, which is considered to be suitable for a distributed heterogeneous metacomputing environment since it performs dynamic load balancing with low overhead. The performance results indicate that the communication overhead caused by the Nexus Proxy is not a severe problem in metacomputing environments.

We have previously developed a navigation system for an autonomous mobile robot. It can find the landmarks on the ceiling for navigation. However, when the robot fails to recognize a landmark, the robot can not calculate the self-position. So, the robot has to prepare for the behavior of error recovery. But an error that is not able to recover may occur. In such a case, the robot needs to relocate the self-position. In this paper, we define global matching as the behavior that is able to relocate the self-position and propose to use this module for the error recovery in HALAS (hierarchical adaptive and learning architecture system) that we have proposed as a hierarchical system.

We have previously developed a navigation system for autonomous mobile robots. It can find landmarks on the ceiling to move and work. However, it is impossible to recognize landmarks when the size of landmark is different from the size of template. In such cases, the robot cannot continue to work. Therefore we proposed variable template matching (VTM) that makes it possible to recognize landmarks of different sizes. We realize VTM by using the optimization method called the evolution strategy (ES). The robot can recognize the landmark faster, inferring landmark parameters by binary density projection.

We present a vision based navigation system which has an error recovery function for an autonomous mobile robot. A misrecognition of landmark causes the robot to mislocate itself. When this happens, the robot is unable to perform its primary objective. In order to solve this kind of problem, we have proposed a hierarchical control architecture "HALAS" (hierarchical adaptive and learning architecture system). This consists of some modules which are arranged hierarchically and each module means a single function of this robot. When one module fails in its aim, the upper module detects the error and recovers from it using other useful information from its knowledge data base. We show experimentally that the robot gets a higher reliability in autonomous mobility because the map correspondence module finds the error which has been caused by an anemo perception module (landmark recognition module) and it corrects the error utilizing its map information.

A template matching is a popular method for visual recognition. It costs much time to detect the target object. We have applied fuzzy and neural network algorithm to solve this problem. However, the problem to mistake recognizing the target object is not solved. Hence, in this paper, we propose robust visual recognition algorithm for the autonomous robot navigation. The proposing algorithm employs a fuzzy template matching with the target object as well as the similar objects in the environment. As a result, the robot verifies the tentative recognition by comparing the likelihood of plus elements (target objects) with that of minus elements (similar objects). To integrate this algorithm into the robot navigation system, we propose a hierarchical control architecture. The validity of the proposing navigation system is shown with experimental results.

A new type of mobile pipeline maintenance robot has been designed to clarify the feasibility of lazy tongs mechanism for adaptation to various size pipelines. The robot basically consists of three modules with two joints between modules and can move straight and spirally along the surface of pipeline. The robot can change its arm length in order to move on the pipeline of various size from 25 A to 100 A, and pass over obstacles such as reducer, diffuser, flange and other plant equipment. The control of the robot is so complicated that the modified dual control mode is introduced using the coordinate transformation matrix. By calculating the position and attitude of the robot under the control mode, the manoeuvrability of passing over a reducer is shown with experimental results.

We propose a vision based navigation system for autonomous mobile robots that recognizes outlets of air conditioning system (anemo) located on the ceiling as landmarks. Because landmarks on the ceiling are not obstructed by other objects, robots can find them easier than landmarks on the floor. To detect the landmark, we applied the fuzzy template matching (FTM) method. Based on FTM, a fuzzy template (FT) is modeled based on the edge information of landmarks in the image. In addition, we use neural network (NN) to detect landmarks quickly. Inputs of NN are the grades of fuzzy memberships and the learned NN directs FT to the landmark in the image. Because FTM does not use image processing, except edge extraction, it does not make processing error and the landmark can be detected precisely. The navigation system can calculate the distance and the angle of the anemo from the map information. As a result, the navigation system can identify the present robot's position and orientation. By the proposed algorithm, precise and fast landmark detection become possible even if the landmark follows the shadow.