Various sensor data from Internet of Things (IoT) devices are expected to be routinely collected, analyzed, and utilized in the cloud. However, the required communication throughput and latency for various services must be maintained when collecting IoT data in mobile environments. IoT communication involves a large amount of small-scale streaming data, necessitating efficient transmission methods tailored to the communication environment. In this study, we investigate the effectiveness of compression processing by varying the publish/subscribe data characteristics and data compression ratios to improve performance. In experiments, the performance of MQTT communication over SINETStream is investigated, varying parameters such as data size, compression algorithm, data characteristics, and data compression ratio. The results show that performance is improved for highly compressible data, and the performance difference becomes more pronounced as data size increases. There is no correlation between compression time and data compression ratio, and the impact of compression time on overall execution time is slight, thereby confirming that selecting appropriate algorithms for each data characteristic and applying compression processing according to the data size effectively improves performance.

The data collected by Internet of Things (IoT) devices equipped with sensors enable smart home services such as monitoring the elderly, pets, and the indoor environment. Building an IoT system to collect data from individual households in the cloud requires measures to reduce communication latency and the amount of data transferred, and protect privacy. When collecting diverse data in an indoor environment, installing sensors in multiple indoor locations is necessary. However, installing numerous sensors increases costs and makes it difficult to relocate the sensors to obtain the necessary information. In this study, we construct an IoT system for a smart home that collects indoor environmental information using a wheeled mobile robot implemented in a Robot Operating System (ROS) and performs analysis processing in a cloud via an edge server. We attempt to demonstrate the effectiveness of sensor data collection using a robot by developing a prototype system for indoor carbon dioxide concentration monitoring application. We also the performance characteristics of ROS communication between the sensor robot and the edge server, and IoT communication between the edge server and the cloud server under different communication environments to identify technical issues in the smart home.

Running neural network models on edge devices is attracting much attention by neural network researchers since edge computing technology is becoming more powerful than ever. However, deploying large neural network models on edge devices is challenging due to the limitation in available computing resources and storage space. Therefore, model compression techniques have been recently studied to reduce the model size and fit models on resource-limited edge devices. Compressing neural network models reduces the size of a model, but also degrades the accuracy of the model since it reduces the precision of weights in the model. Consequently, a retraining method is required to recover the accuracy of compressed models. Most existing retraining methods require the original labeled training datasets to retrain the models, but labeling is a time-consuming process. In particular, we cannot always access the original labeled datasets because of privacy policies and license limitations. In this paper, we propose a method to retrain a compressed neural network model with an unlabeled dataset that is different from the original labeled dataset. We compress the neural network model using quantization to decrease the size of the model. Subsequently, the compressed model is retrained by our proposed retraining method without using a labeled dataset to recover the accuracy of the model. We compared the proposed retraining method against the conventional retraining. The proposed method reduced the size of VGG-16 and ResNet-50 by 81.10% and 52.45%, respectively without significant accuracy loss. In addition, our proposed retraining method is clearly faster than the conventional retraining method.

With the development of cameras and sensors, and the spread of cloud computing, life logs can be acquired and stored in general households for various services using the logs. However, it is difficult to analyze moving images acquired by a home sensor in real time using machine learning because the data size and the computational complexity are large. New computing paradigm called edge computing or fog computing, which enables distributed computing over edge and cloud, has the possibility to address this issue. The feature vectors are extracted from moving images by preprocessing on the sensor side and the only small feature vectors are sent to the cloud and used for learning. But, it is not clear how accurately we can recognize actions using only the feature vectors in the learning and inferring. We investigate the accuracies of action recognition with various machine learning methods using feature vector information obtained from moving images. We use the pose estimation library OpenPose for detection of the feature vectors and recognize actions using logistic regression, random forest, support vector machine, and neural network (NN) models, general NN and LSTM, as machine learning methods. The experimental results show that it is possible to recognize an action with 80% accuracy or higher when using random forest and neural network models. We also discuss a method to further improve the accuracy based on the experimental results.

The spread of various sensors and the development of cloud computing technologies enable the accumulation and use of many live logs in ordinary homes. In addition, deep learning technologies have been widely used for image and speech recognition processing. However, a key issue for deep learning is heavy processing loads. To operate a service that utilizes sensor data, those data are transmitted from sensors in ordinary homes to a cloud and analyzed in the cloud. However, services that involve moving image analysis require large amounts of data to be transferred continuously and high computing power for the analysis; hence, it is difficult to process them in real time in the cloud using a conventional stream data processing framework. First, we perform preliminary experiments using Apache Spark [3] (hereinafter called Spark), which is a representative cluster computing platform that is designed to be fast and versatile, and Ray [4] , which is a distributed execution framework. We investigate the characteristics of their distributed recognition processing and demonstrate that Ray enables scalable distributed processing. Next, We implement a prototype system of the proposed distributed stream processing infrastructure using Ray and Apache Kafka [1] (hereinafter called Kafka), which is a distributed messaging system, and demonstrate its performance.

As the use of various sensors and cloud computing technologies has spread, many life-log analysis applications for safety services for the elderly and children have been developed. However, it is difficult to perform real-time large data processing in clouds due to the computational complexity of the analysis because efficient deployment schemes of streaming computing components over cloud resources have not been well-investigated. In this study, we propose a video analysis framework that collects videos from multiple cameras and analyzes them using Apache Kafka and Apache Spark Streaming. We first investigate the data transfer performance of Apache Kafka and examine efficient cluster configuration and parameter settings. We then apply this configuration to the proposed framework and measure the data analysis throughput. The experimental results show that the overall throughput varies depending on the number of broker nodes that store data, the number of topic partitions of data, and the number of nodes that conduct analysis processing. In addition, it is confirmed that the number of cores is needed to consider for the efficient cluster configuration, and that the network bandwidth between the nodes becomes a bottleneck as the amount of data and the number of components increase.

The Spread of cameras and sensors and cloud technologies enable us to obtain life logs at ordinary homes and transmit the captured data to a cloud for life log analysis. However, the amount of processing for video data analysis in a cloud drastically increases when a very large number of homes send data to the cloud. In this research, we aim to improve the efficiency of distributed video data analysis processing by using the parallel deep learning framework Chainer [2] and the distribution processing platform Apache Spark [1] (Spark). In this paper, we construct a Spark cluster and investigate the performance of parallel data processing using Spark varying parameter settings.

Apache Spark is a parallel data processing framework that executes fast for iterative calculations and interactive processing, by caching intermediate data in memory with a lineage-based data recovery from faults. The Spark system can also manage data sets larger than memory capacity by placing some cache or all of them on disks on processing nodes. However, the disadvantage is potential performance degradation due to disk I/O and/or serialization. This study aims to clarify efficient/inefficient use of disks in intermediate data caching in Spark and also to improve the usability of disks for end users. In order to achieve the purpose, influence of disk use in data caching was firstly investigated in various aspects, such as caching options, data abstractions and storage devices. The results indicate that serialization cost is dominant rather than disk I/O in most cases. Secondly, a method of combined use of memory and disk was further evaluated under a high memory pressure. Then the method was improved to avoid an excessive re-caching problem, which achieved at most 20-30% reduction of total execution time under a high memory pressure and did not degrade the performance under a low memory pressure, in our experiment with 4 machine learning benchmarks. Finally, this paper summarizes important factors and potential improvements for efficiently using disks in data caching in Spark.

The spread of various sensors and Cloud technologies has made it easy to acquire life-logs and accumulate data. As a result, many life-log analysis applications, which transfer data from sensors, especially cameras to a Cloud and analyze them in the Cloud, have been developed. Cameras with a server function called network cameras have become cheap and readily available for security services and the monitoring of pets and children from remote locations. In these services, raw data from sensors, including cameras, are generally transferred to a Cloud and processed there. However, it is difficult to transfer raw data from sensors to a Cloud because of the limitation of network bandwidth between sensors and a Cloud and privacy issues caused by sending raw sensor data to a Cloud.

Distributed file systems, which manage large amounts of data over multiple commercially available machines, have attracted attention as a management and processing system for big data applications. A distributed file system consists of multiple data nodes and provides reliability and availability by holding multiple replicas of data. Due to system failure or maintenance, a data node may be removed from the system and the data blocks the removed data node held are lost. If data blocks are missing, the access load of the other data nodes that hold the lost data blocks increases, and as a result the performance of data processing over the distributed file system decreases. Therefore, replica reconstruction is an important issue to reallocate the missing data blocks in order to prevent such performance degradation. The Hadoop Distributed File System (HDFS) is a widely used distributed file system. In the HDFS replica reconstruction process, source and destination data nodes for replication are selected randomly. We found that this replica reconstruction scheme is inefficient because data transfer is biased. Therefore, we propose two more effective replica reconstruction schemes that aim to balance the workloads of replication processes. Our proposed replication scheduling strategy assumes that nodes are arranged in a ring and data blocks are transferred based on this one-directional ring structure to minimize the difference of the amount of transfer data of each node. Based on this strategy, we propose two replica reconstruction schemes, an optimization scheme and a heuristic scheme. We have implemented the proposed schemes in HDFS and evaluated them on an actual HDFS cluster. From the experiments, we confirm that the replica reconstruction throughput of the proposed schemes show a 45% improvement compared to that of the default scheme. We also verify that the heuristic scheme is effective because it shows performance comparable to the optimization scheme and can be more scalable than the optimization scheme.

This paper proposes a new cloud computing service model, Hardware as a Service (HaaS), that is based on the idea of implementing ``elastic data centers'' that provide a data center administrator with resources located at different data centers as demand requires. To demonstrate the feasibility of the proposed model, we have developed what we call an Inter-cloud Resource Integration System (Iris) by using nested virtualization and OpenFlow technologies. Iris dynamically configures and provides a virtual infrastructure over inter-cloud resources, on which an IaaS cloud can run. Using Iris, we have confirmed an IaaS cloud can seamlessly extend and manage resources over multiple data centers. The experimental results on an emulated inter-cloud environment show that the overheads of the HaaS layer are acceptable when the network latency is less than 10 msec. We believe these results provide new insight to help establish inter-cloud computing.

We propose a Map Reduce based stream processing system, called SSS, which is capable of processing stream along with large scale static data. Unlike the existing stream processing systems that can work only on the relatively small on-memory data-set, SSS can process incoming streamed data consulting the stored data. SSS processes streamed data with continuous Mappers and Reducers, which are periodically invoked by the system. It also supports merge operation on two sets of data, which enables stream data processing with large static data. This poster shows overview of SSS stream processing and preliminary evaluation results.

Virtual Machine(VM)-based flexible capacity management is an effective scheme to reduce total power consumption in the data centers. However, there remain the following issues, trade-off between power-saving and user experience, decision on VM packing plans within a feasible calculation time, and collision avoidance for multiple VM live migration processes. In order to resolve these issues, we propose two VM packing algorithms, a matching-based (MBA) and a greedy-type heuristic (GREEDY). MBA enables to decide an optimal plan in polynomial time, while GREEDY is an aggressive packing approach faster than MBA. We investigate the basic performance and the feasibility of proposed algorithms under both artificial and realistic simulation scenarios, respectively. The basic performance experiments show that the algorithms reduce total power consumption by between 18% and 50%, and MBA makes suitable VM packing plans within a feasible calculation time. The feasibility experiments show that the proposed algorithms are feasible to make packing plans for an actual supercomputer, and GREEDY has the advantage in power consumption, but MBA shows the better performance in user experience.

We have been developing a middleware suite called GridARS that enables co-allocation of computing and network resources from multiple administration sites. In such middleware, it is important to provide each user application with a slice which is a set of dynamically allocated resources distributed across sites. However, there are the following issues in constructing such a slice automatically: 1) multi-site administration heterogeneity, 2) dynamic determination of application configuration information, 3) distributed resource monitoring, and 4) asymmetric network reachability. We design and implement an application execution system that provides each application with a slice, that mimics a conventional computing cluster system over the dynamically allocated resources. From the demonstration of the proposed system on an emulated wide area network environment, we confirmed that: first, the proposed system can fully automate resource allocation, slice construction, application invocation, and resource monitoring, in coordination with GridARS. Second, the proposed system can setup a slice quickly, even if the allocated resources are widely distributed and their communication latencies are high. This is because the overhead for gathering and distributing contextualization information is small, and OS-level virtualization and stackable file system technologies accelerate the contextualization process at each node.

An HPC cloud, a flexible and robust cloud computing service specially dedicated to high performance computing, is a promising future e-Science platform. In cloud computing, virtualization is widely used to achieve flexibility and security. Virtualization makes migration or checkpoint/restart of computing elements (virtual machines) easy, and such features are useful for realizing fault tolerance and server consolidations. However, in widely used virtualization schemes, I/O devices are also virtualized, and thus I/O performance is severely degraded. To cope with this problem, VMM-bypass I/O technologies, including PCI passthrough and SR-IOV, in which the I/O overhead can be significantly reduced, have been introduced. However, such VMM-bypass I/O technologies make it impossible to migrate or checkpoint/restart virtual machines, since virtual machines are directly attached to hardware devices. This paper proposes a novel and practical mechanism, called Symbiotic Virtualization (SymVirt), for enabling migration and checkpoint/restart on a virtualized cluster with VMM-bypass I/O devices, without the virtualization overhead during normal operations. SymVirt allows a VMM to cooperate with a message passing layer on the guest OS, then it realizes VM-level migration and checkpoint/restart by using a combination of a PCI hotplug and coordination of distributed VMMs. We have implemented the proposed mechanism on top of QEMU/KVM and the Open MPI system. All PCI devices, including Infiniband and Myrinet, are supported without implementing specific para-virtualized drivers; and it is not necessary to modify either of the MPI runtime and applications. Using the proposed mechanism, we demonstrate reactive and proactive FT mechanisms on a virtualized Infiniband cluster. We have confirmed the effectiveness using both a memory intensive micro benchmark and the NAS parallel benchmark. Moreover, we also show that postcopy live migration enables us to reduce the down time of an application as the memory footprint increases.

Virtualized IT infrastructures combined with virtual machine migration technologies have a potential to support IT services that are resilient to partial physical infrastructure failures caused by extreme events. This paper experimentally evaluates the migration of multiple VMs across long geographical distances - an activity that is required to move virtualized IT systems from a disaster site to a safe location. Taking into account the resource availability parameters observed after the Great East Japan Earthquake, experimental results show that if (1) service downtime in the order of minutes is acceptable, (2) VMs can be kept with small storage footprint, and (3) power and network are available for tens of minutes, it is possible to migrate tens of VMs from damaged sites to a very distant stable location.

Wide-area live migration of virtual machines (VMs) is a key to advanced cloud federation, allowing dynamic and transparent load balancing among data centers. Although Mobile IPv6 (MIPv6) provides strong network infrastructure for mobile nodes, there still exists a missing link to the achievement of MIPv6-based VM migration. Real-world IaaS datacenters require guest-transparent and flexible tunneling mechanisms, which are not provided by existing MIPv6 programs. In this paper, we propose a guest-transparent MIPv6 tunneling mechanism (Kagemusha), that performs Client MIPv6 signaling and tunneling on a host operating system. No MIPv6 program is required to be installed into a guest operating system. The proposed system is fully compatible with existing home agents. It basically works with most virtual machine monitors, including Qemu/KVM and Xen. We have developed the first proof-of-concept prototype of the proposed mechanism. Our experiments showed that our prototype system successfully created MIPv6 tunnels with existing home agents. Its performance overhead was negligible for normal use cases. We also confirmed that the prototype system successfully supported live migration, transparently achieving continuous network reachability for migrated VMs. The downtime of migration increased only by several hundred milliseconds.

Intercloud is a promising technology for data intensive applications. However, an important issue for Intercloud applications is orchestration of various virtualized and performance-assured resources, not only computers, but also network and storage, provided from multiple domains. We have been developing an advance reservation-based resource management framework, called Grid ARS, which can integrate heterogeneous resources and construct a performance-assured virtual infrastructure over Intercloud environment. Grid ARS provides four services that address resource management, resource allocation planning, provisioning and monitoring of the constructed virtual infrastructure. Grid ARS has been developed using common Web services technologies and standards. In this paper, we present overview of Grid ARS and its service components and describe Grid ARS demonstration challenges, demonstration at GLIF2010 and SC10 and OGF NSI interoperation in 2011.

We are developing an efficient resource management system with aggressive virtual machine (VM) relocation among physical nodes in a data center. Existing live migration technology, however, requires a long time to change the execution host of a VM, it is difficult to optimize VM packing on physical nodes dynamically, corresponding to ever-changing resource usage. In this paper, we propose an advanced live migration mechanism enabling instantaneous relocation of VMs. To minimize the time needed for switching the execution host, memory pages are transferred after a VM resumes at a destination host. A special character device driver allows transparent memory page retrievals from a source host for the running VM at the destination. In comparison with related work, the proposed mechanism supports guest operating systems without any modifications to them (i.e, no special device drivers and programs are needed in VMs). It is implemented as a lightweight extension to KVM (Kernel-based Virtual Machine Monitor). It is not required to modify critical parts of the VMM code. Experiments were conducted using the SPECweb2005 benchmark. A running VM with heavily-loaded web servers was successfully relocated to a destination within one second. Temporal performance degradation after relocation was resolved by means of a precaching mechanism for memory pages. In addition, for memory intensive workloads, our migration mechanism moved all the states of a VM faster than existing migration technology.

MapReduce has been very successful in implementing large-scale data-intensive applications. Because of its simple programming model, MapReduce has also begun being utilized as a programming tool for more general distributed and parallel applications, e.g., HPC applications. However, its applicability is limited due to relatively inefficient runtime performance and hence insufficient support for flexible workflow. In particular, the performance problem is not negligible in iterative MapReduce applications. On the other hand, today, HPC community is going to be able to utilize very fast and energy-efficient Solid State Drives (SSDs) with 10 Gbit/sec-class read/write performance. This fact leads us to the possibility to develop "High-Performance MapReduce'', so called. From this perspective, we have been developing a new MapReduce framework called "SSS'' based on distributed key-value store (KVS). In this paper, we first discuss the limitations of existing MapReduce implementations and present the design and implementation of SSS. Although our implementation of SSS is still in a prototype stage, we conduct two benchmarks for comparing the performance of SSS and Hadoop. The results indicate that SSS performs 1-10 times faster than Hadoop.

IaaS (Infrastructure-as-a-Service) is an emerging concept of cloud computing, which allows users to obtain hardware resources from virtualized data centers. Although many commercial IaaS clouds have recently been launched, dynamic virtual machine (VM) migration is not possible among service providers; users are locked into a particular provider, and cannot transparently relocate their VMs to another one for the best cost-effectiveness. In this paper, we propose an advanced storage access mechanism that strongly supports live VM migration over WAN. It rapidly relocates VM disks between source and destination sites with the minimum impact on I/O performance. The proposed mechanism addresses I/O consistency of virtual disks before/after migration, which is the major issue regarding wide-area live migration. The proposed mechanism works as a storage server of a block-level storage I/O protocol (e.g.,iSCSI and NBD). Two key techniques (on-demand fetching and background copying) move on-line virtual disks among remote sites, transparently and efficiently. Our prototype system works perfectly for Xen and KVM without any modification to them. Experiments showed the prototype system also worked successfully for an emulated WAN environment.

To ensure large-scale and long-time (LSLT) applications to run smoothly in a dynamic and heterogeneous grid environment, we have designed and implemented a WSRF-based framework with which users can reserve resources and request on-demand computing resources. The framework can be architecturely divided into three tiers: the tier providing client-side reservation and allocation APIs, the tier for reservation brokerage and resource allocation, and the tier for backend services. The reservation API is implemented for making and releasing a reservation, as well as for showing available reservations. The allocation API is implemented to request, to check, and to release a resource in a convenient way. The middle tier is designed to hide the complexity of the underlying grid infrastructure, and implemented to provide several allocation algorithms. One of the main backend services is Maui-based reservation service at present. A portal to facilitate the resource management is also available. In this paper, we present the API specification, the architecture, and the implementation of this framework. We also show a detailed experimental example.

Despite recent developments in higher-level middleware for the Grid supporting high level of ease-of-programming, hurdles for widespread adoption of Grids remain high, due to (1) assumption of peer-to-peer connectivity of all Grid nodes, as well as (2) lack of scalable programming and deployment support. We propose a Java-based programming environment for a hierarchically organized Grid named Jojo, that allow seamless utilization of privately addressed clusters. Jojo provides several features, including secure private remote invocation using Globus GRAM and ssh/rsh to privately addressed nodes in clusters, intuitive message passing API suitable for overlapped execution using multiple threads, and automatic user/system program staging. Using Jojo, users can easily construct and execute parallel distributed applications on the Grid. We show the design and implementation of its programming API, a working example, as well as preliminary performance evaluation results that prove the effectiveness of hierarchal execution.

High-performance peer-to-peer transfer between clusters will be fundamental technology base for various Grid middleware, such as large-scale data transfer in DataGrid settings, or collective communication in Grid-wide MPIs. There, two major factors are involved: on one hand network pipes with large RTT /spl times/ bandwidth typically become data-starved, resulting in bandwidth loss; on the other hand when multiple nodes on the clusters attempt simultaneous transfer, the network pipe could become saturated, resulting in packet loss which again may result in bandwidth degradation in large RTT /spl times/ bandwidth networks. By dynamically and automatically adjusting transfer parameters between the two clusters, such as the number of network nodes, number of socket stripes, we could achieve optimal bandwidth even when the network is under heavy contention. In order to arrive at a proper performance model for automated adjustment, we have conducted several simulations by which we have discovered that such automatic tuning would beneficial, but the ideal number of network pipes does not exactly match the simple transfer model of traditional peer-to-peer settings between single nodes.

GridRPC is a class of Grid middleware for scientific computing. Interoperability has been an important issue, because current GridRPC systems each employ its own protocol. Web services, where XML-based standards such as SOAP and WSDL are expected to see widespread use, could be the medium of interoperability; however it is not clear if 1) XML-based schemas have sufficient expressive power for GridRPC, and 2) whether performance could be made sufficient. Our experiments indicate that the use of such technologies are more promising. than previously reported. Although a naive implementation of SOAP-based GridRPC has severe performance overhead, application of a series of optimizations improves performance. However encoding of various features of GridRPC proved to be somewhat difficult due to WSDL limitations. The results show that GridRPC systems can be based on Web technologies, but there needs to be work to extend WSDL specifications, possibly impacting OGSA-based Grid services directions.

JiPANG(A Jini-based Portal Augmenting Grids) is a portal system and a toolkit which provides uniform access interface layer to a variety of Grid systems, and is built on top of Jini distributed object technology. JiPANG performs uniform higher-level management of the computing services and resources being managed by individual Grid systems such as Ninf, NetSolve, Globus, etc. In order to give the user a uniform interface to the Grids JiPANG provides a set of simple Java APIs called the JiPANG Toolkits, and furthermore, allows the user to interact with Grid systems, again in a uniform way, using the JiPANG Browser application. With JiPANG, users need not install any client packages before-hand to interact with Grid systems, nor be concerned about updating to the latest version. Such uniform, transparent services available in a ubiquitous manner we believe is essential for the success of Grid as a viable computing platform for the next generation.

Presents a performance evaluation of a wide-area cluster system based on a firewall-enabled Globus metacomputing toolkit. In order to establish communication links beyond the firewall, we have designed and implemented a resource manager called RMF (Resource Manager beyond the Firewall) and the Nexus Proxy, which relays TCP communication links beyond the firewall. In order to extend the Globus metacomputing toolkit to become firewall-enabled, we have built the Nexus Proxy into the Globus toolkit. We have built a firewall-enabled Globus-based wide-area cluster system in Japan and run some benchmarks on it. In this paper, we report various performance results, such as the communication bandwidth and latencies obtained, as well as application performance involving a tree search problem. In a wide-area environment, the communication latency through the Nexus Proxy is approximately six times larger when compared to that of direct communications. As the message size increases, however, the communication overhead caused by the Nexus Proxy can be negligible. We have developed a tree search problem using MPICH-G. We used a self-scheduling algorithm, which is considered to be suitable for a distributed heterogeneous metacomputing environment since it performs dynamic load balancing with low overhead. The performance results indicate that the communication overhead caused by the Nexus Proxy is not a severe problem in metacomputing environments.

Recent developments of global computing systems such as Ninf, NetSolve and Globus have opened up the opportunities for providing high-performance computing services over wide-area networks. However, most research focused on the individual architectural aspects of the system, or application deployment examples, instead of the necessary characteristics such systems should intrinsically satisfy, nor how such systems relate with each other. Our comparative study performs deployment of example applications of network-based libraries using Ninf, NetSolve, and CORBA systems. There, we discover that dedicated systems for global computing such as Ninf and NetSolve have management, programmability, and it does not suffer performance disadvantages over more generic distributed computing capabilities provided by CORBA. Such results indicate the advantage of dedicated global computing systems over general systems, stemming further basic research is necessary across multiple systems to identify the ideal software architectures for global computing.

While there have been several proposals of high-performance global computing systems, scheduling schemes for the systems have not been well investigated. The reason is difficulties of evaluation by large-scale benchmarks with reproducible results. Our Bricks performance evaluation system allows the analysis and comparison of various scheduling schemes in a typical high-performance global computing setting. Bricks can simulate various behaviors of global computing systems, especially the behavior of networks and resource scheduling algorithms. Moreover, Bricks is partitioned into components such that not only can its constituents be replaced to simulate various different system algorithms, but it also allows the incorporation of existing global computing components via its foreign interface. To test the validity of the latter characteristics, we incorporated the NWS (Network Weather Service) system, which monitors and forecasts global computing systems behavior. Experiments were conducted by running NWS under a real environment versus a Bricks-simulated environment, given the observed parameters of the real environment. We observed that Bricks behaved in the same manner as the real environment, and NWS also behaved similarly, making quite comparative forecasts under both environments.

The paper proposes a performance evaluation model for effective job scheduling in global computing systems. The proposed model represents a global computing system by a queueing network, in which servers and networks are represented by queueing systems. Evaluation of the proposed model showed that the model could simulate behavior of an actual global computing system and job scheduling on the system effectively.

Ninflet is a Java-based global computing system that builds on our experiences with the Ninf system which facilitated RPC-based computing of numerical tasks in a wide-area network. The goal of Ninflet is to become a new generation of concurrent object-oriented systems which harness abundant idle computing powers, and also seamlessly integrate global as well as local network parallel computing. Ninflet is designed to make use of Java features to implement important features in global computing, such as resource allocation, inter-Ninflet communication, security, checkpointing, object migration, and easy server management via HTTP. © 1998 John Wiley & Sons, Ltd.

Rapid increase in speed and availability of network of supercomputers is making high-performance global computing possible, including our Ninf system. However, critical issues regarding system performance characteristics in global computing have been little investigated, especially under multi-client, multi-site WAN settings. In order to investigate the feasibility of Ninf and similar systems, we conducted benchmarks under various LAN and WAN environments, and observed the following results: 1) Given sufficient communication bandwidth, Ninf performance quickly overtakes client local performance, 2) current supercomputers are sufficient platforms for supporting Ninf and similar systems in terms of performance and OS fault resiliency, 3) for a vector-parallel machine (Cray J90), employing optimized data-parallel library is a better choice compared to conventional task-parallel execution employed for non-numerical data servers, 4) computationally intensive tasks such as EP can readily be supported under the current Ninf infrastructure, and 5) for communication-intensive applications such as Linpack, server CPU utilization dominates LAN performance, while communication bandwidth dominates WAN performance, and furthermore, aggregate bandwidth could be sustained for multiple clients located at different Internet sites; as a result, distribution of multiple tasks to computing servers on different networks would be essential for achieving higher client-observed performance. Our results are not necessarily restricted to the Ninf system, but rather, would be applicable to other similar global computing systems.