Companies depend on mining data to grow their business more than ever. To achieve optimal performance of Big Data analytics workloads, a careful configuration of the cluster and the employed software framework is required. The lack of flexible and accurate performance models, however, render this a challenging task. This article fills this gap by presenting accurate performance prediction models based on Stochastic Activity Networks (SANs). In contrast to existing work, the presented models consider multiple work queues, a critical feature to achieve high accuracy in realistic usage scenarios. We first introduce a monolithic analytical model for a multi-queue YARN cluster running DAG-based Big Data applications that models each queue individually. To overcome the limited scalability of the monolithic model, we then present a fixed-point model that iteratively computes the throughput of a single queue with respect to the rest of the system until a fixed-point is reached. The models are evaluated on a real-world cluster running the widely-used Apache Spark framework and the YARN scheduler. Experiments with the common transaction-based TPC-DS benchmark show that the proposed models achieve an average error of only$5.6\%$in predicting the execution time of the Spark jobs. The presented models enable businesses to optimize their cluster configuration for a given workload and thus to reduce their expenses and minimize service level agreement (SLA) violations. Makespan minimization and per-stage analysis are examined as representative efforts to further assess the applicability of our proposition.

The era of personal resources being sufficient for enterprise big data computations has passed. As computations are executed in the cloud, small policy changes of cloud operators may cause considerable changes in operational costs. Carefully choosing the amount of resources for a given application is thus of great importance. This, however, requires a priori knowledge of the application's performance under different configurations. Creating a performance prediction model needs to account for the heterogeneity of resources and the diversity in application workflows. Previous approaches for heterogeneous environments consider a black-box representation of the application which results in single-purpose models. This paper addresses the problem with two gray-box prediction models using linear programming (LP) and mixed-integer linear programming (MILP). Given a set of available resources, the models consider Apache Spark applications and their Directed Acyclic Graph (DAG) of workflow running on top of a Hadoop-YARN cluster. We then propose a configuration recommendation algorithm to optimize the cost-performance trade-offs when renting machine instances. The accuracy of the proposed models is evaluated with real-world executions of several representative applications on the Wikipedia dataset and the TPC-DS benchmark. The average error of only 3.28% for the proposed prediction models demonstrates the practicality of the proposed approach in handling cost-performance trade-offs.

Abstract In cloud computing services, high availability is one of the quality of service requirements which is necessary to maintain customer confidence. High availability systems can be built by applying redundant nodes and multiple clusters in order to cope with software and hardware failures. Due to cloud computing complexity, dependability analysis of the cloud may require combining state‐based and nonstate‐based modeling techniques. This article proposes a hierarchical model combining reliability block diagrams and continuous time Markov chains to evaluate the availability of OpenStack private clouds, by considering different scenarios. The steady‐state availability, downtime, and cost are used as measures to compare different scenarios studied in the article. The heterogeneous workloads are considered in the proposed models by varying the number of CPUs requested by each customer. Both hardware and software failure rates of OpenStack components used in the model are collected via setting up a real OpenStack environment applying redundancy techniques. Results obtained from the proposed models emphasize the positive impact of redundancy on availability and downtime. Considering the tradeoff between availability and cost, system providers can choose an appropriate scenario for a specific purpose.

The increasing demand for Web services encourages commercial cloud service providers to publish their own services with various functional and nonfunctional capabilities in different cloud platforms. The aggregation of atomic services from multiple service repositories is the main idea of the service composition concept in multiclouds. The cloud Web service composition is a suitable way for satisfying users' complex requests by integrating services from different clouds in order to create a new value-added composite service. The time required to serve a composite service by a multicloud environment is an important parameter, which depends on different factors, ranging from the service composition and selection algorithm to the number of atomic services published in the clouds. In this paper, a model based on timed colored Petri nets (TCPNs) is proposed to evaluate the service composition in multicloud environments while minimizing the number of clouds involved in serving a composite service request. The proposed TCPN graphically models the process of request submission, composite service analysis, service selection, and service provisioning in a multicloud environment. It also assesses both mean response time of the environment and probability of dropping composite requests. The verification of the accuracy of the proposed model is done by comparing the results obtained from the TCPN model, in two different scenarios, with the results from the CloudSim framework. These results confirm that our proposed TCPN model can appropriately model the system and evaluate its performance more efficiently than the CloudSim.

This paper investigates the performance of epidemic routing, in mobile social networks (MSNs), which makes use of the store-carry-forward paradigm for communication. Real-life mobility traces show that people have skewed location visiting preferences, with some places visited frequently and some others infrequently. In order to model epidemic routing in MSNs, we first analyze the time taken for a node to meet the first node belonging to a set of nodes restricted to move in a specific subarea. Afterwards, a monolithic stochastic reward net (SRN) is proposed to evaluate the delivery delay and the average number of transmissions under epidemic routing by considering skewed location visiting preferences. This monolithic model is not scalable enough, in terms of the number of nodes and frequently visited locations. In order to achieve higher scalability, the folding technique is applied to the monolithic SRN and an approximate folded SRN is proposed to evaluate the performance of epidemic routing. Discrete-event simulation is applied to cross-validate the proposed models. Results indicate that the monolithic model has higher accuracy in predicting the performance of epidemic routing. The approximate folded model also achieves a good accuracy and can be solved for a network with a large number of nodes/frequently visited locations. This model is more accurate than the ordinary differential equation approach.

Infrastructure as a Service (IaaS) is one of the most significant and fastest growing fields in cloud computing. To efficiently use the resources of an IaaS cloud, several important factors such as performance, availability, and power consumption need to be considered and evaluated carefully. Evaluation of these metrics is essential for cost-benefit prediction and quantification of different strategies which can be applied to cloud management. In this paper, analytical models based on Stochastic Reward Nets (SRNs) are proposed to model and evaluate an IaaS cloud system at different levels. To achieve this, an SRN is initially presented to model a group of physical machines which are controlled by a management layer. Afterwards, the SRN models presented for the groups of physical machines in the first stage are combined to capture a monolithic model representing an entire IaaS cloud. Since the monolithic model does not scale well for large cloud systems, two approximate SRN models using folding and fixed-point iteration techniques are proposed to evaluate the performance, availability, and power consumption of the IaaS cloud. The existence of a solution for the fixed-point approximate model is proved using Brouwer's fixed-point theorem. A validation of the proposed monolithic and approximate models against both an ad-hoc discrete-event simulator developed in Java and the CloudSim framework is presented. The analytic-numeric results obtained from applying the proposed models to sample cloud systems show that the errors introduced by approximate models are insignificant while an improvement of several orders of magnitude in the state space reduction of the monolithic model is obtained.

In some applicable scenarios, such as community patrolling, mobile nodes are restricted to move only in their own communities. Exploiting the meetings of the nodes within the same community and the nodes within the neighboring communities, a delay tolerant network (DTN) can provide communication between any two nodes. In this paper, two analytical models based on stochastic reward nets (SRNs) are proposed to evaluate the performance of the epidemic content retrieval in such multi-community DTNs. Performance measures computed by the proposed models are the average retrieval delay and the average number of transmissions. The monolithic SRN model proposed in the first step is not scalable, in terms of the number of communities and nodes, due to the state space explosion in the underlying Markov chain. In order to solve the scalability problem of the monolithic model, an approximate model based on the folding technique is presented which allows us to evaluate the performance of large-scale DTNs. In order to cross-validate the results obtained from the proposed models, we extend the ONE simulator to support our network model. The analytic-numeric results indicate that both models have good accuracy, and the folded model reduces the state space highly, achieving good scalability without any significant loss of accuracy.

In this paper, the performance of a grid resource is modeled and evaluated using stochastic reward nets (SRNs), wherein the failure–repair behavior of its processors is taken into account. The proposed SRN is used to compute the blocking probability and service time of a resource for two different types of tasks: grid and local tasks. After modeling a grid resource and evaluating the performability measures, an algorithm is presented to find the probability mass function (pmf) of the service time of the grid resource for a program which is composed of grid tasks. The proposed algorithm exploits the universal generating function to find the pmf of service time of a single grid resource for a given program. Therefore, it can be used to compute the pmf of the service time of entire grid environment for a workflow with several dependent programs. Each possible scheduling of programs on grid resources may result in different service times and successful execution probabilities. Due to this fact, a genetic-based scheduling algorithm is proposed to appropriately dispatch programs of a workflow application to the resources distributed within a grid computing environment. Numerical results obtained by applying the proposed SRN model, the algorithm to find the pmf of grid service time, and the genetic-based scheduling algorithm to a comprehensive case study demonstrate the applicability of the proposed approach to real systems.

In this paper, performance of grid computing environment is studied in the presence of failure-repair of the resources. To achieve this, in the first step, each of the grid resource is individually modeled using Stochastic Reward Nets (SRNs), and mean response time of the resource for grid tasks is computed as a performance measure. In individual models, three different scheduling schemes called random selection, non-preemptive priority, and preemptive priority are considered to simultaneously schedule local and grid tasks to the processors of a single resource. In the next step, single resource models are combined to shape an entire grid environment. Since the number of the resources in a large-scale grid environment is more than can be handled using such a monolithic SRN, two approximate SRN models using folding and fixed-point techniques are proposed to evaluate the performance of the whole grid environment. Brouwer's fixed-point theorem is used to theoretically prove the existence of a solution to the fixed-point approximate model. Numerical results indicate an improvement of several orders of magnitude in the model state space reduction without a significant loss of accuracy.

Availability of the service delivered by cloud providers is one of the most important QoS factors of the service level agreements between providers and customers. Since current Infrastructure-as-a-Service providers use virtualization technology to manage data centers, virtual data centers (VDCs) have become a popular infrastructure for cloud computing. In order to study the service availability, a stochastic activity network (SAN) model is presented in this paper. The proposed SAN model can be appropriately used to investigate the impact of different characteristics and policies on service availability of VDCs.