Combining edge processing (at data capture site) with analysis carried out while data is enroute from the capture site to a data center offers a variety of different processing models. Such in-transit nodes include network data centers that have generally been used to support content distribution (providing support for data multicast and caching), but have recently started to offer user-defined programmability, through Software Defined Networks (SDN) capability, e.g., OpenFlow and Network Function Visualization (NFV). We demonstrate how this multi-site computational capability can be aggregated to support video analytics, with Quality of Service and cost constraints (e.g., latency-bound analysis). The use of SDN technology enables separation of the data path from the control path, enabling in-network processing capabilities to be supported as data is migrated across the network. We propose to leverage SDN capability to gain control over the data transport service with the purpose of dynamically establishing data routes such that we can opportunistically exploit the latent computational capabilities located along the network path. Using a number of scenarios, we demonstrate the benefits and limitations of this approach for video analysis, comparing this with the baseline scenario of undertaking all such analysis at a data center located at the core of the infrastructure.

Cloud computing is emerging as a viable platform for scientific exploration. Elastic and on-demand access to resources (and other services), the abstraction of “unlimited” resources, and attractive pricing models provide incentives for scientists to move their workflows into clouds. Generalizing these concepts beyond a single virtualized datacenter, it is possible to create federated marketplaces where different types of resources (e.g., clouds, HPC grids, supercomputers) that may be geographically distributed, are collectively exposed as a single elastic infrastructure. This presents opportunities for optimizing the execution of application workflows with heterogeneous and dynamic requirements, and tackling larger scale problems. In this paper, we introduce a framework to manage the end-to-end execution of data-intensive application workflows in dynamic software-defined resource federation. This framework enables the autonomic execution of workflows by elastically provisioning an appropriate set of resources that meet application requirements, and by adapting this set of resources at runtime as the requirements change. It also allows users to customize scheduling policies that drive the way resources federated and used. To demonstrate the benefits of our approach, we study the execution of two different data-intensive scientific workflows in a multi-cloud federation using different policies and objective functions.

The exponential growth of digital data sources has the potential to transform all aspects of society and our lives. However, to achieve this impact, the data has to be processed promptly to extract insights that can drive decision making. Further, traditional approaches that rely on moving data to remote data centers for processing are no longer feasible. Instead, new approaches that effectively leverage distributed computational infrastructure and services are necessary. Specifically, these approaches must seamlessly combine resources and services at the edge, in the core, and along the data path as needed. This paper presents our vision for enabling an approach for computing in the continuum, i.e., realizing a fluid ecosystem where distributed resources and services are programmatically aggregated on-demand to support emerging data-driven application workflows. This vision calls for novel solutions for federating infrastructure, programming applications and services, and composing dynamic workflows, which are capable of reacting in real-time to unpredictable data sizes, availabilities, locations, and rates.

Data-driven application workflows that leverage compute capabilities and hosted services near the network edge can support latency-sensitive and critical applications in emerging areas such as Internet of Things (IoT) and smart infrastructure. However, distributed instantiation and execution of these workflows using resources across service providers and datacenters can be challenging. In this paper, we present the formulation of a decentralized workflow management approach for the autonomous instantiation and execution of dynamic data-driven workflows based on the opportunistic discovery and composition of services on-demand. Given a workflow template specification, this approach allows us to decouple workflow stages, allowing the execution of different stages to be performed by individual services, which are discovered and instantiated dynamically, and can be independently scaled as needed. These services may be geographically distributed and may be offered by different service providers using various QoS levels and cost models. The design, implementation and experimental evaluation of a decentralized workflow management framework using a live media stream application in a multi-cloud infrastructure is presented. Evaluations using a sample topology shows up to 2.5 times increase in QoS-meeting throughput when using our dynamic multi-cloud approach instead of using a fixed centralized cloud of identical capacity.

Pervasive computational ecosystems that combine data sources and computing/communication resources in self-managed environments, such as the ones powered by Internet of Things (IoT) devices, have the potential to automate and facilitate many aspects of our lives, and impact a variety of applications, from the management of extreme events to the optimization of everyday processes. However, this vision remains mostly unrealized despite the fact that the technology to achieve it exists, largely because of the gap between our ability to collect data and our ability to gain insight from it. In this paper, we discuss the challenges associated with providing a pervasive computational ecosystem. We then present our vision of how to best support data-driven computational ecosystems and propose a conceptual architecture that leverages ideas from software-defined environments in order to combine data, computing, and communication resources. In addition, we show how this proposed architecture enables the execution of data-driven workflows on top of these resources.

Cloud computing has generally involved the use of specialist data centres to support computation and data storage at a central site (or a limited number of sites). The motivation for this has come from the need to provide economies of scale (and subsequent reduction in cost) for supporting large scale computation for multiple user applications over (generally) a shared, multi-tenancy infrastructure. The use of such infrastructures requires moving data to a central location (data may be pre-staged to such a location prior to processing using terrestrial delivery channels and does not always require the use of a network-based transfer), undertaking processing on the data, and subsequently enabling users to download results of analysis. We extend this model using software defined networks (SDNs), whereby capability within the network can be used to support in-transit processing while data is in movement from source to destination. Using a smart building infrastructure scenario, consisting of sensors and actuators embedded within a built environment, we describe how an SDN-based architecture can be used to support real time data processing. This significantly influences the processing times to support energy optimisation of the building and reduces costs. We describe an architecture for such a distributed, multi-layered Cloud system and discuss a prototype that has been implemented using the CometCloud system, deployed across three sites in the UK and the US. Wevalidate the prototype using data from sensors within a Sports facility and making use of EnergyPlus.

Multi-cloud systems have enabled resource and service providers to co-exist in a market where the relationship between clients and services depends on the nature of an application and can be subject to a variety of different Quality of Service (QoS) constraints. Deciding whether a cloud provider should host (or finds it profitable to host) a service in the long-term would be influenced by parameters such as the service price, the QoS guarantees required by customers, the deployment cost (taking into account both cost of resource provisioning and operational expenditure, e.g. energy costs) and the constraints over which these guarantees should be met. In a federated cloud system users can combine specialist capabilities offered by a limited number of providers, at particular cost bands-such as availability of specialist co-processors and software libraries. In addition, federation also enables applications to be scaled on-demand and restricts lock in to the capabilities of a particular provider. We devise a market model to support federated clouds and investigate its efficiency in two real application scenarios:(i) energy optimisation in built environments and (ii) cancer image processing both requiring significant computational resources to execute simulations. We describe and evaluate the establishment of such an application based federation and identify a cost-decision based mechanism to determine when tasks should be outsourced to external sites in the federation. The following contributions are provided: (i) understanding the criteria for accessing sites within a federated cloud dynamically, taking into account factors such as performance, cost, user perceived value, and specific application requirements; (ii) developing and deploying a cost based federated cloud framework for supporting real applications over three federated sites at Cardiff (UK), Rutgers and Indiana (USA), (iii) a performance analysis of the application scenarios to determine how task submission could be supported across these three sites, subject to particular revenue targets.

Cloud computing is emerging as a viable platform for scientific exploration. The ideas of on-demand access to resources, "unlimited" resources as well as interesting pricing models are making scientist to move their workflows into cloud computing. However, the amount of services and different pricing models offered by the providers often overwhelm users when deciding which option is best for them. Moreover, interoperability across providers remains an open topic that forces users to develop specific solutions for each provider. In this paper, we present a service framework that enables the autonomic execution of dynamic workflows in multi-cloud environments. It also allows users to customize scheduling policies to use those resources that best fit their needs. To demonstrate the benefits of this framework, we study the execution of a real scientific workflow, with data dependencies across stages, in a multi-cloud federation using different policies and objective functions.

With increasing availability of instrumented infrastructures in built environments, it is necessary to understand how such data will be stored, processed and analysed in a timely manner. Many "smart cities" applications, for instance, identify how data from building sensors can be combined together to support applications such as emergency response, energy management, etc. Enabling sensor data to be transmitted to a Cloud environment for processing provides a number of benefits, such as scalability and elastic provisioning of computational resources - as the total data size may not be known apriori. In this application-based case study, we describe the integration of an in-building sensor network (both for sensing and actuation) with a distributed Cloud environment. Energy optimisation in buildings represents a class of problems that requires significant computational resources and generally is a time consuming process. We describe the use of Cloud computing for efficiently running and deploying Energy Plus simulations with sensor data in order to fulfil a number of energy related objectives for buildings. We describe and evaluate the establishment of such a sensor based application using a Comet Cloud implementation with data collection from a real building pilot. Although our focus is on a single application, the general architecture and analysis carried out can be generalised to other similar scenarios.

One of the key benefits of Cloud systems is their ability to provide elastic, on-demand (seemingly infinite) computing capability and performance for supporting service delivery. With the resource availability in single data centres proving to be limited, the option of obtaining extra-resources from a collection of Cloud providers has appeared as an efficacious solution. The ability to utilize resources from multiple Cloud providers is also often mentioned as a means to: (i) prevent vendor lock in, (ii) to enable in house capacity to be combined with an external Cloud provider, (iii) combine specialist capability from multiple Cloud vendors (especially when one vendor does not offer such capability or where such capability may come at a higher price). Such federation of Cloud systems can therefore overcome a limit in capacity and enable providers to dynamically increase the availability of resources to serve requests. We describe and evaluate the establishment of such a federation using a CometCloud based implementation, and consider a number of federation policies with associated scenarios and determine the impact of such policies on the overall status of our system. CometCloud provides an overlay that enables multiple types of Cloud systems (both public and private) to be federated through the use of specialist gateways. We describe how two physical sites, in the UK and the US, can be federated in a seamless way using this system.

The variety and complexity of virus harm cause there is few practical evaluation methods currently. The paper first presents harms' definition and classification of the virus. Then representative first-level and second-level evaluation indexes are proposed. However, as these evaluation factors could hardly be assigned with definite values and somewhat have fuzziness, the paper constructs a fuzzy evaluation model for the harm of computer virus. Once an obscure value is assigned to each evaluation indexes in the second level, the model can calculate the membership degree to each of the five harm grades, and then obtains a reasonable harm evaluation result. Finally, the paper evaluates the harms of ldquoWorm.WhBoy.hrdquo, ldquoAutoRunrdquo, ldquoJS.Yamanner.ardquo and ldquoAn-ti-virusrdquo by the fuzzy model; the evaluation result is similar to the statistics of the virus.