Large Language Models (LLMs) have emerged as a disruptive technology for Natural Language Processing (NLP), achieving success in NLP-related generative applications. However, the potential capability of LLMs in other domains remains largely unexplored. To explore the potential of task scheduling with LLMs, we model a typical task scheduling scenario in cloud computing and transfer scheduling problems as natural language prompts. Afterward, the knowledge and reasoning abilities of LLMs are enabled to generate scheduling decisions. Six well-known and open-source LLMs are integrated into our framework to perform experimental studies, and the results are evaluated from multiple perspectives and compared with each other. Besides, traditional heuristic algorithms and a basic Reinforcement Learning (RL) method are all performed for comparison. Our results demonstrate: 1) compared to most heuristic methods, the decisions made by LLMs achieve better scheduling performance; 2) compared to the basic RL method, LLMs exhibit better generalization on various workload patterns; 3) the larger parameter size of the LLMs has, the better scheduling performance it achieves. To the best of our knowledge, our experimental study is the first exploration to apply LLMs in task scheduling. Our findings highlight the promising potential of LLMs as a novel approach to task scheduling, offering new avenues for research and practice.

The extensive deployment of drone swarms in emergency response and logistics applications presents stringent requirements for mobile ad-hoc network protocols, particularly due to their large-scale and highly dynamic characteristics. As a widely adopted protocol in industrial applications, BATMAN-ADV (Better Approach To Mobile Ad-Hoc Networking Advanced) employs a fixed-interval transmission mechanism for control packets, which leads to inefficient utilization of bandwidth resources. Therefore, we propose ROBIN, a distributed Multi-Agent Reinforcement Learning(MARL) optimization method. Our approach enables each drone to act as an independent agent, autonomously adjusting the control packet transmission interval and optimizing transmission strategies based on localized network states. The solution maintains the protocol’s inherent millisecond end-to-end latency while establishing a dynamic balance between bandwidth consumption and latency performance. Moreover, our method features extremely low resource consumption, making it suitable for drone with limited resources. Experimental results show that ROBIN reduces control packet overhead by 75.9% across diverse network scales and mobility scenarios, while maintaining only 2.29% CPU utilization and 0.25% memory consumption per node on commercial drones.

Leveraging satellites and aerials vehicle, nonterrestrial network (NTN)-enabled IoT networks enhance coverage and reliability, enabling global data connections in remote and underserved regions. A key application within these networks is the location-related publish/subscribe service (LPSS), which is geospatial location sensitive, real time, and energy efficient, supporting disaster early warning and environmental monitoring. We demonstrate that, compared to IP technology, named data networking (NDN) is more suited to supporting LPSS. However, current NTN-enabled IoT networks lack mechanisms to utilize geospatial characteristics effectively. Additionally, interactions between IoT devices and aerial vehicles or satellites face challenges, such as low bandwidth, high latency, and intermittent connectivity, which hinder the efficiency of LPSS. We propose geospatially enhanced NDN (GENDN), an adapted NDN framework for supporting LPSS. GENDN incorporates Geohash encoding in content names, allowing flexible use of geospatial characteristics in data subscription. GENDN enhances request aggregation, enabling a single Interest packet (I-pkt) to subscribe to all data in adjacent areas without sequential matching and retrieval. Simulation experiments demonstrate that, compared to traditional NDN, GENDN: 1) effectively leverages geospatial data characteristics, increasing the hit rate of I-pkts in LPSS; 2) reduces the PIT size and network communication overhead, enhancing real-time performance and energy efficiency; and 3) shows potential for large-scale deployment in NTN-enabled IoT environments.

High-performance key-value (KV) storage is critical for the cloud, providing KV services to various cloud applications. A key challenge for KV services is that workloads of cloud applications are often write-intensive and exhibit highly-skewed characteristics, which result in load imbalance among storage servers thus lowering system performance. To address this problem, prior works set up an in-switch write-back caching mechanism, which adopts a centralized controller to balance writes. However, the limited bandwidth between the switches and the controller is a severe bottleneck for achieving high performance. In this paper, we present Magneto, a novel key-value service architecture for the cloud. At the core of Magneto is a delayed-write mechanism in the switch data plane to absorb frequent write queries for hot items. It effectively balances the load under write-intensive workloads without involving the controller. Magneto also designs a reliability mechanism to ensure system reliability during switch state transitions and failures. We implement a prototype using an FPGA-integrated switch, which has high packet-processing performance and contains enough memory to provide both the in-switch cache and the write buffer. Extensive evaluation shows that Magneto can achieve 8.4x system throughput gains compared to baseline systems when handling a skewed workload consisting of 70% reads and 30% writes. Moreover, it can reduce the load on back-end servers up to 56% in total.

For the applications of artificial intelligence, training models with GPUs are widely noted, while inferring requirements are somehow neglected. In some scenario, it is quite important to finish the deep learning inference (DLI) task and get the response in time, e.g., anomaly detection in AIOps or QoE (Quality of Experience) assurance for customers. However, the challenges of GPU inferring are as follows: 1) the interference among inferring tasks on the shared GPU are not well studied and even not considered, which may cause a surge in the inference latency due to the hardware contention; 2) the deadline miss rate caused by the arrival rate is not clearly considered, which often exhibits significant fluctuations in real-world cases. Therefore, the interference among tasks and arrival rates of tasks should be well designed to decrease the deadline miss rate, when sharing GPU resources. To tackle the issue, we propose the algorithm, DeInfer, through following manners: 1) we identify the key factors that lead to interference, conduct a systematic study and develop a highly accurate interference prediction algorithm based on the random forest algorithm, achieving a four times improvement compared to the state-of-the-art interference prediction algorithms; 2) we utilize the queue theory to model the randomness of the arrival process and put forward a GPU resource allocation algorithm, which reduces the deadline miss rate by an average of over 30%.

Named Data Networking (NDN) is an emerging paradigm for future networks that facilitates content retrieval by managing data directly through their names. However, in Non-Terrestrial Networks (NTN), including satellites and UAVs, data often have strong geospatial characteristics due to dynamic topologies and geographic dependencies. The current NDN protocol struggles to leverage these geospatial characteristics due to the mismatch between two-dimensional geographical coordinates and one-dimensional data names, resulting in inefficient data retrieval. We propose GeoNDN, a Geohash-based naming scheme that effectively utilizes geospatial characteristics during data retrieval. GeoNDN enhances the aggregation of Interest packets when consumers retrieve data from nearby areas, allowing all data from a target area to be obtained with a single Interest packet instead of sequentially matching each piece of data. Simulation experiments tailored to NTN wireless scenarios demonstrate that GeoNDN reduces the Pending Interest Table (PIT) scale by approximately 25%, decreases network communication overhead by 21%, and increases the Interest packet hit rate by about 12% compared to the traditional NDN naming. The experimental results also show that GeoNDN’s optimization effects are proportional to request density, highlighting its potential for large-scale deployment.

Key-value stores are critical for modern data centers, yet they struggle with performance degradation under write-intensive workloads due to frequent cache invalidations. Traditional read-optimized caching mechanisms fail to maintain efficiency as frequent updates lead to obsolete cached data. As a result, queries sent by clients will suffer from long queuing delays or even be dropped and overall system performance will be degraded severely. This paper presents a novel key-value stores architecture, named Freq-absorb, which addresses this issue through a delayed-write mechanism delaying writes commitment to backend servers. By buffering write queries within the switch for a specific period, Freq-absorb reduces the impact of cache invalidations, improves switch hit rates, and enhances overall system performance. We implement a prototype using an FPGA-based switch that acts as the ToR (Top of Rack) switch to achieve cache and the appended write buffer. Our experimental results show that compared to the read cache mechanism, our approach can reduce switch miss from 99% to 48.4% when handling write-intensive workloads, significantly enhancing performance.

No abstract available.

Industrial applications often require federated cloud services from multiple providers to improve reliability and flexibility. Traditional selection methods through auctions usually involve a centralized auctioneer to coordinate the auction procedure. Blockchain and smart contracts provide a decentralized mechanism to automate the cloud auction process; however, existing solutions fail in the selection of the most suitable providers and the violation detection of the signed auction agreements, which are also known as service-level agreements (SLAs). To tackle these problems, we propose an integrated auction model using Bayesian game theory and blockchain techniques. The proposed model is enhanced with two Bayesian Nash Equilibriums (BNEs); the first BNE enables the selection of cost-effective providers to construct the federated cloud services, while the second BNE ensures consistent and trustworthy monitoring of federated SLAs. Moreover, a timed message submission (TMS) algorithm is proposed to protect the auction privacy during the message submission phase. This paper validates the equilibrium results of two BNEs and implements the proposed model on the Ethereum blockchain. The analytical and experimental results demonstrate the feasibility, trustworthiness, and cost-effectiveness of our model.

Summary The current cloud market is dominated by a few providers, which offer cloud services in a take‐it‐or‐leave‐it manner. However, the dynamism and uncertainty of cloud environments may require the change over time of both application requirements and service capabilities. The current service‐level agreement (SLA) management solutions cannot easily guarantee a trustworthy, distributed SLA adaptation due to the centralized authority of the cloud provider who could also misbehave to pursue individual goals. To address the above issues, we propose a novel SLA management framework, which facilitates the specification and enforcement of dynamic SLAs that enable one to describe how, and under which conditions, the offered service level can change over time. The proposed framework relies on a two‐level blockchain architecture. At the first level, the smart SLA is transformed into a smart contract that dynamically guides service provisioning. At the second level, a permissioned blockchain is built through a federation of monitoring entities to generate objective measurements for the smart SLA/contract assessment. The scalability of this permissioned blockchain is also thoroughly evaluated. The proposed framework enables creating open distributed clouds, which offer manageable and dynamic services, and facilitates cost reduction for cloud consumers, while it increases flexibility in resource management and trust in the offered cloud services.

There lacks trust between the cloud customer and provider to enforce traditional cloud SLA (Service Level Agreement) where the blockchain technique seems a promising solution. However, current explorations still face challenges to prove that the off-chain SLO (Service Level Objective) violations really happen before recorded into the on-chain transactions. In this paper, a witness model is proposed implemented with smart contracts to solve this trust issue. The introduced role, "Witness", gains rewards as an incentive for performing the SLO violation report, and the payoff function is carefully designed in a way that the witness has to tell the truth, for maximizing the rewards. This fact that the witness has to be honest is analyzed and proved using the Nash Equilibrium principle of game theory. For ensuring the chosen witnesses are random and independent, an unbiased selection algorithm is proposed to avoid possible collusions. An auditing mechanism is also introduced to detect potential malicious witnesses. Specifically, we define three types of malicious behaviors and propose quantitative indicators to audit and detect these behaviors. Moreover, experimental studies based on Ethereum blockchain demonstrate the proposed model is feasible, and indicate that the performance, ie, transaction fee, of each interface follows the design expectations.

With the development of edge computing, edge storage solutions are attracting widespread attention. When facing the requirements of lower latency and faster access speed from end devices, edge storage solutions are considered to be an alternative to the cloud. However, edges are usually owned by small organizations which have limited operations and maintenance capabilities. This makes these edge devices can be easily disabled by external attacks or internal hardware failures. Besides, the heterogeneity of the edge devices will also make it difficult to price the edge resources uniformly. To tackle these problems, we propose SmartStore: an auction mechanism based on blockchain to allocate edge resources. Considering centralized solutions have access bottlenecks and trust issues, we built SmartStore on the smart contract. With Bayesian game theory, SmartStore can analyze how data owners (DO) and edges price the resources can maximize their benefits. From an economic perspective, both DO and edges can make full use of edge heterogeneous resources with SmartStore. Besides, a two-stage submission strategy is proposed to complete the sealed auction. Furthermore, considering the reliability of edge storage, we propose a cluster-based block distribution algorithm for SmartStore's intelligent edge recommendation process. SmartStore ensures the reliability of edge storage while maximizing the benefits and resource utilization of both parties. Finally, we conduct specific experiments on the proposed auction smart contract through “Ethereum” and the experimental results of implementation show the effectiveness and efficiency of our SmartStore.

Edge computing emerges as an alternative to cloud computing in the scenarios where the end devices require lower latency and faster access speeds. Edge nodes are deployed at the proximity of the end devices to reduce response time. On the other hand, the edge nodes are usually owned by small organizations that have limited operations and maintenance capabilities. Data on the edge may be easily damaged, due to external attacks or internal hardware failures. Therefore, it is essential to verify data integrity in edge computing. However, edge environment requires a different trust model compared with other computing and storage paradigm. Besides, compared with cloud storage, edge storage is decentralized and storage service participants may pose greater internal and external threats. This paper proposes a blockchain-based intelligent crowdsourcing audit approach (Crowdauditing) to achieve on-chain and off-chain credibility of audit results. The model relies on an untrusted auditor committee from the crowd to audit data integrity and uses smart contracts as the core of the intelligent system to ensure the reliability of result submission, the accuracy of the result judgment, and reasonable punishments and rewards. Specifically, an unbiased selection algorithm is proposed to achieve fairness during the auditor committee construction. An innovative two-stage submission strategy is proposed to ensure that the auditor committee can reach a consensus on the off-chain audit results. An incentive mechanism is carefully designed to force auditors providing audit services honestly to maximize their own rewards. Moreover, we modeled that as a game of n players, which proves the reliability of the result. Finally, we implement a prototype of Crowdauditing based on smart contracts. The extensive experimental results demonstrate the effectiveness of Crowdauditing.

The rapid development of wearable sensors and the 5G network empowers traditional medical treatment with the ability to collect patients’ information remotely for monitoring and diagnosing purposes. Meanwhile, the health‐related mobile apps and devices also generate a large amount of medical data, which is critical for promoting disease research and diagnosis. However, medical data is too sensitive to share, which is also a common issue for IoT (Internet of Things) data. The traditional centralized cloud‐based medical data sharing schemes have to rely on a single trusted third party. Therefore, the schemes suffer from single‐point failure and lack of privacy protection and access control for the data. Blockchain is an emerging technique to provide an approach for managing data in a decentralized manner. Especially, the blockchain‐based smart contract technique enables the programmability for participants to access the data. All the interactions are authenticated and recorded by the other participants of the blockchain network, which is tamper resistant. In this paper, we leverage the K‐anonymity and searchable encryption techniques and propose a blockchain‐based privacy‐preserving scheme for medical data sharing among medical institutions and data users. To be specific, the consortium blockchain, Hyperledger Fabric, is adopted to allow data users to search for encrypted medical data records. The smart contract, i.e., the chaincode, implements the attribute‐based access control mechanisms to guarantee that the data can only be accessed by the user with proper attributes. The K‐anonymity and searchable encryption ensure that the medical data is shared without privacy leaking, i.e., figuring out an individual patient from queries. We implement a prototype system using the chaincode of Hyperledger Fabric. From the functional perspective, security analysis shows that the proposed scheme satisfies security goals and precedes others. From the performance perspective, we conduct experiments by simulating different numbers of medical institutions. The experimental results demonstrate that the scalability and performance of our scheme are practical.

Last decades, Cloud computing has made significant impacts on traditional applications to change their development and operation methods. We witnessed ever more newly-built Clouds and data centers. However, the centralized management mechanism of current Clouds lacks the dispersion to satisfy the requirements of emerging collaborative applications, including AI, IoT, and autopilot. On the other hand, the Edge computing stays at the conceptual and experimental stage. Most organizations construct their own Edge nodes to operate applications. An efficient and incentive mechanism is missing to motivate the Edge and micro Cloud resource providers to join and constitute a more generalized and decentralized ecosystem. To address this issue, we propose ALLSTAR, a blockchain based architecture for equally combining all the Cloud and Edge resources to be seamlessly leveraged by the application in the DevOps (development and operations) lifecycle. The ALLSTAR architecture is a systematic solution to realize the "Cloud+Edge" management and contributes to constructing the corresponding ALLSTAR ecosystem. This paper describes the overall architecture of ALLSTAR, the related key techniques, and detailed application DevOps processes as well as the new business model.

Summary The increasing volume of data being produced, curated, and made available by research infrastructures in the environmental science domain require services that are able to optimize the delivery and staging of data for researchers and other users of scientific data. Specialized data services for managing data life cycle, for creating and delivering data products, and for customized data processing and analysis all play a crucial role in how these research infrastructures serve their communities, and many of these activities are time‐critical—needing to be carried out frequently within specific time windows. We describe our experiences identifying the time‐critical requirements of environmental scientists making use of computational research support environments. We present a microservice‐based infrastructure optimization suite, the Dynamic Real‐Time Infrastructure Planner, used for constructing virtual infrastructures for research applications on demand. We provide a case study whereby our suite is used to optimize runtime service quality for a data subscription service provided by the Euro‐Argo using EGI Federated Cloud and EUDAT's B2SAFE services, and to consider how such a case study relates to other application scenarios.

By effectively virtualizing operating systems and encapsulating necessary runtime contexts of software components and services, container technologies can significantly improve portability and efficiency for distributed application deployment. It flexibly extends virtual machine based cloud (Infrastructure-as-a-Service) as a much lighter virtual environment (container cluster) for agile application management. However, existing container management systems are not capable of handling concurrent requests efficiently, particularly for the underlying clusters with heterogeneous machines and the requested containers with multi-resource demands. In this paper, we propose an Enhanced Container Scheduler (ECSched) for efficiently scheduling concurrent container requests on heterogeneous clusters with multi-resource constraints. We formulate the container scheduling problem as a minimum cost flow problem (MCFP), and represent the container requirements using a specific graph data structure (flow network). ECSched affords flexibility in constructing the flow network based on a batch of concurrent requests, and performs the MCFP algorithm to schedule the concurrent requests in an online manner. We evaluate ECSched in different testbed clusters, and measure the scheduling overhead with large-scale simulations. The experimental results show that ECSched outperforms state-of-the-art container schedulers in container performance and resource efficiency, and only introduces a small and acceptable scheduling overhead in large-scale clusters.

The permissioned blockchains have demonstrated their potential to provide trustworthy and security services in various industrial scenarios, especially in the Cloud-based virtualized environments. To customize the configuration of a blockchain application, an operator needs the performance characteristics of a blockchain network in different Cloud environments. However, manually profiling the performance characteristics of a blockchain network is very time-consuming. Therefore, in this paper, we propose a BlockchaIn-infRAstructure CustomIzation and Auto-profiLing (BIRACIAL) framework to automate the whole process of blockchain deployment and performance profiling. Based on the profile and performance requirements of a blockchain application, the framework aims to plan the virtual infrastructure for permissioned blockchain, to automate the provision of the required infrastructure, to deploy the customized permissioned blockchain, and to enable continuous monitoring of blockchain performance. Our evaluation results show that the proposed framework can achieve automated deployment of different permissioned blockchain networks under certain overheads. The performance profiling results can be used to compare and select the appropriate blockchain platforms and consensus algorithms.

Traditional cloud Service Level Agreement (SLA) suffers from lacking a trustworthy platform for automatic enforcement. The emerging blockchain technique brings in an immutable solution for tracking transactions among business partners. However, it is still very challenging to prove the credibility of possible violations in the SLA before recording them onto the blockchain. To tackle this challenge, we propose a witness model using game theory and the smart contract techniques. The proposed model extends the existing service model with a new role called “witness” for detecting and reporting service violations. Witnesses gain revenue as an incentive for performing these duties, and the payoff function is carefully designed in a way that trustworthiness is guaranteed: in order to get the maximum profit, the witness has to always tell the truth. This is analyzed and proved through game theory using the Nash equilibrium principle. In addition, an unbiased sortition algorithm is proposed to ensure the randomness of the independent witnesses selection from the decentralized witness pool, to avoid possible unfairness or collusion. An auditing mechanism is also introduced in the paper to detect potential irrational or malicious witnesses. We have prototyped the system leveraging the smart contracts of Ethereum blockchain. Experimental results demonstrate the feasibility of the proposed model and indicate good performance in accordance with the design expectations.

With ever more IoT (Internet of Things) and bigdata applications, the emerging blockchain techniques provide fundamental supports to credibly track the transactions of digital assets. Public blockchains, e.g., bitcoin, are often energy-consuming and low efficient. Therefore, an empirical study of operating permissioned blockchains in clouds is urgently needed. In this paper, we study the performance of Sawtooth, a well-known permissioned blockchain platforms from Hyperledger, in cloud environments. Our results provide insights for blockchain operators to optimize the performance of Sawtooth through adjusting the two configuration parameters, i.e., Scheduler and Maximum Batches Per Block. Our approach can be used to test other blockchain platforms.

Summary The infrastructure‐as‐a‐service (IaaS) model of cloud computing provides virtual infrastructure functions (VIFs), which allow application developers to flexibly provision suitable virtual machines' (VM) types and locations, and even configure the network connection for each VM. Because of the pay‐as‐you‐go business model, IaaS provides an elastic way to operate applications on demand. However, in current cloud applications DevOps (software development and operations) lifecycle, the VM provisioning steps mainly rely on manually leveraging these VIFs. Moreover, these functions cannot be programmatically embedded into the application logic to control the infrastructure at runtime. Especially, the vendor lock‐in issue, which different clouds provide different VIFs, also enlarges this gap between the cloud infrastructure management and application operation. To mitigate this gap, we designed and implemented a framework, CloudsStorm, which enables developers to easily leverage VIFs of different clouds and program them into their cloud applications. To be specific, CloudsStorm empowers applications with infrastructure programmability at design‐level, infrastructure‐level, and application‐level. CloudsStorm also provides two infrastructure controlling modes, ie, active and passive mode, for applications at runtime. Besides, case studies about operating task‐based and big data applications on clouds show that the monetary cost is significantly reduced through the seamless and on‐demand infrastructure management provided by CloudsStorm. Finally, the scaling and recovery operation evaluations of CloudsStorm are performed to show its controlling performance. Compared with other tools, ie, “jcloud” and “cloudinit.d”, the scaling and provisioning performance evaluations demonstrate that CloudsStorm can achieve at least 10% efficiency improvement in our experiment settings.

The IaaS (Infrastructure-as-a-Service) offered by Clouds provides applications with the capability of customizing VMs and configuring their network. Compared to traditional service-based IaaS applications such as persistent web services, most task-based applications have a relatively short duration but are triggered on demand. A typical way to support such kinds of application is to provision a shared and fixed virtual infrastructure based on pre-estimated size in advance, and then perform all the processing tasks. However, due to unpredictable workloads, this solution can lead to either cost inefficiency caused by over-provisioning, or failure to deliver the performance required by applications. CloudsStorm is a dynamic control framework proposed to provide applications with agile programmability and flexibility in controlling the virtual infrastructure. With its front end, applications can design their networked infrastructure and program that infrastructure with our interpreted infrastructure code language. With the back-end engine, the infrastructure code can be executed to provision the networked infrastructure, deploy and execute the application to obtain results, and release resources. Moreover, we adopt multi-threading to support parallel operation. Finally, we conduct experiments in an assumed scenario to demonstrate functionalities of CloudsStorm. The evaluation results prove CloudsStorm is efficient for task-based applications that need to exploit Clouds but reduce the monetary cost.

Cloud Service Level Agreement (SLA) is challengeable due to lacking a trustworthy platform. This paper presents a witness model to credibly enforce the cloud service level agreement. Through introducing the witness role and using the blockchain based smart contract, we solve the trust issues about who can detect the service violation, how the violation is confirmed and the compensation is guaranteed. In this model, a verifiable consensus sortition algorithm proposed by us is firstly leveraged to select independent witnesses to form a witness committee. They are responsible for a specific service level agreement and get paid by monitoring and detecting service violation. Through carefully designing the witness' payoff function in the agreement, we further leverage game theory to analyze and prove that it is not the witness itself is trustworthy. Instead, the witness has to tell the truth because of its greedy nature, which is the desire to maximize its own revenue. As long as the service violation is confirmed by the witness committee, the compensation is automatically transferred to the customer by the smart contract. Finally, we implement a proof-of-concept prototype with the smart contract of Ethereum blockchain. It demonstrates the feasibility of our model.

Data-intensive applications usually need to deal with huge volumes of data within their deadlines. These applications can be modelled as DAGs and require parallel computation frameworks such as MapReduce and Spark to enhance the performance. The network communication has a crucial impact on the performance of an application. Coflow is intended to address the application-specific network level Quality-of-Service (QoS) requirements in cloud-based data centres. However, existing works mainly focus on scheduling coflows in a single stage. How to schedule coflows in multi-stage applications (represented as DAGs) remains to be an open problem. In this paper we study the problem of scheduling coflows in a DAG to meet its deadline requirement. Single stage coflow scheduling has been proven to be NP-hard. Multiple stages in a DAG make our problem even more complex. Owing to the complexity of the problem, we propose a genetic algorithm-based method for solving the problem. The effectiveness of our solution is verified through numerical evaluation. Experimental results show that our solution can effectively guarantee the deadline of the DAGs compared with existing single stage coflow scheduling algorithms.

Speculative execution is the mechanism adopted by current MapReduce framework when dealing with the straggler problem, and it functions through creating redundant copies for identified stragglers. The result of the quicker task will be adopted to improve the overall job execution performance. Although proved to be effective for contention caused stragglers, speculative execution can easily meet its bottleneck when mitigating data skew caused stragglers due to its replication nature: the identical unbalanced input data will lead to a slow speculative task. The Map inputs are typically even in size according to the HDFS block configuration, therefore the skew caused stragglers happen mainly in the Reduce phase because of the unknown intermediate key distribution. In this paper, we focus on mitigating data skew caused Reduce stragglers, propose ImKP, an Intermediate Key Pre-processing framework that enables the even distributed partition for Reduce inputs. A group based ranking technique has been developed that dramatically decreases the pre-processing time, and ImKP manages to eliminate this timing overhead through parallelizing the pre-processing with the file uploading procedure (from local file system to HDFS). For jobs that take input directly from HDFS, ImKP minimizes the overhead by storing themapping result on every node within the cluster for reuse. Experiments are conducted on different datasets with various workloads. Results show that, compared to the popular hash partition, ImKP can dramatically decrease Reduce skew, achieving a 99.8% reduction in the coefficient of variation of the input sizes in average, and improve up to 29.37% job response performance.

When deploying an application in the cloud, a developer often wants to know which of the wide variety of cloud resources is best to use. Most cloud providers only provide static information about different cloud resources which is often not enough because static information does not take into account the hardware and software that is being used or the policy that has been applied by the cloud provider. Therefore, dynamic benchmarking of cloud resources is needed to find out how a certain workload is going to behave on a certain instance. However, benchmarking various cloud resources is a time consuming process. Thus, using a tool which automatically benchmarks various cloud resources will be of great use. In this paper, we present the Cloud Performance Collector, a modular cloud benchmarking tool aimed to automatically benchmark a wide variety of applications. To demonstrate the benefit of the tool, we did three experiments with three synthetic benchmark applications and one real-world application using the ExoGENI testbed.

Resource provisioning is a key step in the deployment of applications onto clouds. When some datacenter is not accessible or some part of the infrastructure is crashed, the provisioning mechanism is therefore essential for these applications to recover quickly from sudden failures, especially for time critical applications. However, most current solutions focus on the cloud provider's hardware to achieve the fast provisioning of cloud resources. This paper proposes a co-provisioning mechanism to partition the customer's cloud resource requests while preserving their connectivity. This mechanism uses a brokering approach that is totally transparent to both the customer and the cloud provider, specifically considering the network topology. We carry out experiments on an NIaaS (networked infrastructure-as-a-service) platform, called ExoGENI. Experimental results and data analysis show that this mechanism is feasible and can dramatically improve the speed of resource provisioning.

As many quality critical applications are migrating to clouds, Quality of Service (QoS) and Quality of Experience (QoE) have become vital properties for cloud applications. Therefore, the provisioning mechanism, which aims to make the virtual infrastructure recover from sudden failures quickly or adapt dynamic properties of applications, is essential. However, most current provisioning mechanisms focus on the cloud provider and are developed for specific hardware. This paper proposes a mechanism to partition a customer's cloud resource requests efficiently across multiple domains or clouds, while ensuring that the partitions are still connected with each other. This mechanism exploits networked infrastructure to make dynamic cloud resource provisioning as fast as possible. It works using a broker-based model that is transparent both to the customer and to the cloud provider. It is easy for customers to use and does not force providers to make any changes to their services. Moreover, the dynamic property makes the provisioned infrastructure better able to recover from failures quickly. We implement the mechanism and carry out experiments on ExoGENI, a networked infrastructure-as-a-service (NIaaS) platform. Comprehensive experimental results and theoretical analysis demonstrate that the mechanism we propose is feasible and can dramatically improve the speed of resource provisioning.

For Securing BGP Protocol, this paper proposes an identity-based private key infrastructure (iPKI) for managing self-attested IP (sIP) addresses. An sIP address endows the current IP address self-attested characteristic, which does not rely on any credential based PKI. Based on the sIP address, we design the In-Band Self Origin Verification (IBSOV) protocol and self Route Origin Authorization (sROA) to provide a lightweight origin verification for the BGP protocol, which has a much lower overhead than the existing works.

In this demo, we propose the self-trustworthy and secure Internet protocol (T-IP) for authenticated and encrypted network layer communications. T-IP has the following advantages: 1) Self-Trustworthy IP address. 2) Low connection latency and transmission overhead. 3) Reserving to be stateless (an important merit of IP). 4) Compatible with the existing TCP/IP architecture. We have implemented the protocol and deployed it in our campus network. Compared with IPsec, the evaluation shows that T-IP has a much lower transmission overhead and connection latency.