Hierarchical Federated Learning (HFL) frameworks place edge servers between IoT devices and the cloud server to reduce communication costs and preserve privacy. In practice, however, HFL must handle hierarchical non-IID data across both device and edge levels. At the edge-level, heterogeneity arises because devices connected to the same edge server often share geographic or contextual similarities, giving each server its own optimization goal aligned with its region-specific data distribution rather than with a shared global objective. Existing HFL methods largely ignore this distinction, focusing on training a single global model that can obscure severe underperformance at the edge-level with underrepresented data. Since edge servers often act as operational units, poor performance at an edge implies degraded service quality, undermining system reliability and user trust. We propose Personalized Hierarchical Edge-enabled Federated Learning (PHE-FL), a novel method that produces personalized edge models by adaptively integrating edge- and cloud-level knowledge based on the data distribution of each edge, without incurring additional computational overhead or compromising client privacy. We deploy edge-specific test sets at each edge to ensure its unique data distribution is accurately reflected during evaluation. To the best of our knowledge, this is the first work to explicitly address hierarchical data heterogeneity in a 3-level HFL framework, both in terms of personalization and evaluation. Extensive experiments show that PHE-FL achieves up to 83% higher accuracy than existing edge-accommodated FL methods and maintains robust performance across edge-level non-IIDness, with reduced accuracy fluctuations compared to the state-of-the-art FedAvg with two levels (edge and cloud) aggregation.

In many real-world situations, the processing time of computational jobs can be shortened by lowering the processing quality. This is referred to as discretely controllable processing time, where the original processing time can be shortened to a number of levels with lower processing qualities. In this paper, we study the scheduling problem of periodic jobs with discretely controllable processing times on two machines. The problem is NP-hard, as directly solving it through dynamic programming leads to exponential computational complexity. This is because we need to memorise a set of processed jobs to avoid reprocessing. In order to address this issue, we prove the Ordered Scheduling Structure (OSS) Property and the Consecutive Decision Making (CDM) Property. The OSS Property allows us to search for an optimal solution in which jobs on the same machine are orderly started. The CDM Property allows us to memorise only two jobs to completely avoid the job reprocessing. These two properties greatly decrease the searching space, and the resultant dynamic programming solution to find an optimal solution is with pseudo-polynomial computational complexity.

Personalized Federated Learning (pFL) tailors models to each client’s local data distribution in heterogeneous federated learning settings. Federated Meta-Learning (FML) is a branch of pFL that uses meta-learning to achieve fast adaptation, where clients start with a meta-model and personalize it by fine-tuning it with local data. Since a single global meta-model has limitations when the data distribution of clients varies significantly, meta-model personalization should be considered in FML. However, most benchmark pFL methods lack meta-model personalization, and usually lack meta-learning or relying on a single global meta-model. Besides, these methods can neither provide meta-model personalization nor guarantee generalization and convergence, due to the challenges in measuring the distance between the meta-model and the client model in FML. To address these issues, we combine FML with hypernetwork and propose a constrained hypernetwork-based FML framework called FMLH, which innovatively utilizes hypernetwork to capture the differences in fine-tuned models, thereby providing personalized meta-models for each client. We provide rigorous mathematical proofs illustrating how the hypernetwork affects the convergence and generalization bounds of FMLH. Experimental results demonstrate that FMLH significantly improves the generalization of the model in cross-client shifts, with the lowest decile accuracy improved by up to 18.71%. FMLH also outperforms representative pFL algorithms by up to 5.6% in terms of maximum accuracy improvement.

Federated learning, as a compelling machine learning framework, enables collaborative model training without exposing private data. However, the excessive communication overhead remains a major challenge. To tackle this challenge, hierarchical federated edge learning (HFEL) framework has been proposed for reducing the communication load via migrating the model aggregation partially from cloud to edge servers. Although HFEL has significant potential, it is still constrained by end-devices with limited computational capabilities and unfavorable network conditions. A common approach to reduce this effect is to involve only the fastest end-devices in the training process. But because only parts of end-devices' data samples can be selected by such means, it damages the diversity of training data, and hence greatly affects the model's quality. In addition, for further improving the training performance, a proper bandwidth allocation strategy is also needed to make full use of the shared network resource of edge servers. To this end, we proposeDAHFF, aDiversity-AwareHierarchicalFastFederated learning framework consisting ofVirtual Queue based Device Selectionphase andBinary Search based Bandwidth Allocation, which are responsible for selecting participated end-devices and allocating bandwidth for selected devices, respectively. Extensive experiments on different deep learning models show that our proposed framework can averagely speed up the training performance by$2.07\times$in comparison with state-of-the-art approaches.

The emergence of Cooperative Intelligent Transportation Systems (C-ITS) has revolutionized urban mobility by enabling seamless collaboration among vehicles, infrastructure, and individuals to improve traffic management, safety, and efficiency. Deep Reinforcement Learning (DRL) has become a key technology in this ecosystem, empowering autonomous agents to make real-time decisions that optimize traffic flow, reduce congestion, and enhance road safety. Although many surveys on Intelligent Transportation Systems (ITS) either overlook cooperative aspects or primarily emphasize security, this paper bridges the gap by examining the diverse applications of DRL in C-ITS. It examines critical areas such as traffic signal control, AV coordination, route planning, and human-vehicle interaction. The study also traces the evolution of DRL algorithms, their adaptation to transportation challenges, and their integration with cutting-edge projects and standards. Additionally, the paper provides a comprehensive analysis of current research trends, identifying achievements, unresolved challenges, and future directions in the field. By synthesizing existing literature and highlighting the synergy between DRL and C-ITS, this survey serves as a valuable resource for researchers, policymakers, and industry professionals striving to develop intelligent, cooperative, and sustainable transportation systems. The insights offered aim to guide advancements in this rapidly growing domain, fostering innovation and practical implementation.

WiFi devices are ubiquitous in modern environments, from smartphones and laptops to IoT sensors and AR/VR headsets. Identifying device types/models within these populations enables crowd analysis, network optimization, and detection of unusual devices. Current identification methods struggle with MAC address randomization, require large training datasets, and perform poorly in real-world deployments. This paper introduces a device identification method based on Information Element (IE) attributes extracted from WiFi probe requests. We evaluate the approach using probe requests captured in the 2.4 GHz band. Evaluation across 70+ device types yields 99% precision, 98% recall, and 99% F1 score, exceeding deep learning approaches (92% F1 score) under similar training conditions. Our approach maintains accuracy despite MAC randomization and requires minimal training data. We demonstrate practical applicability through an operational dashboard tested in real-world scenarios for urban planning and network management. Case studies across diverse environments confirm the effectiveness of the method for operational use.

Digital twin (DT) technology enables smooth integrations of cyber and physical worlds in alignment with the Industry 4.0 initiative. DTs are virtual presentations of physical objects. Through synchronizations with physical objects in real-time, DTs can reflect the states of their objects with high fidelity. Orthogonal to the DT technology, mobile edge computing (MEC) is a promising computing paradigm that shifts computing power to the edge network, which is appropriate for delay-sensitive intelligent services. In this paper, we study fidelity-aware inference services in a DT-assisted MEC environment, where machine learning-based inference models must be continuously retrained using updated DT data in order to provide high-fidelity services for consumers. To this end, we first formulate two novel optimization problems: the initial DT and model placement problem with the aim of minimizing the total cost of various resources consumed, and the cumulative fidelity maximization problem to maximize the long-term cumulative fidelity of service models while minimizing the cost of resource consumption on service model fidelity enhancements over a given time horizon, through jointly scheduling mobile devices to upload their update data to synchronize with their DTs and determining whether DTs and/or models to be migrated at each time slot. We then develop an efficient algorithm for the initial DT and model placement problem, through a reduction to a series of minimum-cost maximum matching problems in auxiliary graphs. We also devise an online algorithm with a provable competitive ratio for the cumulative fidelity maximization problem, by designing an elegant service request admission strategy. Finally, we evaluate the performance of the proposed algorithms via simulations. Simulation results demonstrate that the proposed algorithms are promising, and outperform their baselines by no less than 28%.

Performance interference in multi-tenant container-ized environments-such as those utilizing Linux Containers (LX C)-poses a critical challenge to maintaining Quality of Service (QoS) and adhering to Service Level Agreements (SLAs), especially under conditions of high resource contention. Con-ventional static resource allocation strategies often fail to adapt to dynamic workload behaviors and lack cross-resource coor-dination, leading to inefficiencies and degraded performance. While machine learning-based approaches, such as Long Short-Term Memory (LSTM) predictors, have demonstrated improved forecasting capabilities, their computational complexity and training requirements introduce latency and overhead, rendering them impractical for real-time control in resource-constrained deployments. In this paper, we introduce a lightweight, real-time interference-aware resource management solution that integrates predictive modeling with fine-grained CPU and I/O capping mechanisms using Linux cgroup subsystems. Our solution lever-ages continuous profiling of key performance metrics, including QoS violation frequency, CPU throttling rates, and I/O contention signals, to identify emerging interference patterns across co-located containers. The solution dynamically adjusts resource quotas and scheduling parameters in response to runtime observations, enabling adaptive capacity provisioning with minimal system overhead. We implement and evaluate our solution on a heterogeneous LXC-based container cluster with up to 32 concurrently running containers. Experimental results show that our proposed framework achieves an average speedup of 71.4 % for latency-sensitive high-priority workloads compared to the default LX C scheduler, while significantly reducing interference-induced performance degradation across mixed-priority services.

Breast cancer remains a leading cause of mortality among women, with millions of new cases diagnosed annually. Early detection through screening is crucial. Using neural networks to improve the accuracy of breast cancer screening has become increasingly important. In accordance with radiologists' practices, we proposed using images from the unaffected side to create adversarial samples with critical medical implications in our adversarial learning process. By introducing beneficial perturbations, this method aims to reduce overconfidence and improve the precision and robustness of breast cancer classification. Our proposed framework is an adversarial quadruple-view classification network (NaFV-Net) incorporating images from both affected and unaffected perspectives. By comprehensively capturing local and global information and implementing adversarial learning from four mammography views, this framework allows for the fusion of features and the integration of medical principles and radiologist evaluation techniques, thus facilitating the accurate identification and characterization of breast tissues. Extensive experiments have shown the high effectiveness of our model in accurately distinguishing between benign and malignant findings, demonstrating state-of-the-art classification performance on both internal and public datasets.

Advanced Vector Extensions 512 (AVX-512), a modern SIMD instruction set for x86 architectures, enables data-level parallelism through 512-bit wide ZMM registers capable of processing multiple data elements concurrently within a single instruction cycle. In this study, we present a high-throughput, lock-free, in-memory architecture for key-value data-stores that exploits AVX-512 vector operations to accelerate fundamental operations such as insertion and lookup. Our design introduces an optimized memory layout that partitions the key space into two disjoint regions (primary and secondary) and employs three independent hash functions to identify candidate slots. This asymmetric layout improves key distribution, reduces collision probability, and enhances overall lookup efficiency. Experimental evaluation shows that this strategy yields the lowest insertion failure rate among tested memory partitioning schemes. By leveraging AVX-512 instructions in combination with most optimized memory layout, our implementation achieves insertion throughput within 6% of Intel TBB's highly optimized multithreaded hash map, despite avoiding explicit synchronization or thread-level parallelism. Under workloads with 550 million entries and a 90% miss rate, our approach delivers 4.0-5.1x speedup over standard STL, Boost, Robin-Hood, and Abseil hash maps, and up to$2.5 x$improvement relative to TBB and Abseil. These gains are consistently observed for both 32-bit and 64-bit floating-point key types. The results confirm the viability of AVX-512-centric designs as a cost-effective alternative to thread-level parallelism, particularly in environments where minimizing synchronization overhead and ensuring deterministic execution are critical. Our findings suggest for a paradigm shift in CPU and system architecture, emphasizing wider vector units and improved memory bandwidth utilization as primary levers for scalable high-performance computing. These findings suggest that future extensions of AVX-512 capabilities, such as non-blocking memory loads, expanded vector registers, and asynchronous prefetching, could enhance the efficiency of data-intensive workloads.

Beyond the worst-case analysis of algorithms, the learning-augmented framework considers that an algorithm can leverage possibly imperfect predictions about the unknown variables to have guarantees tied to the prediction quality. We consider online non-clairvoyant scheduling to minimize the max-stretch under this framework, where the scheduler can access job size predictions. We present a family of algorithms: Relaxed-Greedy (RG) with an $O(\eta^3 \cdot \sqrt{P})$ competitive ratio, where $\eta$ denotes the prediction error for job sizes and $P$ the maximum job size ratio; Adaptive Relaxed-Greedy with an $O(\lambda^{0.5} \cdot \eta^{2.5} \cdot \sqrt{P})$ competitive ratio, where $\lambda$ denotes the error for the minimum job size; Predictive Relaxed-Greedy with an $O(\lambda^{0.5} \cdot \varphi^{0.5} \cdot \eta \cdot \max \\\{ \eta, \varphi \\\} \cdot \sqrt{P})$ competitive ratio, where $\varphi$ denotes the error for the maximum job size. We also present *${RG}^x$*, an algorithm that represents a trade-off between consistency and smoothness, with an $O(\eta^{2+2x} \cdot P^{1-x})$ competitive ratio. We introduce a general method using resource augmentation to bound robustness, resulting in *RR*-augmented *RG*, with a $(1 + \epsilon)$-speed $O(\min \\\{ \eta^3 \sqrt{P}, \frac{n}{\epsilon} \\\})$ competitive ratio. Finally, we conduct simulations on synthetic and real-world datasets to evaluate the practical performance of these algorithms.

Federated Learning (FL) enables privacy-preserving machine learning by allowing clients to collaboratively train models without sharing raw data. Federated Learning as a Service (FLaaS) extends this approach to cloud infrastructures. However, conventional secure aggregation protocols, such as Google's SecAgg and SecAgg+, introduce high computation and communication overheads, particularly in large-scale FLaaS deployments where client dropout rates are limited. To address these challenges, we propose ACCESS-FL, a lightweight, secure aggregation method designed for honest-but-curious FLaaS scenarios with stable network conditions. ACCESS-FL eliminates double masking, Shamir's Secret Sharing, and excessive encryption/decryption by creating shared secrets only between two peers per client, which reduces computation and communication complexity to constant$O(1)$and makes the algorithm independent of network size and comparable to standard FL. ACCESS-FL preserves privacy against inversion attacks and maintains model accuracy equivalent to the FL, SecAgg, and SecAgg+ protocols, proving that reducing overhead does not compromise learning performance and achieves communication and computation costs comparable to standard FL. Experimental evaluations on benchmark datasets (MNIST, FMNIST, and CIFAR-10) demonstrate lower overhead, making ACCESS-FL practical for service-based stable FLaaS applications such as healthcare analytics.

Recent key-value (KV) caches are offloaded to programmable switches to offer high query processing performance. However, they suffer from both low accuracy in hot key detection and high latency in cache updates due to the strict limitations on switch registers. We propose TurboCache, a switch-accelerated KV cache with accurate hot key detection and fast cache updates. Our key idea is to leverage the switch recirculation capability to build a novel data structure that caches hot KV pairs. With this hardware-compatible cache data structure, TurboCache designs efficient data plane algorithms that accurately detects new hot keys and quickly updates its cache entirely within switch ASIC pipelines. We have implemented TurboCache on a${64}\times {100}$Gbps Tofino switch. Testbed results indicate that TurboCache improves the hot key detection accuracy and decreases the cache update latency of existing KV caches by several orders of magnitude.

Online portfolio selection seeks to determine a sequence of allocations to maximize capital growth. Classical universal strategies asymptotically match the best constant-rebalanced portfolio but ignore potential forecasts, whereas heuristic methods often collapse when belief fails. We formalize this tension in a learning-augmented setting in which an investor observes (possibly erroneous) predictions prior to each decision moment, and we introduce the Rebalanced Arithmetic Mean portfolio with predictions (RAM). Under arbitrary return sequences, we prove that RAM captures at least a constant fraction of the hindsight-optimal wealth when forecasts are perfect while still exceeding the geometric mean of the sequence even when the predictions are adversarial. Comprehensive experiments on large-scale equity data strengthen our theory, spanning both synthetic prediction streams and production-grade machine-learning models. RAM advantages over universal-portfolio variants equipped with side information across various regimes. These results demonstrate that modest predictive power can be reliably converted into tangible gains without sacrificing worst-case guarantees.

Recent cyberattacks have increasingly targeted distributed networking environments like IoT networks. To detect these attacks, hidden under network traffic encryption, many centralized Machine Learning (ML) based solutions have been introduced, which are not well suited for IoT networks. This work proposes PIFL a practical approach to secure IoT networks by combining federated learning, in-network ML using P4-enabled devices, software-defined networks, and binarized neural networks. PIFL detects compromised edge devices and isolates them into separate network slices based on trust parameters derived from their behavior. We demonstrate the feasibility of PIFL using an experimental testbed with three intelligent network devices and seven IoT devices implemented on Raspberry Pi devices.

Identifying attacks on Internet of Things (IoT) systems through anomaly detection is an effective approach and remains a crucial area of research. The core method involves collecting system-related data during normal operation to establish a baseline of typical behavior and then continuously monitoring for deviations from this baseline. Using system call sequences for anomaly detection is a well-established and important field. System call sequences effectively capture the behavior of a target system at a low level, allowing identification of any changes in this behavior; however, these approaches face several challenges, including high false-positive rates, the need for segmentation of long sequences, and the difficulty of detecting anomalies when the system call data comes from multiple processes. This work presents a novel anomaly-detection approach that uses a position-specific scoring mechanism to analyze the content and structural properties of system call sequences. The proposed approach addresses key challenges in this field, including fixed-length segmentation of system call sequences, predetermined anomaly-detection thresholds, the detection of anomalies in both single and multiple processes, and high false-positive rates. We extensively evaluated the proposed approach using system-call-specific public datasets (ADFA-LD and UNM) of a diverse nature. The performance of the proposed content-based, structure-based, and combined content- and structure-based anomaly-detection methods was evaluated using ten-fold cross-validation. The proposed anomaly-detection approach achieves an impressive detection rate of 1.0, along with exceptionally low false-positive rates of 0.001 and 0.017 when evaluated on the UNM and ADFA-LD datasets, respectively.

With the rapid expansion of next-generation networking, Internet of Things (IoT) devices have become central components of federated learning (FL) networks. FL offers a paradigm for distributed training machine learning models while preserving user data privacy. However, existing network security measures often struggle to identify legitimate contributors from opportunistic free riders within these networks. The Free Rider (FR) problem arises when participants seek to benefit from the FL processes without contributing. In particular, free riders are known to exist within or outside of the network, whereas outside free riders can hardly be identified. The Zero Trust model proposes an environment where no entity, including the network itself, is inherently trusted, providing a foundation to counter external threats seeking to exploit the network. This study proposes a novel framework strengthened by the Zero Trust model to identify external free riders in FL networks. Leveraging a Deep Autoencoding Gaussian Mixture Model (DAGMM)-based technique for internal free rider detection, our framework demonstrates superior performance in identifying free riders across various FR scenarios compared to current state-of-the-art solutions. Through our proposed framework and the principles of Zero Trust, we establish a robust security guarantee for FL networks, ensuring the integrity of the learning process.

Asynchronous Byzantine Fault Tolerant (BFT) consensus protocols have garnered significant attention with the rise of blockchain technology. A typical asynchronous protocol is designed by executing sequential instances of the Asynchronous Common Sub-seQuence (ACSQ). The ACSQ protocol consists of two primary components: the Asynchronous Common Subset (ACS) protocol and a block sorting mechanism, with the ACS protocol comprising two stages: broadcast and agreement. However, current protocols encounter three critical issues: high latency arising from the execution of the agreement stage, latency instability due to the integral-sorting mechanism, and reduced throughput caused by block discarding. To address these issues, we propose Falcon, an asynchronous BFT protocol that achieves low latency and enhanced throughput. Falcon introduces a novel broadcast protocol, Graded Broadcast (GBC), which enables a block to be included in the ACS set directly, bypassing the agreement stage and thereby reducing latency. To ensure safety, Falcon incorporates a new binary agreement protocol called Asymmetrical Asynchronous Binary Agreement (AABA), designed to complement GBC. Additionally, Falcon employs a partial-sorting mechanism, allowing continuous rather than simultaneous block committing, enhancing latency stability. Finally, we incorporate an agreement trigger that, before its activation, enables nodes to wait for more blocks to be delivered and committed, thereby boosting throughput. We conduct a series of experiments to evaluate Falcon, demonstrating its superior performance.

Standing as a foundational element within blockchain systems, theByzantine Fault Tolerant(BFT) consensus has garnered significant attention over the past decade. The introduction of aDirected Acyclic Directed(DAG) structure into BFT consensus design, termed DAG-based BFT, has emerged to bolster throughput. However, prevalent DAG-based protocols grapple with substantial latency issues, suffering from a latency gap compared to non-DAG protocols. For instance, leading-edge DAG-based protocols named GradedDAG and BullShark exhibit a good-case latency of$4$and$6$communication rounds, respectively. In contrast, the non-DAG protocol, exemplified by PBFT, attains a latency of$3$rounds in favorable conditions. To bridge this latency gap, we propose Remora, a novel DAG-based BFT protocol. Remora achieves a reduced latency of$3$rounds by incorporating optimistic paths. At its core, Remora endeavors to commit blocks through the optimistic path initially, facilitating low latency in favorable situations. Conversely, in unfavorable scenarios, Remora seamlessly transitions to a pessimistic path to ensure liveness. Various experiments validate Remora's feasibility and efficiency, highlighting its potential as a robust solution in the realm of BFT consensus protocols.

In Mobile Edge Computing (MEC) scenarios, computational tasks are popularly deployed using containerization to isolate the runtime environment. To complete the execution of the task, the edge server first pulls the image, then instantiates and runs the container. Since it takes a lot of time for the edge server to download the image from the cloud, image reuse reduces the pulling latency significantly. However, the limited storage capacity of edge servers hinders image reuse. Recent works have enhanced reuse efficiency by leveraging the hierarchical structure of images and caching high-value layers. However, their efficiency remains limited due to the lack of multi-container collaboration. This paper proposes a novel container scheduling strategy based on image layer reuse and sequence arrangement (ILR-SA) for MEC scenarios, which achieves efficient scheduling by collaborating multiple containers. First, containers are greedily deployed into the edge cluster. Then, the execution sequence of containers is modeled as an optimal Hamiltonian path problem, efficiently solved by our proposed decomposition algorithm. Finally, an efficient image layer update strategy is used to achieve layer reuse. We conduct rigorous experiments to demonstrate that our proposed container scheduling strategy reduces the computational task completion time by up to 91.3% compared to existing approaches.

As a promising vehicular communication technology, Cellular Vehicle-to-Everything (C-V2X) is expected to ensure the safety and convenience of Intelligent Transportation Systems (ITS) by providing global road information. However, it is difficult to obtain global road information in practical scenarios since there will still be many vehicles on the road without onboard units (OBUs) in the near future. Specifically, although C-V2X vehicles have sensors that can perceive their surroundings and broadcast their perceived information to the C-V2X system, their line-of-sight (LoS) is limited and obscured by the environment, such as other vehicles and terrain. Besides, vehicles without OBUs cannot share their perceived information. These two problems cause extensive areas with unperceived information in the C-V2X system, and whether vehicles are in these areas is unknown. Thus, extending the perceivable range of the limited scenario for C-V2X applications that require global road information is necessary. To this end, this paper pioneers investigating the scenario inpainting task problem in C-V2X. To solve this challenging problem, we propose an effiCient trAfficScenario inpainTing (CAST) solution consisting of a generative architecture and knowledge distillation, simultaneously considering the inpainting precision and computation efficiency. Extensive experiments have been conducted to demonstrate the effectiveness of CAST in terms of Precise Inpaint Rate (PIR), Rough Inpaint Rate (RIR), Lane-Level Inpaint Rate (LLIR), and Inpaint Confidence Error (ICE), paving the way for novel solutions for the inpainting problem in more complex road scenarios.

The emergence of the mobile edge computing paradigm enables the deployment of microservices on edge servers, which greatly improves the quality of services and reduces network transmission costs. However, due to limited computing and storage resources, an individual edge server can host only a limited number of microservice instances. Moreover, user mobility often results in uneven distribution of service requests in mobile edge computing systems. To this end, it is a key problem to dispatch microservice requests to appropriate edge servers to minimize the average service response time. Current solutions to this problem rely on centralized methods and suffer from serious problems of single point of failure, error-proneness, difficult expansion, low robustness, etc. To resolve these problems, this paper proposes a decentralized game-theoretic approach for dispatching microservice requests effectively and efficiently in mobile edge computing systems. Specifically, we formulate the request dispatching problem as a decentralized non-cooperative game and propose a decentralized request dispatching algorithm that can find the Nash equilibrium through finite iterations. We conduct a series of experiments to demonstrate that our approach beats benchmarking approaches with close-to-optimal performance and high efficiency measured by convergence time.

Attributing cyberattacks in Internet of Things (IoT) environments is challenging due to their distributed, heterogeneous natures and the limitations of traditional Digital Forensics (DF) tools for preserving privacy and scalability. This paper presents FedCAD, a novel Federated Learning (FL)-based forensic method for multi-attribute cyberattack attribution that operates directly on IoT devices. It leverages a lightweight 1D Convolutional Neural Network (CNN) architecture with a shared feature extractor and three parallel sub-networks, each of which targets a distinct attribute, i.e., attack types, tactics/tools, and motives. Its architecture consists of three layers: an IoT device one collecting traces and local training; an FL one for privacy-preserving model aggregation via FedAvg; and a DF analysis one for multi-attribute inference and reporting. Also, FedCAD integrates the MITRE ATT&CK framework to enrich training data with real-world adversarial knowledge. Evaluations on the three public datasets TON-IoT, Bot-IoT, and UNSW-NB15 show that FedCAD outperforms centralized models, with gains of 2.1% accuracy, 1.0% precision, 1.5% recall, and 1.3% f1-score on TON-IoT. Experimental results demonstrate FedCAD's effectiveness as a scalable, privacy-preserving solution for cyberattack attribution in dynamic IoT ecosystems.

In recent years, social media has emerged as a powerful tool for sustainability marketing. It leverages its extensive reach and interactive characteristics to raise awareness of environmental issues, promote community engagement, and influence policy development. This paper comprehensively analyzes different strategies used on social media platforms to support sustainability goals. We have classified these strategies into six broad categories: campaigns and education to increase awareness; activities that engage and build communities; efforts to advocate and influence policies; initiatives related to corporate social responsibility and branding; projects that use crowdsourcing and collaboration; and behavior change campaigns. We assess the effectiveness of these strategies based on case studies and key performance metrics for sustainable practices, public behaviors, and corporate marketing. Additionally, the presented survey highlights issues such as misinformation, engagement fatigue, and authenticity, necessitating solutions to address these concerns. This study highlights the vital role of social media in achieving sustainability objectives and outlines future research directions to improve its effectiveness in supporting these goals. Therefore, this research offers valuable insights for practitioners, policymakers, and scholars aiming to leverage social media platforms to advance sustainability goals.

Artificial intelligence (AI) has immense potential in time series prediction, but most explainable tools have limited capabilities in providing a systematic understanding of important features over time. These tools typically rely on evaluating a single time point, overlook the time ordering of inputs, and neglect the time-sensitive nature of time series applications. These factors make it difficult for users, particularly those without domain knowledge, to comprehend AI model decisions and obtain meaningful explanations. We propose CGS-Mask, a post-hoc and model-agnostic cellular genetic strip mask-based saliency approach to address these challenges. CGS-Mask uses consecutive time steps as a cohesive entity to evaluate the impact of features on the final prediction, providing binary and sustained feature importance scores over time. Our algorithm optimizes the mask population iteratively to obtain the optimal mask in a reasonable time. We evaluated CGS-Mask on synthetic and real-world datasets, and it outperformed state-of-the-art methods in elucidating the importance of features over time. According to our pilot user study via a questionnaire survey, CGS-Mask is the most effective approach in presenting easily understandable time series prediction results, enabling users to comprehend the decision-making process of AI models with ease.

Pain, a primary reason for seeking medical help, requires essential pain assessment for effective management. Studies have recognized electrodermal activity (EDA) signaling's potential for automated pain assessment, but traditional algorithms often ignore the noise and uncertainty inherent in pain data. To address this, we propose a learning framework predicated on data uncertainty, introducing two forms: a) subject-level stimulation-reaction drift; b) ambiguity in self-reporting scores. We formulate an uncertainty assessment using Heart Rate Variability (HRV) features to guide the selection of responsive pain profiles and reweight subtask importance based on the vagueness of self-reported data. These methods are integrated within an end-to-end neural network learning paradigm, focusing the detector on more accurate insights within the uncertainty domain. Extensive experimentation on both the publicly available biovid dataset and the proprietary Apon dataset demonstrates our approach's effectiveness. In the biovid dataset, we achieved a 6% enhancement over the state-of-the-art methodology, and on the Apon dataset, our method outperformed baseline approaches by over 20%.

The surge in real-time IoT data introduces scalability and computational challenges, necessitating advanced architectural and technological solutions. Streamed data processing is increasingly adopted across industries to enhance operational efficiency by extracting insights from vast, unstructured datasets. However, complex analytical tasks, such as multi-join queries, often require stateful iterative calculations on high-volume, high-velocity data, which challenges conventional programming models like MapReduce. This paper introduces an architectural model enabling application developers to create intricate streaming computational logic within an IoT platform. Our architecture supports scalable applications across distributed edge-tier nodes, particularly for iterative analytical operations on streamed data. We discuss core concepts and a timestamp model (borrowed from the timely data-flow concept) attached to data items circulating between computational blocks, which can execute concurrently on different edge-tier nodes. Additionally, we detail a buffer management mechanism that dynamically adjusts memory size in each computational block on nodes with limited capacity. This mechanism considers application performance requirements and runtime conditions to optimize buffer sizes. Performance evaluation against cloud-tier alternatives confirms the effectiveness of our solution. Experimental results show a significant reduction in p-99 delay compared to cloud-tier deployment with a database engine for analytical applications involving multi-join operations.

Distributed Deep Neural Network (DNN) inference is a promising technology to explore the distributed resources in edge cloud to realize edge intelligence. Meanwhile the inherent resource sharing nature of edge cloud infrastructure also raises serious concerns on security and privacy. Software Guard Ex-tensions (SGX) emerges as a potential hardware-level solution but its limited secure memory (i.e., enclave page cache) imposes new challenges, especially in contrast to memory-hungry DNN models. A task's performance will be severely affected when its memory footprint is beyond the enclave page cache size, due to expensive secure page swapping. In this case, how to appropriately partition a DNN model and assign the partitions to distributed edge servers to efficiently utilize edge resources for fast secure inference becomes a challenging problem. In this paper, we first show that this problem is NP-hard. We further propose a MEmory -aware Distributed Inference Acceleration (MEDIA) algorithm, whose guaranteed approximation ratio is also formally analyzed. We have implemented a prototype system and applied some well-known representative DNN models to evaluate MEDIA's performance. Through extensive experiments, we verify the efficiency of MEDIA by the fact that it reduces the inference time by 19.5%-38.1 % in comparison with state-of-the-art approaches.

Multi-tenant containerized environments offer numerous benefits to virtualized computing platforms, providing a lightweight and consistent environment for application development. However, the shared nature of resources among coresident containers introduces interference, potentially leading to performance degradation and violating service level agreements (SLA) set by end-users. Addressing the inherent interference in multi-tenant containerized environments is a challenging yet promising endeavor, as it can significantly impact performance and violate SLAs. This paper presents a lightweight system designed to diagnose and control interference in a multitenant containerized environment. We have implemented the proposed solution on top of the Linux container environment and conducted experiments with diverse CPU- and I/O-intensive workloads, including real-world applications like distributed data processing and event stream processing pipelines. The results highlight that our solution achieves an average prediction error below 24% in CPU-bound workloads, with none surpassing 38% across diverse workloads.

The development of the Internet of Things (IoT), Big Data, and deep learning technologies has brought convenience to people’s lives. As personal privacy data protection laws and regulations tighten, the cost of acquiring high-quality annotated data from vast IoT datasets has significantly increased, resulting in prevalent issues such as data acquisition challenges and label noise in training data. In this work, we focus on the demand for privacy protection and trustworthy sharing of IoT data, and propose a method for addressing data bias in IoT through federated learning and blockchain by utilizing the theory of local intrinsic dimension (LID), incorporating committee consensus to achieve noise label identification and correction at the data level, reducing information loss in the training data. Additionally, it performs screening of low-quality local model updates at the model level, leveraging blockchain technology that addresses the single point of failure issues in traditional federated learning, ensuring the performance and security of the federated learning models. Analysis, proof of convergence, and experimentations on the proposed framework demonstrate good security and robustness in noisy environments, effectively addressing data bias in intelligent IoT settings. In scenarios with a noise level of 0.3, 0.6, and 0.9, the average model accuracy improved respectively by 7.75%, 7.30%, and 14.04% compared to FedAvg. Similarly, when compared to FedCorr, the average improvement in model accuracy is 5.19%, 3.63%, and 8.74% respectively. Moreover, the training time remains within an acceptable range for all cases.

The development of cloud computing delivery models inspires the emergence of cloud-native computing. Cloud-native computing, as the most influential development principle for web applications, has already attracted increasingly more attention in both industry and academia. Despite the momentum in the cloud-native industrial community, a clear research roadmap on this topic is still missing. As a contribution to this knowledge, this article surveys key issues during the life cycle of cloud-native applications, from the perspective of services. Specifically, we elaborate on the research domains by decoupling the life cycle of cloud-native applications into four states: building, orchestration, operation, and maintenance. We also discuss the fundamental necessities and summarize the key performance metrics that play critical roles during the development and management of cloud-native applications. We highlight the key implications and limitations of existing works in each state. The challenges, future directions, and research opportunities are also discussed.

Cache miss analysis has become one of the most important things to improve the execution performance of a program. Generally, the approaches for analyzing cache misses can be categorized into dynamic analysis and static analysis. The former collects sampling statistics during program execution but is limited to specialized hardware support and incurs expensive execution overhead. The latter avoids the limitations but faces two challenges: inaccurate execution path prediction and inefficient analysis resulted by the explosion of the program state graph. To overcome these challenges, we propose CBANA, an LLVM- and process address space-based lightweight, efficient, and flexible cache behavior analysis framework. CBANA significantly improves the prediction accuracy of the execution path with awareness of inputs. To improve analysis efficiency and utilize the program preprocessing, CBANA refactors loop structures to reduce search space and dynamically splices intermediate results to reduce unnecessary or redundant computations. CBANA also supports configurable hardware parameter settings, and decouples the module of cache replacement policy from other modules. Thus, its flexibility is established. We evaluate CBANA by using the popular open benchmark PolyBench, graph workloads, and our synthetic workloads with good and poor data locality. Compared with the popular dynamic cache analysis tools Perf and Valgrind, the cache miss gap is less than 3.79% and 2.74% respectively with over ten thousand data accesses for the synthetic workloads, and the time reduction is up to 92.38% and 97.51% for the multiple-path workloads. Compared with the popular static cache analysis tool Heptane, CBANA achieves a time reduction of 97.71% while ensuring accuracy at the same time.

Vehicular Edge Computing (VEC) is a transportation-specific version of Mobile Edge Computing (MEC) designed for vehicular scenarios. Task offloading allows vehicles to send computational tasks to nearby Roadside Units (RSUs) in order to reduce the computation cost for the overall system. However, the state-of-the-art solutions have not fully addressed the challenge of large-scale task result feedback with low delay, due to the extremely flexible network structure and complex traffic data. In this paper, we explore the joint task offloading and resource allocation problem with result feedback cost in the VEC. In particular, this study develops a VEC computing offloading scheme, namely, a Lagrange multipliers-based adaptive computing offloading with prediction model, considering multiple RSUs and vehicles within their coverage areas. First, the VEC network architecture employs GAN to establish a prediction model, utilizing the powerful predictive capabilities of GAN to forecast the maximum distance of future trajectories, thereby reducing the decision space for task offloading. Subsequently, we propose a real-time adaptive model and adjust the parameters in different scenarios to accommodate the dynamic characteristic of the VEC network. Finally, we apply Lagrange Multiplier-based Non-Uniform Genetic Algorithm (LM-NUGA) to make task offloading decision. Effectively, this algorithm provides reliable and efficient computing services. The results from simulation indicate that our proposed scheme efficiently reduces the computation cost for the whole VEC system. This paves the way for a new generation of disruptive and reliable offloading schemes.

The deep neural networks are envisaged for the early disease diagnosis from medical images. However, in the early stage of the disease, the medical images of patients and healthy people have only subtle visual differences. Distinguishing the medical images for early diagnosis belongs to the Fine-Grained Visual Classification (FGVC) task. Many recent works are based on a standard FGVC learning paradigm: locate the discriminative regions first and then classify by fusing the information of these regions. However, it is still not enough for medical images. Because the shape and size of the lesions are variable, and the relationship between lesions and the background is complex. In order to solve these problems, we propose a fine-grained lesion classification framework for early auxiliary diagnosis. We first locate and extract multiple lesions with different sizes and shapes from the original image and then fuse the feature of lesion and background based on attention mechanism. As shown by experiment results in two real-world clinical data sets, our model can locate accurately and perform better.

Model prediction based on machine learning is provided as a service in cloud environments, but how to verify that the model prediction service is entirely conducted becomes a critical challenge. Although zero-knowledge proof techniques potentially solve the integrity verification problem when applied to the prediction integrity of massive privacy-preserving Convolutional Neural Networks (CNNs), the significant proof burden results in low practicality. In this research, we present psvCNN (parallel splitting zero-knowledge technique for integrity verification). The psvCNN scheme effectively improves the utilization of computational resources in CNN prediction integrity, proving by an independent splitting design. Through a convolutional kernel-based model splitting design and an underlying zero-knowledge succinct non-interactive knowledge argument, our psvCNN develops parallelizable zero-knowledge proof circuits for CNN prediction. Furthermore, psvCNN presents an updated Freivalds algorithm for a faster integrity verification process. Experiments show that psvCNN is practical and efficient in terms of proof time and storage, generating a prediction integrity proof with a proof size of 1.2MB in 7.65s for the structurally complicated CNN model VGG16. psvCNN is 3765 times faster than the latest zk-SNARK-based non-interactive method vCNN, and 12 times faster than the latest sumcheck-based interactive technique zkCNN in terms of proving time.

Software-defined networking (SDN) allows flexible and centralized control in cloud data centers. An elastic set of distributed SDN controllers is often required to provide sufficient yet cost-effective processing capacity. However, this introduces a new challenge: Request Dispatching among the controllers by SDN switches. It is essential to design a dispatching policy for each switch to guide the request distribution. Existing policies are designed under certain assumptions, including a single centralized agent, global network knowledge, and a fixed number of controllers, which often cannot be satisfied in practice. This article proposes MADRina, Multiagent Deep Reinforcement Learning for request dispatching, to design policies with high dispatching adaptability and performance. First, we design a multiagent system to address the limitation of using a centralized agent with global network knowledge. Second, we propose a Deep Neural Network-based adaptive policy to enable request dispatching over an elastic set of controllers. Third, we develop a new algorithm to train the adaptive policies in a multiagent context. We prototype MADRina and build a simulation tool to evaluate its performance using real-world network data and topology. The results show that MADRina can significantly reduce response time by up to 30% compared to existing approaches.

Personalized recommendation is deemed ubiquitous. Indeed, it has been applied to several online services (e.g., E-commerce, advertising, and social media applications, to name a few). Learning unknown user preferences from user-provided data lies at the core of modern collaborative filtering recommender systems. However, there is an incentive for malicious attackers to manipulate the learned preferences, which could affect business decision making, by injecting poisoned data. In the face of such a poisoning attack, while previous works have proposed a number of defense methods succeeding in other machine learning (ML) tasks, little is effective for collaborative filtering (CF). Thereof, we present a new defense scheme called poison-tolerant collaborative filtering (PTCF), which is highly robust against poisoning attacks on collaborative filtering. Different from the defenses that remove outliers or search a min-loss subset, the PTCF scheme enables collaborative filtering on an attacked training dataset while guarantees system's availability and integrity. We evaluate extensively the PTCF scheme on a public dataset (Jester) and two real-world datasets (Movie and E-Shopping), and demonstrate that the PTCF scheme is significantly effective in providing robustness.

Blockchain technology, with its unique decentralized and tamper-resistant features, is being utilized to address the issue of information silos in traditional electronic healthcare. However, as healthcare data sources become increasingly complex and numerous, the limited scalability and transaction throughput of traditional blockchains result in challenges such as slow processing efficiency and vulnerability to attacks in modern healthcare blockchain systems. To address these issues, we propose a Dynamic Security Resource Configuration scheme based on Medical Consortium Blockchain (MC-DSC). This scheme allows for dynamic blockchain configuration based on the varying urgency levels of data, enhancing data processing efficiency. It ensures the security of the data processing process through identity control and data encryption methods. Experimental results demonstrate that, compared to existing blockchain configuration algorithms (SsHealth and Medge-Chain), the proposed scheme achieves approximately a 15% performance improvement by dynamically configuring the blockchain for three data types (secure, urgent, and normal). Additionally, the security module accounts for only 7% of the total time overhead, efficiently safeguarding the security of healthcare data while effectively handling data with different urgency levels.

Intelligent transportation systems (ITS) utilize advanced technologies to enhance traffic safety and efficiency, contributing significantly to modern transportation. The integration of Vehicle-to-Everything (V2X) further elevates road safety and fosters the progress of ITS through enabling direct vehicle communication and interaction with infrastructure. However, the penetration rate of V2X vehicles is advancing gradually. Consequently, there will be mixed scenarios on the road, involving both on-board units (OBUs)-equipped and non-equipped vehicles. This results in disparities in communication capabilities, highlighting the need to ensure the efficient and safe operation of vehicles in such mixed scenarios. This paper addresses this challenge by presenting a feasibility analysis and prediction method for lane-changing overtaking maneuvers in mixed scenarios, specifically for vehicles equipped with OBUs. This method assists vehicles in completing overtaking maneuvers by offering a non-binary lane-changing overtaking feasibility index along with corresponding speed guidance. First, vehicle sensors are used to sense the state of surrounding vehicles, addressing any missing sensor data due to occlusions. Moreover, the future driving behavior of the vehicle is taken into account to more accurately predict the future state of the vehicle. Then, a deep reinforcement learning algorithm is deployed to process the hybrid action space to train a lane-changing overtaking model, which also takes into account the influence of the flow of each lane in front of the vehicle, and finally predicts the feasibility of the vehicle performing lane-changing overtaking. Experimental results demonstrate that our method can accurately predict the vehicle’s future state and effectively assist the vehicle in completing lane-changing overtaking maneuvers. This research provides strong support for the integration of ITS and V2X technologies.

By combining edge computing and parallel computing, distributed edge computing has emerged as a new paradigm to exploit the booming IoT devices at the edge. To accelerate computation at the edge,i.e., the inference tasks for DNN-driven applications, the parallelism of both computation and communication needs to be considered for distributed edge computing, and thus, the problem of Minimum Latency joint Communication and Computation Scheduling (MLCCS) is proposed. However, existing works have rigid assumptions that the communication time of each device is fixed and the workload can be split arbitrarily small. Aiming at making the work more practical and general, the MLCCS problem without the above assumptions is studied in this paper. Firstly, the MLCCS problem under a general model is formulated and proved to be NP-hard. Secondly, a pyramid-based computing model is proposed to consider the parallelism of communication and computation jointly, which has an approximation ratio of$1+\delta$, where$\delta$is related to devices' communication rates. An interesting property under such a computing model is identified and proved,i.e., the optimal latency can be obtained under arbitrary scheduling order when all the devices share the same communication rate. When the workload cannot be split arbitrarily, an approximation algorithm with a ratio of at most$2\cdot (1+\delta )$is proposed. Additionally, for handling the dynamically changing network scenarios, several algorithms are also proposed accordingly. Finally, the theoretical analysis and simulation results verify that the proposed algorithm has high performance in terms of latency. Two testbed experiments are also conducted, which show that the proposed method outperforms the existing methods, reducing the latency by up to 29.2% for inference tasks at the edge.

Mobile Edge Computing (MEC) has been identified as a desirable computing paradigm that provides efficient and effective services for various applications, while meeting stringent service delay requirements. Orthogonal to the MEC computing paradigm, Network Function Virtualization (NFV) technology is another enabling technology that provides the network resource management with great flexibility and scalability, where the instances of Virtual Network Functions (VNFs) are deployed in edge servers as Service Function Chains (SFCs) for SFC-enabled services. Although reliable service provisioning in MEC environments is fundamentally important, the deployed VNF instances usually are not reliable, which can be affected by their software implementation, their execution duration, the workload among edge servers, and so on. Empowered by digital twin techniques, the states of VNF instances can be maintained by their digital twins in a real-time manner and their reliability can be accurately predicted through their digital twins. In this paper, we study digital twin-assisted, SFC-enabled reliable service provisioning in MEC networks by exploiting the dynamics of VNF instance reliability. We concentrate on two novel optimization problems of reliable service provisioning: the service cost minimization problem, and the dynamic service admission maximization problem. We first show their NP-hardness. We then formulate an Integer Linear Program (ILP) solution, and devise an approximation algorithm with a constant approximation ratio for the service cost minimization problem. We thirdly provide an ILP solution to the offline version of the dynamic service admission maximization problem. Built upon this offline ILP solution, we also develop an online algorithm with a provable competitive ratio for the problem, by adopting the primal-dual dynamic updating technique. We finally evaluate the performance of the proposed algorithms via simulations. Simulation results demonstrate that the proposed algorithms outperform their comparison benchmarks, and improve the performance of their comparison counterparts by no less than$10.2 \%$.

This paper investigates the implementation of Federated Learning (FL) in an over-the-air computation system with volatile clients, where each client operates under a limited energy budget and may unexpectedly drop out during local training sessions. The dropout of clients not only wastes energy but also diminishes their participation frequency, necessitating careful client selection by the server in each communication round. However, the diversity of training tasks and the random nature of client dropout present challenges such as the absence of an explicit objective function and the unavailability of client performance metrics. To address these challenges, we first analyze the convergence of the over-the-air federated learning system with volatile clients to identify the key factor influencing the model's convergence speed. Building upon this analysis, we propose an approximation of the objective function as the optimization goal for client selection. To mitigate energy waste, we introduce a dynamic client selection strategy termed DCSE, based on Exp3 with multiple plays and energy constraints, aiming to reconcile the dilemma of unknown local training states and limited resource constraints. Theoretical analysis demonstrates that our proposed solution maintains a constant bound on the difference from the optimal solution, affirming its theoretical feasibility. Furthermore, experimental results validate the effectiveness of the proposed strategy in enhancing FL by accelerating convergence speed, improving test accuracy, and reducing wasted energy.

In recent years, the Internet of Vehicles (IoV) has experienced significant growth, but the lack of effective secret key establishment remains a security concern due to the dynamic and ad-hoc nature of IoV communications. Physical layer key generation has emerged as a promising solution for establishing a pair of cryptographic keys in a lightweight and information-theoretic secure manner. However, previous works have primarily focused on legacy communication technologies, such as Wi-Fi, ZigBee, and 5 G, which are limited to short-range IoV communications. With the emergence of Long-range (LoRa) communication technology, which features long-range, low power, and extremely low data rates, new challenges arise for key generation in long-range IoV scenarios. This paper presentsVehicle-Key, a secret key generation system designed to secure LoRa-enabled IoV communications.Vehicle-Keypresents an innovative scenario adaptive deep learning model that performs channel prediction and quantization concurrently while reducing the training cost through a data augmentation pipeline and enhancing the model's generalization using a domain-adaption method. Additionally, we propose a bloom filter-assisted autoencoder-based reconciliation method to significantly improve the key agreement rate. Comprehensive real-world experiments show thatVehicle-Keysurpasses the State-of-the-Art, achieving a 15.26%–50.35% improvement in key agreement rate and a 9–15× increase in key generation rate. Moreover, the proposed method attains a 4.37--9.33% improvement when adapted to new scenarios with limited data sizes. A security analysis demonstrates thatVehicle-Keyis resilient against several common attacks. Furthermore, we implementVehicle-Keyon a Raspberry Pi and demonstrate its ability to execute within 3.5 ms.

Generative adversarial networks (GANs) have been advancing and gaining tremendous interests from both academia and industry. With the development of wireless technologies, a huge amount of data generated at the network edge provides an unprecedented opportunity to develop GANs applications. However, due to the constraints such as bandwidth, privacy, and legal issues, it is inappropriate to collect and send all data to the cloud or servers for analysis, training, and mining. Thus, deploying and training GANs at the edge becomes a promising alternative solution. The instability of GANs introduced by non-independent and identical data (Non-IID) poses significant challenges to training GANs. To address these challenges, this paper presents a novel federated learning framework for GANs, namely,Collaborated gAmeParallel Learning (CAP). CAP supports parallel training of data and models for GANs, breaking the isolated training among generators that exists in the previous distributed algorithms, and achieving collaborative learning among cloud, edge servers, and devices. Then, to further enhance the ability of CAP-GAN for addressing Non-IID issues, we propose a Mix-Generator module (Mix-G) which divides a generator into the sharing layer and personalizing layer. The Mix-G module extracts the generic and personalization features and improves the performance of CAP-GAN on extremely personalizing datasets. Experimental results and analysis substantiate the usefulness and superiority of our proposed CAP-GAN scheme which can achieve better results in the Non-IID scenarios compared with the state-of-the-art algorithms.

Vehicular Edge Computing (VEC) is the transportation version of Mobile Edge Computing (MEC) in road scenarios. One key technology of VEC is task offloading, which allows vehicles to send their computation tasks to the surrounding Roadside Units (RSUs) or other vehicles for execution, thereby reducing computation delay and energy consumption. However, the existing task offloading schemes still have various gaps and face challenges that should be addressed because vehicles with time-varying trajectories need to process massive data with high complexity and diversity. In this paper, a VEC-based computation offloading model is developed with consideration of data dependency of tasks. The minimization of the average response time and average energy consumption of the system is defined as a combinatorial optimization problem. To solve this problem, we propose aMobility-aware dependent taskoffloading (MESON) Scheme for urban VEC and develop a DRL-based algorithm to train the offloading strategy. To improve the training efficiency, a vehicle mobility detection algorithm is further designed to detect the communication time between vehicles and RSUs. In this way, MESON can avoid unreasonable decisions by lowering the size of the action space. Moreover, to improve the system stability and the offloading successful rate, we design a task priority determination scheme to prioritize the tasks in the waiting queue. The experimental results show that MESON is superior compared to other task offloading schemes in terms of the average response time, average system energy consumption, and offloading successful rate.

Multiview clustering algorithms have attracted intensive attention and achieved superior performance in various fields recently. Despite the great success of multiview clustering methods in realistic applications, we observe that most of them are difficult to apply to large-scale datasets due to their cubic complexity. Moreover, they usually use a two-stage scheme to obtain the discrete clustering labels, which inevitably causes a suboptimal solution. In light of this, an efficient and effective one-step multiview clustering (E2OMVC) method is proposed to directly obtain clustering indicators with a small-time burden. Specifically, according to the anchor graphs, the smaller similarity graph of each view is constructed, from which the low-dimensional latent features are generated to form the latent partition representation. By introducing a label discretization mechanism, the binary indicator matrix can be directly obtained from the unified partition representation which is formed by fusing all latent partition representations from different views. In addition, by coupling the fusion of all latent information and the clustering task into a joint framework, the two processes can help each other and obtain a better clustering result. Extensive experimental results demonstrate that the proposed method can achieve comparable or better performance than the state-of-the-art methods. The demo code of this work is publicly available at https://github.com/WangJun2023/EEOMVC.

We propose an innovative approach named MEC-RDESN /mek”r:dI’saIn/ (MECQoS forecasting based onRegion recognition andDynamicEchoStateNetwork) enabling mobility-aware and swift QoS forecasting in the mobile edge computing environment. MEC-RDESN offers efficient QoS forecasting while maintaining high accuracy. We can identify the edge region to which a user belongs in real time while moving by leveraging mobile sensing technology. We employ adynamic echo state networkcharacterized by multi-service adaptability to retain information about services invoked by users to ensure real-time training and forecasting accuracy. Our approach is validated through a series of experiments using both public and collected datasets. The experiments demonstrate that MEC-RDESN achieves the goal of fast forecasting while ensuring its forecasting accuracy in diverse application scenarios.

Multi-server jobs that request multiple computing resources and hold onto them during their execution dominate modern computing clusters. When allocating the multi-type resources to several co-located multi-server jobs simultaneously in online settings, it is difficult to make the tradeoff between the parallel computation gain and the internal communication overhead, apart from the resource contention between jobs. To study the computation-communication tradeoff, we model the computation gain as the speedup on the job completion time when it is executed in parallelism on multiple computing instances, and fit it with utilities of different concavities. Meanwhile, we take the dominant communication overhead as the penalty to be subtracted. To achieve a better gain-overhead tradeoff, we formulate an cumulative reward maximization program and design an online algorithm, namedOgaSched, to schedule multi-server jobs.OgaSchedallocates the multi-type resources to each arrived job in the ascending direction of the reward gradients. It has several parallel sub-procedures to accelerate its computation, which greatly reduces the complexity. We proved that it has a sublinear regret with general concave rewards. We also conduct extensive trace-driven simulations to validate the performance ofOgaSched. The results demonstrate thatOgaSchedoutperforms widely used heuristics by 11.33%, 7.75%, 13.89%, and 13.44%, respectively.

Efficient green software solutions require being aware of the characteristics of both the software and the hardware where it is executed. Separately optimizing them leads to inefficient results, and there is a need for a perfect synergy between software and hardware for optimal outcomes. We present a novel combinatorial optimization problem for the minimization of the software execution time on a specific hardware, taking into account the existing uncertainty in the system. A solution to the problem is a sequence of LLVM code transformations, and a cellular genetic algorithm is used to find it. Assuming that hardware does not change, reducing the software runtime typically leads to a greener version with lower consumption. To cope with the uncertainty, two novel approaches relying on bootstrap method to compute confident intervals of the software runtime at negligible cost are proposed and compared to three other techniques and -O3 Clang compilation flag over four hardware architectures. Results show how the proposed approach effectively copes with the uncertainty, providing more robust solutions with respect to the compared methods. The execution time of the raw program is reduced from 28.1% to up to 63.2%, outperforming -O3 flag by 13.9% to 26.3%, for the different architectures.

Traffic prediction is crucial for intelligent transportation systems, assisting in making travel decisions, minimizing traffic congestion, and improving traffic operation efficiency. Although effective, existing centralized traffic prediction methods have privacy leakage risks. Federated learning-based traffic prediction methods keep raw data local and train the global model in a distributed way, thus preserving data privacy. Nevertheless, the spatial correlations between local clients will be broken as data exchange between local clients is not allowed in federated learning, leading to missing spatial information and inferior prediction accuracy. To this end, we propose a federated graph neural network with spatial information completion (FedGCN) for privacy-preserving traffic prediction by adopting a federated learning scheme to protect confidentiality and presenting a mending graph convolutional neural network to mend the missing spatial information during capturing spatial dependency to improve prediction accuracy. To complete the missing spatial information efficiently and capture the client-specific spatial pattern, we design a personalized training scheme for the mending graph neural network, reducing communication overhead. The experiments on four public traffic datasets demonstrate that the proposed model outperforms the best baseline with a ratio of 3.82%, 1.82%, 2.13%, and 1.49% in terms of absolute mean error while preserving privacy.

Reputation systems are widely used to provide a trustworthy environment and improve the sustainability of online discussions. They help users understand and evaluate the quality of information by collecting and counting feedback from different users. However, a common issue in most reputation systems is how to maintain users’ reputation and protect their anonymity simultaneously. In this paper, we introduce a new practical anonymous reputation system based on SGX. The establishment of an anonymous reputation system has a positive effect on sustainable trust in reputation-based online applications. Our system achieves the combination of reputation and anonymity by utilizing Intel SGX and the Bloom filter. The Path ORAM algorithm is also implemented to resist side-channel attacks. The experiments demonstrate that our system achieves high performance in terms of computation and storage costs. When compared to two state-of-the-art anonymous reputation systems, our system has better computation performance with at least three orders of magnitude.

The rapid expansion of cloud infrastructure services as dominating computing paradigm across diverse applications has introduced new complexities in the allocation of on-demand cloud resources. Efficiently provisioning computing resources for various job types with distinct requirements poses a fundamental challenge for cloud infrastructure providers, who are responsible for managing data centers. In this paper, we propose a new clustering framework for large-scale workload traces from the Google cloud platform. Our approach intelligently identifies groups of similar jobs by considering key factors such as the centroid of clusters (representing CPU and memory usage deviation), fluctuations in CPU and memory usage, and the time distribution updates following each clustering step. We evaluate our proposed clustering approach by conducting empirical analysis on a substantial dataset consisting of 2.4 TB workload traces from eight different Borg cells within Google's cluster infrastructure. Through this evaluation, we derived seven key observations that shed light on the characteristics of the workload traces. These observations encompass diverse job types, resource utilization patterns, the efficiency of current resource allocation strategies, and the duration of resource usage for different job types and priorities. The insights gained from these observations can provide empirical evidence and valuable guidance for both cloud providers and researchers in designing optimal resource allocation and management strategies for large-scale cloud data centers.

Intraoperative hypotension (IOH) events warning plays a crucial role in preventing postoperative complications, such as postoperative delirium and mortality. Despite significant efforts, two fundamental problems limit its wide clinical use. The well-established IOH event warning systems are often built on proprietary medical devices that may not be available in all hospitals. The warnings are also triggered mainly through a predefined IOH event that might not be suitable for all patients. This work proposes a composite multi-attention (CMA) framework to tackle these problems by conducting short-term predictions on user-definable IOH events using vital signals in a low sampling rate with demographic characteristics. Our framework leverages a multi-modal fusion network to make four vital signals and three demographic characteristics as input modalities. For each modality, a multi-attention mechanism is used for feature extraction for better model training. Experiments on two large-scale real-world data sets show that our method can achieve up to 94.1% accuracy on IOH events early warning while the signals sampling rate is reduced by 3000 times. Our proposal CMA can achieve a mean absolute error of 4.50 mm Hg in the most challenging 15-minute mean arterial pressure prediction task and the error reduction by 42.9% compared to existing solutions.

Early diagnosis of osteonecrosis of the femoral head (ONFH) can inhibit the progression and improve femoral head preservation. The radiograph difference between early ONFH and healthy ones is not apparent to the naked eye. It is also hard to produce a large dataset to train the classification model. In this paper, we propose Asymmetric-Sensitive Transformer (AsT) to capture the uneven development of the bilateral femoral head to enable robust ONFH detection. Our ONFH detection is realized using the self-attention mechanism to femoral head regions while conferring sensitivity to the uneven development by the attention-shared transformer. The real-world experiment studies show that AsT achieves the best performance of AUC 0.9313 in the early diagnosis of ONFH and can find out misdiagnosis cases firmly.

Machine learning models are increasingly used in time series prediction with promising results. The model explanation of time series prediction falls behind the model development and makes less sense to users in understanding model decisions. This paper proposes ES-Mask, a post-hoc and model-agnostic evolutionary strip mask-based saliency approach for time series applications. ES-Mask designs the mask consisting of strips with the same salient value in consecutive time steps to produce binary and sustained feature importance scores over time for easy understanding and interpretation of time series. ES-Mask uses an evolutionary algorithm to search for the optimal mask by manipulating strips in rounds, thus is agnostic to models by involving no internal model states in the search. The initial experiments on MIMIC-III data set show that ES-Mask outperforms state-of-the-art methods.

Many real-time OLAP systems have been proposed to query evolving data with sub-second latency. Although this feature is highly attractive, it is very hard to be achieved on analytic graph queries that can only be answered after accessing every connected vertex. Fortunately, researchers recently observed that answering pairwise queries is enough for many real-world scenarios. These pairwise queries avoid the exhaustive nature and hence may only need to access a small portion of the graph. Obviously, the crux of achieving low latency is to what extent the system can eliminate unnecessary computations. This pruning process, according to our investigation, is usually achieved by estimating certain upper bounds of the query result in existing systems.

Vehicle trajectory prediction is a promising technology for improving the performance of Cellular Vehicle-to-Everything (C-V2X) applications by providing future road states. Various vehicle trajectory prediction methods have been proposed to increase the accuracy of the predicted trajectory. Although the existing vehicle trajectory prediction methods can accurately predict the future trajectory under the assumption that data comply with the Independent and Identically Distributed (IID), their performance is seriously degraded in practical implementation due to the ubiquitous distribution shifts in vehicle trajectory data. To improve the universality of the vehicle trajectory prediction method, generalizing the method to an environment that never appeared in the training data, namely, the Domain Generalization (DG) task, should be considered. Thus, we propose a plug-and-play inFORmaTive caUsality-based vehicle trajectory predictioN architecturE (FORTUNE) to improve the DG capability of vehicle trajectory prediction methods. First, a novel structural causal model (SCM) of vehicle trajectory prediction is established to simulate the causality of the data-generating process. Second, we utilize the principle of mutual information to learn the invariant representation of the SCM. Third, an invariant knowledge-transferring module is proposed to increase learning ability without destroying the structure of the original model. The results from simulation experiments demonstrate that the proposed scheme can significantly improve the DG capability of vehicle trajectory prediction methods.

Blockchain technology has attracted significant industry, academic, and governmental attention since its emerged in 2008. Blockchain use cases are now being explored by traditional, transaction-oriented businesses in the finance, insurance, logistics and healthcare sectors to name a few. This has expanded further with the widespread use of Internet of Things (IoT) devices. Massive amounts of data are generated by IoT devices and are recorded in the blockchain. While blockchain provides many advantages, such as immutability and transparency, its serialized nature makes impossible to read in a single query. Multiple requests are required even for simple tasks, such as displaying an account's transaction history. This further leads to the difficulty in understanding the data in the blockchain. In this paper, we address the problem of smart contract visualization in a real-time manner. To this end, we design a visualization dashboard for smart contracts. A visual aid for massive amounts of data helps users understand the blockchain's overall activities, uncover operational risks and provide critical intelligence by visualising unusual activities and connections. Such insights may enable the user to investigate and predict any anomalies or reveal any network vulnerabilities. Cattle farm selected as a use case because the voluminous data can be acquired from IoT sensors on the farm cattle. Our dashboard has been proven to help visualize the life cycle of animals, the distribution of activities and time factor analysis. This visualization can give a user a better perspective of the token functions and results as well as animal management issues.

Time-Slotted Channel Hopping (TSCH) is a synchronous medium access mode of the IEEE 802.15.4e standard designed for providing low-latency and highly-reliable end-to-end communication. TSCH constructs a communication schedule by combining frequency channel hopping with Time Division Multiple Access (TDMA). In recent years, IETF designed several standards to define general mechanisms for the implementation of TSCH. However, the problem of updating the TSCH schedule according to the changes of the wireless link quality and node's traffic load left unresolved. In this paper, we use non-cooperative game theory to propose GT-TSCH, a distributed TSCH scheduler designed for low-power IoT applications. By considering selfish behavior of nodes in packet forwarding, GT-TSCH updates the TSCH schedule in a distributed approach with low control overhead by monitoring the queue length, the place of the node in the Directed Acyclic Graph (DAG) topology, the quality of the wireless link, and the data packet generation rate. We prove the existence and uniqueness of Nash equilibrium in our game model and we find the optimal number of TSCH Tx timeslots to update the TSCH slotframe. To examine the performance of our contribution, we implement GT-TSCH on Zolertia Firefly IoT motes and the Contiki-NG Operating System (OS). The evaluation results reveal that GT-TSCH improves performance in terms of throughput and end-to-end delay compared to the state-of-the-art method.

By combing edge computing and parallel computing, distributed edge computing has emerged as a new paradigm to accelerate computation at the edge. Considering the parallelism of both computation and communication, the problem of Minimum Latency joint Communication and Computation Scheduling (MLCCS) is studied recently. However, existing works have rigid assumptions that the communication time of each device is fixed and the workload can be split arbitrarily small. Aiming at making the work more practical and general, the MLCCS problem without the above assumptions is studied in this paper. Firstly, the MLCCS problem under a general model is formulated and proved to be NP-hard. Secondly, a pyramid-based computing model is proposed to consider the parallelism of communication and computation jointly, which has an approximation ratio of 1 + δ, where δ is related to devices’ communication rates. An interesting property under such computing model is identified and proved, i.e., the optimal latency can be obtained under arbitrary scheduling order when all the devices share the same communication rate. When the devices own different communication rates, the optimal scheduling order is also obtained. Additionally, when the workload cannot be split arbitrarily, an approximation algorithm with ratio of at most 2 (1 + δ) is proposed. Finally, the theoretical analysis and simulation results verify that the proposed algorithm has high performance in terms of latency. Two testbed experiments are also conducted, which show that the proposed method outperforms the existing methods, reducing the latency by up to 29.2% in real-world applications.

Sharding scales throughput by splitting blockchain nodes into parallel groups. However, different shards’ independent and random scheduling for cross-shard transactions results in numerous conflicts and aborts, since cross-shard transactions from different shards may access the same account. A deterministic ordering can eliminate conflicts by determining a global order for transactions before processing, as proved in the database field. Unfortunately, due to the intertwining of the Byzantine environment and information isolation among shards, there is no trusted party able to predetermine such an order for cross-shard transactions. To tackle this challenge, this paper proposes Prophet, a conflict-free sharding blockchain based on Byzantine-tolerant deterministic ordering. It first depends on untrusted self-organizing coalitions of nodes from different shards to pre-execute cross-shard transactions for prerequisite information about ordering. It then determines a trusted global order based on stateless ordering and post-verification for pre-executed results, through shard cooperation. Following the order, the shards thus orderly execute and commit transactions without conflicts. Prophet orchestrates the pre-execution, ordering, and execution processes in the sharding consensus for minimal overhead. We rigorously prove the determinism and serializability of transactions under the Byzantine and sharded environment. An evaluation of our prototype shows that Prophet improves the throughput by 3.11× and achieves nearly no aborts on 1 million Ethereum transactions compared with state-of-the-art sharding.

The emerging digital twin technique enhances the network management efficiency and provides comprehensive insights, through mapping physical objects to their digital twins. The user satisfaction on digital twin-enabled query services relies on the freshness of digital twin data, which is measured by the Age of Information (AoI). Because the remote cloud faces challenges in providing data for users due to long service delays, Mobile Edge Computing (MEC), as a promising technology, offers real-time data communication between physical objects and their digital twins at the edge of the core network. However, the mobility of physical objects and dynamic query arrivals make efficient service provisioning in MEC become challenging. In this paper, we investigate the dynamic digital twin placement for improving user service satisfaction in MEC environments. We focus on two user service satisfaction augmentation problems under both static and dynamic digital twin placement schemes: the static and dynamic utility maximization problems. We first formulate an Integer Linear Programming (ILP) solution to the static utility maximization problem when the problem size is small; otherwise, we propose a performance- guaranteed approximation algorithm for it. We then devise an online algorithm for the dynamic utility maximization problem with a provable competitive ratio. Finally, we evaluate the performance of the proposed algorithms through experimental simulations. Simulation results demonstrate that the proposed algorithms outperform the comparison baseline algorithms, and the performance improvement is no less than 11.6%, compared with the baseline algorithms.

Non-intrusive load monitoring (NILM) is a cost-effective energy disaggregation means to estimate the energy consumption of individual appliances from a central load reading. Learning-based methods are the new trends in NILM implementations but require large labeled data to work properly at end-user premises. We first formulate an unsupervised multi-source domain adaptation problem to address this challenge by leveraging rich public datasets for building the NILM model. Then, we prove a new generalization bound for the target domain under multi-source settings. A hybrid loss-driven multi-source domain adversarial network (HLD-MDAN) is developed by approximating and optimizing the bound to tackle the domain shift between source and target domains. We conduct extensive experiments on three real-world residential energy datasets to evaluate the effectiveness of HLD-MDAN, showing that it is superior to other methods in single-source and multi-source learning scenarios.

Summary Apache Storm is a distributed processing engine that can reliably process unbounded streams of data for real‐time applications. While recent research activities mostly focused on devising a resource allocation and task scheduling algorithm to satisfy high performance or low latency requirements of Storm applications across a distributed and multi‐core system, finding a solution that can optimize the energy consumption of running applications remains an important research question to be further explored. In this article, we present a controlling strategy for CPU throttling that continuously optimize the level of consumed energy of a Storm platform by adjusting the voltage and frequency of the CPU cores while running the assigned tasks under latency constraints defined by the end‐users. The experimental results running over a Storm cluster with 4 physical nodes (total 24 cores) validates the effectiveness of proposed solution when running multiple compute‐intensive operations. In particular, the proposed controller can keep the latency of analytic tasks, in terms of 99th latency percentile, within the quality of service requirement specified by the end‐user while reducing the total energy consumption by 18% on average across the entire Storm platform.

The Internet of Things (IoT) has revolutionized the industrial field with numerous facilities and advancements. The industrial IoT system demands delay-aware workload execution with the aid of a fog computing platform, and precise resource allocation is required in fog nodes (FNs) to execute the fluctuating industrial IoT workloads with minimal cost and delay. In view of the issue mentioned above, we introduce an autonomic workload prediction and resource allocation framework that efficiently allocates resources among FNs. In the proposed framework, the workloads are predicted in the analysis phase with the guidance of the deep autoencoder (DAE) model, and the FNs are scaled based on the demand of Industrial IoT workloads. The crow search algorithm (CSA) is integrated with the framework for optimal FN selection to improve cost and delay objectives. The proposed scheme is evaluated and compared with the existing optimization models in terms of execution cost, request rejection ratio, throughput, and response time. The simulation results establish that the proposed scheme outperformed other optimization models. The method provided a suitable solution for the optimal FN placement problems in efficiently executing dynamic industrial IoT workloads.

Although the field of eXplainable Artificial Intelligence (XAI) has a significant interest these days, its implementation within cyber security applications still needs further investigation to understand its effectiveness in discovering attack surfaces and vectors. In cyber defence, especially anomaly-based Intrusion Detection Systems (IDS), the emerging applications of machine/deep learning models require the interpretation of the models' architecture and the explanation of models' prediction to examine how cyberattacks would occur. This paper proposes a novel explainable intrusion detection framework in the Internet of Things (IoT) networks. We have developed an IDS using a Short-Term Long Memory (LSTM) model to identify cyberattacks and explain the model's decisions. This uses a novel set of input features extracted by a novel SPIP (S: Shapley Additive exPlanations, P: Permutation Feature Importance, I: Individual Conditional Expectation, P: Partial Dependence Plot) framework to train and evaluate the LSTM model. The framework was validated using the NSL-KDD, UNSW-NB15 and TON_IoT datasets. The SPIP framework achieved high detection accuracy, processing time, and high interpretability of data features and model outputs compared with other peer techniques. The proposed framework has the potential to assist administrators and decision-makers in understanding complex attack behaviour.

Blockchain databases have attracted widespread attention but suffer from poor scalability due to underlying non-scalable blockchains. While blockchain sharding is necessary for a scalable blockchain database, it poses a new challenge named on-chain cross-shard database services. Each cross-shard database service (e.g., cross-shard queries or inter-shard load balancing) involves massive cross-shard data exchanges, while the existing cross-shard mechanisms need to process each cross-shard data exchange via the consensus of all nodes in the related shards (i.e., on-chain) to resist a Byzantine environment of blockchain, which eliminates sharding benefits. To tackle the challenge, this paper presents GriDB, the first scalable blockchain database, by designing a novel off-chain cross-shard mechanism for efficient cross-shard database services. Borrowing the idea of off-chain payments, GriDB delegates massive cross-shard data exchange to a few nodes, each of which is randomly picked from a different shard. Considering the Byzantine environment, the untrusted delegates cooperate to generate succinct proof for cross-shard data exchanges, while the consensus is only responsible for the low-cost proof verification. However, different from payments, the database services' verification has more requirements (e.g., completeness, correctness, freshness, and availability); thus, we introduce several new authenticated data structures (ADS). Particularly, we utilize consensus to extend the threat model and reduce the complexity of traditional accumulator-based ADS for verifiable cross-shard queries with a rich set of relational operators. Moreover, we study the necessity of inter-shard load balancing for a scalable blockchain database and design an off-chain and live approach for both efficiency and availability during balancing. An evaluation of our prototype shows the performance of GriDB in terms of scalability in workloads with queries and updates.

Edge computing has gained momentum in recent years, and can provide more immediate analysis of streaming video data. However, the edge devices often lack the computing capabilities (processing power, memory) to guarantee reasonable performance (e.g., accuracy, latency, throughput) for complex video analytics tasks. To alleviate this critical problem, the prevalent trend is to offload some video analytics tasks from the edge devices to the cloud. However, existing offloading approaches fail to consider the dynamic nature of the video analytical tasks (e.g., varying encoding format for different video content) and are unable to adapt system dynamics (e.g., varying workload between the edge and the cloud). To overcome the limitation of existing approaches, we develop an edge-cloud offloading performance model based on the concept of hierarchical queues. The resource constraints (e.g., computing capacity and network bandwidth) of each edge nodes and dynamic edge-cloud network conditions are used to parameterize the performance model. Since finding optimal solutions for the performance model is NP-hard, we develop a two-stage gradient-based algorithm and compare it with some state-of-the-art (SOTA) solutions (e.g., FastVA, DeepDecision, Hill Climbing). Experiments have shown our performance model's advantages and the stability of the proposed offloading approach given different systems (edge-cloud) and video analytics application dynamics.

Federated Distillation (FD) extends classic Federated Learning (FL) to a more general training framework that enables model-heterogeneous collaborative learning by Knowledge Distillation (KD) across multiple clients and the server. However, existing KD-based algorithms usually require a set of shared input samples for each client to produce soft-prediction for distillation. Worse still, such a manual selection is accompanied by careful deliberations or prior information on clients’ private data distribution, which is not in line with the privacy-preserving characteristic of classic FL. In this paper, we propose a novel training framework to achieve data-independent knowledge transfer by properly designing a distributed generative adversarial network (GAN) between the server and clients that can synthesize shared feature representations to facilitate the FD training. Specifically, we deploy a generator on the server and reuse each local model as a federated discriminator to form a lightweight efficient distributed GAN that can automatically synthesize simulated global feature representations for distillation. Moreover, since the synthesized feature representations are usually more faithful and homologous with global data distribution, faster and better training convergence can be obtained. Extensive experiments on different tasks and heterogeneous models demonstrate the effectiveness of the proposed framework on model accuracy and communication overhead.

BCube is a well-known recursively defined network structure. It provides multiple low-diameter paths and good fault-tolerance for data center networks (DCNs). Its distributed routing algorithm, BCube Source Routing (BSR), can be deployed rapidly and conveniently to build multiple parallel path sets. But in the worst case, BSR may suffer from flow collisions and waste$50\%$of the capacity of each BCube link. In this paper, to decrease collisions and improve bandwidth utilization, we supplement the BCube topology with a central master computer and design two centralized dynamic parallel flow scheduling algorithms: CDPFS and CDPFSMP, for single-path and multi-path respectively. We focus on finding the least congested path for each flow by analyzing the information about the state of the global network. Furthermore, we allocate those paths to each flow in parallel. The simulation result shows that our proposed algorithms take advantage of BCube structure and deliver high-performance solutions for load balancing problems, which have improved 44.1% of the throughput in random bijective traffic pattern and 36.2% of throughput in data shuffle compared with BSR algorithm.

With the rapid surge in cloud services, cloud load balancing has become a paramount research issue. The major part of a cloud computing system's operational costs is attributed to energy consumption. Therefore, to provide better QoS, considering the energy minimization factor in load balancing is essential. This paper addresses the latency and energy-aware load balancing problem in a cloud computing system. Specifically, two fundamental performance criteria–response time and energy–for the load balancing problem are considered. To solve this problem, first, the load balancing problem is formulated as an optimization problem. Then it is modeled as a cooperative game so that the solution of the game, called the Nash bargaining solution (NBS), can simultaneously optimize both criteria. The existence and computation of NBS are analyzed theoretically, and an efficient algorithm, called${{\sf L}}$atency and${{\sf E}}$nergy a${{\sf W}}$are load balancIng${{\sf S}}$cheme (${{\sf LEWIS}}$), is proposed to compute the NBS. Further, to assess the efficacy of${{\sf LEWIS}}$, it is compared with three other approaches, i.e.,${\mathsf {Coop\_{RT}}}$,${\mathsf {Coop\_{EN}}}$, and${\mathsf {NCG}}$, on problem instances of various settings. The experimental results show that${{\sf LEWIS}}$not only provides less response time while consuming less energy but also gauntness fairness to the end-users.

Distributed cloud/edge resources are needed to execute pervasive artificial intelligence tasks, collectively. The AI workload and data sets have variable multitasking granularity, privacy constraints, and communication latency concerns. This article presents a novelfederated cloud/edge(FCE)framework, illustrated by distributed medical image processing across multiple hospital sites. This federated cloud system appeals to train many machine learning models efficiently with workload balancing and reduced communication overheads. We tested the FCE model on a multi-cloud platform recently built at the Chinese University of Hong Kong in Shenzhen. We claim three distinct advantages in using the FCE system. First, our federated cloud system results in 41.3% reduction in total AI processing time in large-scale ML/DL experiments. Second, high machine model accuracy was achieved at 87% level in telemedicine experiments. The virtual graph helps reduce internode traffic latencies to avoid ML inference slowdowns. Third, the system can tolerate multiple cloud failures to enter a graceful degradation mode in case of node failures. The scalable performance gains in AI processing speed, model accuracy, and fault tolerance make our federated clouds a truly viable approach to solving massive AI multitasking problems in pervasive AI applications.

With the expansion scale of interconnected power systems and refined state perception, scientific power calculations become more complex and diverse. They need faster computation speed and better scalability to support power flow calculation, reactive power optimization, and static/transient stability analysis for unit scheduling. Therefore, this paper proposes a novel cloud data center task mapping algorithm of the Stoer-Wagner binary tree (SWBT) to support accelerated executions of these calculations. Firstly, based on the block bordered-diagonal form of the admittance matrix, high-time complexity scientific power calculations are transformed into a unified multi-task decomposition-coordination directed acyclic graph (DC-DAG). And then, the critical tasks in this DC-DAG are found and the virtual machines encapsulating them are matched with physical machines in the data center preferentially. Finally, on CloudSim, a cloud computing platform, the multi-job mixed experiments of 118-13659 bus power systems are carried out. In addition, real-time workload performance is enhanced in two very large real-world power systems. Studies illustrate that SWBT can improve the underlying physical machine resource utilization and reduce data interaction transmission hops to achieve better computing acceleration performance.

Hyperspectral band selection aims to identify an optimal subset of bands for hyperspectral images (HSIs). For most existing clustering-based band selection methods, they directly stretch each band into a single feature vector and employ the pixelwise features to address band redundancy. In this way, they do not take full consideration of the spatial information and deal with the importance of different regions in HSIs, which leads to a nonoptimal selection. To address these issues, a region-aware hierarchical latent feature representation learning-guided clustering (HLFC) method is proposed. Specifically, in order to fully preserve the spatial information of HSIs, the superpixel segmentation algorithm is adopted to segment HSIs into multiple regions first. For each segmented region, the similarity graph is constructed to reflect the bands-wise similarity, and its corresponding Laplacian matrix is generated for learning low-dimensional latent features in a hierarchical way. All latent features are then fused to form a unified feature representation of HSIs. Finally, k -means clustering is utilized on the unified feature representation matrix to generate multiple clusters from which the band with maximum information entropy is selected to form the final subset of bands. Extensive experimental results demonstrate that the proposed clustering method can achieve superior performance than the state-of-the-art representative methods on the band selection. The demo code of this work is publicly available at https://github.com/WangJun2023/HLFC.

Remote sensing change detection (RSCD) aims to explore surface changes from co-registered pair of images. However, the high cost of memory and computation in previous convolutional neural network (CNN)-based methods prevent their successes from being applied to real-world applications. Therefore, we propose a novel lightweight network, which identifies changes based on the features extracted by mobile networks via progressive feature aggregation and supervised attention, termed as A2Net. Considering the less powerful representation capability of mobile networks, we design a neighbor aggregation module (NAM) to fuse features within nearby stages of the backbone to strengthen the representation capability of temporal features. Then, we propose a progressive change identifying module (PCIM) to extract temporal difference information from bitemporal features. Besides, we design a supervised attention module (SAM) to reweight features for effectively aggregating multilevel features from high levels to low levels. With NAM, PCIM, and SAM incorporated, A2Net can achieve favorable results compared with the state-of-the-art methods on three challenging RSCD datasets with fewer parameters (3.78 M) and lower computation costs (6.02 G). The demo code of this work is publicly available athttps://github.com/guanyuezhen/A2Net.

Greedy routing efficiently achieves routing solutions for vehicular networks due to its simplicity and reliability. However, the existing greedy routing algorithms have mainly considered simple routing metrics only, e.g., distance based on the local view of an individual vehicle. This consideration is insufficient for analysing dynamic and complicated vehicular communication scenarios which inevitably degrades the overall routing performance. Software-Defined Vehicular Network (SDVN) and Graph Convolutional Network (GCN) can overcome these limitations. Thus, this paper presents a novel GCN-based greedy routing algorithm (NGGRA) in the hybrid SDVN. The SDVN control plane trains the GCN decision model based on the globally collected data. Vehicles with transmission requirements can adopt this model for inferring and making the routing decisions. The proposed node-importance-based graph convolutional network (NiGCN) model analyses multiple correlated metrics to accurately evaluate the dynamic vehicular network is available at:https://github.com/a824899245/NiGCN. Meanwhile, the SDVN architecture offers a global view for model training and routing computation. Extensive simulation results demonstrate that NiGCN outperforms popular GCN models in training efficiency and accuracy. In addition, NGGRA can improve the packet delivery ratio and substantially reduce delay compared with its counterparts.

Collaborative machine learning, especially Federated Learning (FL), is widely used to build high-quality Machine Learning (ML) models in the Internet of Vehicles (IoV). In this paper, we study the performance evaluation problem in an inherently heterogeneous IoV, where the final models across the network are not identical and are computed on different standards. Previous studies assume that local agents are receiving data from the same phenomenon, and a same final model is fitted to them. However, this “one model fits all” approach leads to a biased performance evaluation of individual agents. We propose a general approach to measure the performance of individual agents, where the common knowledge and correlation between different agents are explored. Experimental results indicate that our evaluation scheme is efficient in these settings.

Multivariate time series data (Mv-TSD) portray the evolving processes of the system(s) under examination in a “multi-view” manner. Factorization methods are salient for Mv-TSD analysis with the potentials of structural feature construction correlating various data attributes. However, research challenges remain in the derivation of factors due to highly scattered data distribution of Mv-TSD and intensive interferences/outliers embedded in the source data. The proposed Enhanced Bayesian Factorization approach (Enhanced-BF) addresses the challenges in three phases: (1) variant scale partitioning applies to Mv-TSD according to degree of amplitude and obtains the blocks of variant scales; (2) hierarchical Bayesian model for tensor factorization automatically derives the factors of each block with interferences suppressed; (3) Bayesian unification model merges those block factors to construct the final structural features.Enhanced-BFhas been evaluated using a case study of brain data engineering with multivariate electroencephalogram (EEG). Experimental results indicate that the proposed method manifests robustness to the interferences and outperforms the counterparts in terms of operation efficiency and error when factorizing EEG tensor. Besides,Enhanced-BFexcels in factorization-based analysis of ongoing autism spectrum disorder (ASD) EEG: 3 times speed-up in factorization and$87.35\%$accuracy in ASD discrimination. The latent factors (“biomarkers”) can distinctly interpret the typical EEG characteristics of ASD subjects.

Applications deployed on edge servers improve users’ experience, when compared to deployments on cloud servers. Existing works usually assume that a central scheduler helps in making decisions, but they are often inefficient, inaccurate, or time-consuming. In this paper, we present a proactive application deployment system, which consists of three modules (i.e., incentive, profit, and latency). Based on the architecture of a fully distributed edge network, our system includes SELL, a Spontaneous Edge depLoyment aLgorithm in the incentive module. SELL lets edge servers compete with each other in a two-stage Stackelberg game to win deployment rights, and the winners get paid for their deployment efforts. The other two modules help recursively adjust service prices and deployment intentions in view of their own profits. Simulations on the proactive edge application deployment system demonstrate that SELL can help an application provider find appropriate edge servers to deploy applications while maximizing the profits for both parties in a low latency.

Mobile Edge Computing (MEC) promises to provide mobile users with delay-sensitive services at the edge of network, and each user service request usually is associated with a Service Function Chain (SFC) requirement that consists of Virtualized Network Functions (VNFs) in order. The satisfaction of a user on his requested service is heavily impacted by the service reliability. In this paper, we study user satisfaction on services provided by an MEC network through introducing a submodular function based metric to measure user satisfaction. We first formulate a novel user satisfaction problem with the aim to maximize the accumulative user satisfaction, assuming that all available computing resource in the MEC network can be used for service reliability enhancement. We show that the problem is NP-hard, and devise an approximation algorithm with a provable approximation ratio for it. We then consider the problem under a given computing resource budget constraint, for which we devise an approximation algorithm with a provable approximation ratio, at the expense of moderate budget violations. We finally evaluate the performance of the proposed algorithms through experimental simulations. Simulation results demonstrate that the proposed algorithms outperform the comparison baseline algorithms, improving the performance by more$16.1\%$in comparison with the baseline algorithms.

Lip reading can help people with speech disorders to communicate with others and provide them with a new channel to interact with the world. In this paper, we design and implementHearMe, an accurate and real-time lip-reading system built on commercial RFID devices. HearMe can be used to accurately recognize different words in a pre-defined vocabulary without limitations in light conditions and can be used in multiple user scenarios by leveraging RFID's ability in identifying different users. We design an effective data collection strategy to well capture the tiny and complex signal patterns caused by mouth motion and propose a set of algorithms to extract signal profiles related to mouth motions and mitigate interference factors like multi-path. A carefully designed set of features, including time-domain statistical features and frequency-domain features, are then extracted from the signal to lift the recognition accuracy at the word level. To reduce training costs when the model is used in a new environment, a transfer-learning-based approach is adopted to enhance the robustness of the model in cross-environment scenarios. Experimental results show that HearMe detects speaking actions of the user with an accuracy higher than 0.95 and recognizes different words in a 20-words vocabulary with an average accuracy higher than 0.88. Moreover, the latency of HearMe ($\sim$150ms) is nearly two orders of magnitude less than traditional approaches, making it applicable to practical scenarios that require real-time lip reading.

The surging of deep learning brings new vigor and vitality to shape the prospect of intelligent Internet of Things (IoT), and the rise of edge intelligence enables provisioning real-time deep neural network (DNN) inference services for mobile users. To perform efficient and effective DNN model training in edge computing environments while preserving training data security and privacy of IoT devices, federated learning has been envisioned as an ideal learning paradigm for this purpose. In this article, we study energy-aware DNN model training in edge computing. We first formulate a novel energy-aware, Device-to-Device (D2D) assisted federated learning problem with the aim to minimize the global loss of a training DNN model, subject to bandwidth capacity on an edge server and energy capacity on each IoT device. We then devise a near-optimal learning algorithm for the problem when the training data follows the i.i.d. data distribution. The crux of the proposed algorithm is to explore using the energy of neighboring devices of each device for its local model uploading, by reducing the problem to a series of weighted maximum matching problems in corresponding auxiliary graphs. We also consider the problem without the assumption of the i.i.d. data distribution, for which we propose an efficient heuristic algorithm. We finally evaluate the performance of the proposed algorithms through experimental simulations. Experimental results show that the proposed algorithms are promising.

In this article, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of$\mathcal {O}(\frac{1}{T})$. In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum. We also propose HierOPT, which optimizes the aggregation periods (worker-edge and edge-cloud aggregation periods) to minimize the loss given a limited training time. By conducting the experiment, we verify that HierMo outperforms existing mainstream benchmarks under a wide range of settings. In addition, HierOPT can achieve a near-optimal performance when we test HierMo under different aggregation periods.

Dependency-aware jobs, such as the big data analytic workflows, are commonly executed on the cloud. They are compiled to directed acyclic graphs, with tasks linked in regarding the dependency. The cloud scheduler, which maintains a large number of resources, is responsible to execute tasks in parallel. To resolve the complex dependencies, Deep Reinforcement Learning (DRL) based schedulers are widely applied. However, we find that the DRL-based schedulers are vulnerable to the perturbations in the input jobs and may generate falsified decisions to benefit a particular job while delaying the others. By perturbation, we mean a slight adjustment to the job's node features or dependencies, while not changing its functionality. In this paper, we first explore the vulnerability of DRL-based schedulers to job perturbations without accessing the information of the DRL models used in the scheduler. We devise the black-box perturbation system, in which, a proxy model is trained to mimic the DRL-based scheduling policy. We show that the high-faith proxy model can help to craft effective perturbations. The DRL-based schedulers can be as high as 60% likely to be badly affected by the perturbations. Then, we investigate the solution to improve the robustness of DRL-based schedulers to such perturbations. We propose an adversarial training framework to force the neural model to adapt to the perturbation patterns during training so as to eliminate the potential damage during applications. Experiments show that the adversarial-trained scheduler is more robust, reducing the chance of being affected to 3-fold less and the potential bad effects halved.

Multi-server jobs are imperative in modern cloud computing systems. A noteworthy feature of multi-server jobs is that, they usually request multiple computing devices simultaneously for their execution. How to schedule multi-server jobs online with a high system efficiency is a topic of great concern. First, the scheduling decisions have to satisfy the service locality constraints. Second, the scheduling decisions needs to be made online without the knowledge of future job arrivals. Third, and most importantly, the actual service rate experienced by a job is usually in fluctuation because of the dynamic voltage and frequency scaling (DVFS) and power oversubscription techniques when multiple types of jobs co-locate. A majority of online algorithms with theoretical performance guarantees are proposed. However, most of them require the processing speeds to be knowable, thereby the job completion times can be exactly calculated. To present a theoretically guaranteed online scheduling algorithm for multi-server jobs without knowing actual processing speeds apriori, in this article, we proposeEsdp(Efficient Sampling-based Dynamic Programming), which learns the distribution of the fluctuated processing speeds over time and simultaneously seeks to maximize the cumulative overall utility. The cumulative overall utility is formulated as the sum of the utilities of successfully serving each multi-server job minus the penalty on the operating, maintaining, and energy cost.Esdpis proved to have a polynomial complexity and a logarithmic regret, which is a State-of-the-Art result. We also validate it with extensive simulations and the results show that the proposed algorithm outperforms several benchmark policies with improvements by up to 73%, 36%, and 28%, respectively.

The paradigm of edge computing has formed an innovative scope within the domain of the Internet of Things (IoT) through expanding the services of the cloud to the network edge to design distributed architectures and securely enhance decision-making applications. Due to the heterogeneous, distributed and resource-constrained essence of edge Computing, edge applications are required to be developed as a set of lightweight and interdependent modules. As this concept aligns with the objectives of microservice architecture, effective implementation of microservices-based edge applications within IoT networks has the prospective of fully leveraging edge nodes capabilities. Deploying microservices at IoT edge faces plenty of challenges associated with security and privacy. Advances in Artificial Intelligence (AI) (especially Machine Learning), and the easy access to resources with powerful computing providing opportunities for deriving precise models and developing different intelligent applications at the edge of network. In this study, an extensive survey is presented for securing edge computing-based AI Microservices to elucidate the challenges of IoT management and enable secure decision-making systems at the edge. We present recent research studies on edge AI and microservices orchestration and highlight key requirements as well as challenges of securing Microservices at IoT edge. We also propose a Microservices-based edge computing framework that provides secure edge AI algorithms as Microservices utilizing the containerization technology to offer automated and secure AI-based applications at the network edge.

Cloud services are experiencing a remarkable increase in the number of users and the resource required over the past few years. Thus, it has become a great challenge for the internet vendors to make a robust framework to serve the customers with low cost and delay. Congestion control is one of the essential topics of routing algorithms in cloud data center networks. In this paper, we propose a weighted optimal scheduling scheme WSPR for congestion control in cloud data center networks which prevents the congestion in advance with the global view so that it can make good use of vacant network resources. We choose BCube as our network model and modify the network topology to fit software-defined networks so as to have a full view of the topology. First, we design the SP graph which contains all shortest paths between a source server and a destination server. Second, we propose WSPR to allocate the most appropriate path to each flow for congestion control. We implement a system to simulate a data center, and evaluate our proposed scheme WSPR by comparing WSPR with other classical methods. The experimental results demonstrate that our proposed scheme WSPR has the best performance in terms of the maximum delay, average delay, and throughput among all compared methods.

This article focuses on a simple, yet fundamental question of distributed edge computing: “how to handle IoT traffic with different levels of sensitivity and criticality by satisfying the application-specific latency constraints?” This question arises in the practical deployment of edge computing, where user data can arrive at a much faster rate than that they can be processed by an edge node. Addressing this question is critical for meeting the latency requirement for latency-sensitive applications, but existing approaches are inadequate to the problem. We presentJanus, a multi-level traffic scheduling system for managing multiple data streams with various degrees of latency constraints. At the edge node level,Janususes multi-level queues to manage data streams with different latency constraints. It then allocates the output bandwidth of the edge node according to the requirements of applications in different priority queues, aiming to reduce the queuing and processing delay of latency-sensitive streams while maximizing the edge-node throughput. At the network level,Janusactively redirects incoming data streams to the less-loaded ones to achieve better network-wide load balance and improve the overall throughput. Experiments show thatJanusreduces the latency to only 16.6% of a non-priority based solution and improves the throughput by 1.7x of a state-of-the-art priority-aware data stream scheduling approach.

A 6-hour early detection of sepsis leads to a significant increase in the chance of surviving it. Previous sepsis early detection studies have focused on improving the performance of supervised learning algorithms while ignoring the potential correlation in data mining, and there was no reliable method to deal with the problem of incomplete data. In this paper, we proposed the Denoising Transformer AutoEncoder (DTAE) for the first time combining transformer and unsupervised learning. DTAE can learn the correlation of the features required for early detection of sepsis without the label. This method can effectively solve the problems of data sparsity and noise and discover the potential correlation of features by adding DTAE enhancement module without modifying the existing algorithms. Finally, the experimental results show that the proposed method improves the existing algorithms and achieves the best results of early detection.

Recent years have witnessed the remarkable growth of the Internet of Vehicles (IoV). Due to the high dynamics and ad-hoc nature of IoV communication, the lack of effective secret key establishment in IoV remains a security bottleneck. Physical layer key generation has emerged as a promising technology to establish a pair of cryptographic keys in a lightweight and information-theoretic secure way. However, prior works mainly focus on legacy communication technologies such as Wi-Fi, ZigBee, and 5G which can only achieve short range IoV communications. The emergence of Long-range (LoRa) communication technology that features long-range, low power, and extremely low data rate, brings new challenges for key generation in long range IoV scenarios. In this paper, we present Vehicle-Key, which is a secret key generation system to secure LoRa-enabled IoV communications. In Vehicle-Key, we design a novel deep learning model that can achieve channel prediction and quantization simultaneously. Additionally, we propose an autoencoder-based reconciliation method that improves the key agreement rate significantly. Extensive real-world experiments show that Vehicle-Key improves the key agreement rate by 15.10%–49.81% and key generation rate by 9–14× compared with the state-of-the-art. Security analysis demonstrates that Vehicle-Key is secure against several common attacks. Moreover, we implement Vehicle-Key on a Raspberry Pi and show that it can be executed in 3.4 ms.

Canonical Polyadic Decomposition (CPD) of sparse tensors is an effective tool in various machine learning and data analytics applications, in which sparse Matricized Tensor Times Khatri-Rao Product (MTTKRP) is the major performance bottleneck. To overcome this bottleneck and support efficient applications, this paper presents HPSpTM, an efficient sparse MTTKRP framework, to exploit the multi-level parallelism and reusability on heterogeneous HPC systems. HPSpTM incorporates: (1) a multi-level matrix-driven tiling engine that leverages the process- and thread-level parallelism of the underlying platform and data reusability based on the derived factor matrix-driven MTTKRP algorithm; (2) a tensor-driven parallel execution that enables buffering-aware scheduling and pipeline scheduling to optimize the performance in the tile granularity; (3) a partition-aware light weight data storage that exploits better data locality based on the proposed hierarchical and fine-grained execution; and (4) a performance auto-tuning technique that offers large flexibility for tile size auto-adjusting across various input datasets based on a designed runtime model. Our experiments show that HPSpTM on a Nvidia Tesla P100 obtains the average performance improvement of up to 76.46% over the state-of-the-arts, and HPSpTM achieves the speedup of up to 15.39× when scaling from 8 to 128 core groups, corresponding to processes, on the Sunway TaihuLight supercomputer.

We study the multi-machine task scheduling problem in an integrated serverless edge and cloud computing system, where tasks can be scheduled locally on edge processors or offloaded to cloud servers, with the objective of minimizing the makespan, i.e., the total time to finish all tasks. The system is semi-online, where the edge processing delays of the tasks are known as priori, but the cloud processing delays remain unknown due to the uncertainty introduced by uploading and loading delay (loading the software environment). The problem is NP-hard in nature, and therefore we resort to approximation schemes and propose a novel algorithm named multi-machine with restart scheduling (MRS). MRS utilizes task restart, where a task that is cancelled will be restarted later when its processing time exceeds the threshold, and the threshold can be adaptively adjusted. We derive an competitive ratio for MRS so that its worst-case gap from the optimal solution is bounded. We also implement the MRS scheduler in a real-world system, which schedules a diverse set of Deep Neural Network (DNN) inference tasks. It shows that MRS achieves significant reduction in makespan compared to existing benchmark schemes.

The surging of deep learning brings new vigor and vitality to shape the prospect of intelligent Internet of Things (IoT), and edge intelligence arises to provision real-time deep neural network (DNN) inference services for mobile users. To perform efficient and effective DNN model training in edge environments while preserving training data security and privacy of IoT devices, federated learning has been envisioned as an ideal learning paradigm for this purpose. In this paper we study energy-aware DNN model training in an edge environment. We first formulate a novel energy-aware, device-to-device (D2D) assisted federated learning problem with the aim to minimize the global loss of a training DNN model, subject to bandwidth capacity on an edge server and the energy capacity on each IoT device. We then devise an efficient heuristic algorithm for the problem. The crux of the proposed algorithm is to explore the energy usage of neighboring devices of each device for its local model uploading, by reducing the problem to a series of maximum weight matching problems in corresponding auxiliary graphs. We finally evaluate the performance of the proposed algorithm through experimental simulations. Experimental results show that the proposed algorithm is promising.

360° videos have become popular creating an immersive streaming experience for the user, nevertheless, these videos demand high bandwidth in operational networks and have strict latency requirements. Viewport (VP) aware streaming with variable tiling has been proposed as a promising solution to reduce bandwidth consumption while providing fine granularity to the user VP. Content caching at the edge has also been proposed to reduce the delivery latency. Though, combining these two mechanisms have potential advantages, applying conventional tile-based caching, which primarily tries to find identical tiles, is not feasible due to the high diversity in tile area and location in variable tiles. To this end, we propose OpCASH, an ILP based mechanism to devise optimal cache tile configuration at a MEC server to provide a non-overlapping tile cover for a given VP request in variable tiles, while minimizing the requests to remote servers and reducing the delivery latency. Experimental trace-driven simulation results show that we can achieve more than 95% of VP coverage from cache after just 24 views of the video. Compared to a baseline which represents conventional tile-based caching, OpCASH reduces the data fetched from the content servers by 85% and total content delivery time by 74%.

The recent revival in learning theory gives us improved capabilities for accurate predictions and increased opportunities for performance enhancement. This work extends the research agenda of augmenting algorithms with predictions to one of the central scheduling problems – soft real-time scheduling on single and parallel machines to minimize the mean response time. We design an algorithm, PEDRMLF (Predictions Enhanced Dynamic Randomized MultiLevel Feedback), that incorporates job size predictions, achieving an optimal competitive ratio under perfect predictions and the best-known competitive ratio under any predictions. PEDRMLF is the first algorithm that simultaneously achieves optimal consistency and bounded robustness. Simulations show that the proposed algorithm performs close to the theoretically optimal bound while consistently outperforming state-of-the-art benchmarks.

We consider the problem of non-clairvoyant scheduling on single machine to minimize the total flow time with job size predictions. The existing algorithm achieves 2-consistency to predictions, but no algorithm can simultaneously attain bounded robustness. This work finds a sufficient condition for any algorithm to achieve optimal O(P)-robustness, where P is the maximum ratio of any two job sizes. We give the first algorithm that achieves optimal robustness up to a constant multiplicative factor and optimal consistency using this condition. Finally, for addressing small prediction errors, we present an algorithm that we conjecture to achieve the optimal O(η^2) competitive ratio, where η is the prediction error. Proving the claimed bound is our ongoing work.

Abstract In a virtualized computer system with shared resources, consolidated virtual services (VSs) fiercely compete with each other to obtain the required capacity of resources, and this causes significant system's performance degradation. The performance of input output (I/O)‐bound applications running inside their own VS is mainly determined by the total time required to schedule every read/write request, plus the actual time needed by the device driver to complete the request. To achieve a right performance isolation of shared resources (e.g., the last level cache, memory bandwidth, and the disk buffer), it is essential to limit the performance degradation level among collocated applications, as simultaneously several I/O operations are requested by VSs, perhaps with different priorities. This article proposes a resource allocation controller that uses a fully polynomial‐time randomized approximation scheme to enable performance isolation of concurrent I/O requests in a shared system with multiple consolidated VSs. This controller uses a Monte Carlo sampling approach to measure and estimate the unknown attributes of operational requests originating from each VS. This is formalized as an optimization problem with the aim to minimize the degree of total quality of service (QoS) violation incidents in the entire platform. We associated a reward function to every working machine that represents the fulfillment degree of quality of service metric among all running VSs. The conducted comprehensive set of experiments showed that the proposed algorithm can reduce the QoS violation incidents by 32%, compared with the result which is obtained by employing the default resource allocation policy embedded in the existing Linux container layer.

With astonishing speed, bandwidth, and scale, mobile-edge computing (MEC) has played an increasingly important role in the next generation of connectivity and service delivery. Yet, along with the massive deployment of MEC servers, the ensuing energy issue is now on an increasingly urgent agenda. In the current context, the large-scale deployment of renewable-energy-supplied MEC servers is perhaps the most promising solution for the incoming energy issue. Nonetheless, as a result of the intermittent nature of their power sources, these special design MEC servers must be more cautious about their energy usage, in a bid to maintain their service sustainability as well as service standard. Targeting optimization on a single-server MEC scenario, we, in this article, propose neural network-based adaptive frequency adjustment (NAFA), an adaptive processor frequency adjustment solution, to enable an effective plan of the server’s energy usage. By learning from the historical data revealing request arrival and energy harvest pattern, the deep reinforcement learning-based solution is capable of making intelligent schedules on the server’s processor frequency, so as to strike a good balance between service sustainability and service quality. The superior performance of NAFA is substantiated by real-data-based experiments, wherein NAFA demonstrates up to 20% increase in the average request acceptance ratio and up to 50% reduction in average request processing time.

Federated learning (FL), arising as a privacy-preserving machine learning paradigm, has received notable attention from the public. In each round of synchronous FL training, only a fraction of available clients are chosen to participate, and the selection decision might have a significant effect on the training efficiency, as well as the final model performance. In this article, we investigate the client selection problem under a volatile context, in which the local training of heterogeneous clients is likely to fail due to various kinds of reasons and in different levels of frequency. Intuitively, too much training failure might potentially reduce the training efficiency, while too much selection on clients with greater stability might introduce bias, thereby resulting in degradation of the training effectiveness. To tackle this tradeoff, we, in this article, formulate the client selection problem under joint consideration of effective participation and fairness. Furthermore, we propose E3CS, a stochastic client selection scheme as a solution. According to our experimental results over a public data set, the proposed selection scheme is able to achieve up to$2\times $faster convergence to a fixed model accuracy while maintaining the same level of final model accuracy, compared with the state-of-the-art selection schemes.

Industrial Internet of Things (IIoT) and Industry 4.0 empower interrelation among manufacturing processes, industrial machines, and utility services. The time-critical data collected from heterogeneous sensing devices are usually communicated to processing points for analysis and aggregation as the basis of IIoT. The IIoTs’ service quality typically depends on data integrity and accuracy, which could be exploited by injecting malicious events, such as false data injection and data poisoning attacks. Thus, effective anomaly recognition and explanation are critical for ensuring quality services and empowering security administrators to interpret the causal reasoning of prediction decisions and underlying data evidence. This study proposes an autoencoder-based detection framework using convolutional and recurrent networks to discover cyber threats in IIoT networks and explain the model. A two-step sliding window (SW) is applied to learn the latent representations of data features better. Malicious points from the raw time series are transformed into fixed-length series through the first-step SW. Every series is converted into continuous-time-reliant subseries via another smaller SW to learn latent representations of malicious events. Fully connected networks use the extracted temporal and spatial features for the classification and explanation of attack events. The empirical results revealed that this framework effectively extracts features that include contexts of malicious patterns. This demonstrated that the proposed framework is robust in detecting malicious events using multiple evaluation metrics and outperforming the contemporary state-of-the-art methods, indicating its suitability as an operative application method in real-world IIoT-based networks.

COVID-19 has spread rapidly across the globe and become a deadly pandemic. Recently, many artificial intelligence-based approaches have been used for COVID-19 detection, but they often require public data sharing with cloud datacentres and thus remain privacy concerns. This paper proposes a new federated learning scheme, called FedGAN, to generate realistic COVID-19 images for facilitating privacy-enhanced COVID-19 detection with generative adversarial networks (GANs) in edge cloud computing. Particularly, we first propose a GAN where a discriminator and a generator based on convolutional neural networks (CNNs) at each edge-based medical institution alternatively are trained to mimic the real COVID-19 data distribution. Then, we propose a new federated learning solution which allows local GANs to collaborate and exchange learned parameters with a cloud server, aiming to enrich the global GAN model for generating realistic COVID-19 images without the need for sharing actual data. To enhance the privacy in federated COVID-19 data analytics, we integrate a differential privacy solution at each hospital institution. Moreover, we propose a new blockchain-based FedGAN framework for secure COVID-19 data analytics, by decentralizing the FL process with a new mining solution for low running latency. Simulations results demonstrate the superiority of our approach for COVID-19 detection over the state-of-the-art schemes.

Smart space has emerged as a new paradigm that combines sensing, communication, and artificial intelligence technologies to offer various customized services. A fundamental requirement of these services is person identification. Although a variety of person-identification approaches has been proposed, they suffer from several limitations in practical applications, such as low energy efficiency, accuracy degradation, and privacy issue. This article proposes an energy-harvesting-based privacy-preserving gait recognition scheme for smart space, which is named PrivGait. In PrivGait, we extract discriminative features from 1-D gait signal and design an attention-based long short-term memory (LSTM) network to classify different people. Moreover, we leverage a novel Bloom filter-based privacy-preserving technique to address the privacy leakage problem. To demonstrate the feasibility of PrivGait, we design a proof-of-concept prototype using off-the-shelf energy-harvesting hardware. Extensive evaluation results show that the proposed scheme outperforms state of the art by 6%–10% and incurs low system cost while preserving user’s privacy.

Albeit great success has been achieved in image defocus blur detection, there are still several unsolved challenges, e.g., interference of background clutter, scale sensitivity and missing boundary details of blur regions. To deal with these issues, we propose a deep neural network which recurrently fuses and refines multi-scale deep features (DeFusionNet) for defocus blur detection. We first fuse the features from different layers of FCN as shallow features and semantic features, respectively. Then, the fused shallow features are propagated to deep layers for refining the details of detected defocus blur regions, and the fused semantic features are propagated to shallow layers to assist in better locating blur regions. The fusion and refinement are carried out recurrently. In order to narrow the gap between low-level and high-level features, we embed a feature adaptation module before feature propagating to exploit the complementary information as well as reduce the contradictory response of different feature layers. Since different feature channels are with different extents of discrimination for detecting blur regions, we design a channel attention module to select discriminative features for feature refinement. Finally, the output of each layer at last recurrent step are fused to obtain the final result. We collect a new dataset consists of various challenging images and their pixel-wise annotations for promoting further study. Extensive experiments on two commonly used datasets and our newly collected one are conducted to demonstrate both the efficacy and efficiency of DeFusionNet.

Big data processing systems, such as Hadoop and Spark, usually work in large-scale, highly-concurrent, and multi-tenant environments that can easily cause hardware and software malfunctions or failures, thereby leading to performance degradation. Several systems and methods exist to detect big data processing systems’ performance degradation, perform root-cause analysis, and even overcome the issues causing such degradation. However, these solutions focus on specific problems such as stragglers and inefficient resource utilization. There is a lack of a generic and extensible framework to support the real-time diagnosis of big data systems. In this article, we propose, develop and validate AutoDiagn. This generic and flexible framework provides holistic monitoring of a big data system while detecting performance degradation and enabling root-cause analysis. We present an implementation and evaluation of AutoDiagn that interacts with a Hadoop cluster deployed on a public cloud and tested with real-world benchmark applications. Experimental results show that AutoDiagn can offer a high accuracy root-cause analysis framework, at the same time as offering a small resource footprint, high throughput, and low latency.

Current spatial-keyword publish/subscribe systems need to handle spatial-keyword skyline queries over geo-textual streams to continuously obtain good results. The skyline queries in such systems face two main problems: (1) query problems, because the powerful query capability is required for the strict limit of the response time and the large number of items concerned by the users, and (2) scalability issue, because millions of active users are maintained simultaneously with many network-connected machines. Unfortunately, the current approach is towards static data. Thus, this paper first proposes a distributed skyline query processing framework. Then, we optimize the skyline computing by introducing MF-R$^t$-tree, which is an update-efficient and space-saving indexing structure and a fast approach for processing a continuous spatial-keyword skyline query called$eager^*$. Finally, a spatial and textual signature-based communication optimization method is proposed to support scalability. The experimental results indicate that (1) MF-R$^t$-tree can significantly reduce update costs, while maintaining a low storage cost, and a query performance comparable to IL-Quadtree, (2)$eager^*$can averagely accelerate 79.72 × faster than the method based on BNL, (3) the communication optimization method significantly reduces the communication cost, and (4) the distributed framework can efficiently support large-scale skyline queries.

Understanding the interconnected relationships of large-scale information networks like social, scholar and Internet of Things networks is vital for tasks like recommendation and fraud detection. The vast majority of the real-world networks are inherently heterogeneous and dynamic, containing many different types of nodes and edges and can change drastically over time. The dynamicity and heterogeneity make it extremely challenging to reason about the network structure. Unfortunately, existing approaches are inadequate in modeling real-life dynamical networks as they either have strong assumption of a given stochastic process or fail to capture the heterogeneity of network structure, and they all require extensive computational resources. We introduceLime, a better approach for modeling dynamic and heterogeneous information networks.Limeis designed to extract high-quality network representation with significantly lower memory resources and computational time over the state-of-the-arts. Unlike prior work that uses a vector to encode each network node, we exploit the semantic relationships among network nodes to encode multiple nodes with similar semantics in shared vectors. By using many fewer node vectors, our approach significantly reduces the required memory space for encoding large-scale networks. To effectively trade information sharing for reduced memory footprint, we employ the recursive neural network (RsNN) with carefully designed optimization strategies to explore the node semantics in a novel cuboid space. We then go further by showing, for the first time, how an effective incremental learning approach can be developed – with the help of RsNN, our cuboid structure, and a set of novel optimization techniques – to allow a learning framework to quickly and efficiently adapt to a constantly evolving network. We evaluateLimeby applying it to three representative network-based tasks, node classification, node clustering and anomaly detection, performing on three large-scale datasets. We compareLimeagainst eleven prior state-of-the-art approaches for learning network representation. Our extensive experiments demonstrate thatLimenot only reduces the memory footprint by over 80 percent and the processing time over 2x when learning network representation but also delivers comparable performance for downstream processing tasks. We show that our incremental learning method can boost the learning time by up to 20x without compromising the quality of the learned network representation.

Edge computing is an emerging promising computing paradigm that brings computation and storage resources to the network edge, hence significantly reducing the service latency and network traffic. In edge computing, many applications are composed of dependent tasks where the outputs of some are the inputs of others. How to offload these tasks to the network edge is a vital and challenging problem which aims to determine the placement of each running task in order to maximize the Quality-of-Service (QoS). Most of the existing studies either design heuristic algorithms that lack strong adaptivity or learning-based methods but without considering the intrinsic task dependency. Different from the existing work, we propose an intelligent task offloading scheme leveraging off-policy reinforcement learning empowered by a Sequence-to-Sequence (S2S) neural network, where the dependent tasks are represented by a Directed Acyclic Graph (DAG). To improve the training efficiency, we combine a specific off-policy policy gradient algorithm with a clipped surrogate objective. We then conduct extensive simulation experiments using heterogeneous applications modelled by synthetic DAGs. The results demonstrate that: 1) our method converges fast and steadily in training; 2) it outperforms the existing methods and approximates the optimal solution in latency and energy consumption under various scenarios.

Cloud computing offers a wide range of services through a pool of heterogeneous Physical Machines (PMs) hosted on cloud data centers, where each PM can host several Virtual Machines (VMs). Resource sharing among VMs comes with major benefits, but it can create technical challenges that have a detrimental effect on the performance. To ensure a specific service level requested by the cloud-based applications, there is a need for an approach to assign adequate resources to each VM. To this end, we present our novel Multi-Loop Control approach, calledMultiScaler, to allocate resources to VMs based on the Service Level Agreement (SLA) requirements and the run-time conditions.MultiScaleris mainly composed of three different levels working closely with each other to achieve an optimal resource allocation. We propose a set of tailor-made controllers to monitor VMs and take actions accordingly to regulate contention among collocated VMs, to reallocate resources if required, and to migrate VMs from one PM to another. The evaluation in a VMware cluster have shown that theMultiScalerapproach can meet applications performance goals and guarantee the SLA by assigning the exact resources that the applications require. Compared with sophisticated baselines,MultiScalerproduces significantly better reaction to changes in workloads even under the presence of noisy neighbors.

For a number of years, due to an exponential increase in the demand for an eco-friendly environment, there has been a rapid increase in the green city revolution across the globe. Subsequently, load shifting of major energy consumers from conventional power grids to renewable energy sources (RES) has become inevitable. Towards this end, cloud data centers (DCs) have emerged as significant consumers of energy that solely rely on power grids to fuel their day-to-day operations. Nevertheless, their energy consumption has increased significantly which in turn has substantially raised the global carbon footprint rate. These challenges can be best addressed by the judicious utilization of RES which have well established advantages like reduced operational costs and carbon emissions. Keeping in view of the above facts, the ultimate goal of the proposed work is to design a comprehensive workload classification; and job scheduling and Vitual machine placement architecture for cloud DCs powered by RES and power grids. For this, a multi-objective optimization scheme is proposed which operates in two phases. In phase I,a random forest-based wrapper schemeknown as Boruta, is used for relevant feature set selection for the incoming workload. This is followed by classification of the workload using a locality sensitive hashing-based support vector machines approach. In phase II, a multi-objective optimization problem for job scheduling and VM placement is formulated with respect to parameters such as service level agreement (SLA), energy cost, carbon footprint rate (CFR), and availability of RES. It is further solved using an enhanced heuristic approach based on a greedy strategy. Our experimental evaluations show an average improvement of approximately 31 percent in energy utilization, 28 percent in energy cost, and 36 percent in CFR, with a slight degradation in SLA assurance (about 2 percent) compared with the existing schemes.

In recent years, renewable energy, such as wind and photovoltaic electric power has been increasingly integrated into data center power provisioning systems to address high energy consumption of data centers. However, in reality, the intermittency and randomness of renewable energy (power supply fluctuation) is detrimental to the reliable operation of sophisticated IT equipment in those so-called green data centers. In this article, we address the problem of data center power regulation explicitly taking into account the unreliability and instability of renewable energy sources. To this extent, we design a novel data center power control framework that smoothens the power fluctuation and instability of renewable energy sources. The core of our framework is two power regulation optimization algorithms. In particular, a server workload scheduling algorithm deals with high frequency fluctuations while an Uninterruptable Power Supply (UPS) power regulation algorithm handles low frequency and large extent power fluctuations. These algorithms are also designed to satisfy service level agreement (SLA) and standby power supply capacity. We have conducted an extensive evaluation study using trace data of a real data center of 30000-node cluster with 50 x 250 UPS battery groups and 24-hour power generation data from real wind farm and photovoltaic power station. The experimental results show our framework effectively smoothens fluctuations of data center power supply, more effective use of renewable energy, and extend the UPS batteries’ lives to reduce the skyrocketed data center operating expenses.

Remote sensing change detection (RSCD), which identifies the changed and unchanged pixels from a registered pair of remote sensing images, has enjoyed remarkable success recently. However, locating changed objects with fine structural details is still a challenging problem in RSCD. In this paper, we propose a novel remote sensing change detection network via temporal feature interaction and guided refinement (TFI-GR) to solve this issue. Specifically, unlike previous methods, which just employ one single concatenation or subtraction operation for bi-temporal feature fusion, we design a temporal feature interaction module (TFIM) to enhance interaction between bi-temporal features and capture temporal difference information at diverse feature levels. Afterword, a guided refinement modules (GRM), which aggregates both low- and high-level temporal difference representations to polish the location information of high-level features and filter the background clutters of low-level features, is repeatedly performed. Finally, the multi-level temporal difference features are progressively fused to generate change maps for change detection. To demonstrate the effectiveness of the proposed TFI-GR, comprehensive experiments are performed on three high spatial resolution remote sensing change detection datasets. Experimental results indicate that the proposed method is superior to other state-of-the-art change detection methods. The demo code of this work is publicly available at https://github.com/guanyuezhen/TFI-GR.

In Industry 5.0, the massive number of Internet of Things devices have increasing demands for group communication with a high communication efficiency and low energy consumption. However, group communication meets continuously increasing security risk challenges. Existing authentication and group key agreement schemes have encountered many problems, such as lack of anonymity and untraceability. In this article, we propose an anonymous authentication and dynamic group key agreement scheme based on the Blockchain and token mechanism, where each group member can apply for a time-sensitive token during the first authentication and only needs to check the validity of the token in the subsequent authentication, reducing the computational and transmission costs considerably. The verification on the security of the proposed scheme is tackled through mathematical analysis and validated using ProVerif, and comparisons with existing schemes demonstrate that the proposed scheme reduces the security risks and each group member’s energy consumption.

Wireless capsule endoscopy is a modern non-invasive Internet of Medical Imaging Things that has been increasingly used in gastrointestinal tract examination. With about one gigabyte image data generated for a patient in each examination, automatic lesion detection is highly desirable to improve the efficiency of the diagnosis process and mitigate human errors. Despite many approaches for lesion detection have been proposed, they mainly focus on large lesions and are not directly applicable to tiny lesions due to the limitations of feature representation. As bleeding lesions are a common symptom in most serious gastrointestinal diseases, detecting tiny bleeding lesions is extremely important for early diagnosis of those diseases, which is highly relevant to the survival, treatment, and expenses of patients. In this article, a method is proposed to extract and fuse multi-scale deep features for detecting and locating both large and tiny lesions. A feature extracting network is first used as our backbone network to extract the basic features from wireless capsule endoscopy images, and then at each layer multiple regions could be identified as potential lesions. As a result, the features maps of those potential lesions are obtained at each level and fused in a top-down manner to the fully connected layer for producing final detection results. Our proposed method has been evaluated on a clinical dataset that contains 20,000 wireless capsule endoscopy images with clinical annotation. Experimental results demonstrate that our method can achieve 98.9% prediction accuracy and 93.5% score, which has a significant performance improvement of up to 31.69% and 22.12% in terms of recall rate and score, respectively, when compared to the state-of-the-art approaches for both large and tiny bleeding lesions. Moreover, our model also has the highest AP and the best medical diagnosis performance compared to state-of-the-art multi-scale models.

Temporal information is critical for routing computation in the vehicular network. It plays a vital role in the vehicular network. Till now, most existing routing schemes in vehicular networks consider the networks as a sequence of static graphs. We need to find an appropriate method to process temporal information into routing computation. Thus, in this paper, we propose a routing algorithm based on the Hidden Markov Model (HMM) and temporal graph, namely, Prediction-Based Temporal Graph Routing Algorithm (PT-GROUT). This new algorithm considers the vehicular network as a temporal graph, in which each data transmission as an edge has its specific temporal information. To better capture the temporal information, we select Software-Defined Vehicular Network (SDVN) as our network architecture, which is a preferred architecture for processing the temporal graph regarding the vehicular network since all vehicle statuses can be easily managed. To compute the future routing path accurately and efficiently, the future temporal graph is predicted by applying HMM, in which we model the current vehicular network with dynamic programming and greedy strategies. With the temporal information and reasonable setting of HMM, PT-GROUT can better evaluate the vehicular network and discover the evolution of the internal structure of the network. The optimal routing path can be achieved more efficiently. The simulation results demonstrate that PT-GROUT can substantially improve the computation efficiency and reduce packet loss and delivery delay compared with its counterparts.

Federated Learning (FL) is a promising machine learning paradigm to cooperatively train a global model with highly distributed data located on mobile devices. Aiming to optimize the communication efficiency for gradient aggregation and model synchronization among large-scale devices, we propose a relay-assisted FL framework. By breaking the traditional transmission-order constraint and exploiting the broadcast characteristic of relay nodes, we design a novel synchronization scheme named Partial Synchronization Parallel (PSP), in which models and gradients are transmitted simultaneously and aggregated at relay nodes, resulting in traffic reduction. We prove that PSP has the same convergence rate as the sequential synchronization approaches via rigorous analysis. To further accelerate the training process, we integrate PSP with any unbiased and error-bounded compression technologies and prove that the convergence properties of the resulting scheme still hold. Extensive experiments are conducted in a distributed cluster environment with real-world datasets and the results demonstrate that our proposed approach reduces the training time up to 37 percent compared to state-of-the-art methods.

Network slicing is the key to enable virtualized resource sharing among vertical industries in the era of 5G communication. Efficient resource allocation is of vital importance to realize network slicing in real-world business scenarios. To deal with the high algorithm complexity, privacy leakage, and unrealistic offline setting of current network slicing algorithms, in this paper we propose a fully decentralized and low-complexity online algorithm, DPoS, for multi-resource slicing. We first formulate the problem as a global social welfare maximization problem. Next, we design the online algorithm DPoS based on the primal-dual approach and posted price mechanism. In DPoS, each tenant is incentivized to make its own decision based on its true preferences without disclosing any private information to the mobile virtual network operator and other tenants. We provide a rigorous theoretical analysis to show that DPoS has the optimal competitive ratio when the cost function of each resource is linear. Extensive simulation experiments are conducted to evaluate the performance of DPoS. The results show that DPoS can not only achieve close-to-offline-optimal performance, but also have low algorithmic overheads.

Knee osteoarthritis (KOA) as a disabling joint disease has doubled in prevalence since the mid-20th century. Early diagnosis for the longitudinal KOA grades has been increasingly important for effective monitoring and intervention. Although recent studies have achieved promising performance for baseline KOA grading, longitudinal KOA grading has been seldom studied and the KOA domain knowledge has not been well explored yet. In this paper, a novel deep learning architecture, namely adversarial evolving neural network (A-ENN), is proposed for longitudinal grading of KOA severity. As the disease progresses from mild to severe level, ENN involves the progression patterns for accurately characterizing the disease by comparing an input image it to the template images of different KL grades using convolution and deconvolution computations. In addition, an adversarial training scheme with a discriminator is developed to obtain the evolution traces. Thus, the evolution traces as fine-grained domain knowledge are further fused with the general convolutional image representations for longitudinal grading. Note that ENN can be applied to other learning tasks together with existing deep architectures, in which the responses characterize progressive representations. Comprehensive experiments on the Osteoarthritis Initiative (OAI) dataset were conducted to evaluate the proposed method. An overall accuracy was achieved as 62.7%, with the baseline, 12-month, 24-month, 36-month, and 48-month accuracy as 64.6%, 63.9%, 63.2%, 61.8% and 60.2%, respectively.

In mobile edge computing (MEC) systems, mobile users (MUs) are capable of allocating local resources (CPU frequency and transmission power) and offloading tasks to edge servers in the vicinity in order to enhance their computation capabilities and reduce back-and-forth transmission over backhaul link. Nevertheless, mobile environment makes it hard to draw offloading and resource allocation decisions under dynamical wireless channel state and users’ locations. In real life, social relationship is also provably a significant factor affecting integral performance in collaborative work, which results in MUs decisions strongly coupled and renders this problem further intractable. Most of previous works ignore the impact of inter-user dependency (or data dependency among IoT devices). To bridge this gap, we study the service collaboration with master-slave dependency among service chains of MUs and formulate this combinational optimization problem as a mixed integer non-linear programming (MINLP) problem. To this end, we derive the closed-form expression of resource allocation solution by convex optimization and transform it to integer linear programming (ILP) problem. Subsequently, we propose a distributed algorithm based on Markov approximation which has polynomial computation complexity. Experimental result on real-world dataset substantiates the usefulness and superiority of our scheme, in terms of reducing latency and energy consumption.

Edge computing is booming as a promising paradigm to extend service provisioning from the centralized cloud to the network edge. Benefit from the development of serverless computing, an edge server can be configured as a carrier of limited serverless functions, in the way of deploying Docker runtime and Kubernetes engine. Meanwhile, an application generally takes the form of directed acyclic graphs (DAGs), where vertices represent dependent functions and edges represent data traffic. The status quo of minimizing the completion time (a.k.a. makespan) of the application motivates the study on optimal function placement. However, current approaches lose sight of proactively splitting and mapping the traffic to the logical data paths between the heterogeneous edge servers, which could affect the makespan significantly. To remedy that, we propose an algorithm, termed as Dependent Function Embedding (DPE), to get the optimal edge server for each function to execute and the moment it starts executing. DPE finds the best segmentation of each data traffic by exquisitely solving several infinity norm minimization problems. DPE is theoretically verified to achieve the global optimality. Extensive experiments on Alibaba cluster trace show that DPE significantly outperforms two baseline algorithms in makespan by 43.19% and 40.71%, respectively.

There is growing interest in applying distributed machine learning to edge computing, formingfederated edge learning. Federated edge learning faces non-i.i.d. and heterogeneous data, and the communication between edge workers, possibly through distant locations and with unstable wireless networks, is more costly than their local computational overhead. In this work, we propose${{\sf DONE}}$, a distributed approximate Newton-type algorithm with fast convergence rate for communication-efficient federated edge learning. First, with strongly convex and smooth loss functions,${{\sf DONE}}$approximates the Newton direction in a distributed manner using the classical Richardson iteration on each edge worker. Second, we prove that${{\sf DONE}}$has linear-quadratic convergence and analyze its communication complexities. Finally, the experimental results with non-i.i.d. and heterogeneous data show that${{\sf DONE}}$attains a comparable performance to Newton's method. Notably,${{\sf DONE}}$requires fewer communication iterations compared to distributed gradient descent and outperforms DANE, FEDL, and GIANT, state-of-the-art approaches, in the case of non-quadratic loss functions.

Edge intelligence has emerged as a prevalent enabling technology to support various intelligent applications. Along with the prosperity, it also raises great concern on the security and privacy since the edge servers are usually shared and untrusted. The security-sensitive code (i.e., the pre-trained model) and data may be easily stolen by malicious tenants, and even untrusted infrastructure providers. To this end, Software Guard Extensions (SGX) is proposed to provide an isolated Trust Execution Environment (TEE) for security and privacy guarantee. However, we find that running tasks in SGX suffer certain performance degradation due to the limited Enclave Page Cache (EPC) size. This further leads to frequent page swapping operations and the high enclave call overhead, which are also influenced by the task (i.e., DNN layer) dispatching and scheduling. To this end, in this paper, we designLasagna, as an SGX based secure DNN inference acceleration framework, which explores the layered-structure of DNN models to well balance the usage of the scarce EPC resources and the computation resources. Lasagna mainly consists of a global task balancer and a local task scheduler, responding for task dispatching across distributed edge servers and task scheduling in local server, respectively. We evaluate Lasagna over different well-known DNN models, and the results show that Lasagna effectively speeds up the inference performance by$1.11\times -1.51\times$.

Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL (FedAvg) employs gradient descent algorithm, which may not be efficient enough. Momentum is able to improve the situation by adding an additional momentum step to accelerate the convergence and has demonstrated its benefits in both centralized and FL environments. It is well-known that Nesterov Accelerated Gradient (NAG) is a more advantageous form of momentum, but it is not clear how to quantify the benefits of NAG in FL so far. This motives us to propose FedNAG, which employs NAG in each worker as well as NAG momentum and model aggregation in the aggregator. We provide a detailed convergence analysis of FedNAG and compare it with FedAvg. Extensive experiments based on real-world datasets and trace-driven simulation are conducted, demonstrating that FedNAG increases the learning accuracy by 3–24% and decreases the total training time by 11–70% compared with the benchmarks under a wide range of settings.

Most of the existing FL systems focus on a data-parallel architecture where training data are partitioned by samples among several parties. In some real-life applications, however, partitioning by features is also of practical relevance and the number of features is usually unbalanced among parties. The corresponding learning framework is referred to as Vertical Federated Learning (VFL). Though some pioneering work focused on VFL, the convergence properties of VFL on unbalanced features, especially when parties conduct different numbers of local updates concerning heterogeneous computational capabilities are still unknown. In this article, we propose a new learning framework to improve the training efficiency of VFL on unbalanced features. Given the number of features and the computational capability owned by each party, our thorough theoretical analysis exhibits that the number of local updates conducted by each party has a great effect on the convergence rate and the computational complexity, both of which jointly determine the overall training efficiency in an interrelated and sophisticated way. Based on our theoretical findings, we formulate an optimization problem and derive the optimal solution by selecting an adaptive number of local training rounds for each party. Extensive experiments on various datasets and models demonstrate that our approach significantly improves the training efficiency of VFL.

Matrix factorization is a powerful method to implement collaborative filtering recommender systems. This article addresses two major challenges, privacy and efficiency, which matrix factorization is facing. We based our work on DS-ADMM, a distributed matrix factorization algorithm with decent efficiency, to achieve the following two pieces of work: (1) Integrated local differential privacy paradigm into DS-ADMM to provide the privacy-preserving property; (2) Introduced a stochastic quantized function to reduce transmission overheads in ADMM to further improve efficiency. We named our work DS-ADMM++, in which one ’+’ refers to differential privacy, and the other ’+’ refers to quantized techniques. DS-ADMM++ is the first to perform efficient and private matrix factorization under the scenarios of differential privacy and DS-ADMM. We conducted experiments with benchmark data sets to demonstrate that our approach provides differential privacy and excellent scalability with a decent loss of accuracy.

To achieve a high degree of resource utilization, production clusters need to co-schedule diverse workloads – including both batch analytic jobs with short-lived tasks and long-running applications (LRAs) that execute for a long time frame from hours to months – onto the shared resources. Microservice architecture advances the manifestation of distributed LRAs (DLRAs), comprising multiple interconnected microservices that are executed in long-lived distributed containers and serve massive user requests. Detecting and mitigating QoS violation become even more intractable due to the network uncertainties and latency propagation across dependent microservices. However, current resource managers are only responsible for resource allocation among applications/jobs but agnostic to runtime QoS such as latency at application level. The state-of-the-art QoS-aware scheduling approaches are dedicated for monolithic applications, without considering the temporal-spatio performance variability across distributed microservices. In this paper, we presentToposch, a new scheduling and execution framework to prioritize the QoS of DLRAs whilst balancing the performance of batch jobs and maintaining high cluster utilization through harvesting idle resources.Toposchtracks footprints of every single request across microservices and uses critical path analysis, based on the end-to-end latency graph, to identify microservices that have high risk of QoS violation. Based on microservice and node level risk assessment, we intervene the batch scheduling by adaptively reducing the visible resources to batch tasks and thus delaying their execution to give way to DLRAs. We propose a prediction-based vertical resource auto-scaling mechanism, with the aid of resource-performance modeling and fine-grained resource inference and access control, for prompt recovery of QoS violation. A cost-effective task preemption is leveraged to ensure a low-cost task preemption and resource reclamation during the auto-scaling.Toposchis integrated with Apache YARN and experiments show thatToposchoutperforms other baselines in terms of performance guarantee of DLRAs, at an acceptable cost of batch job slowdown. The tail latency of DLRAs is merely 1.12x of the case of executing alone on average inToposchwith a 26% JCT increase of Spark analytic jobs.

Sustainable energy management systems have been increasingly studied in recent years. Non-intrusive load monitoring (NILM), as a key component, estimates the power consumption of individual appliances from the main readings only. However, most NILM approaches are computationally expensive, and their generality is negatively affected by the data drift occurred when the models are used across domains. Besides, the threats of privacy violation will rise in the model transfer due to the possible leakage of the personal information of the users from the source domain. To address all these challenges, we designed a cost-efficient learning method using LightGBM for energy disaggregation. We also proposed a model-based transfer learning algorithm using feature importance analysis, which enhances the generalisation capability of tree-based ensemble models applied in different domains while protecting privacy. We conducted experiments with real-world data sets. The performance of our approach is superior to the state-of-the-art solutions.

With Cloud computing becoming mainstream for the execution of various applications, the multi-objective scheduling algorithms for providing the most suitable services to users have gained much attention. As provisioning Cloud services that satisfy end-users quality of service (QoS) requirements is complex and challenging, scheduling algorithms for cloud computing tend to focus on optimizing the execution cost or the execution time within user-defined deadline constraints. This paper addresses the problem of efficiently allocating Cloud services among competing jobs to achieve multiple end-users QoS. We design and develop a framework called Autonomic Resource Provisioning and Scheduling (ARPS) framework. ARPS framework has the decision-making capability to schedule the jobs at the best resources within the deadline and optimizes both the execution time and the cost simultaneously. The ARPS framework is also integrated with the spider monkey optimization (SMO) algorithm based scheduling mechanism. Our proposed mechanism is intended to solve a multi-objective optimization problem, including minimizing processing time, cost, and energy consumption. We study the effectiveness of the proposed scheduling mechanism through extensive simulation analysis using Cloudsim To assess the relative performance of our method, we compare it against four existing mechanisms. Experimental results show that the proposed mechanism outperforms its counterparts.

The increasing use of renewable energy sources and electrical energy storage systems creates a new energy paradigm for residential houses and buildings. Such design reduces carbon footprint, but it also introduces a new challenge for minimizing the electricity bill while still meeting users’ needs. This challenge is often accompanied by deep uncertainty in user load demands, electricity tariffs, and renewable energy generations. To address this challenge, we propose an online algorithm, Virtual Algorithm-based Lightweight Online Scheduling (VALOS), to manage electricity purchasing and battery operations. Our solution does not use any prediction components in dealing with uncertainties. Instead, our algorithm employs a lightweight routine, Virtual Algorithm (VA), for making critical decisions to manage uncertainty. We prove that VA achieves an expected probability of 1/e for choosing the optimal purchasing timing online and incurs only a logarithmic computational cost. With VA, VALOS decides how the energy storage reacts to the load demands, which incurs only a linear-logarithmic online computational cost while achieving optimal performance under the specified conditions. Finally, we conduct extensive trace-driven simulations on real-world datasets to confirm the theoretical results and demonstrate the potential of VALOS.

The protection of data processing is emerging as an essential aspect of data analytics, machine learning, delegation of computation, Internet of Things, medical and financial analysis, smart cities, genomics, non-disclosure searching, among others. Often, they use sensitive information that cannot be protected by traditional cryptosystems. Homomorphic Encryption (HE) schemes and secure Multi-Party Computation (MPC) are considered suitable solutions for privacy protection. In this paper, we propose and analyze the performance of three homomorphic Logistic Regression (LR) models with Gradient Descent (GD) algorithms based on the Residue Number System (RNS). We compare their performance with four traditional non-homomorphic versions, one homomorphic algorithm based on RNS with Batch GD, and two state-of-the-art homomorphic algorithms. To validate our approach, we consider six public datasets of different medicine domains (diabetes, cancer, drugs, etc.) and genomics. We use a 5-fold cross-validation technique for a fair comparison in terms of the solution quality and training time. The results show that propose homomorphic solutions have similar accuracy with non-homomorphic algorithms, increased classification performance, and decreased training time compared with the state-of-the-art HE algorithms.

Edge intelligence has already been widely regarded as a key enabling technology in a variety of domains. Along with the prosperity, increasing concern is raised on the security and privacy of intelligent applications. As these applications are usually deployed on shared and untrusted edge servers, malicious co-located attackers, or even untrustworthy infrastructure providers, may acquire highly security-sensitive data and code (i.e., the pre-trained model). Software Guard Extensions (SGX) provides an isolated Trust Execution Environment (TEE) for task security guarantee. However, we notice that DNN inference performance in SGX is severely affected by the limited enclave memory space due to the resultant frequent page swapping operations and the high enclave call overhead. To tackle this problem, we propose Lasagna, an SGX oriented DNN inference performance acceleration framework without compromising the task security. Lasagna consists of a local task scheduler and a global task balancer to optimize the system performance by exploring the layered-structure of DNN models. Our experiment results show that our layer-aware Lasagna effectively speeds up the well-known DNN inference in SGX by 1.31x-1.97x.

In this paper, we propose a hyperspectral band selection method via spatial-spectral weighted region-wise multiple graph fusion-based spectral clustering, referred to as RMGF briefly. Considering that different objects have different reflection characteristics, we use a superpixel segmentation algorithm to segment the first principal component of original hyperspectral image cube into homogeneous regions. For each superpixel, we construct a corresponding similarity graph to reflect the similarity between band pairs. Then, a multiple graph diffusion strategy with theoretical convergence guarantee is designed to learn a unified graph for partitioning the whole hyperspectral cube into several subcubes via spectral clustering. During the graph diffusion process, the spatial and spectral information of each superpixel are embedded to make spatial/spectral similar superpixels contribute more to each other. Finally, the band containing minimum noise in each subcube is selected to represent the whole subcube. Extensive experiments are conducted on three public datasets to validate the superiority of the proposed method when compared with other state-of-the-art ones.

Container, as a light-weight virtualization technology with the advantages of continuous integration and easy deployment, has been widely adopted to support diverse microservices. At runtime, non-local container images need to be frequently pulled from remote registries to local servers, resulting in large pulling traffic and hence long startup time. A distinctive feature in container-based microservice, which has not been exploited, is that container images are in layered structure and some common base layers can be shared between co-located microservices. In this paper, we propose a layer sharing microservice deployment and image pulling strategy which explores the advantage of layer sharing to speedup microservice startup and lower image storage consumption. The problem is formulated into an Integer Linear Programming (ILP) form. An Accelerated Distributed Augmented Lagrangian (ADAL) based distributed algorithm executed cooperatively by registries and servers is proposed. Through extensive trace driven experiments, we validate the high efficiency of our ADAL based algorithm as it accelerates the microservice startup by 2.30 times in average and reduces the storage consumption by 55.33%.

Time-Slotted Channel Hopping (TSCH) is a synchronous Medium Access Control (MAC) technology standardized as a part of IEEE 802.15.4e to provide highly reliable communications for resource-constrained devices. While IETF and IEEE standards defined solutions for the configuration of TSCH nodes, the problem of creating dynamic TSCH schedules has been left open. In this poster, we introduce GT-SF, a distributed TSCH scheduling function that is designed based on the non-cooperative game-theory for low-power Internet of Things (IoT) applications. We implement GT-SF on Zolerita firefly IoT motes and the Contiki-NG operating system to examine its effectiveness. The evaluation results demonstrate that GT-SF outperforms Orchestra (the state-of-the-art method) by enhancing the packet delivery ratio and reducing the latency.

Secure Device-to-Device (D2D) communication is becoming increasingly important with the ever-growing number of Internet-of-Things (IoT) devices in our daily life. To achieve secure D2D communication, the key agreement between different IoT devices without any prior knowledge is becoming desirable. Although various approaches have been proposed in the literature, they suffer from a number of limitations, such as low key generation rate and short pairing distance. In this paper, we present InaudibleKey, an inaudible acoustic signal based key generation protocol for mobile devices. Based on acoustic channel reciprocity, InaudibleKey exploits the acoustic channel frequency response of two legitimate devices as a common secret to generating keys. InaudibleKey employs several novel technologies to significantly improve its performance. We conduct extensive experiments to evaluate the proposed system in different real environments. Compared to state-of-the-art works, InaudibleKey improves key generation rate by 3 times, extends pairing distance by 3.2 times, and reduces information reconciliation counts by 2.5 times. Security analysis demonstrates that InaudibleKey is resilient to a number of malicious attacks. We also implement InaudibleKey on modern smartphones and resource-limited IoT devices. Results show that it is energy-efficient and can run on both powerful and resource-limited IoT devices without incurring excessive resource consumption.

A significant amount of research studies in the past years has been devoted on developing efficient mechanisms to control the level of degradation among consolidate workloads in a shared platform. Workload consolidation is a promising feature that is employed by most service providers to reduce the total operating costs in traditional computing systems [1]–[3]. Serverless paradigm - also known as Function as a Service, FaaS, and Lambda - recently emerged as a new virtualization run-time model that disentangles the traditional state of applications' users from the burden of provisioning physical computing resources, leaving the difficulty of providing the adequate resource capacity on the service provider's side. This paper focuses on a number of challenges associated with workload consolidation when a serverless platform is expected to execute several data-intensive functional units. Each functional unit is considered to be the atomic component that reacts to a stream of input data. A serverless application in the proposed model is composed of a series of functional units. Through a systematic approach, we highlight the main challenges for devising an efficient workload consolidation process in a data-intensive serverless platform. To this end, we first study the performance interference among multiple workloads to obtain the capacity of last level cache (LLC). We show how such contention among workloads can lead to a significant throughput degradation on a single physical server. We expand our investigation into a general case with the aim to prevent the total throughput never falling below a predefined utilization level. Based on the empirical results, we develop a consolidation model and then design a computationally efficient controller to optimize the throughput degradation among a platform consists fs multiple machines. The performance evaluation is conducted using modern workloads inspired by data management services, and data analytic benchmark tools in our in-house four node platform showing the efficiency of the proposed solution to mitigate the QoS violation rate for high priority applications by 90% while can enhance the normalized throughput usage of disk devices by 39 %.

A significant portion of research work in the past decade has been devoted on developing resource allocation and task scheduling solutions for large-scale data processing platforms. Such algorithms are designed to facilitate deployment of data analytic applications across either conventional cluster computing systems or modern virtualized data-centers. The main reason for such a huge research effort stems from the fact that even a slight improvement in the performance of such platforms can bring a considerable monetary savings for vendors, especially for modern data processing engines that are designed solely to perform high throughput or/and low-latency computations over massive-scale batch or streaming data. A challenging question to be yet answered in such a context is to design an effective resource allocation solution that can prevent low resource utilization while meeting the enforced performance level (such as 99-th latency percentile) in circumstances where contention among applications to obtain the capacity of shared resources is a non negligible performance-limiting parameter. This paper proposes a resource controller system, called QSpark, to cope with the problem of (i) low performance (i.e., resource utilization in the batch mode and p-99 response time in the streaming mode), and (ii) the shared resource interference among collocated applications in a multi-tenancy modern Spark platform. The proposed solution leverages a set of controlling mechanisms for dynamic partitioning of the allocation of computing resources, in a way that it can fulfill the QoS re-quirements of latency-critical data processing applications, while enhancing the throughput for all working nodes without reaching their saturation points. Through extensive experiments in our in-house Spark cluster, we compared the achieved performance of proposed solution against the default Spark resource allocation policy for a variety of Machine Learning (ML), Artificial Intelligence (AI), and Deep Learning (DL) applications. Experimental results show the effectiveness of the proposed solution by reducing the p-99 latency of high priority applications by 32 % during the burst traffic periods (for both batch and stream modes), while it can enhance the QoS satisfaction level by 65 % for applications with the highest priority (compared with the results of default Spark resource allocation strategy).

Time-Slotted Channel Hopping (TSCH) was standardized as a part of IEEE 802.15.4e to address the strict reliability and timeliness requirements of low-power Internet of Things (IoT) applications. Setting the size of the TSCH slotframe has a considerable effect on the performance of scheduling algorithms used in IoT networks. Although IETF and IEEE standards define general mechanisms for communication of TSCH nodes, finding the optimal size of the TSCH slotframe has been left open and unresolved. In this poster, we propose an algorithm called S-TSCH to find the optimal size of the TSCH slotframe for maximizing network throughput based on 1) the number of nodes placed in the topology, 2) the data generation rate of applications running on IoT nodes, 3) and the maximum rate of generating TSCH/RPL control packets. To evaluate the performance of our contribution, we implement S-TSCH on Zolerita Firefly IoT motes and the Contiki-NG operating system. Evaluation results show that our proposed method improves the performance of distributed TSCH scheduling algorithms in terms of reliability and delay.

Summary Mobile data offloading has already appeared to offer the means of addressing the challenges of limited computing capability and battery life of mobile devices. Most existing code offloading frameworks only consider migrating an application within a single network at a time and failed to fully utilize the energy efficiency mechanism of the latest CPU chips. To tackle these issues, we propose an offloading approach under both a multitasking environment and a heterogeneous network to increase energy and execution efficiency. The offloading problem was formulated as a biobjective optimization problem that aims to save energy and keep a good performance by combining mobile cloud computing with big.LITTLE technology under a heterogeneous network equipped with cellular and WiFi connectivity. By varying the applications and network scenarios, the experimental results show that with offloading a single application using our proposed framework, the data‐intensive application can obviously benefit when the access point density of WiFi reaches 0.0002 unit/m2. Under multiple application scenarios, our proposed framework can increase processing speed by an average of 0.3× over the single application and can save an average of 25% power over the single application.

The paradigm of the Internet of Things (IoT) and cloud-edge computing plays a significant role in future smart grids. The data-driven solution integrating the artificial intelligence functionalities brings novel methods to address the nontrivial task of economic dispatch in microgrids in the presence of uncertainties of renewable generations and loads. This article proposes a learning-based decision-making framework for the economic energy dispatch of an islanding microgrid based on the cloud-edge computing architecture. Cloud resources are utilized to solve the optimal dispatch decision sequences over historical operating patterns. It can be considered as a sample labeling process for the supervised training that can implement the complex mapping of input-output space through an advanced machine learning model. Then, the well-trained model can be adopted locally at edge computing devices keeping the long-term parameters unchanged for implement the real-time microgrid energy dispatch. The key benefit of the proposed solution is that it effectively avoids the prediction of multiple stochastic variables and the design of sophisticated regulation strategies or reward policy functions for real-time dispatch. The solution is extensively assessed through simulation experiments by the use of real data measurements for a set of operational scenarios and the numerical results validate the effectiveness and benefit of the proposed algorithmic solution.

The epoch of the Internet of Things (IoT) has come by enabling almost everything to gather and share electronic information. Considering the unreliable factors of public IoT, how to outsource huge amounts of indispensable stream data generated by the nodes to the remote storage (RS) efficiently and securely is one of the most challenging issues. In this article, we propose a secure lightweight stream data outsourcing framework for IoT based on identity and blockchain. Taking advantage of identity-based cryptography and blockchain, for public IoT containing untrusted communication channels, nodes, RS, and even verifiers, we introduce a private mobile network and multiple verifiers to ensure that the stream data are stored intact and updated correctly, without the costs and risks brought by the public-key infrastructures (PKI). Meanwhile, the framework can also achieve privacy-preserving checking, by revealing no data to the other entities besides the RS, even in the blockchains. Our comprehensive analysis and experiments demonstrate that the proposed framework is suitable for lightweight devices and practical for IoT.

Prediction of the future location of vehicles and other mobile targets is instrumental in intelligent transportation system applications. In fact, networking schemes and protocols based on machine learning can benefit from the results of such accurate trajectory predictions. This is because routing decisions always need to be made for the future scenario due to the inevitable latency caused by the processing and propagation of the routing request and response. Thus, to predict the high-precision trajectory beyond the state of the art, we propose a generative adversarial network (GAN)-based vehicle trajectory prediction method, GAN-VEEP, for urban roads. The proposed method consists of three components: 1) vehicle coordinate transformation for data set preparation; 2) neural network prediction model trained by GAN; and 3) vehicle turning model to adjust the prediction process. The vehicle coordinate transformation model is introduced to deal with the complex spatial dependence in the urban road topology. Then, the neural network prediction model learns from the behavior of vehicle drivers. Finally, the vehicle turning model can refine the driving path based on the driver’s psychology. Compared with its counterparts, the experimental results show that GAN-VEEP exhibits higher effectiveness in terms of the average accuracy, mean absolute error, and root-mean-squared error.

A simplified approach to accelerate matrix factorization of big data is to parallelize it. A commonly used method is to divide the matrix into multiple non-intersecting blocks and concurrently calculate them. This operation causes the Load balance problem, which significantly impacts parallel performance and is a big concern. A general belief is that the load balance across blocks is impossible by balancing rows and columns separately. We challenge the belief by proposing an approach of “Balanced Partitioning (BaPa)”. We demonstrate under what circumstance independently balancing rows and columns can lead to the balanced intersection of rows and columns, why, and how. We formally prove the feasibility of BaPa by observing the variance of rating numbers across blocks, and empirically validate its soundness by applying it to two standard parallel matrix factorization algorithms, DSGD and CCD++. Besides, we establish a mathematical model of “Imbalance Degree” to explain further why BaPa works well. BaPa is applied to synchronous parallel matrix factorization, but as a general load balance solution, it has significant application potential.

For cloud storage service vendors, balancing the client-perceived IO performance and the self-perceived space cost is always one of the standing challenges. When applying deduplication techniques for the cloud storage systems, the demand for optimizing such tradeoff becomes more pressing. Enabling deduplication decreases the storage space cost, whereas the IO performance will be somewhat affected due to extra processing overhead and data fragmentation. In this article, we address this challenge by proposing MUSE, a MUti-tiered and SLA-drivEn deduplication framework for cloud storage systems. First, we propose a novel notation of Dedup-SLA (deduplication-oriented service level agreement). With different levels of quantified performance/space-cost combinations, the Dedup-SLA serves as a refined service quality protocol between service vendor and customer. Second, MUSE adopts multi-tiered deduplication that orchestrates several combinational forms of deduplication into multiple tiers with varied “deduplication strength”. Third, we implement a mechanism called dynamic deduplication regulation (DDR) to adjust the deduplication behavior during runtime. MUSE's deduplication behavior is periodically switched between tiers according to the predefined Dedup-SLA and instant system status. We conduct comprehensive experiments to compare MUSE with several other types of deduplication schemes. The results demonstrate that MUSE significantly optimizes the IO-performance/space-cost balance compared to other schemes, hence delivering higher deduplication service quality for deduplication-enabled cloud storage systems.

SLA violations do happen in real world. An SLA violation represents the failure of guaranteeing a service, which leads to unwanted consequences such as penalty payments, profit margin reduction, reputation degradation, customer churn and service interruptions. Hence, in the context of cloud-hosted big data analytics applications (BDAAs), it is paramount for providers to predict and prevent SLA violations. While machine learning-based techniques have been applied to detect SLA violations for web service or general cloud service, the study on detecting SLA violations dedicated for cloud-hosted BDAAs is still lacking. In this article, we propose four machine learning techniques and integrate 12 resampling methods to detect SLA violations for batch-based BDAAs in the cloud. We evaluate the efficiency of the proposed techniques in comparison with ideal and baseline classifiers based on a real-world trace dataset (Alibaba). Our work not only helps providers to choose the best performing prediction technique, but also provides them capabilities to uncover the hidden pattern of multiple configurations of BDAAs across layers.

In recent years, the rapid development of the Internet of Things (IoT) has produced a large amount of data that needs to be processed in a timely manner. Traditional cloud computing systems can provide us with plentiful resources to process such data. However, the increasing requirements of IoT applications on data privacy, energy consumption savings and location-aware data processing pushes the emergence and the interplay of fog computing and cloud computing. This paper examines the resource scheduling issue under such a system to minimize makespan and energy consumption. A multi-objective estimation of distribution algorithm (EDA) as well as a partition operator is adopted to divide the graph and determine the task processing permutation and processor assignment. Single and multiple application simulation were both conducted. The comparative results show that the Pareto set produced by our proposed algorithm is able to dominate a large proportion of those solutions by the heuristic method and the simple EDA under single application simulation. When it comes to multi-application simulation, IoT devices can have a much longer lifetime with our proposed scheduling algorithm as well having similar performance to the other algorithms on fog node energy consumption and much better on makespan.

This article presents an efficient fingerprint identification system that implements an initial classification for search-space reduction followed by minutiae neighbor-based feature encoding and matching. The current state-of-the-art fingerprint classification methods use a deep convolutional neural network (DCNN) to assign confidence for the classification prediction, and based on this prediction, the input fingerprint is matched with only the subset of the database that belongs to the predicted class. It can be observed for the DCNNs that as the architectures deepen, the farthest layers of the network learn more abstract information from the input images that result in higher prediction accuracies. However, the downside is that the DCNNs are data hungry and require lots of annotated (labeled) data to learn generalized network parameters for deeper layers. In this article, a shallow multifeature view CNN (SMV-CNN) fingerprint classifier is proposed that extracts: 1) fine-grained features from the input image and 2) abstract features from explicitly derived representations obtained from the input image. The multifeature views are fed to a fully connected neural network (NN) to compute a global classification prediction. The classification results show that the SMV-CNN demonstrated an improvement of 2.8% when compared to baseline CNN consisting of a single grayscale view on an open-source database. Moreover, in comparison with the state-of-the-art residual network (ResNet-50) image classification model, the proposed method performs comparably while being less complex and more efficient during training. The result of classification-based fingerprint identification has shown that the search space is reduced by over 50% without degradation of identification accuracies.

Industry 4.0 have automated the entire manufacturing sector (including technologies and processes) by adopting Internet of Things and cloud computing. To handle the workflows from Industrial Cyber-Physical systems, more and more data centers have been built across the globe to serve the growing needs of computing and storage. This has led to an enormous increase in energy usage by cloud data centers, which is not only a financial burden but also increases their carbon footprint. The private software defined wide area network (SDWAN) connects a cloud provider's data centers across the planet. This gives the opportunity to develop new scheduling strategies to manage cloud providers workload in a more energy-efficient manner. In this context, this article addresses the problem of scheduling data-driven industrial workflow applications over a set of private SDWAN connected data centers in an energy-efficient manner while managing tradeoff of a cloud provider' revenue. Our proposed algorithm aims to minimize the cloud provider's revenue and the usage of nonrenewable energy by utilizing the real-world electricity prices with the availability of green energy on different cloud data centers, where the energy consumption consists of the usage of running application over multiple data centers and transferring the data among them through SDWAN. The evaluation shows that our proposed method can increase usage of green energy for the execution of industrial workflow up to 3× times with a slight increase in the cost when compared to cost-based workflow scheduling methods.

Due to the limited local computation resource, industrial vehicular computation requires offloading the computation tasks with time-delay sensitive and complex demands to other intelligent devices (IDs) once the data is sensed and collected collaboratively. This article considers offloading partial computation tasks of the industrial vehicles (IVs) to multiple available IDs of the industrial mobile edge computing (MEC), including unmanned aerial vehicles (UAVs), and the fixed-position MEC servers, to optimize the system cost including execution time, energy consumption, and the ID rental price. Moreover, to increase the access probability of IV by the UAVs, the geographical area is divided into small partitions and schedule the UAVs regarding the regional IV density dynamically. A minimum incremental task allocation algorithm is proposed to divide the whole task and assign the divided units for the minimum cost increment each time. Experimental results show the proposed solution can significantly reduce the system cost.

The dramatically increasing of mobile applications make it convenient for users to complete complex tasks on their mobile devices. However, the latency brought by unstable wireless networks and the computation failures caused by constrained resources limit the development of mobile computing. A popular approach to solve this problem is to establish a mobile service provisioning system based on a mobile edge computing (MEC) paradigm. In the MEC paradigm, plenty of machines are placed at the edge of the network so that the performance of applications can be optimized by using the involved microservice instances deployed on them. In this paper, we explore the deployment problem of microserivce-based applications in the MEC environment and propose an approach to help to optimize the cost of application deployment with the constraints of resources and the requirement of performance. We conduct a series of experiments to evaluate the performance of our approach. The result shows that our approach can improve the average response time of mobile services.

We discover a new capacity scaling law in ultra-dense networks under practical system assumptions, such as a general multi-piece path loss model, a non-zero base station to user equipment antenna height difference, and a finite user equipment density. The intuition and implication of this new capacity scaling law are completely different from those found in the year 2011. That law indicated that the increase of the interference power caused by a denser network would be exactly compensated by the increase of the signal power due to the reduced distance between transmitters and receivers, and thus, network capacity should grow linearly with network densification. However, we find that both the signal and interference powers become bounded in practical ultra-dense networks, which leads to a constant capacity scaling law. Moreover, our new discovery on the constant capacity scaling law indicates three network optimization problems respectively for base station deployment, user equipment scheduling and base station coordination. These three optimization problems are justified and solved in this paper, shedding new light on the deployment and optimization of ultra-dense networks.

There is an increasing interest in a fast-growing machine learning technique called Federated Learning (FL), in which the model training is distributed over mobile user equipment (UEs), exploiting UEs' local computation and training data. Despite its advantages such as preserving data privacy, FL still has challenges of heterogeneity across UEs' data and physical resources. To address these challenges, we first propose FEDL, a FL algorithm which can handle heterogeneous UE data without further assumptions except strongly convex and smooth loss functions. We provide a convergence rate characterizing the trade-off between local computation rounds of each UE to update its local model and global communication rounds to update the FL global model. We then employ FEDL in wireless networks as a resource allocation optimization problem that captures the trade-off between FEDL convergence wall clock time and energy consumption of UEs with heterogeneous computing and power resources. Even though the wireless resource allocation problem of FEDL is non-convex, we exploit this problem's structure to decompose it into three sub-problems and analyze their closed-form solutions as well as insights into problem design. Finally, we empirically evaluate the convergence of FEDL with PyTorch experiments, and provide extensive numerical results for the wireless resource allocation sub-problems. Experimental results show that FEDL outperforms the vanilla FedAvg algorithm in terms of convergence rate and test accuracy in various settings.

Only when we understand how hackers think, can we defend against their attacks. Towards this end, this paper studies the cyber-attacks that aim to remove nodes or links from network topologies. We particularly focus on one type of such attacks called Maximum Vertex Coverage Attacks under Knapsack constraint (MVCAK), in which a hacker has a fixed budget to remove nodes from a network with the nodes involving different costs for removal, and the hacker's goal is to maximize the number of links incident to the nodes removed. Since the MVCAK problem is NP-hard, we firstly propose an optimal solution by Integer Linear Program formulation. Secondly, we give an approximate solution by Linear Programming relaxation that achieves an approximation ratio of 3/4, outperforming the existing 1 - 1/sqrt(e) (about 0.39). Thirdly, since the straightforward implementation of our approximate solution has a high time complexity, we propose two heuristics to significantly reduce its complexity while preserving the approximation ratio. We formally prove the correctness and the effectiveness of these two heuristics. Finally, we conduct extensive experiments on both artificial and real-world networks, showing that our approximate solution produces almost the same results as the optimal solution in practice and has an acceptable running time.

Many IoT applications and services such as smart parking and smart traffic control contain a network of different analytical components, which are composed in the form of a workflow to make better decisions. These workflows are also known as stream workflows. The focus of existing research works is on the streaming operator graph, which differs from stream workflow application as it involves heterogeneity, multiple data sources and multiple outputs. Considering the complexity and dynamism of stream workflow, meeting real-time data analysis requirements at deployment time is not the whole story as the velocity of data changes over time. This change is the most dynamic form of stream workflow that occurs frequently during the execution of this application. In this article, we propose a new dynamic scheduling technique that manages cloud resources over time to handle data velocity changes in stream workflow while maintaining user-defined real-time data analysis requirements and minimising execution cost. The efficiency of the proposed technique is evaluated, and experimental results showed that this technique outperformed its competitors and is close to the lower bound.

Edge computing, a fast evolving computing paradigm, has spawned a variety of new system architectures and computing methods discussed in both academia and industry. Edge servers are directly deployed near users' equipment or devices owned by telecommunications companies. This allows for offloading computing tasks of various devices nearby to edge servers. Due to the shortage of computing resources in edge computing networks, they are often not as sufficient as the computing resources in a cloud computing center. This leads to the problem of service load imbalance once the load in the edge computing network increases suddenly. To solve the problem of “load evacuation” in edge environments, we introduce a strategy when the number of service requests for mobile devices or IoT devices increases rapidly within a short period of time. Therefore, to prevent poor QoS in edge computing, service load should be migrated to other edge servers to reduce the overall delay of these service requests. In this article, we have introduced a strategy with two stages during the burst load evacuation. Based on an optimal routing search at the dispatching stage, tasks will be migrated from the server in which the burst load occurs to other servers as soon as possible. Subsequently, with the assistance of the remote server and edge servers, these tasks are processed with the highest efficiency through the proposed parallel structure at the scheduling stage. Finally, we conduct numerical experiments to clarify the superiority of our algorithm in an edge environment simulation.

The issue of potential privacy leakage during centralized AI's model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally, without the necessity to upload their personal sensitive data. In FL, the number of clients could be sufficiently large, but the bandwidth available for model distribution and re-upload is quite limited, making it sensible to only involve part of the volunteers to participate in the training process. The client selection policy is critical to an FL process in terms of training efficiency, the final model's quality as well as fairness. In this article, we will model the fairness guaranteed client selection as a Lyapunov optimization problem and then a C2MAB-based method is proposed for estimation of the model exchange time between each client and the server, based on which we design a fairness guaranteed algorithm termed RBCS-F for problem-solving. The regret of RBCS-F is strictly bounded by a finite constant, justifying its theoretical feasibility. Barring the theoretical results, more empirical data can be derived from our real training experiments on public datasets.

Multi-access edge computing (MEC) aims to extend cloud service to the network edge to reduce network traffic and service latency. A fundamental problem in MEC is how to efficiently offload heterogeneous tasks of mobile applications from user equipment (UE) to MEC hosts. Recently, many deep reinforcement learning (DRL)-based methods have been proposed to learn offloading policies through interacting with the MEC environment that consists of UE, wireless channels, and MEC hosts. However, these methods have weak adaptability to new environments because they have low sample efficiency and need full retraining to learn updated policies for new environments. To overcome this weakness, we propose a task offloading method based on meta reinforcement learning, which can adapt fast to new environments with a small number of gradient updates and samples. We model mobile applications as Directed Acyclic Graphs (DAGs) and the offloading policy by a custom sequence-to-sequence (seq2seq) neural network. To efficiently train the seq2seq network, we propose a method that synergizes the first order approximation and clipped surrogate objective. The experimental results demonstrate that this new offloading method can reduce the latency by up to 25 percent compared to three baselines while being able to adapt fast to new environments.

The Sunway TaihuLight, equipped with 10 million cores, is currently the world's third fastest supercomputer. SpMV is one of core algorithms in many high-performance computing applications. This paper implements a fine-grained design for generic parallel SpMV based on the special Sunway architecture and finds three main performance limitations, i.e., storage limitation, load imbalance, and huge overhead of irregular memory accesses. To address these problems, this paper introduces a customized and accelerative framework for SpMV (CASpMV) on the Sunway. The CASpMV customizes an auto-tuning four-way partition scheme for SpMV based on the proposed statistical model, which describes the sparse matrix structure characteristics, to make it better fit in with the computing architecture and memory hierarchy of the Sunway. Moreover, the CASpMV provides an accelerative method and customized optimizations to avoid irregular memory accesses and further improve its performance on the Sunway. Our CASpMV achieves a performance improvement that ranges from 588.05 to 2118.62 percent over the generic parallel SpMV on a CG (which corresponds to an MPI process) of the Sunway on average and has good scalability on multiple CGs. The performance comparisons of the CASpMV with state-of-the-art methods on the Sunway indicate that the sparsity and irregularity of data structures have less impact on CASpMV.

Presents the introductory editorial for this issue of the publication.

To provide efficient networking services at the edge of Internet-of-Vehicles (IoV), Software-Defined Vehicular Network (SDVN) has been a promising technology to enable intelligent data exchange without giving additional duties to the resource constrained vehicles. Compared with conventional centralized SDVNs, hybrid SDVNs combine the centralized control of SDVNs and self-organized distributed routing of Vehicular Ad-hoc NETworks (VANETs) to mitigate the burden on the central controller caused by the frequent uplink and downlink transmissions. Although a wide variety of routing protocols have been developed, existing protocols are designed for specific scenarios without considering flexibility and adaptivity in dynamic vehicular networks. To address this problem, we propose an efficient online sequential learning-based adaptive routing scheme, namely, Penicillium reproduction-based Online Learning Adaptive Routing scheme (POLAR) for hybrid SDVNs. By utilizing the computational power of edge servers, this scheme can dynamically select a routing strategy for a specific traffic scenario by learning the pattern from network traffic. Firstly, this paper applies Geohash to divide the large geographical area into multiple grids, which facilitates the collection and processing of real-time traffic data for regional management in controller. Secondly, a new Penicillium Reproduction Algorithm (PRA) with outstanding optimization capabilities is designed to improve the learning effectiveness of Online Sequential Extreme Learning Machine (OS-ELM). Finally, POLAR is deployed in control plane to generate decision-making model (i.e., routing policy). Based on the real-time featured data, this scheme can choose the optimal routing strategy for a specific area. Extensive simulation results show that POLAR is superior to a single traditional routing protocol in terms of packet delivery ratio and latency.

Spark has emerged as one of the most widely and successfully used data analytical engine for large-scale enterprise, mainly due to its unique characteristics that facilitate computations to be scaled out in a distributed environment. This paper deals with the performance degradation due to resource contention among collocated analytical applications with different priority and dissimilar intrinsic characteristics in a shared Spark platform. We propose an auto-tuning strategy of computing resources in a distributed Spark platform for handling scenarios in which submitted analytical applications have different quality of service (QoS) requirements (e.g., latency constraints), while the interference among computing resources is considered as a key performance-limiting parameter. We compared Spark-Tuner to two widely used resource allocation heuristics in a large scale Spark cluster through extensive experimental settings across several traffic patterns with uncertain rate and application types. Experimental results show that with Spark-Tuner, the Spark engine can decrease the p-99 latency of high priority applications by 43% during the high-rate traffic periods, while maintaining the same level of CPU throughput across a cluster.

Defocus blur detection aims to separate the in-focus and out-of-focus regions in an image. Although attracting more and more attention due to its remarkable potential applications, there are still several challenges for accurate defocus blur detection, such as the interference of background clutter, sensitivity to scales and missing boundary details of defocus blur regions. In order to address these issues, we propose a deep neural network which Recurrently Refines Multi-scale Residual Features (R2MRF) for defocus blur detection. We firstly extract multi-scale deep features by utilizing a fully convolutional network. For each layer, we design a novel recurrent residual refinement branch embedded with multiple residual refinement modules (RRMs) to more accurately detect blur regions from the input image. Considering that the features from bottom layers are able to capture rich low-level features for details preservation while the features from top layers are capable of characterizing the semantic information for locating blur regions, we aggregate the deep features from different layers to learn the residual between the intermediate prediction and the ground truth for each recurrent step in each residual refinement branch. Since the defocus degree is sensitive to image scales, we finally fuse the side output of each branch to obtain the final blur detection map. We evaluate the proposed network on two commonly used defocus blur detection benchmark datasets by comparing it with other 11 state-of-the-art methods. Extensive experimental results with ablation studies demonstrate that R2MRF consistently and significantly outperforms the competitors in terms of both efficiency and accuracy.

Modern stream-data processing platforms are required to execute processing pipelines over high-volume, yet high-velocity, datasets under tight latency constraints. Apache Flink has emerged as an important new technology of large-scale platform that can distribute processing over a large number of computing nodes in a cluster (i.e., scale-out processing). Flink allows application developers to design and execute queries over continuous raw-inputs to analyze a large amount of streaming data in a parallel and distributed fashion. To increase the throughput of computing resources in stream processing platforms, a service provider might be tempted to use a consolidation strategy to pack as many processing applications as possible on the working nodes, with the hope of increasing the total revenue by improving the overall resource utilization. However, there is a hidden trap for achieving such a higher throughput solely by relying on an interference-oblivious consolidation strategy. In practice, collocated applications in a shared platform can fiercely compete with each others for obtaining the capacity of shared resources (e.g., cache and memory bandwidth) which in turn can lead to a severe performance degradation for all consolidated workloads.This paper addresses the shared resource contention problem associated with the auto-resource controlling mechanism of Apache Flink engine running across a distributed cluster. A controlling strategy is proposed to handle scenarios in which stream processing applications may have different quality of service (QoS) requirements while the resource interference is considered as the key performance-limiting parameter. The performance evaluation is carried out by comparing the proposed controller with the default Flink resource allocation strategy in a testbed cluster with total 32 Intel Xeon cores under different workload traffic with up to 4000 streaming applications chosen from various benchmarking tools. Experimental results demonstrate that the proposed controller can successfully decrease the average latency of high priority applications by 223% during the burst traffic while maintaining the requested QoS enforcement levels.

ARM processors, with their low power consumption and heat dissipation, have been highly successful in embedded systems. In the recent past, there have been attempts to adopt these energy-efficient processors for servers in data centers. However, a fundamental question remains open with ARM-based systems on server side is whether they are capable of handling compute-intensive workloads at scale. This paper gives our answer to this question with an empirical approach. We study the performance characteristics of the Amazon Graviton Processor - an ARM64 processor with the Cortex-A72 micro-architecture - using the A1 (Graviton) product family on AWS EC2, with comparisons to the I3 and M5 product families based on Intel Xeon processors. We use a combination of micro benchmark and performance counters to identify the lack of L3 cache and the slower memory access speed limit Graviton's capability in achieving higher performance. We confirm Graviton's capability in handling various large-scale horizontally scalable compute-intensive workloads, including multi-tier web service, video transcoding and terabyte scale sorting. In our large-scale evaluations, the test worker fleet has up to 1600 vCPU cores, which is by far the largest ARM64 cluster that has been reported. We observe that the A1 product family achieves the same price-performance in multi-tier web service, up to 37% cost saving in video transcoding, and up to 65% cost saving in terabyte scale sorting, as compared with the I3 and M5 product families.

Recently, Deep Learning (DL) is widely applied to intelligent systems equipped with resource-constraint hardware accelerators. With multiple DL applications sharing the resource, the execution model can be divided into two stages: (i) batching independent inference tasks initiated by each application, and (ii) scheduling batches to run in a time-sharing manner. The state-of-the-art DL serving systems employ the execution model by organizing sequential tasks into batches and then scheduling batches concerning their targeting deep neural network (DNN) models in a round-robin manner. However, we demonstrated that these practices fail to alleviate the slowdown of tasks, and there is a need to re-visit batching and scheduling in terms of efficiency and fairness. To this end, we formulated batching as a resource allocation problem and investigated scheduling in terms of each application's utilization on the device. Then, we proposed the fine-grained batching scheme and fairness-driven scheduling scheme for DL serving and implemented a prototype system called DyBatch. To be exact, DyBatch accomplishes efficient batching by taking into account Pareto efficiency of and envy between batches. Besides, DyBatch's fair scheduler monitors the resource utilization of all applications and assigns a batch from the application with the lowest utilization for execution first. Evaluation under various benchmarks with comparison to the baseline system Tensorflow Serving (TFS) shows the superiority of DyBatch, which achieves up to 55% reduction of slowdown, and up to 12% improvement of throughput.

As more analytical applications today require real-time processing over high volume data streams, finding an optimal implementation of traditional algorithms which possess iterative computations are gaining popularity and become crucial in most commercial contexts, particularly in edge processing and cloud applications. In this work, we propose an auto-tuning mechanism for enhancing the run-time performance of real-world iterative and cyclic stream processing applications (Multi-Join Operation as the study case) to correctly adjust the right performance bounds for workloads with different characteristics and data-sizes running on modern streaming data processing platform.

Edge computing promises a considerable reduction in latency and data volume by placing edge frameworks near the data source. Stream processing framework is a critical use case for edge computing, presenting in-situ data processing and low latency. However, careful CPU resource control is more important than cloud computing to provide effective multi-tenancy in resource-constrained edge computing. In this work, we investigate practical ways of controlling CPU resources of stream processing frameworks. Evaluation results from the different stream processor parallelism and cgroup parameters give a clear direction of designing an essential controller for stream processing frameworks in edge computing.

Internet of Things (IoT) allows the creation of virtually endless connections into a global array of distributed intelligence. However, the design, development, and deployment of IoT applications are complex and complicated due to various unwarranted challenges. For instance, addressing the IoT application users' subjective and objective opinions with IoT workflow instances remains a challenge for the design of a more holistic approach. Moreover, the complexity of IoT applications increased exponentially due to the heterogeneous nature of the Edge/Cloud services, utilised with the aim of lowering latency in data transformation and increase re-usability. Hence, in this paper, we present an IoT workflow composition system (IoTWC) to allow IoT users to pipeline their workflows with proposed IoT workflow activity abstract patterns. IoTWC leverages the analytic hierarchy process (AHP) to compose the multi-level IoT workflow that satisfies the requirements of any IoT application. Moreover, the users are befitted with recommended IoT workflow configurations using an AHP based multi-level composition framework. The proposed IoTWC is validated on a user case study to evaluate the coverage of IoT workflow activity abstract patterns and a real-world scenario for smart buildings. The comprehensive analysis shows the effectiveness of IoTWC in terms of IoT workflow abstraction and composition.

The post-discharge support is increasingly suggested for stroke patients to be discharged earlier and start rehabilitation at home. Considering that stroke patients usually have a high chance of recurrence, a good prognostic program is essential to improve diagnostic capabilities while reducing readmission rate to further save medical sources. In this context, various machine learning methods have been leveraged to obtain diagnostic findings and guide further treatments. However, those approaches mainly focus on performing analysis using a single data source obtained from the hospital, which could ignore the information complementarity between different groups of features and several subtle and discrete differences of physical interpretation among them. In this paper, we propose an Edge-based system design for post-stroke surveillance and warning prediction, called PSMART (Post-Stroke Mobile Auxiliary Rudiment Treatment), for processing enriched pathogenic factors of ischemic stroke from multi-sensors (views) to make readmission warning predictions. Our approach can considerably enrich the distinctive features from raw data, as well as exploit the consistency and complementary proprieties of different views, leading to better learning results. We evaluate the performance of the proposed approach on a real-world dataset, and the accuracy can reach up to 98.98%. Moreover, experiment results also show that our proposed approach can provide better accuracy when compared to the single-view ones.

With the Internet of Things continuously penetrating into all spheres of our daily lives, the increasing use of smart devices enabled the emergence of the edge computing paradigm. To meet the needs of saving energy and reducing electricity bills for each household, solar energy is exploited by using photovoltaic (PV) panels that can be integrated into an edge computing platform based on a cost-effective scheduling scheme. However, it is still a major challenge to determine the optimal energy allocation of renewable energy due to the intermittent nature of renewable energy generation. In this paper, we propose a unified clustering-based prediction framework with two tree-based algorithms to provide short-term prediction of PV power output. We also provide the in-terpretability analysis for our approach to reveal the features that are important for the prediction. The experimental results show our proposed framework is superior to other benchmark machine learning algorithms.

Federated Learning (FL) is a fast-developing distributed machine learning technique involving the participation of a massive number of user devices. While FL has benefits of data privacy and the abundance of user-generated data, its challenges of heterogeneity across users’ data and devices complicate algorithm design and convergence analysis. To tackle these challenges, we propose an algorithm that exploits proximal stochastic variance reduced gradient methods for non-convex FL. The proposed algorithm consists of two nested loops, which allow user devices to update their local models approximately up to an accuracy threshold (inner loop) before sending these local models to the server for global model update (outer loop). We characterize the convergence conditions for both local and global model updates and extract various insights from these conditions via the algorithm’s parameter control. We also propose how to optimize these parameters such that the training time of FL is minimized. Experimental results not only validate the theoretical convergence but also show that the proposed algorithm outperforms existing Stochastic Gradient Descent-based methods in terms of convergence speed in FL setting.

A scheduler is essential for resource management in a shared computer cluster, particularly scheduling algorithms play an important role in meeting service level objectives of user applications in large scale clusters that underlie cloud computing. Traditional cluster schedulers are often based on empirical observations of patterns of jobs running on them. It is unclear how effective they are for capturing the patterns of a variety of jobs in clouds. Recent advances in Deep Reinforcement Learning (DRL) promise a new optimization framework for a scheduler to systematically address the problem. A DRL-based scheduler can extract detailed patterns from job features and the dynamics of cloud resource utilization for better scheduling decisions. However, the deep neural network models used by the scheduler might be vulnerable to adversarial attacks. There is limited research investigating the vulnerability in DRL-based schedulers. In this paper, we give a white-box attack method to show that malicious users can exploit the scheduling vulnerability to benefit certain jobs. The proposed attack method only requires minor perturbations job features to significantly change the scheduling priority of these jobs. We implement both greedy and critical path based algorithms to facilitate the attacks to a state-of-the-art DRL based scheduler called Decima. Our extensive experiments on TPC-H workloads show a 62% and 66% success rate of attacks with the two algorithms. Successful attacks achieve a 18.6% and 17.5% completion time reduction.

With the continuous growth in population and energy demands more attention has been paid to energy consumption issues in residential environments. At the user-end, the home energy management system (HEMS) has been proposed as a cost-effective solution to reduce the electricity cost in households, while maintaining users' comfort and reducing the pressure on energy providers. However, it is a challenge to design a cost-effective scheduling strategies for HEMS which takes many objectives into consideration while potentially benefiting both users and providers. In our work, we propose a new approach named adaptive multi-objective salp swarm algorithm (AMSSA) based on traditional multi-objective salp swarm algorithm (MSSA) to realise a multi-objective optimisation approach for the power scheduling problem. AMSSA not only fulfils the trade-off among users' comfort, electricity cost and peak to average ratio (PAR), but also enhances the convergence speed for the overall optimisation process. Moreover, we also set up a testbed by using smart appliances and implemented our design on an edge-based energy management system. The experiment results demonstrated a reduction in both electricity cost (47.55%) and PAR (45.73%), compared with the case without a scheduling scheme.

Tag searching is a fundamental problem for a variety of radio frequency identification (RFID) applications. Prior works focus on single group searching, which refers to determining which ones in a given set of tags exist in the system. In this paper, we propose PTS, a protocol that can perform fast Parallel Tag Searching for multiple users simultaneously. Different from prior works that have to execute k times to search k groups separately, PTS obtains searching results for all the k groups with only one-shot execution. PTS achieves high parallelism due to some novel designs. First, we develop a grouping filter that encodes the membership of tags in different groups, with which non-target tags can be efficiently filtered out for all the k groups simultaneously. Second, we design two new codes, grouping code and mapping code, with which the remaining tags can quickly verify and confirm which group they belong to. We theoretically analyze how to set optimal parameters for PTS to minimize the execution time and conduct extensive simulation experiments to evaluate its performance. Compared with the state-of-the-art solutions, PTS significantly improves time efficiency in multiple group searching scenarios (by a factor of up to 7.54X when k = 10) and achieves the same time efficiency in single group searching scenarios.

To meet the needs for energy savings in Internet of Things (IoT) and mobile systems, solar energy has been increasingly exploited to serve as a green and renewable source to allow systems to better operate in an energy-efficient way. In this respect, accurate photovoltaics (PV) power output prediction is a prerequisite for any energy saving scheme employed in these systems [1].

Devising an elastic resource allocation controller of data analytical applications in virtualized data-center has received a great attention recently, mainly due to the fact that even a slight performance improvement can translate to huge monetary savings in practical large-scale execution. Apache Flink is among modern streamed data processing run-times that can provide both low latency and high throughput computation in to execute processing pipelines over high-volume and high-velocity data-items under tight latency constraints. However, a yet to be answered challenge in a large-scale platform with tens of worker nodes is how to resolve the run-time violation in the quality of service (QoS) level in a multi-tenant data streaming platforms, particularly when the amount of workload generated by different users fluctuates. Studies showed that a static resource allocation algorithm (round-robin), which is used by default in Apache Flink, suffer from lack of responsiveness to sudden traffic surges happening unpredictably during the run-time. In this paper, we address the problem of resource management in a Flink platform for ensuring different QoS enforcement levels in a platform with shared computing resources. The proposed solution applies theoretical principals borrowed from close-loop control theory to design a CPU and memory adjustment mechanism with the primary goal to fulfill the different QoS levels requested by submitted applications while the resource interference is considered as the critical performance-limiting factor. The performance evaluation is carried out by comparing the proposed resource allocation mechanism with two static heuristics (round robin and class-based weighted fair queuing) in a 80-core cluster under multiple traffic patterns resembling sudden changes in the incoming workloads of low-priory streaming applications. The experimental results confirm the stability of the proposed controller to regulate the underlying platform resources to smoothly follow the target values (QoS violation rates). Particularly, the proposed solution can achieve higher efficiency compared to the other heuristics by reducing the response-time of high priority applications by 53% while maintaining the enforced QoS levels during the burst traffic periods.

Time-Slotted Channel Hopping (TSCH) is defined in the IEEE 802.15.4e standard as a share medium access control technology to address reliability and timeliness requirements of low-power Internet of Things (IoT) applications. While standards define mechanisms for the basic configuration and communication of TSCH nodes, the adaptation of the TSCH schedule to traffic dynamics has been left as an open research problem. In this poster, we propose an Optimized Adaptive TSCH Scheduling Function (OA-TSCH) to dynamically adjust the TSCH schedule to the changes in the data traffic loads. We implement OA-TSCH on Zolerita Firefly IoT motes and the Contiki-NG operating system to evaluate its performance. Evaluation results show that our proposed scheduling function can improve the packet delivery ratio and throughput significantly.

Secure Device-to-Device (D2D) communication is becoming increasingly important with the ever-growing number of Internet-of-Things (IoT) devices in our daily life. To achieve secure D2D communication, the key agreement between different IoT devices without any prior knowledge is becoming desirable. Although various approaches have been proposed in the literature, they suffer from a number of limitations, such as low key generation rate and short pairing distance. In this paper, we present an inaudible acoustic signal based key generation protocol for mobile devices. Based on acoustic channel reciprocity, our system exploits channel frequency response of two legitimate devices as a common secret to generate keys. Extensive experiments are conducted to evaluate the proposed system in different real environments. Evaluation results show that the proposed system can generate the same secret key for two mobile devices with high probability.

Along with the rapid developments in communication technologies and the surge in the use of mobile devices, a brand-new computation paradigm, edge computing, is surging in popularity. Meanwhile, the artificial intelligence (AI) applications are thriving with the breakthroughs in deep learning and the many improvements in hardware architectures. Billions of data bytes, generated at the network edge, put massive demands on data processing and structural optimization. Thus, there exists a strong demand to integrate edge computing and AI, which gives birth to edge intelligence. In this article, we divide edge intelligence into AI for edge (intelligence-enabled edge computing) and AI on edge (artificial intelligence on edge). The former focuses on providing more optimal solutions to key problems in edge computing with the help of popular and effective AI technologies while the latter studies how to carry out the entire process of building AI models, i.e., model training and inference, on the edge. This article provides insights into this new interdisciplinary field from a broader perspective. It discusses the core concepts and the research roadmap, which should provide the necessary background for potential future research initiatives in edge intelligence.

The power efficiency of photovoltaic (PV) modules is highly correlated with their health status. Under dynamically changing environments, PV defects could spontaneously form and develop into fatal faults during the daily operation of PV power plants. To facilitate defect detection with less human intervention, a nondestructive, contactless, and automatical visual inspection system with the help of unmanned aerial vehicles and edge computing is proposed in this article. During the processing of the incoming data stream, the system may collect some new, unknown, and unlabeled defects that have not been identified yet in the existing database. To distinguish them from the existing ones, a deep embedded restricted cluster algorithm is designed to identify the unknown and unlabeled PV module defects in an unsupervised manner. Limited by the resources of edge devices and the availability of images of PV defects for training, we developed an online solution combined with deep learning, data argumentation, and transfer learning to properly address the issues of running resource-hungry applications on edge devices and lack of training samples faced by the deep learning approaches used in the field. In addition, pointwise convolution layers are introduced into the network to reduce the parameters and the size of the model. With the reduction of the network depth of the deep convolutional neural network model and the features transferred from the learned defects, the resource consumption of our proposed approach is significantly reduced, and thus can be used on a wide range of edge devices to complete defect detection in a timely manner with high accuracy. The experimental results clearly demonstrate the practicality and effectiveness.

Network slicing, as a key 5G enabling technology, is promising to support with more flexibility, agility, and intelligence towards the provisioned services and infrastructure management. Fulfilling these tasks is challenging, as nowadays networks are increasingly heterogeneous, dynamic and large-dimensioned. This contradicts the dominant network slicing solutions that only customize immediate performance over one snapshot of the system in the literature. Instead, this paper first presents a two-stage slicing optimization model with time-averaged metrics to safeguard the network slicing in the dynamical networks, where prior environmental knowledge is absent but can be partially observed at runtime. Directly solving an off-line solution to this problem is intractable since the future system realizations are unknown before decisions. Therefore, we propose a learning augmented optimization approach with deep learning and Lyapunov stability theories. This enables the system to learn a safe slicing solution from both historical records and run-time observations. We prove that the proposed solution is always feasible and nearly optimal, up to a constant additive factor. Finally, we demonstrate up to 2.6× improvement in the simulation when compared with three state-of-the-art algorithms.

Hosting virtualized network functions (VNF) has been regarded as an effective way to realize network function virtualization (NFV). Considering the cost diversity in cloud computing, from the perspective of service providers, it is significant to orchestrate the VNFs and schedule the traffic flows for network utility maximization (NUM) as it implies maximal revenue. However, traditional heuristic solutions based on optimization models usually follow some assumptions, limiting their applicability. Recent studies have shown that deep reinforcement learning (DRL) is a promising way to tackle such limitations. However, DRL agent training also suffers from slow convergence problem, especially with complex control problems. We notice that optimization models actually can be applied to accelerate the DRL training. Therefore, we are motivated to design a model-assisted DRL framework for VNF orchestration in this paper. Other than letting the agent blindly explore actions, the heuristic solutions are used to guide the training process. Based on such principle, the DRL framework is also redesigned accordingly. Experiment results validate the high efficiency of our model-assisted DRL framework as it not only converges 23× faster than traditional DRL algorithm, but also with higher performance at the same time.

Summary With the proliferation of Internet of Things (IoT) and edge computing paradigms, billions of IoT devices are being networked to support data‐driven and real‐time decision making across numerous application domains, including smart homes, smart transport, and smart buildings. These ubiquitously distributed IoT devices send the raw data to their respective edge device (eg, IoT gateways) or the cloud directly. The wide spectrum of possible application use cases make the design and networking of IoT and edge computing layers a very tedious process due to the: (i) complexity and heterogeneity of end‐point networks (eg, Wi‐Fi, 4G, and Bluetooth); (ii) heterogeneity of edge and IoT hardware resources and software stack; (iv) mobility of IoT devices; and (iii) the complex interplay between the IoT and edge layers. Unlike cloud computing, where researchers and developers seeking to test capacity planning, resource selection, network configuration, computation placement, and security management strategies had access to public cloud infrastructure (eg, Amazon and Azure), establishing an IoT and edge computing testbed that offers a high degree of verisimilitude is not only complex, costly, and resource‐intensive but also time‐intensive. Moreover, testing in real IoT and edge computing environments is not feasible due to the high cost and diverse domain knowledge required in order to reason about their diversity, scalability, and usability. To support performance testing and validation of IoT and edge computing configurations and algorithms at scale, simulation frameworks should be developed. Hence, this article proposes a novel simulator IoTSim‐Edge, which captures the behavior of heterogeneous IoT and edge computing infrastructure and allows users to test their infrastructure and framework in an easy and configurable manner. IoTSim‐Edge extends the capability of CloudSim to incorporate the different features of edge and IoT devices. The effectiveness of IoTSim‐Edge is described using three test cases. Results show the varying capability of IoTSim‐Edge in terms of application composition, battery‐oriented modeling, heterogeneous protocols modeling, and mobility modeling along with the resources provisioning for IoT applications.

An isolated power system (IPS) usually operates in an islanded mode. Because of the lack of support from an external power grid, stability is a prominent issue for IPSs. This article proposes a novel stability analysis approach for IPSs based on the input-to-state stability (ISS) theory. Compared with existing stability analyses that use simulations and direct methods, the proposed method decomposes the stability analysis process by considering the ISS properties of subsystems and a network equation that reflects their connections. Thus, it has good adaptability for the stability analysis of systems with flexible operating conditions. Algorithms are presented for estimating the ISS properties of subsystems, and asymptotic gains in a piecewise linear form are adopted. The small gain theorem is used for the decomposed stability analysis, and a practical algorithm to numerically check the small gain condition is presented. Time-domain simulations were performed with a test system to verify the effectiveness of the proposed decomposition-based stability analysis approach.Note to Practitioners—Power systems used in shipboards, airplanes, remote areas, and so on are usually classified as isolated power systems (IPSs). The continuity of power supply in IPSs is the prerequisite of fulfilling certain tasks. Due to the lack of support from the bulk power grid, the normal operation of IPSs can be threatened by various external disturbances, such as disasters, battle damages, device failures, and so on. To maintain the survivability and reliability of IPSs under extreme conditions, fast reconfiguration and emergency control approaches are often performed, which lead to system topology changes and frequent connection/disconnection operation of devices in IPSs. Because of the limited generation capacity of an IPS, a stability analysis after an emergency is important for ensuring that the IPS can perform tasks normally, and can provide guidance for designing fast reconfiguration and emergency control strategies. However, current stability analysis approaches have limited applicability or are overly conservative for analyzing the stability of IPSs. To address the challenge of changeable topologies for the stability analysis of an IPS, this article proposes a decomposition-based analysis approach using input-to-state stability (ISS) theory. By decomposing the entire system into several subsystems, the system’s stability can be checked through the ISS properties of subsystems and their connections. The ISS properties of subsystems can be estimated offline, which saves time for online calculation. To reduce the conservativeness of stability analysis, the asymptotic gains in piecewise linear form are adopted in this article. Practical algorithms are designed for efficiently checking the proposed decomposition-based stability conditions. The research outcome provides a fast and flexible stability analysis approach that can adapt to the frequent changes in the operating conditions of IPSs.

In a smart grid distribution management system, operation, planning, forecasting and decision making relies on demand-side management functions, which require real-time smart grid data. This data has significant dollar value because it is extremely useful for efficient control and intelligent prediction of the energy consumption, and expert management of residential and commercial load. However, the huge amount of (smart grid) data generated at a very high velocity poses a number of challenges. Utility companies have a huge demand for efficient summarization techniques to mine interesting patterns and extracting useful and actionable intelligence. Research from various domains has shown that data summarization can significantly improve the scalability and efficiency of various data analytic tasks (e.g., transactional database mining, data streams mining, network monitoring). This paper proposes a summarization approach (i.e., a set of algorithms, data structures, and query mechanisms) that enables the utility company to accurately infer various energy consumption patterns in real-time by automatic monitoring of smart grid data using significantly less computational resources. The proposed summarization approach is suitable for processing spatiotemporal streams, and it can also provide answers in real-time to various smart grid applications (e.g., demand-side management, direct load control, smart pricing and Volt-VAr control). Both theoretical bound and experimental evaluation are presented in this paper, which shows that the memory required for the proposed data structure grows linearly for the first 52 weeks; but interestingly, after the first year, the memory growth is negligible. The experimental results show that the proposed approach can process around 4 million smart meter readings every second or 120 million readings every minute. The proposed approach outperforms widely commercially used Database Management Systems (DBMSs) in terms of update and query costs: it is about 200 times faster than DBMSs in terms of update time, and about 340 times faster than DBMSs in terms of query time.

Network function virtualization (NFV) is a promising solution to realize a variety of network services. By definition, virtual network functions (VNFs) are chained together to realize different services. However, chaining is not an ideal solution as service latency grows linearly with respect to the length of the chain. Motivated by the fact that many VNFs can be parallelized, we investigate parallelism of VNFs for acceleration. The dependency of the VNFs is characterized by a directed acyclic graph (DAG). We aim to deploy the VNFs in the right place and process them in parallel without violating the DAG, to minimize the overall delay. However, directly solving the delay minimization problem is NP-hard, and it may also introduce a large number of duplicated packets to burden the system. To deal with these issues, we propose the Prune and Plant (P&P) scheme with polynomial computational complexity, to reduce the overall delay while limiting the number of duplicated packets. P&P comprises two stages: in the Prune stage, we prune the original DAG into a series-parallel graph (SP-graph), which eliminates NP-hardness while maintaining parallelism of VNFs. In the Plant stage, we find the optimal placement for the VNFs with respect to the SP-graph. By both simulation and prototyping, we demonstrate that P&P significantly outperforms benchmark schemes.

The pervasive use of virtualization techniques in today's datacenters poses challenges in power monitoring since it is not possible to directly measure the power consumption of a virtual entity such as a virtual machine (VM) and a container. In this paper, we present cWatts++, a lightweight virtual power meter that enables accurate power usage measurement in virtualized computing environments such as VMs and containers of Cloud data centers. At the core of cWatts++ is its application-agnostic power model. To this end, we devise two power models (eventModel and raplModel) that are driven by CPU event counters and the Running Average Power Limit (RAPL) feature of modern Intel CPUs, respectively. While eventModel is more generic and, thus, applicable to a wide range of workloads, raplModel is particularly good for CPU-bound workloads. We have evaluated cWatts++ with its two power models in a real system using the PARSEC benchmark suite and our in-house benchmarks. Our evaluation study demonstrates that these power models have an average error of 4.55 and 1.25 percent, respectively, compared with actual power usage measurements of a real power meter, Cabac Power-Mate.

Enormous energy consumption of data centers has a major impact on power systems by significantly increasing the electrical load. Due to the increase in electrical load, power systems are facing demand and supply miss-management problems. Therefore, power systems require efficient and intelligent ancillary services to maintain robustness, reliability, and stability. Data centers can provide the computational capabilities to manage power systems; however, data centers consume a tremendous amount of energy, and energy price accounts for a significant portion of their operational cost. Power system jobs will make this situation even more critical for data centers. In our work, we seek an Ancillary Services Model (ASM) to service data centers and power systems. In ASM, we find an optimal job scheduling technique for executing power systems' jobs on data centers in terms of low power consumption, reduced makespan, and fewer preempted jobs. The power systems' jobs include Optimal Power Flow (OPF) calculation, transmission line importance index, and bus importance index. Moreover, a Service Level Agreement (SLA) between data centers and power systems is shown to provide mutual benefits.

MapReduce is a crucial framework in the cloud computing architecture, and is implemented by Apache Hadoop and other cloud computing platforms. The resources required for executing jobs in a large data center vary according to the job types. In general, there are two types of jobs, CPU-bound and I/O-bound, which require different resources but run simultaneously in the same cluster. The default job scheduling policy of Hadoop is first-come-first-served and therefore, may cause unbalanced resource utilization. Considering various job workloads, numerous job allocation schedulers were proposed in the literature. However, those schedulers encountered the data locality problem or unreasonable job execution performance. This study proposes a job scheduler based on a dynamic grouping integrated neighboring search strategy, which can balance the resource utilization and improve the performance and data locality in heterogeneous computing environments.

Cloud computing is now a well-adopted computing paradigm. With unprecedented scalability and flexibility, the computational cloud is able to carry out large scale computing tasks in parallel. The datacenter cloud is a new cloud computing model that uses multi-datacenter architectures for large scale massive data processing or computing. In datacenter cloud computing, the overall efficiency of the cloud depends largely on the workload scheduler, which allocates clients' tasks to different Cloud datacenters. Developing high performance workload scheduling techniques in Cloud computing imposes a great challenge which has been extensively studied. Most previous works aim only at minimizing the completion time of all tasks. However, timeliness is not the only concern, reliability and security are also very important. In this work, a comprehensive Quality of Service (QoS) model is proposed to measure the overall performance of datacenter clouds. An advanced Cross-Entropy based stochastic scheduling (CESS) algorithm is developed to optimize the accumulative QoS and sojourn time of all tasks. Experimental results show that our algorithm improves accumulative QoS and sojourn time by up to 56.1 and 25.4 percent respectively compared to the baseline algorithm. The runtime of our algorithm grows only linearly with the number of Cloud datacenters and tasks. Given the same arrival rate and service rate ratio, our algorithm steadily generates scheduling solutions with satisfactory QoS without sacrificing sojourn time.

Service migration between datacenters can reduce the network overhead within a cloud infrastructure; thereby, also improving the quality of service for the clients. Most of the algorithms in the literature assume that the client access pattern remains stable for a sufficiently long period so as to amortize such migrations. However, if such an assumption does not hold, these algorithms can take arbitrarily poor migration decisions that can substantially degrade system performance. In this paper, we approach the issue of performing service migrations for an unknown and dynamically changing client access pattern. We propose an online algorithm that minimizes the inter-datacenter network, taking into account the network load of migrating a service between two datacenters, as well as the fact that the client request pattern may change “quickly”, before such a migration is amortized. We provide a rigorous mathematical proof showing that the algorithm is 3.8-competitive for a cloud network structured as a tree of multiple datacenters. We briefly discuss how the algorithm can be modified to work on general graph networks with an O(log|V|) probabilistic approximation of the optimal algorithm. Finally, we present an experimental evaluation of the algorithm based on extensive simulations.

Edge computing (EC) is now emerging as a key paradigm to handle the increasing Internet-of-Things (IoT) devices connected to the edge of the network. By using the services deployed on the service provisioning system which is made up of edge servers nearby, these IoT devices are enabled to fulfill complex tasks effectively. Nevertheless, it also brings challenges in trustworthiness management. The volatile environment will make it difficult to comply with the service-level agreement (SLA), which is an important index of trustworthiness declared by these IoT services. In this article, by denoting the trustworthiness gain with how well the SLA can comply, we first encode the state of the service provisioning system and the resource allocation scheme and model the adjustment of allocated resources for services as a Markov decision process (MDP). Based on these, we get a trained resource allocating policy with the help of the reinforcement learning (RL) method. The trained policy can always maximize the services' trustworthiness gain by generating appropriate resource allocation schemes dynamically according to the system states. By conducting a series of experiments on the YouTube request dataset, we show that the edge service provisioning system using our approach has 21.72% better performance at least compared to baselines.

Industrial networks are complex and diverse. Among existing intrusion prevention systems available, several of them have problems such as low detection accuracy rate, high false positive (FP) rate, and low real-time performance for impersonation attacks. To address such issues, it is proposed in this article an industrial network intrusion detection algorithm based on multifeature data clustering optimization model, where the weighted distances and security coefficients of data are classified based on the priority threshold of data attribute feature for each node in the network, given that the data modules in the industrial network environment are diverse and easy to diagnose, restore, and rebuild. The proposed algorithm can effectively improve the detection rate and real-time performance of detecting abnormal behavior for the multifeature data in industrial networks. The novel features are twofold, to rapidly select a node with high-security coefficient as the cluster center, and match the multifeature data around the center into a cluster. Experimental results show that the proposed algorithm has good superiority in terms of detection rate and time compared to other algorithms. In the industrial network, the detection accuracy of abnormal data reaches 97.8%, and the FP of detection is decreased by 8.8%.

Workflow scheduling is one of the most difficult tasks due to the variation in the traffic flows generated from diverse cloud applications. Hence, in this article, a container-based virtualization is used to design an energy-efficient workflow scheduling in software-defined data centers. The containers provide the flexibility to the applications to access the underlying resource as per their requirements. Moreover, a runtime scheduler is responsible to handle all the scheduling decisions in the proposed workflow scheduling scheme. Even more, a doubly linked list-based access mechanism is used to provide access to the servers and virtual machines by traversing both ways. Finally, a hashing scheme is used to select an ideal location for the allocation of the containers. The proposed scheme is evaluated with respect to different performance metrics (makespan, execution time, fault tolerance, energy consumption, etc.) on the real data traces. The results obtained depict the superiority of the proposed scheme in comparison to the other existing schemes of its category.

The papers in this special section focus on intelligent informatics for the edge of things in smart industrial ecosystems. In the recent years, Internet of Thing (IoT) has been widely deployed in numerous areas ranging from the development of smart cities and smart homes, smart grid, smart vehicles, smart health to the smart manufacturing and industrial management. By investigating and collecting huge amounts of data in an intelligent manner, these smart systems can improvise the decision making, business flows, automate industrial control processes, production, and economic results. With this motivation, IoT has made way into every corner of modern smart industrial ecosystem and economy.

Expectation maximization (EM) is a clustering-based machine learning algorithm that is widely used in many areas of science (e.g., bioinformatics and computer vision) to find maximum likelihood and maximum a posteriori estimates for models with latent variables. To deploy such an algorithm in cloud environments, security and privacy issues need be considered to avoid data breaches or abuses by external malicious parties or even by cloud service providers. However, the processing performance of the EM algorithm poses a challenge in terms of building a secure environment. This article describes an innovative and practical privacy-preserving EM algorithm for cloud systems that addresses this challenge, and estimates the EM parameters in an accurate and secure manner. Fully homomorphic encryption (FHE) is used to ensure the privacy of both the EM algorithm computations and the users' sensitive data in the cloud. A distributed-based approach is also proposed to overcome the overheads of FHE computations and ensure a fast convergence of the EM algorithm. The conducted experiments demonstrate a significant improvement in the convergence time of the distributed EM algorithm, while achieving a high level of accuracy and reducing the associated computational FHE overheads.

Resource provisioning for cloud computing necessitates the adaptive and accurate prediction of cloud workloads. However, the existing methods cannot effectively predict the high-dimensional and highly-variable cloud workloads. This results in resource wasting and inability to satisfy service level agreements (SLAs). Since recurrent neural network (RNN) is naturally suitable for sequential data analysis, it has been recently used to tackle the problem of workload prediction. However, RNN often performs poorly on learning long-term memory dependencies, and thus cannot make the accurate prediction of workloads. To address these important challenges, we propose a deep Learning based Prediction Algorithm for cloud Workloads (L-PAW). First, a top-sparse auto-encoder (TSA) is designed to effectively extract the essential representations of workloads from the original high-dimensional workload data. Next, we integrate TSA and gated recurrent unit (GRU) block into RNN to achieve the adaptive and accurate prediction for highly-variable workloads. Using real-world workload traces from Google and Alibaba cloud data centers and the DUX-based cluster, extensive experiments are conducted to demonstrate the effectiveness and adaptability of the L-PAW for different types of workloads with various prediction lengths. Moreover, the performance results show that the L-PAW achieves superior prediction accuracy compared to the classic RNN-based and other workload prediction methods for high-dimensional and highly-variable real-world cloud workloads.

Multi-dimensional, large-scale, and sparse data, which can be neatly represented by sparse tensors, are increasingly used in various applications such as data analysis and machine learning. A high-performance sparse tensor-vector product (SpTV), one of the most fundamental operations of processing sparse tensors, is necessary for improving efficiency of related applications. In this article, we propose aeSpTV, an adaptive and efficient SpTV framework on Sunway TaihuLight supercomputer, to solve several challenges of optimizing SpTVon high-performance computing platforms. First, to map SpTV to Sunway architecture and tame expensive memory access latency and parallel writing conflict due to the intrinsic irregularity of SpTV, we introduce an adaptive SpTV parallelization. Second, to co-execute with the parallelization design while still ensuring high efficiency, we design a sparse tensor data structure named CSSoCR. Third, based on the adaptive SpTV parallelization with the novel tensor data structure, we present an autotuner that chooses the most befitting tensor partitioning method for aeSpTV using the variance analysis theory of mathematical statistics to achieve load balance. Fourth, to further leverage the computing power of Sunway, we propose customized optimizations for aeSpTV. Experimental results show that aeSpTV yields good sacalability on both thread-level and process-level parallelism of Sunway. It achieves a maximum GFLOPS of 195.69 on 128 processes. Additionally, it is proved that optimization effects of the partitioning autotuner and optimization techniques are remarkable.

Serverless computing has emerged as a new cloud computing execution model that liberates users and application developers from explicitly managing `physical' resources, leaving such a resource management burden to service providers. In this article, we study the problem of resource allocation for multi-tenant serverless computing platforms explicitly taking into account workload fluctuations including sudden surges. In particular, we investigate different root causes of performance degradation in these platforms where tenants (their applications) have different workload characteristics. To this end, we develop a fine-grained CPU cap control solution as a resource manager that dynamically adjusts CPU usage limit (or CPU cap) concerning applications with same/similar performance requirements, i.e., application groups. The adjustment of CPU caps applies primarily to co-located worker processes of serverless computing platforms to minimize resource contention, which is the major source of performance degradation. The actual adjustment decisions are made based on performance metrics (e.g., throttled time and queue length) using a group-aware scheduling algorithm. The extensive experimental results performed in our local cluster confirm that the proposed resource manager can effectively eliminate the burden of explicit reservation of computing capacity, even when fluctuations and sudden surges in the incoming workload exist. We measure the robustness of the proposed resource manager by comparing it with several heuristics which extensively used in practice, including the enhanced version of round robin and the least length queue scheduling policies, under various workload intensities driven by real-world scenarios. Notably, our resource manager outperforms other heuristics by decreasing skewness and average response time up to 44 and 94 percent, respectively, while it does not over-use the CPU resources.

We investigate a workflow scheduling problem with stochastic task arrival times and fuzzy task processing times and due dates. The problem is common in many real-time and workflow-based applications, where tasks with fixed stage number and linearly dependency are executed on scalable cloud resources with multiple price options. The challenges lie in proposing effective, stable, and robust algorithms under stochastic and fuzzy tasks. A triangle fuzzy number-based model is formulated. Two metrics are explored: the cost and the degree of satisfaction. An iterated heuristic framework is proposed to periodically schedule tasks, which consists of a task collection and a fuzzy task scheduling phases. Two task collection strategies are presented and two task prioritization strategies are employed. In order to achieve a high satisfaction degree, deadline constraints are defined at both job and task levels. By designing delicate experiments and applying sophisticated statistical techniques, experimental results show that the proposed algorithm is more effective and robust than the two existing methods.

Frequently occurred oil leaking accidents can induce significant damage to the ocean ecosystem and environment. Integrated buoy sensing, which functions as a tool for periodically monitoring oil existence, plays an essential role in oil leakage detection in an offshore petroleum Internet of Things (IoT) and Cyber Physical System (CPS). Buoy sensor cyberattack can severely affect the ability to detect the petroleum leakage and hence delay the pollution recovery process. Despite these, existing techniques seldom deal with attacks on buoy sensors and their impacts on marine oil spill detection. In this article, a Partially observable Markov decision process based Buoy Sensor Cyberattack detection (PBSC) technique is proposed. Proposed PBSC technique utilizing Partially Observable Markov Decision Process (POMDP) method, which is a stochastic process based on Markov decision process, to evaluate the cyberattack probability for each buoy sensors. Cyberattack probability is evaluated by cross entropy based oil simulation method. This technique can efficiently identify attacked sensors and locate the oil leaking sources, which facilitates future pollution recovery. Experimental results from a marine area in Shenzhen, China demonstrate that the proposed technique can improve the detection accuracy by up to 50 percent while ruining x6 faster than the state of art cyberattack techniques.

Directed Acyclic Graph (DAG) has been emerging as the so-called Blockchain 3.0 after Bitcoin (Blockchain 1.0) and Ethereum (Blockchain 2.0). This new distributed ledger technology is getting significant attention for its high performance and low transaction fee. There have already been several notable implementations, such as IOTA [1], Nano [2] and Byteball [3]. In this paper, we present DAGBENCH as a performance evaluation framework for DAG implementations. DAGBENCH provides a number of sample workloads and adaptors that make effective and easy evaluation of different DAG implementations. It allows any DAG implementation to be evaluated by adding an adaptor. DAGBENCH allows to measure the performance of DAG implementation in terms of throughput, latency, scalability, success indicator, resource consumption, transaction data size and transaction fee. We demonstrate the efficacy of DAGBENCH with different DAG implementations. In particular, we have conducted experiments, on Amazon EC2, with three popular DAG implementations: IOTA, Nano and Byteball. Our experimental results provide the performance comparison between these implementations that helps developers/users effectively evaluate different performance characteristics; and, this enables them to identify bottlenecks and accordingly to improve performance.

In this paper, we present the challenges involved in large-scale video transcoding application in public clouds. We introduce the architecture of an existing video transcoding system which is tightly coupled with an existing video sharing service. We examine the horizontal scalability of the video transcoding system on AWS EC2. With an online transaction processing (OLTP) model, the system achieves linear horizontal scalability up to 1,000 vCPU cores, but starts to experience performance degradation beyond that. We analyze the resource consumption pattern of the existing system, then introduce an improved architecture by adding a message queue layer. This effectively decouples the video transcoding system from the video sharing service and converts the OLTP model into a batch processing model. Large-scale evaluations on AWS EC2 indicate that the improved design maintains linear horizontal scalability at 10,100 vCPU cores. The hybrid design of the system allows it to be easily adapted for other batch processing use cases without the need to modify or recompile the application.

Defocus blur detection aims to detect out-of-focus regions from an image. Although attracting more and more attention due to its widespread applications, defocus blur detection still confronts several challenges such as the interference of background clutter, sensitivity to scales and missing boundary details of defocus blur regions. To deal with these issues, we propose a deep neural network which recurrently fuses and refines multi-scale deep features (DeFusionNet) for defocus blur detection. We firstly utilize a fully convolutional network to extract multi-scale deep features. The features from bottom layers are able to capture rich low-level features for details preservation, while the features from top layers can characterize the semantic information to locate blur regions. These features from different layers are fused as shallow features and semantic features, respectively. After that, the fused shallow features are propagated to top layers for refining the fine details of detected defocus blur regions, and the fused semantic features are propagated to bottom layers to assist in better locating the defocus regions. The feature fusing and refining are carried out in a recurrent manner. Also, we finally fuse the output of each layer at the last recurrent step to obtain the final defocus blur map by considering the sensitivity to scales of the defocus degree. Experiments on two commonly used defocus blur detection benchmark datasets are conducted to demonstrate the superority of DeFusionNet when compared with other 10 competitors. Code and more results can be found at: http://tangchang.net.

In Software Defined Vehicular Networks (SDVNs), most existing studies of routing consider the vehicular network as a static graph and compute the flow table based on static information. However, a static graph could only contain partial network data. Routing computation based on the static graph could be inefficient because vehicular networks are temporal graphs. Thus, in this paper, we propose a novel routing algorithm based on the Markov model and the temporal graph. Unlike conventional routing algorithms, the proposed algorithm adopts the concept of the temporal graph where every edge has its specific temporal information. We apply the Markov model to predict the future routing of the network and adopt prediction data to get the optimal routing by running the temporal graph optimal path algorithm. A benefit of our proposal is, the proposed algorithm searched on the temporal graph of SDVNs can avoid generating additional routing overhead. Besides, based on the information of the vehicular network which is collected from the data plane, the controller can enhance the Markov model as time flows. By applying the above mechanisms, the flow table (route) could be calculated more precisely to enable efficient vehicular communication. The simulation experiments demonstrate the superiority of the proposed algorithm over its counterparts in high-density vehicular networks.

Edge computing is an effective approach for resource provisioning at the network edge to host virtualized network functions (VNF). Considering the cost diversity in edge computing, from the perspective of service providers, it is significant to orchestrate the VNFs and schedule the traffic flows for network utility maximization (NUM) as it implies maximal revenue. However, traditional model-based optimization methods usually follow some assumptions and impose certain limitations. In this paper, inspired by the success of deep reinforcement learning in solving complicated control problems, we propose a deep deterministic policy gradients (DDPG) based algorithm. We first formulate the NUM problem with the consideration of end-to-end delays and various operation costs into a non-convex optimization problem and prove it to be NP-hard. We then redesign the exploration method and invent a dual replay buffer structure to customize the DDPG. Meanwhile, we also apply our formulation to guide our replay buffer update. Through extensive trace-driven experiments, we show the high efficiency of our customized DDPG based algorithm as it significantly outperforms both model-based methods and traditional non-customized DDPG based algorithm.

With the increasing demand for data-driven decision making, there is an urgent need for processing geographically distributed data streams in real-time. The existing scheduling and resource management schemes efficiently optimize stream processing performance with the awareness of resource, quality-of-service, and network traffic. However, the correlation between network delay and inter-operator communication pattern is not well-understood. In this study, we propose a network-aware and partition-based resource management scheme to deal with the ever-changing network condition and data communication in stream processing. The proposed approach applies operator fusion by considering the computational demand of individual operators and the inter-operator communication patterns. It maps the fused operators to the clustered hosts with the weighted shortest processing time heuristic. Meanwhile, we established a 3-dimensional coordinate system for prompt reflection of the network condition, real-time traffic, and resource availability. We evaluated the proposed approach against two benchmarks, and the results demonstrate the efficiency in throughput and resource utilization. We also conducted a case study and implemented a prototype system supported by the proposed approach that aims to utilize the stream processing paradigm for pedestrian behavior analysis. The prototype application estimates walking time for a given path according to the real crowd traffic. The promising evaluation results of processing performance further illustrate the efficiency of the proposed approach.

Mobile edge computing has significantly reduced the response time of mobile applications by executing services in close proximity to mobile consumers. However, the capabilities and resources of edge servers are typically limited; additionally, service requirements in mobile environments are becoming increasingly complex and diverse. In this context, properly dispatching service requests to edge and cloud servers to improve the quality of services has become a crucial problem. In this paper, we focus on this problem and aim to minimize the response time of service invocations in mobile edge computing systems. The problem is formulated as an optimization problem, and a heuristic algorithm that combines the Genetic algorithm and the simulated Annealing algorithm for service selection in Mobile Edge Computing systems (GAMEC) is proposed to solve the problem. A series of experiments has demonstrated that the method performs well in reducing the response time of service invocations in mobile edge computing systems. Moreover, the execution time of GAMEC is of a low order of magnitude, and the algorithm scales well as the experimental scale increases.

Oral health has significant impact on people’s over-all well-being. While many activity recognition systems exist in the literature, accurately sensing toothbrushing activities remains an unsolved challenging problem due to the diversity of tooth-brushing habits among different users and subtle distinctions between different brushing actions. In this work, we propose Hygiea, an energy-efficient and highly-accurate toothbrushing monitoring system which exploits IMU-based wrist-worn gesture sensing using unmodified toothbrushes. To address toothbrushing variety, Hygiea incorporates a number of novel signal preprocessing techniques to automatically transform the sensory input during arbitrary toothbrushing activities to the consistent user coordinate system. To distinguish different brushing actions, Hygiea leverages an emerging deep learning model (e.g., AT-LSTM) to achieve fine-grained activity recognitions. Moreover, a POMDP model is incorporated for sampling control to balance activity detection and energy efficiency. Extensive real-world experiments show that the Hygiea system achieves a 11.7% accuracy gain compared to the state-of-the-art while maintaining energy-efficiency and zero modification on the toothbrushes.

There is an increasing interest in a new machine learning technique called Federated Learning, in which the model training is distributed over mobile user equipments (UEs), and each UE contributes to the learning model by independently computing the gradient based on its local training data. Federated Learning has several benefits of data privacy and potentially a large amount of UE participants with modern powerful processors and low-delay mobile-edge networks. While most of the existing work focused on designing learning algorithms with provable convergence time, other issues such as uncertainty of wireless channels and UEs with heterogeneous power constraints and local data size, are under-explored. These issues especially affect to various trade-offs: (i) between computation and communication latencies determined by learning accuracy level, and thus (ii) between the Federated Learning time and UE energy consumption. We fill this gap by formulating a Federated Learning over wireless network as an optimization problem FEDL that captures both trade-offs. Even though FEDL is non-convex, we exploit the problem structure to decompose and transform it to three convex sub-problems. We also obtain the globally optimal solution by charactering the closed-form solutions to all sub-problems, which give qualitative insights to problem design via the obtained optimal FEDL learning time, accuracy level, and UE energy cost. Our theoretical analysis is also illustrated by extensive numerical results.

Virtual machine (VM) migration is a widely used technique in cloud computing systems to increase reliability. There are also many other reasons that a VM is migrated during its lifetime, such as reducing energy consumption, improving performance, maintenance, etc. During a live VM migration, the underlying VM continues being up until all or part of its data has been transmitted from source to destination. The remaining data are transmitted in an off-line manner by suspending the corresponding VM. The longer the off-line transmission time, the worse the performance of the respective VM. The above is because during the off-line data transmission, the VM service is down. Because a running VM's memory is subject to changes, already transmitted data pages may get dirtied and thus needing re-transmission. The decision of when suspending the VM is not a trivial task at all. The above is justified by the fact that when suspending the VM early we may result in transmitting off-line a significant amount of data degrading thus the VM's performance. On the other hand, a long waiting time to suspend the VM may result in re-transmitting a huge amount of dirty data, leading in that way to waste of resources. In this paper, we tackle the joint problem of minimizing both the total VM migration time (reflecting the resources spent during a migration) and the VM downtime (reflecting the performance degradation). The aforementioned objective functions are weighted according to the needs of the underlying cloud provider/user. To tackle the problem, we propose an online deterministic algorithm resulting in an strong competitive ratio, as well as a randomized online algorithm achieving significantly better results against the deterministic algorithm.

Summary form only given. Recent technological trends such as Industry 4.0 introduced new challenges that push the limit of current computer and networking architectures. It demands the connection of thousands, if not millions, of sensors and mobile devices coupled with optimized operations to automate various operations inside factories. This led to the new era of Internet of Things (IoTs) where lightweight (possibly mobile) devices are envisaged to send vital information to cloud data centres (mobile and fixed infrastructure) for further processing and decision making. Current cloud computing systems, however, are not able to efficiently digest and process collected information from IoT devices with strict response requests for two main reasons: (1) the round trip delay between IoT devices to the processing engines of cloud could exceed an application's threshold, and (2) network links to cloud resources could be clogged when IoT devices flush data in an uncoordinated fashion. Fog and Edge Computing are two solutions to address both of the previous problems. Though designed to alleviate the same problem, they have fundamental differences that make adopting one more applicable than the other. This talk will overview the practical concerns of exploiting Edge Computing to realize today's IoT implementations through tackling the most important obstacles that hinder their adoption. First, production of applicable network (fixed and mobile) latency models to capture all elements of IoT platforms. Second, building a holistic Edge ecosystem to orchestrate various inter-related layers of IoT platforms, including connectivity, big-data analytics, and workload optimization. Third, proposing viable solutions that can be actually implemented in IoT-based applications. More details will be provided about the above issues during the talk.

In recent years, the rapid development of edge computing enables us to process a wide variety of intelligent applications at the edge, such as real-time video analytics. However, edge computing could suffer from service outage caused by the fluctuated wireless connection or congested computing resource. During the service outage, the only choice is to process the deep neural network (DNN) inference at the local mobile devices. The obstacle is that due to the limited resource, it may not be possible to complete inference tasks on time. Inspired by the recently developedearly exit of DNNs, where we can exit DNN at earlier layers to shorten the inference delay by sacrificing an acceptable level of accuracy, we propose to adopt such mechanism to process inference tasks during the service outage. The challenge is how to obtain the optimal schedule with diverse early exit choices. To this end, we formulate an optimal scheduling problem with the objective to maximize a general overall utility. However, the problem is in the form of integer programming, which cannot be solved by a standard approach. We therefore prove the Ordered Scheduling structure, indicating that a frame arrived earlier must be scheduled earlier. Such structure greatly decreases the searching space for an optimal solution. Then, we propose the Scheduling Early Exit (SEE) algorithm based on dynamic programming, to solve the problem optimally with polynomial computational complexity. Finally, we conduct trace-driven simulations and compare SEE with two benchmarks. The result shows that SEE can outperform the benchmarks by 50.9%.

This paper focuses on Timely dataflow programming model for processing streams of data. We propose a technique to define CPU resource allocation (i.e., CPU capping) with the goal to improve response time latency in such type of applications with different quality of service (QoS) level, as they are concurrently running in a shared multi-core computing system with unknown and volatile demand. The proposed solution predicts the expected performance of the underlying platform using an online approach based on queuing theory and adjusts the corrections required in CPU allocation to achieve the most optimized performance. The experimental results confirms that measured performance of the proposed model is highly accurate while it takes into account the percentiles on the QoS metrics. The theoretical model used for elastic allocation of CPU share in the target platform takes advantage of design principals in model predictive control theory and dynamic programming to solve an optimization problem. While the prediction module in the proposed algorithm tries to predict the temporal changes in the arrival rate of each data flow, the optimization module uses a system model to estimate the interference among collocated applications by continuously monitoring the available CPU utilization in individual nodes along with the number of outstanding messages in every intermediate buffer of all TDF applications. The optimization module eventually performs a cost-benefit analysis to mitigate the total amount of QoS violation incidents by assigning the limited CPU shares among collocated applications. The proposed algorithm is robust (i.e., its worst-case output is guaranteed for arbitrarily volatile incoming demand coming from different data streams), and if the demand volatility is not large, the output is optimal, too. Its implementation is done using the TDF framework in Rust for distributed and shared memory architectures. The experimental results show that the proposed algorithm reduces the average and p99 latency of delay-sensitive applications by 21% and 31.8%, respectively, while can reduce the amount of QoS violation incidents by 98% on average.

The rapid growth in the number of Internet of Things (IoT) devices has increased the demand for exploring high-throughput communications. Low power IoT networks perform poorly under heavy traffic due to severe congestion and high packet loss problems. Controlling the rate of traffic load is advocated as an effective way to reduce the congestion in traditional networks. However, it poses a major challenge to low power IoT networks due to the lack of infrastructure, dynamic changes of the network topology, and using multi-hop communication through unstable lossy wireless links. To overcome this problem, in this paper we propose an optimized Mobility-Aware Rate Adaptation (MARA) framework based on the game theory. We model the rate control problem as a non-cooperative game where IoT nodes compete for higher bandwidth as selfish players. Based on the Rosen's theorem for concave N-person games, we prove the existence and uniqueness of Nash equilibrium. Finding the optimal solution of the game is modeled as a nonlinear programming (NLP) problem which is solved by using Lagrange multipliers and Karush-Kuhn-Tucker (KKT) optimality conditions. MARA can effectively adapt the transmission rate of each node to the changes in the network topology, traffic dynamics, and energy resources. We implement MARA on Zolerita IoT motes and Contiki operating system to evaluate its performance. Emulation results show that MARA improves the packet delivery ratio by up to 42%, and reduces the end-to-end delay and the energy consumption by up to 32% and 30% respectively.

A typical scenario in a stream data-flow processing engine is that users submit continues queries in order to receive the computational result once a new stream of data arrives. The focus of the paper is to design a dynamic CPU cap controller for stream data-flow applications with real-time constraints, in which the result of computations must be available within a short time period, specified by the user, once a recent update in the input data occurs. It is common that the stream data-flow processing engine is deployed over a cluster of dedicated or virtualized server nodes, e.g., Cloud or Edge platform, to achieve a faster data processing. However, the attributes of incoming stream data-flow might fluctuate in an irregular way. To effectively cope with such unpredictable conditions, the underlying resource manager needs to be equipped with a dynamic resource provisioning mechanism to ensure the real-time requirements of different applications. The proposed solution uses control theory principals to achieve a good utilization of computing resources and a reduced average response time. The proposed algorithm dynamically adjusts the required quality of service (QoS) in an environment when multiple stream & data-flow processing applications concurrently run with unknown and volatile workloads. Our study confirms that such a unpredictable demand can negatively degrade the system performance, mainly due to adverse interference in the utilization of shared resources. Unlike prior research studies which assumes a static or zero correlation among the performance variability among consolidated applications, we presume the prevalence of shared-resource interference among collocated applications as a key performance-limiting parameter and confront it in scenarios where several applications have different QoS requirements with unpredictable workload demands. We design a low-overhead controller to achieve two natural optimization objectives of minimizing QoS violation amount and maximizing the average CPU utilization. The algorithm takes advantage of design principals in model predictive control theory for elastic allocation of CPU share. The experimental results confirm that there is a strong correlation in performance degradation among consolidation strategies and the system utilization for obtaining the capacity of shared resources in a non-cooperative manner. The results confirm that the proposed solution can reduce the average latency of delay-sensitive applications by 17% comparing to the results of a well established heuristic called Class-Based Weighted Fair Queuing (CFWFQ). At the same time, the proposed solution can prevent the QoS violation incidents by 62%.

With the increasing popularity of virtual machine monitoring (VMM) technologies, performance variability among collocated virtual machines (VMs) can easily become a severe scalability issue. Particularly, it becomes a necessary for administrative team to control the performance degradation level in a shared environment when multiple I/O-intensive applications simultaneously request their I/O operations [1]. Nevertheless, adding several logical layers between the running applications and the physical storage system, as seen in contemporary virtualized storage devices, makes it considerably difficult to build a low overhead controlling mechanism for such systems (while each VM may running a separate operating system instance) [2]. In this paper, we propose a strategy based on control theory for managing the performance of several I/O requests, such as mean response times and read/write throughput in a consolidated environment where multiple virtual services can share access to a storage system. This scheme uses an approach for measuring the characterization of read/write performance attributes of each virtual services and also takes into account the run-time quality of service enforcement levels requested by them. This is formulated as an optimization problem where a reward function is defined to reduce the overall QoS violation incidents among all consolidated virtual services. Performance evaluation is carried out by comparing the proposed solution with the default embedded Linux controller across a range of emulated application workloads in scenarios with multiple consolidated virtual containers. The results confirm that the proposed solution can reduce the overall QoS violation incident rates in scenarios in which the platform operates at a significant traffic load comparing to the default policy in LXC engine.

Containerized web-applications have gained popularity recently due to the advantages provided by the containers including light-weight, packaged, fast start up and shut down and easy scalability. As there are more than 267 cloud providers, finding a flexible deployment option for containerized web-applications is very difficult as each cloud offers numerous deployment infrastructure. Benchmarking is one of the eminent options to evaluate the provisioned resources before product-level deployment. However, benchmarking the massive infrastructure resources provisioned by various cloud providers is a time consuming, tedious and costly process and is not practical to accomplish manually.

Radio frequency identification (RFID)-based localization and activity recognition have attracted much research attention recently. They rely on accurate measurements of signal features, e.g., phase and received signal strength (RSS), in line-of-sight (LOS) condition to estimate the location or activity status of the target objects. However, the LOS requirement might be frequently breached by obstacles between reader and tags in real deployed RFID systems. The resulting non-LOS (NLOS) signal will greatly reduce localization or activity recognition accuracy. How to filter out NLOS in the localization/activity recognition process is therefore practically important for guaranteeing accuracy. In this paper, we propose the first LOS/NLOS path recognition approach to differentiate the signals by LOS path from the ones by NLOS path. The proposed approach is both precise (with precision higher than 0.95) and real-time in nature (with recognition delay less than 400 ms) due to the following innovative designs. First, we design a new metric that can precisely distinguish LOS and NLOS paths by considering the joint variance of phase and RSS. Second, we propose an efficient method to mitigate the negative impacts of phase ambiguity on recognition precision. Third, we sample over a selected subset of channels and use only a handful of readings to perform LOS/NLOS path recognition, which greatly reduces the recognition delay without sacrificing precision. We conducted extensive experiments with commercial-off-the-shelf RFID devices. The results show that our approach achieves high precision and recall in all testing cases, with a precision of up to 0.969 and a recall of up to 0.991. Furthermore, our approach can also distinguish between different types of obstacles with an accuracy as high as 0.93.

Edge computing is an emerging research area to incorporate cloud computing into edge network devices. An Edge datacenter, also referred to as EDC, processes data streams and user requests in real-time and is therefore used to decrease the latency and congestion in the network. EDC is usually setup as a distributed system and is accordingly placed between the cloud datacenter and the data source . These EDCs work as an intermediate layer in the fog hierarchy between IoT and Cloud datacenter. EDC’s are aided by load balancers, responsible for distributing the workload amongst multiple EDC, in order to optimize resource utilization and response time . The load balancers make sure that the workload is equally divided amongst the available EDCs to avoid over loading of some EDCs while other remain idle as this directly impacts the user response and real-time event detection . Given the fact that EDCs are deployed in remote environments, the need for secure authentication is of major importance. In this paper we propose a novel load balancing technique that enables EDC authentication as well as identification of idle EDCs for better load balancing. The proposed load balancing technique is also compared with existing approaches and proves to be more efficient in locating EDC’s with less workload. In addition to the improved efficiency, the proposed scheme also strengthens the security of the network by incorporating destination EDC authentication.

Network function virtualization (NFV) decouples the traditional network functions from specific or proprietary hardware, such that virtualized network functions (VNFs) can run in software form. By exploring NFV, a consecutive set of VNFs can constitute a service function chain (SFC) to provide the network service. From the perspective of network service providers, how to maximize the network utility is always one of the major concerns. To this end, there are two main issues need to be considered at runtime: 1) how to handle the unpredictable network traffic burst? and 2) how to fairly allocate resources among various flows to satisfy different traffic demands? In this paper, we investigate a fairness-aware flow scheduling problem for network utility maximization, with joint consideration of resource allocation and rate control. Based on a discrete-time queuing model, we propose a low-complexity online-distributed algorithm using the Lyapunov optimization framework, which can achieve arbitrary optimal utility with different fairness levels by tuning the fairness bias parameter. We theoretically analyze the optimality of the algorithm and evaluate its efficiency by both simulation and testbed-based experiments.

Electricity price forecasting is a significant part of smart grid because it makes smart grid cost efficient. Nevertheless, existing methods for price forecasting may be difficult to handle with huge price data in the grid, since the redundancy from feature selection cannot be averted and an integrated infrastructure is also lacked for coordinating the procedures in electricity price forecasting. To solve such a problem, a novel electricity price forecasting model is developed. Specifically, three modules are integrated in the proposed model. First, by merging of Random Forest (RF) and Relief-F algorithm, we propose a hybrid feature selector based on Grey Correlation Analysis (GCA) to eliminate the feature redundancy. Second, an integration of Kernel function and Principle Component Analysis (KPCA) is used in feature extraction process to realize the dimensionality reduction. Finally, to forecast price classification, we put forward a differential evolution (DE) based Support Vector Machine (SVM) classifier. Our proposed electricity price forecasting model is realized via these three parts. Numerical results show that our proposal has superior performance than other methods.

Smart Grid (SG) has emerged as one of the most powerful technologies of the modern era for an efficient energy management by integrating information and communication technologies (ICT) in the existing infrastructure. Among various ICT, cloud computing (CC) has emerged as one of the leading service providers which uses geo-distributed data centers (DCs) to serve the requests of users in SG. In recent times, with an increase in service requests by end users for various resources, there has been an exponential increase in the number of servers deployed at various DCs. With an increase in the size, the energy consumption of DCs has increased many folds which leads to an increase in overall operational cost of DCs. However, efficient resource allocation among these geo-distributed DCs may play a vital role in reducing the energy consumption of DCs. Moreover, with an increase in harmful emissions, the use of renewable energy sources (RES) can benefit DCs, SG, and society at large. Keeping focus on these points, in this paper, an energy-aware resource allocation scheme is proposed using a Stackelberg game for energy management in cloud-based DCs. For this purpose, a cloud controller is used to receive the requests of users which then distributes these requests among geo-distributed DCs in such a way that the energy consumption of DCs is sustained by RES. However, if energy consumption of DCs is not sustained by RES then the energy is drawn from the grid. The requests of users are routed to the DC which is offered lowest energy tariff from the grid. For this purpose, a Stackelberg game for energy trading is also proposed to select the grid offering lowest energy tariff to DCs. The proposed scheme is evaluated using various performance metrics using Google workload traces. The results obtained show the effectiveness of the proposed scheme.

There is a growing trend for employing cyber-physical systems to help smart homes improve the comfort of residents. However, a residential cyber-physical system is different from a common cyber-physical system since it directly involves human interaction, which is full of uncertainty. The existing solutions could be effective for performance enhancement in some cases when no inherent and dominant human factors are involved. Besides, the rapidly rising interest in the deployments of cyber-physical systems at home does not normally integrate with energy management schemes, which is a central issue that smart homes have to face. In this article, we propose a cyber-physical-system-based energy management framework to enable a sustainable-edge computing paradigm while meeting the needs of home energy management and residents. This framework aims to enable the full use of renewable energy while reducing electricity bills for households. A prototype system was implemented using real-world hardware. The experiment results demonstrated that renewable energy is fully capable of supporting the reliable running of home appliances most of the time and electricity bills could be cut by up to 60% when our proposed framework was employed.

Class labels are required for supervised learning but may be corrupted or missing in various applications. In binary classification, for example, when only a subset of positive instances is labeled whereas the remaining are unlabeled, positive-unlabeled (PU) learning is required to model from both positive and unlabeled data. Similarly, when class labels are corrupted by mislabeled instances, methods are needed for learning in the presence of class label noise (LN). Here we propose adaptive sampling (AdaSampling), a framework for both PU learning and learning with class LN. By iteratively estimating the class mislabeling probability with an adaptive sampling procedure, the proposed method progressively reduces the risk of selecting mislabeled instances for model training and subsequently constructs highly generalizable models even when a large proportion of mislabeled instances is present in the data. We demonstrate the utilities of proposed methods using simulation and benchmark data, and compare them to alternative approaches that are commonly used for PU learning and/or learning with LN. We then introduce two novel bioinformatics applications where AdaSampling is used to: 1) identify kinase-substrates from mass spectrometry-based phosphoproteomics data and 2) predict transcription factor target genes by integrating various next-generation sequencing data.

With the emergence of the Internet-of-Things (IoT) and seamless Internet connectivity, the need to process streaming data on real-time basis has become essential. However, the existing data stream management systems are not efficient in analyzing the network log big data for real-time anomaly detection. Further, the existing anomaly detection approaches are not proficient because they cannot be applied to networks, are computationally complex, and suffer from high false positives. Thus, in this paper a hybrid data processing model for network anomaly detection is proposed that leverages grey wolf optimization (GWO) and convolutional neural network (CNN). To enhance the capabilities of the proposed model, GWO and CNN learning approaches were enhanced with: 1) improved exploration, exploitation, and initial population generation abilities and 2) revamped dropout functionality, respectively. These extended variants are referred to as Improved-GWO (ImGWO) and Improved-CNN (ImCNN). The proposed model works in two phases for efficient network anomaly detection. In the first phase, ImGWO is used for feature selection in order to obtain an optimal trade-off between two objectives, i.e., reduced error rate and feature-set minimization. In the second phase, ImCNN is used for network anomaly classification. The efficacy of the proposed model is validated on benchmark (DARPA'98 and KDD'99) and synthetic datasets. The results obtained demonstrate that the proposed cloud-based anomaly detection model is superior in comparison to the other state-of-the-art models (used for network anomaly detection), in terms of accuracy, detection rate, false positive rate, and F-score. In average, the proposed model exhibits an overall improvement of 8.25%, 4.08%, and 3.62% in terms of detection rate, false positives, and accuracy, respectively; relative to standard GWO with CNN.

Enterprise users at different geographic locations generate large-volume data that is stored at different geographic datacenters. These users may also perform big data analytics on the stored data to identify valuable information in order to make strategic decisions. However, it is well known that performing big data analytics on data in geographical-located datacenters usually is time-consuming and costly. In some delay-sensitive applications, the query result may become useless if answering a query takes too long time. Instead, sometimes users may only be interested in timely approximate rather than exact query results. When such approximate query evaluation is the case, applications must sacrifice timeliness to get more accurate evaluation results or tolerate evaluation result with a guaranteed error bound obtained from analyzing the samples of the data to meet their stringent timeline. In this paper, we study quality-of-service (QoS)-aware data replication and placement for approximate query evaluation of big data analytics in a distributed cloud, where the original (source) data of a query is distributed at different geo-distributed datacenters. We focus on the problems of placing data samples of the source data at some strategic datacenters to meet stringent query delay requirements of users, by exploring a non-trivial trade-off between the cost of query evaluation and the error bound of the evaluation result. We first propose an approximation algorithm with a provable approximation ratio for a single approximate query. We then develop an efficient heuristic algorithm for evaluating a set of approximate queries with the aim to minimize the evaluation cost while meeting the delay requirements of these queries. We finally demonstrate the effectiveness and efficiency of the proposed algorithms through both experimental simulations and implementations in a real test-bed, real datasets are employed. Experimental results show that the proposed algorithms are promising.

Workload surges are a serious hindrance to per-formance of even high-throughput key-value data stores, such as Cassandra, MongoDB, and more recently Aerospike. In this paper, we present a decentralized admission controller for high-throughput key-value data stores. The proposed controller dynamically regulates the release time of incoming requests explicitly taking into account different Quality of Service (QoS) classes. In particular, an instance of such controller is assigned to each client for its autonomous admission control specific to the client's QoS requirements. These controllers operate in a decentralized manner with only local performance metrics, response time and queue waiting time. Despite the use of such "minimal" run-time state information, our decentralized admission controller is capable of coping with workload surges respecting QoS requirements. The performance evaluation is carried out by comparing the proposed admission controller with the default scheduling policy of Aerospike, in a testbed cluster under various workload intensity rates. Experimental results confirm that the proposed controller improves QoS satisfaction in terms of end-to-end response time by nearly 12 times, on average, compared with that of Aerospike's, in high-rate workload. Results also show decreases of the average and standard deviation of latency up to 31% and 50%, respectively, during workload surges (peak load) in high-rate workload.

The move from the traditional Software-as-a-Product (SaaP) model to the Software-as-a-Service (SaaS) model is apparent with the wide adoption of cloud computing. Unlike the SaaP model, the SaaS model delivers a diverse set of software features directly from public clouds to a large number of arbitrary users with varying quality of service (QoS) requirements. QoS is typically assured by admission control. However, there are two outstanding issues with traditional QoS systems: (1) they are usually designed and developed with a special purpose, making them difficult to be reused for other use cases; and (2) they have limited scalability (i.e., vertical scalability) due to the write-intensive nature of admission control workload. In this paper, we present Janus - a QoS framework that is generic and scalable for SaaS applications taking full advantage of cloud's inherent horizontal scalability (scaling-out). Janus uses a multi-layer architecture to eliminate the communication between nodes (being scaled out) in the same layer achieving horizontal scalability without sacrificing vertical scalability. Janus ensures accurate admission control (QoS decisions) using a distributed set of leaky buckets with a refill mechanism. Janus also adopts a key-value request-response mechanism for easy integration with the actual application. We extensively evaluate Janus on AWS cloud with both Apache HTTP server benchmarking tool and a photo sharing web application. Our experimental results demonstrate that (a) Janus achieves linear scalability both vertically and horizontally, and (b) Janus can be integrated with existing applications with a minimum amount of code change. In particular, Janus achieves more than 100,000 requests per second with only 10 nodes (4 vCPU cores on each node) in the QoS server layer and 90% of the admission control decisions were made in 3 milliseconds.

Lambda platform is a new concept based on an event-driven server-less computation that empowers application developers to build scalable enterprise software in a virtualized environment without provisioning or managing any physical servers (a server-less solution). In reality, however, devising an effective consolidation method to host multiple Lambda functions into a single machine is challenging. The existing simple resource allocation algorithms, such as the round-robin policy used in many commercial server-less systems, suffer from lack of responsiveness to a sudden surge in the incoming workload. This will result in an unsatisfactory performance degradation that is directly experienced by the end-user of a Lambda application. In this paper, we address the problem of CPU cap management in a Lambda platform for ensuring different QoS enforcement levels in a platform with shared resources, in case of fluctuations and sudden surges in the incoming workload requests. To this end, we present a closed-loop (feedback-based) CPU cap controller, which fulfills the QoS levels enforced by the application owners. The controller adjusts the number of working threads per QoS class and dispatches the outstanding Lambda functions along with the associated events to the most appropriate working thread. The proposed solution reduces the QoS violations by an average of 6.36 times compared to the round-robin policy. It can also maintain the end-to-end response time of applications belonging to the highest priority QoS class close to the target set-point while decreasing the overall response time by up to 52%.

The emerging of mobile edge computing has significantly reduced the response time and Internet risk of service invocations. However, due to the distributed architecture and limited resources, balancing the load between edge servers to minimize the overall response time has become a critical objective for mobile edge computing. This problem is generally related to two aspects, request dispatching and service scheduling. To address this issue, we proposed a novel heuristic method called GASD (combined Genetic algorithm and simulated Annealing algorithm for Service request Dispatching). It tackles the problem by jointly conducting request dispatching and service scheduling. In addition, a solution combination algorithm is applied to reduce the computation complexity of the method. The experimental results show that the GASD method can achieve much lower overall response time than the compared methods. Moreover, the execution time of GASD is in a low order of magnitude and the algorithm performs excellent scalability as the experimental scale increases.

There is a growing trend for engaging edge computing to help smart homes to improve the living comfort of residents. However, the rapidly rising interest in such deployments does not normally integrate with energy management schemes, which is a central issue that smart homes have to face. In this paper, we propose a unified energy management framework for enabling a sustainable edge computing paradigm while meeting the needs of home energy management and smart home applications. This framework aims to enable the full use of renewable energy while reducing electricity bills for households. A prototype system was implemented by using low-cost and easy-to-get hardware. The experiment results demonstrated that renewable energy is fully capable of supporting the reliable running of edge computing devices and electricity bills could be cut by up to 86% when our proposed framework was employed.

Minimizing service-level agreement (SLA) violations and energy consumption through server consolidation is of paramount importance for the sustainability of cloud environments. In this paper, we propose an online method to reduce cloud SLA violations by taking into account: (a) the energy consumption of migrating virtual machines and (b) server consolidation techniques to minimize the energy consumption within the system. Rigorous mathematical competitive analysis shows that the proposed method achieves a 2.4 competitive ratio against a cognitive adversary. Our solution is superior compared to the current state of the art algorithms, such as UP-VMC, KMI, MBFD, both theoretically (competitive ratios of other alternatives are unbounded) and empirically through simulations using CloudSim. More specifically, the competitive ratios of state of the art algorithms are unbounded and our proposed methodology reduces the VM migrations and energy consumption by 85% and 45%, respectively. The aforementioned improvement comes at an expense of a small increase in terms of SLA violations. The above results are achieved without the a priori knowledge of VM utilization patterns.

Due to the limited capabilities and resources, edge servers cannot meet the increasingly complex and diverse service requirements in mobile edge computing environments. In this circumstance, how to dispatch the component tasks of service requests to edge and cloud servers to reduce the time delay has become a crucial problem. Therefore, we focus on this problem and propose a heuristic algorithm called GAMEC (combined Genetic algorithm and simulated Annealing algorithm for service selection in Mobile Edge Computing systems). The simulated experiments have demonstrated the high effectiveness of the method.

In recent years, Edge computing has emerged as a new paradigm that can reduce communication delays over the Internet by moving computation power from far-end cloud servers to be closer to data sources. It is natural to shift the design of cloud-based IoT applications to Edge-based ones. Activity recognition in smart homes is one of the IoT applications that can benefit significantly from such a shift. In this work, we propose an Edge-based solution for addressing the activity recognition problem in smart homes from multiple perspectives, including architecture, algorithm design and system implementation. First, the Edge computing architecture is introduced and several critical management tasks are also investigated. Second, a realization of the Edge computing system is presented by using open source software and low-cost hardware. The consistency and scalability of running jobs on Edge devices are also addressed in our approach. Last, we propose a convolutional neural network model to perform activity recognition tasks on Edge devices. Preliminary experiments are conducted to compare our model with existing machine learning methods, and the results demonstrate that the performance of our model is promising.

Caching is essential for data-intensive mobile applications to reduce duplicated data transmission. In this paper, we study the optimal probabilistic caching and re-caching policies in mobile networks as the file popularity may change over time. We propose a Probabilistic File Re-caching (PFR) policy to match the updated popularity. Through PFR, files cached (resp. not cached) are probabilistically opted out (resp. opted in). PFR is with substantial advantages: (1) PFR can handle a huge combinatorial number of all possible situations. (2) The expected number of replaced files is minimized. (3) The computational complexity of PFR is low. Second, we study a utility optimization problem in the mobile network, in order to optimally decide the probability that each file is cached and whether a file should be downloaded from a peer device or directly from the server. Even though the optimization problem is non-convex programming in nature, we devise a computationally efficient Optimal Probabilistic Caching and Requesting (OPCR) policy, through decoupling the decision variables, to derive a globally optimal solution. Finally, we develop a real-world prototype and conduct trace-driven simulations to validate and evaluate our proposed PFR and OPCR policies.

Fog computing is a promising solution to provide low-latency and ubiquitously available computation offloading services to widely distributed Internet of Things (IoT) devices with limited computing capabilities. One obstacle, however, is how to seamlessly hand over mobile IoT devices among different fog nodes to avoid service interruption. In this paper, we propose seamless fog (sFog), a new framework supporting efficient congestion control and seamless handover schemes. Intrinsically, sFog improves system performance during handovers (achieved by the handover scheme), and guarantees the performance does not degrade when handovers do not occur (achieved by the congestion control scheme). Through the congestion control scheme, jobs are efficiently offloaded without causing unnecessary system idling; through the handover scheme, jobs are pre-migrated to the target fog node when a handover is about to occur, in order to reduce migration delay. In order to evaluate the performance of sFog, we propose a theoretical framework and establish a real-world prototype. Both the theoretical and experimental results show that sFog achieves substantial delay reductions compared with traditional benchmark handover schemes.

Dynamic virtual machine (VM) consolidation is considered an effective approach for improving power consumption and computing resource utilization in cloud-based data centers. However, the ever-changing workload in a data center makes it difficult for VM consolidation to prevent service level agreement (SLA) violations and optimize power consumption. Detection of overutilized and underutilized physical machines (PMs) plays a significant role in effective VM consolidation, immediately improving resource utilization, SLA violations, and power consumption. This paper presents a new proposal for the dynamic adjustment of threshold values that aims to minimize the number of migrations in varying workload environments. The proposed approach, named the `dynamic threshold-based fuzzy approach' (DTFA), is a fuzzy threshold-based approach used for adjusting the threshold values of PMs in a cloud environment. The proposed approach allows the number of migrations caused by overloading to be reduced and SLAs to be met. Three sets of experiments with different workloads were conducted to validate the proposed approach. The results demonstrate that DTFA outperforms existing solutions by an average of 22.52%, 45.63% and 56.68% in power consumption, VM migration count, and SLA violations, respectively.

Efficient power management is essential in ensuring the economic viability of large-scale distributed systems. There is growing interest in applying the Running Average Power Limit (RAPL) feature that is commonly found in today's Intel CPUs to energy monitoring and efficiency applications. We investigate the power consumption characteristics of two different CPUs using the RAPL feature. We present a prototype lightweight software-based virtual power meter that exploits this functionality. Utilizing a simple but very effective application-agnostic power model, it offers comparable or superior performance to existing power models that are more complex. It is portable across a variety of systems. It can be used in containerized or virtualized environments. We demonstrate that our power model has an average error of 1.63 %. This result compares favorably with existing state-of-the-art power models and is achieved using a simple power model. Consequently, our power meter is viable for use in real-world applications such as power estimation for energy-aware scheduling.

In a multi-cloud environment, consumers can access multiple cloud services using a single heterogeneous computing architecture. In such an environment, multiple instances of the same cloud service and its component may be geographically dispersed. So, cloud service broker (CSB) exploits the heterogeneity of multi-cloud environment to provide high performance at a low price to its consumers. The consumer tasks are allocated to the geo-dispersed cloud service components for execution of various services. For this purpose, an optimal service components identification and task allocation are major concerns keeping in view of the heterogeneity in multi-cloud environment. For this purpose, a scheduling algorithm, which takes care of location, price, and performance is required. Therefore, in this paper, SLOPE: A Self Learning Optimization and Prediction Ensembler for Task Scheduling in Multi-cloud Environment is proposed. SLOPE works in two phases, 1) In first phase, Bayes theorem is used to design a self-learning algorithm, to compute the conditional probability (strength) of each service component in order to select the probable rule string, and 2) a roulette wheel method is used to select an optimal scheduling policy for a given service request. SLOPE helps to identify the best possible service component from the pool of resources on the basis of dynamic factors and then schedule a service request to the selected component. Unlike most of the other existing approaches, SLOPE builds an efficient schedule for service selection. Experimental results demonstrate that SLOPE performs better in comparison to other competing schemes of its category.

As a key enabling technology for 5G network softwarization, network function virtualization (NFV) provides an efficient paradigm to optimize network resource utility for the benefits of both network providers and users. However, the inherent network dynamics and uncertainties from 5G infrastructure, resources, and applications are slowing down the further adoption of NFV in many emerging networking applications. Motivated by this, in this paper, we investigate the issues of network utility degradation when implementing NFV in dynamic networks, and design a proactive NFV solution from a fully stochastic perspective. Unlike existing deterministic NFV solutions, which assume given network capacities and/or static service quality demands, this paper explicitly integrates the knowledge of influential network variations into a two-stage stochastic resource utilization model. By exploiting the hierarchical decision structures in this problem, a distributed computing framework with two-level decomposition is designed to facilitate a distributed implementation of the proposed model in large-scale networks. The experimental results demonstrate that the proposed solution not only improves 3~5 folds of network performance, but also effectively reduces the risk of service quality violation.

Deploying deduplication for distributed primary storage is a sophisticated and challenging task, considering that the demands of low read/write latency, stable read/write performance, and efficient space saving are all of paramount importance. Unfortunately, existing schemes cannot present a satisfactory solution for the aforementioned requirements simultaneously. In this article, we propose D$^{3}$, a dynamic dual-phase deduplication framework for distributed primary storage. Several major innovations are established in D$^{3}$. First, we formulate a deduplication-oriented taxonomy calledDedup-Type, to group data with similar deduplication-related characteristics into larger categories. It serves as coarse-grained filter and one of the prioritizing references in D$^{3}$. Second, D$^{3}$is a dual-phase framework—inline-phase and offline-phase deduplication processes work in concert with each other. Third, D$^{3}$operates in a dynamic manner. We design two critical mechanisms:context-aware threshold adjustment(CTA) for local inline-phase deduplication, anddeferred priority-based enforcement(DPE) for global offline-phase deduplication. The CTA mechanism enables selective deduplication under a periodically updated threshold. Data skipped during the inline phase is regarded as a candidate for offline phase, and is handled in a prioritized order under the governance of DPE mechanism. Evaluation results demonstrate that, compared with conventional inline and offline deduplication schemes, D$^{3}$achieves more efficient and stabler read/write performance with competitive space saving.

Outsourcing data to a third-party administrative control, as is done in cloud computing, gives rise to security concerns. The data compromise may occur due to attacks by other users and nodes within the cloud. Therefore, high security measures are required to protect data within the cloud. However, the employed security strategy must also take into account the optimization of the data retrieval time. In this paper, we propose division and replication of data in the cloud for optimal performance and security (DROPS) that collectively approaches the security and performance issues. In the DROPS methodology, we divide a file into fragments, and replicate the fragmented data over the cloud nodes. Each of the nodes stores only a single fragment of a particular data file that ensures that even in case of a successful attack, no meaningful information is revealed to the attacker. Moreover, the nodes storing the fragments, are separated with certain distance by means of graph T-coloring to prohibit an attacker of guessing the locations of the fragments. Furthermore, the DROPS methodology does not rely on the traditional cryptographic techniques for the data security; thereby relieving the system of computationally expensive methodologies. We show that the probability to locate and compromise all of the nodes storing the fragments of a single file is extremely low. We also compare the performance of the DROPS methodology with 10 other schemes. The higher level of security with slight performance overhead was observed.

Energy efficiency of data centers (DCs) has become a major concern as DCs continue to grow large often-Energy efficiency of data centers (DCs) has become a major concern as DCs continue to grow large often hosting tens of thousands of servers or even hundreds of thousands of them. Clearly, such a volume of DCs implies scale of data center network (DCN) with a huge number of network nodes and links. The energy consumption of this communication network has skyrocketed and become the same league as computing servers' costs. With the ever-increasing amount of data that need to be stored and processed in DCs, DCN traffic continues to soar drawing increasingly more power. In particular, more than one-third of the total energy in DCs is consumed by communication links, switching and aggregation elements. In this paper, we concern the energy efficiency of data center explicitly taking into account both servers and DCN. To this end, we present VPTCA, as a collective energy-efficiency approach to data center network planning, which deals with virtual machine (VM) placement and communication traffic configuration. VPTCA aims particularly to reduce the energy consumption of DCN by assigning interrelated VMs into the same server or pod, which effectively helps reduce the amount of transmission load. In the layer of traffic message, VPTCA optimally uses switch ports and link bandwidth to balance the load and avoid congestions, enabling DCN to increase its transmission capacity, and saving a significant amount of network energy. In our evaluation via NS-2 simulations, the performance of VPTCA is measured and compared with two well-known DCN management algorithms, Global First Fit and ElasticTree. Based on our experimental results, VPTCA outperforms existing algorithms in providing DCN more transmission capacity with less energy consumption.

Understanding users’ social relationships plays an important role in many disciplines, including marketing, management science, and so on and is the fundamental context information required in many context-aware applications. However, despite significant research progress in social learning, sensing and reconstructing the off-line social structures in an accurate and nonobtrusive way is still a challenging open problem. In this paper, we propose SODAR, an off-line SOcial colocation Detection And network Reconstruction system, a social learning system that exploits wireless probes emitted by the smartphones carried by users to learn and infer their social relationships and reconstruct their off-line social structures. The probe capturing and filtering mechanisms collects high-quality wireless probe information, and the passive localization techniques are used to process the data and detect colocation events, which are used for the novel social representation learning. The learned social representation vector for each user contains rich social information and can be used to determine the social distances for each pair of users. With projection and clustering techniques, the off-line social structures can be visualized and reconstructed. We implemented the system and deployed the system to different indoor spaces covering more than 1000 m2. The evaluation results show that the SODAR system is able to reliably learn social representations for each user and effectively reconstruct the off-line social structures.

Online Social Networks (OSNs) have not only significantly reformed the social interaction pattern but have also emerged as an effective platform for recommendation of services and products. The upswing in use of the OSNs has also witnessed growth in unwanted activities on social media. On the one hand, the spammers on social media can be a high risk towards the security of legitimate users and on the other hand some of the legitimate users, such as bloggers can pollute the results of recommendation systems that work alongside the OSNs. The polluted results of recommendation systems can be precarious to the masses that track recommendations. Therefore, it is necessary to segregate such type of users from the genuine experts. We propose a framework that separates the spammers and unsolicited bloggers from the genuine experts of a specific domain. The proposed approach employs modified Hyperlink Induced Topic Search (HITS) to separate the unsolicited bloggers from the experts on Twitter on the basis of tweets. The approach considers domain specific keywords in the tweets and several tweet characteristics to identify the unsolicited bloggers. Experimental results demonstrate the effectiveness of the proposed methodology as compared to several state-of-the-art approaches and classifiers.

With the evolution of Internet and extensive usage of smart devices for computing and storage, cloud computing has become popular. It provides seamless services such as e-commerce, e-health, e-banking, etc., to the end users. These services are hosted on massive geodistributed data centers (DCs), which may be managed by different service providers. For faster response time, such a data explosion creates the need to expand DCs. So, to ease the load on DCs, some of the applications may be executed on the edge devices near to the proximity of the end users. However, such a multi-edge-cloud environment involves huge data migrations across the underlying network infrastructure, which may generate long migration delay and cost. Hence, in this paper, an efficient workload slicing scheme is proposed for handling data-intensive applications in multiedge-cloud environment using software-defined networks (SDN). To handle the inter-DC migrations efficiently, an SDN-based control scheme is presented, which provides energy-aware network traffic flow scheduling. Finally, a multileader multifollower Stackelberg game is proposed to provide cost-effective inter-DC migrations. The efficacy of the proposed scheme is evaluated on Google workload traces using various parameters. The results obtained show the effectiveness of the proposed scheme.

Traffic management in roads is one of the major challenges faced in developing efficient intelligent transportation systems. Recently, wireless networks have received significant attention for tackling this challenge. However, wireless technologies face the well-known spectrum scarcity problem due to the explosive demand for radio resources. To overcome this challenge, this study presents a novel intelligent traffic control system, utilizing the unused spectrum. Unlike existing works, the spectrum owners in this study hire free spectrum to drivers. The hired spectrum is deployed to build a short-range cost-effective wireless communication for monitoring traffic and enabling drivers to exchange warning messages, and thus enhancing road safety. Our objectives include minimizing crash probability, utilizing unused spectrum, and enabling spectrum owners to generate extra revenue. Numerical analysis demonstrates the capability of our approach to minimize the crash probability among vehicles under different operating road conditions.

Smart grid (SG) is an integration of traditional power grid with advanced information and communication infrastructure for bidirectional energy flow between grid and end users. A huge amount of data is being generated by various smart devices deployed in SG systems. Such a massive data generation from various smart devices in SG systems may lead to various challenges for the networking infrastructure deployed between users and the grid. Hence, an efficient data transmission technique is required for providing desired QoS to the end users in this environment. Generally, the data generated by smart devices in SG has high dimensions in the form of multiple heterogeneous attributes, values of which are changed with time. The high dimensions of data may affect the performance of most of the designed solutions in this environment. Most of the existing schemes reported in the literature have complex operations for the data dimensionality reduction problem which may deteriorate the performance of any implemented solution for this problem. To address these challenges, in this paper, a tensor-based big data management scheme is proposed for dimensionality reduction problem of big data generated from various smart devices. In the proposed scheme, first the Frobenius norm is applied on high-order-tensors (used for data representation) to minimize the reconstruction error of the reduced tensors. Then, an empirical probability-based control algorithm is designed to estimate an optimal path to forward the reduced data using software-defined networks for minimization of the network load and effective bandwidth utilization. The proposed scheme minimizes the transmission delay incurred during the movement of the dimensionally reduced data between different nodes. The efficacy of the proposed scheme has been evaluated using extensive simulations carried out on the data traces using `R' programming and Matlab. The big data traces considered for evaluation consist of more than two million entries (2,075,259) collected at one minute sampling rate having hetrogenous features such as-voltage, energy, frequency, electric signals, etc. Moreover, a comparative study for different data traces and a real SG testbed is also presented to prove the efficacy of the proposed scheme. The results obtained depict the effectiveness of the proposed scheme with respect to the parameters such asnetwork delay, accuracy, and throughput.

Use of social network is the basic functionality of today's life. With the advent of more and more online social media, the information available and its utilization have come under the threat of several anomalies. Anomalies are the major cause of online frauds which allow information access by unauthorized users as well as information forging. One of the anomalies that act as a silent attacker is the horizontal anomaly. These are the anomalies caused by a user because of his/her variable behavior towards different sources. Horizontal anomalies are difficult to detect and hazardous for any network. In this paper, a self-healing neuro-fuzzy approach (NHAD) is used for the detection, recovery, and removal of horizontal anomalies efficiently and accurately. The proposed approach operates over the five paradigms, namely, missing links, reputation gain, significant difference, trust properties, and trust score. The proposed approach is evaluated with three datasets: DARPA'98 benchmark dataset, synthetic dataset, and real-time traffic. Results show that the accuracy of the proposed NHAD model for 10 to 30 percent anomalies in synthetic dataset ranges between 98.08 and 99.88 percent. The evaluation over DARPA'98 dataset demonstrates that the proposed approach is better than the existing solutions as it provides 99.97 percent detection rate for anomalous class. For real-time traffic, the proposed NHAD model operates with an average accuracy of 99.42 at 99.90 percent detection rate.

In the current cloud business environment, the cloud provider (CP) can provide a means for offering the required quality of service (QoS) for multiple classes of clients. We consider the cloud market where various resources such as CPUs, memory, and storage in the form of Virtual Machine (VM) instances can be provisioned and then leased to clients with QoS guarantees. Unlike existing works, we propose a novel Service Level Agreement (SLA) framework for cloud computing, in which a price control parameter is used to meet QoS demands for all classes in the market. The framework uses reinforcement learning (RL) to derive a VM hiring policy that can adapt to changes in the system to guarantee the QoS for all client classes. These changes include: service cost, system capacity, and the demand for service. In exhibiting solutions, when the CP leases more VMs to a class of clients, the QoS is degraded for other classes due to an inadequate number of VMs. However, our approach integrates computing resources adaptation with service admission control based on the RL model. To the best of our knowledge, this study is the first attempt that facilitates this integration to enhance the CP's profit and avoid SLA violation. Numerical analysis stresses the ability of our approach to avoid SLA violation while maximizing the CP's profit under varying cloud environment conditions.

In real simulation applications, simulations often involve large volumes of three-dimensinal (3D) moving objects. With the rapid growth of the scale of simulation-problem domains, it has become a key requirement to efficiently manage massive 3D moving objects. Conventional indexing approaches for managing 3D moving objects during simulations generally sufferfrom excessive update costs. Aiming to this problem, this paper first proposes an update-efficient indexing structure by fusing a loose Octree and one update-memo structure, namely ML-Octree. ML-Octree significantly reduces the update costs of one simulation involving massive 3D moving objects. Towards providing a more efficient indexing approach, this paper has explored the feasibility of paralleling ML-Octree by employing Graphic Processing Unit (GPU). A load-balancing scheme is used to further improve the update performance of the GPU-aided ML-Octree. Finally, a distributed GPU-aided ML-Octree is proposed for large-scale simulations. The experimental results indicate that (1) ML-Octree can acquire the update-performance gain of an order of magnitude similar to that of Octree, (2) the GPU-aided ML-Octree can accelerate 5.07χ fasterthan a parallel ML-Octree with 8 CPU threads on average, (3) the load-balance scheme can improve GPU-aided ML-Octree by 2.3χ on average, and (4) the distributed GPU-aided ML-Octree can efficiently support large-scale simulations.

Lambda paradigm, also known as Function as a Service (FaaS), is a novel event-driven concept that allows companies to build scalable and reliable enterprise applications in an off-premise computing data-center as a serverless solution. In practice, however, an important goal for the service provider of a Lambda platform is to devise an efficient way to consolidate multiple Lambda functions in a single host. While the majority of existing resource management solutions use only operating-system level metrics (e.g., average utilization of computing and I/O resources) to allocate the available resources among the submitted workloads in a balanced way, a resource allocation schema that is oblivious to the issue of shared-resource contention can result in a significant performance variability and degradation within the entire platform. This paper proposes a predictive controller scheme that dynamically allocates resources in a Lambda platform. This scheme uses a prediction tool to estimate the future rate of every event stream and takes into account the quality of service enforcements requested by the owner of each Lambda function. This is formulated as an optimization problem where a set of cost functions are introduced (i) to reduce the total QoS violation incidents; (ii) to keep the CPU utilization level within an accepted range; and (iii) to avoid the fierce contention among collocated applications for obtaining shared resources. Performance evaluation is carried out by comparing the proposed solution with an enhanced interference-aware version of three well-known heuristics, namely spread, binpack (the two native clustering solutions employed by Docker Swarm) and best-effort resource allocation schema. Experimental results show that the proposed controller improves the overall performance (in terms of reducing the end-to-end response time) by 14.9 percent on average compared to the best result of the other heuristics. The proposed solution also increases the overall CPU utilization by 18 percent on average (for lightweight workloads), while achieves an average 87 percent (maximum 146 percent) improvement in preventing QoS violation incidents.

There are many application domains, where hierarchical data is inherent, but surprisingly, there are few techniques for mining patterns from such important data. Hierarchical Heavy Hitters (HHH) and multilevel and Cross-Level Association Rules (CLAR) mining are well-known hierarchical pattern mining techniques. The problem in these techniques; however, is that they focus on capturing only global patterns from data but cannot identify local contextual patterns. Another problem in these techniques is that they treat all data items in the transaction equally and do not consider the sequential nature of the relationship among items within a transaction; hence, they cannot capture the correlation semantic within the transactions of the data items. There are many applications such as clickstream mining, healthcare data mining, network monitoring, and recommender systems, which require to identify local contextual patterns and correlation semantics. In this work, we introduce a new concept, which can capture the sequential nature of the relationship between pairs of hierarchical items at multiple concept levels and can capture local contextual patterns within the context of the global patterns. We call this notion Hierarchically Correlated Heavy Hitters (HCHH). Specifically, the proposed approach finds the correlation between items corresponding to hierarchically discounted frequency counts. We have provided formal definitions of the proposed concept and developed algorithmic approaches for computing HCHH in data streams efficiently. The proposed HCHH algorithm have deterministic error guarantees, and space bounds. It requires O(η/ϵpϵs) memory, where h is a small constant, and ϵp∈ [0,1], ϵs∈ [0,1] are user defined parameters on upper bounds of estimation error. We have compared the proposed HCHH concept with existing hierarchical pattern mining approaches both theoretically as well as experimentally.

Cloud computing has arisen as the mainstream platform of utility computing paradigm that offers reliable and robust infrastructure for storing data remotely, and provides on demand applications and services. Currently, establishments that produce huge volume of sensitive data, leverage data outsourcing to reduce the burden of local data storage and maintenance. The outsourced data, however, in the cloud are not always trustworthy because of the inadequacy of physical control over the data for data owners. To better streamline this issue, scientists have now focused on relieving the security threats by designing remote data checking (RDC) techniques. However, the majority of these techniques are inapplicable to big data storage due to incurring huge computation cost on the user and cloud sides. Such schemes in existence suffer from data dynamicity problem from two sides. First, they are only applicable for static archive data and are not subject to audit the dynamic outsourced data. Second, although, some of the existence methods are able to support dynamic data update, increasing the number of update operations impose high computation and communication cost on the auditor due to maintenance of data structure, i.e., merkle hash tree. This paper presents an efficient RDC method on the basis of algebraic properties of the outsourced files in cloud computing, which inflicts the least computation and communication cost. The main contribution of this paper is to present a new data structure, called Divide and Conquer Table (D&CT), which proficiently supports dynamic data for normal file sizes. Moreover, this data structure empowers our method to be applicable for large-scale data storage with minimum computation cost. The one-way analysis of variance shows that there are significant differences between the proposed method and the existing methods in terms of the computation and communication cost on the auditor and cloud.

Network Function Virtualization (NFV) is an emerging network architecture to increase flexibility and agility within operator's networks by placing virtualized services on demand in Cloud data centers (CDCs). One of the main challenges for the NFV environment is how to minimize network latency in the rapidly changing network environments. Although many researchers have already studied in the field of Virtual Machine (VM) migration and Virtual Network Function (VNF) placement for efficient resource management in CDCs, VNF migration problem for low network latency among VNFs has not been studied yet to the best of our knowledge. To address this issue in this article, we i) formulate the VNF migration problem and ii) develop a novel VNF migration algorithm called VNF Real-time Migration (VNF-RM) for lower network latency in dynamically changing resource availability. As a result of experiments, the effectiveness of our algorithm is demonstrated by reducing network latency by up to 70.90% after latency-aware VNF migrations.

Consolidating applications of conflicting service level objectives (SLOs) to share virtualized resources in cloud datacenters requires efficient resource management to ensure overall high Quality-of-Service (QoS). Applications of different performance targets often exhibit different resource demands. Thus, it is not trivial to translate individual application SLOs to corresponding resource shares in a shared virtualized environment to meet performance targets. In this paper, we present CtrlCloud, a performance-aware resource controlling system, that adaptively allocates resources, with a resource-share controller and an allocation optimization model. The controller automatically adapts resource demands based on performance deviations, while the optimization model resolves conflicts in resource demands from multiple co-located applications based on their ongoing performance achieved. We implement a proof-of-concept prototype of CtrlCloud in Python on top of Xen hypervisor. Our experimental results indicate that CtrlCloud can optimize allocations of CPU resources across multiple applications to maintain the 95th percentile latency within predefined SLO targets. CtrlCloud also provides QoS differentiation and yet fulfilling of CPU share demands from applications is maximized given resource availability. We further compare CtrlCloud against two other resource allocation methods commonly used in current clouds. CtrlCloud improves resource utilization by allocating resource shares optimal to 'actual needs' as it employs share-performance online modeling.

Many servers use technologies such as virtualization or containerization to improve server utilization. These technologies pose challenges for power monitoring since it is not possible to directly measure the power use of an abstraction such as a virtual machine. Much work has been done in modeling the power use of CPUs, virtual machines and entire servers, however, there is a scarcity of work in building lightweight power monitoring middleware that can be deployed across a range of systems. In this paper, we present cWatts+ as a prototype lightweight software-based virtual power meter. Utilizing a simple but powerful application-agnostic power model, it offers comparable performance to existing "more complex and heavier-weight" power models. It uses a small number of widely available CPU event counters and the Performance Application Programming Interface Library to estimate power usage on a per-thread basis. It has minimal overhead and is portable across a variety of systems. It can be used in containerized or virtualized environments. We evaluate the estimation performance of cWatts+ for a variety of real-world benchmarks that are relevant to large distributed systems. Also, we examine the importance of including CPU core temperature data in the power model. We demonstrate that our power model has an average error of less than 5%. This result compares favorably with existing state-of-the-art power models and is achieved using a relatively simple power model that exhibits minimal power consumption (overhead). Consequently, our power monitoring middleware is viable for use in real-world applications such as power estimation for energy-aware scheduling.

Network Function Virtualization (NFV) is an emerging network architecture to increase flexibility and agility within operator's networks by placing virtualized services on demand in Cloud data centers (CDCs). One of the main challenges for the NFV environment is how to efficiently allocate Virtual Network Functions (VNF) to Virtual Machines (VMs). Although a significant amount of work/research has been already conducted for the generic VNF placement problem, network latency among various network components has not been comprehensively considered yet. To address this concern, in this article, we design a more comprehensive model based on real measurements to capture network latency among VNFs with more granularity to optimize placement of VNFs in CDCs. Experimental results are promising and indicate that our approach, namely VNF Low-Latency Placement (VNF-LLP), can reduce network latency by up to 64.24% (50.33% in average) compared with two generic algorithms. Furthermore, it has a lower lead time (time to find a suitable VM to host a VNF) as compared with two classic approaches.

Virtual machine (VM) consolidation is necessary for increasing the server utilization; however, it also leads to VM performance degradation. This work presents a method to predict the consolidated VMs performance from the critical system events data. Experiments are designed to demonstrate the effect of system events like interrupts, page faults, mutex operations, and context switching on the consolidated VMs. Results show that the host server counters are not reliable for such predictions. On the other hand, the coupling of the task execution time with the VM system events is an effective way to predict the consolidation performance. Results further show that the VM memory allocation plays an important part in the consolidated tasks performance. The system event data is also used to train an Artificial Neural Network (ANN) for performance prediction on three hypervisors; ESXi, Xen, and XenServer and similar results are observed in all three.

This paper addresses the shared resource contention problem associated with the auto-parallelization of running queries in distributed stream processing engines. In such platforms, analyzing a large amount of data often requires to execute user-defined queries over continues raw-inputs in a parallel fashion at each single host. However, previous studies showed that the collocated applications can fiercely compete for shared resources, resulting in a severe performance degradation among applications. This paper presents an advanced resource allocation strategy for handling scenarios in which the target applications have different quality of service (QoS) requirements while shared-resource interference is considered as a key performance-limiting parameter. To properly allocate the best possible resource to each query, the proposed controller predicts the performance degradation of the running pane-level as well as the window-level queries when co-running with other queries. This is addressed as an optimization problem where a set of cost functions is defined to achieve the following goals: a) reduce the sum of QoS violation incidents over all machines; b) keep the CPU utilization level within an accepted range; and c) avoid fierce shared resource interference among collocated applications. Particle swarm optimization is used to find an acceptable solution at each round of the controlling period. The performance of the proposed solution is benchmarked with Round-Robin and best-effort strategies, and the experimental results clearly demonstrate that the proposed controller has the following advantages over its opponents: it increases the overall resource utilization by 15% on average while can reduce the average tuple latencies by 14%. It also achieves an average 123% improvement in preventing QoS violation incidents.

Petroleum industry has started to embrace the advanced Petroleum Cyber-Physical System (CPS) technologies. Offshore petroleum CPS is particularly difficult to build, mainly due to the challenge in detecting and preventing offshore oil leaking. During the oil exploration and transportation process, the remote multi-sensing technology is typically used for leak detection, enabling the underwater modeling of an offshore petroleum CPS. However, such a technology suffers from insufficient remote sensing resources and large computational overhead. In this work, a cross entropy optimization based leak detection technique is proposed to detect the oil leak, which also facilitates the understanding of the oil leak induced marine pollution. Experimental results on a real Penglai oil spill event demonstrate that the proposed technique can effectively identify the sources of oil spills with accuracy of up to 90.78%.

Lambda architecture is a novel event-driven serverless paradigm that allows companies to build scalable and reliable enterprise applications. As an attractive alternative to traditional service oriented architecture (SOA), Lambda architecture can be used in many use cases including BI tools, in-memory graph databases, OLAP, and streaming data processing. In practice, an important aim of Lambda's service providers is devising an efficient way to co-locate multiple Lambda functions with different attributes into a set of available computing resources. However, previous studies showed that consolidated workloads can compete fiercely for shared resources, resulting in severe performance variability/degradation. This paper proposes a resource allocation mechanism for a Lambda platform based on the model predictive control framework. Performance evaluation is carried out by comparing the proposed solution with multiple resource allocation heuristics, namely enhanced versions of spread and binpack, and best-effort approaches. Results confirm that the proposed controller increases the overall resource utilization by 37% on average and achieves a significant improvement in preventing QoS violation incidents compared to others.

The emerging Internet of Things (IoT) paradigm creates a growing need to analyze a significant amount of data produced by the interconnected IoT devices. Since IoT devices have limited computation capabilities, Fog Computing is a natural complement, to provide distributed, location-aware, and easy-to-access computation resources. In this work, we address the problem of application processing and data offloading in a Fog-integrated IoT ecosystem. By leveraging the Lyapunov optimization technique, we design an online and distributed system control policy called the Distributed Weighted Backpressure (DWB) policy that asymptotically minimizes the cost of IoT devices. A three-way tradeoff among queue backlogs, communication cost, and computation cost is then investigated. Finally, simulation study has been conducted to validate the correctness and usefulness of the proposed DWB policy.

Recent technological trends such as Industry 4.0 introduced new challenges that push the limit of current computer and networking architectures. It demands the connection of thousands, if not millions, of sensors and mobile devices coupled with optimized operations to automate various operations inside factories. This led to the new era of Internet of Things (IoTs) where lightweight (possibly mobile) devices are envisaged to send vital information to cloud data centres (mobile and fixed infrastructure) for further processing and decision making [1].

The key to maintaining high standards of quality and power conservation of physical machines in data centers lies in efficient consolidation of virtual machines (VMs). Several schemes have been proposed for this purpose; and these include online migration and VM placement - which can offer the best in terms of resource utilization. The consolidation process can be made effective by finding “opportunities” to migrate VMs as well approximating the resource utilization for the VM placement. An inefficient placement scheme, however, will lead to a substantial overloading of physical machines. This proposed VM placement scheme uses correlation coefficient and predicted future requirements of computing resources to accurately compute the value/s of variable, and has been termed LIFE - Lowest Interdependence Factor Exponent. This variable shows the extent to which a VM can be associated with a target physical machine. Higher value of LIFE will correspondingly result in a larger impact factor influencing the performance of existing VMs whenever a VM is selected for migration to a target machine. To minimize performance degradation, migration of a VM to a target machine will only take place if it is found to correspond with a value of LIFE that is found to be the lowest. Intensive experiments show that the proposed scheme offers better performance attributes over Minimum Correlation Coefficient (MCC) and Power Aware Best Fit Decreasing (PABFD) schemes measured in terms of the following metrics: power consumption by 44.08% and 27.52%, SLA violation by 50.90% and 19.53% and number of VM migration by 52.91% and 9.66% respectively.

Nowadays, cloud computing has emerged as the most popular computing platform due to its ability to reduce the operating costs. A sizable chunk of the operating costs is composed of power consumption of the cloud data center alone. It has been observed that improperly sized virtual machines result in comparatively higher power consumption. Although efforts have been made to address the issue of high power consumption, still efficient resource allocation in a cloud environment continues to pose a significant challenge. The proposed design is based on the concept of clustering tasks based on computing resource requirements, and then distributing the tasks among appropriately sized virtual machines on the basis of computing resources. The current paper evaluates the proposed model against representational techniques in the field. A benchmark dataset from Google cloud trace is used for the evaluation. Empirical results analysis has shown that the proposed model offers significant advantages over the existing schemes in so far as load balancing of active physical servers is concerned in terms of defined performance metrics.

Scientific workflows are increasingly containerised, which requires rethinking central processing unit (CPU) sharing policies to accommodate different workload types. However, container engines running scientific workflows struggle to share the CPU fairly, as workload characteristics are not taken into account. This paper proposes a sharing policy called the Adaptive Completely Fair Scheduling policy (adCFS), which considers the future state of CPU usage and proactively shares CPU cycles between various containers based on their corresponding workload metrics (e.g., CPU usage, task runtime, #tasks). adCFS estimates the weight of workload characteristics and redistributes the CPU based on the corresponding weights. The Markov chain model is used to predict CPU state use, and the adCFS policy is triggered to dynamically allocate containers to the proper CPU portions. Experimental results show enhanced container CPU response time for those containers that run heavy and large jobs: these display 12% faster response time compared with the default CFS (Completely Fair Scheduler). adCFS therefore enhances CFS by considering workload metrics, which leads to the CPU being shared fairly when it is fully used.

Recent distributed key-value data stores, such as Aerospike are getting the momentum with ever-increasing need for large-scale real-time data processing. While these data stores can provide significantly improved performance, they still struggle to meet Quality of Service (QoS) during workload surges. In this paper, we address the problem of QoS-aware resource allocation for burst workloads in key-value data stores. To this end, we design a resource allocation controller, which enables each application to independently regulate the releases of its requests taking into account QoS. In particular, the proposed controller monitors the actual performance metrics of the target system and dynamically releases requests from a buffer owned by each application accordingly. We have implemented the proposed controller in an Aerospike cluster for our performance evaluation. Experiments have been conducted with various workload intensities (with up to 36,180 write operations per second) in comparison with the default Aerospike policy. Experimental results confirm that the proposed controller decreases the overall average latency up to 41% on high-rate workload while maintaining the QoS of high priority applications.

This paper addresses the challenging problem of guaranteeing quality-of-service (QoS) requirements associated with parallel running queries in distributed stream processing engines. In such platforms, the real-time processing of streaming data often requires executing a set of user-defined queries over continues data flows. However, previous studies showed that guaranteeing QoS enforcement (such as end-to-end response time) for a collection of applications is a complex problem. This paper presents an advanced resource allocation strategy to tackle such a problem by considering the traffic pattern of individual data streams. To properly allocate resource for streaming queries execution, we define a certain number of priority channels to categorize the streaming data across the system. The resource allocation is addressed as an optimization problem where a set of cost functions is defined to achieve the following goals: a) reduce the sum of QoS violation incidents across all applications; b) increase the CPU utilization level, and (c) avoid the additional costs caused by frequent reconfigurations. The proposed solution does not depend on any assumption about the incoming data rate or the query processing time. The performance of the proposed solution is benchmarked, and the experimental results reveal that the proposed scheme increases the overall resource utilization by 23% on average and reduces the QoS violations by 29% against round-robin strategy. It could also prevent QoS violation incidents at different levels by tuning the cost function.

Virtual machine (VM) live migration has been applied to system load balancing in cloud environments for the purpose of minimizing VM downtime and maximizing resource utilization. However, the migration process is both time- and cost-consuming as it requires the transfer of large size files or memory pages and consumes a huge amount of power and memory for the origin and destination physical machine (PM), especially for storage VM migration. This process also leads to VM downtime or slowdown. To deal with these shortcomings, we develop a Multi-objective Load Balancing (MO-LB) system that avoids VM migration and achieves system load balancing by transferring extra workload from a set of VMs allocated on an overloaded PM to other compatible VMs in the cluster with greater capacity. To reduce the time factor even more and optimize load balancing over a cloud cluster, MO-LB contains a CPU Usage Prediction (CUP) sub-system. The CUP not only predicts the performance of the VMs but also determines a set of appropriate VMs with the potential to execute the extra workload imposed on the VMs of an overloaded PM. We also design a Multi-Objective Task Scheduling optimization model using Particle Swarm Optimization to migrate the extra workload to the compatible VMs. The proposed method is evaluated using a VMware-vSphere-based private cloud in contrast to the VM migration technique applied by vMotion. The evaluation results show that the MO-LB system dramatically increases VM performance while reducing service response time, memory usage, job makespan, power consumption and the time taken for the load balancing process.

High reliability, efficient I/O performance and flexible consistency provided with low storage cost are all desirable properties of cloud storage systems. Due to the inherent conflicts, however, simultaneously achieving optimum on all these properties is impractical. N-way Replication and Erasure Coding, two extensively-applied storage schemes with high reliability, adopt opposite and unbalanced strategies on the tradeoff among these properties, thus considerably restraining their effectiveness on wide range of workloads. To address the aforementioned obstacle, we propose a novel storage scheme called ASSER, an ASSembling chain of Erasure coding and Replication. ASSER stores each object in two parts: a full copy and a certain amount of erasure-coded segments. We establish dedicated read/write protocols for ASSER leveraging the unique structural advantages. On the basis of elementary protocols, we implement sequential and PRAM (Pipeline-RAM) consistency to make ASSER feasible for various services with different performance/consistency requirements. Evaluation results demonstrate that under the same fault tolerance and consistency level, ASSER outperforms N-way replication and pure erasure coding in I/O throughput under diverse system and workload configurations with superior performance stability. More importantly, ASSER delivers stably efficient I/O performance at much lower storage cost than the other comparatives.

Cloud computing has become a de facto approach for service provisioning over the Internet. It operates relying on a pool of shared computing resources available on demand and usually hosted in data centers. Assessing performance and energy efficiency of data centers becomes fundamental. Industries use a number of metrics to assess efficiency and energy consumption of cloud computing systems, focusing mainly on the efficiency of IT equipment, cooling and power distribution systems. However, none of the existing metrics is precise enough to distinguish and analyze the performance of data center communication systems from IT equipment. This paper proposes a framework of new metrics able to assess performance and energy efficiency of cloud computing communication systems, processes and protocols. The proposed metrics have been evaluated for the most common data center architectures including fat tree three-tier, BCube, DCell and Hypercube.

In recent years, recommendation systems have seen significant evolution in the field of knowledge engineering. Most of the existing recommendation systems based their models on collaborative filtering approaches that make them simple to implement. However, performance of most of the existing collaborative filtering-based recommendation system suffers due to the challenges, such as: (a) cold start, (b) data sparseness, and (c) scalability. Moreover, recommendation problem is often characterized by the presence of many conflicting objectives or decision variables, such as users' preferences and venue closeness. In this paper, we proposed MobiContext, a hybrid cloud-based bi-objective recommendation framework (BORF) for mobile social networks. The MobiContext utilizes multi-objective optimization techniques to generate personalized recommendations. To address the issues pertaining to cold start and data sparseness, the BORF performs data preprocessing by using the Hub-Average (HA) inference model. Moreover, the Weighted Sum Approach (WSA) is implemented for scalar optimization and an evolutionary algorithm (NSGA-II) is applied for vector optimization to provide optimal suggestions to the users about a venue. The results of comprehensive experiments on a large-scale real dataset confirm the accuracy of the proposed recommendation framework.

Utility based cloud services can efficiently provide various supportive services to different service providers. Trust negotiations with federated identity management are vital for preserving privacy in open systems such as distributed collaborative systems. However, due to the large amounts of server based communications involved in trust negotiations scalability issues prove to be less cumbersome when offloaded on to the cloud as a utility service. In this view, we propose trust based federated identity management as a cloud based utility service. The main component of this model is the trust establishment between the cloud service provider and the identity providers. We propose novel trust metrics based on the potential vulnerability to be attacked, the available security enforcements and a novel cost metric based on policy dependencies to rank the cooperativeness of identity providers. Practical use of these trust metrics is demonstrated by analyses using simulated data sets, attack history data: published by MIT Lincoln laboratory, real-life attacks and vulnerabilities extracted from Common Vulnerabilities and Exposures (CVE) repository and fuzzy rule based evaluations. The results of the evaluations imply the significance of the proposed trust model to support cloud based utility services to ensure reliable trust negotiations using federated identity management.

The fusion of remote sensing images with different spatial and temporal resolutions is needed for diverse Earth observation applications. A small number of spatiotemporal fusion methods that use sparse representation appear to be more promising than weighted- and unmixing-based methods in reflecting abruptly changing terrestrial content. However, none of the existing dictionary-based fusion methods consider the downsampling process explicitly, which is the degradation and sparse observation from high-resolution images to the corresponding low-resolution images. In this paper, the downsampling process is described explicitly under the framework of compressed sensing for reconstruction. With the coupled dictionary to constrain the similarity of sparse coefficients, a new dictionary-based spatiotemporal fusion method is built and named compressed sensing for spatiotemporal fusion, for the spatiotemporal fusion of remote sensing images. To deal with images with a high-resolution difference, typically Landsat-7 and Moderate Resolution Imaging Spectrometer (MODIS), the proposed model is performed twice to shorten the gap between the small block size and the large resolution rate. In the experimental procedure, the near-infrared, red, and green bands of Landsat-7 and MODIS are fused with root mean square errors to check the prediction accuracy. It can be concluded from the experiment that the proposed methods can produce higher quality than five state-of-the-art methods, which prove the feasibility of incorporating the downsampling process in the spatiotemporal model under the framework of compressed sensing.

In this paper, we focus on detecting a special type of anomaly in wireless sensor network (WSN), which appears simultaneously in a collection of neighboring nodes and lasts for a significant period of time. Existing point-based techniques, in this context, are not very effective and efficient. With the proposed distributed segment-based recursive kernel density estimation, a global probability density function can be tracked and its difference between every two periods of time is continuously measured for decision making. Kullback-Leibler (KL) divergence is employed as the measure and, in order to implement distributed in-network estimation at a lower communication cost, several types of approximated KL divergence are proposed. In the meantime, an entropic graph-based algorithm that operates in the manner of centralized computing is realized, in comparison with the proposed KL divergence-based algorithms. Finally, the algorithms are evaluated using a real-world data set, which demonstrates that they are able to achieve a comparable performance at a much lower communication cost.

In this paper, we provide the detailed analysis of the achievable throughput of infrastructure-based vehicular network with a finite traffic density under a cooperative communication strategy, which explores the combined use of vehicle-to-infrastructure (V2I) communications, vehicle-to-vehicle (V2V) communications, the mobility of vehicles, and cooperations among vehicles and infrastructure to facilitate the data transmission. A closed form expression of the achievable throughput is obtained, which reveals the relationship between the achievable throughput and its major performance-impacting parameters, such as distance between adjacent infrastructure points, the radio ranges of infrastructure and vehicles, the transmission rates of V2I and V2V communications, and vehicular density. Numerical and simulation results show that the proposed cooperative communication strategy significantly increases the throughput of vehicular networks, compared with its non-cooperative counterpart, even when the traffic density is low. Our results shed insight on the optimum deployment of vehicular network infrastructure and the optimum design of cooperative communication strategies in vehicular networks to maximize the throughput.

Under today's bursty web traffic, the fine-grained per-container control promises more efficient resource provisioning for web services and better resource utilization in cloud datacenters. In this paper, we present Two-stage Stochastic Programming Resource Allocator (2SPRA). It optimizes resource provisioning for containerized n-tier web services in accordance with fluctuations of incoming workload to accommodate predefined SLOs on response latency. In particular, 2SPRA is capable of minimizing resource over-provisioning by addressing dynamics of web traffic as workload uncertainty in a native stochastic optimization model. Using special-purpose OpenOpt optimization framework, we fully implement 2SPRA in Python and evaluate it against three other existing allocation schemes, in a Docker-based CoreOS Linux VMs on Amazon EC2. We generate workloads based on four real-world web traces of various traffic variations: AOL, WorldCup98, ClarkNet, and NASA. Our experimental results demonstrate that 2SPRA achieves the minimum resource over-provisioning outperforming other schemes. In particular, 2SPRA allocates only 6.16 percent more than application's actual demand on average and at most 7.75 percent in the worst case. It achieves 3x further reduction in total resources provisioned compared to other schemes delivering overall cost-savings of 53.6 percent on average and up to 66.8 percent. Furthermore, 2SPRA demonstrates consistency in its provisioning decisions and robust responsiveness against workload fluctuations.

It has long been an important issue in various disciplines to examine massive multidimensional data superimposed by a high level of noises and interferences by extracting the embedded multi-way factors. With the quick increases of data scales and dimensions in the big data era, research challenges arise in order to (1) reflect the dynamics of large tensors while introducing no significant distortions in the factorization procedure and (2) handle influences of the noises in sophisticated applications. A hierarchical parallel processing framework over a GPU cluster, namely H-PARAFAC, has been developed to enable scalable factorization of large tensors upon a “divide-and-conquer” theory for Parallel Factor Analysis (PARAFAC). The H-PARAFAC framework incorporates a coarse-grained model for coordinating the processing of sub-tensors and a fine-grained parallel model for computing each sub-tensor and fusing sub-factors. Experimental results indicate that (1) the proposed method breaks the limitation on the scale of multidimensional data to be factorized and dramatically outperforms the traditional counterparts in terms of both scalability and efficiency, e.g., the runtime increases in the order of n2 when the data volume increases in the order of n3, (2) H-PARAFAC has potentials in refraining the influences of significant noises, and (3) H-PARAFAC is far superior to the conventional window-based counterparts in preserving the features of multiple modes of large tensors.

Big data analytics has attracted close attention from both industry and academic because of its great benefits in cost reduction and better decision making. As the fast growth of various global services, there is an increasing need for big data analytics across multiple data centers (DCs) located in different countries or regions. It asks for the support of a cross-DC data processing platform optimized for the geo-distributed computing environment. Although some recent efforts have been made for geo-distributed big data analytics, they cannot guarantee predictable job completion time, and would incur excessive traffic overthe inter-DC network that is a scarce resource shared by many applications. In this paper, we study to minimize the inter-DC traffic generated by MapReduce jobs targeting on geo-distributed big data, while providing predicted job completion time. To achieve this goal, we formulate an optimization problem by jointly considering input data movement and task placement. Furthermore, we guarantee predictable job completion time by applying the chance-constrained optimization technique, such that the MapReduce job can finish within a predefined job completion time with high probability. To evaluate the performance of our proposal, we conduct extensive simulations using real traces generated by a set of queries on Hive. The results show that our proposal can reduce 55 percent inter-DC traffic compared with centralized processing by aggregating all data to a single data center.

Distributed data stream mining in a sliding window has emerged recently, due to its applications in many domains including large Telecoms and Internet Service Providers, financial tickers, ATM and credit card operations in banks and transactions in retail chains. Many of these large-scale applications prohibit monitoring data centrally at a single location due to their massive volume of the data; therefore, data acquisition, processing, and mining tasks are often distributed to a number of processing nodes, which monitor their local streams and exchange only the summary of data either periodically or on demand. While this offer many advantages, distributed stream applications possess significant challenges including problems related to an online analysis of the recent data, communication efficiency and various estimation of various complex queries. There are few existing techniques which solve problems related to distributed sliding window data stream; however, those techniques are focused on solving only simple problems and require high space, query, and communication cost, which can be a bottleneck for many of these large scale applications. In this paper, we propose an efficient query estimation technique by constructing a small sketch of the data stream. The constructed sketch uses a deterministic sliding window model and can estimate various complex queries, for both centralized and distributed applications; including point queries (i.e., range queries and heavy hitter queries), quantiles, inner product, and self-join size queries, with deterministic guarantees on the precision. The proposed approach improves upon recent existing work for these problems, in terms of the memory and query cost in a centralized setting and in terms of communication cost and merge complexity in a distributed setting. It requires O(1/ε21 log (εN)) memory (where 0 <; ε <; 1 is a user defined parameter), can provide estimates in O(1) time, and processes each incoming record in O(1) amortized time. Detailed experimental analysis, both in centralized and distributed settings demonstrates that in practice the proposed approach uses about six times less memory, and has about eight times less query time when compared to ECM sketches. In a distributed application, the proposed technique also significantly improves (around seven times) on the communication cost between distributed sites.

Hybrid clouds have gained popularity in recent times in a variety of organizations due to their ability to provide additional capacity in a public cloud, to augment private cloud capacity, when it is needed. However, scheduling distributed applications' jobs (e.g, workflow tasks) on hybrid cloud resources introduces new challenges. One key problem is the danger of exposing private data and jobs in a third-party public cloud infrastructure, for example in healthcare applications. In this article, we tackle the problem of designing workflow scheduling algorithms to meet customers' deadlines, while not compromising data and task privacy requirements. Our work is different from most studies on workflow scheduling where the main goal is to achieve a balance between desirable, yet incompatible constraints, such as meeting the deadline and/or minimizing the execution time. Although many others have addressed the trade-off between cost and time, or privacy and cost, their work still suffers from an insufficient consideration of the trade-off between privacy and time. To address such shortcomings in the literature, we present a new SaaS scheduling broker composed of MPHC-P1, MPHCP2, and MPHC-P3 policies to preserve privacy while scheduling the workflows' tasks under customers' deadlines. We evaluated our approach using real workflows running on a VMware based hybrid cloud. Results demonstrate that under our scheduling policies, MPHC-P2 and MPHC-P3 are promising in time-critical scenarios by reducing the total cost by 10-20 percent compared to alternatives. Overall, results show that our approach is efficient in reducing the cost of executing workflows while satisfying both their privacy and deadline constraints.

Distributed storage systems, e.g., Hadoop Distributed File System (HDFS), have been widely used in datacenters for handling large amounts of data due to their excellent performance in terms of fault tolerance, reliability and scalability. However, these storage systems usually adopt the same replication and storage strategy to guarantee data availability, i.e., creating the same number of replicas for all data sets and randomly storing them across data nodes. Such strategies do not fully consider the difference requirements of data availability on different data sets. More servers than necessary should thus be used to store replicas of rarely-used data, which will lead to increased energy consumption. To address this issue, we propose an energy-efficient storage strategy for cloud datacenters based on a novel hypergraph coverage model. According to users' requirements of data availability in different applications, our proposed algorithm can selectively determine the corresponding minimum hyperedge coverage, which represents the minimum set of data nodes required in the datacenter. Hence, some other data nodes can be turned off for the purpose of energy saving. We have also implemented our proposed algorithm as a dynamic runtime strategy in a HDFS based prototype datacenter for performance evaluation. Experimental results show that the variable hypergraph coverage based strategy can not only reduce energy consumption, but can also improve the network performance in the datacenter.

To respond to emergencies in a fast and an effective manner, it is of critical importance to have efficient evacuation plans that lead to minimum road congestions. Although emergency evacuation systems have been studied in the past, the existing approaches, mostly based on multi-objective optimizations, are not scalable enough when involve numerous time varying parameters, such as traffic volume, safety status, and weather conditions. In this paper, we propose a scalable emergency evacuation service, termed the MacroServ that recommends the evacuees with the most preferred routes towards safe locations during a disaster. Unlike many existing approaches that model systems with static network characteristics, our approach considers real-time road conditions to compute the maximum flow capacity of routes in the transportation network. The evacuees are directed towards those routes that are safe and have least congestion resulting in decreased evacuation time. We utilized probability distributions to model the real-life stochastic behaviors of evacuees during emergency scenarios. The results indicate that recommendation of appropriate routes during emergency scenarios play a critical role in quicker and safe evacuation of the population.

The advances in mobile technologies enable mobile devices to perform tasks that are traditionally run by personal computers as well as provide services to the others. Mobile users can form a service sharing community within an area by using their mobile devices. This paper highlights several challenges involved in building such service compositions in mobile communities when both service requesters and providers are mobile. To deal with them, we first propose a mobile service provisioning architecture named a mobile service sharing community and then propose a service composition approach by utilizing the Krill-Herd algorithm. To evaluate the effectiveness and efficiency of our approach, we build a simulation tool. The experimental results demonstrate that our approach can obtain superior solutions as compared with current standard composition methods in mobile environments. It can yield near-optimal solutions and has a nearly linear complexity with respect to a problem size.

Reports on the current scope and state of the IEEE Transactions on Sustainable Computing.

In this paper, we analyze the coverage probability and the area spectral efficiency (ASE) for the uplink (UL) of dense small cell networks (SCNs) considering a practical path loss model incorporating both line-of-sight (LoS) and non-line-ofsight (NLoS) transmissions. Compared with the existing work, we adopt the following novel approaches in this paper: 1) we assume a practical user association strategy (UAS) based on the smallest path loss, or equivalently the strongest received signal strength; 2) we model the positions of both base stations (BSs) and the user equipments (UEs) as two independent homogeneous Poisson point processes; and 3) the correlation of BSs' and UEs' positions is considered, thus making our analytical results more accurate. The performance impact of LoS and NLoS transmissions on the ASE for the UL of dense SCNs is shown to be significant, both quantitatively and qualitatively, compared with existing work that does not differentiate LoS and NLoS transmissions. In particular, existing work predicted that a larger UL power compensation factor would always result in a better ASE in the practical range of BS density, i.e., 101~ 103BSs/km2. However, our results show that a smaller UL power compensation factor can greatly boost the ASE in the UL of dense SCNs, i.e., 102~ 103BSs/km2, while a larger UL power compensation factor is more suitable for sparse SCNs, i.e., 101 ~ 102 BSs/km2.

Mobile cloud applications have become extremely popular in the last years. Location-based services, navigation, online gaming and social networking are a representative set of “always on” cloud applications in which the same or partially overlapping content is delivered to multiple users. Network coding is a well matching solution to improve content delivery. In this paper we propose the vNC-CELL technique, which uses network coding to combine information flows carrying the same or overlapping content that has to be delivered to co-located users. vNC-CELL executes coding functionalities in a mobile cloud through virtualization as these operations are computationally intensive if performed locally at the base station. Performance evaluation obtained from NS-3 simulations confirms vNC-CELL ability to improve network throughput and reduce download times for the users.

Temperature prediction can enhance datacenter thermal management towards minimizing cooling power draw. Traditional approaches achieve this through analyzing task-temperature profiles or resistor-capacitor circuit models to predict CPU temperature. However, they are unable to capture task resource heterogeneity within multi-tenant environments and make predictions under dynamic scenarios such as virtual machine migration, which is one of the main characteristics of Cloud computing. This paper proposes virtual machine level temperature prediction in Cloud datacenters. Experiments show that the mean squared error of stable CPU temperature prediction is within 1.10, and dynamic CPU temperature prediction can achieve 1.60 in most scenarios.

Data center efficiency is primarily sought by sharing physical resources, such as processors, memory, and disks in the form of virtual machines or containers among multiple users, i.e., workload consolidation. However, the reality is co-located applications in these virtual platforms compete for resources and interfere with each others' performance, resulting in performance variability/degradation. In this paper, we present the contentionaware resource allocation (CARA) solution, which optimizes data center efficiency. It is essentially devised based on a model predictive control that enables to make judicious consolidation decisions with future system states. CARA consolidates workloads explicitly taking into account the correlation between shared and isolated resource usage patterns. Based on our experimental results, CARA improves the overall resource utilization by 32%, without a significant impact on the quality-of-service (QoS) enforcement level. Such improvement results in a fewer number of active servers and in turn contributes to an overall energy saving by 33%.

Apache Storm has recently emerged as an attractive fault-tolerant open-source distributed data processing platform that has been chosen by many industry leaders to develop real-time applications for processing a huge amount of data in a scalable manner. A key aspect to achieve the best performance in this system lies on the design of an efficient scheduler for component execution, called topology, on the available computing resources. In response to workload fluctuations, we propose an advanced scheduler for Apache Storm that provides improved performance with highly dynamic behavior. While enforcing the required Quality-of-Service (QoS) of individual data streams, the controller allocates computing resources based on decisions that consider the future states of non-controllable disturbance parameters, e.g. arriving rate of tuples or resource utilization in each worker node. The performance evaluation is carried out by comparing the proposed solution with two well-known alternatives, namely the Storm's default scheduler and the best-effort approach (i.e. the heuristic that is based on the first-fit decreasing approximation algorithm). Experimental results clearly show that the proposed controller increases the overall resource utilization by 31% on average compared to the two others solutions, without significant negative impact on the QoS enforcement level.

The steep rise of Internet of Things (IoT) applications along with the limitations of Cloud Computing to address all IoT requirements promotes a new distributed computing paradigm called Fog Computing, which aims to process data at the edge of the network. With the help of Fog Computing, the transmission latency and monetary spending caused by Cloud Computing can be effectively reduced. However, executing all applications in fog nodes will increase the average response time since the processing capabilities of fog is not as powerful as cloud. A tradeoff issue needs to be addressed within such systems in terms of average response time and average cost. In this paper, we develop an online algorithm, unit-slot optimization, based on the technique of Lyapunov optimization. It is a quantified near optimal solution and can online adjust the tradeoff between average response time and average cost. We evaluate the performance of our proposed algorithm by a number of experiments. The experimental results not only match up the theoretical analyses properly, but also demonstrate that our proposed algorithm can provide cost-effective processing while guaranteeing average response time.

Exploiting temporal and spatial locality is a way to improve the performance of data compression and deduplication in a storage system. Through our evaluation, we find that content level similarity measures such as similar tags of photos have a certain correlation to data compressibility. Raw images with similar tags can be compressed together to get better storage space savings. Furthermore, storing similar raw images together enables rapid data sorting, searching, and retrieval if the images are stored in a distributed and large-scale environment with reduced fragmentation. In this paper, we present the correlation results between content similarity and data compressibility using a dataset built from Flickr. The system design we proposed has been based on the evaluation and it optimizes storage efficiency for Top-N relevant images with the same tag. On one hand, the storage space is saved. On the other hand, the design may accelerate the query performance for Top-N relevance search.

Summary This work presents a genetic algorithm (GA)‐based optimization technique, called GA‐ParFnt, to find the Pareto frontier for optimizing data transfer versus job execution time in grids. As the performance of a generic GA is not suitable to find such Pareto relationship, major modifications are applied to it so that it can efficiently discover such relationship. The frontier curve representing this relationship is then matched against performance of several scheduling techniques—for both data intensive and computationally intensive applications—to measure their overall performances. Results show that few of these algorithms are far from the Pareto front despite their claims of being efficient in optimizing their targeted objectives. Results also provide invaluable insights into this formidable problem and should aid in the design of future schedulers. Copyright © 2012 John Wiley & Sons, Ltd.

Although most current cloud providers, such as Amazon Web Services (AWS) and Microsoft Azure offer different types of computing instances with different capacities, cloud users tend to hire a cluster of instances of particular type to ensure performance predictability for their applications. Nowadays, many large-scale applications including big data analytics applications feature workload patterns that have heterogeneous resource demands, for which, accounting for heterogeneity of virtual cloud instances to allocate would be highly advantageous to the application performance. However, performance predictability has been always an issue in such clusters of heterogeneous resources. In particular, to precisely decide on what instances from which types to enclose in a cluster, such that the desired performance is attained, remains an open question. To this end, we devise a resource allocation mechanism by formulating it as a Mixed-Integer programming model representing an optimization problem. Our resource allocation mechanism incorporates predictable average performance as a unified performance metric, which concerns two key performance-related issues: (a) Performance variation within same-type instances, and (b) Correlations of performance variabilities across different types. Our experimental results demonstrate that target performance is predictable and attainable for clusters of heterogeneous resources. Our mechanism constructs clusters whose performance is within 95 percent of the performance of optimal ones, hence deadlines are always met. By reoptimisation, our mechanism can react to performance mispredictions and support autoscaling for varying workloads. We experimentally verify our findings using clusters on Amazon EC2 with MapReduce workloads, and on a private cloud as well. We conduct comparison experiments with an existing recent resource allocation approach in literature.

The unprecedented capabilities of monitoring and responding to stimuli in the physical world of wireless sensor and actuator networks (WSAN) enable these networks to provide the underpinning for several Smart City applications, such as structural health monitoring (SHM). In such applications, civil structures, endowed with wireless smart devices, are able to self-monitor and autonomously respond to situations using computational intelligence. This work presents a decentralized algorithm for detecting damage in structures by using a WSAN. As key characteristics, beyond presenting a fully decentralized (in-network) and collaborative approach for detecting damage in structures, our algorithm makes use of cooperative information fusion for calculating a damage coefficient. We conducted experiments for evaluating the algorithm in terms of its accuracy and efficient use of the constrained WSAN resources. We found that our collaborative and information fusion-based approach ensures the accuracy of our algorithm and that it can answer promptly to stimuli (1.091 s), triggering actuators. Moreover, for 100 nodes or less in the WSAN, the communication overhead of our algorithm is tolerable and the WSAN running our algorithm, operating system and protocols can last as long as 468 days.

In a two-server password-authenticated key exchange (PAKE) protocol, a client splits its password and stores two shares of its password in the two servers, respectively, and the two servers then cooperate to authenticate the client without knowing the password of the client. In case one server is compromised by an adversary, the password of the client is required to remain secure. In this paper, we present two compilers that transform any two-party PAKE protocol to a two-server PAKE protocol on the basis of the identity-based cryptography, called ID2S PAKE protocol. By the compilers, we can construct ID2S PAKE protocols which achieve implicit authentication. As long as the underlying two-party PAKE protocol and identity-based encryption or signature scheme have provable security without random oracles, the ID2S PAKE protocols constructed by the compilers can be proven to be secure without random oracles. Compared with the Katz et al.'s two-server PAKE protocol with provable security without random oracles, our ID2S PAKE protocol can save from 22 to 66 percent of computation in each server.

The research on cyber-physical systems (CPSs) addresses the close interactions between the embedded cyber components and the dynamic physical components that could involve mechanical components, energy systems, human activities, and surrounding environment. Some example CPSs include automotive systems, energy systems, robot systems, and cyber-physical biochip systems.

Supervisory control and data acquisition (SCADA) systems have become a salient part in controlling critical infrastructures, such as power plants, energy grids, and water distribution systems. In the past decades, these systems were isolated and use proprietary software, operating systems, and protocols. In recent years, SCADA systems have been interfaced with enterprise systems, which therefore exposed them to the vulnerabilities of the Internet and the security threats. Traditional security solutions (e.g., firewalls, antivirus software, and intrusion detection systems) cannot fully protect SCADA systems, because they have different requirements. This paper presents an innovative intrusion detection approach to detect SCADA tailored attacks. This is based on a data-driven clustering technique of process parameters, which automatically identifies the normal and critical states of a given system. Later, it extracts proximity-based detection rules from the identified states for monitoring purposes. The effectiveness of the proposed approach is tested by conducting experiments on eight data sets that consist of process parameters' values. The empirical results demonstrated an average accuracy of 98% in automatically identifying the critical states, while facilitating the monitoring of the SCADA system.

The concept of smart home has recently gained significant popularity. Despite that it offers improved convenience and cost reduction, the prevailing smart home infrastructure suffers from vulnerability due to cyberattacks. It is possible for hackers to launch cyberattacks at the community level while causing a large area power system blackout through cascading effects. In this paper, the cascading impacts of two cyberattacks on the predicted dynamic electricity pricing are analyzed. In the first cyberattack, the hacker manipulates the electricity price to form peak energy loads such that some transmission lines are overloaded. Those transmission lines are then tripped and the power system is separated into isolated islands due to the cascading effect. In the second cyberattack, the hacker manipulates the electricity price to increase the fluctuation of the energy load to interfere the frequency of the generators. The generators are then tripped by the protective procedures and cascading outages are induced in the transmission network. The existing technique only tackles overloading cyberattack while still suffering from the severe limitation in scalability. Therefore, based on partially observable Markov decision processes, a hierarchical detection framework exploring community decomposition and global policy optimization is proposed in this work. The simulation results demonstrate that our proposed hierarchical computing technique can effectively and efficiently detect those cyberattacks, achieving the detection accuracy of above 98%, while improving the scalability.

Cloud data owners prefer to outsource documents in an encrypted form for the purpose of privacy preserving. Therefore it is essential to develop efficient and reliable ciphertext search techniques. One challenge is that the relationship between documents will be normally concealed in the process of encryption, which will lead to significant search accuracy performance degradation. Also the volume of data in data centers has experienced a dramatic growth. This will make it even more challenging to design ciphertext search schemes that can provide efficient and reliable online information retrieval on large volume of encrypted data. In this paper, a hierarchical clustering method is proposed to support more search semantics and also to meet the demand for fast ciphertext search within a big data environment. The proposed hierarchical approach clusters the documents based on the minimum relevance threshold, and then partitions the resulting clusters into sub-clusters until the constraint on the maximum size of cluster is reached. In the search phase, this approach can reach a linear computational complexity against an exponential size increase of document collection. In order to verify the authenticity of search results, a structure called minimum hash sub-tree is designed in this paper. Experiments have been conducted using the collection set built from the IEEE Xplore. The results show that with a sharp increase of documents in the dataset the search time of the proposed method increases linearly whereas the search time of the traditional method increases exponentially. Furthermore, the proposed method has an advantage over the traditional method in the rank privacy and relevance of retrieved documents.

Service mashups are applications created by combining single-functional services (or APIs) dispersed over the web. With the development of cloud computing and web technologies, service mashups are becoming more and more widely used and a large number of mashup platforms have been produced. However, due to the proliferation of services on the web, how to select component services to create mashups has become a challenging issue. Most developers pay more attention to the quality of service (QoS) and cost of services. Beside service selection, mashup deployment is another pivotal process, as the platform can significantly affect the quality of mashups. In this paper, we focus on creating service mashups from the perspective of developers. A genetic algorithm-based method, genetic algorithm for mashup creation (GA4MC), is proposed to select component services and deployment platforms in order to create service mashups with optimal cost performance. A series of experiments are conducted to evaluate the performance of GA4MC. The results show that the GA4MC method can achieve mashups whose cost performance is extremely close to the optimal. Moreover, the execution time of GA4MC is in a low order of magnitude and the algorithm performs good scalability as the experimental scale increases.

Software-defined networking (SDN) is an emerging network paradigm that simplifies network management by decoupling the control plane and data plane, such that switches become simple data forwarding devices and network management is controlled by logically centralized servers. In SDN-enabled networks, network flow is managed by a set of associated rules that are maintained by switches in their local Ternary Content Addressable Memories (TCAMs) which support high-speed parallel lookup on wildcard patterns. Since TCAM is an expensive hardware and extremely power-hungry, each switch has only limited TCAM space and it is inefficient and even infeasible to maintain all rules at local switches. On the other hand, if we eliminate TCAM occupation by forwarding all packets to the centralized controller for processing, it results in a long delay and heavy processing burden on the controller. In this paper, we strive for the fine balance between rule caching and remote packet processing by formulating a minimum weighted flow provisioning ( MWFP) problem with an objective of minimizing the total cost of TCAM occupation and remote packet processing. We propose an efficient offline algorithm if the network traffic is given, otherwise, we propose two online algorithms with guaranteed competitive ratios. Finally, we conduct extensive experiments by simulations using real network traffic traces. The simulation results demonstrate that our proposed algorithms can significantly reduce the total cost of remote controller processing and TCAM occupation, and the solutions obtained are nearly optimal.

In this paper, we focus on price bidding strategies of multiple users competition for resource usage in cloud computing. We consider the problem from a game theoretic perspective and formulate it into a non-cooperative game among the multiple cloud users, in which each cloud user is informed with incomplete information of other users. For each user, we design a utility function which combines the net profit with time efficiency and try to maximize its value. We design a mechanism for the multiple users to evaluate their utilities and decide whether to use the cloud service. Furthermore, we propose a framework for each cloud user to compute an appropriate bidding price. At the beginning, by relaxing the condition that the allocated number of servers can be fractional, we prove the existence of Nash equilibrium solution set for the formulated game. Then, we propose an iterative algorithm ($\mathcal {IA}$), which is designed to compute a Nash equilibrium solution. The convergency of the proposed algorithm is also analyzed and we find that it converges to a Nash equilibrium if several conditions are satisfied. Finally, we revise the obtained solution and propose a near-equilibrium price bidding algorithm ($\mathcal {NPBA}$) to characterize the whole process of our proposed framework. The experimental results show that the obtained near-equilibrium solution is close to the equilibrium one.

With the advancement of information and communication technology, data are being generated at an exponential rate via various instruments and collected at an unprecedented scale. Such large volume of data generated is referred to as big data, which now are revolutionizing all aspects of our life ranging from enterprises to individuals, from science communities to governments, as they exhibit great potentials to improve efficiency of enterprises and the quality of life. To obtain nontrivial patterns and derive valuable information from big data, a fundamental problem is how to properly place the collected data by different users to distributed clouds and to efficiently analyze the collected data to save user costs in data storage and processing, particularly the cost savings of users who share data. By doing so, it needs the close collaborations among the users, by sharing and utilizing the big data in distributed clouds due to the complexity and volume of big data. Since computing, storage and bandwidth resources in a distributed cloud usually are limited, and such resource provisioning typically is expensive, the collaborative users require to make use of the resources fairly. In this paper, we study a novel collaboration- and fairness-aware big data management problem in distributed cloud environments that aims to maximize the system throughout, while minimizing the operational cost of service providers to achieve the system throughput, subject to resource capacity and user fairness constraints. We first propose a novel optimization framework for the problem. We then devise a fast yet scalable approximation algorithm based on the built optimization framework. We also analyze the time complexity and approximation ratio of the proposed algorithm. We finally conduct experiments by simulations to evaluate the performance of the proposed algorithm. Experimental results demonstrate that the proposed algorithm is promising, and outperforms other heuristics.

With an ever increasing amount of data generated by scientific experiments, social networks and mobile as well as wireless sensor networks, reducing resource consumption by big data applications becomes of paramount importance. Towards this end, filtering data close to the data sources is a common strategy in order to reduce network traffic. Assuming a network of nodes, each potentially generating data and a query in the form of a single operator to be applied in these data, the basic statement of the operator placement problem is: find the best node to place the operator so that the network traffic is minimized. In this paper we study the problem of placing a set of communicating operators exhibiting a tree structure over a tree network of nodes with capacity constraints. We take advantage of our previous work on unconstrained placement in order to develop a new approach enabling both single and group operator migrations using evictions of hosted operators if free space is required. To enhance their applicability, the algorithms work in a distributed asynchronous manner, requiring only minimal knowledge at each network node. Results from simulation experiments show that the proposed algorithms reduce considerably network overhead against their counterparts.

Presents the introductory editorial for this issue of the publication.

Cloud computing has become a de facto approach for service provisioning over the Internet. It operates relying on a pool of shared computing resources available on demand and usually hosted in data centers. Assessing performance and energy efficiency of data centers becomes fundamental. Industries use a number of metrics to assess efficiency and energy consumption of cloud computing systems, focusing mainly on the efficiency of IT equipment, cooling and power distribution systems. However, none of the existing metrics is precise enough to distinguish and analyze the performance of data center communication systems from IT equipment. This paper proposes a framework of new metrics able to assess performance and energy efficiency of cloud computing communication systems, processes and protocols. The proposed metrics have been evaluated for the most common data center architectures including fat-tree three-tier, BCube and DCell.

The power of the crowd, more precisely crowdsourced resources, is in its ubiquity. Accounting for traditional desktop/laptop computers and recent mobile computing devices including tablets and smart phones far surpasses the number of servers in cloud data centers. Besides, the capacity and capability of these resources owned by the crowd (crowd-sourced resources) has increased dramatically with GPUs in particular. Although a myriad of public-resource (or volunteer) computing projects, including SETI@home and Milkyway@home, have attracted the participation of crowd-sourced resources at very large scale, the sustainability of such participation is in doubt due primarily to ever-increasing energy costs. In this paper, we present Crowdware, a framework for enabling sustainable GPU-based public-resource computing with a realistic financial incentive mechanism. To this end, we design an auction-based resource allocation algorithm and a profit-based resource participation algorithm, explicitly considering the electricity cost of participating resources. Our results show that Crowdware greatly promotes profitability and cost efficiency for resource providers and resource consumers, respectively. Specifically, Crowdware has enabled the execution of MD5 password recovery jobs, in our testbed, with only 2.2% of the cost of using Amazon EC2 GPU instances while the participation of crowd-sourced resources is profitable with an average profit rate of 9.2%. Crowdware also shows great scalability with its fat-client and thin-server design. Together, Crowdware significantly improves the sustainability of public-resource computing.

Cloud computing is a computing model where users access ICT services and resources without regard to where the services are hosted. Communication resources often become a bottleneck in service provisioning for many cloud applications. Therefore, data replication which brings data (e.g., databases) closer to data consumers (e.g., cloud applications) is seen as a promising solution. In this paper, we present models for energy consumption and bandwidth demand of database access in cloud computing datacenter. In addition, we propose an energy efficient replication strategy based on the proposed models, which results in improved Quality of Service (QoS) with reduced communication delays. The evaluation results obtained with extensive simulations help to unveil performance and energy efficiency tradeoffs and guide the design of future data replication solutions.

Data traffic from mobile devices experiences unprecedented growth, which current cellular network capacities cannot sustain. Traffic offloading to other type of networks, such as WiFi, can be used to reduce load in cellular networks. In this paper, we propose a novel solution, which unlike other existing methodologies, implements tight cooperation with the cellular network to optimize traffic offloading. The cellular network provides information about channel usage statistics, user mobility patterns, available resources and other parameters. The offloading decisions aim at optimizing the balance between user application requirements and availability of network resources. The validation results, obtained from NS-3 simulations, confirm effectiveness of the proposed solution in balancing cellular traffic load while ensuring QoS.

The shortest path routing (SPR) problem is a well-known challenge in the field of mobile network routing. The aim is to find the least cost path that connect a specific source node with a specific destination node. Although there are numerous algorithms to solve SPR, most of them consider only static environments in which the network topology and link-cost never change. A network with dynamic topologies and cost are indeed more challenging but more practical in real world applications. This paper presents a memetic algorithm for dynamic SPR (DSPR) problems in a mobile network. The proposed approach consists of three stages: genetic algorithm, local search and elitism-based immigrants procedure. Genetic algorithm (GA) is applied in the first stage to explore the search space and generate a new set of solutions. The generated solutions are further improved in the second stage by a local search algorithm. In third stage, an elitism-based immigrants procedure is activated to handle the dynamic changes by maintaining the diversity of the search process. The performance of the proposed algorithm has been evaluated on dynamic shortest path routing problem instances under both cyclic and acyclic environments. The study shows that, on both circumstances, the proposed algorithm is very stable with regards to dynamic network changes. This method is highly competitive compared to state-of-the-art algorithms in the literature as it outperformed these algorithms on all instances of dynamic routing during evaluation.

Data deduplication is used to overcome two issues on Solid State Drives (SSDs). One is price per GB of storage space, and the other is the write limit or disk endurance. By eliminating duplicate data, the deduplication system improves storage efficiency and protects SSD from unnecessary writes. CAFTL [1] is a known solution for deduplication on SSD. We propose a system architecture for inline deduplication based on existing protocol of The Hadoop Distributed File System (HDFS), aiming at addressing performance challenges for primary storage. However, simply applying CAFTL to SSDs in a cluster does not work well. Two routing algorithms are presented and evaluated using selective real-life data sets. Compared to prior work, one routing algorithm (MMHR) may improve the deduplication ratio by 8% at minimal costs while the other (FFFR) can achieve about 30% higher deduplication ratio with tradeoff on chunk level fragmentation. A new research problem of chunk assignment into more than one node for deduplication is also formulated for more studies in this area.

Scientists in different fields, such as high energy physics, earth science, and astronomy are developing large-scale workflow applications. In many use cases, scientists need to run a set of interrelated but independent workflows (i.e., Workflow ensembles) for the entire scientific analysis. As a workflow ensemble usually contains many sub-workflows in each of which hundreds or thousands of jobs exist with precedence constraints, the execution of such a workflow ensemble makes a great concern with cost even using elastic and pay-as-you-go cloud resources. In this paper, we address two main challenges in executing large-scale workflow ensembles in public clouds with both cost and deadline constraints: (1) execution coordination, and (2) resource provisioning. To this end, we develop a new pulling based workflow execution system with a profiling-based resource provisioning strategy. The idea is homogeneity in both scientific workflows and cloud resources can be exploited to remove scheduling overhead (in execution coordination) and to minimize cost meeting deadline. Our results show that our solution system can achieve 80% speed-up, by removing scheduling overhead, compared to the well-known Pegasus workflow management system when running scientific workflow ensembles. Besides, our evaluation using Montage (an astronomical image mosaic engine) workflow ensembles on around 1000-core Amazon EC2 clusters has demonstrated the efficacy of our resource provisioning strategy in terms of cost effectiveness within deadline.

Summary In today's highly parametrized distributed computational environments, such as green grid clusters and clouds, the growing power and cooling rates are becoming the dominant part of the users' and system managers' budgets. Computational grids, owing to their sheer sizes, still require advanced methodologies and strategies for supporting the scheduling of the users' tasks and applications to the distributed resources. The efficient resource allocation becomes even more challenging when energy utilization, beyond the conventional scheduling criteria, such as Makespan , is treated as first‐class additional scheduling objective. In this paper, we address the independent batch scheduling in computational grid as a bi‐objective global minimization problem with Makespan and energy consumption as the main criteria. We apply the dynamic voltage and frequency scaling model for the management of the cumulative power energy utilized by the grid resources. We develop three genetic algorithms as energy‐aware grid schedulers, which were empirically evaluated in three grid size scenarios in static and dynamic modes. The simulation results confirmed the effectiveness of the proposed genetic algorithm‐based schedulers in the reduction of the energy consumed by the whole system and in dynamic load balancing of the resources in grid clusters, which is sufficient to maintain the desired quality level(s). Copyright © 2012 John Wiley & Sons, Ltd.