Accurate forecasting of resource consumption, such as gas, is essential for efficient energy management, cost reduction, and sustainability. Time series forecasting (TSF) techniques like recurrent neural networks (RNNs), convolutional networks (TCNs), and Transformers have been employed to model complex temporal variations but face challenges in capturing long-term dependencies or inter-series relationships. In this paper, we propose a Hierarchical Contrastive Learning approach for Time Series (HCLTS) to improve gas consumption prediction for industrial and commercial users. HCLTS employs a prototype-based approach to construct positive and negative samples, leveraging industry labels and hierarchical sampling. We further adopt a hierarchical contrastive training objective to model underlying consumption patterns across industries. Extensive experiments on a real-world gas consumption dataset demonstrate that HCLTS achieves superior performance over existing methods, highlighting its potential for practical applications in diverse industrial contexts.

The zero-shot retrieval task aims to retrieve the most relevant documents to a user’s query without relevance labels. Current approaches expand input queries by generating pseudo-documents with large language models (LLMs) and perform document retrieval based on the expanded queries. However, their retrieval methods are limited to either sparse or dense retrieval methods alone. In this paper, we propose a hybrid retriever to further improve the quality of the pseudo-documents and to obtain the relevant information more effectively. Specifically, we use sparse retrievers to obtain keyword information and dense retrievers to obtain contextual information. Then, we introduce reciprocal ranking fusion and weighted scoring fusion into both the pre-retrieval of candidate documents and the final-retrieval of final results to calculate the overall hybrid matching score. Experimental results on TREC DL19/DL20 and several datasets from the BEIR benchmark indicate the superiority of our proposed method.

In multivariate time series forecasting, transformer-based methods have gained attention for their ability to capture complex dependencies and are often integrated with graph neural networks to improve forecasting performance. However, these approaches are computationally intensive. Mamba, a more computationally efficient architecture, has emerged as a potential solution. Despite its efficiency, Mamba fails to capture interactions across time and dependencies between variables. Furthermore, the graph structures used in most existing methods to model inter-series relationships primarily capture correlations between pairs of variables, neglecting the similarity of patterns among different variable series. In light of this, we propose a Tribe Graph enhanced Bidirectional Mamba for multivariate time series forecasting (TGBiMamba). This approach leverages bidirectional Mamba as the backbone to capture intra-series interactions and employs a designed tribe graph structure to capture inter-series intrinsic similarities. Extensive experiments on various public datasets and a proprietary industrial dataset highlight the superiority of our method.

Auction-based Federated Learning (AFL) has emerged as an important research field in recent years. The prevailing strategies for FL data consumers (DCs) assume that the entire team of the required data owners (DOs) for an FL task must be assembled before training can commence. In practice, a DC can trigger the FL training process multiple times. DOs can thus be gradually recruited over multiple FL model training sessions. Existing bidding strategies for AFL DCs are not designed to handle such scenarios. Therefore, the problem of multi-session AFL remains open. To address this problem, we propose the Multi-session Budget Optimization Strategy for forward Auction-based Federated Learning (MBOS-AFL). Based on hierarchical reinforcement learning, MBOS-AFL jointly optimizes intersession budget pacing and intra-session bidding for AFL DCs, with the objective of maximizing the total utility. Extensive experiments on six benchmark datasets show that it significantly outperforms seven state-of-the-art approaches. On average, MBOS-AFL achieves 12.28% higher utility, 14.52% more data acquired through auctions for a given budget, and 1.23% higher test accuracy achieved by the resulting FL model compared to the best baseline. To the best of our knowledge, it is the first budget optimization decision support method with budget pacing capability designed for DCs in multi-session forward AFL.

Intelligent fault diagnosis (IFD) plays a crucial role in ensuring the safe operation of industrial machinery and improving production efficiency. However, traditional supervised deep learning methods require a large amount of training data and labels, which are often located in different clients. Additionally, the cost of data labeling is high, making labels difficult to acquire. Meanwhile, differences in data distribution among clients may also hinder the model’s performance. To tackle these challenges, this paper proposes a semi-supervised federated learning framework, SSFL-DCSL, which integrates dual contrastive loss and soft labeling to address data and label scarcity for distributed clients with few labeled samples while safeguarding user privacy. It enables representation learning using unlabeled data on the client side and facilitates joint learning among clients through prototypes, thereby achieving mutual knowledge sharing and preventing local model divergence. Specifically, first, a sample weighting function based on the Laplace distribution is designed to alleviate bias caused by low confidence in pseudo labels during the semi-supervised training process. Second, a dual contrastive loss is introduced to mitigate model divergence caused by different data distributions, comprising local contrastive loss and global contrastive loss. Third, local prototypes are aggregated on the server with weighted averaging and updated with momentum to share knowledge among clients. To evaluate the proposed SSFL-DCSL framework, experiments are conducted on two publicly available datasets and a dataset collected on motors from the factory. In the most challenging task, where only 10% of the data are labeled, the proposed SSFL-DCSL can improve accuracy by 1.15% to 7.85% over state-of-the-art methods.

The lack of complete labels and data heterogeneity are obstacles to the application of artificial intelligence-based methods in industrial scenarios, such as machinery fault diagnosis. To address these challenges, this article proposes a federated learning (FL) framework for the industrial Internet of Things based on bidirectional knowledge distillation (KD) and hard sample selection. In the framework, the cloud server provides a pretrained deep learning (DL) model based on the cross-domain public dataset to facilitate the cold start in real-world applications. Then during the training process, each participating factory trains its heterogeneous local DL model according to local data volume and computing resources. Bidirectional KD with feature maps and hard sample selection is then carried out on a shared dataset between the server and factories to share knowledge efficiently. Moreover, all the DL models used in the application of the proposed framework are designed based on expertise and attention mechanism to diagnose multiple types of machinery and faults. Case studies using the vibration data collected from multiple factories show that the proposed framework improves the fault diagnosis accuracy compared to other FL methods while significantly reducing communication overhead.

In this article, we propose a method, generative image reconstruction from gradients (GIRG), for recovering training images from gradients in a federated learning (FL) setting, where privacy is preserved by sharing model weights and gradients rather than raw training data. Previous studies have shown the potential for revealing clients' private information or even pixel-level recovery of training images from shared gradients. However, existing methods are limited to low-resolution images and small batch sizes (BSs) or require prior knowledge about the client data. GIRG utilizes a conditional generative model to reconstruct training images and their corresponding labels from the shared gradients. Unlike previous generative model-based methods, GIRG does not require prior knowledge of the training data. Furthermore, GIRG optimizes the weights of the conditional generative model to generate highly accurate "dummy" images instead of optimizing the input vectors of the generative model. Comprehensive empirical results show that GIRG is able to recover high-resolution images with large BSs and can even recover images from the aggregation of gradients from multiple participants. These results reveal the vulnerability of current FL practices and call for immediate efforts to prevent inversion attacks in gradient-sharing-based collaborative training.

Gas usage estimation plays a critical role in various aspects of the power generation and delivery business, including budgeting, resource planning, and environmental preservation. Federated Learning (FL) has demonstrated its potential in enhancing the accuracy and reliability of gas usage estimation by enabling distributedly owned data to be leveraged, while ensuring privacy and confidentiality. However, to effectively motivate stakeholders to contribute their high-quality local data and computational resources for this purpose, incentive mechanism design is key. In this paper, we report our experience designing and deploying the Hierarchical FL Incentive mechanism for Gas usage estimation (HiFi-Gas) system. It is designed to cater to the unique structure of gas companies and their affiliated heating stations. HiFi-Gas provides effective incentivization in a hierarchical federated learning framework that consists of a horizontal federated learning (HFL) component for effective collaboration among gas companies and multiple vertical federated learning (VFL) components for the gas company and its affiliated heating stations. To motivate active participation and ensure fairness among gas companies and heating stations, we incorporate a multi-dimensional contribution-aware reward distribution function that considers both data quality and model contributions. Since its deployment in the ENN Group in December 2022, HiFi-Gas has successfully provided incentives for gas companies and heating stations to actively participate in FL training, resulting in more than 12% higher average gas usage estimation accuracy and substantial gas procurement cost savings. This implementation marks the first successful deployment of a hierarchical FL incentive approach in the energy industry.

Time series forecasting is a critical and challenging task in practical application. Recent advancements in pre-trained foundation models for time series forecasting have gained significant interest. However, current methods often overlook the multi-scale nature of time series, which is essential for accurate forecasting. To address this, we propose HiMTM, a hierarchical multi-scale masked time series modeling with self-distillation for long-term forecasting. HiMTM integrates four key components: (1) hierarchical multi-scale transformer (HMT) to capture temporal information at different scales; (2) decoupled encoder-decoder (DED) that directs the encoder towards feature extraction while the decoder focuses on pretext tasks; (3) hierarchical self-distillation (HSD) for multi-stage feature-level supervision signals during pre-training; and (4) cross-scale attention fine-tuning (CSA-FT) to capture dependencies between different scales for downstream tasks. These components collectively enhance multi-scale feature extraction in masked time series modeling, improving forecasting accuracy. Extensive experiments on seven mainstream datasets show that HiMTM surpasses state-of-the-art self-supervised and end-to-end learning methods by a considerable margin of 3.16-68.54%. Additionally, HiMTM outperforms the latest robust self-supervised learning method, PatchTST, in cross-domain forecasting by a significant margin of 2.3%. The effectiveness of HiMTM is further demonstrated through its application in natural gas demand forecasting.

Federated learning (FL) addresses data privacy concerns by enabling collaborative training of AI models across distributed data owners. Wide adoption of FL faces the fundamental challenges of data heterogeneity and the large scale of data owners involved. In this paper, we investigate the prospect of Foundation Model (e.g., transformers)-based FL for achieving generalization and personalization in this setting. Different from existing research efforts which mostly focus on studying Transformer-based FL on small scales, we conduct extensive comparative experiments involving FL with Transformers, ResNet, and personalized ResNet-based FL approaches under various large-scale scenarios. These experiments consider varying numbers of data owners to demonstrate Transformers’ advantages over deep neural networks in large-scale heterogeneous FL tasks. In addition, we analyze the superior performance of Transformers by comparing the Centered Kernel Alignment (CKA) representation similarity across different layers and FL models to gain insight into the reasons behind their promising capabilities.

Artificial Intelligence (AI) has been widely applied for Safety Inspection in a number of industrial domains. However, individual company usually could not provide sufficient data to support well-trained AI models. Federated Learning, as a new AI paradigm, enables a number of participants to contribute training data to co-create high-performance models without compromising data privacy. However, an effective incentive mechanism is essential to encourage participants to contribute high-quality data, and the fundamental of the incentive mechanism is to evaluate participants' contribution fairly. Shapley Value (SV) is a well-known approach to evaluate individual's marginal contribution in a coalition, but the canonical SV calculation and its available variants are very costly. In this paper, we proposed an FL framework to enable a number of natural gas companies from different cities to jointly train an object detection computer vision deep learning model for the purpose of identifying potential hazards, without sharing their confidential inspection photos directly. We improve state-of-the-art SV algorithm by proposing Weighted Truncation (WT) for unnecessary computations, and go further with Dynamic Programming (WTDP), achieving better trade-off between efficiency and accuracy. Based on our proposed WTDP-Shapley participant contribution evaluation approach, an effective end-to-end incentive mechanism is designed by leveraging knowledge of both data scientists and domain experts. According to our experiments, it could encourage participants to contribute scarce photos with potential hazards, and thus co-create a high-performance AI model to identify various hazards accurately for residential natural gas installation safety inspection.

Multi-Party Computation (MPC) provides an effective cryptographic solution for distributed computing systems so that local models with sensitive information are encrypted before sending to the centralized servers for aggregation. Though direct local knowledge leakages are eliminated in MPC-based algorithms, we observe the server can still obtain the local information indirectly in many scenarios, or even reveal the groundtruth images through methods like Deep Leakage from Gradients (DLG). To eliminate such possibilities and provide stronger protections, we propose an augmented MPC approach by encrypting local models with two rounds of decomposition before transmitting to the server. The proposed solution allows us to remove the constraint that servers must be honest in the general federated learning settings since the true global model is hidden from the servers. Specifically, the augmented MPC algorithm encodes local models into multiple secret shares in the first round, then each share is furthermore split into a public share and a private share. Consequences of such a two-round decomposition are that the augmented algorithm fully inherits the advantages of standard MPC by providing lossless encryption and decryption while simultaneously rendering the global model invisible to the central server. Both theoretical analysis and experimental verification demonstrate that such an augmented solution can provide stronger protections for the security and privacy of the training data, with minimal extra communication and computation costs incurred.

In modern industrial systems, components have complex interactions with each other, which makes it become a challenging task to identify the operational conditions of industrial systems. Considering that an industrial system, the embedded components and their interactions can be expressed as nodes and edges in a graph, respectively. Therefore, graph representation algorithms are powerful tools for fault diagnosis of industrial systems. As one of the most commonly used graph representation algorithms, graph neural networks (GNN) mainly follow the law of “learning to attend.” GNN extract training data features learn the statistical correlations between features and labels, resulting in the attended graph favoring for accessing noncausal features as a shortcut for prediction. This shortcut feature is unstable and depends on the data distribution characteristics in the training dataset, which reduces the generalization ability of the classifier. By performing the causal analysis of GNN modeling for graph representation, the results show that shortcut features act as confounding factors between causal features and predictions, causing classifiers to learn wrong correlations. Therefore, to discover patterns of causality and weaken the confounding effects of shortcut features, a causal-trivial attention graph neural network strategy is proposed. First, node and edge representations are given by estimating soft masks. Second, through disentanglement, both causal features and shortcut features are obtained from the graph. Third, the backdoor adjustment of the causal theory is parameterized to combine each causal feature with a variety of shortcut features. Finally, comparative experiments on the three-phase flow facility dataset illustrate the effectiveness of the proposed method.

Artificial intelligence (AI)-empowered industrial fault diagnostics is important in ensuring the safe operation of industrial applications. Since complex industrial systems often involve multiple industrial plants (possibly belonging to different companies or subsidiaries) with sensitive data collected and stored in a distributed manner, collaborative fault diagnostic model training often needs to leverage federated learning (FL). As the scale of the industrial fault diagnostic models are often large and communication channels in such systems are often not exclusively used for FL model training, existing deployed FL model training frameworks cannot train such models efficiently across multiple institutions. In this paper, we report our experience developing and deploying the Federated Opportunistic Block Dropout (FedOBD) approach for industrial fault diagnostic model training. By decomposing large-scale models into semantic blocks and enabling FL participants to opportunistically upload selected important blocks in a quantized manner, it significantly reduces the communication overhead while maintaining model performance. Since its deployment in ENN Group in February 2022, FedOBD has served two coal chemical plants across two cities in China to build industrial fault prediction models. It helped the company reduce the training communication overhead by over 70% compared to its previous AI Engine, while maintaining model performance at over 85% test F1 score. To our knowledge, it is the first successfully deployed dropout-based FL approach.

The vanilla federated learning (FedAvg) takes a weighted aggregation of uploaded models based on the amount of data from each participant. By addressing varying data quality and heterogeneous data distributions across participants, the existing work calculates the contribution of participants based on Shapley Value (SV) and attempt to guide model aggregation under SV. Our empirical study found that vanilla SV exhibits low sensitivity and instability issues. To address these issues, we propose a Shapley Value Weighted FL aggregation method based on Adaptive Temperature augmentation and Momentum update (SWATM). The adaptive temperature moderately augments the sensitivity of SV according to aggregation performance, while the momentum update slows down the fluctuation and improves stability. Experiments on CIFAR-10 show that, our approach can significantly improve stability compared to the state-of-the-art, and experiments on household safety inspection show that our method accelerate the convergence of the global model. Finally, we deploy SWATM for predictive maintenance of equipment with a combined F1-score improvement of 2.57%.

Federated learning can collaboratively train AI models while protecting data privacy. In practical industry environment, non-independent and identically distributed (Non-IID) characteristics of data affect the effectiveness of federated learning. Personalized federated learning can help resolve this, but it cannot adapt to unknown data. In addition, practical applications also call for trusted training environment and remain stable when there are security threats. In this article, we propose a credible federated self-learning (CFSL), based on the idea of hypernetwork supported by blockchain to achieve secured, credible, personalized federated self-learning, especially, for unknown data in Non-IID environment. Extensive experiments on three Non-IID data sets demonstrate the capabilities on adaptive resilience for security attacks and on accuracy of recognizing unknown objects, with good performance at the same time. CFSL outperforms the existing personalized federated learning methods, with an increase in average accuracy by 4.11%.

In[1], on page 1824,Fig. 3should be as follows:

Federated learning has become an emerging hot research field in industry because of its ability to perform large-scale distributed learning while preserving data privacy. However, recent studies have shown that in the actual use of federated learning, there are device heterogeneity and data not identically and independently distributed (Non-IID) characteristics between client nodes, which will affect the effect of federated learning. In this work, we propose resilient reinforcement federated learning (R$^{2}$Fed), a R$^{2}$Fed method, which applies reinforcement learning to federated learning and uses reinforcement learning for weighted fusion of client models instead of average fusion. We conduct experiments on object detection, object classification, and sentiment classification tasks in the context of Non-IID and heterogeneity, and the experimental results show that the R$^{2}$Fed method outperforms traditional federated learning, increasing the average accuracy by 4.7%. Experiments also demonstrate that R$^{2}$Fed is resilient to federation attacks.

Federated learning (FL) has become a promising machine learning (ML) paradigm for training machine learning models over distributed datasets, owing to its low communication costs and privacy preserving property. To date, the most commonly adopted model fusion mechanism in FL is average aggregation. However, it has been shown that this average aggregation mechanism performs poorly in heterogeneous systems, especially for non-independent and identically distributed (NonIID) data. In order to address this challenge, we propose a weighted FL model aggregation strategy for each client based on clustering, termed CluFL. Specifically, CluFL first measures the similarities among uploaded models from clients through their parameters using a spectral clustering algorithm. Then, CluFL assigns aggregation weights according to the similarity of the intra-cluster global model for each cluster and the average model across the clusters. Further, we derive a convergence bound on the CluFL algorithm considering a practical nonconvex setting of neural network training. This bound reveals that the proposed CluFL algorithm can achieve a convergence speed in the order of O(1/T). Extensive experiments have been conducted on both FashionMNIST and CIFAR-10 datasets and show that CluFL outperforms the state-of-the-art FL algorithms in terms of accuracy and communication efficiency.

Multi-party computation (MPC) allows distributed machine learning to be performed in a privacy-preserving manner so that end-hosts are unaware of the true models on the clients. However, the standard MPC algorithm also triggers additional communication and computation costs, due to those expensive cryptography operations and protocols. In this paper, instead of applying heavy MPC over the entire local models for secure model aggregation, we propose to encrypt critical part of model (gradients) parameters to reduce communication cost, while maintaining MPC's advantages on privacy-preserving without sacrificing accuracy of the learnt joint model. Theoretical analysis and experimental results are provided to verify that our proposed method could prevent deep leakage from gradients attacks from reconstructing original data of individual participants. Experiments using deep learning models over the MNIST and CIFAR-10 datasets empirically demonstrate that our proposed partially encrypted MPC method can reduce the communication and computation cost significantly when compared with conventional MPC, and it achieves as high accuracy as traditional distributed learning which aggregates local models using plain text.

Home appliance manufacturers strive to obtain feedback from users to improve their products and services to build a smart home system. To help manufacturers develop a smart home system, we design a federated learning (FL) system leveraging a reputation mechanism to assist home appliance manufacturers to train a machine learning model based on customers’ data. Then, manufacturers can predict customers’ requirements and consumption behaviors in the future. The working flow of the system includes two stages: in the first stage, customers train the initial model provided by the manufacturer using both the mobile phone and the mobile-edge computing (MEC) server. Customers collect data from various home appliances using phones, and then they download and train the initial model with their local data. After deriving local models, customers sign on their models and send them to the blockchain. In case customers or manufacturers are malicious, we use the blockchain to replace the centralized aggregator in the traditional FL system. Since records on the blockchain are untampered, malicious customers or manufacturers’ activities are traceable. In the second stage, manufacturers select customers or organizations as miners for calculating the averaged model using received models from customers. By the end of the crowdsourcing task, one of the miners, who is selected as the temporary leader, uploads the model to the blockchain. To protect customers’ privacy and improve the test accuracy, we enforce differential privacy (DP) on the extracted features and propose a new normalization technique. We experimentally demonstrate that our normalization technique outperforms batch normalization when features are under DP protection. In addition, to attract more customers to participate in the crowdsourcing FL task, we design an incentive mechanism to award participants.

Countries across the globe have been pushing strict regulations on the protection of personal or private data collected. The traditional centralized machine learning method, where data is collected from end-users or IoT devices, so that it can discover insights behind real-world data, may not be feasible for many data-driven industry applications in light of such regulations. A new machine learning method, coined by Google as Federated Learning (FL) enables multiple participants to train a machine learning model collectively without directly exchanging data. However, recent studies have shown that there is still a possibility to exploit the shared models to extract personal or confidential data. In this paper, we propose to adopt Multi-Party Computation (MPC) to achieve privacy-preserving model aggregation for FL. The MPC-enabled model aggregation in a peer-to-peer manner incurs high communication overhead with low scalability. To address this problem, the authors proposed to develop a two-phase mechanism by 1) electing a small committee and 2) providing MPC-enabled model aggregation service to a larger number of participants through the committee. The MPC-enabled FL framework has been integrated in an IoT platform for smart manufacturing. It enables a set of companies to train high quality models collectively by leveraging their complementary data-sets on their own premises, without compromising privacy, model accuracy vis-a`-vis traditional machine learning methods and execution efficiency in terms of communication cost and execution time.

Formal analyses of blockchain protocols have received much attention recently. Consistency results of Nakamoto's blockchain protocol are often expressed in a quantity c, which denotes the expected number of network delays before some block is mined. With μ (resp., ν) denoting the fraction of computational power controlled by benign miners (resp., the adversary), where μ+ν =1, we prove for the first time that to ensure the consistency property of Nakamoto's blockchain protocol in an asynchronous network, it suffices to have c to be just slightly greater than 2μ/(ln(μ/ν)). Such a result is both neater and stronger than existing ones. In the proof, we formulate novel Markov chains which characterize the numbers of mined blocks in different rounds.

In cryptocurrency and blockchain-based distributed ledgers, transfer of money (digital coins) can be presented as a transaction. Due to the irreversibility nature of blockchain transactions, a single fraudulent use of private key (used to sign transactions) could have significant consequences (e.g. financial loss). Key protection alone is not adequate in protecting cryptocurrencies, and threshold signature is a viable method to avoid fraudulent key usage or key theft. In this paper, we focus on the Edwards-curve digital security algorithm (EdDSA), which has been applied in several cryptocurrencies (e.g. Cardano, Zcash, and Decred) and design the first efficient two-party EdDSA signing protocol. Unlike standard secret sharing, a valid signature is generated using an interactive protocol without the original key ever being exposed. We mathematically prove the security of our proposed protocol. Findings from the performance evalation of the protocol show that it achieves good performance for curve Ed25519, with a single signing operation in the malicious setting taking approximately 3.32 ms between two devices.

In this study, we initiate a cyberinfrastructure solution by synergizing both the blockchain and Internet of Things (IoT) technologies for transportation insurance. The insurance premium related services are encapsulated in “on-chain” chaincodes to perform over the facts on vehicle's trip and driver's behavior, which are deduced through “off-chain” analytic services using the sensing data collected from vehicles' on-board sensors. A hybrid scheme coordinating both the permissioned (Hyperledger) and public (Ethereum) blockchains is proposed to exploit their respective capabilities in terms of high transaction throughput and built-in cryptocurrency. A working prototype platform is implemented with a basic premium calculation model. The prototype system is deployed across Amazon Web Services (AWSs) cloud in a real-world Internet environment. A comprehensive performance study from the aspects of throughput, latency, and resource usage under different configurations is presented to show the solution's feasibility. The design practice and research findings are concluded in consort with the experience gained for further enhancing the proposed solution and extending the functional features such as a more realistic insurance policy to be applied in generic vehicle insurance applications.

Estimation of path travel time provides great value to applications like bus line designs and route plannings. Existing approaches are mainly based on single-source trajectory datasets that are usually large in size to ensure a satisfactory performance. This leads to two limitations: 1) Large-scale data may not always be attainable, e.g. city-scale public bus data is usually small compared to taxi data due to relative fewer bus trips in a day. 2) Considering only single-source trajectory data neglects the potential estimation-improving insights of external data, e.g. trajectory dataset of other vehicle sources obtained from the same geographical region. A challenge is how to effectively utilize such other trajectory sources. Moreover, existing work does not attend the important attributes of a trajectory including vehicle ID, day of week, rainfall level etc., which are important for estimating the path travel time. Motivated by these and the recent successes of neural network models, we propose Attribute-related Hybrid Trajectories Network~(AtHy-TNet), a neural model that effectively utilizes the attribute correlations, as well as the spatial and temporal relationships across hybrid trajectory data. We apply this to a novel problem of estimating path travel time of a type of vehicles using a hybrid trajectory dataset that includes trajectories from other vehicle types. We demonstrate in our experiments the benefits of considering hybrid data for travel time estimation, and show that AtHy-TNet significantly outperforms state-of-the-art methods on real-world trajectory datasets.

Secure Multi-Party Computation (MPC) is a promising privacy-preserving technology to enable multiple trustless parties to compute a function jointly without revealing private inputs to each other. With the fast development of MPC protocols, software implementation, and underlying computation infrastructure, MPC has developed from purely theoretical interest to tangible platform implementations for real-world applications. In this paper, we investigate multiple mechanisms to design efficient MPC algorithms by avoiding costly MPC operations and leveraging parallel operations. In order to speed up database table searching, a machine learning-based approach is proposed to completely avoid equality-check operations, playing the trade-off between efficiency and accuracy. According to our experimental results, a well-designed MPC algorithm could improve performance and scalability significantly, and thus make MPC technology practicable.

Large-scale graph-structured computation is becoming increasingly important for various data analytics applications. However, most distributed graph processing frameworks do not directly support efficient implementation of sophisticated algorithms requiring massive graph traversals. In this paper, we propose concurrent hybrid breadth-first-search (BFS) algorithm on a popular distributed graph processing framework (Power-Graph and its optimized version PowerLyra). It leverages the small-world property of skewed graphs which apply to most realworld data sets. Hybrid BFS algorithm changes graph traversal direction to save computational workload and message transmissions. Concurrent BFS algorithm enables running multiple BFS simultaneously while sharing vertex explorations efficiently with bit operations. Extensive experiments are conducted to evaluate performance and scalability on a multi-core computer and a cluster of AWS instances. Concurrent hybrid BFS algorithm dramatically increases graph traversal efficiency with respect to the increasing number of concurrent BFS traversals. Compared with PowerGraph, PowerLyra uses fewer resources, while achieving significantly higher performance and better scalability.

Innovations such as the Cloud, Internet of Things (IoT) and data analytics have already dramatically altered the customer experience in many, if not all, industries. Blockchain, as another emerging technology, is expected to be the next generation infrastructure to established trusted multiparty collaborations. In this paper, we investigated the convergence of aforementioned technologies, by presenting a prototype of fine-grained transportation insurance. Insurance premium were assessed based on vehicles usage and driver's behavior, which were deduced from streaming IoT data collected from mobile sensors. This incentive mechanism promotes fairness among drivers and encourages safer driving style. The prototype takes advantage of both private blockchain (e.g., high transaction rate in Hyperledger) and public blockchain (e.g., inbuilt cryptocurrency). Besides system architecture and implementation details, preliminary performance evaluations are presented and discussed.

With the development of information and storage technologies, electronic document recording has become an unalterable trend, which transforms the way that people store, access and operate the data generated in various applications. Healthcare is the leader application domain that pioneers in usage of electronic medical records (EMRs). Cross-organizational EMRs' sharing has many constructive effects in motivating the domain innovation, introducing better domain understanding and overall the domain intelligence. However, the privacy concern, trust issue as well as the sophisticated legal regulation of the sensitive EMRs' use leads to inefficiency in the data sharing process. In this paper, we propose a cross-organizational medical data sharing framework based on permissioned blockchain technology, named “EMRShare“, to resolve the trust concern existing in EMRs sharing practice among different participants like patients, clinicians and researchers, and other relevant parties such as the insurance agent and government, to make medical data sharing and access secure, efficient, transparent, immutable, traceable and auditable. A working prototype system is implemented to demonstrate the key features for the cross-organizational medical data sharing and access management. The objective of this work targets at explaining the essential design considerations along with the working principle and operation logics using the blockchain technology to facilitate the medical data sharing in a highly-cooperative healthcare ecosystem.

Blockchain is first introduced by Bitcoin in 2009 and developers all around the world have been trying to apply blockchain in different areas, like finance services, credit and ownership management, resource sharing, investment management, Internet of Things (IoT) etc. Ethereum is a Blockchain 2.0 platform that allows developers to build a Decentralized Application (DApp) without building a new blockchain from the scratch. IoT is the technology to embed all the physical devices with sensors and chips to provide automation process via machine-to-machine communication. Blynk is a platform that provides iOS and Android application for the users and developers to collect data from control microcontroller. This paper is aiming to build a system with Ethereum private Blockchain, Raspberry Pi (RPi), Blynk platform, DHT11 temperature and humidity sensors. The system is a proof-of-concept prototype to simulate smart home applications. It collects the real-time room temperature and humidity by DHT11 via Raspberry Pi. The sensor data will be updated to the Blynk App and stored on the smart contract deployed on the Ethereum private Blockchain. If the real-time temperature or humidity value exceeds the threshold value set by the users, red or green LEDs will be turned on as warnings. This system can be improved by some possible future work.

Container technology is being adopted to simplify workflow execution. In this paper, we investigate a workflow scheduling policy for container-based systems. A workflow, representing an application, consists of a set of tasks. Each task can be executed in a container within a virtual machine (VM), where the container packaging the function for the task should be loaded into the VM before task execution. To reduce the workflow execution time and the network bandwidth consumption, we propose a cost-efficient and latency-aware workflow scheduling algorithm that strategically loads the containers into VMs and executes the tasks on the VMs. The algorithm is based on “Stretch Out and Compact”, which can stretch out the tasks along the resources by critical path analysis and then find the inefficient slots within the computing resources and eventually compact the tasks into those slots. We introduce a concept of “virtual task” into the algorithm, where container loading is regarded as a virtual task that should be executed before the real task. The introduction of the virtual task can be more effective in finding the inefficient slots for the compaction, thus resulting in a more efficient workflow scheduling policy. Simulation results show that compared to the algorithms that fully or selectively load the dockers, the proposed algorithm can achieve less execution time while saving network bandwidth consumption for loading dockers.

Graph Processing has been widely used to capture complex data dependency and uncover relationship insights. Due to the ever-growing graph scale and algorithm complexity, distributed graph processing has become more and more popular. In this paper, we investigate how to balance performance and cost for large scale graph processing on configurable virtual machines (VMs). We analyze the system architecture and implementation details of a Pregel-like distributed graph processing framework and develop a system-aware model to predict the execution time. Consequently, cost effective execution scenarios are recommended by selecting a certain number of VMs with specified capability subject to the predefined resource price and user preference. Experiments using synthetic and real world graphs have verified that system-aware model can achieve much higher prediction accuracy than popular machine-learning models which treat graph processing framework as a black box. As a result, the recommended execution scenarios have comparable cost efficiency to the optimal scenarios.

The FP-tree based edge weight computation (EWC for short) with MapReduce has demonstrated its remarkable performance for extracting weighted graphs from big data for data analysis. However, our investigation finds that existing algorithm includes unnecessary scan on the datasets as well as unnecessary information for the FP-tree construction, which prolong the runtime execution. In addition, applying inappropriate Reducers-to-cores mapping strategy may make it exhaust the resources and fail to complete the job execution. This paper designs, implements and evaluates an optimized FP-tree based graph EWC algorithm with MapReduce on Multi-core Clusters. First, we design a more compact FP-tree based EWC with 2-phase MapReduce, reducing one phase scan of the dataset. Second, we propose a reduced FP-tree data structure to reduce the FP-tree construction cost. Third, we examine two strategies for mapping Reducers to cores for EWC on each multi-core computer: {\em one-Reducer-one-core} and {\em one-Reducer-multiple-cores}. Finally, an empirical comparison performance study has been carried out on the optimized EWC algorithm against the existing one over a massive application dataset generated by a real social network. The results demonstrate that the optimized FP-tree based EWC algorithm obtains about 39\% to 55\% percentage improvement in execution time, and in the meantime achieves better scale-out and scale-up speedup. This paper's findings can also be applied to improve the scalability and efficiency of the parallel and distributed execution of applications involving large scale all-pairs set intersection computation over multi-core MapReduce clusters.

Social contact network (SCN) models the contacts between people by their daily activities. It can be formalized by an agent-to-location bipartite graph. The simulations over SCN are employed to study the complex social dynamics such as information propagation and disease spread among large-scale population. A challenge to the simulation is the skewed degree distribution of SCN, which contains a few hub locations with large numbers of visitors. The skewed degree distribution can cause load imbalance for parallel simulation and greatly degrade the execution performance. This paper proposes an approach which decomposes hub locations into small splits. Thus, SCN can be partitioned with better balanced workloads and multiple splits are able to run in parallel. Based on the pattern of information transmission between agents, we duplicate necessary data among splits to ensure the correctness of simulation. Furthermore, we enhance the parallel algorithm of SCN simulation to reduce the additional overhead from communication between splits. Finally, we build an experiment with epidemic simulation on an open dataset. The experimental results demonstrate that our approach achieves 14~35% performance improvement compared with the partitioning method without decomposition of hub locations.

Design space exploration refers to the evaluation of implementation alternatives for many engineering and design problems. A popular exploration approach is to run a large number of simulations of the actual system with varying sets of configuration parameters to search for the optimal ones. Due to the potentially huge resource requirements, cloud-based simulation execution strategies should be considered in many cases. In this paper, we look at the issue of running large-scale simulation-based design space exploration problems on commercial Infrastructure-as-a-Service clouds, namely Amazon EC2, Microsoft Azure and Google Compute Engine. To efficiently manage cloud resources used for execution, the key problem would be to accurately predict the running time for each simulation instance in advance. This is not trivial due to the currently wide range of cloud resource types which offer varying levels of performance. In addition, the widespread use of virtualization techniques in most cloud providers often introduces unpredictable performance interference. In this paper, we propose a resource and application-aware (RA2) prediction approach to combat performance variability on clouds. In particular, we employ neural network based techniques coupled with non-intrusive monitoring of resource availability to obtain more accurate predictions. We conducted extensive experiments on commercial cloud platforms using an evacuation planning design problem over a month-long period. The results demonstrate that it is possible to predict simulation execution times in most cases with high accuracy. The experiments also provide some interesting insights on how we should run similar simulation problems on various commercially available clouds.

Personalized PageRank (PPR) computation is a fundamental operation in web search, social networks, and graph analysis. Given a graph G , a source s, and a target t, the PPR query Π( s, t ) returns the probability that a random walk on G starting from s terminates at t. Unlike global PageRank which can be effectively pre-computed and materialized, the PPR result depends on both the source and the target, rendering results materialization infeasible for large graphs. Existing indexing techniques have rather limited effectiveness; in fact, the current state-of-the-art solution, BiPPR, answers individual PPR queries without pre-computation or indexing, and yet it outperforms all previous index-based solutions. Motivated by this, we propose HubPPR, an effective indexing scheme for PPR computation with controllable tradeoffs for accuracy, query time, and memory consumption. The main idea is to pre-compute and index auxiliary information for selected hub nodes that are often involved in PPR processing. Going one step further, we extend HubPPR to answer top- k PPR queries, which returns the k nodes with the highest PPR values with respect to a source s , among a given set T of target nodes. Extensive experiments demonstrate that compared to the current best solution BiPPR, HubPPR achieves up to 10x and 220x speedup for PPR and top- k PPR processing, respectively, with moderate memory consumption. Notably, with a single commodity server, HubPPR answers a top- k PPR query in seconds on graphs with billions of edges, with high accuracy and strong result quality guarantees.

With more parallel and distributed applications moving to Cloud and data centers, it is challenging to provide predictable and controllable resources to multiple tenants, and thus guarantee application performance. In this paper, we propose an integrated QoS-aware resource provisioning platform based on virtualization technology for computing, storage and network resources. Coarse-grained CPU mapping and fine-grained CPU scheduling mechanisms are proposed to enable adjustable computing power. A hierarchical distributed scheduling mechanism is implemented on a scalable storage system to guarantee I/O throughput for individual tenants and applications. A network manager has also been developed to guarantee the data transmission rate. Web-based interface enables users to monitor real time resource utilization and to adjust resource QoS levels on the fly. According to our experimental results, the resource cost can be saved up to 45% without degrading the performance of a distributed data processing benchmark, and the performance of a parallel agent-based simulation can be improved by 91% using the same amount of resources.

Although cloud computing is a thriving technology trend in industry and academy, the resource renting cost is still the main obstacle for users to switch to cloud. The existing pricing models are not flexible enough for users. On-demand pricing model does not guarantee resource availability, while reserved pricing model may result in high risk of resource wasting. In this paper, we propose OMTiR: An Open Market for Trading Idle Cloud Resources, enabling users to sell their unused or underutilized resources on negotiable prices. Consequently, users, either as a resource seller or buyer, can reduce the resource renting cost. In addition, the cloud provider can increase revenue by taking arbitrage profit in the market and serving more users using the same amount of resource. A comparative study is conducted using a real world workload trace to show the advantages of the open market model over the existing price models in terms of resource utilization rate and task waiting time.

Graph processing has become popular for various big data analytic applications. Google's Pregel framework enables vertex-centric graph processing in distributed environment based on Bulk Synchronous Parallel (BSP) model. However, the BSP model is inefficient for many complex graph algorithms requiring graph traversals, as only a small number of vertices really update states in each super step. In this paper, we propose an hierarchical parallelization mechanism, taking the advantages of both synchronous (warp-level) and asynchronous (task-level) parallelization approaches. In addition, a runtime task scheduling mechanism is proposed, relying on real-time monitoring or prediction of resource utilization. Experiments have verified that the hierarchical parallelization mechanism can expose greater parallelism, and thus, increase resource utilization significantly. Moreover, the runtime scheduling mechanism can avoid aggressive resource competition, and thus, further enhance the performance of the parallelized graph processing.

Agent-based modeling is one of the promising modeling tools that can be used in the study of population dynamics. Two of the main obstacles hindering the use of agent-based simulation in practice are its scalability when the analysis requires large-scale models as in policy research, and its ease-of-use especially for users with no programming experience. While there has been a significant work on the scalability issue, ease-of-use aspect has not been addressed in the same intensity. This paper presents a graphical user interface designed for a simulation tool which allows modelers with no programming background to specify agent-based demographic models and run them on parallel environments. The interface eases the definition of models to describe individual and group dynamics processes with both qualitative and quantitative data. The main advantage is to allow users to transparently run the models on high performance computing infrastructures.

With more applications moving to cloud, scalable storage systems, composed of a cluster of storage servers and gateways, are deployed as the back-end infrastructure to accommodate high-volume data. In such an environment, it is a challenge to provide predictable and controllable storage performance for multitenanted users with multiple applications, due to performance violation from misbehaving applications. In this paper, we propose a two-level QoS controller over scalable storage system. On the higher level, I/O throughput rented by each tenant is guaranteed and strictly limited by a CAP value. On the lower level, this rented service can be on-demand served among multiple applications under the same tenant. Thus our distributed controller not only shields performance violation from "noisy" tenants but also allows tenants to fully utilizing the rented I/O throughput. Furthermore, the QoS controller is implemented in an efficient manner, by reusing the communication channels among gateways and storage servers and piggybacking control signals on data communications. The experimental results have shown that the two-level QoS controller can guarantee I/O throughput at tenant level by controlling the CAP value while accelerating applications by on-demand serving at a very little computation cost.

More and more interests have been shown to move large-scale simulations on modern data centers composed of a large number of virtualized multi-core computers. However, the simulation components (Federates) consolidated in the same computer may have imbalanced simulation workloads. Similarly, the computers involved in the same simulation execution (Federation) may also have imbalanced simulation workloads. Hence, federates may waste a lot of computer resources on time synchronization with each other. In this paper, a hierarchical resource management system is proposed to enhance simulation execution performance. Federates in the federation are enraptured in their individual Virtual Machines (VMs), which are consolidated on a group of virtualized multi-core computers. On the computer level, multiple VMs share the resource of the computer according to the simulation workloads of their corresponding federates. On the federation level, some VMs are migrated for workload balance purpose. Therefore, computer resources are fully utilized to conduct useful simulation workloads, avoiding the synchronization overheads. Experiments using synthetic and real simulation workloads have verified that the hierarchical resource management system enhances simulation performance significantly.

High Level Architecture (HLA)-based simulations employing optimistic synchronization allows federates to process event and to advance simulation time freely at the risk of over-optimistic execution and execution rollbacks. In this paper, an adaptive resource provisioning system is proposed to accelerate optimistic HLA-based simulations in Virtual Execution Environment (VEE). A performance monitor is introduced using a middleware approach to measure the performance of individual federates transparently to the simulation application. Based on the performance measurements, a resource manager distributes the available computational resources to the federates, making them advance simulation time with comparable speeds. Our proposed approach is evaluated using a real-world simulation model with various workload inputs and different parameter settings. The experimental results show that, compared with distributing resources evenly among federates, our proposed approach can accelerate the simulation execution significantly using the same amount of computational resources.

A large scale HLA-based simulation (federation) is composed of a large number of simulation components (federates), which may be developed by different participants and executed at different locations. Byzantine failures, caused by malicious attacks and software/hardware bugs, might happen to federates and propagate in the federation execution. In this paper, a three-phases (i.e., failure detection, failure location, and failure recovery) Byzantine Fault Tolerance (BFT) mechanism is proposed based on the decoupled federate architecture. By combining the replication, check pointing and message logging techniques, some redundant executions of federate replicas are avoided. The BFT mechanism is implemented using both Barrier and No-Barrier federate replication structures. Protocols are also developed to remove the epidemic effect caused by Byzantine failures. As the experiment results show, the BFT mechanism using No-Barrier replication outperforms that using Barrier replication significantly in the case that federate replicas have different runtime performance.

Interactive multi-user Internet games require frequent state updates between players to accommodate the great demand for reality and interactivity. The large latency and limited bandwidth on the Internet greatly affects the game's scalability. The High Level Architecture (HLA) is the IEEE standard for distributed simulation with its Data Distribution Management (DDM) service group assuming the functionalities of interest management. With its support for reuse and interoperability and its DDM support for communication optimization, the HLA is promising at supporting multi-user gaming on the Internet. However, this usually requires particular prior security setup across administrative domains according to the specific Run Time Infrastructure (RTI) used. We have previously developed a Service Oriented HLA RTI (SOHR) which enables distributed simulations to be conducted across administrated domains on the Grid. This paper discusses multi-user gaming on the Grid using SOHR. Specifically, a maze game is used to illustrate how SOHR enables users to join a game conveniently. Experiments have been carried out to show how DDM can improve the communication efficiency.

The High Level Architecture provides a general framework for distributed simulation, promoting reusability and interoperability of simulation components (federates). Large scale distributed simulation, in which federates run on many heterogenous computing machines may benefit from migrating federates among these machines for load- balancing and fault-tolerance. However, the HLA framework does not provide formal support for federate migration currently. We have previously developed a Service Oriented HLA RTL (SOHR) framework, which provides HLA RTL functionalities via the cooperation of a set of Grid services. SOHR is developed with migration support features by using a decoupled federate design. In this paper, a basic federate migration protocol is first proposed to illustrate the process of federate migration in SOHR. Then two optimized protocols are further developed to overlap federate migration with federate execution for the purpose of reducing migration overhead. Experiments show that the migration overhead is reduced considerably in the optimized protocols.

Modeling and simulation permeate all areas of business, science and engineering. To promote the interoperability and reusability of simulation applications and link geographically dispersed simulation components, distributed simulation was introduced. While the high level architecture (HLA) is the IEEE standard for distributed simulation, a run time infrastructure (RTI) provides the actual implementation of the HLA. With increased size and complexity of simulation applications, large amounts of distributed computational and data resources are required. The Grid provides a flexible, secure and coordinated resource sharing environment which can facilitate distributed simulation execution. In this paper, we propose a service oriented HLA RTI (SOHR) framework which provides the functionalities of an RTI as Grid services and enables large scale distributed simulations to be conducted on a heterogeneous Grid environment. The various services in SOHR can be dynamically deployed, discovered and undeployed, leading to a scalable distributed simulation environment. While the communications between simulators are through Grid service invocations, the standard HLA interface is provided as a library to increase simulator reusability and interoperability. A subset of HLA specifications was implemented in a SOHR prototype based on GT4 and the experimental results have verified the feasibility of SOHR.