The rapid development of the Internet of Things (IoT) has propelled mobile edge computing (MEC) into the forefront of both academia and industry. Nevertheless, the surge in urban activities driven by economic development is putting a strain on infrastructure like transportation and utilities. Increased demand for computing tasks and server failures from natural disasters can severely strain MEC in a specific region due to its reliance on static edge servers. To address these challenges, we introduce an MEC framework called onboard edge computing (OBEC), which explores onboard servers to provide computational offloading services in mobile scenarios. To determine the end devices that each onboard server will serve, we propose the concept of “hunger value” to accurately measure the resource idleness of an onboard server. We also implement a Genetic Optimization-based Hunting-Predation Algorithm, an onboard server as a predator and a service end device as a prey, to minimize the overall hunger value of the whole system. Taking into account the power consumption of onboard servers and the satisfaction of end devices, we introduce a Stackelberg game to allow each onboard server to select the optimal serviced end devices and efficiently provide the required resources. Since this Stackelberg game lacks an analytical solution, we employ gradient descent to calculate the optimal offloading and resource allocation strategy. Finally, we conduct simulation experiments to demonstrate the superiority of the proposed OBEC framework over other state-of-art methods across various scenarios, underscoring its potential to foster synergistic interactions between servers and end devices.

As the proliferation of Internet of Things (IoT) devices continues, vast amounts of data are being collected on various end devices. However, uploading these data to the cloud for centralized processing poses significant privacy risks. Federated learning (FL) addresses this issue by sharing model updates instead of raw data, which helps mitigate privacy concerns. Nonetheless, model updates transmitted in plaintext remain vulnerable to inference and reconstruction attacks. While homomorphic encryption (HE) can enhance security, traditional single-key schemes struggle to defend against collusion. Multikey HE schemes introduce substantial computational and communication overhead, making them impractical for resource-constrained IoT environments. In this article, we propose a novel FL framework that integrates single-key HE, elliptic curve cryptography (ECC), and trusted execution environments (TEE) to achieve robust protection against collusion attacks. Specifically, we utilize ECC-based public key encryption to secure HE private key and employ secret sharing to split the ECC private key into multiple subsecrets, which are distributed to edge nodes, ensuring no single party can independently decrypt model updates. Additionally, we delegate the HE private key reconstruction and global model decryption processes to the TEE, and introduce a hash verification mechanism to ensure that only aggregated global model updates can be decrypted. Finally, we provide comprehensive security proofs and extensive experimental results, demonstrating the effectiveness and superiority of the proposed framework.

The rapid evolution of the Internet of Things (IoT) and vehicular ad-hoc networks (VANETs) has underscored the urgent need for robust data access control mechanisms. ciphertext-policy attribute-based encryption (CP-ABE) emerges as a promising solution by leveraging users’ inherent attributes to define access privileges, thus providing a secure and flexible approach to access control. Despite the flexibility of ciphertext-policy ABE (CP-ABE), its deployment in VANETs faces several challenges, including limited edge resources, heavy reliance on a centralized trusted authority that may hinder scalability and introduce single points of failure, and inefficiency in addressing key misuse and identity forgery, all of which significantly impact system performance and scalability. In response to these challenges, we propose an efficient decentralized cloud-based CP-ABE system that integrates active and robust black-box traitor tracing, wherein each user is bound to unique and implicit logical location. These bindings enable precise tracing and accountability, which allows the scheme to efficiently identify the traitor through multiple interactions and prevent collusion between entities. The distributed scheme offers strong traitor tracing capabilities, ensures IND-chosen-plaintext attack (CPA) security, and maintains low-computational overhead, thereby making it well-suited for resource-constrained edge devices in VANETs. Finally, extensive deployment experiments on edge devices and formal security proofs are provided to demonstrate the scheme’s performance.

With the rapid evolution of the Internet of Things (IoT), there is a noticeable surge in both the proliferation of edge devices and the voluminous data they generate. These edge devices are progressively furnished with AI processors, harnessing the power of deep learning to augment their data processing capabilities. However, in edge environments, traditional federated learning methods typically send multiple models to a central server for aggregation, which gives rise to several tough challenges such as low data transmission efficiency, privacy concerns, and the threat of model poisoning attacks. In this paper, we introduce a distributed machine learning framework with an innovative collaborative voting mechanism to integrate the results of adaptive pruned models on various end devices for edge computing. The main goals of this framework are to mitigate the risk of data privacy and strengthen the system’s resilience against model poisoning attacks. Additionally, an adaptive model pruning mechanism is implemented to tailor diverse models according to the limited computational resources available on end devices for enhancing training efficiency. Experiments reveal that our framework can effectively mitigate the impact of poisoning attacks, but also provide superior efficiency and accuracy for edge computing compared with other prevalent federated learning methods.

Unsupervised attribute graph representation learning allows for embedding node information into compact vectors without relying on any labels, which greatly facilitates downstream tasks. Graph contrastive learning, founded on the principle of maximizing mutual information, has emerged as a pivotal technique in unsupervised graph representation learning. It achieves node discriminative representations by bringing positive samples closer together and pushing negative samples further apart. However, most existing graph contrastive learning methods primarily concentrate on node-level comparisons, capturing highly abstract node differences to discriminate them, while overlooking the wealth of information present in community substructures within a graph. Additionally, nodes within the same community in a graph often exhibit similar semantics, and considering all other nodes as negative samples unavoidably leads to sampling bias issues. In this work, we propose Community-aware unsupervised graph debiased contrastive representation learning (CAGDCL). Specifically, CAGDCL employs a novel edge-density driven contrastive objective on the augmented graph for community detection to generate robust community prototypes. We introduce the node-prototype contrastive objective based on the node-level contrastive objective. The primary purpose is to encourage the encoder to capture more community-related semantics, enabling the obtained embeddings to maintain intra-community alignment and inter-community uniformity in the embedding space. To mitigate the issue of sampling bias, we propose a weighting scheme of negative samples based on community assignment result and community prototype similarity. Through extensive experiments on several real-world datasets, we demonstrate the effectiveness of CAGDCL.

Message Passing Interface (MPI) is a standard communication protocol utilized in parallel computing. In MPI programs, communication deadlock is one of the most serious problems. To detect deadlocks, existing methods usually traverse all possible execution paths. However, with the increase of wildcard receives, these methods face the problem of path explosion, resulting in low detection efficiency. To address the issue, we propose a deadlock detection approach with path compression and focus matching. In the approach, consecutive identical send operations within a process in an MPI program are combined to a new compressed operation, which reduces the number of communication operations to be analyzed. Then, for each receive operation, a match-set containing all possible compressed send operations from which the receive operation can receive a message is formed based on the Lazy Lamport Clocks Protocol. Finally, a focus matching algorithm based on the match-sets is applied for deadlock detection. We have implemented our approach in a tool called PCMPI and evaluated PCMPI by experimenting on 16 test programs from the Umpire test suite and open-source projects in the real-world. The experimental results demonstrate that PCMPI exhibits higher efficiency in deadlock detection than the two most related tools SAMPI and PDMPI.

With the exponential growth of data at edge environments, federated learning has emerged. Nevertheless, it encounters persistent challenges in defending against malicious attacks, such as inference attacks and reconstruction attacks. In this paper, we enhance the security of federated learning by integrating cryptographic primitives, including Elliptic Curve Cryptography (ECC) and homomorphic encryption, with cloud federation to protect private user data from being inferred by adversaries using localized models. Specifically, we employ two cloud service providers (CSPs) to manage encrypted model aggregation and global model decryption services separately, which ensures robust privacy protection and provides a strong solution for safeguarding sensitive information. ECC is utilized to encrypt the private key used in homomorphic encryption. Additionally, we partition the ECC private key into multiple sub-secrets and distribute them among the clients to defend against collusion attacks during the decryption process of the global model. Furthermore, we harness the gradient offset between local and global models to design an optimized objective function tailored for clients handling non-Independent and Identically Distributed (non-IID) and unbalanced distributed datasets. Our experimental results show that our method improves accuracy by 1.5% over the baseline, while also ensuring robust privacy protection.

With the rapid development of the Internet of Things (IoT), data generated by IoT devices are also increasing exponentially. The edge computing has alleviated the problems of limited network and transmission delay when processing tasks of IoT devices in traditional cloud computing. And with the popularity of deep-learning, more and more terminal devices are embedded with artificial intelligence (AI) processors for higher processing capability at the edge. However, the problems of deep-learning task offloading in a heterogeneous edge computing environment have not been fully investigated. In this article, a multimodel edge computing offloading framework is proposed, using NVIDIA Jetson edge devices (Jetson TX2, Jetson Xavier NX, and Jetson Nano) and GeForce RTX GPU servers (RTX3080 and RTX2080) to simulate the edge computing environment, and make binary computational offloading decisions for face detection tasks. We also introduce a Bayesian optimization algorithm, namely, modified tree-structured Parzen estimator (MTPE), to reduce the total cost of edge computation within a time slot including response time and energy consumption, and ensure the accuracy requirements of face detection. In addition, we employ the Lyapunov model to obtain the harvesting energy between time slots to keep the energy queue stable. Experiments reveal that MTPE algorithm can achieve the globally optimal solution in fewer iterations. The total cost of multimodel edge computing framework is reduced by an average of 17.94% compared to a single-model framework. In contrast to the double deep Q-network (DDQN), our proposed algorithm can decrease the computational consumption by 23.01% for obtaining the offloading decision.

Deadlock is one of the critical problems in the message passing interface. At present, most techniques for detecting the MPI deadlock issue rely on exhausting all execution paths of a program, which is extremely inefficient. In addition, with the increasing number of wildcards that receive events and processes, the number of execution paths raises exponentially, further worsening the situation. To alleviate the problem, we propose a deadlock detection approach called SAMPI based on match-sets to avoid exploring execution paths. In this approach, a match detection rule is employed to form the rough match-sets based on Lazy Lamport Clocks Protocol. Then we design three refining algorithms based on the non-overtaking rule and MPI communication mechanism to refine the match-sets. Finally, deadlocks are detected by analyzing the refined match-sets. We performed the experimental evaluation on 15 various programs, and the experimental results show that SAMPI is really efficient in detecting deadlocks in MPI programs, especially in handling programs with many interleavings.

In the era of data explosion, a growing number of data-intensive computing frameworks, such as Apache Hadoop and Spark, have been proposed to handle the massive volume of unstructured data in parallel. Since programming models provided by these frameworks allow users to specify complex and diversified user-defined functions (UDFs) with predefined operations, the grand challenge of tuning up entire system performance arises if programmers do not fully understand the semantics of code, data, and runtime systems. In this paper, we design a holistic semantics-aware (optimization for data-intensive applications using hybrid program analysis (SODA) to assist programmers to tune performance issues. SODA is a two-phase framework: the offline phase is a static analysis that analyzes code and performance profiling data from the online phase of prior executions to generate a parameterized and instrumented application; the online phase is a dynamic analysis that keeps track of the application's execution and collects runtime information of data and system. Extensive experimental results on four real-world Spark applications show that SODA can gain up to 60%, 10%, 8%, faster than its original implementation, with the three proposed optimization strategies, i.e., cache management, operation reordering, and element pruning, respectively.

Many data analyzing applications highly rely on timely response from execution, and are referred as time-critical data analyzing applications. Due to frequent appearing of gigantic amount of data and analytical computations, running them on large scale distributed computing environments is often advantageous. The workload of big data applications is often hybrid, i.e., contains a combination of time-critical and regular non-time-critical applications. Resource management for hybrid workloads in complex distributed computing environment is becoming more critical and needs more studies. However, it is difficult to design rule-based approaches best suited for such complex scenarios because many complicated characteristics need to be taken into account.Therefore, we present an innovative reinforcement learning (RL) based resource management approach for hybrid workloads in distributed computing environment. We utilize neural networks to capture desired resource management model, use reinforcement learning with designed value definition to gradually improve the model and use ε-greedy methodology to extend exploration along the reinforcement process. The extensive experiments show that our obtained resource management solution through reinforcement learning is able to greatly surpass the baseline rule-based models. Specifically, the model is good at reducing both the missing deadline occurrences for time-critical applications and lowering average job delay for all jobs in the hybrid workloads. Our reinforcement learning based approach has been demonstrated to be able to provide an efficient resource manager for desired scenarios.

Along with the explosive growth of data, there is a great demand to speedup the ability to process them. Although there are several platforms such as Spark that have made analysis easier to developers, the performance tuning for such platforms meanwhile becomes complex. In this paper, we propose an efficient performance optimization engine called Hedgehog to evaluate the performance based on "Law of Diminishing Marginal Utility" and give an optimal configuration setting. The initial experiments show that our optimization can gain 19.6% performance improvement compared to the naive configuration by tuning only 3 parameters.

Nowadays, with the increasing burst of newly generated data everyday, as well as the vast expanding needs for corresponding data analyses, grand challenges have been brought to big data computing platforms. Computing resources in a single cluster are often not able to fulfill the computing capability needs. The requests of distributed computing resources are dramatically arising. In addition, with increasing popularity of cloud computing platforms, many organizations with data security concerns are more favor to hybrid cloud, a multi-cluster environment composed by both public cloud and private cloud in purpose of keeping sensitive data local. All these scenarios show great necessity of migrating big data computing to multi-cluster environment. In this paper, we present a hierarchical multi-cluster big data computing framework built upon Apache Spark. Our framework supports combination of heterogeneous Spark computing clusters. With an integrated controller within the framework, it also facilitates ability for submitting, monitoring, executing of Spark workflow. Our experimental results show that the proposed framework not only enables possibility of distributing Spark workflow throughout multiple clusters, but also provides significant performance improvement compared to single cluster environment by optimizing utilization of multi-cluster computing resources.

Data have been generated and collected at an accelerating pace. Hadoop has made analyzing large scale data much simpler to developers/analysts using commodity hardware. Interestingly, it has been shown that most Hadoop jobs have small input size and do not run for long time. For example, higher level query languages, such as Hive and Pig, would handle a complex query by breaking it into smaller adhoc ones. Although Hadoop is designed for handling complex queries with large data sets, we found that it is highly inefficient to operate at small scale data, despite a new Uber mode was introduced specifically to handle jobs with small input size. In this paper, we propose an optimized Hadoop extension called MRapid, which significantly speeds up the execution of short jobs. It is completely backward compatible to Hadoop, and imposes negligible overhead. Our experiments on Microsoft Azure public cloud show that MRapid can improve performance by up to 88% compared to the original Hadoop.

Recent research has demonstrated that social media could provide valuable spatio-temporal data about users activities. However, information extraction and computation from big amount of data pose various challenges. To effectively process massive datasets, several platforms have been developed. Our previous study [20] explored Hadoop-based cloud computing for processing big amount of social media data [9] to study geographic distributions of social media users. In this paper, we investigate an emerging system named Spark and present a timely pilot experience on geospatial big data research. In our study, Spark has been utilized to perform some classic geospatial analyses like K-Nearest Neighbors (KNN), geographic mean and median points, and the distribution of the median points. Our design is tested on an Amazon EC2 cluster. An exemplary study using 60GB, 120GB and 180GB Twitter data has demonstrated the performance achievements by migrating computing tasks from Hadoop to Spark. In our experiments, the Spark-based solution can be up to 2.3x faster than the Hadoop-based solution due to its in-memory processing and coarse-grained resource allocation strategy. In the paper, we also discuss optimization strategies on using Spark for different geospatial computing tasks.

In the field of big data research, analytics on spatio-temporal data from social media is one of the fastest growing areas and poses a major challenge on research and application. An efficient and flexible computing and storage platform is needed for users to analyze spatio-temporal patterns in huge amount of social media data. This paper introduces a scalable and distributed geographic information system, called Dart, based on Hadoop and HBase. Dart provides a hybrid table schema to store spatial data in HBase so that the Reduce process can be omitted for operations like calculating the mean center and the median center. It employs reasonable pre-splitting and hash techniques to avoid data imbalance and hot region problems. It also supports massive spatial data analysis like K-Nearest Neighbors (KNN) and Geometric Median Distribution. In our experiments, we evaluate the performance of Dart by processing 160 GB Twitter data on an Amazon EC2 cluster. The experimental results show that Dart is very scalable and efficient.

Hadoop is a popular open-source implementation of the MapReduce programming model to handle large data sets, and HDFS is one of Hadoop's most commonly used distributed file systems. Surprisingly, we found that HDFS is inefficient when handling upload of data files from client local file system, especially when the storage cluster is configured to use replicas. The root cause is HDFS's synchronous pipeline design. In this paper, we introduce an improved HDFS design called SMARTH. It utilizes asynchronous multi-pipeline data transfers instead of a single pipeline stop-and-wait mechanism. SMARTH records the actual transfer speed of data blocks and sends this information to the namenode along with periodic heartbeat messages. The namenode sorts datanodes according to their past performance and tracks this information continuously. When a client initiates an upload request, the namenode will send it a list of "high performance" datanodes that it thinks will yield the highest throughput for the client. By choosing higher performance datanodes relative to each client and by taking advantage of the multi-pipeline design, our experiments show that SMARTH significantly improves the performance of data write operations compared to HDFS. Specifically, SMARTH is able to improve the throughput of data transfer by 27-245% in a heterogeneous virtual cluster on Amazon EC2.