Machine unlearning is a newly popularized technique for removing specific training data from a trained model, enabling it to comply with data deletion requests. While it protects the rights of users requesting unlearning, it also introduces new privacy risks. Prior works have primarily focused on the privacy of data that has been unlearned, while the risks to retained data remain largely unexplored. To address this gap, we focus on the privacy risks of retained data and, for the first time, reveal the vulnerabilities introduced by machine unlearning under the dual-view setting, where an adversary can query both the original and the unlearned models. From an information-theoretic perspective, we introduce the concept of privacy knowledge gain and demonstrate that the dual-view setting allows adversaries to obtain more information than querying either model alone, thereby amplifying privacy leakage. To effectively demonstrate this threat, we propose DVIA, a Dual-View Inference Attack, which extracts membership information on retained data using black-box queries to both models. DVIA eliminates the need to train an attack model and employs a lightweight likelihood ratio inference module for efficient inference. Experiments across different datasets and model architectures validate the effectiveness of DVIA and highlight the privacy risks inherent in the dual-view setting.

Data augmentation in reinforcement learning (RL) aims to generate diverse and extensive datasets to enhance the learning process. Most existing studies on RL augmentation employ sample-based approaches that modify existing samples. However, many of these methods directly adopt augmentation strategies from other domains, which may present limitations when applied to RL. For instance, approaches derived from computer vision techniques are often not well-suited for general RL applications that do not involve visual inputs. Moreover, such sample-based methods frequently overlook the broader characteristics of the training environment, rendering the augmented data potentially less effective in complex scenarios, such as intelligent transportation systems. In addition, these methods can introduce significant risks to critical and sensitive data. For example, introducing noise to samples as a method of augmentation can precipitate adversarial attacks. Thus, a reliable and stable method of augmentation is necessary. To address these concerns, we propose a novel large language model (LLM)-based augmentation strategy in RL with machine unlearning and fine-tuning. This method utilizes LLMs for data augmentation, ensuring the reliability of the augmented data by tailoring it to specific environmental contexts. Additionally, it enhances the quality of augmentation by mitigating the impact on critical samples through the proposed novel machine unlearning method, while simultaneously fine-tuning the model to improve overall performance. Our experimental results indicate that this innovative approach significantly surpasses traditional augmentation methods in learning performance. By mitigating the impact on critical samples, our strategy not only generates more reliable augmented data but also enhances the effectiveness of RL models.

Due to the growing emphasis on privacy and data governance in machine learning, federated unlearning, an emerging concept in the domain of federated learning, stems from the growing need to address the dynamic nature of data and the evolving requirements related to privacy, compliance, and data management. However, there are some security risks during the unlearning process, including the potential for adversarial manipulation of model integrity, privacy breaches, and performance degradation in a federated learning framework. Although existing research has proposed various defenses to mitigate these risks, significant vulnerabilities remain that can be exploited to undermine the integrity and effectiveness of the unlearning process. Current attack methods are limited by their detectability during training, lack of persistence, and reliance on test-time triggers, which reduces their overall effectiveness. In this paper, we introduce camouflaged poisoning attacks, a novel attack paradigm relevant to federated unlearning. In this approach, some adversary clients initially infuse a small number of meticulously designed points into the dataset, ensuring that the model's predictions are barely influenced. The adversary then makes a request to the exclusion of some of these malicious clients. At this juncture, the attack is activated, leading to a detrimental impact on the model's predictions. The outcomes reveal a substantial potential for these strategies to compromise the effectiveness of models in unlearning scenarios. The essence of this attack involves the creation of deceptive clients that conceal the influence of a contaminated dataset during the federated unlearning process.

The deployment of large language models (LLMs) like ChatGPT and Gemini has shown their powerful natural language generation capabilities. However, these models can inadvertently learn and retain sensitive information and harmful content during training, raising significant ethical and legal concerns. To address these issues, machine unlearning has been introduced as a potential solution. While existing unlearning methods take into account the specific characteristics of LLMs, they often suffer from high computational demands, limited applicability, or the risk of catastrophic forgetting. To address these limitations, we propose a lightweight behavioral unlearning framework based on Retrieval-Augmented Generation (RAG) technology. By modifying the external knowledge base of RAG, we simulate the effects of forgetting without directly interacting with the unlearned LLM. We approach the construction of unlearned knowledge as a constrained optimization problem, deriving two key components that underpin the effectiveness of RAG-based unlearning. This RAG-based approach is particularly effective for closed-source LLMs, where existing unlearning methods often fail. We evaluate our framework through extensive experiments on both open-source and closed-source models, including ChatGPT, Gemini, Llama-2-7b-chat, and PaLM 2. The results demonstrate that our approach meets five key unlearning criteria: effectiveness, universality, harmlessness, simplicity, and robustness. Meanwhile, this approach can extend to multimodal large language models and LLM-based agents.

The development of Large Language Models (LLMs) faces a significant challenge: the exhaustion of publicly available fresh data. This is because training an LLM requires a large demand for new data. Federated learning emerges as a promising solution, enabling collaborative model to contribute their private data to LLM global model. However, integrating federated learning with LLMs introduces new challenges, including the lack of transparency and the need for effective unlearning mechanisms. Transparency is essential to ensuring trust and fairness among participants, while accountability is crucial for deterring malicious behaviour and enabling corrective actions when necessary. To address these challenges, we propose a novel blockchain-based federated learning framework for LLMs that enhances transparency, accountability, and unlearning capabilities. Our framework leverages blockchain technology to create a tamper- proof record of each model's contributions and introduces an innovative unlearning function that seamlessly integrates with the federated learning mechanism. We investigate the impact of Low-Rank Adaptation (LoRA) hyperparameters on unlearning performance and integrate Hyperledger Fabric to ensure the security, transparency, and verifiability of the unlearning process. Through comprehensive experiments and analysis, we showcase the effectiveness of our proposed framework in achieving highly effective unlearning in LLMs trained using federated learning. Our findings highlight the feasibility of integrating blockchain technology into federated learning frameworks for LLM.

With growing attention to data privacy in federated learning, federated unlearning has become an important solution to meet increasing demands for privacy compliance. However, unlearning may bring in new security concerns, such as dangers of adversarial manipulation, where the adversary may launch malicious updates or inputs to hurt the model performance or prediction, privacy-attacks, as the sensitive data can be possibly deduced from the process of unlearning, and performance degradation, because the unlearning process may break the consistency or performance of the model. In this paper, to address such issues and acquire a good and adaptive unlearning policy without causing much negative effect to the federated system, we present a reinforcement learning based method to facilitate the data unlearning method in federated learning. Our approach iteratively disposes of clients through partial unlearning, complete unlearning, or no unlearning using a DQN combined with clients’ properties like contribution, privacy cost, and computational overhead. We show that by utilizing the reinforcement learning technique, the performance decay can be defended effectively, and adversarial behaviors are indeed a common concern for the federated unlearning scenario. Our analysis can inform the development of federated unlearning frameworks that defend against performance and security threats.

Transfer learning is an important approach that produces pre-trained teacher models which can be used to quickly build specialized student models. However, recent research on transfer learning has found that it is vulnerable to various attacks, e.g., misclassification and backdoor attacks. However, it is still not clear whether transfer learning is vulnerable to model inversion attacks. Launching a model inversion attack against transfer learning scheme is challenging. Not only does the student model hide its structural parameters, but it is also not queried to the adversary. Hence, when targeting a student model, existing model inversion attacks fail, as they typically rely on querying the target model. In this paper, we initiate research into model inversion attacks against transfer learning with two novel attack methods. Both are black-box attacks, suiting different situations, that do not rely on queries to the target student model. In the first method, the adversary has the data samples that share the same distribution as the training set of the teacher model. In the second method, the adversary does not have any such samples. Experiments show that highly recognizable data records can be inverted with both of these methods. This research underscores the critical insight that even when a model is shielded from public queries, it can still be susceptible to model inversion attacks.

Federated learning (FL) trains a global model by aggregating local updates from multiple clients under a server's guidance. Despite its potential, FL is vulnerable to poisoning attacks where malicious clients intentionally corrupt their updates, compromising the global model's accuracy. Current defense strategies aim to tolerate or remove such corrupt updates, but they are not fully effective to prevent malicious clients from sending poisonous updates to the server, leaving the global model at risk. We propose a novel approach based on the evolution of cooperation, which promotes system-wide collaboration. Our defense method allows the server to selectively engage clients in the training process, encouraging them to provide clean updates or exclude those persistently malicious. We also introduce an attack framework where clients initially send clean updates to gain trust before sending malicious ones later. This model, designed to simulate advanced threats, can adapt to various attack types to increase its impact. Our experimental results show that this defense significantly improves resilience against such attacks, effectively safeguarding the global model even under complex threat scenarios.

Model inversion attacks pose a significant privacy threat to machine learning models by reconstructing sensitive data from their outputs. While various defenses have been proposed to counteract these attacks, they often come at the cost of the classifier's utility, thus creating a challenging trade-off between privacy protection and model utility. Moreover, most existing defenses require retraining the classifier for enhanced robustness, which is impractical for large-scale, well-established models. This article introduces a novel defense mechanism to better balance privacy and utility, particularly against adversaries who employ a machine learning model (i.e., inversion model) to reconstruct private data. Drawing inspiration from data poisoning attacks, which can compromise the performance of machine learning models, we propose a strategy that leverages data poisoning to contaminate the training data of inversion models, thereby preventing model inversion attacks. Two defense methods are presented. The first, termed label-preserving poisoning attacks for all output vectors (LPA), involves subtle perturbations to all output vectors while preserving their labels. Our findings demonstrate that these minor perturbations, introduced through a data poisoning approach, significantly increase the difficulty of data reconstruction without compromising the utility of the classifier. Subsequently, we introduce a second method, label-flipping poisoning for partial output vectors (LFP), which selectively perturbs a small subset of output vectors and alters their labels during the process. Empirical results indicate that LPA is notably effective, outperforming the current state-of-the-art defenses. Our data poisoning-based defense provides a new retraining-free defense paradigm that preserves the victim classifier's utility.

The rapid growth and escalating complexity of the Internet of Things (IoT) necessitate meticulous attention to ensure efficient and secure transactions among various autonomous components. To address this critical issue, this study proposes the integration of multiagent systems (MASs) and blockchain technology within the IoT domain. Uniquely, our approach employs smart contracts to manage exchanges between autonomous entities, thereby offering enhanced security, transparency, and reliability. The study introduces a set of innovative algorithms that regulate agent activities, such as creating blocks, sharing information, and conducting rating processes. Additionally, it provides a detailed analysis of their privacy and security aspects. Compared to traditional multiagent frameworks, empirical evidence demonstrates significant improvements in efficiency, adaptability, and scalability. This scholarly effort lays a robust foundation for further investigations into applying blockchain to enhance MASs, potentially paving the way for more sophisticated and context-specific strategies across various IoT fields.

Federated learning (FL) FL has emerged as a promising privacy-preserving machine-learning technology, enabling multiple clients to collaboratively train a global model without sharing raw data. With the increasing adoption of FL in Internet of Things (IoT) scenarios, concerns about security and privacy have become critical. In particular, gradient inversion attacks and poisoning attacks pose significant threats to the integrity and effectiveness of the global model. In response, we propose a comprehensive blockchain-based defense mechanism that effectively protects FL systems from such attacks. We develop a novel combination of techniques, including public blockchain level protection and private blockchain level protection, which work in tandem to prevent attackers from reconstructing figures using the obtained gradients. This unique combination of methods provides a robust defense against gradient inversion attacks in FL IoT scenarios. We conduct extensive experiments to validate the effectiveness of our proposed approach against gradient inversion and poisoning attacks. Our results demonstrate improved accuracy and stable convergence of training loss under poisoning attacks, indicating that our method can be applied to a wide range of FL IoT scenarios, enhancing both the security and privacy of distributed machine-learning systems.

Implementing federated learning within the Internet of Everything (IoE) framework poses substantial computational challenges, stemming from extensive client involvement, which can lead to escalated training expenses and diminished convergence rates. While many studies have investigated the combination of federated learning and blockchain networks, the integration of public and private chains to enhance federated learning performance remains largely unexplored. In this study, we introduce an innovative methodology that unifies public and private chains to mitigate clients’ computational demands while preserving data privacy and security, demonstrating compatibility within the IoE milieu and yielding favorable outcomes. To facilitate secure model migration and expedite training without incurring excessive computation costs, we delineate a blockchain-anchored model migration scheme tailored for resource-limited IoT infrastructures, establishing a private chain mechanism to incentivize companies possessing multiple devices or clients to prioritize model training. Employing blockchain technology guarantees trustworthiness in model migration, precluding the disclosure of devices’ confidential data. Overall, our innovative method provides an effective solution that improves the accuracy, privacy, and security of federated learning while reducing clients’ computational burdens within the context of the Internet of Everything (IoE).

Resource allocation in an edge-based Internet of Things (IoT) systems can be a challenging task, especially when the system contains many devices. Hence, in recent years, scholars have devoted some attention to designing different resource allocation strategies. Among these strategies, reinforcement learning is considered to be one of the best methods for maximizing the efficiency of resource allocation schemes. In a typical reinforcement learning scheme, the edge hosts would be required to upload their local parameters to a central server. However, this process has privacy implications given some of the data processed by the edge hosts is likely to be highly sensitive. To tackle this privacy issue, we developed a concurrent joint reinforcement learning method based on local differential privacy. Our approach allows the edge host to add noise during local training to preserve privacy, and to make joint decisions with the central server to devise an optimal resource allocation strategy. Experiments show that this approach yields high performance while preserving the privacy of the edge hosts.

Adaptive traffic signal control (ATSC) attempts to alleviate traffic congestion by dynamically adjusting the timing of traffic lights in real time, and multiagent reinforcement learning is one of the ways these systems learn how and when to change signals. However, traffic congestion continues to be a problem in most highly populated cities. We know that the current research into ATSC still has much ground to cover in terms of traffic efficiency, global optimality, and convergence stability. Hence, in this article, we outline a method that provides an advising method to the multiagent traffic signal control based on relative location in real time. ATSC is regarded as a multiagent environment, in which each traffic intersection is an agent to observe the distribution of the number of vehicles (state) at the intersection to control the change of signal lights (action). In our learning framework, each agent can not only take action by its advantage actor–critic model but can also ask its neighboring agent for advice when it is not confident in its decision. The advice is generated by a real-time updated advising model, which is based on the state and relative location of neighboring agents. Because the advising model provides real-time feedback, we find that learning is more effective and convergence is more stable. Moreover, drawing on neighboring states during taking action avoids falling into a local optimality caused by only observing local states. Comparisons with similar methods show that our method brings a significant improvement in a range of evaluation criteria, such as queue lengths, vehicle speeds, and trip delays.

Model inversion attacks involve reconstructing the training data of a target model, which raises serious privacy concerns for machine learning models. However, these attacks, especially learning-based methods, are likely to suffer from low attack accuracy, i.e., low classification accuracy of these reconstructed data by machine learning classifiers. Recent studies showed an alternative strategy of model inversion attacks, GAN-based optimization, can improve the attack accuracy effectively. However, these series of GAN-based attacks reconstruct only class-representative training data for a class, whereas learning-based attacks can reconstruct diverse data for different training data in each class. Hence, in this paper, we propose a new training paradigm for a learning-based model inversion attack that can achieve higher attack accuracy in a black-box setting. First, we regularize the training process of the attack model with an added semantic loss function and, second, we inject adversarial examples into the training data to increase the diversity of the class-related parts (i.e., he essential features for classification tasks) in training data. This scheme guides the attack model to pay more attention to the class-related parts of the original data during the data reconstruction process. The experimental results show that our method greatly boosts the performance of existing learning-based model inversion attacks. Even when no extra queries to the target model are allowed, the approach can still improve the attack accuracy of reconstructed data. This new attack shows that the severity of the threat from learning-based model inversion adversaries is underestimated and more robust defenses are required.

Machine learning models are susceptible to a range of adversarial activities. These attacks are designed to either infer private information from the target model or deceive it. For instance, an attacker may attempt to discern if a given data example is from the model’s training set (membership inference attacks) or create adversarial examples to mislead the model to make incorrect predictions (adversarial example attacks). Numerous defense methods have been proposed to counter these attacks. However, these methods typically share two common limitations. Firstly, most are not designed to address label-only attacks, which is a newly emerged kind of attacks that rely solely on the hard labels predicted by the target model. Secondly, they are often developed to mitigate specific attacks rather than universally various attacks. To address these limitations, this paper proposes a novel defense method that focuses on the most challenging attacks, i.e., label-only attacks, and can handle various types of label-only attacks. The key idea is to strategically modify the target model’s predicted labels using a meta-reinforcement learning technique. This ensures that attackers receive incorrect labels while benign users continue to receive correct labels. Notably, the defender, i.e., the owner of the target model, can make effective decisions without knowledge of the attacker’s behavior. The experimental results demonstrate that our proposed method is an effective defense against a range of attacks, including label-only model stealing, label-only membership inference, label-only model inversion, and label-only adversarial example attacks.

Model stealing attacks involve creating copies of machine learning models that have similar functionalities to the original model without proper authorization. Such attacks raise significant concerns about the intellectual property of the machine learning models. Nonetheless, current defense mechanisms against such attacks tend to exhibit certain drawbacks, notably in terms of utility, and robustness. For example, watermarking-based defenses require victim models to be retrained for embedding watermarks, which can potentially impact the main task performance. Moreover, other defenses, especially fingerprinting-based methods, often rely on specific samples like adversarial examples to verify ownership of the target model. These approaches might prove less robust against adaptive attacks, such as model stealing with adversarial training. It remains unclear whether normal examples, as opposed to adversarial ones, can effectively reflect the characteristics of stolen models. To tackle these challenges, we propose a novel method that leverages a neural network as a decoder to inverse the suspicious model’s outputs. Inspired by model inversion attacks, we argue that this decoding process will unveil hidden patterns inherent in the original outputs of the suspicious model. Drawing from these decoding outcomes, we calculate specific metrics to determine the legitimacy of the suspicious models. We validate the efficacy of our defense technique against diverse model stealing attacks, specifically within the domain of classification tasks based on deep neural networks.

Advanced persistent threats (APTs) are one of today’s major threats to cyber security. Highly determined attackers along with novel and evasive exfiltration techniques mean APT attacks elude most intrusion detection and prevention systems. The result has been significant losses for governments, organizations, and commercial entities. Intriguingly, despite greater efforts to defend against APTs in recent times, frequent upgrades in defense strategies are not leading to increased security and protection. In this paper, we demonstrate this phenomenon in an appropriately designed APT rivalry game that captures the interactions between attackers and defenders. What is shown is that the defender’s strategy adjustments actually leave useful information for the attackers, and thus intelligent and rational attackers can improve themselves by analyzing this information. Hence, a critical part of one’s defense strategy must be finding a suitable time to adjust one’s strategy to ensure attackers learn the least possible information. Another challenge for defenders is determining how to make the best use of one’s resources to achieve a satisfactory defense level. In support of these efforts, we figured out the optimal timings of a player’s strategy adjustment in terms of information leakage, which form a family of Nash equilibria. Moreover, two learning mechanisms are proposed to help defenders find an appropriate defense level and allocate their resources reasonably. One is based on adversarial bandits, and the other is based on deep reinforcement learning. Experimental simulations show the rationales behind the game and the optimality of the equilibria. The results also demonstrate that players indeed have the ability to improve themselves by learning from past experiences, which shows the necessity of specifying optimal strategy adjustment timings when defending against APTs.

In a model inversion attack, an adversary attempts to reconstruct the training data records of a target model using only the model’s output. In launching a contemporary model inversion attack, the strategies discussed are generally based on either predicted confidence score vectors, i.e., black-box attacks, or the parameters of a target model, i.e., white-box attacks. However, in the real world, model owners usually only give out the predicted labels; the confidence score vectors and model parameters are hidden as a defense mechanism to prevent such attacks. Unfortunately, we have found a model inversion method that can reconstruct representative samples of the target model’s training data based only on the output labels. We believe this attack requires the least information to succeed and, therefore, has the best applicability. The key idea is to exploit the error rate of the target model to compute the median distance from a set of data records to the decision boundary of the target model. The distance is then used to generate confidence score vectors which are adopted to train an attack model to reconstruct the representative samples. The experimental results show that highly recognizable representative samples can be reconstructed with far less information than existing methods.

In multiagent learning, one of the main ways to improve learning performance is to ask for advice from another agent. Contemporary advising methods share a common limitation that a teacher agent can only advise a student agent if the teacher has experience with an identical state. However, in highly complex learning scenarios, such as autonomous driving, it is rare for two agents to experience exactly the same state, which makes the advice less of a learning aid and more of a one-time instruction. In these scenarios, with contemporary methods, agents do not really help each other learn, and the main outcome of their back and forth requests for advice is an exorbitant communications' overhead. In human interactions, teachers are often asked for advice on what to do in situations that students are personally unfamiliar with. In these, we generally draw from similar experiences to formulate advice. This inspired us to provide agents with the same ability when asked for advice on an unfamiliar state. Hence, we propose a model-based self-advising method that allows agents to train a model based on states similar to the state in question to inform its response. As a result, the advice given can not only be used to resolve the current dilemma but also many other similar situations that the student may come across in the future via self-advising. Compared with contemporary methods, our method brings a significant improvement in learning performance with much lower communication overheads.

A cyber system may face to multiple attackers from diverse adversaries, who usually employ sophisticated techniques to both continuously steal sensitive data and avoid being detected by defense strategies. This continuous process is typically involved in an advanced persistent threat (APT). Since the game theory is an ideal mathematical model for investigating continuous decision making of competing players, it is broadly used to research the interaction between defenders and APT attackers. Although many researchers are now using the game theory to defend against APT attacks, most of the existing solutions are limited to single-defender, single-attacker scenarios. In the real world, threats by multiple attackers are not uncommon and multiple defenders can be put in place. Therefore, to overcome the limitation of the existing solutions, we develop a multiagent deep reinforcement learning (MADRL) method with a novel sampling approach. The MADRL method allows defenders to create strategies on the fly and share their experience with other defenders. To develop this method, we create a multidefender, multiattacker game model and analyze the equilibrium of this model. The results of a series of experiments demonstrate that, with MADRL, defenders can quickly learn efficient strategies against attackers.

Summary The growing popularity of users in online social network gives a big opportunity for online auction. The famous Information Diffusion Mechanism (IDM) is an excellent methods even meet the incentive compatibility and individual rationality. Although the existing auction in online social network has considered the buyers' information has not known by the seller, current mechanism still cannot preserve the information such as prices. In this paper, we propose a novel mechanism which modeled the auction process in online social network and preserved users' privacy by using differential privacy mechanism. Our mechanism can successfully process the auction and at the same time preserve clients' price information from neighbors. We achieved these by adding Laplace noise for its valuation and the number of valuation seller received in the auction process. We also formulate this mechanism on the real network to show the feasibility and effective of the proposed mechanism.

Summary Due to the rapid development of the cloud computing environment, it is widely accepted that cloud servers are important for users to improve work efficiency. Users need to know servers' capabilities and make optimal decisions on selecting the best available servers for users' tasks. We consider the process of learning servers' capabilities by users as a multiagent reinforcement learning process. The learning speed and efficiency in reinforcement learning can be improved by sharing the learning experience among learning agents which is defined as advising. However, existing advising frameworks are limited by the requirement that during advising all learning agents in a reinforcement learning environment must have exactly the same actions. To address the above limitation, this article proposes a novel differentially private advising framework for multiagent reinforcement learning. Our proposed approach can significantly improve the application of conventional advising frameworks when agents have one different action. The approach can also widen the applicable field of advising and speed up reinforcement learning by triggering more potential advising processes among agents with different actions.

In multi-agent reinforcement learning, transfer learning is one of the key techniques used to speed up learning performance through the exchange of knowledge among agents. However, there are three challenges associated with applying this technique to real-world problems. First, most real-world domains are partially rather than fully observable. Second, it is difficult to pre-collect knowledge in unknown domains. Third, negative transfer impedes the learning progress. We observe that differentially private mechanisms can overcome these challenges due to their randomization property. Therefore, we propose a novel differential transfer learning method for multi-agent reinforcement learning problems, characterized by the following three key features. First, our method allows agents to implement real-time knowledge transfers between each other in partially observable domains. Second, our method eliminates the constraints on the relevance of transferred knowledge, which expands the knowledge set to a large extent. Third, our method improves robustness to negative transfers by applying differentially exponential noise and relevance weights to transferred knowledge. The proposed method is the first to use the randomization property of differential privacy to stimulate the learning performance in multi-agent reinforcement learning system. We further implement extensive experiments to demonstrate the effectiveness of our proposed method.

Resource allocation is a fundamental research issue in IoT edge computing, and reinforcement learning is fast becoming a common solution. The majority of the current techniques involve decision makers who determine how and where resources should be distributed. In a standard cloud system, this decision maker is a central server. In an edge system, the decision maker is an edge host. Both approaches have drawbacks. Edge hosts do not always have access to enough global information to create the most optimal resource allocation strategy. Central servers do but at the cost of privacy. A solution is needed that can do both. This article, therefore, presents a novel resource allocation method called concurrent federated reinforcement learning. The scheme inherits the privacy protection of federated learning, the complex problem solving power of reinforcement learning and adds concurrency in the form of joint decision making so the resource allocation strategies work to the benefit of the global system. The experiments demonstrate that the approach provides the state-of-the-art performance in system-wide utility, speed of task completion, and resource utilization.

Agent advising is one of the main approaches to improve agent learning performance by enabling agents to share advice. Existing advising methods have a common limitation that an adviser agent can offer advice to an advisee agent only if the advice is created in the same state as the advisee's state. However, in complex environments, it is a very strong requirement that two states are the same, because a state may consist of multiple dimensions and two states being the same means that all these dimensions in the two states are correspondingly identical. Therefore, this requirement may limit the applicability of existing advising methods to complex environments. In this article, inspired by the differential privacy scheme, we propose a differential advising method that relaxes this requirement by enabling agents to use advice in a state even if the advice is created in a slightly different state. Compared with the existing methods, agents using the proposed method have more opportunity to take advice from others. This article is the first to adopt the concept of differential privacy on advising to improve agent learning performance instead of addressing security issues. The experimental results demonstrate that the proposed method is more efficient in complex environments than the existing methods.

Planning is one of the main approaches used to improve agents’ working efficiency by making plans beforehand. However, during planning, agents face the risk of having their private information leaked. This article proposes a novel strong privacy-preserving planning approach for logistic-like problems. This approach outperforms existing approaches by addressing two challenges: 1) simultaneously achieving strong privacy; completeness and efficiency; and 2) addressing communication constraints. These two challenges are prevalent in many real-world applications including logistics in military environments and packet routing in networks. To tackle these two challenges, our approach adopts the differential privacy technique, which can both guarantee strong privacy and control communication overhead. To the best of our knowledge, this article is the first to apply differential privacy to the field of multi-agent planning as a means of preserving the privacy of agents for logistic-like problems. We theoretically prove the strong privacy and completeness of our approach and empirically demonstrate its efficiency. We also theoretically analyze the communication overhead of our approach and illustrate how differential privacy can be used to control it.

Machine learning models are vulnerable to data inference attacks, such as membership inference and model inversion attacks. In these types of breaches, an adversary attempts to infer a data record’s membership in a dataset or even reconstruct this data record using a confidence score vector predicted by the target model. However, most existing defense methods only protect against membership inference attacks. Methods that can combat both types of attacks require a new model to be trained, which may not be time-efficient. In this paper, we propose a differentially private defense method that handles both types of attacks in a time-efficient manner by tuning only one parameter, the privacy budget. The central idea is to modify and normalize the confidence score vectors with a differential privacy mechanism which preserves privacy and obscures membership and reconstructed data. Moreover, this method can guarantee the order of scores in the vector to avoid any loss in classification accuracy. The experimental results show the method to be an effective and timely defense against both membership inference and model inversion attacks with no reduction in accuracy.

Artificial Intelligence (AI) has attracted a great deal of attention in recent years. However, alongside all its advancements, problems have also emerged, such as privacy violations, security issues and model fairness. Differential privacy, as a promising mathematical model, has several attractive properties that can help solve these problems, making it quite a valuable tool. For this reason, differential privacy has been broadly applied in AI but to date, no study has documented which differential privacy mechanisms can or have been leveraged to overcome its issues or the properties that make this possible. In this paper, we show that differential privacy can do more than just preserve privacy. It can also be used to improve security, stabilize learning, build fair models, and impose composition in selected areas of AI. With a focus on regular machine learning, distributed machine learning, deep learning, and multi-agent systems, the purpose of this article is to deliver a new view on many possibilities for improving AI performance with differential privacy techniques.

Abstract To alleviate the traffic congestion caused by the sharp increase in the number of private cars and save commuting costs, taxi carpooling service has become the choice of many people. Current research on taxi carpooling services has focused on shortening the detour distances. While with the development of intelligent cities, efficiently match passengers and vehicles and planning routes become urgent. And the privacy between passengers in the taxi carpooling service also needs to be considered. In this paper, we propose a time-optimal and privacy-preserving carpool route planning system via deep reinforcement learning. This system uses the traffic information around the carpooling vehicle to optimize passengers’ travel time, not only to efficiently match passengers and vehicles but also to generate detailed route planning for carpooling vehicles. We conducted experiments on an Internet of Vehicles simulator CARLA, and the results demonstrate that our method is better than other advanced methods and has better performance in complex environments.

Cyber deception is one of the key approaches used to mislead attackers by hiding or providing inaccurate system information. There are two main factors limiting the real-world application of existing cyber deception approaches. The first limitation is that the number of systems in a network is assumed to be fixed. However, in the real world, the number of systems may be dynamically changed. The second limitation is that attackers' strategies are simplified in the literature. However, in the real world, attackers may be more powerful than theory suggests. To overcome these two limitations, we propose a novel differentially private game theoretic approach to cyber deception. In this proposed approach, a defender adopts differential privacy mechanisms to strategically change the number of systems and obfuscate the configurations of systems, while an attacker adopts a Bayesian inference approach to infer the real configurations of systems. By using the differential privacy technique, the proposed approach can 1) reduce the impacts on network security resulting from changes in the number of systems and 2) resist attacks regardless of attackers' reasoning power. The experimental results demonstrate the effectiveness of the proposed approach.

Agent advising is one of the key approaches to improve agent learning performance by enabling agents to ask for advice between each other. Existing agent advising approaches have two limitations. The first limitation is that all the agents in a system are assumed to be friendly and cooperative. However, in the real world, malicious agents may exist and provide false advice to hinder the learning performance of other agents. The second limitation is that the analysis of communication overhead in these approaches is either overlooked or simplified. However, in communication-constrained environments, communication overhead has to be carefully considered. To overcome the two limitations, this paper proposes a novel differentially private agent advising approach. Our approach employs the Laplace mechanism to add noise on the rewards used by student agents to select teacher agents. By using the differential privacy technique, the proposed approach can reduce the impact of malicious agents without identifying them. Also, by adopting the privacy budget concept, the proposed approach can naturally control communication overhead. The experimental results demonstrate the effectiveness of the proposed approach.

The popularity of cloud computing has fueled the growth in multi-tenant service-based systems (SBSs) that are composed of selected cloud services. In the cloud environment, a multi-tenant SBS simultaneously serves multiple tenants that usually have differentiated QoS requirements. This unique characteristic further complicates the problems of QoS-aware service selection at build-time and system adaptation at runtime, and renders conventional approaches obsolete and inefficient. In the dynamic and volatile cloud environment, the efficiency of building and adapting a multi-tenant SBS is of paramount importance. In this paper, we present two service recommendation approaches for multi-tenant SBSs, one for build-time and one for runtime, based on K-Means clustering and Locality-Sensitive Hashing (LSH) techniques respectively, aiming at finding appropriate services efficiently. Extensive experimental results demonstrate that our approaches can facilitate fast multi-tenant SBS construction and rapid system adaptation.

Service composition is an important research problem in service computing systems, which combines simple and individual services into composite services to fulfill users' complex requirements. Service composition usually consists of four stages, i.e., service discovery, candidate selection, service negotiation and task execution. In self-organising systems, there is the fifth stage of service composition: self-evolution. Most of existing works study only some of the five stages. However, these five stages should be systematically studied so as to develop an integrated and efficient service composition approach. Against this background, this paper proposes an agent-based integrated self-evolving service composition approach. This approach systematically takes the five stages of service composition into consideration. It is also decentralised and self-evolvable. Experimental results demonstrate that the proposed approach can achieve almost the same success rates while uses much less communication overhead and time consumption in comparison with three existing representative approaches.

Summary In market‐based cloud environments, both resource consumers and providers are self‐interested; additionally, they can come and leave the environment freely. Therefore, the environment is competitive and uncertain. Because of the competition, participants may cheat in making deals, and this represents that the environment is insecure to resource providers who intend to earn profits through renting their resources to the tasks of resource consumers. Against this, in this paper, intelligent agents are designed to strategically quote for the tasks that they are interested in, on behalf of resource providers. Agents could quote according to the messages it obtained and the information learnt and predicted from the messages, to minimize the influence of insecure factors, such as cheating, competition, and dynamism. The experimental evaluation shows that the proposed method outperforms both a well‐known multiresource negotiation‐based task allocation method and a max‐sum belief propagation–based method.

Sleep/wake-up scheduling is one of the fundamental problems in wireless sensor networks, since the energy of sensor nodes is limited and they are usually unrechargeable. The purpose of sleep/wake-up scheduling is to save the energy of each node by keeping nodes in sleep mode as long as possible (without sacrificing packet delivery efficiency) and thereby maximizing their lifetime. In this paper, a self-adaptive sleep/wake-up scheduling approach is proposed. Unlike most existing studies that use the duty cycling technique, which incurs a tradeoff between packet delivery delay and energy saving, the proposed approach, which does not us duty cycling, avoids such a tradeoff. The proposed approach, based on the reinforcement learning technique, enables each node to autonomously decide its own operation mode (sleep, listen, or transmission) in each time slot in a decentralized manner. Simulation results demonstrate the good performance of the proposed approach in various circumstances.

The proliferation of cloud computing has fueled the rapid growth of multi-tenant service-based systems (SBSs), which serve multiple tenants simultaneously by composing existing services in the form of business processes. In a distributed and volatile operating environment, runtime anomalies may occur to the component services of an SBS and cause end-to-end quality violations. Engineering multi-tenant SBSs that can quickly handle runtime anomalies cost effectively has become a significant challenge. Different approaches have been proposed to formulate fault tolerance strategies for engineering SBSs. However, none of the existing approaches has sufficiently considered the service criticality based on multi-tenancy where multiple tenants share the same SBS instance with different multi-dimensional quality preferences. In this paper, we propose Criticality-based Fault Tolerance for Multi-Tenant SBSs (CFT4MTS), a novel approach that formulates cost-effective fault tolerance strategies for multi-tenant SBSs by providing redundancy for the critical component services. First, the criticality of each component service is evaluated based on its multi-dimensional quality and multiple tenants sharing the component service with differentiated quality preferences. Then, the fault tolerance problem is modelled as an Integer Programming problem to identify the optimal fault tolerance strategy. The experimental results show that, compared with three existing representative approaches, CFT4MTS can alleviate degradation in the quality of multi-tenant SBSs in a much more effective and efficient way.

Multi-tenant service-based systems (SBSs) have gained unprecedented prominence in recent years. Network-accessible Web services are composed in the form of business process to simultaneously fulfill multiple tenants' functional and multi-dimensional quality-of-service (QoS) requirements. Those services often operate in a distributed and volatile environment. It is of tremendous importance to monitor the services to detect or predict the anomalies timely to avoid QoS violations and Service Level Agreement (SLA) breaches. However, monitoring Web services consumes resources and incurs system overhead. It is impractical to constantly monitor all the services in an SBS. Thus, it is a significant challenge to monitor the services of a multi-tenant SBS in a cost-effective manner. In this paper, we propose CM4MTS (Criticality-based Monitoring for Multi-Tenant SBSs) for formulating cost-effective monitoring strategy. The criticality of a service is evaluated based on its impacts on the quality of the SBS upon runtime anomalies and the tenants sharing the service. The services with higher criticalities in an SBS are given higher priorities in monitoring resource allocation. Extensive experiments show that CM4MTS outperforms representative approaches in ensuring the quality of multi-tenant SBSs.

Service monitoring is an important research problem in service-based systems (SBSs), which aims to monitor the failure of services in a timely manner while using resources as few as possible. Most of the existing service monitoring approaches are centralised which suffer the potential of single point of failure and are not suitable in distributed large scale SBSs. Moreover, these centralised monitoring approaches are designed only in single-tenant SBSs. Nowadays, the scale of SBSs is extremely large, i.e., including a large number of services and clients. Thus, it is essential for service monitoring approaches to work well in distributed large scale SBSs and support multi-tenancy. Towards this end, in this paper, an agent-based decentralised service monitoring approach is developed in multi-tenant SBSs. Compared to the centralised approaches, the proposed decentralised approach can avoid the single point of failure and can balance the computation over the monitoring agents. Also, unlike existing approaches, the proposed approach is developed in multi-tenant SBSs. Experimental results demonstrate that the proposed approach can respond as quickly as centralised approaches but has much less computation overhead than centralised approaches.

In a distributed, dynamic and volatile operating environment, runtime anomalies occurring in service-oriented systems (SOSs) must be located and fixed in a timely manner in order to guarantee successful delivery of outcomes in response to user requests. Monitoring all component services constantly and inspecting the entire SOS upon a runtime anomaly are impractical due to excessive resource and time consumption required, especially in large-scale scenarios. We present a spectrum-based approach that goes through a five-phase process to quickly localize runtime anomalies occurring in SOSs based on end-to-end system delays. Upon runtime anomalies, our approach calculates the similarity coefficient for each basic component (BC) of the SOS to evaluate their suspiciousness of being faulty. Our approach also calculates the delay coefficients to evaluate each BC's contribution to the severity of the end-to-end system delays. Finally, the BCs are ranked by their similarity coefficient scores and delay coefficient scores to determine the order of them being inspected. Extensive experiments are conducted to evaluate the effectiveness and efficiency of the proposed approach. The results indicate that our approach significantly outperforms random inspection and the popular Ochiai-based inspection in localizing single and multiple runtime anomalies effectively. Thus, our approach can help save time and effort for localizing runtime anomalies occuring in SOSs.

With the fast growth of applications of service-oriented architecture (SOA) in software engineering, there has been a rapid increase in demand for building service-based systems (SBSs) by composing existing Web services. Finding appropriate component services to compose is a key step in the SBS engineering process. Existing approaches require that system engineers have detailed knowledge of SOA techniques which is often too demanding. To address this issue, we propose Keyword Search for Service-based Systems (KS3), a novel approach that integrates and automates the system planning, service discovery and service selection operations for building SBSs based on keyword search. KS3 assists system engineers without detailed knowledge of SOA techniques in searching for component services to build SBSs by typing a few keywords that represent the tasks of the SBSs with quality constraints and optimisation goals for system quality, e.g., reliability, throughput and cost. KS3 offers a new paradigm for SBS engineering that can significantly save the time and effort during the system engineering process. We conducted large-scale experiments using two real-world Web service datasets to demonstrate the practicality, effectiveness and efficiency of KS3.

This paper surveys the literature over the last decades in the field of self-organizing multiagent systems. Self-organization has been extensively studied and applied in multiagent systems and other fields, e.g., sensor networks and grid systems. Self-organization mechanisms in other fields have been thoroughly surveyed. However, there has not been a survey of self-organization mechanisms developed for use in multiagent systems. In this paper, we provide a survey of existing literature on self-organization mechanisms in multiagent systems. We also highlight the future work on key research issues in multiagent systems. This paper can serve as a guide and a starting point for anyone who will conduct research on self-organization in multiagent systems. Also, this paper complements existing survey studies on self-organization in multiagent systems.

To solve the problem of group task allocation with time constraints in open and dynamic network environments, this paper proposes a decentralized combinatorial auction-based approach for group task allocation. In the proposed approach, both resource providers and consumers are modeled as intelligent agents. The proposed approach is decentralized, so all the agents are limited to communicating with their neighboring agents. The proposed approach also allows agents to enter and leave the network environments freely, and is robust for the dynamism and openness of the network environments. Tasks in the proposed approach have deadlines, and may need the collaboration of a group of self-interested providers. The experimental results demonstrate that the proposed approach outperforms two well-known task allocation approaches in terms of success rate of task allocation, the individual utility of the agents, the speed of task allocation and scalability.

Summary This paper addresses the task allocation problem in an open, dynamic grid environments and service‐oriented environments. In such environments, both grid/service providers and consumers can be modelled as intelligent agents. These agents can leave and enter the environment freely at any time. Task allocation under time constraints becomes a challenging issue in such environments because it is difficult to apply a central controller during the allocation process due to the openness and dynamism of the environments. This paper proposes a negotiation‐based method for task allocation under time constraints in an open, dynamic grid environment, where both consumer and provider agents can freely enter or leave the environment. In this method, there is no central controller available, and agents negotiate with each other for task allocation based only on local views. The experimental results show that the proposed method can outperform the current methods in terms of the success rate of task allocation and the total profit obtained from the allocated tasks by agents under different time constraints. Copyright © 2014 John Wiley & Sons, Ltd.

This paper studies the phenomenon of the evolution of cooperation in distributed networks by using an iterated game. An iterated game in a distributed network is a multiple round game, where in each round, a player gains a payoff by playing a game with its neighbours and updates its action based on the actions and/or payoffs of its neighbours. The interaction model between players is usually represented as a two-player, two-action (namely cooperation and defection) Prisoner’s Dilemma game (which is a prototypical model for interaction between selfish individuals). Many researchers have developed strategies (also called update rules) for the evolution of cooperation in distributed networks in order to enhance cooperation, i.e., to increase the proportion of cooperators. Experimental results reported in the current literature, however, have demonstrated that each of these strategies has both advantages and disadvantages. In this paper, a self-adaptive strategy is proposed for the evolution of cooperation in distributed networks, which can utilise the strengths and avoid the limitations of existing strategies. Moreover, we have a theoretical finding about the final proportion of cooperators, evolved by any pure (or deterministic) strategies, in four types of a game. This finding is independent of the initial proportion of cooperators, the topology of the network (e.g., a small-world network or a scale-free network), and the specific game (e.g., the Prisoner’s Dilemma game or the Snow Drift game).

Self-organisation provides a suitable paradigm for developing self-managed complex distributed systems, such as grid computing and sensor networks. In this paper, an integrative self-organisation mechanism is proposed. Unlike current related studies, which propose only a single principle of self-organisation, this mechanism synthesises the three principles of self-organisation: cloning/spawning, resource exchange and relation adaptation. Based on this mechanism, an agent can autonomously generate new agents when it is overloaded, exchange resources with other agents if necessary, and modify relations with other agents to achieve a better agent network structure. In this way, agents can adapt to dynamic environments. The proposed mechanism is evaluated through a comparison with three other approaches, each of which represents state-of-the-art research in each of the three self-organisation principles. Experimental results demonstrate that the proposed mechanism outperforms the three approaches in terms of the profit of individual agents and the entire agent network, the load-balancing among agents, and the time consumption to finish a simulation run.

In some real systems, e.g., distributed sensor networks, individual agents often need to form coalitions to accomplish complex tasks. Due to communication and computation constraints, it is infeasible for agents to directly interact with all other agents to form coalitions. Most previous coalition formation studies, however, overlooked this aspect. Those studies did not provide an explicitly modeled agent network or assumed that agents were in a fully connected network, where an agent can directly communicate with all other agents. Thus, to alleviate this problem, it is necessary to provide a neighborhood network structure, within which agents can directly interact only with their neighbors. Toward this end, in this paper, a self-adaptation-based dynamic coalition formation mechanism is proposed. The proposed mechanism operates in a neighborhood agent network. Based on self-adaptation principles, this mechanism enables agents to dynamically adjust their degrees of involvement in multiple coalitions and to join new coalitions at any time. The self-adaptation process, i.e., agents adjusting their degrees of involvement in multiple coalitions, is realized by exploiting a negotiation protocol. The proposed mechanism is evaluated through a comparison with a centralized mechanism (CM) and three other coalition formation mechanisms. Experimental results demonstrate the good performance of the proposed mechanism in terms of the entire network profit and time consumption. Additionally, a brief survey of current coalition formation research is also provided. From this survey, readers can have a general understanding of the focuses and progress of current research. This survey provides a classification of the primary emphasis of each related work in coalition formation, so readers can conveniently find the most related studies.

Recently, task allocation in multi-agent systems has been investigated by many researchers. Some researchers suggested to have a central controller which has a global view about the environment to allocate tasks. Although centralized control brings convenience during task allocation processes, it also has some obvious weaknesses. Firstly, a central controller plays an important role in a multi-agent system, but task allocation procedures will break down if the central controller of a system cannot work properly. Secondly, centralized multi-agent architecture is not suitable for distributed working environments. In order to overcome some limitations caused by centralized control, some researchers proposed distributed task allocation protocols. They supposed that each agent has a limited local view about its direct linked neighbors, and can allocate tasks to its neighbors. However, only involving direct linked neighbors could limit resource origins, so that the task allocation efficiency will be greatly reduced. In this paper, we propose an efficient task allocation protocol for P2P multi-agent systems. This protocol allows not only neighboring agents but also indirect linked agents in the system to help with a task if needed. Through this way, agents can achieve more efficient and robust task allocations in loosely coupled distributed environments (e.g. P2P multi-agent systems). A set of experiments are presented in this paper to evaluate the efficiency and adaptability of the protocol. The experiment result shows that the protocol can work efficiently in different situations.