It is known that deep neural networks may exhibit dangerous behaviors under various security threats (e.g., backdoor attacks, adversarial attacks and safety property violation) and there exists an ongoing arms race between attackers and defenders. In this work, we propose a complementary perspective to utilize recent progress on ''neural network repair'' to mitigate these security threats and repair various kinds of neural network defects (arising from different security threats) within a unified framework, offering a potential silver bullet solution to real-world scenarios. To substantially push the boundary of existing repair techniques (suffering from limitations such as lack of guarantees, limited scalability, considerable overhead, etc) in addressing more practical contexts, we propose ProRepair, a novel provable neural network repair framework driven by formal preimage synthesis and property refinement. The key intuitions are: (i) synthesizing a precise proxy box to characterize the feature space preimage, which can derive a bounded distance term sufficient to guide the subsequent repair step towards the correct outputs, and (ii) performing property refinement to enable surgical corrections and scale to more complex tasks. We evaluate ProRepair across four security threats repair tasks on six benchmarks and the results demonstrate it outperforms existing methods in effectiveness, efficiency and scalability. For point-wise repair, ProRepair corrects models while preserving performance and achieving significantly improved generalization, with a speed-up of 5× to 2000× over existing provable approaches. In region-wise repair, ProRepair successfully repairs all 36 safety property violation instances (compared to 8 by the best existing method), and can handle 18× higher dimensional spaces.

Deep neural networks (DNNs) are suffering from ethical issues such as individual discrimination. In response, extensive NN repair techniques have been developed to adjust models and mitigate such undesired behaviors. However, existing fairness repair methods are typically data-centric, which often lack provable guarantees and generalization to unseen samples. To overcome these limitations, we propose PROF, a novel fairness repair framework with provable guarantees. The key intuition of PROF is to leverage interval bound propagation (a widely used NN verification technique) to soundly capture model outputs over the whole set ${\mathcal{S}}\left(x\right)$ around a biased sample x. The derived bounds are utilized to guide fairness repair which encourages the model to produce consistent outputs on ${\mathcal{S}}\left(x\right)$. Specifically, we integrate fairness constraints and model modifications into a unified constraint-solving formulation, which can be transformed to a Mixed-Integer Linear Programming (MILP) problem solvable by off-the-shelf solvers. The solution to the MILP problem effectively induces a repaired model with guaranteed fairness over the whole set ${\mathcal{S}}\left(x\right)$. We evaluate PROF on four widely used benchmark datasets and demonstrate that it achieves provable fairness repair, with generalization of up to 95.93% on full datasets and 93.16% on the entire input space. Notably, PROF can be easily configured to support multiple sensitive attributes and more practical fairness definitions, while providing provable repair guarantees and delivering around 90% fairness improvement. Our code is available in this $\color{red}{\text{repository}}$.

Large-scale matrix multiplication is a critical operation in various fields such as machine learning, scientific computing, and graphics processing, but performing it on a single machine introduces significant computational latency. Therefore, matrices are partitioned along different dimensions, decomposed into multiple subtasks, and executed in distributed systems. However, the presence of stragglers in distributed systems can severely impact the speed of matrix multiplication. So, coding schemes are introduced to mitigate the straggler problem. Recently, coding schemes for three-dimensional matrix partitioning have gained increasing attention, including DEP codes. However, these schemes have not focused on decoding accuracy and job completion time. In this paper, with the aim of enhancing decoding accuracy and reducing job completion time, we combine the grouping strategy with Systematic Matdot codes to propose Grouped Systematic Matdot (GSM) codes. Experimental results demonstrate that, compared to DEP codes, GSM codes ensure 100% decoding accuracy and achieve shorter encoding time, communication time, and local computation time, thereby reducing job completion time by at least 45%. Moreover, GSM codes consume fewer memory resources, and as the matrix size increases, their time advantage becomes more pronounced.

Neural network repair aims to fix the 'bugs'1 of neural networks by modifying the model's architecture or parameters. However, due to the data-driven nature of neural networks, it is difficult to explain the relationship between the internal neurons and erroneous behaviors, making further repair challenging. While several work exists to identify responsible neurons based on gradient or causality analysis, their effectiveness heavily rely on the quality of available 'bugged' data and multiple heuristics in layer or neuron selection. In this work, we address the issue utilizing the power of formal verification (in particular for neural networks). Specifically, we propose VeRe, a verification-guided neural network repair framework that performs fault localization based on linear relaxation to symbolically calculate the repair significance of neurons and furthermore optimize the parameters of problematic neurons to repair erroneous behaviors. We evaluated VeRe on various repair tasks, and our experimental results show that VeRe can efficiently and effectively repair all neural networks without degrading the model's performance. For the task of removing backdoors, VeRe successfully reduces attack success rate from 98.47% to 0.38% on average, while causing an average performance drop of 0.9%. For the task of repairing safety properties, VeRe successfully repairs all the 36 tasks and achieves 99.87% generalization on average.

Sybil attacks are a prevalent concern within the realm of crowdsourcing, underscoring the significance of quality control in this domain. Truth discovery has been extensively studied to deduce the most trustworthy information from conflicting data based on the principle that reliable workers yield reliable answers. However, existing truth discovery approaches overlook the metric of workers' reputations, e.g., workers' historical approval rates on crowdsourcing platforms, despite being inflated and noisy, they offer a rough indication of workers' ability. In this paper, we first refine the approval rate using Wilson Lower Bound to enhance its confidence, and then mitigate its noise and inflation through a method based on ranking similarity. Specifically, we propose a method called RCTD (Reputation-Constrained Truth Discovery), which introduces a similarity metric between the rankings of workers' weights and the refined approval rates. This metric serves as a penalizing factor in the objective function of the truth discovery, restricting workers' weights to avoid excessively deviating from their historical reputation during the weight estimation process. We solve the objective function by introducing the block coordinate descent coupled with heuristics approach method. Experimental results on real-world datasets demonstrate that our approach achieves more accurate inference of true results in the Sybil attack environment compared to the state-of-the-art methods.

Social manufacturing has significantly advanced the industrial Internet of Things (IIoT), integrating information technology and operation technology to enhance production efficiency and quality, and to foster new business models. This integration, however, introduces novel risks, including advanced persistent threats, which demand robust security measures to safeguard IIoT systems. This article proposes an iCyberGuard game model, tailored for IIoT environments, designed to imitate the cyber and physical attacks for information and operation technologies. Then, we used a reinforcement learning algorithm to compute the optimal strategy. We conducted comprehensive simulation experiments, which demonstrate that our model the strategic interactions between attackers and defenders. Participants are enabled to learn adaptively, discerning optimal strategies based on the intelligence of their adversaries. Finally, we explain the practical significance of the best strategy of defenders or attackers, and how users can rely on these best strategies to strengthen the security performance of the network.

Graph neural networks (GNNs) have achieved remarkable success in various real-world applications. However, recent studies highlight the vulnerability of GNNs to malicious perturbations. Previous adversaries primarily focus on graph modifications or node injections to existing graphs, yielding promising results but with notable limitations. Graph modification attack (GMA) requires manipulation of the original graph, which is often impractical, while graph injection attack (GIA) necessitates training a surrogate model in the black-box setting, leading to significant performance degradation due to divergence between the surrogate architecture and the actual victim model. Furthermore, most methods concentrate on a single attack goal and lack a generalizable adversary to develop distinct attack strategies for diverse goals, thus limiting precise control over victim model behavior in real-world scenarios. To address these issues, we present a gradient-free generalizable adversary that injects a single malicious node to manipulate the classification result of a target node in the black-box evasion setting. Specifically, we model the single-node injection label specificity attack as a Markov decision process (MDP) and propose gradient-free generalizable single node injection attack, namely G2-SNIA, a reinforcement learning framework employing proximal policy optimization (PPO). By directly querying the victim model, G2-SNIA learns patterns from exploration to achieve diverse attack goals with extremely limited attack budgets. Through comprehensive experiments over three acknowledged benchmark datasets and four prominent GNNs in the most challenging and realistic scenario, we demonstrate the superior performance of our proposed G2-SNIA over the existing state-of-the-art baselines. Moreover, by comparing G2-SNIA with multiple white-box evasion baselines, we confirm its capacity to generate solutions comparable to those of the best adversaries.

As a challenge of practical significance, fraud detection has great potential for telecom fraud prevention, economic crime prevention, and personal property preservation. Fraudulent activities are always buried in massive regular transactions, making it hard to find them. Traditional rule-based approaches need multiple domain-specific rules and multistep verification, which limits their transferability and efficiency. Machine learning-based methods might ignore the intricate interactions or the temporal relations among accounts. Meanwhile, the lack of sufficient manual labels restricts their performance. To overcome the above limitations, we present a multipattern integrated network (MPIN) in this article to identify fraudulent accounts in transaction networks. Specifically, MPIN considers the interactions among nodes from three perspectives: inflows, outflows, and their mutual influences. To learn the behavior pattern of each node, MPIN first applies an attention mechanism to integrate the short-term information and then learns the long-term patterns by aggregating multiple short-term patterns. Behavior patterns from different perspectives together with long short-term modeling enable the model to precisely distinguish fraudulent accounts from the normal ones. Moreover, contrastive pretraining with temporal consistency and local tightness guarantee is adopted to alleviate the label sparsity issue and provide the model with low-variance performance. We conducted experiments on two real-world transaction networks, and the results showed the effectiveness of MPIN compared with five state-of-the-art baselines.

Model inversion attacks (MIAs) aim to recover private data from inaccessible training sets of deep learning models, posing a privacy threat. MIAs primarily focus on the white-box scenario where attackers have full access to the model’s structure and parameters. However, practical applications are usually in black-box scenarios or label-only scenarios, i.e., the attackers can only obtain the output confidence vectors or labels by accessing the model. Therefore, the attack models in existing MIAs are difficult to effectively train with the knowledge of the target model, resulting in sub-optimal attacks. To the best of our knowledge, we pioneer the research of a powerful and practical attack model in the label-only scenario. In this paper, we develop a novel MIA method, leveraging a conditional diffusion model (CDM) to recover representative samples under the target label from the training set. Two techniques are introduced: selecting an auxiliary dataset relevant to the target model task and using predicted labels as conditions to guide training CDM; and inputting target label, pre-defined guidance strength, and random noise into the trained attack model to generate and correct multiple results for final selection. This method is evaluated using Learned Perceptual Image Patch Similarity as a new metric and as a judgment basis for deciding the values of hyper-parameters. Experimental results show that this method can generate similar and accurate samples to the target label, outperforming generators of previous approaches.

Personal data custodian services enable data owners to share their data with data consumers in a convenient manner, anytime and anywhere. However, with data hosted in these services being beyond the control of the data owners, it raises significant concerns about privacy in personal data sharing. Many schemes have been proposed to realize fine-grained access control and privacy protection in data sharing. However, they fail to protect the rights of data owners to their data under the law, since their designs focus on the management of system administrators rather than enhancing the data owners’ privacy. In this paper, we introduce a novel task-driven personal data sharing system based on the data capsule paradigm realizing personal data sharing autonomy. It enables data owners in our system to fully control their data, and share it autonomously. Specifically, we present a tamper-resistant data capsule encapsulation method, where the data capsule is the minimal unit for independent and secure personal data storage and sharing. Additionally, to realize selective sharing and informed-consent based authorization, we propose a task-driven data sharing mechanism that is resistant to collusion and EDoS attacks. Furthermore, by updating parts of the data capsules, the permissions granted to data consumers can be immediately revoked. Finally, we conduct a security and performance analysis, proving that our scheme is correct, sound, and secure, as well as revealing more advantageous features in practicality, compared with the state-of-the-art schemes.

In security games, the patrolling problem is usually modeled as a Stackelberg game to obtain patrol schemes. However, solving Stackelberg games is challenging, as the player strategy space grows exponentially with patrolling scenarios that expand in space and time. Recent work on reducing the player strategy space does not consider adversarial graph features, making the process of solving the Stackelberg game inefficient. To address this issue, we propose a novel algorithm called “Digger,” and the following important findings are made: the defender's optimal strategy can prevent an attacker from reaching a target, and this is achieved by a set of available interdiction vertices that separate the attacker from the target. The defender can arrive at the interdiction vertices earlier than the attacker. After arriving at the set of interdiction vertices, the next defender action subgraphs are not all useful, and the expected utility of the related player's pure strategy is not optimal. Finally, we build a player support set that does not contain useless strategies to improve the speed of the security game algorithm. An experimental evaluation of warehouse graphs demonstrates that Digger dramatically improves the existing algorithms.

Fine-tuning large-scale pre-trained language models has been demonstrated effective for various natural language processing (NLP) tasks. Previous studies have established that incorporating adversarial training during the fine-tuning stage can significantly enhance model generalization and robustness. However, from the perspective of game theory, such utilizations of adversarial training correspond to pure-strategy games, which are inherently limited in terms of the scope of their strategies, thereby still having room for improvement. In order to push the performance boundaries, we propose a novel Mixed-strategy Adversarial Training algorithm (MAT). Methodologically, we derive the Nash equilibrium of a mixed-strategy game for adversarial training using Entropy Mirror Descent to establish MAT by sampling method. To verify the effectiveness of MAT, we conducted extensive benchmark experiments on large-scale pre-trained models, such as BERT and RoBERTa. MAT significantly outperforms the state-of-the-art methods on both the GLUE and ANLI benchmarks in terms of generalization and robustness.

Abstract As mobile communications, the Internet, databases, distributed computing, and other technologies continue to develop, the Internet of Things (IoT) has emerged as prevalent technique. However, attacks on security and sensitive data in IoT occur frequently, and these attacks often evade intrusion detection systems strategically by mutating their traffic. To prevent security threats and sensitive data leakage, we propose a game approach based on adversarial deep learning to optimize a dynamic security threshold strategy. We introduce a mobile edge computing framework and utilize a game model to describe the adversarial interaction between the two participants. To solve the complexity of the game problem to gain dynamically randomized adversarial attacks, we present a column generation (CG) framework, which uses a feedforward neural network to quantify data flowing through IoT devices. Considering the limited resources of IoT devices, we calculate an optimal response to cyberattacks via a particle swarm optimization algorithm, aiming to reduce the false alarm rate. The adversarial dynamic threshold (ADT)‐based column generation (CG‐ADT) algorithm generates the set of detection threshold and the probability. Finally, we present the results of experiments conducted to demonstrate the effectiveness and robustness of the proposed dynamic threshold scheme for sensitive data security protection in IoT and its suitability for implementation in production systems.

In an autonomous system, rational agents aim to maximize their benefits. Rational agents gradually give up cooperation with others and only request services, which leads to the phenomenon of free-riding. This phenomenon generally exists in the real society and application system. Therefore, how to reduce the behavior, which is similar to free-riding, has become an important research topic. In addition, the traditional pairwise interaction network cannot truly reflect the interaction properties of cooperative events. Social norms provide an effective way to promote cooperation for autonomous systems. Axelrod’s metanorm game model achieves the emergence of social norms by conducting punishment to connivance based on punishing defection. However, links in networks only allow for pairwise interactions. Therefore, we introduce hypergraphs, which are more suitable to reflect group interaction to model metanorm games. The establishment efforts of norms are examined on hypergraphs (uniform random hypergraph (URH), hyperdegree-heterogeneous random hypergraphs, and real-world hypergraphs). We show that the difference in group sizes affects norms emergence of agents and realize the establishment of social norms on URH. To a certain extent, the probability of being seen is positively correlated with agents’ vengefulness and learning time and negatively correlated with boldness. However, the usage of the boldness and vengefulness learning (BV-learning) algorithm on HRHs cannot make norms emergence because leavers cannot participate in learning and the fixed learning step. Therefore, we propose the dynamic relevance BV-learning algorithm to overcome the aforementioned problems, so hub agents and leavers can jointly establish better social norms. We also verify the evolution process of agents’ cooperation rate by BV-learning and the dynamic relevance BV-learning, respectively, which demonstrate that the establishment of social norms can indeed promote agents’ cooperation. Finally, in order to illustrate the universality of the dynamic relevance BV-learning clearly, we study norms establishment on real-world hypergraphs by and compare it with BV-learning. It can be found that the dynamic relevance BV-learning can effectively establish social norms on real-world hypergraphs.

Artificial intelligence (AI) continues demonstrating its positive impact on society and successful adoptions in data-rich domains including social computing systems. There are serious ethical and legal concerns about AI’s ability to make decisions in a responsible way. Many principles and guidelines for responsible AI (RAI) have been issued by governments, research organizations, and enterprises. For instance, the Institute for Ethical Machine Learning provides various RAI resources[1], including higher level guidelines and frameworks, tools, standards, regulations, course, and so on. However, high-level principles are far from ensuring the trustworthiness of AI systems.

The topic of source identification has attracted wide attention from researchers. In practice, the source identification method aims to locate the sources of rumors, computer viruses, and epidemics, such as COVID-19. However, there are two main problems with existing propagation source detection methods. First, most source detection methods are based on infinite networks, not in line with reality. Second, sources are often randomly selected in simulations, but different sources often cause significantly different detection results in real-world applications. To this end, we study how does the source location impact the effectiveness of source detection in finite networks. This paper first proposes a diameter-based node division method to classify the nodes based on their structural location. We further offer different evaluation indicators to measure the effectiveness of source detection methods. Then, we conduct systematic experiments on three synthetic networks and two real-world networks. Our experiments demonstrate that the location of the source directly effects detection effectiveness in finite networks for all source detection methods. Specifically, sources closer to the network boundary will lead to worse detection performance. It means that attackers can choose sources close to the network boundary to reduce the probability of detection to achieve a larger spreading scale.

As one of the most powerful topic models, Latent Dirichlet Allocation (LDA) has been used in a vast range of tasks, including document understanding, information retrieval and peer-reviewer assignment. Despite its tremendous popularity, the security of LDA has rarely been studied. This poses severe risks to security-critical tasks such as sentiment analysis and peer-reviewer assignment that are based on LDA. In this paper, we are interested in knowing whether LDA models are vulnerable to adversarial perturbations of benign document examples during inference time. We formalize the evasion attack to LDA models as an optimization problem and prove it to be NP-hard. We then propose a novel and efficient algorithm, EvaLDA to solve it. We show the effectiveness of EvaLDA via extensive empirical evaluations. For instance, in the NIPS dataset, EvaLDA can averagely promote the rank of a target topic from 10 to around 7 by only replacing 1% of the words with similar words in a victim document. Our work provides significant insights into the power and limitations of evasion attacks to LDA models.

There exist two severe challenges in cloud file-sharing networks: cooperation dilemma and trust dilemma. The mechanism designed to promote cooperation could suffer from malicious users, while the trust management that only considers the trust dilemma is subjected to denial-of-service attacks. To address these two dilemmas simultaneously, we present a dependable cloud file-sharing scheme-EigenCloud. The main contributions include the following. First, we propose a modified EigenTrust algorithm to calculate the global cooperation value and global trust value of each cloud user based on her/his past behaviors. Second, we propose cooperation and trust-aware worker recommendation mechanism by determining a Pareto front from all cloud users. Thus, a cloud user who adopts the recommendation mechanism by paying an additional fee could have a higher probability of receiving a valid file in one transaction. Last but not least, we use the evolutionary game theory (EGT) to study the acceptance and effectiveness of the proposed EigenCloud by strategically modeling cloud users. The Lyapunov stability theory is employed to mathematically investigate the stability of evolutionary equilibriums of our EigenCloud. Finally, both numerical simulations and simulator-driving experiments illustrate that our EigenCloud has an outstanding performance in promoting cooperation and inhibiting malicious activity.

Countermeasure selection is a key process of the Intrusion Response System (IRS). Many cost-sensitive schemes have been proposed to select the optimal countermeasure to maximize security utility by attuning attack damage and response cost. However, existing schemes ignore the interaction between different countermeasures for different attack paths, and neglect the uncertainty between alerts and attacks, which may lead to excessive or insufficient responses. ignore the interaction between different countermeasures for multiple attack paths. To address this problem, in this paper, we propose a combined countermeasures selection scheme based on probabilistic attack tree (PAT). First, we employ Bayesian networks to calculate the probability of each atomic attack in the PAT. Next, the exploitation probability of each attack path is evaluated and multiple possible attack paths are identified. In addition, we quantify the damage of each identified attack path and formulate the countermeasure selection for single attack path as a multi-objective optimization problem. Finally, by considering the security utilities of the countermeasures for different attack paths, we use a greedy strategy to select the combined countermeasures and maximize overall security utility. The experimental results demonstrate the effectiveness of the proposed scheme.

The epidemic containment game is a formulation to describe voluntary vaccination behaviors before epidemic spreading. This game relies on the characterization of the susceptible-infected- susceptible (SIS) model in terms of the spectral radius of the network. Existing researches showed that finding the worst Nash Equilibrium (NE) is NP-hard and used a heuristic algorithm called Low Degree (LDG) to estimate the maximum social cost under the worst NE (Max NE cost). By comparing the results of the LDG algorithm and exhaustive search, we found the LDG algorithm cannot estimate Max NE cost well, thus, we proposed a new neighbor information based algorithm to estimate Max NE cost in this paper. Moreover, we discussed Stackelberg strategies in which some nodes are secured first by a leader, then other agents choose their strategies voluntarily. We found the target (TAR) strategy is effective to reduce Max NE cost in a scale- free network when T is large and useless when T is low (T is the ratio of the recovery rate to the transmission rate in the SIS model). Moreover, we found that a lot of nodes with small degrees are secured voluntarily under the TAR strategy when T is low, which leads to high Max NE cost. At last, we proposed a new greedy algorithm to select nodes secured first, which can reduce Max NE cost when T is low.

Security measurement matters to every stakeholder in network security. It provides security practitioners the exact security awareness. However, most of the works are not applicable to the unknown threat. What is more, existing efforts on security metric mainly focus on the ease of certain attack from a theoretical point of view, ignoring the “likelihood of exploitation.” To help administrator have a better understanding, we analyze the behavior of attackers who exploit the zero-day vulnerabilities and predict their attack timing. Based on the prediction, we propose a method of security measurement. In detail, we compute the optimal attack timing from the perspective of attacker, using a long-term game to estimate the risk of being found and then choose the optimal timing based on the risk and profit. We design a learning strategy to model the information sharing mechanism among multiattackers and use spatial structure to model the long-term process. After calculating the Nash equilibrium for each subgame, we consider the likelihood of being attacked for each node as the security metric result. The experiment results show the efficiency of our approach.

Summary In peer‐to‐peer (P2P) networks, contributions are made by peers voluntarily for the autonomous character of peers. However, selfish peers may refuse to be cooperative when considering their limited transmission resources. Incentive mechanisms are always used to guarantee successful cooperations among peers. Although the inventive mechanisms have been widely investigated on the basis of game theory, most researches assume that peers are well mixed in the network, regardless of the influence of peers' transaction relationships. In this paper, a novel analysis framework based on spatial evolutionary game theory is proposed to verify the effectiveness of incentive mechanisms. In the framework, a transaction overlay network is used to model the transaction relationships of peers. The transactions between clients and servers are modeled as the donor‐recipient game to satisfy their asymmetric characters. Influences of the learning noise and some common behaviors of peers on incentive mechanisms are also considered. Moreover, in order to demonstrate the utility of the framework, a reciprocation‐based incentive mechanism, which considers the requestors' behaviors of providing and consuming services, is thoroughly investigated under the framework in scenarios with homogeneous and heterogeneous benefits of services. By using the framework, besides the effectiveness of incentive mechanisms, the detailed spatiotemporal evolutions of peers' strategies driven by incentive mechanisms can also be obtained. Copyright © 2014 John Wiley & Sons, Ltd.

In peer-to-peer (P2P) network, incentive mechanism is crucial to encourage cooperation among peers. Hence, how to construct a framework to analyze and evaluate the effectiveness of incentive mechanism is a very significant problem. Considering the peers' interactions are influenced by the network structure in real network, we propose a novel framework based on spatial evolutionary game theory. Different from most of other researches based on classical and evolutionary game theory, square lattice network is adopted as the network structure in this paper, without the assumption that peers are well-mixed in P2P network. The square lattice network structure can be easily extended to other realistic complex networks, such as small-world network and scale-free network. The reciprocative incentive mechanism is analyzed and evaluated under the framework with different service benefit. Through the simulation, the range of the parameter Q (cost/benefit) that makes the incentive mechanism work effectively under the framework is got, and the reason is analyzed. In addition, the influences of zero-cost identity and strategy mutation of peers on the incentive mechanism are evaluated. The framework is general to analyze and evaluate the effectiveness of other incentive mechanisms.