Advanced Persistent Threats (APTs) have become one of the most prominent cybersecurity risks globally. The external network-facing (ENF) services (e.g., e-commerce platforms) within a system are particularly vulnerable, as they are directly exposed to the Internet and often serve as the primary targets for attackers. By deploying deception resources to protect these ENF services, defenders can detect threats early, block potential attacks, and enhance overall system resilience. However, most existing studies on cyber deception strategies assume simultaneous moves by both attacker and defender. Furthermore, few works have considered the evolution of the system state resulting from APT attacks on the ENF services. To address these limitations, this paper proposes a Cyber Deception Stackelberg Markov Game (CDSMG) for protecting ENF services, which dynamically captures state transitions and accurately characterizes the strategic interactions between defenders and APT attackers. In CDSMG, the defender acts as the leader, who proactively selects a subset of services to deploy the deception resources based on the current system state, while the APT attacker plays as the follower, making a best response which incorporates the defender’s policy into its own strategy. To overcome the challenge of the combinatorial optimization problem of selecting a subset of services, we propose a revised version of the PPO algorithm by using no-replacement sampling to select multiple services at once, thereby significantly reducing the action space size. Finally, experimental results demonstrate that our approach effectively defends against APT attacks. It not only outperforms several baseline methods but also exhibits better scalability and robustness under varied model parameter settings.

In Internet of Things (IoT) environments, anti-honeypot attacks exploit device fingerprinting and protocol analysis to undermine traditional honeypot defenses. To this end, we introduce an innovative camouflage defense strategy that dynamically disguises real IoT devices as honeypots to mislead attackers and protect critical resources. We formalize this adversarial interaction using a multi-stage signaling game, thereby reversing the information asymmetry that typically favors attackers. Subsequently, we theoretically derive the Perfect Bayesian Nash Equilibrium (PBNE) under conditions of incomplete information, thereby confirming the existence of mixed-strategy equilibria. Practically, we develop a signaling game-based multi-agent proximal policy optimization (SG-MAPPO) algorithm by integrating LSTM networks and Bayesian updates, overcoming challenges in long-term strategy optimization and parameter convergence in dynamic IoT contexts. Experiments demonstrate that SG-MAPPO outperforms existing methods in terms of training efficiency and stability, effectively reducing attacker success rates and providing a proactive defense solution for IoT security.

As a key technology in the Industrial Internet of Things (IIoT), the combination of digital twins (DTs) and mobile-edge computing (MEC) facilitates edge intelligence in B5G, while the application of generative artificial intelligence (GenAI) further enhances edge intelligence in resource allocation. However, the DT-enabled MEC system in IIoT faces the challenges of the low latency and high reliability demands, especially with the massive increase in machine-type communication devices and limited wireless and computing resources. To reduce overall delay and ensure high-reliability communication among machine devices (MDs), we propose an optimization problem for joint wireless and MEC computation resource allocation (JWMC-RA), which is shown to be NP-hard and intractable. Thereafter, the original problem is decomposed into two stages, i.e., machine-to-machine (M2M) links clustering, and MEC computing resource management. In the step of M2M links clustering, a heuristic clustering scheme via spectrum radius (HCS-SR) is presented for reducing interference of MDs which uses GenAI and graph theory. In the step of MEC computing resource management, the initial optimization problem is transformed to a convex problem, then, the optimal task offloading ratios of MD and MEC computing resource allocation are obtained. Finally, simulations show that the JWMC-RA scheme can reduce the overall delay and ensure the communication reliability.

Many real-world scenarios including fleet management and Ad auctions can be modeled as Stackelberg mean-field games (SMFGs) where a leader aims to incentivize a large number of homogeneous self-interested followers to maximize her utility. Existing works focus on cases with a small number of heterogeneous followers, e.g., 5-10, and suffer from scalability issue when the number of followers increases. There are three major challenges in solving large-scale SMFGs: i) classical methods based on solving differential equations fail as they require exact dynamics parameters, ii) learning by interacting with environment is data-inefficient, and iii) complex interaction between the leader and followers makes the learning performance unstable. We address these challenges through transition-informed reinforcement learning. Our main contributions are threefold: i) we first propose an RL framework, the Stackelberg mean-field update, to learn the leader's policy without priors of the environment, ii) to improve the data efficiency and accelerate the learning process, we then propose the Transition-Informed Reinforcement Learning (TIRL) by leveraging the instantiated empirical Fokker-Planck equation, and iii) we develop a regularized TIRL by employing various regularizers to alleviate the sensitivity of the learning performance to the initialization of the leader's policy. Extensive experiments on fleet management and food gathering demonstrate that our approach can scale up to 100,000 followers and significantly outperform existing baselines.

Decision-making problems, categorized as single-agent, e.g., Atari, cooperative multi-agent, e.g., Hanabi, competitive multi-agent, e.g., Hold'em poker, and mixed cooperative and competitive, e.g., football, are ubiquitous in the real world. Although various methods have been proposed to address the specific decision-making categories, these methods typically evolve independently and cannot generalize to other categories. Therefore, a fundamental question for decision-making is: *Can we develop **a single algorithm** to tackle **ALL** categories of decision-making problems?* There are several main challenges to address this question: i) different decision-making categories involve different numbers of agents and different relationships between agents, ii) different categories have different solution concepts and evaluation measures, and iii) there lacks a comprehensive benchmark covering all the categories. This work presents a preliminary attempt to address the question with three main contributions. i) We propose the generalized mirror descent (GMD), a generalization of MD variants, which considers multiple historical policies and works with a broader class of Bregman divergences. ii) We propose the configurable mirror descent (CMD) where a meta-controller is introduced to dynamically adjust the hyper-parameters in GMD conditional on the evaluation measures. iii) We construct the GameBench with 15 academic-friendly games across different decision-making categories. Extensive experiments demonstrate that CMD achieves empirically competitive or better outcomes compared to baselines while providing the capability of exploring diverse dimensions of decision making.

Advanced persistent threat (APT) for cyber espionage poses a great threat to modern organizations. In order to mitigate the impact of APT on an organization, all the compromised systems in the organization must be quarantined and recovered in a timely and effective way. This article focuses on the problem of customizing a dynamic quarantine and recovery (QAR) scheme for an organization so that the APT impact is minimized. Based on a novel node-level epidemic model characterizing the effect of the QAR scheme on the expected state of the underlying network, we estimate the expected impact of APT under a QAR scheme. On this basis, we model the original problem as an optimal control problem. By use of optimal control theory, we derive the optimality system for the optimal control problem and thereby introduce the concept of normal potential optimal (NPO) control. Next, through comparative experiments, we find that the NPO control outperforms a set of heuristic controls. Hence, the QAR scheme associated with the NPO control is satisfactory in terms of the effectiveness of defending against APT. Finally, we examine the effect of some factors on the expected APT impact under the NPO control. This article would be helpful to the defense against APT for cyber espionage.

The advanced persistent threat (APT) as a new kind of cyber attack has posed a severe threat to modern organizations. When the APT has been detected, the organization has to deal with the APT response problem, i.e., to allocate the available response resources to fix her insecure hosts so as to mitigate her potential loss. This paper addresses the APT response problem by using the risk management approach. First, we introduce a model characterizing the evolution of the organization's expected state. By analyzing this model, we find the organization's expected state approaches a common limit expected state. Then, we use the organization's expected loss per unit time to measure her potential loss, and we find this measure is determined by the organization's limit expected state. On this basis, we model the APT response problem as a game-theoretic problem (the APT response game) in which the organization seeks a Nash equilibrium. We present a greedy algorithm for solving the game. Comparative experiments show that the algorithm is effective. Therefore, we recommend the response strategy generated by performing the algorithm. These findings contribute to defending against the APT. To our knowledge, this is the first time the APT response problem is addressed.

To restrain escalating computer viruses, new virus patches must be constantly injected into networks. In this scenario, the patch-developing cost should be balanced against the negative impact of virus. This article focuses on seeking best-balanced patch-injecting strategies. First, based on a novel virus-patch interactive model, the original problem is reduced to an optimal control problem, in which (a) each admissible control stands for a feasible patch-injecting strategy and (b) the objective functional measures the balance of a feasible patch-injecting strategy. Second, the solvability of the optimal control problem is proved, and the optimality system for solving the problem is derived. Next, a few best-balanced patch-injecting strategies are presented by solving the corresponding optimality systems. Finally, the effects of some factors on the best balance of a patch-injecting strategy are examined. Our results will be helpful in defending against virus attacks in a cost-effective way.

Advanced persistent threat (APT) is a new kind of cyberattack that poses a serious threat to modern society. When an APT campaign on an organization has been identified, the available repair resources must be reasonably allocated to the potentially insecure hosts to mitigate the potential loss of the organization. We refer to the feasible repair resource allocation strategies as repair strategies. This paper focuses on the APT repair problem, i.e., the problem of developing effective repair strategies for organizations. First, for an organization with time-varying communication relationship, we establish an evolution model of the organization's expected state, in which the impact of lateral movement of APT is accommodated. On this basis, we model the APT repair problem as a differential Nash game problem (the APT repair game) in which the attacker attempts to maximize his potential benefit, and the organization manages to minimize its potential loss. Second, we derive a system (the potential system) for calculating a potential Nash equilibrium of an APT repair game, and we examine the structure of the potential attack and repair strategies in a potential Nash equilibrium. Next, we solve some potential systems to get the corresponding potential Nash equilibria. Finally, by comparison with a large number of randomly generated attack and repair strategies, we conclude that the potential Nash equilibrium of each APT repair game is a Nash equilibrium of the game. Therefore, we recommend to organizations their respective potential repair strategies. Our findings help to better understand and effectively defend against APT.

The new cyberattack pattern of advanced persistent threat (APT) has posed a serious threat to modern society. This paper addresses the APT defense problem, that is, the problem of how to effectively defend against an APT campaign. Based on a novel APT attack-defense model, the effectiveness of an APT defense strategy is quantified. Thereby, the APT defense problem is modeled as an optimal control problem, in which an optimal control stands for a most effective APT defense strategy. The existence of an optimal control is proved, and an optimality system is derived. Consequently, an optimal control can be figured out by solving the optimality system. Some examples of the optimal control are given. Finally, the influence of some factors on the effectiveness of an optimal control is examined through computer experiments. These findings help organizations to work out policies of defending against APTs.