Strategic mining attacks, such as selfish mining, exploit blockchain consensus protocols by deviating from honest behavior to maximize rewards. Markov Decision Process (MDP) analysis faces scalability challenges in modern digital economics, including blockchain. To address these limitations, reinforcement learning (RL) provides a scalable alternative, enabling adaptive strategy optimization in complex dynamic environments. In this survey, we examine RL’s role in strategic mining analysis, comparing it to MDP-based approaches. We begin by reviewing foundational MDP models and their limitations, before exploring RL frameworks that can learn near-optimal strategies across various protocols. Building on this analysis, we compare RL techniques and their effectiveness in deriving security thresholds, such as the minimum attacker power required for profitable attacks. Expanding the discussion further, we classify consensus protocols and propose open challenges, such as multi-agent dynamics and real-world validation. This survey highlights the potential of reinforcement learning to address the challenges of selfish mining, including protocol design, threat detection, and security analysis, while offering a strategic roadmap for researchers in decentralized systems and AI-driven analytics.

Proof-of-Stake (PoS) is an energy-efficient consensus where proposer election based on validator stakes causes centralization, especially in stake pools. However, existing reward allocation mechanisms aim to address centralization but increase the risk of Sybil attacks. Therefore, designing a reward allocation mechanism for stake pools that reconciles decentralization and Sybil resilience remains a key challenge. In this paper, we propose Dream-SR, a reward allocation mechanism established with formal function properties. Dream-SR enhances decentralization by imposing reward constraints to limit the dominance of large stake pools. Sybil resilience is ensured by aligning reward allocation with both the economic incentives and the influence, effectively addressing utility-maximizing and influence-maximizing Sybil attacks. Furthermore, we leverage a multi-leader-multi-follower (MLMF) Stackelberg game model to capture the interactions between validators and users regarding commission pricing and stake delegation within stake pools. The game model is used to analyze the impact of the reward allocation mechanism on the strategies of participants and the system equilibrium. Compared with other relevant mechanisms, numerical results reveal that Dream-SR effectively improves decentralization and resilience to Sybil attacks at equilibrium.