One of the key challenges of current online social platforms is predicting the size of information cascades, also known as popularity prediction or cascade prediction. Accurate popularity prediction can benefit various fields, including news distribution, market decisions, and rumor detection. However, existing popularity prediction approaches concentrate more on the historical sequences of single messages, overlooking the interactions between message diffusion and the dynamic nature of social networks, which limits the timeliness and accuracy of predictions. To address this, we propose an online popularity prediction service based on minimal substitution reinforcement learning calledMSRL. Specifically, we explore a substitution theory and design a minimal substitution reinforcement learning method that models diffusion as message substitution and considers mutual information diffusion. That helps the model gain a broader perspective, allowing it to fully exploit the cooperative, competitive, or dependent relationships between information diffusions. Furthermore, the reinforcement learning scheme enables the service to dynamically adjust its parameters to respond to the dynamic social network environment in real-time. Finally, extensive experiments on real-world datasets show that the MSRL outperforms state-of-the-art methods regarding accuracy and service agility.