Personalized exercise recommendation systems aim to enhance learning efficiency by dynamically guiding students toward content aligned with their evolving knowledge states. Among various approaches, Reinforcement Learning (RL) has emerged as an effective framework for modeling student-environment interactions as sequential decision-making processes. However, most existing RL-based methods typically reward recommendations that target unmastered knowledge concepts.Such reward-driven strategies are prone to local optima, restricting exploration of unattempted or less familiar knowledge concepts. In addition, they often overlook key learning factors, such as forgetting dynamics and exercise difficulty, leading to suboptimal outcomes. To address these issues, we propose a novelIntegratingEntropyRegulation andDual-objectiveOptimizationExerciseRecommendation (IERDO-ER)method. Specifically, we introduce an entropy-based function to encourage broader exploration in the exercise space. We also design an end-to-end policy network that generates candidate exercises as next-step actions for the recommendation agent. To enable more adaptive and pedagogically sound recommendations, we develop a dual-objective reward mechanism that incorporates both anti-forgetting and gentleness objectives. This mechanism continuously balances policy optimization through a synergy of rewards and penalties. Experiments on three real-world educational datasets show that our approach consistently outperforms strong baselines, validating its effectiveness.