Reinforcement learning with verifiable rewards (RLVR) typically evaluates only final outcomes, providing limited learning signal about whether the generated reasoning is consistent with the correct answer.As a result, even when groundtruth answers are available during training, onpolicy rollouts can repeatedly produce reasoning that is inconsistent with the answer.We propose Answer-Guided Group Relative Policy Optimization (AG-GRPO) for masked diffusion language models (dLLMs), which generate text through iterative masked-token restoration.AG-GRPO combines standard answerfree (AF) rollouts, sampled without access to the ground-truth answer, with answer-guided (AG) rollouts.In AG rollouts, the model generates reasoning conditioned on an anchored ground-truth answer suffix, and then re-predicts the answer from the generated reasoning for reward computation.We compute group-relative advantages over the combined AF/AG rollout set, allowing answer-guided training signals to improve the answer-free policy used at test time.Across mathematics, puzzle-solving, and code-generation benchmarks, AG-GRPO consistently improves over the pretrained dLLM and prior RL method for masked dLLMs.We further analyze optimization dynamics to study how shared group-relative advantages support signal transfer and affect convergence.Our code is available at https://github.com/ JuHyng/ag_grpo.