Cognitive diagnosis aims to infer students' mastery levels over knowledge components from their learning interactions, supporting personalized education applications. However, existing models encode responses as binary correctness labels, discarding information about which specific option a student selected. This input-level information loss limits their ability to distinguish qualitatively different error types. As a result, providing interpretable diagnostic outputs becomes challenging. To address these limitations, we propose SACD, a semantic-augmented cognitive diagnosis framework that integrates LLM-based semantic analysis with student behavioral modeling. SACD comprises the following key components. First, an LLM-based exercise diagnostic generator analyzes exercise content and produces structured semantic annotations for each answer choice, capturing the specific misconceptions each option represents. Second, a kernel-based alignment mechanism projects semantic embeddings and behavioral representations into a unified kernel space, enabling effective fusion of heterogeneous information. Third, an interpretable diagnosis layer predicts student performance and generates fine-grained mastery estimates, which LLMs further process to produce actionable learning plans. Extensive experiments on three real-world datasets demonstrate that SACD achieves superior prediction accuracy while enabling interpretable, actionable diagnostics.