The increasing deployment of complex "black box" AI models in anomaly-based Intrusion Detection Systems (IDS) for future networks has opened up a trust gap that requires human-interpretable explanations in order for analysts to feel confident in acting on alerts. Current approaches to Explainable AI (XAI), such as SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations), do not properly address the challenges inherent in the problem domain. These techniques fundamentally fail from a fidelity standpoint due to their incorrect assumption of independence between features that results in untrustworthy explanations, which are fundamentally based on correlated network data. We address these shortcomings by proposing HiFi-XAI, which leverages a new, novel framework to provide faithful and semantically rich explanations. HiFi-XAI introduces a model-agnostic Conditional Value Attribution Explanation (CVAE), a method based on probabilistic Shapley values that models feature dependencies to ensure explanations are derived from plausible data distributions. These high-fidelity attributions are then translated into actionable, natural-language narratives by a fine-tuned Large Language Model (LLM). We validate our framework through allaware scenario feature ablation studies on the CICIDS2017 and CICIOT2023 datasets. This demonstrates that CVAE consistently identifies more impactful features than SHAP and LIME across five anomaly-based IDS models. Furthermore, we deploy the HiFi-XAI to prove its practical feasibility and test it on a resource-constrained Raspberry Pi 4. Our work presents a complete, end-to-end solution for building trust in AI-driven IDS.