Despite their impressive capabilities, large language models (LLMs) remain highly susceptible to adversarial perturbations, often producing misleading or inconsistent outputs, thus undermining their reliability. In this paper, we propose CVshield, a two-stage training-free interpretable black-box defense framework for LLMs against adversarial perturbations. Under the framework of CVshield, we first design a set of Chain-Of-Thought (CoT) templates tailored to different types of adversarial perturbations, enabling the LLM to analyze and reconstruct potentially corrupted inputs from multiple semantic perspectives. We then propose a semantic alignment verification (SAV) module to evaluate semantic consistency across the LLM outputs generated by the CoT templates and the original LLM output. Significant discrepancies indicate the likely presence of a specific type of adversarial perturbation corresponding to the applied CoT template. To precisely identify the type of disturbance, we further introduce a decision mechanism based on Bayes’ theorem, which integrates evidence from the SAV comparison process to make a robust and probabilistically sound detection decision.We evaluated CVshield on LLaMA2-7B-Chat, Qwen-1.5B, and Mistral-7B models. CVshield successfully reduces the average attack success rate (ASR) to 24.67% in three datasets (MIX, SQuAD, and MedQuAD), significantly outperforming baseline methods such as perplexity (58.80%), tokenization (47.07%) and SmoothLLM (58.76%). Meanwhile, for the identification of attack type, CVshield achieved an average precision of 78.16% on the three datasets, substantially exceeding Perplexity (49.66%), Retokenization (30.65%) and SmoothLLM (38.34%).