Adversarial SQL injection (SQLi) refers to the process in which attackers dynamically modify their attack strategies based on feedback from the target Web Application Firewall (WAF) in an attempt to bypass it. The emergence of automated adversarial SQLi tools in recent years, such as AdvSQLi, RAT, and GPTFuzzer, demonstrates that adversarial SQLi has become a critical method for vulnerability detection and SQLi execution. These tools' intelligent nature poses a significant threat to existing SQLi defense mechanisms. Payloads subjected to adversarial mutation can effectively deceive machine learning-based WAFs, while rule-based WAFs are more susceptible to having vulnerabilities discovered. To address this issue, we propose a hybrid model based on enhanced Character-Level CNN (CLCNN) and Reinforcement Learning (RL), which detects malicious patterns through multi-scale feature extraction by CLCNN from a pattern-matching perspective. Additionally, the reinforcement learning module is employed for adversarial training, executing de-obfuscation operations on payloads for which the CLCNN outputs a low confidence level, further processing ambiguous payloads. The experimental results indicate that even without a priori knowledge, CLCNN demonstrates strong resistance to such tools. And the RL module can further enhance Recall through adversarial training, with minimal reduction in the hybrid model's performance on standard SQLi datasets.