Static analyzers are vital to ensure software quality, but often produce false alarms. In this paper, we focus on the challenging task, directly refining defective static detection rules in the analyzer with Large Language Models to mitigate false positives/negatives fundamentally. This paper introduces RuleRefiner, a novel multi-stage framework for static analyzer rule refinement. Specifically, RuleRefiner systematically employs LLMs by integrating dynamic profiling information for fact-based rule-code alignment, performing differential fault localization to accurately pinpoint error sources, and utilizing targeted templates to guide and constrain LLM-based modifications for precise and minimally disruptive enhancements. Evaluated on 218 real-world refinement tasks, RuleRefiner achieved a pass@5 score of 80.28%, significantly outperforming all selected LLM-based baselines under the same settings. Moreover, the rules refined by RuleRefiner demonstrated high generalization capability comparable to those written by human experts.