Large language models (LLMs) have advanced information extraction (IE) by enabling zeroshot and few-shot named entity recognition (NER), yet their generative outputs still show persistent and systematic errors.Despite progress through instruction fine-tuning, zeroshot NER still lags far behind supervised systems.These recurring errors mirror inconsistencies observed in early-stage human annotation processes that resolve disagreements through pilot annotation.Motivated by this analogy, we introduce DiZiNER (Disagreementguided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition), a framework that simulates the pilot annotation process, employing LLMs to act as both annotators and supervisors.Multiple heterogeneous LLMs annotate shared texts, and a supervisor model analyzes intermodel disagreements to refine task instructions.Across 18 benchmarks, DiZiNER achieves zero-shot SOTA results on 14 datasets, improving prior bests by +8.0 F1 and reducing the zero-shot to supervised gap by over +11 points.It also consistently outperforms its supervisor, GPT-5 mini, indicating that improvements stem from disagreement-guided instruction refinement rather than model capacity.Pairwise agreement between models shows a strong correlation with NER performance, further supporting this finding.1