Weakly Supervised Object Detection (WSOD) is fundamentally limited by instance ambiguity, manifesting as either part domination (focusing on discriminative fragments) or merged detection (confusing objects with context). Unlike existing approaches that rely on object-level patterns, we draw inspiration from human cognition, where objects are perceived structurally by integrating constituent parts. Building on this perspective, we propose P2WDet (Part-to-Whole Detection), a novel framework that shifts WSOD from traditional instance selection to semantic reconstruction. P2WDet comprises three systematic stages: (1) constructing a comprehensive visually-grounded part vocabulary leveraging large language models; (2) training dedicated part detectors by mining consistent patterns from cross-image proposal clusters; and (3) assembling detected parts into complete objects to enforce structural consistency. Extensive experiments show that P2WDet not only outperforms state-of-the-art methods on standard benchmarks but also demonstrates superior generalization in open-world settings. Furthermore, by decoupling detection from fixed category labels, P2WDet enables the flexible detection of novel or ambiguous objects defined solely by their components.