Terahertz (THz) time-domain spectroscopy gains its popularity in internal defect detection of nonpolar dielectrics, and emerges as a promising inspection technique for delamination defects in composite materials due to its submillimeter level resolution and high penetrability. However, the contrast of THz images decreases significantly with an increasing number of layers due to dramatic signal attenuation and severe multiple reflections, which degrades the temporal and spatial resolution and subsequently hinders the accuracy of defect characterization. In this article, we propose a novel THz defect characterization framework for inspecting ultralayered glass fiber-reinforced polymer composites. The framework reconstructs the laminate structure by utilizing the local symmetry characteristic of the reflected THz pulses and the point cloud density. Then, THz images are extracted from both the reflected time-domain signal and local symmetry function. Pixel-level clustering of THz images is performed for accurate defect characterization. The defect intersection over union for 25 delamination defects in a 31-layer composite can reach more than 0.87. Our proposed strategy offers a promising route for accurate delamination assessment in ultralayered composites, and can be extended to automatic THz characterization in industrial applications.