Suboptimal design of power delivery networks (PDNs) may cause performance deterioration and severe functional failures on high-speed computer platforms. Voltage regulators (VRs) distribute controlled voltage in the PDN to the active devices, providing a steady power supply at a desired DC voltage level with an acceptable noise level or ripple. Unacceptable voltage drops can be caused by transient switching currents at the devices. Many decoupling capacitors are commonly used to lower the PDN impedance profile in order to reduce power supply noise and to supply fast transient current to switching devices. However, commercially available decoupling capacitors typically present large manufacturing variability. In this article, we first propose an optimization methodology that gradually finds the best compensation parameter values of a buck converter VR to meet suitable stability criteria. Simultaneously, the number of parallel decoupling capacitors in the PDN is minimized while meeting a frequency-domain impedance profile specification and a time-domain minimum voltage droop requirement under nominal parameter values. Finally, a statistical analysis, yield estimation, and yield optimization of the nominally optimized PDN subject to large decoupling capacitor tolerances is presented. We consider the impedance profile, transient voltage droop, and VR stability as the responses of interest for yield calculation.