In this paper, we develop a general pessimism reduction framework for design optimization of timing closure. Although the modified graph based timing analysis (mGBA) slack model can be readily formulated into a quadratic programming problem with constraints, the realistic difficulty is the size of the problem. A critical path selection scheme, a uniform sampling method with the sparse characteristics of the optimal solution, and a stochastic conjugate gradient method are proposed to accelerate the optimization solver. This modified GBA is embedded into design optimization of timing closure. Experimental results show that the proposed solver can achieve 13.82x speedup than gradient descent method with similar accuracy. With mGBA, the optimization of timing closure can achieve a better performance on area, leakage power, buffer counts.