This paper proposes an optimized H.264 Intra Prediction arithmetic to reduce the time required to complete the intra prediction by saving ninety percent waste time during the process of intra prediction. Firstly, the 16×16 luminance prediction can be decomposed equally into 16 parts and inserted into the same macro-block's 4×4 luminance prediction to save the 16×16 luminance prediction time in one macro-block's luminance prediction. Then, by means of changing the computation sequence of 4×4 luminance prediction, a large amount of waiting time between 4×4 blocks' luminance predictions can be further removed. Almost all the waste time can be reused or eliminated by these optimizations. At the last part of this paper, an optimized hardware implementation aiming at fitting to the whole H.264 video coding system and a method to realize 16×16 luminance DC prediction on it are proposed.