In this paper we propose an optimized bitstream compression algorithm based on LZ and a novel architecture of decompressor, the proposed algorithm improves the Compression Ratio by fully utilizing the regularity of configuration bits of CLB (Configurable Logic Box) in FPGA and using the variable length Golomb coding method. The experimental results show that the Optimized method can improve the Compression Ratio of LZSS by 32.3% for bitstream with high regularity and 10.3% for bitstream with low regularity, and our approach shows a higher flexibility than the BMC+RLE arithmetic when compressing the bitstream with high regularity for various FPGA. Moreover, we design a two-buffer-window decompressor to download the compressed bitstreams. In order to increase the throughput of the proposed decompressor, we design a multi-stage data selector in it. The post-simulation of the decompressor shows that its throughput is up to 9280 Mbps under 65nm CMOS process. And that is 4352Mbps when verified on a Virtex-5 FPGA.