The Preconditioned Conjugate Gradient method is one of the most important solvers in linear algebra system, and is widely used in scientific and engineering computing applications. Based on the Sunway heterogeneous many-core architecture, we propose a Coupled Incomplete Cholesky and Jacobi preconditioner (CICJ). The preconditioner applies a block Jacobi method to the matrix inversion in preconditioning process, localizes matrix inversions and completely eliminates the data correlation on slave cores. It strikes a better trade-off between convergence and parallelism than other preconditioners on the Sunway heterogeneous many-core architecture. Besides, a two-level software-controlled cache is designed for sparse matrix-vector multiplication operations, which makes full use of the Sunway heterogeneous many-core architecture. We apply our CICJ method on Intel, GPU, and Sunway, and the results show great generality on all three architectures. We also conduct experiments on the underwater submarine models using the open-source framework OpenFOAM. The results show that when the matrix column size is 0.82 billion and the number of non-zero values is 59 billion, our method accelerates the whole algorithm by 8.42 times compared with the diagonal incomplete Cholesky preconditioner (DIC) and 6.5 times compared with the geometric algebraic multi-grid preconditioner (GAMG) using 133,120 processors on the Sunway system.