The Asmuth and Bloom threshold secret sharing (AB-SS) is a classical introduction of the Chinese remainder theorem (CRT) to secret sharing, offering low computational complexity compared to other branches of secret sharing. For decades, numerous schemes have been proposed for practical applications of AB-SS, such as secret image sharing (SIS). However, in terms of security, AB-SS has proved to be neither ideal nor perfect, and its derivatives in image sharing exhibit vulnerabilities associated with secret leakage. This paper studies the security issues in the SIS schemes derived from AB-SS and improves the core sharing principle of AB-SS to enhance security in image protection. First, for$(2,n)$-CRTSIS schemes, we exploit the vulnerability in a single share image to crack the confidential information of the original image, including secret pixel values and the ratio of different pixels. Then, by employing the XOR operation, we introduce a chain obfuscation technology and propose a secure image sharing scheme based on the Chinese remainder theorem (COxor-CRTSIS). The COxor-CRTSIS scheme utilizes integer linear programming for achieving lossless recovery without segmentation and eliminates potential secret disclosure risks without additional encryption. Furthermore, to comprehensively evaluate the security of existing schemes, this paper presents three metrics, information loss rate, fluctuation degree, and coverage rate, enabling a quantitative comparison of security for the first time. Theoretical analyses and experiments are conducted to validate the effectiveness of our scheme.