Kazakh image-text retrieval is a challenging task with no dedicated research to date. Although existing multilingual vision-language pretraining models provide limited support for aligning Kazakh text with images, their performance remains poor due to the scarcity of annotated Kazakh resources and the complex expression patterns arising from its agglutinative linguistic nature, which hinder accurate modeling of text-image alignment. To address these challenges, we propose a new Kazakh image-text retrieval framework that integrates Semantic-Agnostic Feature Enhancement and Adaptive Knowledge Transfer (SAFE-AKT). SAFE employs an adversarial training strategy to generate semantic-agnostic features from Kazakh texts (e.g., expression patterns) and incorporates them into the kazakh text encoding process, enhancing the model’s robustness to diverse Kazakh expressions. AKT estimates sample-level alignment confidence by computing the entropy of the teacher distribution and weights the KL divergence loss accordingly, enabling effective transfer of high-quality English image-text alignment knowledge to the Kazakh representation space while mitigating overfitting to low-confidence samples, thereby improving Kazakh image-text alignment. Meanwhile, to address the scarcity of Kazakh data, we construct the first Kazakh image-text dataset, Flickr30k-kaz, based on machine translation and manual refinement. Experimental results on the Flickr30k-kaz and the Multi30K benchmark demonstrate that our method significantly outperforms existing state-of-the-art approaches and achieves new best performance.