Surrogate-assisted evolutionary algorithms (SAEAs) are well-suited for optimizing computationally expensive black-box problems in diverse real-world scenarios. The sample efficiency of SAEAs depends largely on the configuration of the surrogate model and sampling criteria. However, configuring these core components requires substantial manual effort and expert knowledge, limiting the broader applicability of SAEAs. To address these challenges, we propose CoE-SAEA, a novel paradigm that co-evolves large language models (LLMs) and configuration strategies to enhance SAEAs. Specifically, the paradigm consists of three populations with distinct roles: one evolves LLM prompts to generate robust configuration strategy instructions, another optimizes the configuration strategies, and the third solves the optimization problem using the selected algorithm configuration. Additionally, an exploration-exploitation module is incorporated to decide whether to explore new configuration strategies via LLMs or exploit existing ones. We empirically validate the efficacy of CoE-SAEA by comparing it to state-of-the-art algorithms across various benchmark problems and a real-world traffic signal optimization task. The source code of the proposed CoE-SAEA is publicly available at: https://github.com/ForrestXie9/CoE-SAEA.