Foundation models, which exploit self-supervised learning on vast amounts of data, have demonstrated superior performance in various biomedical domains. However, the differences in existing foundation models, such as pre-training data sources, model architectures, and pre-training tasks, lead to noticeable variations in their performance across diverse biomedical tasks. Therefore, it remains unclear how to select the most appropriate foundation model for a new task. Here, we propose an alternative solution scMoE to integrate multiple foundation models within the same domain through a mixture-of-expert framework. The key idea of scMoE is to selectively activate certain foundation models according to the specific task and input using a multi-head sparse router. scMoE can be further extended to integrate foundation models across modalities, such as integrating multiple single-cell foundation models and multiple molecular foundation models. Our experiments across three biomedical tasks, including cancer drug response prediction, drug combination perturbation prediction, and synergistic drug combination prediction, demonstrate that scMoE significantly outperforms individual foundation models, highlighting its effectiveness and robustness in complex biomedical applications. Overall, scMoE provides a novel integrative framework for utilizing heterogeneous foundation models, paving the path for applying multiple diverse foundation models to new tasks.

MOTIVATION: High-resolution Hi-C contact matrices reveal the detailed three-dimensional architecture of the genome, but high-coverage experimental Hi-C data are expensive to generate. Simultaneously, chromatin structure analyses struggle with extremely sparse contact matrices. To address this problem, computational methods to enhance low-coverage contact matrices have been developed, but existing methods are largely based on resolution enhancement methods for natural images and hence often employ models that do not distinguish between biologically meaningful contacts, such as loops and other stochastic contacts. RESULTS: We present Capricorn, a machine learning model for Hi-C resolution enhancement that incorporates small-scale chromatin features as additional views of the input Hi-C contact matrix and leverages a diffusion probability model backbone to generate a high-coverage matrix. We show that Capricorn outperforms the state of the art in a cross-cell-line setting, improving on existing methods by 17% in mean squared error and 26% in F1 score for chromatin loop identification from the generated high-coverage data. We also demonstrate that Capricorn performs well in the cross-chromosome setting and cross-chromosome, cross-cell-line setting, improving the downstream loop F1 score by 14% relative to existing methods. We further show that our multiview idea can also be used to improve several existing methods, HiCARN and HiCNN, indicating the wide applicability of this approach. Finally, we use DNA sequence to validate discovered loops and find that the fraction of CTCF-supported loops from Capricorn is similar to those identified from the high-coverage data. Capricorn is a powerful Hi-C resolution enhancement method that enables scientists to find chromatin features that cannot be identified in the low-coverage contact matrix. AVAILABILITY AND IMPLEMENTATION: Implementation of Capricorn and source code for reproducing all figures in this paper are available at https://github.com/CHNFTQ/Capricorn.