SUMMARY: Multimodal approaches are increasingly leveraged for integrating omics data with textual biological knowledge. Yet there is still no accessible, standardized framework that enables systematic comparison of omics representations with different text encoders within a unified workflow. We present mmContext, a lightweight and extensible multimodal embedding framework built on top of the open-source Sentence Transformers library. The software allows researchers to train or apply models that jointly embed omics and text data using any numeric representation stored in an AnnData.obsm layer and any text encoder available in Hugging Face. mmContext supports integration of diverse biological text sources and provides pipelines for training, evaluation, and data preparation. We train and evaluate models for a RNA-Seq and text integration task, and demonstrate their utility through zero-shot classification of cell types and diseases across four independent datasets. By releasing all models, datasets, and tutorials openly, mmContext enables reproducible and accessible multimodal learning for omics-text integration. AVAILABILITY AND IMPLEMENTATION: Pretrained checkpoints and full source code for our custom MMContextEncoder are available on Hugging Face huggingface.co/jo-mengr. The Python package github.com/mengerj/mmcontext provides the model implementation and training and evaluation scripts for custom training. The releases for the publication can be accessed via zenodo: adata_hf_datasets: doi.org/10.5281/zenodo.19185217 and mmContext: doi.org/10.5281/zenodo.19185493.

There are various dimensionality reduction techniques for visually inspecting dynamical patterns in time-series single-cell RNA-sequencing (scRNA-seq) data. However, the lack of one-to-one correspondence between cells across time points makes it difficult to uniquely uncover temporal structure in a low-dimensional manifold. The use of different techniques may thus lead to discrepancies in the representation of dynamical patterns. However, The extent of these discrepancies remains unclear. To investigate this, we propose an approach for reasoning about such discrepancies based on synthetic time-series scRNA-seq data generated by variational autoencoders. The synthetic dynamical patterns induced in a low-dimensional manifold reflect biologically plausible temporal patterns, such as dividing cell clusters during a differentiation process. We consider manifolds from different dimensionality reduction techniques, such as principal component analysis, t-distributed stochastic neighbor embedding, uniform manifold approximation, and projection and single-cell variational inference. We illustrate how the proposed approach allows for reasoning about to what extent low-dimensional manifolds, obtained from different techniques, can capture different dynamical patterns. None of these techniques was found to be consistently superior and the results indicate that they may not reliably represent dynamics when used in isolation, underscoring the need to compare multiple perspectives. Thus, the proposed synthetic dynamical pattern approach provides a foundation for guiding future methods development to detect complex patterns in time-series scRNA-seq data.

MOTIVATION: Following many successful applications to image data, deep learning is now also increasingly considered for omics data. In particular, generative deep learning not only provides competitive prediction performance, but also allows for uncovering structure by generating synthetic samples. However, exploration and visualization is not as straightforward as with image applications. RESULTS: We demonstrate how log-linear models, fitted to the generated, synthetic data can be used to extract patterns from omics data, learned by deep generative techniques. Specifically, interactions between latent representations learned by the approaches and generated synthetic data are used to determine sets of joint patterns. Distances of patterns with respect to the distribution of latent representations are then visualized in low-dimensional coordinate systems, e.g. for monitoring training progress. This is illustrated with simulated data and subsequently with cortical single-cell gene expression data. Using different kinds of deep generative techniques, specifically variational autoencoders and deep Boltzmann machines, the proposed approach highlights how the techniques uncover underlying structure. It facilitates the real-world use of such generative deep learning techniques to gain biological insights from omics data. AVAILABILITY AND IMPLEMENTATION: The code for the approach as well as an accompanying Jupyter notebook, which illustrates the application of our approach, is available via the GitHub repository: https://github.com/ssehztirom/Exploring-generative-deep-learning-for-omics-data-by-using-log-linear-models. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.