MOTIVATION: High-dimensional omics data are typically measured on limited sample sizes, which challenges model-based clustering methods such as Gaussian mixture models (GMMs), often leading to instability and poor generalization under complex mixture structures. To address these limitations, we developed Praxis-BGM, a natural-gradient variational inference framework for GMMs. Praxis-BGM enables semi-supervised transfer learning by incorporating an informative prior GMM estimated from large-scale reference data with robust cluster structures. The prior model can encode cluster-specific means, covariance structures, and structural connectivity patterns, and is updated using the target data with variational inference to improve clustering in small-sample settings. RESULTS: Using the Variational Online Newton (VON) algorithm, we derived natural-gradient updates for the standard parameters of GMMs. Implemented in the Python library JAX for accelerator-oriented computation, Praxis-BGM is computationally efficient and scalable. Across extensive simulations and two real-world applications-breast cancer bulk transcriptomics for subtype recovery and single-cell transcriptomics for cross-platform cell-type label transfer-Praxis-BGM improves posterior clustering performance, stability, and biological interpretability, even when priors are partially mismatched. AVAILABILITY AND IMPLEMENTATION: Praxis-BGM is freely available at https://github.com/ContiLab-usc/Praxis-BGM, and an archival version is available on Zenodo at https://doi.org/10.5281/zenodo.19657680.