We evaluate how well large language model embeddings represent continuous numerical values across different precisions and ranges. Using linear models and principal component analysis on models from major providers, we show that while embeddings can reconstruct numbers with high fidelity (R2 ≥ 0.95), they introduce substantial noise, with principal components explaining less than 40% of embedding variance. Performance degrades with increasing decimal precision and mixed-sign values, revealing fundamental limitations in how these models encode numerical information.

The principal benefit of unsupervised representation learning is that a pre-trained model can be fine-tuned where data or labels are scarce. Existing approaches for graph representation learning are domain specific, maintaining consistent node and edge features across the pre-training and target datasets. This has precluded transfer to multiple domains. We present Topology Only Pre-Training (ToP), a graph pre-training method based on node and edge feature exclusion. We show positive transfer on evaluation datasets from multiple domains, including domains not present in pre-training data, running directly contrary to assumptions made in contemporary works. On 75% of experiments, ToP models perform significantly ([Formula: see text]) better than a supervised baseline. Performance is significantly positive on 85.7% of tasks when node and edge features are used in fine-tuning. We further show that out-of-domain topologies can produce more useful pre-training than in-domain. Under ToP we show better transfer from non-molecule pre-training, compared to molecule pre-training, on 79% of molecular benchmarks. Against the limited set of other generalist graph models ToP performs strongly, including against models with many orders of magnitude larger. These findings show that ToP opens broad areas of research in both transfer learning on scarcely populated graph domains and in graph foundation models. Supplementary Information: The online version contains supplementary material available at 10.1007/s10618-026-01210-1.

This doctoral dissertation establishes and addresses frontiers in graph generation. I first apply a Graph Neural Network (GNN) model on social network data, a new domain, to establish what frontiers exist for graph generators. I establish that GNN models are currently limited in the diversity of feature sets that they can produce, the variety of graph structure types they can generate, and highly limited in the size of generated graphs. Further, I find that the quality metrics available for graph generation are aggregate-based and un-expressive. To address the issue of scale I propose Hierarchical Generation of Graphs (HiGGs), a framework for producing graphs orders of magnitude larger than is possible with a single model. As a step towards more expressive metrics I develop Topology only Pre-training (ToP), a pre-training framework for graph models that is capable of representing multiple domains of graphs simultaneously, without relying on tertiary models in downstream applications. The next stage of research will adapt ToP as a model based metric for graph generators.