Large Language Models (LLMs) have demonstrated strong capabilities in various natural language processing tasks; however, their application to graph-related problems remains limited, primarily due to scalability constraints and the absence of dedicated mechanisms for processing graph structures. Existing approaches predominantly integrate LLMs with Graph Neural Networks (GNNs), using GNNs as feature encoders or auxiliary components. However, directly encoding graph structures within LLMs has been underexplored, particularly in the context of large-scale graphs where token limitations hinder effective representation. To address these challenges, we propose SDM-InstructGLM, a novel instruction-tuned Graph Language Model (InstructGLM) framework that enhances scalability and efficiency without relying on GNNs. Our method introduces a similarity-degree-based biased random walk mechanism, which selectively samples and encodes graph information based on node-feature similarity and degree centrality, ensuring an adaptive and structured representation within the LLM. This approach significantly improves token efficiency, mitigates information loss due to random sampling, and enhances performance on graph-based tasks such as node classification and link prediction. Furthermore, our results demonstrate the feasibility of LLM-only graph processing, enabling scalable and interpretable Graph Language Models (GLMs) optimized through instruction-based fine-tuning. This work paves the way for GNN-free approaches to graph learning, leveraging LLMs as standalone graph reasoning models. Our source code is available on GitHub1.

Modern large-scale computing clusters face scalability challenges with traditional DRAM-based memory systems due to issues like increasing cell leakage current and reduced reliability. To overcome these limitations, alternative memory solutions have emerged, including 3D-stacked DRAM and emerging non-volatile memory (NVM) technologies. However, these alternatives are unlikely to fully replace DRAM due to capacity constraints and higher cost-per-bit. Hybrid memory systems, combining DRAM with NVM technologies, offer a cost-effective solution by leveraging the strengths of both memory types. Effective data placement decisions are crucial for optimizing hybrid memory systems. This paper introduces MemFlex, an machine learning (ML)-driven approach for migrating pages between memory tiers in a hybrid memory system based on predicted page lifetimes. MemFlex utilizes application-specific ML models to predict death-time ranges with high accuracy, guiding placement decisions to the appropriate storage tier. Evaluation results demonstrate MemFlex's superiority over state-of-the-art techniques, achieving an average performance improvement of 19% over evaluated baselines, with minimal performance degradation. This paper contributes to hybrid memory management research, leveraging real-world traces for evaluation, and introducing an ML based approach for optimized data placement decisions.

Neuro-Symbolic models combine the best of two worlds, knowledge representation capabilities of symbolic models and representation learning power of deep networks. In this paper, we develop a Neuro-Symbolic approach to infer unknown facts from relational data. A well-known approach is to use statistical relational models such as Markov Logic Networks (MLNs) to perform probabilistic inference. However, these approaches are known to be non-scalable and inaccurate for large, real-world problems. Therefore, given symbolic knowledge, we train a Neural Tensor Network (NTN) to learn representations for symmetries implied by the symbolic knowledge. Further, since the data is interconnected, predicting one fact can positively or negatively impact the prediction of other facts. Therefore, we train the NTN using open-world semantics over multiple possible worlds, learning to represent symmetries in each world. We evaluate our approach in several real-world benchmarks comparing with state-of-the-art relational learning methods, Neuro-Symbolic methods and purely symbolic methods clearly illustrating the generality, accuracy and scalability of our proposed approach.

Neuro-symbolic learning, where deep networks are combined with symbolic knowledge can help regularize the model and control overfitting. In particular, for applications where data instances are not independent, domain knowledge can be used to specify relational dependencies which may be hard to infer purely from the data. Symbolic AI models such as Markov Logic networks (MLNs) which are based on first-order logic are designed to represent and reason with uncertain background knowledge. However learning and inference algorithms in such models is known to be slow and inaccurate. In this paper, we develop a novel model that combines the best of both worlds, namely, the scalable learning capabilities of DNNs and symbolic knowledge specified in MLNs. To do this, we infer symmetries in the data based on the relational knowledge encoded in an MLN knowledge base and train a Convolutional Neural Network (CNN) to learn kernels combining symmetrical variables. However, by doing this, we are forced to split the relational data into independent instances for CNN training which may result is a loss of relational dependencies adding noise/uncertainty to the learned model. Therefore, instead of a single model, we learn a distribution over the model parameters. Our experiments illustrate that our model outperforms purely-MLN or purely-DNN based models in several different problem domains.

Weight learning is a challenging problem in Markov Logic Networks (MLNs) due to the large size of the ground propositional probabilistic graphical model that underlies the first-order representation of MLNs. Though more sophisticated weight learning methods that use lifted inference have been proposed, such methods can typically scale up only in the absence of evidence, namely in generative weight learning. In discriminative learning, where the evidence typically destroys symmetries, existing approaches are lacking in scalability. In this paper, we propose a novel, intuitive approach for learning MLNs discriminatively by utilizing approximate symmetries. Specifically, we reduce the size of the training database by clustering approximately symmetric atoms together and selecting a representative atom from each cluster. However, each choice made from the clusters induces a different distribution, increasing the uncertainty in our learned model. To reduce this uncertainty, we learn a finite mixture model by stacking the different distributions, where the parameters of the model are learned using an EM approach. Our results on several benchmarks show that our approach is much more scalable and accurate as compared to existing state-of-the-art MLN learning methods.

Typically, inference algorithms for big data address non-relational data. However, clearly, a lot of real-world data such as social network data, healthcare data, etc. are relational in nature. Therefore, we need more powerful techniques that can scale up richer inference algorithms on relational data. Markov Logic Networks (MLNs) are arguably one of the most popular statistical relational models that can represent complex, uncertain knowledge succinctly. In this paper, we scale up inference algorithms for MLNs to big relational data. Specifically, the probabilistic graphical model underlying an MLN is typically extremely large even for small-sized problems, and performing inference on this model is highly challenging. A pre-dominant approach that is used to improve scalability is to perform lifted inference that does not construct the full graphical model underlying the MLN. Instead, the idea in lifted inference is to use symmetries in the distribution to reduce the size of the model, thus improving scalability. A popular approach to perform lifting utilizes clustering techniques to group together variables with similar distributional characteristics. However, for big relational data, it quickly becomes infeasible to identify these symmetries scalably. In this paper, we design a novel lifted inference system built on top of Spark that takes advantage of parallelism to identify symmetries in the MLN. Thus our work unifies advances in inference for relational data with advances in big data processing technologies. Utilizing the power of Spark, we show that we can perform more accurate inference and scale up relational inference to orders of magnitude larger sized datasets than currently possible by state-of-the-art MLN systems.

Inference is a central problem in probabilistic graphical models, and is often the main sub-step in probabilistic learning procedures. Thus, accurate inference algorithms are essential to both answer queries on a learned model, as well as to learn a robust model. Gibbs sampling is arguably one of the most popular approximate inference methods that has been widely used for probabilistic inference in several different domains including natural language processing, computer vision. etc. Here, we develop an approach that improves the performance of blocked Gibbs sampling, an advanced variant of the Gibbs sampling algorithm. Specifically, we utilize correlation among variables in the probabilistic graphical model to develop an adaptive blocked Gibbs sampler that automatically tunes its proposal distribution based on statistics derived from previous samples. Specifically, we adapt the proposal such that we sample blocks containing highly correlated variables more often than the others. This in turn helps improve probability estimates given by the sampler, by selecting hard-to-sample variables more often during the sampling procedure. Further, since adaptation breaks the Markovian property of the sampler, we develop a method to guarantee that our sampler converges to the correct stationary distribution despite being non-Markovian, by diminishing the adaptation of the selection probabilities over time. We evaluate our method with several discrete probabilistic graphical models taken from UAI challenge problems corresponding to different domains, and show that our approach is superior in terms of accuracy as compared to methods that ignore correlation information in the proposal distribution of the sampler.