Large graphs are becoming ubiquitous, presenting significant computational hurdles in data processing and analysis. Graph Coarsening algorithms are frequently employed to condense large graphs while preserving key graph properties. Real-world graphs also have features or contexts associated with each node. However, existing coarsening methods often overlook simultaneity across node features and structural information. Recent approaches to alleviate this limitation are computationally intensive, and primarily suited for homophilic datasets. Most existing approaches are unsuitable for streaming and evolving graphs, as they require recomputation of the coarsened graph at every timestamp. In this paper, we introduce a Fast and Scalable Hashing-Based Universal Graph Coarsening (UGC) Framework, that integrates locality-sensitive hashing, and feature augmentation to effectively coarsen graphs. UGC is exceptionally fast, straightforward to implement, and capable of handling homophilic, heterophilic, and streaming graphs making it a truly universal solution for graph coarsening. We use an optimization-based framework to minimize a constrained $\epsilon$ε similarity between the original and coarsened graphs, where $\epsilon$ε is between zero and one. Through extensive experimentation on real and synthetic datasets, we demonstrate the effectiveness of our approach in terms of improved runtime complexity and generalization to heterophilic and streaming graphs. Furthermore, we showcase its utility in downstream tasks, emphasizing its scalability for training graph neural networks on coarsened graphs from benchmark real-world datasets.

Graph Neural Networks (GNNs) have emerged as powerful tools for modeling financial data as networks, effectively capturing both individual attributes and complex relationships. However, their inherent message-passing and aggregation operations introduce significant inference latency, limiting their applicability in latency-sensitive domains such as finance, healthcare, and robotics. Recent efforts have attempted to mitigate this limitation by distilling GNN knowledge into more efficient Multi-Layer Perceptrons (MLPs). While promising in reducing inference costs, existing GNN-to-MLP distillation approaches face three critical challenges: (1) reliance on labeled data, (2) limited robustness to noisy or perturbed inputs due to the absence of structural information, and (3) the existence of representational bias. To address these issues, we propose REFINE, a novel self-supervised GNN-to-MLP knowledge distillation framework. Our method enhances model stability and fairness through structure-free feature augmentations, including noise injection and counterfactual generation. Extensive experiments on two real-world financial datasets and one social network benchmark demonstrate that our approach consistently outperforms existing distillation baselines, achieving a favorable trade-off between predictive utility, stability, and fairness.

Ensuring fairness in Recommendation Systems (RSs) across demographic groups is critical due to the increased integration of RSs in applications such as personalized healthcare, finance, and e-commerce. Graph-based RSs play a crucial role in capturing intricate higher-order interactions among entities. However, integrating these graph models into the Federated Learning (FL) paradigm with fairness constraints poses formidable challenges as this requires access to the entire interaction graph and sensitive user information (such as gender, age, etc.) at the central server. This paper addresses the pervasive issue of inherent bias within RSs for different demographic groups without compromising the privacy of sensitive user attributes in FL environment with the graph-based model. To address the group bias, we propose F2PGNN (Fair Federated Personalized Graph Neural Network), a novel framework that leverages the power of Personalized Graph Neural Network (GNN) coupled with fairness considerations. Additionally, we use differential privacy techniques to fortify privacy protection. Experimental evaluation on three publicly available datasets showcases the efficacy of F2PGNN in mitigating group unfairness by 47% ∼ 99% compared to the state-of-the-art while preserving privacy and maintaining the utility. The results validate the significance of our framework in achieving equitable and personalized recommendations using GNN within the FL landscape. Source code is at: https://github.com/nimeshagrawal/F2PGNN-AAAI24

ML models are increasingly being used to increase the test coverage and decrease the overall testing time. This field is still in its nascent stage and up till now there were no algorithms that could match or outperform commercial tools in terms of speed and accuracy for large circuits. We propose an ATPG algorithm HybMT in this paper that finally breaks this barrier Like sister methods, we augment the classical PODEM algorithm that uses recursive backtracking. We design a custom 2-level predictor that predicts the input net of a logic gate whose value needs to be set to ensure that the output is a given value (0 or 1). Our predictor chooses the output from among two first-level predictors, where the most effective one is a bespoke neural network and the other is an SVM regressor. As compared to a popular, state-of-the-art commercial ATPG tool, HybMT shows an overall reduction of 56.6% in the CPU time without compromising on the fault coverage for the EPFL benchmark circuits. HybMT also shows a speedup of 126.4% over the best ML-based algorithm while obtaining an equal or better fault coverage for the EPFL benchmark circuits.

Neural networks (NNs) that exploit strong inductive biases based on physical laws and symmetries have shown remarkable success in learning the dynamics of physical systems directly from their trajectory. However, these works focus only on the systems that follow deterministic dynamics, such as Newtonian or Hamiltonian. Here, we propose a framework, namely Brownian graph neural networks (BroGNet), combining stochastic differential equations (SDEs) and GNNs to learn Brownian dynamics directly from the trajectory. We modify the architecture of BroGNet to enforce linear momentum conservation of the system, which, in turn, provides superior performance on learning dynamics as revealed empirically. We demonstrate this approach on several systems, namely, linear spring, linear spring with binary particle types, and non-linear spring systems, all following Brownian dynamics at finite temperatures. We show that BroGNet significantly outperforms proposed baselines across all the benchmarked Brownian systems. In addition, we demonstrate zero-shot generalizability of BroGNet to simulate unseen system sizes that are two orders of magnitude larger and to different temperatures than those used during training. Finally, we show that BroGNet conserves the momentum of the system resulting in superior performance and data efficiency. Altogether, our study contributes to advancing the understanding of the intricate dynamics of Brownian motion and demonstrates the effectiveness of graph neural networks in modeling such complex systems.

In the era of big data, graphs have emerged as a natural representation of intricate relationships. However, graph sizes often become unwieldy, leading to storage, computation, and analysis challenges. A crucial demand arises for methods that can effectively downsize large graphs while retaining vital insights. Graph coarsening seeks to simplify large graphs while maintaining the basic statistics of the graphs, such as spectral properties and $\epsilon$-similarity in the coarsened graph. This ensures that downstream processes are more efficient and effective. Most published methods are suitable for homophilic datasets, limiting their universal use. We propose **U**niversal **G**raph **C**oarsening (UGC), a framework equally suitable for homophilic and heterophilic datasets. UGC integrates node attributes and adjacency information, leveraging the dataset's heterophily factor. Results on benchmark datasets demonstrate that UGC preserves spectral similarity while coarsening. In comparison to existing methods, UGC is 4x to 15x faster, has lower eigen-error, and yields superior performance on downstream processing tasks even at 70% coarsening ratios.

Due to the nonsparse representation, the use of compressed sensing (CS) for physiological signals, such as a multichannel electroencephalogram (EEG), has been a challenge. We present a generalized Bayesian CS framework that is capable of handling representations that arise in the spatiotemporal setting. The proposed model utilizes the standard linear Gaussian observation model associated with the hierarchical modeling of data using the matrix-variate Gaussian scale mixture (GSM). It deploys various random and deterministic parameters to incorporate the knowledge of spatial and temporal correlation present in data. By varying distributions over random parameters, a family of generalized hyperbolic matrix variate distributions is derived. For estimation, we rely on variational Bayes (VB) for random parameters and expectation-maximization (EM) for deterministic parameters. Furthermore, the model is compared with recent developments in matrix-variate distribution-based modeling of data, and we briefly discuss its extension to finite mixtures of skewed distributions. Finally, the framework is applied to the steady-state visual evoked potential (SSVEP)-based EEG benchmark data set, and a comparative study is conducted to show its effectiveness for the frequency detection task. One of the crucial features of the proposed model is that it simultaneously processes multichannel signals with low computational cost and time, making it suitable for real-time systems, especially in a resource-constrained environment.

We propose Enhash, a fast ensemble learner that detects concept drift in a data stream.A stream may consist of abrupt, gradual, virtual, or recurring events, or a mixture of various types of drift.Enhash employs projection hash to insert an incoming sample.Benchmark tests on 6 artificial and 4 real data sets consisting of various types of drift show that Enhash is competitive with stateof-the-art ensemble learners while being significantly faster.It also has moderate resource requirements. 59

Implementing machine learning models on resource-constrained platforms such as hardware devices requires sparse models that can generalize well. This article analyzes the effect of parameter (or weight) quantization on the performance, number of support vectors, model size in bits,$L_2$norm and training time on various Support Vector Machines (SVM) and Minimal Complexity Machhines (MCM)-based kernel methods. We show that, Empirical Feature Space (EFS) and hinge loss-based MCM algorithms result in comparable accuracy, (8–190)x smaller model size in bits and (10k–16k)x smaller$L_2$norm at full precision compared with LIBSVM. The Least Squares (LS) variants of MCM based methods results in$\approx$2% improvement in accuracy, upto 16x reduction in model size and upto 3x reduction in$L_2$norm at full precision compared with its state-of-the-art counterpart Sparse Fixed Size variant of LS-SVM (SFS-LS-SVM). We quantize the weights of the compared variants post-training and demonstrate that our methods can retain their accuracies even with 7 bits as opposed to 10 and 14 bits used by LIBSVM and SFS-LS-SVM, respectively. Our experiments illustrate that quantization further improves upon the model sizes used by our methods by upto 300x and 30x compared with the LIBSVM and SFS-LS-SVM. This has significant implications for implementation in Internet of Things (IoT) devices, which benefit from model sparsity and good generalization.

This paper addresses the problem of Bayesian Block Sparse Modeling when coefficients within the blocks are correlated. In contrast to the current hierarchical methods which do not exploit correlation structure within the blocks, we propose a three level hierarchical estimation framework. It employs heavy-tailed priors for block sparse modeling and variational inference for Bayesian estimation. This paper also describes the relationship between proposed framework and some of the existing Block Sparse Bayesian Learning (SBL) methods and show that these SBL methods can be viewed as its special cases. Extensive experimental results for synthetic signals are provided, demonstrating the superior performance of the proposed framework in terms of failure rate, relative reconstruction error, to name a few. We also demonstrate the applicability of this framework in telemonitoring of Fetal Electrocardiogram.

Classification of ElectroEncephaloGram (EEG) signals has found several applications in developing Brain Computer Interfaces (BCIs), as well as other clinical and nonclinical applications based on EEG signals. Processing of EEG signals in this context is challenged by its non-stationarity, high dimensionality and the problem of class imbalance for training classifiers, particularly in case of multi-class classification. Our recent work demonstrated the utility of Twin Support Vector Machine (TWSVM) classifiers for robust classification of imbalanced datasets, specifically EEG signal classification. However, the architecture of the TWSVM is not scalable for large datasets as it involves computing the kernel and matrix inversion operations. In this paper, we present an application of the recently proposed neural network architecture for the Twin SVM, the Twin Neural Network (Twin NN), for robust classification of EEG signals. Results on datasets from BCI competitions illustrate the improved generalization and scalability of the Twin NN for the binary and multi-class classification tasks.

This paper extends the idea of Least Squares Minimal Complexity Machines [1] (LS-MCMs) to non-Mercer kernels. There are no efficient solvers for LS-SVMs with non-Mercer kernels for large scale datasets. Here, we propose two variants of our novel multiclass loss LS-MCMs. Firstly, with an L1regularizer and secondly, with an explicit margin regularizer along with L1norm in the Empirical Feature Space (EFS). Both these methods can be scaled to large datasets with the use of “prototype vectors” selected from the dataset. The first optimization algorithm can be solved efficiently using Stochastic Gradient Descent (SGD) directly as the problem remains convex in the parameter space. The second problem however, is solved using difference of convex functions (DC) programming with SGD due to the non-convex nature of margin regularizer. Our method also obtains a sparse solution as opposed to the one obtained using LS-SVMs which tend to be non-sparse.

Minimal complexity machines (MCMs) are a class of hyperplane classifiers that try to minimize a tight bound on the Vapnik-Chervonenkis dimension. MCMs can be used both in the input space and in a higher dimensional feature space via the kernel trick. MCMs tend to produce very sparse solutions in comparison to support vector machines, often using three to ten times fewer support vectors. However, large datasets present significant challenges in terms of storage and operations on the kernel matrix. In this paper, we present a stochastic subgradient descent solver for large-scale machine learning with the MCM. The proposed approach uses an explicit feature map-based approximation of the kernel, to improve the scalability of the algorithm.

This paper describes a variant of the Incremental Ant Colony Optimization algorithm for continuous optimization (IACOℝ). The original IACOℝapproach estimates the Probability Density Function (PDF) using Gaussians constructed around candidate solutions to generate new solutions. We use Support Vector Regression (SVR) to fit a regressor to the candidate solutions. The minima of the fitted regressor are found using a variant of EigenAnt. This approach is based on the observation that minima tend to be clustered in real problems, and estimating the landscape of minima is more efficient than estimating the landscape of the original function. We present results on two fronts. We demonstrate the effect of the use of SVR and modified EigenAnt. Further, we also demonstrate the performance of our approach on the Soft Computing (SOCO) benchmark functions for global optimization, and obtain appreciable results.

Identifying wrist and finger flexions from surface Electromyogram (sEMG) signals finds several applications for developing prosthesis-based device control. However, sEMG signals can be corrupted by muscular activity from multiple sources at the site of acquisition, and hence the identification of intents from these signals presents a challenge. Moreover, there can be multiple intents which need to be recognized, hence a robust classifier is required. The accurate recognition of these movements is imperative as it enables reliable control of devices. In this paper, we use the Twin Support Vector Machine (Twin SVM) classifier to identify 15 classes of wrist and finger flexions using one-v/s-rest classification approach. Our work uses sEMG data obtained from nine subjects, including an amputee volunteer. We compare the improved accuracy obtained in using Twin SVM against LIBSVM (a standard SVM implementation) to demonstrate the effectiveness of the classifier. We use a simple feature - the Root Mean Square (RMS) value of the signal during the trial as features for the classifier. Our results demonstrate the effectiveness of using the Twin SVM in a multi-class scenario with unbalanced datasets, which holds significance in addressing the broader challenges in classification presented in several applications based on processing of biomedical signals.

A widely known approach for continuous global optimization has been the Incremental Ant Colony Framework (IACOR). In this paper, we propose a strategy to introduce hybridization within the exploitation phase of the IACOR framework by using the Multi-Trajectory Local Search (Mtsls1) algorithm and Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithms. Our approach entails making a probabilistic choice between these algorithms. In case of stagnation, we switch the algorithm being used based on the last iteration. We evaluate our approach on the Soft Computing (SOCO) benchmark functions and present results by computing the mean and median errors on the global optima achieved, as well as the iterations required. We compare our approach with competing methods on a number of benchmark functions, and show that the proposed approach achieves improved results. In particular, we obtain the global optima in terms of average value for 14 out of 19 benchmark functions, and in terms of the median value for all SOCO benchmarks. At the same time, the proposed approach uses fewer function evaluations on several benchmarks when compared with competing methods, which have been found to use 54% more function evaluations.

Though the Extreme Learning Machine (ELM) has become quite popular in recent years, there are no performance guarantees; the resultant networks also tend to be densely connected. The complexity of a learning machine may be measured by the Vapnik-Chervonenkis (VC) dimension, and a small VC dimension leads to good generalization and lower test set errors. The Minimal Complexity Machine (MCM), that has been proposed very recently, shows that it is possible to learn a classifier with minimal VC dimension, leading to sparse representations and good generalization. In this paper, we draw on results from the MCM to propose a hybrid variant of the ELM, termed the Minimal Complexity - Extreme Learning Machine (MC-ELM), in order to realize a robust classifier that minimizes an exact bound on the VC dimension. The MC-ELM solves a linear programming problem for the last layer and offers the advantages of large margin and low VC dimension. In effect, the learning paradigm elucidated in this paper helps us build a classifier which is based on a minimal representation of the training data owing to MCM, and high training speed attributed to ELM. This makes it feasible for use in complex machine learning applications, where these advantages are of significance.

This paper introduces an ACO model and associated algorithm, called Coupled EigenAnt, for the problem of finding the shortest of N paths between a source and a destination node. It is based on the recently introduced EigenAnt algorithm, the novelty being that it allows probabilistic path choice on both the forward and return journeys, as well as the fact that it introduces decay of pheromone deposition following a geometric progression. Equilibrium points of the model are calculated and the local stability of the two path synchronous version analyzed. Simulations illustrate the main features of the algorithm.

In this paper, we shed light on how powerful congestion control based on local interactions may be obtained. We show how ants can use repellent pheromones and incorporate the effect of crowding to avoid traffic congestion on the optimal path. Based on these interactions, we propose an ant algorithm, the M-unit EigenAnt algorithm, that leads to the selection of the M shortest paths. The ratio of selection of each of these paths is also optimal and regulated by an optimal amount of pheromone on each of them. To the best of our knowledge, the M-unit EigenAnt algorithm is the first antalgorithm that explicitly ensures the selection of the M shortest paths and regulates the amount of pheromone on them such that it is asymptotically optimal. In fact, it is in contrast with most ant algorithms that aim to discover just a single best path. We provide its convergence analysis and show that the steady state distribution of pheromone aligns with the eigenvectors of the cost matrix, and thus is related to its measure of quality. We also provide analysis to show that this property ensues even when the food is moved or path lengths change during foraging. We show that this behavior is robust in the presence of fluctuations and quickly reflects the change in the M optimal solutions. This makes it suitable for not only distributed applications butalso dynamic ones as well. Finally, we provide simulation results for the convergence to the optimal solution under different initial biases, dynamism in lengths of paths, and discovery of new paths.

Surface electromyogram (sEMG) is a measure of the muscle activity from the skin surface, and is an excellent indicator of the strength of muscle contraction. It is an obvious choice for control of prostheses and identification of body gestures. Using sEMG to identify posture and actions that are a result of overlapping multiple active muscles is rendered difficult by interference between different muscle activities. In the literature, attempts have been made to apply independent component analysis to separate sEMG into components corresponding to the activities of different muscles, but this has not been very successful, because some muscles are larger and more active than the others. We address the problem of how to learn to separate each gesture or activity from all others. Multicategory classification problems are usually solved by solving many one-versus-rest binary classification tasks. These subtasks naturally involve unbalanced datasets. Therefore, we require a learning methodology that can take into account unbalanced datasets, as well as large variations in the distributions of patterns corresponding to different classes. This paper reports the use of twin support vector machine for gesture classification based on sEMG, and shows that this technique is eminently suited to such applications.

We propose Twin SVM, a binary SVM classifier that determines two nonparallel planes by solving two related SVM-type problems, each of which is smaller than in a conventional SVM. The Twin SVM formulation is in the spirit of proximal SVMs via generalized eigenvalues. On several benchmark data sets, Twin SVM is not only fast, but shows good generalization. Twin SVM is also useful for automatically discovering two-dimensional projections of the data.

In this paper, we use the approach of adaptive critic design (ACD) for control, specifically, the action-dependent heuristic dynamic programming (ADHDP) method. A least squares support vector machine (SVM) regressor has been used for generating the control actions, while an SVM-based tree-type neural network (NN) is used as the critic. After a failure occurs, the critic and action are retrained in tandem using the failure data. Failure data is binary classification data, where the number of failure states are very few as compared to the number of no-failure states. The difficulty of conventional multilayer feedforward NNs in learning this type of classification data has been overcome by using the SVM-based tree-type NN, which due to its feature to add neurons to learn misclassified data, has the capability to learn any binary classification data without a priori choice of the number of neurons or the structure of the network. The capability of the trained controller to handle unforeseen situations is demonstrated.

In this paper, we propose a novel approach, called Regularized Least Squares Fuzzy Support Vector Regression, to handle time series forecasting. Two key problems in time series forecasting are noise and non-stationarity. Here, we assign a higher membership value to data samples that contain more relevant information. The approach requires only a single matrix inversion, and for the linear case, the matrix order depends only on the dimension in which the data samples lie, and is independent of the number of samples.

In a recent publication, Lazaro et al. addressed the problem of simultaneously approximating a function and its derivative using support vector machines. In this paper, we propose a new approach termed as regularized least squares twin support vector regression, for the simultaneous learning of a function and its derivatives. The regressor is obtained by solving one of two related support vector machine-type problems, each of which is of a smaller size than the one obtained in Lazaro's approach. The proposed algorithm is simple and fast, as no quadratic programming problem needs to be solved. Effectively, only the solution of a pair of linear systems of equations is needed.