Recent years have witnessed significant advancements in machine learning methods on graphs. However, transferring knowledge effectively from one graph to another remains a critical challenge. This highlights the need for algorithms capable of applying information extracted from a source graph to an unlabeled target graph, a task known as unsupervised graph domain adaptation (GDA). One key difficulty in unsupervised GDA is conditional shift, which hinders transferability. In this paper, we show that conditional shift can be observed only if there exists local dependencies among node features. To support this claim, we perform a rigorous analysis and also further provide generalization bounds of GDA when dependent node features are modeled using markov chains. Guided by the theoretical findings, we propose to improve GDA by decorrelating node features, which can be specifically implemented through decorrelated GCN layers and graph transformer layers. Our experimental results demonstrate the effectiveness of this approach, showing not only substantial performance enhancements over baseline GDA methods but also clear visualizations of small intra-class distances in the learned representations. Our code is available at https://github.com/TechnologyAiGroup/DFT.

Ensemble learning is a pivotal machine learning strategy that combines multiple base learners to achieve prediction accuracy surpassing that of any individual model. Despite its effectiveness, large-scale ensemble learning consumes a considerable amount of resources. Ensemble pruning addresses this issue by selecting a subset of base learners from the original ensemble to form a sub-ensemble, while maintaining or even improving the performance of the original model. However, existing ensemble pruning strategies often rely on heuristic solutions that may fail to capture complex interactions among base learners. To address this limitation, in this work, we model the base learners in an ensemble as a weighted and attributed graph, where node features represent characteristics of each learner and edge weights represent relationships between the base learners. Leveraging this representation, we propose a novel ensemble pruning method based on graph neural networks (GNNs). Our approach incorporates specialized GNN architectures designed for bagging and boosting ensembles. Experimental results demonstrate that our method not only improves prediction accuracy but also significantly reduces inference time across diverse datasets. Our implementation is available at the anonymous repository: https://github.com/TechnologyAiGroup/GRE.

Graph anomaly detection (GAD) has become an increasingly important task across various domains. With the rapid development of graph neural networks (GNNs), GAD methods have achieved significant performance improvements. However, fairness considerations in GAD remain largely underexplored. Indeed, GNN-based GAD models can inherit and amplify biases present in training data, potentially leading to unfair outcomes. While existing efforts have focused on developing fair GNNs, most approaches target node classification tasks, where models often rely on simple layer architectures rather than autoencoder-based structures, which are the most widely used architecturs for anomaly detection. To address fairness in autoencoder-based GAD models, we propose DisEntangled Counterfactual Adversarial Fair (DECAF)-GAD, a framework that alleviates bias while preserving GAD performance. Specifically, we introduce a structural causal model (SCM) to disentangle sensitive attributes from learned representations. Based on this causal framework, we formulate a specialized autoencoder architecture along with a fairness-guided loss function. Through extensive experiments on both synthetic and real-world datasets, we demonstrate that DECAF-GAD not only achieves competitive anomaly detection performance but also significantly enhances fairness metrics compared to baseline GAD methods. Our code is available at https://github.com/Tlhey/decaf_code.

Hyperbolic neural networks (HNNs) are emerging as a promising tool for representing data embedded in non-Euclidean geometries, yet their adoption has been hindered by challenges related to stability and robustness. In this work, we conduct a rigorous Lipschitz analysis for HNNs and propose using Lipschitz regularization as a novel strategy to enhance their robustness. Our comprehensive investigation spans both the Poincaré ball model and the hyperboloid model, establishing Lipschitz bounds for HNN layers. Importantly, our analysis provides detailed insights into the behavior of the Lipschitz bounds as they relate to feature norms, particularly distinguishing between scenarios where features have unit norms and those with large norms. Further, we study regularization using the derived Lipschitz bounds. Our empirical validations demonstrate consistent improvements in HNN robustness against noisy perturbations.

The pursuit of accurate diagnosis with good resolution is driven by yield learning during both early bring-up and production excursions. Unfortunately, fault callouts from diagnosis tools often render poor resolution that hinders the follow-up failure analysis. In this work, we propose a method that significantly improves diagnosis. By modeling the logic circuits under test as graphs, the method employs graph neural networks to determine each fault candidate from the diagnosis callout as either the true fault or the false candidate. This novel deep learning method mainly makes full use of circuitry topology with underlying structural information, which was largely ignored or insufficiently analyzed by previous approaches. Other contributions include the finding of the dependency among candidates that can be leveraged to improve diagnoses. Extensive experiments on various benchmark circuits including industrial designs demonstrate that the diagnostic resolution can be improved by 4.51× compared with a fault simulator-based diagnosis tool, and increased by 5.98× compared with one state-of-the-art commercial diagnosis tool. Moreover, experiments also reveal that our method can successfully identify 62.96% of true candidates that were originally not given high priority by the commercial tool (non top-scoring candidates). This means our method can rectify the existing commercial diagnosis for better characterizing failure Pareto, in addition to boost diagnostic resolution.

The recent rapid growth in mobile data traffic entails a pressing demand for improving the throughput of the underlying wireless communication networks. Network node deployment has been considered as an effective approach for throughput enhancement which, however, often leads to highly nontrivial nonconvex optimizations. Although convex-approximation-based solutions are considered in the literature, their approximation to the actual throughput may be loose and sometimes lead to unsatisfactory performance. With this consideration, in this article, we propose a novel graph neural network (GNN) method for the network node deployment problem. Specifically, we fit a GNN to the network throughput and use the gradients of this GNN to iteratively update the locations of the network nodes. Besides, we show that an expressive GNN has the capacity to approximate both the function value and the gradients of a multivariate permutation-invariant function, as a theoretic support to the proposed method. To further improve the throughput, we also study a hybrid node deployment method based on this approach. To train the desired GNN, we adopt a policy gradient algorithm to create datasets containing good training samples. Numerical experiments show that the proposed methods produce competitive results compared with the baselines.

Failure diagnosis is a software-based, data-driven procedure. Collecting an excessive amount of fail data not only increases the overall test cost but can also potentially reduce diagnostic resolution. Thus, test-termination prediction is proposed to dynamically determine the appropriate failing test pattern to terminate testing, producing an amount of test data that is sufficient for an accurate diagnosis analysis. In this work, we describe a set of novel methods utilizing advanced machine learning techniques for efficient test-termination prediction. To implement this approach, we first generate images representing failing test responses from failure-log files. These images are then used to train a multi-layer convolutional neural network (CNN) incorporating a residual block. The trained CNN model leverages the images and known diagnostic results to determine the optimal test-termination strategy within the testing process, ensuring efficient and high-quality diagnosis. In addition to the integration of test response-to-image translation, our approach harnesses two cutting-edge learning strategies to enhance fail data and boost performance in subsequent tasks. The first strategy is transfer learning, which utilizes sample-label information from one circuit to guide the decision of whether to continue or stop testing for another circuit lacking labels. The second strategy involves the use of a generative deep model to generate fail data in the form of synthetic images. This technique increases the modeling effectiveness by expanding the volume of training samples. Experimental results conducted on actual failing chips and standard benchmarks validate that our proposed method surpasses existing approaches. Our method creates opportunities to harness the power of recent advances in machine learning for improving test and diagnosis efficiency.

We propose a novel and trainable graph unpooling layer for effective graph generation. The unpooling layer receives an input graph with features and outputs an enlarged graph with desired structure and features. We prove that the output graph of the unpooling layer remains connected and for any connected graph there exists a series of unpooling layers that can produce it from a 3-node graph. We apply the unpooling layer within the generator of a generative adversarial network as well as the decoder of a variational autoencoder. We give extensive experimental evidence demonstrating the competitive performance of our proposed method on synthetic and real data.

We propose a new method for novelty detection that can tolerate high corruption of the training points, whereas previous works assumed either no or very low corruption. Our method trains a robust variational autoencoder (VAE), which aims to generate a model for the uncorrupted training points. To gain robustness to high corruption, we incorporate the following four changes to the common VAE: 1. Extracting crucial features of the latent code by a carefully designed dimension reduction component for distributions; 2. Modeling the latent distribution as a mixture of Gaussian low-rank inliers and full-rank outliers, where the testing only uses the inlier model; 3. Applying the Wasserstein-1 metric for regularization, instead of the Kullback-Leibler (KL) divergence; and 4. Using a robust error for reconstruction. We establish both robustness to outliers and suitability to low-rank modeling of the Wasserstein metric as opposed to the KL divergence. We illustrate state-of-the-art results on standard benchmarks.

Graph neural networks (GNNs) have shown considerable promise for graph-structured data. However, they are also known to be unstable and vulnerable to perturbations and attacks. Recently, the Lipschitz constant has been adopted as a control on the stability of Euclidean neural networks, but calculating the exact constant is also known to be difficult even for very shallow networks. In this paper, we extend the Lipschitz analysis to graphs by providing a systematic scheme for estimating upper bounds of the Lipschitz constants of GNNs. We also derive concrete bounds for widely used GNN architectures including GCN, GraphSAGE and GAT. We then use these Lipschitz bounds for regularized GNN training for improved stability. Our numerical results on Lipschitz regularization of GNNs not only illustrate enhanced test accuracy under random noise, but also show consistent improvement for state-of-the-art defense methods against adversarial attacks.

Many convolutional neural networks (CNN's) have a feed-forward structure. In this paper, we model a general framework for analyzing the Lipschitz bounds of CNN's and propose a linear program that estimates these bounds. Several CNN's, including the scattering networks, the AlexNet and the GoogleNet, are studied numerically. In these practical numerical examples, estimations of local Lipschitz bounds are compared to these theoretical bounds. Based on the Lipschitz bounds, we next establish concentration inequalities for the output distribution with respect to a stationary random input signal. The Lipschitz bound is further used to perform nonlinear discriminant analysis that measures the separation between features of different classes.

Generative networks have made it possible to generate meaningful signals such as images and texts from simple noise. Recently, generative methods based on GAN and VAE were developed for graphs and graph signals. However, the mathematical properties of these methods are unclear, and training good generative models is difficult. This work proposes a graph generation model that uses a recent adaptation of Mallat's scattering transform to graphs. The proposed model is naturally composed of an encoder and a decoder. The encoder is a Gaussianized graph scattering transform, which is robust to signal and graph manipulation. The decoder is a simple fully connected network that is adapted to specific tasks, such as link prediction, signal generation on graphs and full graph and signal generation. The training of our proposed system is efficient since it is only applied to the decoder and the hardware requirements are moderate. Numerical results demonstrate state-of-the-art performance of the proposed system for both link prediction and graph and signal generation.