Graph data collected from the real world often contains noise, making it imperative to develop robust representation learning tools for graphs. While existing research has primarily focused on feature smoothing, the robustness of the underlying geometric structure is frequently overlooked. In addition, the prevalent use of the$\mathbb {L}_{2}$-norm for achieving global smoothness in graph neural networks shrinks many local characteristics, limiting their expressivity on a node's neighboring information. This paper introduces novel regularizers designed to address noise in both feature and structural aspects of graph data. We employ the alternating direction method of multipliers (ADMM) to optimize the objective function. Our proposed approach effectively prevents oversmoothing graph signal representations when applying multiple layers and ensures convergence to optimal solutions. Empirical results from our study demonstrate the superior performance of our proposedDoTover popular graph convolutions, especially in scenarios where the graph is heavily contaminated.

Graph representation learning has many real-world applications, from self-driving LiDAR, 3D computer vision to drug repurposing, protein classification, social networks analysis. An adequate representation of graph data is vital to the learning performance of a statistical or machine learning model for graph-structured data. This paper proposes a novel multiscale representation system for graph data, called decimated framelets, which form a localized tight frame on the graph. The decimated framelet system allows storage of the graph data representation on a coarse-grained chain and processes the graph data at multi scales where at each scale, the data is stored on a subgraph. Based on this, we establish decimated G-framelet transforms for the decomposition and reconstruction of the graph data at multi resolutions via a constructive data-driven filter bank. The graph framelets are built on a chain-based orthonormal basis that supports fast graph Fourier transforms. From this, we give a fast algorithm for the decimated G-framelet transforms, or FGT, that has linear computational complexity O(N) for a graph of size N. The effectiveness for constructing the decimated framelet system and the FGT is demonstrated by a simulated example of random graphs and real-world applications, including multiresolution analysis for traffic network and representation learning of graph neural networks for graph classification tasks.

This paper presents a new approach for assembling graph neural networks based on framelet transforms. The latter provides a multi-scale representation for graph-structured data. We decompose an input graph into low-pass and high-pass frequencies coefficients for network training, which then defines a framelet-based graph convolution. The framelet decomposition naturally induces a graph pooling strategy by aggregating the graph feature into low-pass and high-pass spectra, which considers both the feature values and geometry of the graph data and conserves the total information. The graph neural networks with the proposed framelet convolution and pooling achieve state-of-the-art performance in many node and graph prediction tasks. Moreover, we propose shrinkage as a new activation for the framelet convolution, which thresholds high-frequency information at different scales. Compared to ReLU, shrinkage activation improves model performance on denoising and signal compression: noises in both node and structure can be significantly reduced by accurately cutting off the high-pass coefficients from framelet decomposition, and the signal can be compressed to less than half its original size with well-preserved prediction performance.

Adam-type optimizers, as a class of adaptive moment estimation methods with the exponential moving average scheme, have been successfully used in many applications of deep learning. Such methods are appealing due to the capability on large-scale sparse datasets with high computational efficiency. In this paper, we present a new framework for Adam-type methods with the trend information when updating the parameters with the adaptive step size and gradients. The additional terms in the algorithm promise an efficient movement on the complex cost surface, and thus the loss would converge more rapidly. We show empirically the importance of adding the trend component, where our framework outperforms the conventional Adam and AMSGrad methods constantly on the classical models with several real-world datasets.

This paper unifies our earlier work on detection of brain activation (Rajapakse and Piyaratna, 2001) and connectivity (Rajapakse and Zhou, 2007) in a probabilistic framework for analyzing effective connectivity among activated brain regions from functional magnetic resonance imaging (fMRI) data. Interactions among brain regions are expressed by a dynamic Bayesian network (DBN) while contextual dependencies within functional images are formulated by a Markov random field. The approach simultaneously considers both the detection of brain activation and the estimation of effective connectivity and does not require a priori model of connectivity. Experimental results show that the present approach outperforms earlier fMRI analysis techniques on synthetic functional images and robustly derives brain connectivity from real fMRI data.

This paper proposes a novel approach for analysis of brain connectivity shown in functional MRI (fMRI), using graphical models. Structural equation modeling (SEM) is currently used to model neural systems by using partial covariance values, which is only able to affirm or refute functional connectivity of a previously known anatomical model or select the best fit model from a set of a priori models. Our approach is exploratory in the sense that it does not require a priori model such as an anatomical model. The SEM uses covariances which describe only second order behavior of a network while conditional probabilities used in graphical models, in theory, describe the complete behavior of a network. It renders the interactions among brain regions with conditional densities and allows simulation of disconnectivity of neural systems.