A significant challenge for Graph Neural Networks (GNNs) lies in accurately performing node classification tasks on heterophilous graphs. Traditional GNN models, which largely rely on the assumption of homophily, often struggle under heterophilous conditions due to the inherent low-pass filtering nature during message-passing among neighboring nodes. Motivated by these limitations, our research explores the dendritic structures found in real neurons, which dynamically amplify and inhibit signals. We demonstrate how the biophysical properties of dendrites enable GNNs to adaptively modulate the Dirichlet energy of graph representations based on the specific context, providing a theoretical justification for integrating neural dendrites into GNNs to improve learning on heterophilous graphs. In this work, we introduce the GDendrite framework, which dynamically adjusts GNN's output response to the various contexts. Our empirical studies, conducted across 13 homophilous and heterophilous benchmark datasets, show that our framework significantly enhances the performance of traditional GNNs in heterophilous settings and outperforms state-of-the-art models in most cases.

Recently, many works on Graph Neural Networks (GNNs) have been well developed for graph-level representation learning tasks and continuously improve graph classification accuracy. These works, however, mainly focus on the principle of different blocks of GNNs to strengthen their representation learning capability and pay less attention to the choice of a more appropriate loss function as learning objectives. In this paper, we aim to facilitate the representation learning process of any existing GNN frameworks by incorporating more task-oriented objectives. To this end, we propose approaches based on Laplacian eigenmaps to enhance the common-used cross-entropy loss, called LEELoss. Our study shows, with the property of Laplacian eigenmaps for classification problems, that utilizing the Laplacian eigenmaps as a regularizer in the original loss function can further enhance the performance of the graph-level learning tasks. Finally, via extensive experiments on popular benchmark datasets, we demonstrate that the proposed approach actually facilitates the classification performance on various GNN frameworks.

Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning tasks or model architectures. The research on flexible activation is quite limited in both designing philosophy and application scenarios. In this study, three principles of choosing flexible activation components are proposed and a general combined form of flexible activation functions is implemented. Based on this, a novel family of flexible activation functions that can replace sigmoid or tanh in LSTM cells are implemented, as well as a new family by combining ReLU and ELUs. Also, two new regularisation terms based on assumptions as prior knowledge are introduced. It has been shown that LSTM models with proposed flexible activations P-Sig- Ramp provide significant improvements in time series forecasting, while the proposed P-E2-ReLU achieves better and more stable performance on lossy image compression tasks with convolutional auto-encoders. In addition, the proposed regularization terms improve the convergence, performance and stability of the models with flexible activation functions. The code for this paper is available at https://github.com/9NXJRDDRQK/Flexible_Activation.