Graph Neural Networks (GNNs) have demonstrated a great representation learning capability on graph data and have been utilized in various downstream applications. However, real-world data in web-based applications (e.g., recommendation and advertising) always contains bias, preventing GNNs from learning fair representations. Although many works were proposed to address the fairness issue, they suffer from the significant problem of insufficient learnable knowledge with limited attributes after debiasing. To address this problem, we develop Graph-Fairness Mixture of Experts (G-Fame), a novel plug-and-play method to assist any GNNs to learn distinguishable representations with unbiased attributes. Furthermore, based on G-Fame, we propose G-Fame++, which introduces three novel strategies to improve the representation fairness from node representations, model layer, and parameter redundancy perspectives. In particular, we first present the embedding diversified method to learn distinguishable node representations. Second, we design the layer diversified strategy to maximize the output difference of distinct model layers. Third, we introduce the expert diversified method to minimize expert parameter similarities to learn diverse and complementary representations. Extensive experiments demonstrate the superiority of G-Fame and G-Fame++ in both accuracy and fairness, compared to state-of-the-art methods across multiple graph datasets.

The development of social media (e.g., Twitter) allows users to make speeches with low cost and broad influence. Thus, social media has become a perfect place for users’ malicious behaviors like committing hate crimes, spreading toxic information, abetting crimes, etc. Malicious behaviors are covert and widespread, with potential relevance regarding topic, person, place, and so on. Therefore, it is necessary to develop novel techniques to detect and disrupt malicious behavior on social media effectively. Previous research has shown promising results in extracting semantic text (speech) representation using natural language processing methods. Yet the latent relation between speeches and the connection between users behind speeches is rarely explored. In light of this, we propose a holistic model named Graph adaption BERT (GraphBERT) to detect malicious behaviors on Twitter with both semantic and relational information. Specifically, we first present a novel and a large-scale corpus of tweet data to benefit both graph-based and language-based malicious behavior detection research. Then, we design a novel model GraphBERT to learn comprehensive tweet and user representation with the integration of both semantic information encoded by transformers (i.e., BERT) and relational information encoded by graph neural network. GraphBERT further leverages a weight adaption BERT module implemented between transformer layers to refine tweet embedding using relational information for malicious tweet classification. Finally, the adapted tweet embedding is used with the initial tweet representation to generate user embedding for malicious user detection. The extensive experiments on the collected Twitter data show that our model outperforms the state-of-the-art baseline methods for both tasks (i.e., malicious tweet classification and malicious user detection).