Federated learning has emerged as a promising paradigm for collaborative model training while preserving data privacy. However, many existing FL methods implicitly assume that clients have sufficient computational and storage resources, making them less applicable in real-world scenarios with severe system heterogeneity. To address this, submodel extraction has recently gained attention as a promising strategy to tailor the global model to resource-constrained clients. Despite this progress, existing methods often suffer from noticeable performance gaps across clients and structural inconsistency in the extracted models, leading to degraded global performance and increased communication overhead. In this work, we propose FedLAGC, a novel federated framework that jointly tackles performance imbalance and communication inefficiency through Layer-Adaptive submodel extraction and Gradient Correction. Specifically, FedLAGC constructs client-specific submodels by selecting structurally important parameters according to layer-wise importance scores, ensuring both resource adaptiveness and architectural consistency. Additionally, we propose a lightweight correction mechanism that captures historical optimization drift, helping to align local updates with the global direction and reduce redundant communication. The rigorous convergence analysis of FedLAGC for system-heterogeneous federated learning under non-convex objectives is given. Extensive experiments on CIFAR-10 and CIFAR-100 with ResNet-18 and ResNet-34 under various system and data heterogeneity settings demonstrate the significant superiority of FedLAGC (up to 24% accuracy improvement and 3.66× communication efficiency) over state-of-the-art methods.

Recently, the success of large models has demonstrated the importance of scaling up model sizes. However, it is difficult to directly train large models locally on multiple mobile devices due to their intrinsic computational constraints. To address this challenge, it becomes a crucial need to train larger global models by training small local models on devices. As a distributed learning approach, federated learning (FL) allows multiple devices to train models locally and aggregate them to form the global model by sharing the updated parameters with the server, thus enabling the co-training of models. This promising feature has spurred an increasing interest in exploring the collaborative training of large models. Despite the advent of existing device-heterogeneity FL approaches, they still have limitations in fully covering the parameter space of the global model. To fill this gap, we propose a novel approach calledFedBRB(Block- wiseRolling and weightedBroadcast). The core idea of FedBRB is to utilize local models of small devices to train all modules of a large global model and broadcast the trained parameters to the entire space, thereby enabling faster information sharing. This approach not only improves training efficiency but also fully utilizes limited computational resources. Experiments demonstrate that FedBRB can produce significant performance gains, achieving state-of-the-art results. Additionally, this paper provides theoretical and experimental analyses of FedBRB convergence, thereby paving a theoretical ground and providing practical guidance for further research and application of the FedBRB method.

Graph node clustering is a fundamental unsupervised task. Existing methods typically train an encoder through self-supervised learning and then apply K-means to the encoder output. Some methods use this clustering result directly as the final assignment, while others initialize centroids based on this initial clustering and then finetune both the encoder and these learnable centroids. However, due to their reliance on K-means, these methods inherit its drawbacks when the cluster separability of encoder output is low, facing challenges from the Uniform Effect and Cluster Assimilation. We summarize three reasons for the low cluster separability in existing methods: (1) lack of contextual information prevents discrimination between similar nodes from different clusters; (2) training tasks are not sufficiently aligned with the downstream clustering task; (3) the cluster information in the graph structure is not appropriately exploited. To address these issues, we propose conTrastive grapH clustEring by SwApping fUsed gRomov-wasserstein coUplingS (THESAURUS). Our method introduces semantic prototypes to provide contextual information, and employs a cross-view assignment prediction pretext task that aligns well with the downstream clustering task. Additionally, it utilizes Gromov-Wasserstein Optimal Transport (GW-OT) along with the proposed prototype graph to thoroughly exploit cluster information in the graph structure. To adapt to diverse real-world data, THESAURUS updates the prototype graph and the prototype marginal distribution in OT by using momentum. Extensive experiments demonstrate that THESAURUS achieves higher cluster separability than the prior art, effectively mitigating the Uniform Effect and Cluster Assimilation issues.

Federated prototype learning is in the spotlight as global prototypes are effective in enhancing the learning of local representation spaces, facilitating the ability to generalize the global model. However, when encountering domain-skewed data, conventional federated prototype learning is susceptible to two dilemmas: 1) Local prototypes obtained by averaging intra-class embedding carry domain-specific markers, the margins among aggregated global prototypes could be attenuated and detrimental to inter-class separation. 2) Local domain-skewed embedding may not exhibit a uniform distribution in Euclidean space, which is not conductive to the prototype-induced intra-class compactness. To address the two drawbacks, we go beyond conventional paradigm of federated prototype learning, and propose learnable semantic anchors with hyperspherical contrast (FedLSA) for domain-skewed data. Specifically, we eschew the pattern of yielding prototypes via averaging intra-class embedding and directly learn a set of semantic anchors aided by the global semantic-aware classifier. Meanwhile, the margins between anchors are augmented via pulling apart them, ensuring decent inter-class separation. To guarantee that local domain-skewed representations can be uniformly distributed, local data is projected into the hyperspherical space, and the intra-class compactness is achieved by optimizing the contrastive loss derived from the von Mises-Fisher distribution. Finally, extensive experimental results on three multi-domain datasets show the superiority of the proposed FedLSA compared to existing typical and state-of-the-state methods.

Differential equations provide a dynamical perspective for understanding and designing graph neural networks (GNNs). By generalizing the discrete Ricci flow (DRF) to attributed graphs, we can leverage a new paradigm for the evolution of node features with the help of curvature. We show that in the attributed graphs, DRF guarantees a vital property: The curvature of each edge concentrates toward zero over time. This property leads to two interesting consequences: 1) graph Dirichlet energy with bilateral bounds and 2) data-independent curvature decay rate. Based on these theoretical results, we propose the Graph Neural Ricci Flow (GNRF), a novel curvature-aware continuous-depth GNN. Compared to traditional curvature-based graph learning methods, GNRF is not limited to a specific curvature definition. It computes and adjusts time-varying curvature efficiently in linear time. We also empirically illustrate the operating mechanism of GNRF and verify that it performs excellently on diverse datasets.

Generalized Category Discovery (GCD) aims to identify both known and novel categories in unlabeled data by leveraging knowledge from old classes. However, existing methods are limited to non-graph data; lack theoretical foundations to answer *When and how known classes can help GCD*. We introduce the Graph GCD task; provide the first rigorous theoretical analysis of *parametric GCD*. By quantifying the relationship between old and new classes in the embedding space using the Wasserstein distance W, we derive the first provable GCD loss bound based on W. This analysis highlights two necessary conditions for effective GCD. However, we uncover, through a Pairwise Markov Random Field perspective, that popular graph contrastive learning (GCL) methods inherently violate these conditions. To address this limitation, we propose SWIRL, a novel GCL method for GCD. Experimental results validate our (theoretical) findings and demonstrate SWIRL's effectiveness.

Federated graph learning (FGL) aims to collaboratively train a global graph neural network (GNN) on multiple private graphs with preserving the local data privacy. Besides the common cases of data heterogeneity in conventional federated learning, FGL faces the unique challenge of topology heterogeneity. Most of existing FGL methods alleviate the negative impact of heterogeneity by introducing global signals. However, the manners of creating increments might not be effective and significantly increase the computation amount. In light of this, we propose the FedATH, an FGL method with Alleviating Topology Heterogeneity from a causal perspective. Inspired by the causal theory, we argue that not all edges in a topology are necessary for the training objective, less topology information might make more sense. With the aid of edge evaluator, the local graphs are divided into causal and biased subgraphs. A dual-GNN architecture is used to encode the two subgraphs into corresponding representations. Thus, the causal representations are drawn closer to the training objective while the biased representations are pulled away from it. Further, the Hilbert-Schmidt Independence Criterion is employed to strengthen the separability of the two subgraphs. Extensive experiments on six real-world graph datasets are conducted to demonstrate the superiority of the proposed FedATH over the compared approaches.

To defend against privacy leakage of user data, differential privacy is widely used in federated learning, but it is not free. The addition of noise randomly disrupts the semantic integrity of the model and this disturbance accumulates with increased communication rounds. In this paper, we introduce a novel federated learning framework with rigorous privacy guarantees, named FedCEO, designed to strike a trade-off between model utility and user privacy by letting clients "C*ollaborate with Each Other". Specifically, we perform efficient tensor low-rank proximal optimization on stacked local model parameters at the server, demonstrating its capability to flexibly truncate high-frequency components in spectral space. This capability implies that our FedCEO can effectively recover the disrupted semantic information by smoothing the global semantic space for different privacy settings and continuous training processes. Moreover, we improve the SOTA utility-privacy trade-off bound by order of $\sqrt{d}$, where $d$ is the input dimension. We illustrate our theoretical results with experiments on representative datasets and observe significant performance improvements and strict privacy guarantees under different privacy settings. The *code is available at https://github.com/6lyc/FedCEO_Collaborate-with-Each-Other.

Multi-view graph clustering (MVGC) has been of widespread interest owing to the ability of capturing the complementary information among views, thereby enhancing the performance of node clustering. Despite the impressive achievements of existing methods, they are limited by a common deficiency, namely, the curse of local manifold while failing to perceive the global manifold structure. In light of this drawback, we propose a Consistent Context-Aware Representation Learning (CCARL) method for MVGC, aiming to learn node representations from global space rather than just local topology. Concretely, we define a set of anchors to establish the global coordinate, which are optimally mapped to multi-view graphs with minimal cost via fused Gromov-Wasserstein optimal transport. To fuse the complementary information in various views, the attention mechanism is employed to integrate multiple graph embeddings into a consistent representation. By transforming to the global coordinate connecting with anchors, the consistent representation captures the contextual information, and its clustering-friendliness is further enhanced through a self-training strategy. Finally, extensive experiments on four multi-view graph datasets demonstrate the effectiveness of the proposed CCARL over existing MVGC methods.

Federated graph learning is focused on aggregating knowledge from multi-source graph data and training graph neural networks. Unlike the data that traditional federated learning needs to deal with, federated graph learning also needs to face additional topological information. Further, there are also biases in features and topologies among clients, increasing the difficulty of training models. Previous methods usually seek global calibration information, however, this approach may suffer from information bias caused by data skews, and it is also difficult to naturally combine feature and topology information. Therefore, adjusting the bias before it occurs will hopefully address the learning difficulties caused by the skew. In view of this, we employ background graph data, which works as reference information for local training, to proactively correct bias before it occurs. As a kind of graph data, background graphs are naturally capable of combining feature and topology information to accomplish bias correction among clients in a comprehensive way. Mixing strategy is employed on the background graph to additionally provide privacy-preserving capabilities. Graph generation methods are employed to restore the diversity of background graphs that are blurred by the mixing strategy. Extensive experiments on two real-world datasets demonstrate the sufficient motivation and effectiveness of the proposed method.

Federated domain generalization addresses the crucial challenge of developing models that can generalize across diverse domains while maintaining data privacy in federated learning settings. Current approaches either compromise privacy constraints or focus narrowly on specific aspects of model invariance, often incurring significant computational overhead. We propose a novel approach FedDIM, which leverages the concept of “insight matrix” - a fine-grained representation of the model's decision-making process derived from element-wise products between feature vectors and classifier weights. By introducing a regularization term that promotes consistency between individual sample insight matrices and their class-wise mean representations, our method effectively captures both feature and classifier invariance. This approach not only maintains strict privacy requirements but also introduces minimal computational overhead as it utilizes intermediate computations already present in the forward pass. Extensive experiments demonstrate that our method achieves superior out-of-distribution generalization compared to existing federated learning approaches while being simple to implement. Our work provides a new perspective on achieving robust generalization in federated learning settings through the lens of decision-making processes.

Inspired by the workings of biological brains, humans have designed artificial neural networks (ANNs), sparking profound advancements across various fields. However, the biological brain possesses high plasticity, enabling it to develop simple, efficient, and powerful structures to cope with complex external environments. In contrast, the superior performance of ANNs often relies on meticulously crafted architectures, which can make them vulnerable when handling complex inputs. Moreover, overparameterization often characterizes the most advanced ANNs. This paper explores the path toward building streamlined and plastic ANNs. Firstly, we introduce the Graph Perceptron (GP), which extends the most fundamental ANN, the Multi-Layer Perceptron (MLP). Subsequently, we incorporate a self-assembly mechanism on top of GP called Self-Assembling Graph Perceptron (SAGP). During training, SAGP can autonomously adjust the network's number of neurons and synapses and their connectivity. SAGP achieves comparable or even superior performance with only about 5% of the size of an MLP. We also demonstrate the SAGP's advantages in enhancing model interpretability and feature selection.

Graph classification has long assumed a closed-world setting, limiting its applicability to real-world scenarios where new categories often emerge. To address this limitation, we introduce Graph-Level Novel Category Discovery (GLNCD), a new task aimed at identifying unseen graph categories without supervision from novel classes. We first adapt classical Novel Category Discovery (NCD) methods for images to the graph domain and evaluate these baseline methods on four diverse graph datasets curated for the GLNCD task. Our analysis reveals that these methods suffer a notable performance degradation compared to their image-based counterparts, due to two key challenges: (1) insufficient utilization of structural information in graph self-supervised learning (SSL), and (2) ineffective pseudo-labeling strategies based on ranking statistics (RS) that neglect graph structure. To alleviate these issues, we propose ProtoFGW-NCD, a framework consisting of two core components: ProtoFGW-CL, a novel graph SSL framework, and FGW-RS, a structure-aware pseudo-labeling method. Both components employ a differentiable Fused Gromov-Wasserstein (FGW) distance to effectively compare graphs by incorporating structural information. These components are built upon learnable prototype graphs, which enable efficient, parallel FGW-based graph comparisons and capture representative patterns within graph datasets. Experiments on four GLNCD benchmark datasets demonstrate the effectiveness of ProtoFGW-NCD.

Federated graph learning (FGL) is a privacy-preserving paradigm for modeling distributed graph data, designed to train a powerful global graph neural network. Existing FGL methods predominantly rely on label information during training, effective FGL in an unsupervised setting remains largely unexplored territory. In this paper, we address two key challenges in unsupervised FGL: 1) Local models tend to converge in divergent directions due to the lack of shared semantic information across clients. Then, how to align representation spaces among multiple clients is the first challenge. 2) Conventional federated weighted aggregation easily results in degrading the performance of the global model, then which raises another challenge, namely how to adaptively learn the global model parameters. In response to the two questions, we propose a tailored framework named FedPAM, which is composed of two modules: Representation Space Alignment (RSA) and Adaptive Global Parameter Learning (AGPL). RSA leverages a set of learnable anchors to define the global representation space, then local subgraphs are aligned with them through the fused Gromov-Wasserstein optimal transport, achieving the representation space alignment across clients. AGPL stacks local model parameters into third-order tensors, and adaptively integrates the global model parameters in a low-rank tensor space, which facilitates to fuse the high-order knowledge among clients. Extensive experiments on eight graph datasets are conducted, the results demonstrate that the proposed FedPAM is superior over classical and SOTA compared methods.

Federated learning enables collaborative training while preserving the privacy of all participants. However, the heterogeneity in data distribution across multiple training nodes poses significant challenges to the construction of federated models. Prior studies were dedicated to mitigating the effects of data heterogeneity by using global information as a blueprint and restricting the local update of the model for reaching a "hard consensus". But this practice makes it difficult to balance local and global information, and it neglects to negotiate amicably between local and global models to reach mutually agreeable results, called ``soft consensus". In this paper, a multiple-path solving method is proposed to balance global and local features and combine these two feature preference paths to reach a soft consensus. Rather than relying on global information as the sole criterion, a negotiation process is employed to address the same objective by accommodating diverse feature preferences, thereby facilitating the discovery of a more plausible solution through multiple distinct pathways. Considering the overwhelming power of local features during local training, a swapping strategy is applied to weaken them to balance the solution paths. Moreover, to minimize the additional communication cost caused by the introduction of multiple paths, the solution of the task network is converted into data adaptation to reduce the amount of parameter transmission. Extensive experiments are conducted to demonstrate the advantages of the proposed method.

In the field of heterogeneous federated learning (FL), the key challenge is to efficiently and collaboratively train models across multiple clients with different data distributions, model structures, task objectives, computational capabilities, and communication resources. This diversity leads to significant heterogeneity, which increases the complexity of model training. In this paper, we first outline the basic concepts of heterogeneous FL and summarize the research challenges in FL in terms of five aspects: data, model, task, device and communication. In addition, we explore how existing state-of-the-art approaches cope with the heterogeneity of FL, and categorize and review these approaches at three different levels: data-level, model-level, and architecture-level. Subsequently, the paper extensively discusses privacy-preserving strategies in heterogeneous FL environments. Finally, the paper discusses current open issues and directions for future research, aiming to promote the further development of heterogeneous FL.

Vertical federated learning (VFL) allows parties to build robust shared machine learning models based on learning from distributed features of the same samples, without exposing their own data. However, current VFL solutions are limited in their ability to perform inference on non-overlapping samples, and data stored on clients is often subject to loss due to various unavoidable factors. This leads to incomplete client data, where client missing features (MF) are frequently overlooked in VFL. The main aim of this paper is to propose a VFL framework to handle missing features (MFVFL), which is a tensor decomposition network-based approach that can effectively learn intra- and inter-client feature information from client data with missing features to improve VFL performance. In the proposed MFVFL method each client imputes missing values and encodes features to learn intra-feature information, and the server collects the uploaded feature embeddings as input to our developed low-rank tensor decomposition network to learn inter-feature information. Finally, the server aggregates the representations from tensor decomposition to train a global classifier. In the paper, we theoretically guarantee the convergence of MFVFL. In addition, differential privacy (DP) for data privacy protection is always used, and the proposed framework (MFVFL-DP) can deal with such degraded data by using a tensor robust PCA to alleviate the impact of noise while preserving data privacy. We conduct extensive experiments on six datasets of different sample sizes and feature dimensions, and demonstrate that MFVFL significantly outperforms state-of-the-art methods, especially under high missing ratios. The experimental results also show that MFVFL-DP possesses excellent denoising capabilities and illustrate that the noisy effect by the DP mechanism can be alleviated.

Gradient Inversion Attacks invert the transmitted gradients in Federated Learning (FL) systems to reconstruct the sensitive data of local clients and have raised considerable privacy concerns. A majority of gradient inversion methods rely heavily on explicit prior knowledge (e.g., a well pre-trained generative model), which is often unavailable in realistic scenarios. This is because real-world client data distributions are often highly heterogeneous, domain-specific, and unavailable to attackers, making it impractical for attackers to obtain perfectly matched pre-trained models, which inevitably suffer from fundamental distribution shifts relative to target private data. To alleviate this issue, researchers have proposed to leverage the implicit prior knowledge of an over-parameterized network. However, they only utilize a fixed neural architecture for all the attack settings. This would hinder the adaptive use of implicit architectural priors and consequently limit the generalizability. In this paper, we further exploit such implicit prior knowledge by proposing Gradient Inversion via Neural Architecture Search (GI-NAS), which adaptively searches the network and captures the implicit priors behind neural architectures. Extensive experiments verify that our proposed GI-NAS can achieve superior attack performance compared to state-of-the-art gradient inversion methods, even under more practical settings with high-resolution images, large-sized batches, and advanced defense strategies. To the best of our knowledge, we are the first to successfully introduce NAS to the gradient inversion community. We believe that this work exposes critical vulnerabilities in real-world federated learning by demonstrating high-fidelity reconstruction of sensitive data without requiring domain-specific priors, forcing urgent reassessment of FL privacy safeguards.

Unsupervised multi-source domain transfer in federated scenario has become an emerging research direction, which can help unlabeled target domain to obtain the adapted model through source domains under privacy-preserving. However, when local data are graph, the difference of domains (or data heterogeneity) mainly originates from the difference in node attributes and sub-graph structures, leading to serious model drift, which is not considered by the existing related algorithms. Currently, there are two challenges in this scenario: (1) The node representations extracted directly through conventional GNNs lack inter-domain generalized and consistent information, making it difficult to apply existing federated learning algorithms. (2) The knowledge of source domains has quality differences, which may lead to negative transfer. To address these issues, we propose a novel two-phase Federated Graph Transfer Learning (FGTL) framework. In the generalization phase, FGTL utilizes local contrastive learning and global context embedding to force node representations to capture the inter-domain generalized and consistent information, lightly alleviating model drift. In the transfer phase, FGTL utilizes consensus knowledge to force the decision bound of classifier to adapt to the target client. In addition, FGTL+ exploits model grouping to make consensus knowledge generation more efficient, further enhancing the scalability of FGTL. Extensive experiments show that FGTL significantly outperforms state-of-the-art related methods, while FGTL+ further enhances privacy protection and reduces both communication and computation overhead.

Federated learning provides a privacy-preserving modeling schema for distributed data, which coordinates multiple clients to collaboratively train a global model. However, data stored in different clients may be collected from diverse domains, and the resulting feature shift is prone to the degraded performance of global model. In this paper, we propose a Federated Domain-Independent Prototype Learning (FedDP) method with Alignments of Representation and Parameter Spaces for Feature Shift. Concretely, FedDP aims to eliminate the domain-specific information and explore the pure representations via information bottleneck, thus integrating the local and global domain-independent prototypes, respectively. To align the cross-domain representation spaces, the global domain-independent prototypes serve as the supervised signals to enable local intra-class representations to approach them. Further, to mitigate the divergences of optimization directions between multiple clients induced by the feature shift, the global representations are yielded by the global model on the client-side and guide the learning of local representations, thus unifying the parameter spaces of multiple local models. We derive the theoretical lower bound of the optimization objective based on mutual information, which is transformed into a computable loss. The proposed FedDP can be applied in the scenarios of homogeneous and heterogeneous models. Extensive experiments are conducted on three challenging multi-domain datasets. The experimental results illustrate the superiority of FedDP compared with state-of-the-art federated learning methods.

Federated learning, as a privacy-preserving learning paradigm, restricts the access to data of each local client, for protecting the privacy of the parties. However, in the case of heterogeneous data settings, the different data distributions among clients usually lead to the divergence of learning targets, which is an essential challenge for federated learning. In this article, we propose a federated learning framework with a unified coding space, called FedUCS, for learning cross-client uniform coding rules to solve the problem of divergent targets among multiple clients due to heterogeneous data. A cross-client coordinator co-trained by multiple clients is used as a criterion of the coding space to supervise all clients coding to a uniform space, which is the significant contribution of this article. Furthermore, in order to appropriately retain historical information and avoid forgetting previous knowledge, a partial memory mechanism is applied. Moreover, in order to further enhance the distinguishability of the unified encoding space, supervised contrastive learning is used to avoid the intersection of the encoding spaces belonging to different categories. A series of experiments are performed to verify the effectiveness of the proposed method in a federated learning setting with heterogeneous data.

Graph neural networks (GNNs) have achieved state-of-the-art performance in various graph representation learning scenarios. However, when applied to graph data in real world, GNNs have encountered scalability issues. Existing GNNs often have high computational load in both training and inference stages, making them incapable of meeting the performance needs of large-scale scenarios with a large number of nodes. Although several studies on scalable GNNs have developed, they either merely improve GNNs with limited scalability or come at the expense of reduced effectiveness. Inspired by knowledge distillation's (KDs) achievement in preserving performances while balancing scalability in computer vision and natural language processing, we propose an enhanced scalable GNN via KD (KD-SGNN) to improve the scalability and effectiveness of GNNs. On the one hand, KD-SGNN adopts the idea of decoupled GNNs, which decouples feature transformation and feature propagation in GNNs and leverages preprocessing techniques to improve the scalability of GNNs. On the other hand, KD-SGNN proposes two KD mechanisms (i.e., soft-target (ST) distillation and shallow imitation (SI) distillation) to improve the expressiveness. The scalability and effectiveness of KD-SGNN are evaluated on multiple real datasets. Besides, the effectiveness of the proposed KD mechanisms is also verified through comprehensive analyses.

Message passing (MP) is a popular paradigm for designing graph neural networks (GNNs), which iteratively aggregates neighbor information and updates node embeddings. However, this paradigm suffers from several issues: First, long-range information struggles to be fully utilized, known as over-squashing. Second, excessive MP layers lead to indistinguishable representations, referred to as over-smoothing. Finally, vanilla MPNNs fail to meet the ability of training in heterophilic graphs. In this paper, we provide a unified insight into these defects: node embeddings are sent to neighbors at a constant "pace" and are aggregated immediately. Such synchronicity causes embeddings closer to the output to be more important, i.e. local priority, manifesting the aforementioned issues. Based on this, Asyn-MPNN, an asynchronous framework that customizes the speed of information aggregation, is proposed, which can unify many popular GNNs. We further propose the automated asynchronous (a Asyn) layer, which achieves effects similar to Asyn-MPNN but without introducing extra hyperparameters and can be integrated into any GNN. aAsyn-MPNN validates its performance through extensive experiments on both graph-level and node-level tasks and achieves leading results on tasks from long-range graph benchmark.

Federated learning enables multiple clients to collaboratively train a global model without revealing their local data. However, conventional federated learning often overlooks the fact that data stored on different clients may originate from diverse domains, and the resulting domain shift problem can significantly impair the performance of the global model. In this paper, we introduce Federated Semantic Prototype Learning (FedSeProto), a semantic prototype-based approach designed to address the domain shift issue in federated learning. The proposed method comprises two components: feature decoupling and feature alignment. Feature decoupling aims to learn semantic prototypes that can represent semantic information associated with specific categories, while feature alignment utilizes these semantic prototypes to facilitate learning of cross-client consistent features. Two key techniques are employed to achieve feature decoupling. On one hand, feature separation is achieved through the minimization of mutual information between semantic and domain features. On the other hand, the knowledge distillation is leveraged to ensure that both semantic and domain features carry the correct information. For feature alignment, intra-class semantic features are used to generate the local prototypes, which are further aggregated to the global prototypes. These global prototypes serve as guides during the local training process. Specifically, the local intra-class semantic features are driven to close to the corresponding global prototypes, thereby encouraging all clients to learn the globally consistent semantic features. Comprehensive experiments conducted on four challenging multi-domain datasets demonstrate the effectiveness of the proposed method compared with existing federated learning algorithms.

Recently, nonnegative matrix factorization (NMF) has been widely adopted for community detection, because of its better interpretability. However, the existing NMF-based methods have the following three problems: 1) they directly transform the original network into community membership space, so it is difficult for them to capture the hierarchical information; 2) they often only pay attention to the topology of the network and ignore its node attributes; 3) it is hard for them to learn the global structure information necessary for community detection. Therefore, we propose a new community detection algorithm, named Contrastive Deep Nonnegative Matrix Factorization (CDNMF). Firstly, we deepen NMF to strengthen its capacity for information extraction. Subsequently, inspired by contrastive learning, our algorithm creatively constructs network topology and node attributes as two contrasting views. Furthermore, we utilize a debiased negative sampling layer and learn node similarity at the community level, thereby enhancing the suitability of our model for community detection. We conduct experiments on three public real graph datasets and the proposed model has achieved better results than state-of-the-art methods. Code available at https://github.com/6lyc/CDNMF.git.

Knowledge graph reasoning aims at predicting the missing facts from the existing structural and semantic information contained in the graphs. A typical approach to infer new facts is reinforcement learning method, which demonstrates the reasoning process through multi-hop paths. However, previous works tacitly approve such reasoning paths make sense, neglecting the fact that the relational paths connecting two entities may lack explainable meaning. To address the above challenge, we propose a novel knowledge graph reasoning method that leverages logical rules to improve interpretability and reliability. It makes full use of logical rules through a two-stage training framework, where the rule memorization stage learn the reasoning patterns from rules, and then the strategy improvement stage to further enhance the inference policy to obtain correct answers. Experimental results on several real-world datasets show that our method achieves excellent performance, and logical rules help to improve the interpretability of reasoning results.

Graph contrastive learning (GCL) enables graph neural networks (GNNs) to learn generalized node features and achieve better performance on downstream tasks. Meanwhile, with the rapid growth of social media, signed graphs have attracted people’s attention, for they can represent complex relationships among people with positive and negative links. Many efforts for unsigned GCL have been made, while signed GCL is rarely explored. This paper argues that unsigned GCL cannot be directly transferred to signed graphs for two reasons. Firstly, unsigned GCL performs random augmentation on the input graph to generate multi-views for contrast, destroying signed graphs’ balance property. Secondly, when the balance is broken, relationships between nodes become ambiguous, leading to over-smoothing between different channels. Therefore, we propose a novel signed GCL model called Signed Graph Contrastive Learning with Adaptive Augmentation (SGCA) to solve these problems. Specifically, we use a learnable module to generate balanced augmented views adaptively. At the same time, we add a constraint to the signed graph encoder to alleviate the over-smoothing problem caused by the balance violation. Various experiments demonstrate the superiority of SGCA over 12 competitive methods on 5 real-world datasets.

The heterogeneity issue in federated learning (FL) has attracted increasing attention, which is attempted to be addressed by most existing methods. Currently, due to systems and objectives heterogeneity, enabling clients to hold models of different architectures and tasks of different demands has become an important direction in FL. Most existing FL methods are based on the homogeneity assumption, namely, different clients have the same architectural models with the same tasks, which are unable to handle complex and multivariate data and tasks. To flexibly address these heterogeneity limitations, we propose a novel federated multi-task learning framework with the help of tensor trace norm, FedSAK. Specifically, it treats each client as a task and splits the local model into a feature extractor and a prediction head. Clients can flexibly choose shared structures based on heterogeneous situations and upload them to the server, which learns correlations among client models by mining model low-rank structures through tensor trace norm. Furthermore, we derive convergence and generalization bounds under non-convex settings. Evaluated on 6 real-world datasets compared to 13 advanced FL models, FedSAK demonstrates superior performance.

Abstract Anomaly detection on attributed graphs is a crucial topic for practical applications. Existing methods suffer from semantic mixture and imbalance issue because they commonly optimize the model based on the loss function for anomaly discrimination, mainly focusing on anomaly discrimination and ignoring representation learning. Graph Neural networks based techniques usually tend to map adjacent nodes into close semantic space. However, anomalous nodes commonly connect with numerous normal nodes directly, conflicting with the assortativity assumption. Additionally, there are far fewer anomalous nodes than normal nodes, leading to the imbalance problem. To address these challenges, a unique algorithm, decoupled self-supervised learning for anomaly detection (DSLAD), is proposed in this paper. DSLAD is a self-supervised method with anomaly discrimination and representation learning decoupled for anomaly detection. DSLAD employs bilinear pooling and masked autoencoder as the anomaly discriminators. By decoupling anomaly discrimination and representation learning, a balanced feature space is constructed, in which nodes are more semantically discriminative, as well as imbalance issue can be resolved. Experiments conducted on various six benchmark datasets reveal the effectiveness of DSLAD.

Avatars, as promising digital assistants in Vehicular Metaverses, can enable drivers and passengers to immerse in 3-D virtual spaces, serving as a practical emerging example of Artificial Intelligence of Things (AIoT) in intelligent vehicular environments. The immersive experience is achieved through seamless human–avatar interaction, e.g., augmented reality navigation, which requires intensive resources that are inefficient and impractical to process on intelligent vehicles locally. Fortunately, offloading avatar tasks to roadside units (RSUs) or cloud servers for remote execution can effectively reduce resource consumption. However, the high mobility of vehicles, the dynamic workload of RSUs, and the heterogeneity of RSUs pose novel challenges to making avatar migration decisions. To address these challenges, in this article, we propose a dynamic migration framework for avatar tasks based on real-time trajectory prediction and multiagent deep reinforcement learning (MADRL). Specifically, we propose a model to predict the future trajectories of intelligent vehicles based on their historical data, indicating the future workloads of RSUs. Based on the expected workloads of RSUs, we formulate the avatar task migration problem as a long-term mixed-integer programming problem. To tackle this problem efficiently, the problem is transformed into a partially observable Markov decision process (POMDP) and solved by multiple DRL agents with hybrid continuous and discrete actions in decentralized. Numerical results demonstrate that our proposed algorithm can effectively reduce the latency of executing avatar tasks by around 25% without prediction and 30% with prediction and enhance user immersive experiences in the AIoT-enabled Vehicular Metaverse (AeVeM).

Most multi-view clustering methods based on shallow models are limited in sound nonlinear information perception capability, or fail to effectively exploit complementary information hidden in different views. To tackle these issues, we propose a novel Subspace-Contrastive Multi-View Clustering (SCMC) approach. Specifically, SCMC utilizes a set of view-specific auto-encoders to map the original multi-view data into compact features capturing its nonlinear structures. Considering the large semantic gap of data from different modalities, we project multiple heterogeneous features into a joint semantic space, namely the embedded compact features are passed through the self-expression layers to learn the subspace representations, respectively. In order to enhance the discriminability and efficiently excavate the complementarity of various subspace representations, we use the contrastive strategy to maximize the similarity between positive pairs while differentiate negative pairs. Thus, the graph regularization is employed to encode the local geometric structure within varying subspaces for optimizing the consistent affinity matrix. Furthermore, to endow the proposed SCMC with the ability of handling the multi-view out-of-samples, we develop a consistent sparse representation (CSR) learning mechanism over the in-samples. To demonstrate the effectiveness of the proposed model, we conduct a large number of comparative experiments on ten challenging datasets, and the experimental results show that SCMC outperforms existing shallow and deep multi-view clustering methods. In addition, the experimental results on out-of-samples illustrate the effectiveness of the proposed CSR.

With the development of multi-view learning, multi-view outlier detection has received increasing attention in recent years. However, the current research still faces two challenges: (1) The current research lacks theoretical analysis tools for multi-view outliers. (2) Most current multi-view outlier detection algorithms are based on shallow structural assumptions of the data, such as cluster assumptions and subspace assumptions, thus they are not suitable for more complex data distributions. In addressing these two issues, this article proposes three occurrence mechanisms of multi-view outlier, which serve as foundational theoretical analysis tools for multi-view outliers. Utilizing proposed mechanisms, we analyze the impact of multi-view outliers and the information structure of multi-view data and validate our findings through experiments. Finally, we propose a novel algorithm referred to as Information-Aware Multi-View Outlier Detection (IAMOD). In contrast to other methods, IAMOD focuses on the information structure of multi-view data without relying on shallow structural assumptions. By learning a compact representation of the sample that is semantically rich and non-redundant, IAMOD can accurately identify multi-view outliers by comparing the consistency of the representations’ neighbors and views. Extensive experimental results demonstrate that our approach outperforms several state-of-the-art multi-view outlier detection methods.

Federated Learning (FL) provides a novel paradigm for privacy-preserving machine learning, enabling multiple clients to collaborate on model training without sharing private data. To handle multi-source heterogeneous data, Vertical Federated Learning (VFL) has been extensively investigated. However, in the context of VFL, the label information tends to be kept in one authoritative client and is very limited. This poses two challenges for model training in the VFL scenario. On the one hand, a small number of labels cannot guarantee to train a well VFL model with informative network parameters, resulting in unclear boundaries for classification decisions. On the other hand, the large amount of unlabeled data is dominant and should not be discounted, and it is worthwhile to focus on how to leverage them to improve representation modeling capabilities. To address the preceding two challenges, we first introduce supervised contrastive loss to enhance the intra-class aggregation and inter-class estrangement, which is to deeply explore label information and improve the effectiveness of downstream classification tasks. Then, for unlabeled data, we introduce a pseudo-label-guided consistency mechanism to induce the classification results coherent across clients, which allows the representations learned by local networks to absorb the knowledge from other clients, and alleviates the disagreement between different clients for classification tasks. We conduct sufficient experiments on four commonly used datasets, and the experimental results demonstrate that our method is superior to the state-of-the-art methods, especially in the low-label rate scenario, and the improvement becomes more significant.

As special information carriers containing both structure and feature information, graphs are widely used in graph mining, e.g., Graph Neural Networks (GNNs). However, graph data are stored separately in multiple distributed parties in some practical scenarios, which may not be directly shared due to conflicts of interest. Hence, federated graph neural networks are proposed to address such data silo issues while preserving each party’s privacy (or client). Nevertheless, different graph data distributions of various parties, which is known as the statistical heterogeneity, may degrade the performance of naive federated learning algorithms like FedAvg. In this article, we propose FedEgo, a federated graph learning framework based on ego-graphs to tackle the challenges above, in which each client will train their local models while also contributing to the training of a global model. FedEgo applies GraphSAGE over ego-graphs to make full use of the structure information and utilizes Mixup for privacy concerns. To deal with the statistical heterogeneity, we integrate personalization into learning and propose an adaptive mixing coefficient strategy that enables clients to achieve their optimal personalization. Extensive experimental results and in-depth analysis demonstrate the effectiveness of FedEgo.

Graph contrastive learning defines a contrastive task to pull similar instances close and push dissimilar instances away. It learns discriminative node embeddings without supervised labels, which has aroused increasing attention in the past few years. Nevertheless, existing methods of graph contrastive learning ignore the differences between diverse semantics existed in graphs, which learn coarse-grained node embeddings and lead to sub-optimal performances on downstream tasks. To bridge this gap, we propose a novelFine-grainedSemantics enhancedGraphContrastiveLearning (FSGCL) in this paper. Concretely, FSGCL first introduces a motif-based graph construction, which employs graph motifs to extract diverse semantics existed in graphs from the perspective of input data. Then, the semantic-level contrastive task is explored to further enhance the utilization of fine-grained semantics from the perspective of model training. Experiments on five real-world datasets demonstrate the superiority of our proposed FSGCL over state-of-the-art methods. To make the results reproducible, we will make our codes public on GitHub after this paper is accepted.

Tensor analysis has received widespread attention in high-dimensional data learning. Unfortunately, the tensor data are often accompanied by arbitrary signal corruptions, including missing entries and sparse noise. How to recover the characteristics of the corrupted tensor data and make it compatible with the downstream clustering task remains a challenging problem. In this article, we study a generalized transformed tensor low-rank representation (TTLRR) model for simultaneously recovering and clustering the corrupted tensor data. The core idea is to find the latent low-rank tensor structure from the corrupted measurements using the transformed tensor singular value decomposition (SVD). Theoretically, we prove that TTLRR can recover the clean tensor data with a high probability guarantee under mild conditions. Furthermore, by using the transform adaptively learning from the data itself, the proposed TTLRR model can approximately represent and exploit the intrinsic subspace and seek out the cluster structure of the tensor data precisely. An effective algorithm is designed to solve the proposed model under the alternating direction method of multipliers (ADMMs) algorithm framework. The effectiveness and superiority of the proposed method against the compared methods are showcased over different tasks, including video/face data recovery and face/object/scene data clustering.

Anomaly detection plays a vital role as a crucial security measure for edge devices in Artificial Intelligence and Internet of Things (AIoT). With the rapid development of IoT ( Internet of Things), changes in system configurations and the introduction of new devices can lead to significant alterations in device relationships and data flows within the IoT, thereby triggering concept drift. Previously trained anomaly detection models fail to adapt to the changed distribution of streaming data, resulting in a high number of false positive events. This paper aims to address the issue of concept drift in IoT anomaly detection by proposing a comprehensive Concept Drift Detection, Interpretation, and Adaptation framework (CDDIA). We focus on accurately capturing the concept drift of normal data in unsupervised scenarios. To interpret drift samples, we integrate a search optimization algorithm and the SHAP method, providing a comprehensive interpretation of drift samples at both the sample and feature levels. Simultaneously, by utilizing the sample-level interpretation results for filtering new and old samples, we retrain the anomaly detection model to mitigate the impact of concept drift and reduce the false positive rate. This integrated strategy demonstrates significant advantages in maintaining model stability and reliability. The experimental results indicate that our method outperforms five baseline methods in adaptability across three datasets and provides interpretability for samples experiencing concept drift.

In recent years, federated learning has achieved remarkable results in the medical and financial fields, but various attacks have always plagued federated learning. Data poisoning attack and defense research in horizontal federated learning are sufficient, yet vertical federated data poisoning attack and defense remains an open area due to two challenges: (1) Complex data distributions lead to immense attack possibilities, and (2) defense methods are insufficient for complex data distributions. We have discovered that from the perspective of information theory, the above challenges can be addressed elegantly and succinctly with a solution. We first reveal the information-theoretic mechanisms underlying vertical federated data poisoning attacks and then propose an unsupervised vertical federated data poisoning defense method (VFedAD) based on information theory. VFedAD learns semantic-rich client data representations through contrastive learning task and cross-client prediction task to identify anomalies. Experiments show VFedAD effectively detects vertical federated anomalies, protecting subsequent algorithms from vertical federated data poisoning attacks.

In deep multi-view clustering, three intractable problems are posed ahead of researchers, namely, the complementarity exploration problem, the information preservation problem, and the cluster structure discovery problem. In this paper, we consider the deep multi-view clustering from the perspective of mutual information (MI), and attempt to address the three important concerns with a Mutual Information-Driven Multi-View Clustering (MIMC) method, which extracts the common and view-specific information hidden in multi-view data and constructs a clustering-oriented comprehensive representation. Specifically, three constraints based on MI are devised in response to three issues. Correspondingly, we minimize the MI between the common representation and view-specific representations to exploit the inter-view complementary information. Further, we maximize the MI between the refined data representations and original data representations to preserve the principal information. Moreover, to learn a clustering-friendly comprehensive representation, the MI between the comprehensive embedding space and cluster structure is maximized. Finally, we conduct extensive experiments on six benchmark datasets, and the experimental results indicate that the proposed MIMC outperforms other clustering methods.

With the rapidly growing attention to multi-view data in recent years, multi-view outlier detection has become a rising field with intense research. These researches have made some success, but still exist some issues that need to be solved. First, many multi-view outlier detection methods can only handle datasets that conform to the cluster structure but are powerless for complex data distributions such as manifold structures. This overly restrictive data assumption limits the applicability of these methods. In addition, almost the majority of multi-view outlier detection algorithms cannot solve the online detection problem of multi-view outliers. To address these issues, we propose a new detection method based on the local similarity relation and data reconstruction, i.e., the Self-Representation Method with Local Similarity Preserving for fast multi-view outlier detection (SRLSP). By using the local similarity structure, the proposed method fully utilizes the characteristics of outliers and detects outliers with an applicable objective function. Besides, a well-designed optimization algorithm is proposed, which completes each iteration with linear time complexity and can calculate each instance parallelly. Also, the optimization algorithm can be easily extended to the online version, which is more suitable for practical production environments. Extensive experiments on both synthetic and real-world datasets demonstrate the superiority of the proposed method on both performance and time complexity.

Graph convolutional networks (GCNs) have achieved great success in many applications and have caught significant attention in both academic and industrial domains. However, repeatedly employing graph convolutional layers would render the node embeddings indistinguishable. For the sake of avoiding oversmoothing, most GCN-based models are restricted in a shallow architecture. Therefore, the expressive power of these models is insufficient since they ignore information beyond local neighborhoods. Furthermore, existing methods either do not consider the semantics from high-order local structures or neglect the node homophily (i.e., node similarity), which severely limits the performance of the model. In this article, we take above problems into consideration and propose a novel Semantics and Homophily preserving Network Embedding (SHNE) model. In particular, SHNE leverages higher order connectivity patterns to capture structural semantics. To exploit node homophily, SHNE utilizes both structural and feature similarity to discover potential correlated neighbors for each node from the whole graph; thus, distant but informative nodes can also contribute to the model. Moreover, with the proposed dual-attention mechanisms, SHNE learns comprehensive embeddings with additional information from various semantic spaces. Furthermore, we also design a semantic regularizer to improve the quality of the combined representation. Extensive experiments demonstrate that SHNE outperforms state-of-the-art methods on benchmark datasets.

Multi-view clustering (MVC) aims at exploiting the consistent features within different views to divide samples into different clusters. Existing subspace-based MVC algorithms usually assume linear subspace structures and two-stage similarity matrix construction strategies, thereby posing challenges in imprecise low-dimensional subspace representation and inadequacy of exploring consistency. This paper presents a novel hierarchical representation for MVC method via the integration of intra-sample, intra-view, and inter-view representation learning models. In particular, we first adopt the deep autoencoder to adaptively map the original high-dimensional data into the latent low-dimensional representation of each sample. Second, we use the self-expression of the latent representation to explore the global similarity between samples of each view and obtain the subspace representation coefficients. Third, we construct the third-order tensor by arranging multiple subspace representation matrices and impose the tensor low-rank constraint to sufficiently explore the consistency among views. Being incorporated into a unified framework, these three models boost each other to achieve a satisfactory clustering result. Moreover, an alternating direction method of multipliers algorithm is developed to solve the challenging optimization problem. Extensive experiments on both simulated and real-world multi-view datasets show the superiority of the proposed method over eight state-of-the-art baselines.

Graph Neural Networks (GNNs) have exhibited their powerful ability of tackling nontrivial problems on graphs. However, as an extension of deep learning models to graphs, GNNs are vulnerable to noise or adversarial attacks due to the underlying perturbations propagating in message passing scheme, which can affect the ultimate performances dramatically. Thus, it's vital to study a robust GNN framework to defend against various perturbations. In this paper, we propose a Robust Tensor Graph Convolutional Network (RT-GCN) model to improve the robustness. On the one hand, we utilize multi-view augmentation to reduce the augmentation variance and organize them as a third-order tensor, followed by the truncated T-SVD to capture the low-rankness of the multi-view augmented graph, which improves the robustness from the perspective of graph preprocessing. On the other hand, to effectively capture the inter-view and intra-view information on the multi-view augmented graph, we propose tensor GCN (TGCN) framework and analyze the mathematical relationship between TGCN and vanilla GCN, which improves the robustness from the perspective of model architecture. Extensive experimental results have verified the effectiveness of RT-GCN on various datasets, demonstrating the superiority to the state-of-the-art models on diverse adversarial attacks for graphs.

Federated learning is increasingly attractive, however as the number of training samples on a single device is too small and the training tasks of the devices are different, it faces the few-shot multitask learning problem. Moreover, federated learning frameworks are usually vulnerable to malicious attacks of the central server and diverse clients. To address these problems, we propose a decentralized federated meta-learning framework (DFMLF) for few-shot multitask learning. In DFMLF, the devices take the rapid adaptation as objective and learn the meta-knowledge shared by tasks to deal with the few-shot multitask problem. In addition, DFMLF conducts cross-validation and secure aggregation mechanism by a small number of committee nodes, which not only eliminates the central server to avoid the security risks brought by the malicious central server, but also avoids the attack of malicious devices. Moreover, to address the extra communication cost brought by the committee strategy, we propose a communication-efficient method to make the training and aggregation carried out in parallel. We conduct extensive experiments based on real-world data sets, and the experimental results demonstrate the effectiveness, robustness, and efficiency of our framework.

Meta-path-based random walk strategy has attracted tremendous attention in heterogeneous network representation, which can capture network semantics with heterogeneous neighborhoods of nodes. Despite the success of meta-path-based random walk strategy in plain heterogeneous networks which contain no attributes, it remains unexplored how meta-path-based random walk strategy could be utilized on attributed heterogeneous networks to simultaneously capture structural heterogeneity and attribute proximity. Moreover, the importance of node attributes and structural relations generally varies across data sets, thus requiring careful considerations when they are incorporated into representations. To tackle these problems, we propose a novel method, Attributed Heterogeneous Network embedding based on Aggregate-path (AHNA), which generates aggregate-path-based random walks on attributed heterogeneous networks and adaptively fuses topological structures and node attributes based on the learned importance. Specifically, AHNA first converts node attributes to additional links in the network to deal with the heterogeneity of structures and attributes, which is followed by an adaptive random walk strategy to strike the importance balance between node attributes and topological structures, thereby generating high-quality representations. Extensive experiments are conducted on three real-world data sets, where AHNA outperforms state-of-the-art approaches by up to 22.7%, 2.6%, and 2.3% on link prediction, community detection, and node classification, respectively. Moreover, our qualitative analysis indicates that AHNA can capture different balances of topological structures and node attributes on various data sets and thus boost the quality of node representations.

With a growing number of alternative Internet of Things (IoT) services that provide the same functionalities, Quality-of-Service (QoS) prediction has become an important research issue. Conventional central methods require that the historical QoS data is centralized. While the QoS data may be distributed in different edge servers. Due to privacy, these edge servers may not be willing to share their data with others for a better model representation. Besides, as each edge server is in charge of the collecting QoS data from the users in a specific area, the QoS data are likely to be heterogeneous. In this article, we propose a personalized federated tensor factorization framework for distributed privacy-preserving IoT services QoS prediction. We first adopt tensors to represent the QoS data with multidimensions and initialize a personalized model for each edge server. Then, each edge server performs local tensor factorization and exchanges the public component with a parameter master. By constraining the consistency of the global public component and the local personalized public components, the global model can learn information from edge servers during the training process. In addition, we conduct extensive experiments on a real-world QoS data set, the experimental results demonstrate that the proposed framework is efficient and effective.

Mobile devices can generate a tremendous amount of unique data, and thus, create countless opportunities for deep learning tasks. Due to the concerns of data privacy, it is often impractical to log all the data to a central server for training a satisfactory model. In federated learning, the participating devices can train a shared global model collaboratively while keeping their data locally. However, it is not a trivial task to train the deep neural networks (DNNs) with millions and billions of parameters on resource-constrained mobile devices in a federated manner. We replace each fully connected (FC) layer with two low-rank projection matrices to compact the DNNs model, and establish a global error function to recover the outputs of the compressed DNNs model. Then, we design a communication-efficient federated optimation Algorithm to reduce communication cost further. Considering that the heterogeneous devices may run different models at the same time, we devise three different training patterns to integrate the heterogeneous devices running different models. We conduct extensive experiments on both independently identically distribution (IID) and non-IID data sets. The experimental results demonstrate that the proposed framework can significantly reduce the number of parameters and communication cost while maintaining performance.

Federated learning allows multiple participants to collaboratively train an efficient model without exposing data privacy. However, this distributed machine learning training method is prone to attacks from Byzantine clients, which interfere with the training of the global model by modifying the model or uploading the false gradient. In this article, we propose a novel serverless federated learning frameworkCommittee Mechanism based Federated Learning(CMFL), which can ensure the robustness of the algorithm with convergence guarantee. In CMFL, a committee system is set up to screen the uploaded local gradients. The committee system selects the local gradients rated by the elected members for the aggregation procedure through the selection strategy, and replaces the committee member through the election strategy. Based on the different considerations of model performance and defense, two opposite selection strategies are designed for the sake of both accuracy and robustness. Extensive experiments illustrate that CMFL achieves faster convergence and better accuracy than the typical Federated Learning, in the meanwhile obtaining better robustness than the traditional Byzantine-tolerant algorithms, in the manner of a decentralized approach. In addition, we theoretically analyze and prove the convergence of CMFL under different election and selection strategies, which coincides with the experimental results.

This work concerns the evolutionary approaches to distributed stochastic black-box optimization, in which each worker can individually solve an approximation of the problem with nature-inspired algorithms. We propose a distributed evolution strategy (DES) algorithm grounded on a proper modification to evolution strategies, a family of classic evolutionary algorithms, as well as a careful combination with existing distributed frameworks. On smooth and nonconvex landscapes, DES has a convergence rate competitive to existing zeroth-order methods, and can exploit the sparsity, if applicable, to match the rate of first-order methods. The DES method uses a Gaussian probability model to guide the search and avoids the numerical issue resulted from finite-difference techniques in existing zeroth-order methods. The DES method is also fully adaptive to the problem landscape, as its convergence is guaranteed with any parameter setting. We further propose two alternative sampling schemes which significantly improve the sampling efficiency while leading to similar performance. Simulation studies on several machine learning problems suggest that the proposed methods show much promise in reducing the convergence time and improving the robustness to parameter settings.

Recently, blockchain technology has become a topic in the spotlight but also a hotbed of various cybercrimes. Among them, phishing scams on blockchain have been found to make a notable amount of money, thus emerging as a serious threat to the trading security of the blockchain ecosystem. In order to create a favorable environment for investment, an effective method for detecting phishing scams is urgently needed in the blockchain ecosystem. To this end, this article proposes an approach to detect phishing scams on Ethereum by mining its transaction records. Specifically, we first crawl the labeled phishing addresses from two authorized websites and reconstruct the transaction network according to the collected transaction records. Then, by taking the transaction amount and timestamp into consideration, we propose a novel network embedding algorithm calledtrans2vecto extract the features of the addresses for subsequent phishing identification. Finally, we adopt the one-class support vector machine (SVM) to classify the nodes into normal and phishing ones. Experimental results demonstrate that the phishing detection method works effectively on Ethereum, and indicate the efficacy oftrans2vecover existing state-of-the-art algorithms on feature extraction for transaction networks. This work is thefirstinvestigation on phishing detection on Ethereum via network embedding and provides insights into how features of large-scale transaction networks can be embedded.

Graph contrastive representation learning aims to learn discriminative node representations by contrasting positive and negative samples. It helps models learn more generalized representations to achieve better performances on downstream tasks, which has aroused increasing research interest in recent years. Simultaneously, signed graphs consisting of both positive and negative links have become ubiquitous with the growing popularity of social media. However, existing works on graph contrastive representation learning are only proposed for unsigned graphs (containing only positive links) and it remains unexplored how they could be applied to signed graphs due to the distinct semantics and complex relations between positive and negative links. Therefore we propose a novel Signed Graph Contrastive Learning model (SGCL) to bridge this gap, which to the best of our knowledge is the first research to employ graph contrastive representation learning on signed graphs. Concretely, we design two types of graph augmentations specific to signed graphs based on a significant signed social theory, i.e., balance theory. Besides, inter-view and intra-view contrastive learning are proposed to learn discriminative node representations from perspectives of graph augmentations and signed structures respectively. Experimental results demonstrate the superiority of the proposed model over state-of-the-art methods on both real-world social datasets and online game datasets.

Graph Convolutional Network (GCN) has been widely used in graph learning tasks. However, GCN-based models (GCNs) are inherently coupled training frameworks repetitively conducting the recursive neighborhood aggregation, which leads to high computational and memory overheads when processing large-scale graphs. To tackle these issues, we present Node2Grids, a cost-efficient uncoupled training framework that leverages the independent mapped data for obtaining the embedding. Instead of directly processing the coupled nodes as GCNs, Node2Grids supports a more efficacious method in practice, mapping the coupled graph data into the independent grid-like data which can be fed into the uncoupled models as Convolutional Neural Network (CNN). This simple but valid strategy significantly saves memory and computational resources while achieving comparable results with the leading GCN-based models. Specifically, in order to support a general and convenient mapping approach, Node2Grids selects the most influential neighborhood with central node fusion information to construct the grid-like data. To further improve the downstream tasks' efficiency, a simple CNN-based neural network is employed to capture the significant information from the mapped grid-like data. Moreover, the grid-level attention mechanism is implemented, which enables implicitly specifying the different weights for the extracted grids of CNN. In addition to the typical transductive and inductive learning tasks, we also verify our framework on million-scale graphs to demonstrate the superiority of cost performance against the state-of-the-art GCN-based approaches. The codes are available on the GitHub link.

The proliferation of Web services makes it difficult for users to select the most appropriate one among numerous functionally identical or similar service candidates. Quality-of-Service (QoS) describes the non-functional characteristics of Web services, and it has become the key differentiator for service selection. However, users cannot invoke all Web services to obtain the corresponding QoS values due to high time cost and huge resource overhead. Thus, it is essential to predict unknown QoS values. Although various QoS prediction methods have been proposed, few of them have taken outliers into consideration, which may dramatically degrade the prediction performance. To overcome this limitation, we propose an outlier-resilient QoS prediction method in this paper. Our method utilizes Cauchy loss to measure the discrepancy between the observed QoS values and the predicted ones. Owing to the robustness of Cauchy loss, our method is resilient to outliers. We further extend our method to provide time-aware QoS prediction results by taking the temporal information into consideration. Finally, we conduct extensive experiments on both static and dynamic datasets. The results demonstrate that our method is able to achieve better performance than state-of-the-art baseline methods.

With the popularity of cloud computing, the use of services provided by the cloud is increasing. Due to the high communication cost, and data privacy issues, a new crosscloud collaborative computing model is demanded instead of a single giant cloud. Coping with federated learning and Joint-Cloud, we propose a federated learning-based collaborative learning framework, in which the distributed cloud entities are able to learn the same model collaboratively. As compared to traditional cloud-centric approaches, the framework for JointCloud can reduce the network bandwidth overhead and guarantee privacy. However, there are two crucial challenges in the federated manner: heterogeneity and high communication overhead. To address the heterogeneity, we propose a Teacher-Student mechanism, the key of which is a regularization term incorporated with the objective function so as to adjust the gradients from the clients among JointCloud with different data distribution. Then, based on the Teacher-Student mechanism, we further present a communication-efficient federated optimation approach via joint Identification-Verification to reduce the communication rounds. We conduct extensive experiments on Non-IID datasets. The experimental results demonstrate that the proposed framework can significantly reduce communication costs and improve performance.

As a fundamental problem in social network analysis, community detection has recently attracted wide attention, accompanied by the output of numerous community detection methods. However, most existing methods are developed by only exploiting link topology, without taking node homophily (i.e., node similarity) into consideration. Thus, much useful information that can be utilized to improve the quality of detected communities is ignored. To overcome this limitation, we propose a new community detection approach based on nonnegative matrix factorization (NMF), namely, homophily preserving NMF (HPNMF), which models not only link topology but also node homophily of networks. As such, HPNMF is able to better reflect the inherent properties of community structure. In order to capture node homophily from scratch, we provide three similarity measurements that naturally reveal the association relationships between nodes. We further present an efficient learning algorithm with convergence guarantee to solve the proposed model. Finally, extensive experiments are conducted, and the results demonstrate that HPNMF has strong ability to outperform the state-of-the-art baseline methods.

Network embedding which aims to learn the low-dimensional representations for vertices in networks has been extensively studied in recent years. Although there are various models designed for networks with different properties and different structures for different tasks, most of them are only applied to normal networks which only contain pairwise relationships between vertices. In many realistic cases, relationships among objects are not pairwise and such relationships can be better modeled by a hyper-network in which each edge can connect an uncertain number of vertices. In this article, we focus on two properties of hyper-networks: nonuniform and dual property. In order to make full use of these two properties, we firstly propose a flexible model called Hyper2vec to learn the embeddings of hyper-networks by applying a biased second order random walk strategy to hyper-networks in the framework of Skip-gram. Then, we combine the features of hyperedges by considering the dual hyper-networks to build a further model called NHNE based on 1D convolutional neural networks, and train a tuplewise similarity function for the nonuniform relationships in hyper-networks. Extensive experiments demonstrate the significant effectiveness of our methods for hyper-network embedding.

Clustering on multilayer networks has been shown to be a promising approach to enhance the accuracy. Various multilayer networks clustering algorithms assume all networks derive from a latent clustering structure, and jointly learn the compatible and complementary information from different networks to excavate one shared underlying structure. However, such an assumption is in conflict with many emerging real-life applications due to the existence of noisy/irrelevant networks. To address this issue, we propose Centroid-based Multilayer Network Clustering (CMNC), a novel approach which can divide irrelevant relationships into different network groups and uncover the cluster structure in each group simultaneously. The multilayer networks is represented within a unified tensor framework for simultaneously capturing multiple types of relationships between a set of entities. By imposing the rank-(Lr,Lr,1) block term decomposition with nonnegativity, we are able to have well interpretations on the multiple clustering results based on graph cut theory. Numerically, we transform this tensor decomposition problem to an unconstrained optimization, thus can solve it efficiently under the nonlinear least squares (NLS) framework. Extensive experimental results on synthetic and real-world datasets show the effectiveness and robustness of our method against noise and irrelevant data.

Community detection is of significant importance in understanding the structures and functions of networks. Recently, overlapping community detection has drawn much attention due to the ubiquity of overlapping community structures in real-world networks. Nonnegative matrix factorization (NMF), as an emerging standard framework, has been widely employed for overlapping community detection, which obtains nodes' soft community memberships by factorizing the adjacency matrix into low-rank factor matrices. However, in order to determine the ultimate community memberships, we have to post-process the real-valued factor matrix by manually specifying a threshold on it, which is undoubtedly a difficult task. Even worse, a unified threshold may not be suitable for all nodes. To circumvent the cumbersome post-processing step, we propose a novel discrete overlapping community detection approach, i.e., Discrete Nonnegative Matrix Factorization (DNMF), which seeks for a discrete (binary) community membership matrix directly. Thus DNMF is able to assign explicit community memberships to nodes without post-processing. Moreover, DNMF incorporates a pseudo supervision module into it to exploit the discriminative information in an unsupervised manner, which further enhances its robustness. We thoroughly evaluate DNMF using both synthetic and real-world networks. Experiments show that DNMF has the ability to outperform state-of-the-art baseline approaches.

As a fundamental machine learning problem, graph clustering has facilitated various real-world applications, and tremendous efforts had been devoted to it in the past few decades. However, most of the existing methods like spectral clustering suffer from the sparsity, scalability, robustness and handling high dimensional raw information in clustering. To address this issue, we propose a deep probabilistic model, called Variational Graph Embedding and Clustering with Laplacian Eigenmaps (VGECLE), which learns node embeddings and assigns node clusters simultaneously. It represents each node as a Gaussian distribution to disentangle the true embedding position and the uncertainty from the graph. With a Mixture of Gaussian (MoG) prior, VGECLE is capable of learning an interpretable clustering by the variational inference and generative process. In order to learn the pairwise relationships better, we propose a Teacher-Student mechanism encouraging node to learn a better Gaussian from its instant neighbors in the stochastic gradient descent (SGD) training fashion. By optimizing the graph embedding and the graph clustering problem as a whole, our model can fully take the advantages in their correlation. To our best knowledge, we are the first to tackle graph clustering in a deep probabilistic viewpoint. We perform extensive experiments on both synthetic and real-world networks to corroborate the effectiveness and efficiency of the proposed framework.

Representation learning and feature aggregation are usually the two key intermediate steps in natural language processing. Despite deep neural networks have shown strong performance in the text classification task, they are unable to learn adaptive structure features automatically and lack of a method for fully utilizing the extracted features. In this paper, we propose a novel architecture that dynamically routes hierarchical structure feature to attentive capsule, named HAC. Specifically, we first adopt intermediate information of a well-designed deep dilated CNN to form hierarchical structure features. Different levels of structure representations are corresponding to various linguistic units such as word, phrase and clause, respectively. Furthermore, we design a capsule module using dynamic routing and equip it with an attention mechanism. The attentive capsule implements an effective aggregation strategy for feature clustering and selection. Extensive results on eleven benchmark datasets demonstrate that the proposed model obtains competitive performance against several state-of-the-art baselines. Our code is available at https://github.com/zhengwsh/HAC.

The cryptocurrency market is a very huge market without effective supervision. It is of great importance for investors and regulators to recognize whether there are market manipulation and its manipulation patterns. This paper proposes an approach to mine the transaction networks of exchanges for answering this question. By taking the leaked transaction history of Mt. Gox Bitcoin exchange as a sample, we first divide the accounts into three categories according to its characteristic and then construct the transaction history into three graphs. Many observations and findings are obtained via analyzing the constructed graphs. To evaluate the influence of the accounts' transaction behavior on the Bitcoin exchange price, the graphs are reconstructed into series and reshaped as matrices. By using singular value decomposition (SVD) on the matrices, we identify many base networks which have a great correlation with the price fluctuation. When further analyzing the most important accounts in the base networks, plenty of market manipulation patterns are found. According to these findings, we conclude that there was serious market manipulation in Mt. Gox exchange and the cryptocurrency market must strengthen the supervision.

Offloading computation-intensive blockchain mining tasks to the edge servers (ESs) is a promising solution for blockchain-empowered Industrial Internet of Things (IIoT) because the computing capabilities in IIoT are usually limited, whereas the blockchain mining tasks are computationally intensive. However, the computation offloading solutions for data processing tasks and for blockchain mining tasks have been studied separately. Moreover, most of the existing solutions for offloading assume that all IIoT devices can directly connect to the ESs or cloud data centers. To address these issues, in this paper, we propose a multihop cooperative and distributed computation offloading algorithm that considers the data processing tasks and the mining tasks together for blockchain-empowered IIoT. First, we study the multihop computation offloading problem for both the data processing tasks and the mining tasks to minimize the economic cost of IIoT devices. Second, we formulate the offloading problem as a potential game in which the IIoT devices can make their decisions autonomously and prove the existence of Nash equilibrium (NE) for the game. Third, we design an efficient distributed algorithm based on exchanging messages between IIoT devices to achieve the NE with low computational complexity. Lastly, our experimental results demonstrate that our distributed algorithm scales well as the number of IIoT devices increases and has the minimum system cost compared with other approaches.

Lexical-based metrics such as BLEU, NIST, and WER have been widely used in machine translation (MT) evaluation. However, these metrics badly represent semantic relationships and impose strict identity matching, leading to moderate correlation with human judgments. In this paper, we propose a novel MT automatic evaluation metric Semantic Travel Distance (STD) based on word embeddings. STD incorporates both semantic and lexical features (word embeddings and n-gram and word order) into one metric. It measures the semantic distance between the hypothesis and reference by calculating the minimum cumulative cost that the embedded n-grams of the hypothesis need to “travel” to reach the embedded n-grams of the reference. Experiment results show that STD has a better and more robust performance than a range of state-of-the-art metrics for both the segment-level and system-level evaluation.

Multidomain network classification has attracted significant attention in data integration and machine learning, which can enhance network classification or prediction performance by integrating information from different sources. Despite the previous success, existing multidomain network learning methods usually assume that different views are available for the same set of instances, and thus, they seek a consistent classification result for all domains. However, in many real-world problems, each domain has its specific instance set, and one instance in one domain may correspond to multiple instances in another domain. Moreover, due to the rapid growth of data sources, different domains may not be relevant to each other, which asks for selecting domains relevant to the target/focused domain. A key challenge under this setting is how to achieve accurate prediction by integrating different data representations without losing data information. In this paper, we propose a semisupervised classification approach for a multidomain network based on label propagation, i.e., multidomain classification with domain selection (MCS), which can deal with the cross-domain information and different instance sets in domains. In particular, with sparse weight properties, the proposed MCS can automatically identify those domains relevant to our target domain by assigning them higher weights than the other irrelevant domains. This not only significantly improves a classification accuracy but also helps to obtain optimal network partition for the target domain. From the theoretical viewpoint, we equivalently decompose MCS into two simpler subproblems with analytical solutions, which can be efficiently solved by their computational procedures. Extensive experimental results on both synthetic and real-world data sets empirically demonstrate the advantages of the proposed approach in terms of both prediction performance and domain selection ability.

Community structure is ubiquitous in real-world complex networks. The task of community detection over these networks is of paramount importance in a variety of applications. Recently, nonnegative matrix factorization (NMF) has been widely adopted for community detection due to its great interpretability and its natural fitness for capturing the community membership of nodes. However, the existing NMF-based community detection approaches are shallow methods. They learn the community assignment by mapping the original network to the community membership space directly. Considering the complicated and diversified topology structures of real-world networks, it is highly possible that the mapping between the original network and the community membership space contains rather complex hierarchical information, which cannot be interpreted by classic shallow NMF-based approaches. Inspired by the unique feature representation learning capability of deep autoencoder, we propose a novel model, named Deep Autoencoder-like NMF (DANMF), for community detection. Similar to deep autoencoder, DANMF consists of an encoder component and a decoder component. This architecture empowers DANMF to learn the hierarchical mappings between the original network and the final community assignment with implicit low-to-high level hidden attributes of the original network learnt in the intermediate layers. Thus, DANMF should be better suited to the community detection task. Extensive experiments on benchmark datasets demonstrate that DANMF can achieve better performance than the state-of-the-art NMF-based community detection approaches.

The task of community detection has become a fundamental research problem in complex network analysis. Intuitively, similar nodes are more likely to be contained in the same community. However, most existing community detection methods cannot extract the intrinsic similarity between nodes. Thus, they may fail to identify the real community structures. In this paper, we propose to learn an affinity matrix adaptively, which can capture the intrinsic similarity between nodes accurately, and therefore benefit the community detection results. Specifically, the proposed model first embeds each node into a low-dimensional space through a transformation matrix with the community structures being preserved. Then, our model learns the affinity matrix in this low-dimensional space. The affinity matrix is further utilized to guide the learning of the community membership matrix via manifold regularization. The above three matrices are learned simultaneously and updated iteratively under the framework of Alternating Direction Method of Multipliers (ADMM). Extensive experiments show that our model can outperform the state-of-the-art approaches.

It is important to transfer the knowledge from label-rich source domain to unlabeled target domain due to the expensive cost of manual labeling efforts. Prior domain adaptation methods address this problem through aligning the global distribution statistics between source domain and target domain, but a drawback of prior methods is that they ignore the semantic information contained in samples, e.g., features of backpacks in target domain might be mapped near features of cars in source domain. In this paper, we present moving semantic transfer network, which learn semantic representations for unlabeled target samples by aligning labeled source centroid and pseudo-labeled target centroid. Features in same class but different domains are expected to be mapped nearby, resulting in an improved target classification accuracy. Moving average centroid alignment is cautiously designed to compensate the insufficient categorical information within each mini batch. Experiments testify that our model yields state of the art results on standard datasets.

In this paper, we address the total variation (TV)-based nonlinear image restoration problems. In nonlinear image restoration problems, an original image is corrupted by a spatially-invariant blur, the build-in nonlinearity in imaging system, and the additive Gaussian white noise. We study the objective function consisting of the nonlinear least squares data-fitting term and the TV regularization term of the restored image. By making use of the structure of the objective function, an efficient alternating direction method of multipliers can be developed for solving the proposed model. The convergence of the numerical scheme is also studied. Numerical examples, including nonlinear image restoration and high-dynamic range imaging are reported to demonstrate the effectiveness of the proposed model and the efficiency of the proposed numerical scheme.