Generalized federated learning seeks to develop robust models across distributed source domains that generalize well to the unseen target domain. Mainstream methods make strict assumptions about the availability of target domain data, limiting the flexibility and adaptability of real-world applications. In this work, we tackle a real-world challenge that has never been addressed before: federated multi-source domain adaptation for an unseen target domain. We propose federated cross-domain semantic alignment with adversarial feature augmentation, a method that enhances model generalization across domains. Our method operates in the feature space to capture both diversity and invariance between source and target domains through a two-stage local training strategy. In the adversarial training phase, a domain identifier and feature discriminator constrain the generated features to extract target-relevant information. During the contrastive learning stage, a semantic representation alignment loss (SRA) is incorporated to align class prototype distributions between source and target domains, ensuring uniform classification standards. Federated aggregation consolidates model knowledge across clients, facilitating collaborative evolution and rapid adaptation to the unseen target domain. Extensive experimental results on four prevalent datasets demonstrate that our approach outperforms existing benchmarks across different backbones, showcasing its effectiveness in scenarios with data silos.

Federated unsupervised representation learning aims at leveraging unlabeled data from multiple parties to learn visual representations without compromising the data privacy and tackle the non-IID challenge by aligning diverse representation spaces. However, model heterogeneity and communication overhead will directly impact the convergence rate and model accuracy of federated unsupervised learning. And it is challenging to learn visual features for downstream tasks under the premise of compatibility with heterogeneous models and reducing communication overhead. To address these issues, we propose a novel communication-efficient federated unsupervised representation learning framework based on prototype similarity distillation (FLPD). In this framework, the global model builds the feature representation space based on the global dataset and steers the optimization of the prototype relations of the client models. In addition to employing discriminative self-supervised learning for model training, each client fine-tunes the local representation space with global prototype similarity via knowledge distillation, which facilitates local models to fit both the local data distribution and the global representation space. In order to maintain the compactness of prototypes within the same category and enhance the separability between prototypes of different categories, a prototype-based consistency constraint is introduced to alleviate the conflict between local and global representation space. Experimental results demonstrate that our framework outperforms other alternative approaches in terms of communication efficiency and accuracy in the federated settings with statistical heterogeneity and model heterogeneity.

Graph Neural Network (GNN) is capable of applying deep neural networks to graph domains. Recently, Message Passing Neural Networks (MPNNs) have been proposed to generalize several existing graph neural networks into a unified framework. For graph representation learning, MPNNs first generate discriminative node representations using the message passing function and then read from the node representation space to generate a graph representation using the readout function. In this paper, we analyze the representation capacity of the MPNNs for aggregating graph information and observe that the existing approaches ignore the self-loop for graph representation learning, leading to limited representation capacity. To alleviate this issue, we introduce a simple yet effective propagation enhanced extension, Self-Connected Neural Message Passing (SC-NMP), which aggregates the node representations of the current step and the graph representation of the previous step. To further improve the information flow, we also propose a Densely Self-Connected Neural Message Passing (DSC-NMP) that connects each layer to every other layer in a feed-forward fashion. Both proposed architectures are applied at each layer and the graph representation can then be used as input into all subsequent layers. Remarkably, combining these two architectures with existing GNN variants can improve these models’ performance for graph representation learning. Extensive experiments on various benchmark datasets strongly demonstrate the effectiveness, leading to superior performance for graph classification and regression tasks.

Graph matching endeavors to find corresponding nodes across two or more graphs, which plays a fundamental role in many vision and pattern matching tasks. However, existing graph matching algorithms often meet abnormal graphs with missing node features and suffer from numerous cluttered outliers in practical applications. To address these, we propose a novel deep graph matching method called Random Deep Graph Matching (RDGM). Different from the deterministic affinity inference in existing deep graph matching methods, RDGM performs message passing in a random manner during model training through randomly masking some available node features in the source or target graph, so that the affinity inference between nodes is insensitive to specific neighborhoods. In addition, a hierarchical attention graph neural network framework is devised in the node embedding process of RDGM, which can obtain more sufficient high-order structural information to reduce the impact of latent noise on affinity learning. Extensive experiments suggest that the proposed RDGM outperforms state-of-the-art graph matching methods, and demonstrates strong robustness and generalization performance.

Representing nodes in a network as low-dimensional dense vectors can facilitate the analysis of complex networks, which is a challenging task and has attracted increasing attention. However, in the real world, networks are changing over time, such as cooperation in citation networks and communication in email networks. Most of the recent embedding methods only focus on static networks. Thus they ignore the critical temporal information, which serves as a supplement to structure information and has been proved to improve the quality of node embedding. In this work, we propose an unsupervised deep learning model called DTINE, which explores temporal information for further enhancing the robustness of node representations in dynamic networks. To preserve network topology, we pertinently design a temporal weight and sampling strategy to extract features from the neighborhoods. An attention mechanism will be applied on the recurrent neural network to measure the contributions of historical information and capture the evolution of the networks. Experimental results on four real-world networks demonstrate that the proposed method achieves better performance than state-of-the-art methods.

Network embedding has been widely used to solve the network analytics problem. Existing methods mainly focus on networks with single-layered homogeneous or heterogeneous networks. However, many real-world complex systems can be naturally represented by multilayer networks, which is another term of heterogeneous networks with multiple edge/relation types. The problem of how to capture and utilize rich interaction information of multi-type relations causes a major challenge of multilayer network embedding. To address this problem, we propose a fast and scalable multilayer network embedding model, called HMNE, to efficiently preserve and learn information of multi-type relations into a unified embedding space. We develop a heuristic 3D interactive walk technique dedicated for multilayer networks, which can leverage rich interactions among distinct layers and effectively capture important information contained in the layered structure. We evaluate our proposed model HMNE on two downstream analytic applications: node classification and link prediction. Experimental results on seven social and biological multilayer network datasets demonstrate that the proposed model outperforms existing competitive baselines with reduced time and memory occupations.