Recently, the success of large models has demonstrated the importance of scaling up model sizes. However, it is difficult to directly train large models locally on multiple mobile devices due to their intrinsic computational constraints. To address this challenge, it becomes a crucial need to train larger global models by training small local models on devices. As a distributed learning approach, federated learning (FL) allows multiple devices to train models locally and aggregate them to form the global model by sharing the updated parameters with the server, thus enabling the co-training of models. This promising feature has spurred an increasing interest in exploring the collaborative training of large models. Despite the advent of existing device-heterogeneity FL approaches, they still have limitations in fully covering the parameter space of the global model. To fill this gap, we propose a novel approach calledFedBRB(Block- wiseRolling and weightedBroadcast). The core idea of FedBRB is to utilize local models of small devices to train all modules of a large global model and broadcast the trained parameters to the entire space, thereby enabling faster information sharing. This approach not only improves training efficiency but also fully utilizes limited computational resources. Experiments demonstrate that FedBRB can produce significant performance gains, achieving state-of-the-art results. Additionally, this paper provides theoretical and experimental analyses of FedBRB convergence, thereby paving a theoretical ground and providing practical guidance for further research and application of the FedBRB method.

Graph Neural Networks (GNNs) have exhibited their powerful ability of tackling nontrivial problems on graphs. However, as an extension of deep learning models to graphs, GNNs are vulnerable to noise or adversarial attacks due to the underlying perturbations propagating in message passing scheme, which can affect the ultimate performances dramatically. Thus, it's vital to study a robust GNN framework to defend against various perturbations. In this paper, we propose a Robust Tensor Graph Convolutional Network (RT-GCN) model to improve the robustness. On the one hand, we utilize multi-view augmentation to reduce the augmentation variance and organize them as a third-order tensor, followed by the truncated T-SVD to capture the low-rankness of the multi-view augmented graph, which improves the robustness from the perspective of graph preprocessing. On the other hand, to effectively capture the inter-view and intra-view information on the multi-view augmented graph, we propose tensor GCN (TGCN) framework and analyze the mathematical relationship between TGCN and vanilla GCN, which improves the robustness from the perspective of model architecture. Extensive experimental results have verified the effectiveness of RT-GCN on various datasets, demonstrating the superiority to the state-of-the-art models on diverse adversarial attacks for graphs.