Since there exists a single point of server failure in conventional centralized federated learning, the decentralized federated learning (DFL) framework has become increasingly popular in recent years. However, when a large number of edge devices participate in DFL, it requires frequent model interactions between edge devices and long convergence time. In this work, we combat the impact of device heterogeneity in the large-scale DFL framework. To optimize communication efficiency and reduce the network complexity in large-scale DFL framework, we propose a decentralized edge devices clustering (DEDC) approach which leverages dense connectivity as the foundation to group edge devices with similar data distributions into clusters, thereby forming a novel multi-cluster decentralized federated edge learning (MD-FEEL) framework. The clustering method is adaptive, meaning it can effectively work across various network topologies, as long as the network is connected. We propose an asynchronous algorithm in the formed MD-FEEL framework, which consists four steps, i.e., local stochastic gradient descent (SGD) update, gradient consensus, intra-cluster model aggregation and inter-cluster model aggregation. We prove the convergence of our proposed asynchronous MD-FEEL algorithm on a non-convex setting and elaborate on the effect of some hyperparameters. Empirically, we evaluate our proposed asynchronous MD-FEEL on the MNIST and CIFAR-10 datasets. The simulations show that our proposed asynchronous MD-FEEL can perform better in terms of convergence speed and generalization performance than some benchmark algorithms.

With the convergence of the social networks and the Internet of Things (IoT), social IoT (SIoT) has emerged as a promising application scenario of federated learning. Meanwhile, most centralized federated learning (CFL) algorithms encounter single-point-of-failure risks and high bandwidth pressure at the central server. Therefore, decentralized FL (DFL) has been widely studied in recent years. However, when a substantial number of social nodes participate in DFL, the model consensus process requires a significant amount of communication among social nodes. This incurs a high communication overhead and low training efficiency, especially for the SIoT with dynamic network topology. In this work, we propose a communication-effective DFL algorithm for a general dynamic SIoT network with a large number of social nodes. To improve the communication efficiency and simplify network complexity, we employ a limited label propagation algorithm (LLPA) to periodically cluster social nodes into a dynamic multi-cluster decentralized federated learning (DMC-DFL) framework. We design an effective algorithm in the formed DMC-DFL framework, which consists of three steps, i.e., local update, intra-cluster communication and inter-cluster communication. Empirically, we conduct extensive comparison and ablation experiments based on four datasets. The experiment results validate the feasibility of DMC-DFL algorithm in both static and dynamic SIoT networks and illustrate the superiority of DMC-DFL algorithm over some benchmark DFL algorithms.

In conventional federated learning (FL), there exists a single point of failure in the central server. Thus the studies about decentralized federated learning (DFL) paradigm have become popular recently. In DFL, some clients with poor computation capacity may take a long time to train local models, therefore, the convergence speed of the global model is usually slow in existing synchronous algorithms. In this work, we consider a large-scale system with device heterogeneity. To reduce training time and fully utilize edge node computation capacity, we propose an asynchronous algorithm in a novel multi-cluster decentralized federated edge learning (MD-FEEL) framework, where there are many clusters and each cluster consists of some clients. Our proposed asynchronous MD-FEEL contains four steps, i.e., local stochastic gradient descent (SGD) update, gradient consensus, intra-cluster model aggregation and inter-cluster model aggregation. To measure the staleness of cluster model, we introduce age of update (AoU) in inter-cluster aggregation stage and theoretically prove the convergence of our proposed algorithm on a non-convex setting. We evaluate our asynchronous MD-FEEL on MNIST and CIFAR-10 datasets and the simulation results show it can aggregate to a global model with better accuracy performance and faster convergence speed than some existing synchronous algorithms.

Non-coding variants associated with complex traits can alter the motifs of transcription factor (TF)-deoxyribonucleic acid binding. Although many computational models have been developed to predict the effects of non-coding variants on TF binding, their predictive power lacks systematic evaluation. Here we have evaluated 14 different models built on position weight matrices (PWMs), support vector machines, ordinary least squares and deep neural networks (DNNs), using large-scale in vitro (i.e. SNP-SELEX) and in vivo (i.e. allele-specific binding, ASB) TF binding data. Our results show that the accuracy of each model in predicting SNP effects in vitro significantly exceeds that achieved in vivo. For in vitro variant impact prediction, kmer/gkm-based machine learning methods (deltaSVM_HT-SELEX, QBiC-Pred) trained on in vitro datasets exhibit the best performance. For in vivo ASB variant prediction, DNN-based multitask models (DeepSEA, Sei, Enformer) trained on the ChIP-seq dataset exhibit relatively superior performance. Among the PWM-based methods, tRap demonstrates better performance in both in vitro and in vivo evaluations. In addition, we find that TF classes such as basic leucine zipper factors could be predicted more accurately, whereas those such as C2H2 zinc finger factors are predicted less accurately, aligning with the evolutionary conservation of these TF classes. We also underscore the significance of non-sequence factors such as cis-regulatory element type, TF expression, interactions and post-translational modifications in influencing the in vivo predictive performance of TFs. Our research provides valuable insights into selecting prioritization methods for non-coding variants and further optimizing such models.

Mixture model (MM) is a probabilistic framework allows us to define dataset containing K different modes. When each of the modes is associated with a Gaussian distribution, we refer to it as Gaussian MM or GMM. Given a data point x, a GMM may assume the existence of a random index k ϵ {1, ..., K} identifying which Gaussian the particular data is associated with. In a traditional GMM paradigm, it is straightforward to compute in closed-form, the conditional likelihood p(x|k, θ) as well as the responsibility probability p(k| x, θ) describing the distribution weights for each data. Computing the responsibility allows us to retrieve many important statistics of the overall dataset, including the weights of each of the modes/clusters. Modern large datasets are often containing multiple unlabelled modes, such as paintings dataset may contain several styles; fashion images containing several unlabelled categories. In its raw representation, the Euclidean distances between the data (e.g., images) do not allow them to form mixtures naturally, nor it's feasible to compute responsibility distribution analytically, making GMM unable to apply. In this paper, we utilize the generative adversarial network (GAN) framework to achieve a plausible alternative method to compute these probabilities. The key insight is that we compute them at the data's latent space z instead of x. However, this process of z → x is irreversible under GAN which renders the computation of responsibility p(k|x, θ) infeasible. Our paper proposed a novel method to solve it by using a socalled posterior consistency module (PCM). PCM acts like a GAN, except its generator CPCMdoes not output the data, but instead it outputs a distribution to approximate p(k|x, θ). The entire network is trained in an “end-to-end” fashion. Trough these techniques, it allows us to model the dataset of very complex structure using GMM and subsequently to discover interesting properties of an unsupervised dataset, including its segments, as well as generating new “out-distribution” data by smooth linear interpolation across any combinations of the modes in a completely unsupervised manner.

ABSTRACT With the rapid development of wireless sensor networks, the issue of designing a reasonable trust management has attracted more and more research attention. Based on Bayesian and Entropy, this paper proposes a lightweight trust management for wireless sensor networks. First, the evaluated node's direct trust value is calculated by Bayesian and periodically updated according to the combination of effective history records and adaptive decay factor. We use effective history records rather than all the records to save nodes’ memory, and the adaptive decay factor enhances the algorithm's accuracy and dynamic. Then, according to the confidence level of the direct trust value, we decide whether the direct trust is credible enough to be the integrated trust. This can reduce the energy computation and make the algorithm lightweight. Last, if the direct trust is not credible enough, the overall indirect trust value will be calculated. The Entropy Theory is adopted to distribute weights to different trust values, which can improve the problems caused by distributing weights subjectively and also enhance adaptability of the model. Simulation experiments are provided to assess the performance of the proposed trust management in terms of attack–defeat ability and energy consumption. Copyright © 2014 John Wiley & Sons, Ltd.