Transformer models are widely used in AI applications such as Natural Language Processing (NLP), Computer Vision (CV), etc. However, enormous computation workload be-comes an obstacle to train large transformer models efficiently. Recently, some methods focus on reducing the computation workload during the training by skipping some layers. How-ever, these methods use simple probability distribution and coarse-grained probability calculation, which significantly affect the model accuracy. To address the issue, in this paper we propose a novel method to accelerate training—Sensitivity-Based Layer Dropping (SBLD). SBLD uses lay-er-wise sensitivity data to switch on/off transformer layers in proper order to keep high accuracy. Besides, we adjust the probability of skipping transformer layers with a scheduler to accelerate training speed and get faster convergence. Our results show that SBLD solves the accuracy drop issue com-pared with prior layer dropping methods. Our SBLD method can decrease end-to-end training time by 19.67% during training of GPT-3 Medium model, the same time increasing the accuracy by 1.65% w.r.t. baseline. Furthermore, for SwinV2-L model the obtained Top-1 and Top-5 accuracies are also higher vs. the baseline. Thus, the proposed method is efficient and practical to improve the large transformer model training.

The availability of high-throughput sequencing data creates opportunities to comprehensively understand human diseases as well as challenges to train machine learning models using such high dimensions of data. Here, we propose a denoised multi-omics integration framework, which contains a distribution-based feature denoising algorithm, Feature Selection with Distribution (FSD), for dimension reduction and a multi-omics integration framework, Attention Multi-Omics Integration (AttentionMOI) to predict cancer prognosis and identify cancer subtypes. We demonstrated that FSD improved model performance either using single omic data or multi-omics data in 15 The Cancer Genome Atlas Program (TCGA) cancers for survival prediction and kidney cancer subtype identification. And our integration framework AttentionMOI outperformed machine learning models and current multi-omics integration algorithms with high dimensions of features. Furthermore, FSD identified features that were associated to cancer prognosis and could be considered as biomarkers.

The 2019 novel coronavirus (SARS-CoV-2) has spread rapidly worldwide and was declared a pandemic by the WHO in March 2020. The evolution of SARS-CoV-2, either in its natural reservoir or in the human population, is still unclear, but this knowledge is essential for effective prevention and control. We propose a new framework to systematically identify recombination events, excluding those due to noise and convergent evolution. We found that several recombination events occurred for SARS-CoV-2 before its transfer to humans, including a more recent recombination event in the receptor-binding domain. We also constructed a probabilistic mutation network to explore the diversity and evolution of SARS-CoV-2 after human infection. Clustering results show that the novel coronavirus has diverged into several clusters that cocirculate over time in various regions and that several mutations across the genome are fixed during transmission throughout the human population, including D614G in the S gene and two accompanied mutations in ORF1ab. Together, these findings suggest that SARS-CoV-2 experienced a complicated evolution process in the natural environment and point to its continuous adaptation to humans. The new framework proposed in this study can help our understanding of and response to other emerging pathogens.

Abstract Wireless communication network attack detection is important in the field of wireless communication network data mining. Transfer learning is a powerful tool to solve the problem of link prediction in unlabeled wireless networks. In the feature mapping process of transfer learning, the loss of source domain information is very serious, which leads to the loss of knowledge in inverse mapping. To obtain more specific and complete information in the source domain, a link prediction method based on the distribution function fitting method of domain orthogonal term family is proposed. The key samples in the source domain are used to construct a family of orthogonal polynomials, and the least square approximation of the distribution function is obtained by using the family of orthogonal polynomials. The feature vector set of the key samples is extracted from the source domain, and the pseudo label is assigned to the target training set, so that the link classifier can complete the supervised link prediction task. In addition, an advanced performance of TLAD is obtained by comparing with other state‐of‐the‐art models in practice.