In this paper, we propose a new approach for continual learning based on the Visual Transformers (ViTs). The purpose of continual learning is to address the catastrophic forgetting problem. One method for preventing forgetting previous tasks is exemplar replay. However, exemplar replay has limitations such as limited memory capacity, large storage requirements, and difficulty adapting to new tasks, which restrict its practical application. Therefore, we adopted knowledge distillation to implement exemplar-free continual learning. This approach transfers the knowledge of old tasks to the model's prior knowledge, helping the model to learn new tasks better while not increasing the storage burden. Based on this, we propose a new method for training the ViT model called Feature Attention Distillation (FAD). Specifically, we introduced a feature attention mechanism into the knowledge distillation model to help the student model better retain meaningful feature representations and attention distributions during the learning process of new tasks by passing attention information between the teacher and student models, thereby improving the learning efficiency and performance of the model. In addition, we also introduced a new normalization method called Continual Normalization (CN) into the student model. This method dynamically calculates the sample mean and variance of the current and historical tasks, allowing the student network to adapt better to the feature distribution changes between different tasks, thus improving the model's generalization ability and robustness. Extensive experiments on the CIFAR100 and ImageNet-32 datasets show that our exemplar-free method is competitive in performance compared to rehearsal-based ViT methods.

MOTIVATION: Genome-scale CRISPR/Cas9 system has been a democratized gene editing technique and widely used to investigate gene functions in some biological processes and diseases especially cancers. Aiming to characterize gene aberrations and assess their effects on cancer, we designed a pipeline to identify the essential genes for pan-cancer. METHODS: CRISPR screening data were used to identify the essential genes that were collected from published data and integrated by Robust Rank Aggregation algorithm. Then, hypergeometrics test and random walks with restart (RWR) were used to predict additional essential genes on broader scale. Finally, the expression status and potential roles of these genes were explored based on TCGA portal and regulatory network analysis. RESULTS: We collected 926 samples from 10 CRISPR-based screening studies involving 33 different types of cancer to identify cancer-essential genes, which consists of 799 protein-coding genes (PCGs) and 97 long non-coding RNAs (lncRNAs). Then, we constructed a 'bi-colored' network with both PCGs and lncRNAs and applied it to predict additional essential genes including 495 PCGs and 280 lncRNAs on a broader scale using hypergeometrics test and RWR. After obtaining all essential genes, we further investigated their potential roles in cancer and found that essential genes have higher and more stable expression levels, and are associated with multiple cancer-associated biological processes and survival time. The regulatory network analysis detected two intriguing modules of essential genes participating in the regulation of cell cycle and ribosome biogenesis in cancer. AVAILABILITY AND IMPLEMENTATION: . SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.