Distributed deep neural networks (DNN) training systems deployed across workers have been widely used in various domains, while the communication overhead among workers for synchronizing gradient tensors often becomes the performance bottleneck. To optimize communication efficiency, state-of-the-art studies often apply both two techniques: i) gradient sparsification compression, which truncates the gradient to its largest elements to reduce the communication traffic, and ii) tensor fusion, which merges multiple gradient tensors within a fusion buffer to transmit them together to reduce the communication startup overhead. However, we find that existing studies often apply gradient sparsification after tensor fusion (we call sparsification-behind tensor fusion), which leads to a fact that a lot of fused gradient tensors are missed after the sparsification, thus impairing the convergence performance.

This systematic literature review explores the application of multimodal learning analysis (MMLA), with physiological signals collected through wearable devices as the primary data source, in immersive virtual reality (IVR) learning environments. By examining 78 peer-reviewed articles published over the past nine years (2016–2024), the paper addresses two core research questions in IVR learning environments: 1) What are the main multimodal learning indicators? 2) What are the ethical considerations associated with multimodal learning analysis? The findings indicate that cognitive indicators are dominant, while studies on emotional and affective learning indicators remain scarce. Real-time monitoring of cognitive load and dynamic task adjustment mechanisms have yet to be fully implemented, suggesting future research on the design of adaptive learning tasks. Additionally, the review calls for improvements in both technological and ethical frameworks to address issues related to privacy, transparency, and fairness. By reviewing and summarizing current research advancements, this paper offers valuable insights into future research and practice of MMLA in IVR learning environments.

Distributed training of large deep-learning models often leads to failures, so checkpointing is commonly employed for recovery. State-of-the-art studies focus on frequent checkpointing for fast recovery from failures. However, it generates numerous checkpoints, incurring substantial costs and thus degrading training performance. Recently, differential checkpointing has been proposed to reduce costs, but it is limited to recommendation systems, so its application to general distributed training systems remains unexplored.

Phage virion proteins (PVPs), which form the structural components of phages, are crucial for maintaining phage structures and infecting host bacteria. Identifying PVPs can lead to the development of novel therapeutic agents to combat bacterial infections, attracting great research attention in recent years. However, most of the existing methods for PVP identification heavily depend on the effectiveness of feature extraction and have no ability to precisely classify specific classes. In this article, we propose a bidirectional encoder representations-based Transformer model called BERTPVP for the identification and classification of PVPs. BERTPVP uses a stack of transformer encoders to effectively capture contextual information from the entire protein sequence through the multi-head self-attention mechanism. We firstly pre-train the model using the masked language modeling task to learn contextual information from phage protein sequences, which aids in understanding the significance and relevance of individual components in relation to the entire sequence. Subsequently, the pre-trained model is fine-tuned for PVP identification and classification tasks. Our experimental results show the superiority of the proposed BERTPVP over the state-of-the-art methods in identifying and classifying PVPs. Moreover, the ablation study demonstrates the necessity of the pre-training and fine-tuning components of BERTPVP in accelerating convergence and improving prediction performance.

Data-parallel deep neural networks (DNN) training systems deployed across nodes have been widely used in various domains, while the system performance is often bottlenecked by the communication overhead among workers for synchronizing gradients. Top-k sparsification compression is the de facto approach to alleviate the communication bottleneck, which truncates the gradient to its largest k elements before sending it to other nodes.

This study employed multimodal learning analytics methods to evaluate different virtual reality (VR) interfaces, exploring their impact on the learning experience of cultural heritage. We recorded and analyzed the participants’ learning behavior, self-reported perceptions from questionnaires’ responses, electroencephalogram (EEG) signals, and visual fatigue. Preliminary results suggested that VR interfaces with more modules per scene led to higher efficiency of use, although more modules did not improve viewing experience. This study provides insights for immersive interface design in cultural heritage learning.

The identification of protein coding regions is one of the important applications of genome sequence analysis. Many digital signal processing (DSP) based methods, which rely on 3-base periodicity of DNA sequences, have been proposed. However, for most Fourier Transform based methods, a prior time-domain window length limits their performances. Even though several wavelet-based methods get rid of the dependence of window length, an overly wide scale range results in the loss of identification accuracy of these methods. In this paper, we propose a novel method based on Scale-limited Gabor Wavelet Transform (SLGWT) for identifying protein coding regions. This method inherits the advantage of wavelet-based methods in the independence of time-domain window length, while maintaining the consistent performance under different wavelet window lengths. More importantly, compared with other wavelet-based method, SLGWT identifies coding regions under narrower and more suitable scale range, thereby improving the identification accuracy and reducing computational load. The experimentations in the sequence and dataset levels verify the superiority of our proposed method.