Dynamic sequences with varying lengths have been widely used in the training of Transformer-based large language models (LLMs). However, current training frameworks adopt a pre-defined static parallel strategy for these sequences, causing neither communication-parallelization cancellation on short sequences nor out-of-memory on long sequences. To mitigate these issues, we propose ParaDySe, a novel adaptive Parallel strategy switching framework for Dynamic Sequences. ParaDySe enables on-the-fly optimal strategy adoption according to the immediate input sequence. It first implements the modular function libraries for parallel strategies with unified tensor layout specifications, and then builds sequence-aware memory and time cost models with hybrid methods. Guided by cost models, ParaDySe selects optimal layer-wise strategies for dynamic sequences via an efficient heuristic algorithm. By integrating these techniques together, ParaDySe achieves seamless hot-switching of optimal strategies through its well-designed function libraries. We compare ParaDySe with baselines on representative LLMs under datasets with sequence lengths up to 624K. Experimental results indicate that ParaDySe addresses OOM and CPC bottlenecks in LLM training by systematically integrating long-sequence optimizations with existing frameworks.

With the development of the Internet and computer technology, the phenomenon of information explosion has occurred in many fields, such as computer vision and natural language processing. People are gradually moving into the era of information overload, and it is very difficult to find effective information from the massive multi-modal data. Recommender systems are important tools to solve this difficulty. Recommendation systems currently face many challenges, one of the most important challenges in application scenarios is the cold-start problem. General recommendation systems are inaccurate for cold users/items, which have very FEW contact with the recommendation system, because of the lack of interaction data. To address the cold-start problem, this paper proposes a Four-viEW (FEW) multi-modal recommendation model, which improves the recommendation effect by constructing a four-view graph convolutional network for enhancing the multi-modal (mainly visual and textual) data representations of cold users/items. FEW is compared with existing single-modal and multi-modal recommendation models on the Amazon review datasets. Overall, compared to existing state-of-the-art models, FEW improves Recall by a maximum of 15.72% and NDCG by a maximum of 19.86%. In particular, the addition of cold-start technology further enhances the recommendation effect of FEW. For cold users, the maximum improvement is 20.86% for Recall and 27.47% for NDCG; for cold items, the average improvement of Recall is 85.14x and NDCG is 63.08x, which verifies the effectiveness of FEW in solving the cold-start problem.

Abstract Although GPUs have been used to accelerate various convolutional neural network algorithms with good performance, the demand for performance improvement is still continuously increasing. CPU/GPU overclocking technology brings opportunities for further performance improvement in CPU-GPU heterogeneous platforms. However, CPU/GPU overclocking inevitably increases the power of the CPU/GPU, which is not conducive to energy conservation, energy efficiency optimization, or even system stability. How to effectively constrain the total energy to remain roughly unchanged during the CPU/GPU overclocking is a key issue in designing adaptive overclocking algorithms. There are two key factors during solving this key issue. Firstly, the dynamic power upper bound must be set to reflect the real-time behavior characteristics of the program so that algorithm can better meet the total energy unchanging constraints; secondly, instead of independently overclocking at both CPU and GPU sides, coordinately overclocking on CPU-GPU must be considered to adapt to real-time load balance for higher performance improvement and better energy constraints. This paper proposes an Adaptive Overclocking Algorithm (AOA) on CPU-GPU heterogeneous platforms to achieve the goal of performance improvement while the total energy remains roughly unchanged. AOA uses the function $$F_k$$ F k to describe the variable power upper bound and introduces the load imbalance factor W to realize the CPU-GPU coordinated overclocking. Through the verification of several types convolutional neural network algorithms on two CPU-GPU heterogeneous platforms (Intel $$^\circledR $$ ® Xeon E5-2660 & NVIDIA $$^\circledR $$ ® Tesla K80; Intel $$^\circledR $$ ® Core™i9-10920X & NIVIDIA $$^\circledR $$ ® GeForce RTX 2080Ti), AOA achieves an average of 10.7% performance improvement and 4.4% energy savings. To verify the effectiveness of the AOA, we compare AOA with other methods including automatic boost, the highest overclocking and static optimal overclocking.

In the last decades, Wireless Local Area Network (WLAN) has been emerging as one of the most prevailing networking architectures. It is expected that current WLAN technologies will further evolve to obtain much higher performance, more energy efficiency and more robustness. However, the WLAN is still prone to a variety of attacks regardless of the existence of data protection and security association mechanisms. They include but not limited to dictionary attacks against the pre-shared secret key of Wi-Fi Protected Access (WPA)/WPA2, the key reinstallation attack (KRACK) against the handshake procedure of WPA2, etc. Although a brand new WPA3 has been recently standardized by Wi-Fi Alliance to address new security threats, it needs a long time to upgrade currently used access points. Hence, there is a significant gap between security and deployment cost. To fill this gap, we design and implement an intellignet WLAN Early warning Platform (iWEP) to provide an early warning service for clients. Specifically, iWEP adopts intelligence algorithms, e.g., machine learning, to provide the capability of defeating existing popular attacks, including Wired Equivalent Privacy (WEP) secret cracking, WPA/WPA2 dictionary attack, Denial-of-Service and KRACK, by handling behaviour features that are extracted from the compromising procedures in real experimental environments. Moreover, iWEP uses edge computing technology to make a good tradeoff between system performance and WLAN security. Finally, we implement a prototype system of iWEP, and the real results demonstrate its effectiveness.