istributedistributedD parallel training of large-scale deep neural networks (DNN) has attracted the attentions of both artificial intelligence and high-performance distributed computing. One of efficient approaches is the micro-batch-based pipeline parallelism (MBPP), e.g., GPipe and Terapipe. Based on the MBPP, we establish a time-cost model with the basic time function of layers, which considers computing time and communication time simultaneously as well as considers they are nonlinear with the amount of input data. Focusing on the jointly optimal solutions of network division and data partition, we propose a Cross-Search algorithm with Improved Multi-dimensional Dichotomy (CSIMD). Through theoretical derivation, we prove improved multi-dimensional dichotomy (IMD) has appreciable theoretical optimality and linear computational complexity significantly faster than the state-of-the-art methods including dynamic programming and recursive algorithm. Extensive experiments on both CNN-based and transformer-based neural networks demonstrate our proposed CSIMD can obtain optimal network division and data partition schemes under MBPP. On average, the training speeds of CSIMD in CNN- and transformer-based DNNs are respectively (2.0, 2.5)× and (2.66, 5.48)× of (MBPP-R, MBPP-E).

This article addresses the issue of mismatched communication delay (CD) in dual-channel nonlinear networked control systems (NCSs) resulting from the quality of service (QoS) mechanism’s queue management. The focus is on the importance of ensuring communication security in NCSs, particularly in the presence of asynchronous cyber attacks (ACAs). First, improved Lyapunov–Krasovskii functions (LKFs) are constructed, taking into account sampling signals, CDs, and nonlinearities in the system. Additionally, a novel looped-functional approach is introduced to reduce the initial constraint of the criterion. Then, the control algorithm’s performance is optimized by achieving a tighter upper bound on the integral term and applying quadratic scaling. To ensure stability and security under ACAs, a fuzzy memory sample-data controller (MSAC) is proposed. This controller leverages a machine learning algorithm to address the optimization problem of data sampling period selection, thereby minimizing resource usage and operating efficiently within the system’s limited bandwidth. Finally, numerical simulations are conducted using the dynamic equations of the inverted pendulum system (IPS) to validate the practicality of the proposed theoretical approach.

In recent years, the rapid development of mobile devices and embedded system raises a demand for intelligent models to address increasingly complicated problems. However, the complexity of the structure and extensive parameters press significantly on efficiency, storage space, and energy consumption. Additionally, the explosive growth of tasks with enormous model structures and parameters makes it impossible to compress models manually. Thus, a standardized and effective model compression solution achieving lightweight neural networks is established as an urgent demand by the industry. Accordingly, Dynamic Channel Ranking Strategy (DCRS) method is proposed to compress deep convolutional neural networks. DCRS selects channels with high contribution of each prunable layer according to compression ratio searched by reinforcement learning agent. Compared with current model compression methods, DCRS efficaciously applies various channel ranking strategies on prunable layers. Experiments indicate with a 50% compression ratio, compressed MobileNet achieved 70.62% top1 and 88.2% top5 accuracy on ImageNet, and compressed ResNet achieved 92.03% accuracy on CIFAR-10. DCRS reduces more FLOPS in these neural networks. The compressed model achieves the best Top-1 and Top-5 accuracy on ResNet50, the best Top-1 accuracy on MobilNetV1.