The rapid proliferation of large language models (LLMs), such as GPT-4 and Gemini, underscores the intense demand for resources during their training processes, posing significant challenges due to substantial computational and environmental costs. In this paper, we introduce a novel checkpoint merging strategy aimed at making efficient use of intermediate checkpoints during LLM pretraining. This method utilizes intermediate checkpoints with shared training trajectories, and is rooted in an extensive search space exploration for the best merging weight via Bayesian optimization. Through various experiments, we demonstrate that: (1) Our proposed methodology exhibits the capacity to augment pretraining, presenting an opportunity akin to obtaining substantial benefits at minimal cost; (2) Our proposed methodology, despite requiring a given held-out dataset, still demonstrates robust generalization capabilities across diverse domains, a pivotal aspect in pretraining.

This paper investigates the secure cooperative control problem of second-order nonlinear multi-agent systems (MASs) under denial-of-service (DoS) attacks based on dynamic event-triggered schemes. First, a dynamic event-triggered control protocol is designed, which can adaptively adjust the triggering threshold according to the state information of MASs, it can also reduce the number of triggering times and save the communication resources. Then, based on the parameters of control protocol, intensity of DoS attacks, dynamics of MASs, and communication topologies, some sufficient conditions are derived for MASs to guarantee consensus under DoS attacks. In addition, it is proved that under the proposed dynamic event-triggered schemes and control protocol, the MASs can achieve consensus, and the event-triggered schemes can exclude Zeno behavior. Finally, numerical simulation example is presented to illustrate the effectiveness of the main results.

Deyuan Liu, Zhanyue Qin, Hairu Wang, Zhao Yang, Zecheng Wang, Fangying Rong, Qingbin Liu, Yanchao Hao, Bo Li, Xi Chen, Cunhang Fan, Zhao Lv, Dianhui Chu, Zhiying Tu, Dianbo Sui. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Zhanyue Qin, Haochuan Wang, Deyuan Liu, Ziyang Song, Cunhang Fan, Zhao Lv, Jinlin Wu, Zhen Lei, Zhiying Tu, Dianhui Chu, Xiaoyan Yu, Dianbo Sui. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

This paper explores the optimal containment control problem for nonlinear and underactuated quadrotors with multiple team leaders governed by nonlinear dynamics, employing the reinforcement learning. A cascade controller is formulated, comprising a position control component to ensure containment achievement and an attitude control component to govern rotational channel. The proposed optimal control protocols derived from historical data collected from quadrotor systems without requirement for exact knowledge of vehicle dynamics. The simulation illustrates the effectiveness of the proposed controller in managing a quadrotor team with multiple leaders.

This paper investigates the optimal formation control of a heterogeneous multiagent system consisting of multiple quadrotors and ground vehicles via reinforcement learning to achieve the time-varying formation under switching topologies. A distributed observer is firstly constructed to generate references using local information for each vehicle to form time-varying formation and the convergence of the observer under switching topologies is proven. Then, reinforcement learning methods are provided for the heterogeneous vehicle group to realize the optimal tracking control without information of vehicle dynamical model. Simulation tests are given to confirm the effectiveness of the proposed method.

This paper investigates the coordination control problem for a special nonlinear heterogeneous multi-agent system consisting of tail-sitter unmanned aerial vehicles and unmanned ground vehicles with uncertainties and disturbances. A robust hierarchical pinning control scheme is proposed for the heterogeneous multi-agent system to restrain the uncertainties and disturbances and achieve coordination scenarios. The heterogeneous multi-agent system can realize coordination tasks by selecting proper pinning nodes and estimating coupling strength. The robustness of the whole system is proven utilizing the Lyapunov stability theorem. The effectiveness of the robust hierarchical pining control method is validated by simulation scenarios.

Event-based video frame interpolation is promising as event cameras capture dense motion signals that can greatly facilitate motion-aware synthesis. However, training existing frameworks for this task requires high frame-rate videos with synchronized events, posing challenges to collect real training data. In this work we show event-based frame interpolation can be trained without the need of high frame-rate videos. This is achieved via a novel weakly supervised framework that 1) corrects image appearance by extracting complementary information from events and 2) supplants motion dynamics modeling with attention mechanisms. For the latter we propose subpixel attention learning, which supports searching high-resolution correspondence efficiently on low-resolution feature grid. Though trained on low frame-rate videos, our framework outperforms existing models trained with full high frame-rate videos (and events) on both GoPro dataset and a new real event-based dataset. Codes, models and dataset will be made available at: https://github.com/YU-Zhiyang/WEVI.

This paper mainly addresses the formation control problem for a group of tail-sitters in transition flight between forward and vertical flight. A robust formation control method is proposed to achieve the aggressive time-varying formation subject to nonlinear dynamics and uncertainties. For each tail-sitter, the proposed control method results in a composite controller that includes a trajectory tracking controller and an attitude controller to achieve the translational and rotational motion control, respectively. It is proven that tracking errors of the proposed global closed-loop system can converge to a given neighborhood around the origin in a finite time. Finally, the simulation studies for multiple tail-sitters to accomplish the time-varying formation in transition flight are presented to show the effectiveness of the proposed control strategy.

In this paper, the fault-tolerant time-varying formation control problem for a group of tail-sitters with multiple actuator faults and uncertainties is studied. A robust distributed fault-tolerant formation control strategy is developed to achieve aggressive time-varying formation flying in flight mode transitions. For each tail-sitter, the designed controller can be divided into an inner attitude controller and an outer position controller to govern the rotational and translational motions, respectively. The information of the actuator faults does not need to be identified online and the tracking errors of the global closed-loop control system can converge into a given neighborhood of the origin in a finite time. Simulation results are presented to show the effectiveness of the proposed control strategy.

High efficiency video coding (HEVC), which is the newest video coding standard currently, achieves the best coding efficiency compared with all the other existing video coding standards. However, the computational complexity of the typical HEVC encoder dramatically increases because of the recursive searching scheme for finding the best coding unit (CU) partitions. In this paper, an adaptive fast CU size decision algorithm for HEVC Intra prediction is proposed based on CU complexity classification (CC) by using machine learning (ML) technology. Firstly, certain image features are extracted to characterize the CU complexity, which has a strong relationship with CU partitions, and then, the support vector machine is employed to analyze and construct the classification model according to the CU complexity. Finally, the proposed adaptive fast CU size decision algorithm, named as CCML, is released based on the complexity classification. The experimental results show that the proposed algorithm could achieve around 60% encoding time reduction for various test video sequences on average with only 1.26% Bjontegaard delta bit rate increase compared with the reference test model HM15.0 of HEVC.