Checkpointing is becoming a hotspot of interest in both academia and industry as the primary fault-tolerance method for large model training. However, existing checkpoint designs are tightly coupled with the training process, leading to interruptions that reduce overall training efficiency. To reduce the impact of checkpoints on training, this paper presents FlowCheck, a novel checkpointing system that decouples checkpoint operations from the training process, enabling checkpoint saving without blocking the training. Specifically, FlowCheck updates the checkpoints by extracting complete gradient information from the network traffic of normal training. FlowCheck deploys a traffic-mirroring network to support this design. To utilize mirrored traffic for checkpointing operations, two key challenges need to be addressed. First, we need to achieve precise identification and extraction of gradient packets from training traffic. Second, the transmission on the mirror link is unreliable due to its inability to trigger retransmission upon packet loss. Through two key designs: (1) packet-counting-based traffic identification, and (2) packet redundancy recovery mechanism, FlowCheck implements an efficient checkpointing system using the existing training network and solves the above two challenges. Experiments and estimations verify that FlowCheck achieves checkpoint operations with zero impact on training, and demonstrate that FlowCheck achieves over 98% effective training time under practical fault conditions.

The development of video summarization techniques is crucial for quickly grasping video content. However, existing methods often fall short of comprehending long-range contextual dependencies and capturing critical information effectively so that redundant content is kept. To address these limitations, we propose the Global and Local CNN Attention Mechanisms for Video Summarization (GLA-VS). The proposed GLA-VS model is built upon the deep convolutional neural network architecture of GoogleNet and incorporates the Convolutional Block Attention Module (CBAM) to enhance the ability to identify and focus on key features. Moreover, we introduce a global feature attention module, which enables the model to jointly attend to both local details and global features. Integrating CBAM and a global feature attention module significantly enhances the model’s ability to accurately extract key frames and generate concise video summaries. Experiments conducted on the SumMe and TVSum datasets demonstrate the effectiveness of the GLA-VS model. The model outperforms existing methods in terms of Kendall’s Tau coefficient and Spearman’s rank correlation coefficient. Specifically, the GLA-VS model achieves a 10.57% improvement on the SumMe dataset compared to the state-of-the-art model. The improvements presented in this study provide a new perspective for video summarization, highlighting the potential of leveraging advanced attention mechanisms to boost model performance.

The user feedbacks could be delayed in many streaming recommendation scenarios. As an example, the user feedbacks to a recommended coupon consist of the immediate feedback on the click event and the delayed feedback on the resultant conversion. The delayed feedbacks pose a challenge of training recommendation models using instances with incomplete labels. When being applied to real products, the challenge becomes more severe as the streaming recommendation models need to be retrained very frequently and the training instances need to be collected over very short time scales. Existing approaches either simply ignore the unobserved feedbacks or heuristically adjust the feedbacks on a static instance set, resulting in biases in the training data and hurting the accuracy of the learned recommenders. In this paper, we propose a novel and theoretic sound counterfactual approach to adjusting the user feedbacks and learning the recommendation models, called CBDF (Counterfactual Bandit with Delayed Feedback). CBDF formulates the streaming recommendation with delayed feedback as a problem of sequential decision making and models it with a batched bandit. To deal with the issue of delayed feedback, at each iteration (episode), a counterfactual importance sampling model is employed to re-weight the original feedbacks and generate the modified rewards. Based on the modified rewards, a batched bandit is learned for conducting online recommendation at the next iteration. Theoretical analysis showed that the modified rewards are statistically unbiased, and the learned bandit policy enjoys a sub-linear regret bound. Experimental results demonstrated that CBDF can outperform the state-of-the-art baselines on a synthetic dataset, the Criteo dataset, and a dataset from Tencent's WeChat app.