Short-video sharing platforms engaging billions of users have attracted intense interest recently. A key insight is that user feedback on these platforms is heavily influenced by preceding exposed videos in the same request, called context cumulative effects. For example, multiple repeated videos in a request often cause user fatigue and influence user feedback. However, related factors, such as the other exposed items in the same request, are available during model training but not accessible during online serving. Vanilla distillation methods mitigate the training-inference inconsistency, struggling to capture the dynamic dependence between context cumulative effects and user feedback. To address this problem, we propose the Mutual Information-aware Knowledge Distillation (MIKD) framework, which fuses such effects and user-item matching degrees by evaluating their impacts on user feedback based on mutual information estimation. Rigorous analysis and extensive experiments demonstrate that MIKD precisely extracts personal interests and consistently improves performance. We conduct online A/B testing on a leading short-video sharing mobile app, and the results demonstrate the effectiveness of the proposed method. MIKD has been successfully deployed online to serve the main traffic and optimize user experiences.

In many real-world applications of reinforcement learning (RL), performing actions requires consuming certain types of resources that are non-replenishable in each episode. Typical applications include robotic control with limited energy and video games with consumable items. In tasks with non-replenishable resources, we observe that popular RL methods such as soft actor critic suffer from poor sample efficiency. The major reason is that, they tend to exhaust resources fast and thus the subsequent exploration is severely restricted due to the absence of resources. To address this challenge, we first formalize the aforementioned problem as a resource-restricted reinforcement learning, and then propose a novel resource-aware exploration bonus (RAEB) to make reasonable usage of resources. An appealing feature of RAEB is that, it can significantly reduce unnecessary resource-consuming trials while effectively encouraging the agent to explore unvisited states. Experiments demonstrate that the proposed RAEB significantly outperforms state-of-the-art exploration strategies in resource-restricted reinforcement learning environments, improving the sample efficiency by up to an order of magnitude.

Decentralized optimization algorithms have attracted intensive interests recently, as it has a balanced communication pattern, especially when solving large-scale machine learning problems. Stochastic Path Integrated Differential Estimator Stochastic First-Order method (SPIDER-SFO) nearly achieves the algorithmic lower bound in certain regimes for nonconvex problems. However, whether we can find a decentralized algorithm which achieves a similar convergence rate to SPIDER-SFO is still unclear. To tackle this problem, we propose a decentralized variant of SPIDER-SFO, called decentralized SPIDER-SFO (D-SPIDER-SFO). We show that D-SPIDER-SFO achieves a similar gradient computation cost—that is, O(ε−3) for finding an ϵ-approximate first-order stationary point—to its centralized counterpart. To the best of our knowledge, D-SPIDER-SFO achieves the state-of-the-art performance for solving nonconvex optimization problems on decentralized networks in terms of the computational cost. Experiments on different network configurations demonstrate the efficiency of the proposed method.