In computational advertising, predicting the post-click conversion rate (CVR) using deep neural networks (DNNs) benefits a lot from privileged features, which can be collected for offline training but are unavailable during online serving.To utilize these privileged signals, privileged features distillation (PFD) methods incorporate a teacher model with privileged features to guide the CVR model.However, existing PFD approaches fail to put more emphasis on poorly predicted instances where the teacher's guidance is most crucial, and thus suffer from overconfidence on "easy" instances.In this work, we propose Hardness-aware Privileged Features Distillation (HA-PFD) for enhancing CVR prediction in real-world advertising recommender systems.We specifically design focal-style distillation losses that adaptively adjust the weight of each instance based on its "hardness".This method prioritizes poorly predicted instances during the distillation process, resulting in improved ranking performance and better model calibration.Additionally, we incorporate latent-level distillation into the PFD framework for the first time, which facilitates the student's representation learning through a straightforward layer alignment approach.We also propose a method for selecting privileged features based on their relevance to the conversion label.We conduct extensive offline experiments on large-scale, real-world datasets and online experiments on Douyin, a short video platform with billions of live users.In the offline evaluation, HA-PFD exhibits competitive performance and superior model calibration compared to existing state-of-the-art methods.In the online experiments, HA-PFD significantly improves advertiser value and conversions.Now we have deployed HA-PFD as the main online serving model on our short video platform.

Robot sports, characterized by well-defined objectives, explicit rules, and dynamic interactions, present ideal scenarios for demonstrating embodied intelligence. In this paper, we present VolleyBots, a novel robot sports testbed where multiple drones cooperate and compete in the sport of volleyball under physical dynamics. VolleyBots integrates three features within a unified platform: competitive and cooperative gameplay, turn-based interaction structure, and agile 3D maneuvering.These intertwined features yield a complex problem combining motion control and strategic play, with no available expert demonstrations.We provide a comprehensive suite of tasks ranging from single-drone drills to multi-drone cooperative and competitive tasks, accompanied by baseline evaluations of representative reinforcement learning (RL), multi-agent reinforcement learning (MARL) and game-theoretic algorithms. Simulation results show that on-policy RL methods outperform off-policy methods in single-agent tasks, but both approaches struggle in complex tasks that combine motion control and strategic play.We additionally design a hierarchical policy which achieves 69.5% win rate against the strongest baseline in the 3 vs 3 task, demonstrating its potential for tackling the complex interplay between low-level control and high-level strategy.To highlight VolleyBots’ sim-to-real potential, we further demonstrate the zero-shot deployment of a policy trained entirely in simulation on real-world drones.

Aligning the world model with the environment for the agent’s specific task is crucial in model-based reinforcement learning. While value-equivalent models may achieve better task awareness than maximum-likelihood models, they sacrifice a large amount of semantic information and face implementation issues. To combine the benefits of both types of models, we propose Task-aware Environment Modeling Pipeline with bi-level Optimization (TEMPO), a bi-level model learning framework that introduces an additional level of optimization on top of a maximum-likelihood model by incorporating a meta weighter network that weights each training sample. The meta weighter in the upper level learns to generate novel sample weights by minimizing a proposed task-aware model loss. The model in the lower level focuses on important samples while maintaining rich semantic information in state representations. We evaluate TEMPO on a variety of continuous and discrete control tasks from the DeepMind Control Suite and Atari video games. Our results demonstrate that TEMPO achieves state-of-the-art performance regarding asymptotic performance, training stability, and convergence speed.