The rapid development of short-form video platforms has provided new avenues for advertisement recommendation, where advertisements are typically displayed as a thumbnail button at the beginning of a video and expand after a certain period. The audio-visual content of a video usually impacts the emotions and attention of users, which further partially determines user behaviors after the advertisement expands. This impact varies depending on the moment of the expansion. Hence, the task of predicting the expansion moment for each video is essential to improve the quality of advertisement recommendation, which is the focus of this paper. The main challenge of this task lies in the requirement to understand both audio-visual video content and user behaviors. To facilitate our research on this task, we first collect a large-scale dataset containing 18,901 videos with over 1.58 billion real user-watching records as the annotations. We also propose a quantitative metric for the task. To address the challenge of the task as well as predict a proper expansion moment for each video, we develop a prediction framework. Firstly, we utilize pretrained models to extract features, then refine extracted features via intra-modal and inter-modal interactions to enhance the understanding of the audio-visual content. Secondly, we propose an uncertainty-based logit adjustment strategy, which leverages prior preferences of user behaviors to adjust the output logit of the model adaptively, achieving a better prediction of user behaviors. Extensive experiments demonstrate the effectiveness of our proposed framework. We hope this work could be a meaningful step towards better video understanding and advertisement recommendation in short-form video platforms. Code and dataset are available at https://github.com/GeWu-Lab/WCAE.

A key challenge in the task of human pose and shape estimation is occlusion, including self-occlusions, object-human occlusions, and inter-person occlusions. The lack of diverse and accurate pose and shape training data becomes a major bottleneck, especially for scenes with occlusions in the wild. In this paper, we focus on the estimation of human pose and shape in the case of inter-person occlusions, while also handling object-human occlusions and self-occlusion. We propose a novel framework that synthesizes occlusion-aware silhouette and 2D keypoints data and directly regress to the SMPL pose and shape parameters. A neural 3D mesh renderer is exploited to enable silhouette supervision on the fly, which contributes to great improvements in shape estimation. In addition, keypoints-and-silhouette-driven training data in panoramic viewpoints are synthesized to compensate for the lack of viewpoint diversity in any existing dataset. Experimental results show that we are among the state-of-the-art on the 3DPW and 3DPW-Crowd datasets in terms of pose estimation accuracy. The proposed method evidently outperforms Mesh Transformer, 3DCrowdNet and ROMP in terms of shape estimation. Top performance is also achieved on SSP-3D in terms of shape prediction accuracy. Demo and code will be available at https://igame-lab.github.io/LASOR/.