Virtual try-on technology enables users to try various fashion items using augmented reality and provides a convenient online shopping experience. However, most previous works focus on the virtual try-on for clothes while neglecting that for shoes, which is also a promising task. To this concern, this work proposes a real-time augmented reality virtual shoe try-on system for smartphones, namely ARShoe. Specifically, ARShoe adopts a novel multi-branch network to realize pose estimation and segmentation simultaneously. A solution to generate realistic 3D shoe model occlusion during the try-on process is presented. To achieve a smooth and stable try-on effect, this work further develop a novel stabilization method. Moreover, for training and evaluation, we construct the very first large-scale foot benchmark with multiple virtual shoe try-on task-related labels annotated. Exhaustive experiments on our newly constructed benchmark demonstrate the satisfying performance of ARShoe. Practical tests on common smartphones validate the real-time performance and stabilization of the proposed approach.

Product display in e-commerce commonly uses static images or videos. In this paper, we design a novel video composition system “RotateView” which displays real-world products videos on the mobile phone in an interactive and 3D-like way. The system estimates the rotation direction of products using optical flow, segments products using an improved version of online adaptive video object segmentation, and adjusts motion and color for composition. The rotation direction estimation methods are evaluated on a video dataset of 1500 rotated videos that are uploaded by sellers, and on our collected real-world video object segmentation (RVOS) dataset. On our RVOS dataset, experimental results show an improvement in segmentation over the state of the art. Moreover, the proposed algorithms for motion and color adjustment are evaluated using extensive user studies. The RVOS dataset can be downloaded viahttps://realvos.github.io/for further studies and code will be made available.