Virtual try-on of clothes has received much attention recently due to its commercial potential. It can be used for online shopping or intelligent recommendation to narrow down the selections to a few designs and sizes. In this paper, we present a mixed reality system for 3D virtual clothes try-on that enables a user to see herself wearing virtual clothes while looking at a mirror display, without taking off her actual clothes. The user can select various virtual clothes for trying-on. The system physically simulates the selected virtual clothes on the user's body in real-time and the user can see virtual clothes fitting on the her mirror image from various angles as she moves. The major contribution of this paper is that we automatically customize an invisible (or partially visible) avatar based on the user's body size and the skin color and use it for proper clothes fitting, alignment and clothes simulation in our virtual try-on system. We present three scenarios: i) virtual clothes on the avatar, ii) virtual clothes on the user's image and iii) virtual clothes on the avatar blended with the user's face image. We have conducted a user study to evaluate the effectiveness of these three solutions from the end user's perception of quality attributes, cognitive attributes and attitude towards using. The user study shows that among these three scenarios, the second one is most preferred by the users and for 50% of them the experience they had with our system was sufficient to make the purchase decision for the outfits they virtually tried-on.

This technical sketch presents a novel method to reconstruct 3D face model from only a single image. Different from other methods, ours does not require the face in the image to be in a specific pose. This method deforms a generic 3D face model to fit the shape of the face in the image. The reconstructed 3D face model is then textured using the image. There are many practical applications to this method. For example, it provides a photo-editing tool to change the face pose in the picture as required. It can also be used by the police to investigate the picture of a suspect where only 1 picture is available and it is necessary to have the picture from different poses. Another possible application is entertainment: the 3D face model can be used to personalize character in 3D games.