Traditional VR is mostly about headset experiences either in completely virtual environments or 360° videos. On the other hand AR has been mixing realities by inserting the virtual within the real. In this work we present the Augmented VR concept that lies at the middle right of the virtuality continuum, typically referred to as augmented virtuality. We offer another perspective by blending the real within the virtual focusing on capturing actual human performances in three dimensions and emplacing them within virtual environments [1]-[3]. By compressing and transmitting this new type of 3D media we can also achieve real-time interaction, communication and collaboration between users. Being in full 3D our media are compatible with a variety of applications be it either VR, AR, MR and open up new exciting opportunities like free viewpoint spectating while also increasing the feeling of immersion of all participating users. We demonstrate our technology via a prototype two player game that can support spectating in various devices like head mounted displays (VR) or tablet laptops (AR). Our system is easy to setup, requiring minimal non-technical human intervention, and relatively low cost taking one step ahead in making this technology available to the consumer public.

Camera pose estimation is a fundamental problem of Augmented Reality and 3D reconstruction systems. Recently, despite the new better performing direct methods being developed, state-of-the-art methods are still estimating erroneous poses due to sensor noise, environmental conditions and challenging trajectories. Adding a back-end mapping process, SLAM systems achieve better performance and are more robust, but require higher computational resources, limiting their applicability. Therefore, lighter solutions to improve the accuracy of pose estimates are required. In this work we demonstrate the effectiveness of lighter data structures, namely surface elements, and exploit the temporality of sensor data streams to accumulate moving camera frames and improve tracking. This representation allows us to splat a photometric and geometric model simultaneously and use it to improve the performance of dense RGB-D camera pose estimation methods. Exploiting Elliptical Weighted Average splatting to produce high quality photometric results also allows us to detect erroneous poses through a novel visual quality analysis process. We show evidence of the EWA temporal model's effectiveness in publicly available datasets and argue that point-based representations are a good candidate for building lighter systems that should be further explored.