In this paper we introduce an image-based 3D capture process for the creation and display of photorealistic virtual environments (VEs). The resulting VEs aim to realistically recreate existing real-world scenes that can be displayed in a range of immersive VR systems using a high-quality, view-dependent algorithm and further enhanced using advanced vegetation, shadow display algorithms and 3D sound. The scenes, an archaeological site and an urban environment, were chosen according to real-world applications in the areas of urban planning/architecture and cultural heritage education. The users in each case are able to reconstruct or manipulate elements of the VEs according to their needs, as these have been specified through a detailed user requirements survey. Furthermore, a user task analysis and scenario-based approach has been adopted for the design of the virtual prototypes and the evaluation, which is currently underway. This work is being developed in the context of the EU-funded research project CREATE and the first examples of the prototype system in use are described and demonstrated in this paper.

Reconstructing and rendering trees is a challenging problem due to the geometric complexity involved, and the inherent difficulties of capture. In this paper we propose a volumetric approach to capture and render trees with relatively sparse foliage. Photographs of such trees typically have single pixels containing the blended projection of numerous leaves/branches and background. We show how we estimate opacity values on a recursive grid, based on alphamattes extracted from a small number of calibrated photographs of a tree. This data structure is then used to render billboards attached to the centers of the grid cells. Each billboard is assigned a set of view-dependent textures corresponding to each input view. These textures are generated by approximating coverage masks based on opacity and depth from the camera. Rendering is performed using a view-dependent texturing algorithm. The resulting volumetric tree structure has low polygon count, permitting interactive rendering of realistic 3D trees. We illustrate the implementation of our system on several different real trees, and show that we can insert the resulting model in virtual scenes.

The global scope of the CREATE project is to develop a mixed-reality framework that enables highly interactive real-time construction and manipulation of photo-realistic, virtual worlds based on real data sources. This framework will be tested and applied to cultural heritage content in an educational context, as well as to the design and review of architectural/urban planning settings. The evaluation of the project is based on a human-centered, constructivist approach to working and learning, with special attention paid to the evaluation of the resulting mixed reality experience. Through this approach, participants in an activity "construct" their own knowledge by testing ideas and concepts based on their prior knowledge and experience, applying these to a new situation, and integrating the new knowledge gained with pre-existing intellectual constructs. CREATE project uses a high degree of interactivity, and includes provision for other senses (haptics and sound). The application developed in CREATE are designed to run on different platforms, and the targeted running systems are SGI and PC driven, with immersive stereo-displays such as a workbench, a ReaCTor (CAVE-like environment), and a wide projection screen.

Capturing and rendering of real scenes in an immersive virtual environment is still a challenging task. We present a novel workflow, using high-quality, view-dependent projective texturing at low texture memory cost, while reinforcing artist's control over image quality. Photographs of a real scene are first used to create a 3D model with standard tools. Our method automatically orders geometry into optimized visibility layers for each photograph. These layers are subsequently used to create standard 2D image-editing layers, enabling artists to fill in missing texture using standard techniques such as clone brushing. The result of this preprocess is used by our novel layered projective texture rendering algorithm, which has low texture memory consumption, high interactive image quality and avoids the need for subdividing geometry for visibility. We show results of our implementation on a real-world project.