In most Virtual Reality applications, people try to achieve a high degree of realism. Numerous techniques and approaches have been developed to achieve this by using pictures and videos of the real world.At REALVIZ, we have developed software products allowing digital content creators to acquire shape and motion from real images, using computer vision techniques. In this presentation I will give a snapshot of the current status of these technologies, and how they are used in various industries for VR- or AR-related applications.

The global scope of the CREATE project is to develop a mixed-reality framework that enables highly interactive real-time construction and manipulation of photo-realistic, virtual worlds based on real data sources. This framework will be tested and applied to cultural heritage content in an educational context, as well as to the design and review of architectural/urban planning settings. The evaluation of the project is based on a human-centered, constructivist approach to working and learning, with special attention paid to the evaluation of the resulting mixed reality experience. Through this approach, participants in an activity "construct" their own knowledge by testing ideas and concepts based on their prior knowledge and experience, applying these to a new situation, and integrating the new knowledge gained with pre-existing intellectual constructs. CREATE project uses a high degree of interactivity, and includes provision for other senses (haptics and sound). The application developed in CREATE are designed to run on different platforms, and the targeted running systems are SGI and PC driven, with immersive stereo-displays such as a workbench, a ReaCTor (CAVE-like environment), and a wide projection screen.

Computer augmented reality (CAR) is a rapidly emerging field which enables users to mix real and virtual worlds. Our goal is to provide interactive tools to perform common illumination, i.e., light interactions between real and virtual objects, including shadows and relighting (real and virtual light source modification). In particular, we concentrate on virtually modifying real light source intensities and inserting virtual lights and objects into a real scene; such changes can be very useful for virtual lighting design and prototyping. To achieve this, we present a three-step method. We first reconstruct a simplified representation of real scene geometry using semiautomatic vision-based techniques. With the simplified geometry, and by adapting recent hierarchical radiosity algorithms, we construct an approximation of real scene light exchanges. We next perform a preprocessing step, based on the radiosity system, to create unoccluded illumination textures. These replace the original scene textures which contained real light effects such as shadows from real lights. This texture is then modulated by a ratio of the radiosity (which can be changed) over a display factor which corresponds to the radiosity for which occlusion has been ignored. Since our goal is to achieve a convincing relighting effect, rather than an accurate solution, we present a heuristic correction process which results in visually plausible renderings. Finally, we perform an interactive process to compute new illumination with modified real and virtual light intensities.

Many classical image processing tasks can be realized as evaluations of a Boolean function over subsets of an image. For instance, the simplicity test used in 3D thinning requires examining the 26 neighbors of each voxel and computing a single Boolean function of these inputs. In this article, we show how binary decision diagrams can be used to produce automatically very efficient and compact code for such functions. The total number of operations performed by a generated function is at most one test and one branching for each input value (e.g., in the case of 3D thinning, 26 tests and branchings). At each stage, the function is guaranteed to examine only the pertinent input data, i.e., the values which affect the result. As an example, we consider the 3D simplicity test in digital topology, and thinning processes. We produce functions much faster than our previously optimized implementations and than any other implementation we know of. In the case of 3D simplicity test, on average, at each voxel only 8.7 neighboring voxel values are examined.

REALISE was designed to extract from sequences of images, acquired with a moving camera, the information necessary for determining the 3D (CAD-like) structure of a real-life scene together with information about the radiometric signatures of surfaces bounding the extracted 3D objects (e.g. reflectance behaviour). The retrieved information is then integrated in a virtual reality (VR) software environment. The R&D work is been performed principally in the following areas of computer vision and computer graphics: structure from motion, recovery of geometries, recovery of photometric and texture information, highly realistic rendering on the basis of empirically-based reflectance models, and the design and development of improved rendering processes together with a new VR system. Beside this innovative R&D work another key aspect of REALISE is to have computer vision & computer graphics cooperate to produce realistic 3D data efficiently.

Presents a vision system for autonomous navigation based on stereo perception without 3D reconstruction. This approach uses weakly calibrated stereo images, i.e. images for which only the epipolar geometry is known. The vision system first rectifies the images, matches selected points between the two images, and then computes the relative elevation of the points relative to a reference plane as well as the images of their projections on this plane. We have integrated this vision module into a complete navigation system. In this system, the relative elevation is used as a shape indicator in order to compute appropriate steering directions everytime a new stereo pair is processed. We have conducted initial experiments in unstructured, outdoor environments with an wheeled rover.>

We present an original approach to the problem of camera calibration. Contrary to classical techniques, which first extract the image features and then compute the camera parameters, we directly search for the camera parameters that best map three-dimensional points onto the image edges, characterized as maxims of the intensity gradient or zero-crossings of the Laplacian. Expressed as a one-stage optimization problem over the parameters of the camera, the whole calibration process is solved by classical iterative optimization. We describe experiments on synthetic and real data.

Addresses the well-known problem of estimating the motion and structure of a plane, but in the case where the visual system is not calibrated and in an unbounded monocular image sequence. The authors first define plane collineations and analyse some of their properties when utilized to analyse the retinal motion in an uncalibrated image sequence. The authors show how to relate them to the Euclidian parameters of the scene. In particular, the authors discuss how to detect and estimate the collineation of the plane at infinity and use this quantity for auto-calibration.

The authors show some very recent results using the weak-calibration idea. Assuming that only the epipolar geometry is known of a pair of stereo images, encoded in the so-called fundamental matrix, it is shown that some useful three-dimensional information such as relative positions of points and planes and 3-D convex hulls can be computed. The notion of visibility is introduced, which allows deriving those properties. Results of both synthetic and real data are shown.>

This paper describes the development and implementation of a reactive visual module utilized on an autonomous mobile robot to automatically correct in trajectory. The authors use a multisensorial mechanism based on inertial and visual cues. The authors report only on the implementation and the experimentation of this module, whereas the main theoretical aspects have been developed elsewhere.

An edge-based trinocular stereovision algorithm is presented. The primitives it works on are cubic B-spline approximations of the 2-D edges. This allows one to deal conveniently with curvature and to extend to some nonpolyhedral scenes to previous stereo algorithms. To build a matching primitive, the principle of the algorithm is, first, to find a triplet of corresponding points on three splines. This is provided by the bootstrapping part. Second, the algorithm propagates along the three supporting splines to find other matching points. This provides a set of ordered point triplets along these three splines, for which all the matching constraints are verified. This primitive constitutes a trinocular hypothesis. The set of all hypotheses is obtained by propagating from all the point triplets provided by the bootstrapping process. A criterion based on the size of the hypotheses is then used to choose among them a compatible set with respect to the uniqueness constraint. Results of several 3-D reconstructed scenes are shown.>