Camera shake during exposure blurs the captured image. Despite several decades of studies, image deconvolution to restore a blurred image still remains an issue, particularly in blind deconvolution cases in which the actual shape of the blur is unknown. Approaches based on cepstral analysis succeeded in restoring images degraded by a uniform blur caused by a camera moving straight in a single direction. In this paper, we propose to estimate, from a single blurred image, the point spread function (PSF) caused by a normal camera undergoing a 2D curved motion, and to restore the image. To extend the traditional cepstral analysis, we derive assumptions about the PSF effects in the cepstrum domain. In a first phase, we estimate several PSF candidates from the cepstrum of a blurred image and restore the image with a fast deconvolution algorithm. In a second phase, we select the best PSF candidate by evaluating the restored images. Finally, a slower but more accurate deconvolution algorithm recovers the latent image with the chosen PSF. We validate the proposed method with synthetic and real experiments.

We propose a novel augmented reality (AR) setup with an omni directional camera on a table top display. The table acts as a mirror on which real playing cards appear augmented with virtual elements. The omni directional camera captures and recognizes its surrounding based on a feature based image retrieval approach which achieves fast and scalable registration. It allows our system to superimpose virtual visual effects to the omni directional camera image. In our AR card game, users sit around a table top display and show a card to the other players. The system recognizes it and augments it with virtual elements in the omni directional image acting as a mirror. While playing the game, the users can interact with each other directly and through the display. Our setup is a new, simple, and natural approach to augmented reality. It opens new doors to traditional card games.

In this paper, we present an image retrieval method based on feature tracking. Feature tracks are summarized into a compact discreet value and used for video indexing purpose. As opposed to existing space-time features, we do not make any assumption on the motion visible on the indexed videos. As a result, given an example query, our system is able to retrieve related videos from a large database. We evaluated our system with the copy detection benchmark MUSCLE-VCD-2007. We also ran retrieval experiment on hours of TV broadcast.

Tracking is a major issue of virtual and augmented reality applications. Single object tracking on monocular video streams is fairly well understood. However, when it comes to multiple objects, existing methods lack scalability and can recognize only a limited number of objects. Thanks to recent progress in feature matching, state-of-the-art image retrieval techniques can deal with millions of images. However, these methods do not focus on real-time video processing and can not track retrieved objects. In this paper, we present a method that combines the speed and accuracy of tracking with the scalability of image retrieval. At the heart of our approach is a bi-layer clustering process that allows our system to index and retrieve objects based on tracks of features, thereby effectively summarizing the information available on multiple video frames. As a result, our system is able to track in real-time multiple objects, recognized with low delay from a database of more than 300 entries.

This paper describes an artwork that relies on recent computer vision and augmented reality techniques to animate the illustrations of a poetry book. Because we donpsilat need markers, we can achieve seamless integration of real and virtual elements to create the desired atmosphere. The visualization is done on a computer screen to avoid cumbersome head-mounted displays. The camera is hidden into a desk lamp for easing even more the spectator immersion.

We present a nonrigid registration technique that achieves spatial, photometric, and visibility accuracy. It lets us photo-realistically augment 3D deformable surfaces under complex illumination conditions and in spite of severe occlusions. There are many approaches that address some of these issues but very few that simultaneously handle all of them as we do. We use triangulated meshes to model the geometry and introduce explicit visibility maps as well as separate illumination parameters for each mesh vertex. We cast our registration problem in an expectation maximization framework that allows robust and fully automated operation. It provides explicit illumination and occlusion models that can be used for rendering purposes.

Three-dimensional detection and shape recovery of a nonrigid surface from video sequences require deformation models to effectively take advantage of potentially noisy image data. Here, we introduce an approach to creating such models for deformable 3D surfaces. We exploit the fact that the shape of an inextensible triangulated mesh can be parameterized in terms of a small subset of the angles between its facets. We use this set of angles to create a representative set of potential shapes, which we feed to a simple dimensionality reduction technique to produce low-dimensional 3D deformation models. We show that these models can be used to accurately model a wide range of deforming 3D surfaces from video sequences acquired under realistic conditions.

We propose a fully automated approach to calibrating multiple cameras whose fields of view may not all overlap. Our technique only requires waving an arbitrary textured planar pattern in front of the cameras, which is the only manual intervention that is required. The pattern is then automatically detected in the frames where it is visible and used to simultaneously recover geometric and photometric camera calibration parameters. In other words, even a novice user can use our system to extract all the information required to add virtual 3D objects into the scene and light them convincingly. This makes it ideal for Augmented Reality applications and we distribute the code under a GPL license.

We present a real-time method for detecting deformable surfaces, with no need whatsoever for a priori pose knowledge. Our method starts from a set of wide baseline point matches between an undeformed image of the object and the image in which it is to be detected. The matches are used not only to detect but also to compute a precise mapping from one to the other. The algorithm is robust to large deformations, lighting changes, motion blur, and occlusions. It runs at 10 frames per second on a 2.8 GHz PC and we are not aware of any other published technique that produces similar results. Combining deformable meshes with a well designed robust estimator is key to dealing with the large number of parameters involved in modeling deformable surfaces and rejecting erroneous matches for error rates of up to 95%, which is considerably more than what is required in practice.

We present a real-time system that can draw virtual patterns or images on deforming real objects by estimating both the deformations and the shading parameters. We show that this is what is required to render the virtual elements so that they blend convincingly with the surrounding real textures. The whole process of uncompressing the video stream, measuring the deformations, estimating the lighting parameters, and realistically augmenting the input image takes about 100 ms on a 2.8 GHz PC. It is fully automated and does not require any manual initialization or engineering of the scene. It is also robust to large deformations, lighting changes, motion blur, specularities, and occlusions. It can therefore be demonstrated live on a simple laptop.