Since the initial comparison of Seitz et al., the accuracy of dense multiview stereovision methods has been increasing steadily. A number of limitations, however, make most of these methods not suitable to outdoor scenes taken under uncontrolled imaging conditions. The present work consists of a complete dense multiview stereo pipeline which circumvents these limitations, being able to handle large-scale scenes without sacrificing accuracy. Highly detailed reconstructions are produced within very reasonable time thanks to two key stages in our pipeline: a minimum s-t cut optimization over an adaptive domain that robustly and efficiently filters a quasidense point cloud from outliers and reconstructs an initial surface by integrating visibility constraints, followed by a mesh-based variational refinement that captures small details, smartly handling photo-consistency, regularization, and adaptive resolution. The pipeline has been tested over a wide range of scenes: from classic compact objects taken in a laboratory setting, to outdoor architectural scenes, landscapes, and cultural heritage sites. The accuracy of its reconstructions has also been measured on the dense multiview benchmark proposed by Strecha et al., showing the results to compare more than favorably with the current state-of-the-art methods.

In this paper, we present a novel method, the first to date to our knowledge, which is capable of directly and automatically producing a concise and idealized 3D representation from unstructured point data of complex cluttered real-world scenes, with a high level of noise and a significant proportion of outliers, such as those obtained from passive stereo. Our algorithm can digest millions of input points into an optimized lightweight watertight polygonal mesh free of self-intersection, that preserves the structural components of the scene at a user-defined scale, and completes missing scene parts in a plausible manner. To achieve this, our algorithm incorporates priors on urban and architectural scenes, notably the prevalence of vertical structures and orthogonal intersections. A major contribution of our work is an adaptive decomposition of 3D space induced by planar primitives, namely a polyhedral cell complex. We experimentally validate our approach on several challenging noisy point clouds of urban and architectural scenes.

Existing algorithms for surface reconstruction from point sets are defeated by moderate amounts of noise and outliers, which makes them unapplicable to point clouds originating from multi-view image data. In this paper, we present a novel method which incorporates the input images in the surface reconstruction process for a better accuracy and robustness. Our approach is based on the medial axis transform of the scene, which our algorithm estimates through a global photo-consistency optimization by simulated annealing. A faithful polyhedral representation of the scene is then obtained by inversion of the medial axis transform.

Abstract We describe a robust but simple algorithm to reconstruct a surface from a set of merged range scans. Our key contribution is the formulation of the surface reconstruction problem as an energy minimisation problem that explicitly models the scanning process. The adaptivity of the Delaunay triangulation is exploited by restricting the energy to inside/outside labelings of Delaunay tetrahedra. Our energy measures both the output surface quality and how well the surface agrees with soft visibility constraints. Such energy is shown to perfectly fit into the minimum s − t cuts optimisation framework, allowing fast computation of a globally optimal tetrahedra labeling, while avoiding the “shrinking bias” that usually plagues graph cuts methods. The behaviour of our method confronted to noise, undersampling and outliers is evaluated on several data sets and compared with other methods through different experiments: its strong robustness would make our method practical not only for reconstruction from range data but also from typically more difficult dense point clouds, resulting for instance from stereo image matching. Our effective modeling of the surface acquisition inverse problem, along with the unique combination of Delaunay triangulation and minimum s − t cuts, makes the computational requirements of the algorithm scale well with respect to the size of the input point cloud.

We introduce an original approach for the cerebral white matter connectivity mapping from diffusion tensor imaging (DTI). Our method relies on a global modeling of the acquired magnetic resonance imaging volume as a Riemannian manifold whose metric directly derives from the diffusion tensor. These tensors will be used to measure physical three-dimensional distances between different locations of a brain diffusion tensor image. The key concept is the notion of geodesic distance that will allow us to find optimal paths in the white matter. We claim that such optimal paths are reasonable approximations of neural fiber bundles. The geodesic distance function can be seen as the solution of two theoretically equivalent but, in practice, significantly different problems in the partial differential equation framework: an initial value problem which is intrinsically dynamic, and a boundary value problem which is, on the contrary, intrinsically stationary. The two approaches have very different properties which make them more or less adequate for our problem and more or less computationally efficient. The dynamic formulation is quite easy to implement but has several practical drawbacks. On the contrary, the stationary formulation is much more tedious to implement; we will show, however, that it has many virtues which make it more suitable for our connectivity mapping problem. Finally, we will present different possible measures of connectivity, reflecting the degree of connectivity between different regions of the brain. We will illustrate these notions on synthetic and real DTI datasets.

International audience

In this paper, we propose a novel method for creating a high-quality texture atlas from a 3D model and a set of calibrated images. Our method focuses on avoiding visual artifacts such as color discontinuities, ghosting or blurring, which typically arise from photometric and geometric inaccuracies. We first compute a partition of mesh faces which realizes a good trade-off between visual detail and color continuity at patch boundaries: we efficiently obtain a close-to-optimal seam placement using graph cuts optimization. We then apply a pixel-wise color correction in the vicinity of patch boundaries with a principled 3D extension of multi-band image blending: we achieve faultless color continuity while avoiding ghosting artifacts. We demonstrate the effectiveness of our method on two real-world large-scale scenes.

In this paper, we propose a robust and efficient Lagrangian approach, which we call Delaunay deformable models, for modeling moving surfaces undergoing large deformations and topology changes. Our work uses the concept of restricted Delaunay triangulation, borrowed from computational geometry. In our approach, the interface is represented by a triangular mesh embedded in the Delaunay tetrahedralization of interface points. The mesh is iteratively updated by computing the restricted Delaunay triangulation of the deformed objects. Our method has many advantages over popular Eulerian techniques such as the level set method and over hybrid Eulerian-Lagrangian techniques such as the particle level set method: localization accuracy, adaptive resolution, ability to track properties associated to the interface, seamless handling of triple junctions. Our work brings a rigorous and efficient alternative to existing topology-adaptive mesh techniques such as T-snakes.

We propose a novel method for computing a four-dimensional (4D) representation of the spatio-temporal visual hull of a dynamic scene, based on an extension of a recent provably correct Delaunay meshing algorithm. By considering time as an additional dimension, our approach exploits seamlessly the time coherence between different frames to produce a compact and high-quality 4D mesh representation of the visual hull. The 3D visual hull at a given time instant is easily obtained by intersecting this 4D mesh with a temporal plane, thus enabling interpolation of objects' shape between consecutive frames. In addition, our approach offers easy and extensive control over the size and quality of the output mesh as well as over its associated re- projection error. Our numerical experiments demonstrate the effectiveness and flexibility of our approach for generating compact, high-quality, time-coherent visual hull representations from real silhouette image data.

We present a novel method to reconstruct the 3D shape of a scene from several calibrated images. Our motivation is that most existing multi-view stereovision approaches require some knowledge of the scene extent and often even of its approximate geometry (e.g. visual hull). This makes these approaches mainly suited to compact objects admitting a tight enclosing box, imaged on a simple or a known background. In contrast, our approach focuses on large-scale cluttered scenes under uncontrolled imaging conditions. It first generates a quasi-dense 3D point cloud of the scene by matching keypoints across images in a lenient manner, thus possibly retaining many false matches. Then it builds an adaptive tetrahedral decomposition of space by computing the 3D Delaunay triangulation of the 3D point set. Finally, it reconstructs the scene by labeling Delaunay tetrahedra as empty or occupied, thus generating a triangular mesh of the scene. A globally optimal label assignment, as regards photo-consistency of the output mesh and compatibility with the visibility of keypoints in input images, is efficiently found as a minimum cut solution in a graph.

Abstract In this paper, we propose a robust and efficient Lagrangian approach for modeling dynamic interfaces between different materials undergoing large deformations and topology changes, in two dimensions. Our work brings an interesting alternative to popular techniques such as the level set method and the particle level set method, for two‐dimensional and axisymmetric simulations. The principle of our approach is to maintain a two‐dimensional triangulation which embeds the one‐dimensional polygonal description of the interfaces. Topology changes can then be detected as inversions of the faces of this triangulation. Each triangular face is labeled with the type of material it contains. The connectivity of the triangulation and the labels of the faces are updated consistently during deformation, within a neat framework developed in computational geometry: kinetic data structures. Thanks to the exact computation paradigm, the reliability of our algorithm, even in difficult situations such as shocks and topology changes, can be certified. We demonstrate the applicability and the efficiency of our approach with a series of numerical experiments in two dimensions. Finally, we discuss the feasibility of an extension to three dimensions.

We propose a novel, fast and robust technique for the computation of anatomical connectivity in the brain. Our approach exploits the information provided by Diffusion Tensor Magnetic Resonance Imaging (or DTI) and models the white matter by using Riemannian geometry and control theory. We show that it is possible, from a region of interest, to compute the geodesic distance to any other point and the associated optimal vector field. The latter can be used to trace shortest paths coinciding with neural fiber bundles. We also demonstrate that no explicit computation of those 3D curves is necessary to assess the degree of connectivity of the region of interest with the rest of the brain. We finally introduce a general local connectivity measure whose statistics along the optimal paths may be used to evaluate the degree of connectivity of any pair of voxels. All those quantities can be computed simultaneously in a Fast Marching framework, directly yielding the connectivity maps. Apart from being extremely fast, this method has other advantages such as the strict respect of the convoluted geometry of white matter, the fact that it is parameter-free, and its robustness to noise. We illustrate our technique by showing results on real and synthetic datasets. OurGCM(Geodesic Connectivity Mapping) algorithm is implemented in C++ and will be soon available on the web.

In this paper, we present a new variational method for multi-view stereovision and non-rigid three-dimensional motion estimation from multiple video sequences. Our method minimizes the prediction error of the shape and motion estimates. Both problems then translate into a generic image registration task. The latter is entrusted to a similarity measure chosen depending on imaging conditions and scene properties. In particular, our method can be made robust to appearance changes due to non-Lambertian materials and illumination changes. It results in a simpler, more flexible, and more efficient implementation than existing deformable surface approaches. The computation time on large datasets does not exceed thirty minutes. Moreover, our method is compliant with a hardware implementation with graphics processor units. Our stereovision algorithm yields very good results on a variety of datasets including specularities and translucency. We have successfully tested our scene flow algorithm on a very challenging multi-view video sequence of a non-rigid scene.

This paper tackles an important aspect of the variational problems involving active contours, which has been largely overlooked so far: the optimization by gradient flows. Classically, the definition of a gradient depends directly on the choice of an inner product structure. This consideration is largely absent from the active contours literature. Most authors, overtly or covertly, assume that the space of admissible deformations is ruled by the canonical L2inner product. The classical gradient flows reported in the literature are relative to this particular choice. In this paper, we investigate the relevance of using other inner products, yielding other gradient descents, and some other minimizing flows not deriving from any inner product. In particular we, show how to induce different degrees of spatial coherence into the minimizing flow, in order to decrease the probability of getting trapped into irrelevant local minima. We show with some numerical experiments that the sensitivity of the active contours method to initial conditions, which seriously limits its applicability and its efficiency, is alleviated by our application-specific spatially coherent minimizing flows

In this paper, we overcome a major drawback of the level set framework: the lack of point correspondences. We maintain explicit backward correspondences from the evolving interface to the initial one by advecting the initial point coordinates with the same speed as the level set function. Our method leads to a system of coupled Eulerian partial differential equations. We show in a variety of numerical experiments that it can handle both normal and tangential velocities, large deformations, shocks, rarefactions and topological changes. Applications are many in computer vision and elsewhere since our method can upgrade virtually any level set evolution. We complement our work with the design of non zero tangential velocities that preserve the relative area of interface patches; this feature may be crucial in such applications as computational geometry, grid generation or unfolding of the organs' surfaces, e.g. brain, in medical imaging.

We present a common variational framework for dense depth recovery and dense three-dimensional motion field estimation from multiple video sequences, which is robust to camera spectral sensitivity differences and illumination changes. For this purpose, we first show that both problems reduce to a generic image matching problem after backprojecting the input images onto suitable surfaces. We then solve this matching problem in the case of statistical similarity criteria that can handle frequently occurring nonaffine image intensities dependencies. Our method leads to an efficient and elegant implementation based on fast recursive filters. We obtain good results on real images.