In this article, we present two online platforms developed for the structuring and valorization of old geographical iconographic collections: a multimodal search engine for their indexing, retrieval and interlinking, and a 3D navigation platform for their visualization in spatial context. In particular, we show how the joint use of these functionalities, guided by geolocation, brings structure and knowledge to the manipulated collections. In the demonstrator, they consist of 54,000 oblique aerial photographs from several French providers (national archives, a museum and a mapping agency).

We present an out-of-core and distributed surface reconstruction algorithm which scales efficiently on arbitrarily large point clouds (with optical centres) and produces a 3D watertight triangle mesh representing the surface of the underlying scene. Surface reconstruction from a point cloud is a difficult problem and existing state of the art approaches are usually based on complex pipelines making use of global algorithms (i.e. Delaunay triangulation, graph-cut optimisation). For one of these approaches, we investigate the distribution of all the steps (in particular Delaunay triangulation and graph-cut optimisation) in order to propose a fully scalable method. We show that the problem can be tiled and distributed across a cloud or a cluster of PCs by paying a careful attention to the interactions between tiles and using Spark computing framework. We confirm the efficiency of this approach with an in-depth quantitative evaluation and the successful reconstruction of a surface from a very large data set which combines more than 350 million aerial and terrestrial LiDAR points.

Motivated by the needs of a scalable out-of-core surface reconstruction algorithm available on the cloud, this paper addresses the computation of distributed Delaunay triangulations of massive point sets. The proposed algorithm takes as input a point cloud and first partitions it across multiple processing elements into tiles of relatively homogeneous point sizes. The distributed computation and communication between processing elements is orchestrated so that each one discovers the Delaunay neighbors of its input points within the theoretical overall Delaunay triangulation of all points and computes locally a partial view of this triangulation. This approach prevents memory limitations by never materializing the global triangulation. This efficiency is due to our proposed uncentralized model to represent, manage and locally construct the triangulation corresponding to each tile. The point set is first partitioned into non-overlapping tiles, then we construct within each tile the Delaunay triangulation of the local points and a minimal set of replicated foreign points in order to capture the simplices spanning multiple tiles. Inspired by the star splaying approach for Delaunay triangulation computation/repair, communication is limited to exchanging points of potential Delaunay neighbors across tiles. Therefore, our method is guaranteed to reconstruct, within each tile, a triangulation that contains the star of its local points, as though it were computed within the Delaunay triangulation of all points. The proposed algorithm is implemented with Spark for the scheduling and C++ for the geometric computations. This allows both an optimal scheduling on multiple machines and efficient low-level computation. The results show the efficiency of our algorithm in terms of speedup and strong scaling on a classical Spark configuration with both synthetic and real use case datasets.

Color palettes are widely used by artists to define colors of artworks and explore color designs. In general, artists select the colors of a palette by following a set of rules, e.g. contrast or relative luminance. Existing interactive palette exploration tools explore palette spaces following limited constraints defined as geometric configurations in color space e.g. harmony rules on the color wheel. Palette search algorithms sample palettes from color relations learned from an input dataset, however they cannot provide interactive user edits and palette refinement. We introduce in this work a new versatile formulation enabling the creation of constraint-based interactive palette exploration systems. Our technical contribution is a graph-based palette representation, from which we define palette exploration as a minimization problem that can be solved efficiently and provide real-time feedback. Based on our formulation, we introduce two interactive palette exploration strategies: constrained palette exploration, and for the first time, constrained palette interpolation. We demonstrate the performances of our approach on various application cases and evaluate how it helps users finding trade-offs between concurrent constraints.

The need for the real-time interactive co-visualization of 3D urban environments on a Web-based virtual Globe arises naturally in GIS but it still remains challenging due to the complexity of city models and their huge data sizes which largely overload the computational power and memory capacity of client devices. Especially on the Web, the visualization of city models makes their rendering not real-time because of the lack of content adaptation and progressive data transmission. This paper presents technical solutions for the co-visualization of massive city models in a Web-based virtual globe, allowing navigation over 3D cities on the globe in real-time. The volume of 3D city data, such as building data, does not allow us to render them directly, nor to keep them in the main memory. We propose to use not only a hierarchical presentation of geo-spatial data to create a chunk-based multiple resolution data structure which reduces complexity of the geometry being rendered; but also a view dependent algorithm so that only small subsets of 3D city models are streamed progressively in real-time and kept in client memory to contribute efficiently to the rendered image. Experimental results show that we can navigate over 3D cities on the Globe in real-time.

This paper presents a novel variational image inpainting method to solve the problem of generating, from 3-D lidar measures, a dense depth map coherent with a given color image, tackling visibility issues. When projecting the lidar point cloud onto the image plane, we generally obtain a sparse depth map, due to undersampling. Moreover, lidar and image sensor positions generally differ during acquisition, such that depth values referring to objects that are hidden from the image view point might appear with a naive projection. The proposed algorithm estimates the complete depth map, while simultaneously detecting and excluding those hidden points. It consists in a primal-dual optimization method, where a coupled total variation regularization term is included to match the depth and image gradients and a visibility indicator handles the selection of visible points. Tests with real data prove the effectiveness of the proposed strategy.

Precise localization in dense urban areas is a challenging task for both mobile mapping and driver assistance systems. This paper proposes a strategy to use road markings as localization landmarks for vision based systems. First step consists in reconstructing a map of road marks. A mobile mapping system equipped with precise georeferencing devices is applied to scan the scene in 3D and to generate an ortho-image of the road surface. A RJMCMC sampler that is coupled with a simulated annealing method is applied to detect occurrences of road marking templates instanced from an extensible database of road mark patterns. The detected objects are reconstructed in 3D using the height information obtained from 3D points. A calibrated camera and a low cost GPS receiver are embedded on a vehicle and used as localization devices. Local bundle adjustment (LBA) is applied to estimate the trajectory of the vehicle. In order to reduce the drift of the trajectory, images are matched with the reconstructed road marks frequently. The matching is initialized by the initial poses that are estimated by LBA and optimized by a MCMC algorithm. The matching provides ground control points that are integrated in the LBA in order to refine the pose parameters. The method is evaluated on a set of images acquired in a real urban area and is compared with a precise ground-truth.

The internet browsers nowadays show incredible possibilities with HTML5. It makes it possible to use all the power of your device such as the GPU and all its sensors, GPS, accelerometer, camera, etc. The ability to put hardware-accelerated 3D content in the browser provides a way for the creation of new web based applications that were previously the exclusive domain of the desktop environment. This paper introduces a novel implementation of a 3D GIS WebGL-based navigation system which allows end-users to navigate in a 3D realistic and immersive urban scene, to interact with different spatial data such as panoramic image, laser, 3D-city model, and vector data with modern functionalities such as using your smartphone as a remote, render for 3D screen and make the scene dynamic.

We present an original multiview stereo reconstruction algorithm which allows the 3D-modeling of urban scenes as a combination of meshes and geometric primitives. The method provides a compact model while preserving details: Irregular elements such as statues and ornaments are described by meshes, whereas regular structures such as columns and walls are described by primitives (planes, spheres, cylinders, cones, and tori). We adopt a two-step strategy consisting first in segmenting the initial meshbased surface using a multilabel Markov Random Field-based model and second in sampling primitive and mesh components simultaneously on the obtained partition by a Jump-Diffusion process. The quality of a reconstruction is measured by a multi-object energy model which takes into account both photo-consistency and semantic considerations (i.e., geometry and shape layout). The segmentation and sampling steps are embedded into an iterative refinement procedure which provides an increasingly accurate hybrid representation. Experimental results on complex urban structures and large scenes are presented and compared to state-of-the-art multiview stereo meshing algorithms.

Nowadays we see a lot of Rich Internet Applications (RIA), but real interactive 3D applications have just started years ago, and it is only in 2011 that a GPU library was integrated directly in the major internet browsers. This opened a new way of creating 3D applications on the web and a new public, as much more users can access them without complicated plugin installations and this is just what 3D web mapping was waiting for.

Internet search has initially been a strong driving force for the rapid emergence of 3D building models of large urban areas. Additionally, many commercial and governmental initiatives have been started to develop urban 3D geographic information systems in a transition from the classical 2D- to the novel 3D-GIS. The modeling of building roofs is thus a relevant research topic. The focus has been on the use of aerial LiDAR point clouds (Light Detection And Ranging). However, recent progress in digital aerial cameras has rendered possible the acquisition of very dense point clouds from high overlap digital aerial imagery, and to use these point clouds jointly with the image information to generate 3D building models.

We propose a multi-view stereo reconstruction algorithm which recovers urban scenes as a combination of meshes and geometric primitives. It provides a compact model while preserving details: irregular elements such as statues and ornaments are described by meshes whereas regular structures such as columns and walls are described by primitives (planes, spheres, cylinders, cones and tori). A Jump-Diffusion process is designed to sample these two types of elements simultaneously. The quality of a reconstruction is measured by a multi-object energy model which takes into account both photo-consistency and semantic considerations (i.e. geometry and shape layout). The sampler is embedded into an iterative refinement procedure which provides an increasingly accurate hybrid representation. Experimental results on complex urban structures and large scenes are presented and compared to multi-view based meshing algorithms.

We propose an original hybrid modeling process of urban scenes that represents 3-D models as a combination of mesh-based surfaces and geometric 3-D-primitives. Meshes describe details such as ornaments and statues, whereas 3-D-primitives code for regular shapes such as walls and columns. Starting from an 3-D-surface obtained by multiview stereo techniques, these primitives are inserted into the surface after being detected. This strategy allows the introduction of semantic knowledge, the simplification of the modeling, and even correction of errors generated by the acquisition process. We design a hierarchical approach exploring different scales of an observed scene. Each level consists first in segmenting the surface using a multilabel energy model optimized by -expansion and then in fitting 3-D-primitives such as planes, cylinders or tori on the obtained partition where relevant. Experiments on real meshes, depth maps and synthetic surfaces show good potential for the proposed approach.

International audience

This paper describes an automatic method for the detection and detailed reconstruction of 3D building models that include roof superstructures such as dormer windows or chimneys from a very high resolution digital elevation model. Buildings are reconstructed as a set of roof planes with a set of parametric shapes that model roof superstructures. The proposed model-based approach minimizes a minimum description length energy.