Abstract In this paper, we discuss the requirements and technical challenges within the EHEM project, Enhancement of Heritage Experiences: The Middle Ages, an ongoing research program for the acquisition, analysis, documentation, interpretation, digital restoration, and communication of medieval artistic heritage. The project involves multidisciplinary teams comprising art historians and visual computing experts. Despite the vast literature on digital 3D models in support of Cultural Heritage, the field is so rich and diverse that specific projects often imply distinct, unique requirements which often challenge the computational technologies and suggest new research opportunities. As good representatives of such diversity, we describe the three monuments that serve as test cases for the project, all of them with a rich history of architecture and paintings. We discuss the art historians’ view of how digital models can support their research, the expertise and technological solutions adopted so far, as well as the technical challenges in multiple areas spanning geometry and appearance acquisition, color analysis and digital restitution, as well as the representation of the profound transformations due to the alterations suffered over the centuries.

Digital color restitution aims to digitally restore the original colors of a painting. Existing image editing applications can be used for this purpose, but they require a select-and-edit workflow and thus they do not scale well to large collections of paintings or different regions of the same painting. To address this issue, we propose an automated workflow that requires only a few representative source colors and associated target colors as input from art historians. The system then creates a control grid to model a deformation of the CIELAB color space. Such deformation can be applied to arbitrary images of the same painting. The proposed approach is suitable for restituting the color of images from a large photographic campaign, as well as for the textures of 3D reconstructions of a monument. We demonstrate the benefits of our method on a collection of mural paintings from a medieval monument.

High-end Terrestrial Lidar Scanners are often equipped with RGB cameras that are used to colorize the point samples. Some of these scanners produce panoramic HDR images by encompassing the information of multiple pictures with different exposures. Unfortunately, exported RGB color values are not in an absolute color space, and thus point samples with similar reflectivity values might exhibit strong color differences depending on the scan the sample comes from. These color differences produce severe visual artifacts if, as usual, multiple point clouds colorized independently are combined into a single point cloud. In this paper we propose an automatic algorithm to minimize color differences among a collection of registered scans. The basic idea is to find correspondences between pairs of scans, i.e. surface patches that have been captured by both scans. If the patches meet certain requirements, their colors should match in both scans. We build a graph from such pair-wise correspondences, and solve for the gain compensation factors that better uniformize color across scans. The resulting panoramas can be used to colorize the point clouds consistently. We discuss the characterization of good candidate matches, and how to find such correspondences directly on the panorama images instead of in 3D space. We have tested this approach to uniformize color across scans acquired with a Leica RTC360 scanner, with very good results.

Abstract Modelling flow phenomena and their related weathering effects is often cumbersome due their dependence on the environment, materials and geometric properties of objects in the scene. Example‐based modelling provides many advantages for reproducing real textures, but little effort has been devoted to reproducing and transferring complex phenomena. In order to produce realistic flow effects, it is possible to take advantage of the widespread availability of flow images on the Internet, which can be used to gather key information about the flow. In this paper, we present a technique that allows the transfer of flow phenomena between photographs, adapting the flow to the target image and giving the user flexibility and control through specifically tailored parameters. This is done through two types of control curves: a fitted theoretical curve to control the mass of deposited material, and an extended colour map for properly adapting to the target appearance. In addition, our method filters and warps the input flow in order to account for the geometric details of the target surface. This leads to a fast and intuitive approach to easily transfer phenomena between images, providing a set of simple and intuitive parameters to control the process.

Weathering effects are ubiquitous phenomena in cities. Buildings age and deteriorate over time as they interact with the environment. Pollution accumulating on facades is a particularly visible consequence of this. Even though relevant work has been done to produce impressive images of virtual urban environments including weathering effects, so far, no technique using a global approach has been proposed to deal with weathering effects. Here, we propose a technique based on a fast physically-inspired approach, that focuses on modeling the changes in appearance due to pollution soiling on an urban scale. We consider pollution effects to depend on three main factors: wind, rain and sun exposure, and we take into account three intervening steps: deposition, reaction and washing. Using a low-cost pre-computation, we evaluate the pollution distribution throughout the city. Based on this and the use of screen-space operators, our method results in an efficient approach able to generate realistic images of urban scenes by combining the intervening factors at interactive rates. In addition, the pre-computation demands a reduced amount of memory to store the resulting pollution map and, as it is independent from scene complexity, it can suit large and complex models by adapting the map resolution.

Abstract In urban design, estimating solar exposure on complex city models is crucial but existing solutions typically focus on simplified building models and are too demanding in terms of memory and computational time. In this paper, we propose an interactive technique that estimates solar exposure on detailed urban scenes. Given a directional exposure map computed over a given time period, we estimate the sky visibility factor that serves to evaluate the final exposure at each visible point. This is done using a screen‐space method based on a two‐scale approach, which is geometry independent and has low storage costs. Our method performs at interactive rates and is designer‐oriented. The proposed technique is relevant in architecture and sustainable building design as it provides tools to estimate the energy performance of buildings as well as weathering effects in urban environments.

Abstract A common weathering effect is the appearance of cracks due to material fractures. Previous exemplar‐based aging and weathering methods have either reused images or sought to replicate observed patterns exactly. We introduce a new approach to exemplar‐based modeling that creates weathered patterns on synthetic objects by matching thestatisticsof fracture patterns in a photograph. We present a user study to determine which statistics are correlated to visual similarity and how they are perceived by the user. We then describe a revised physically‐based fracture model capable of producing a wide range of crack patterns at interactive rates. We demonstrate how a Bayesian optimization method can determine the parameters of this model so it can produce a pattern with the same key statistics as an exemplar. Finally, we present results using our approach and various exemplars to produce a variety of fracture effects in synthetic renderings of complex environments. The speed of the fracture simulation allows interactive previews of the fractured results and its application on large scale environments.

The simulation of weathered appearance is essential in the realistic modeling of urban environments. A representative and particularly difficult effect to produce on a large scale is the effect of fluid flow. Changes in appearance due to flow are the result of both the global effect of large-scale shape, and local effects, such as the detailed roughness of a surface. With digital photography and Internet image collections, visual examples of flow effects are readily available. These images, however, mix the appearance of flows with the specific local context. We present a methodology to extract parameters and detail maps from existing imagery in a form that allows new target-specific flow effects to be produced, with natural variations in the effects as they are applied in different locations in a new scene. In this article, we focus on producing a library of parameters and detail maps for generating flow patterns; and this methodology can be used to extend the library with additional image exemplars. To illustrate our methodology, we show a rich collection of patterns applied to urban models.

Abstract This paper presents a method for the accurate rendering of path‐based surface details such as grooves, scratches and similar features. The method is based on a continuous representation of the features in texture space, and the rendering is performed by means of two approaches: one for isolated or non‐intersecting grooves and another for special situations like intersections or ends. The proposed solutions perform correct antialiasing and take into account visibility and inter‐reflections with little computational effort and memory requirements. Compared to anisotropic BRDFs and scratch models, we have no limitations on the distribution of grooves over the surface or their geometry, thus allowing more general patterns. Compared to displacement mapping techniques, we can efficiently simulate features of all sizes without requiring additional geometry or multiple representations.

Abstract Individually visible scratches, also called isolated scratches, are very common in real world surfaces. Although their microgeometry is not visible, they are individually perceptible by the human eye, lying into a representation scale between BRDF and texture. In order to simulate this kind of scratches in synthetic images we need to know their position over the surface (texture scale), so we can determine where to use the specific scratch BRDF instead of the ordinary surface BRDF. Computing the BRDF of a scratch is difficult because it depends on the scratch's invisible microgeometry. In this paper, we propose a new physically based model to derive this microgeometry by simulating the formation process of scratches. We allow specifying intuitively the parameters involved in the process such as the scratching tool, the penetration forces, and the material properties of the object. From these parameters, we derive the microgeometries of the scratches by taking into account the real behaviour of the process. This behaviour has been determined by analysing existing models in the field of materials engineering and some ``scratch tests'' that we performed on metals. Our method has the advantages of easily simulating scratches with a wide range of microgeometries and taking into account the variability of their microgeometry along the scratch path. Another contribution is related to the location of the scratches over the surface. Instead of using an image of the paths as in previous work, we present a new representation based on curves defining the paths. This offers an independence on the image resolution or the distance from the observer and accurately provides the scratch direction in order to compute scratch BRDFs. Categories and Subject Descriptors (according to ACM CCS): I.3.7 [Computer Graphics]: Color, shading, shadowing, and texture.