Digital twins of the built environment promise major efficiency gains in managing and maintaining infrastructure. A main component of Digital Twin systems is an information-rich, up-to-date, virtual representation of the physical scene. Deriving such an as-is geometric–semantic model (GSM) of the target environment that is both timely and realistic poses a substantial challenge. In particular, the required data aggregation and processing remain substantial obstacles for robust and highly automated generation. The Scan-to-Twin workflow aims to automatically produce an as-is GSM from reality-capture data, but deep learning–based automation still depends on costly manual annotations. We present SynthRoads, a framework that synthetically generates multi-modal data for every step of Scan-to-Twin, enabling deep learning for point-cloud semantic segmentation and geometric extraction in road environments. SynthRoads combines modular model generation with synthetic laser scanning to efficiently create annotated point clouds, geometry parameters, and GSMs. While broadly applicable, we demonstrate significant performance gains for road semantic segmentation and road centerline regression in Scan-to-Twin use cases. Our framework is highly adaptable, leveraging the realism of a game engine and a procedural modeling approach to efficiently generate diverse and realistic road scenes.

Many infrastructure assets in transportation such as roads and bridges represent challenges for inspection and maintenance due to advanced age, structural deficiencies and modifications. Concepts such as Building Information Modelling (BIM) aim to alleviate the problem of health monitoring and asset management by providing digital building models constructed from survey data to all stakeholders. Ageing and oftentimes poorly-documented infrastructure objects such as bridges in particular benefit from a continuous integration of changes to form a digital twin which reflects the asset’s as-is state. However, the process of reconstructing geometric–semantic models from survey data is a manual and labour-intensive process and makes continuously updating the models a difficult task. To automate this process, a cross-domain approach using an artificial neural network is presented which performs semantic segmentation in the image domain and transfers the results over to the point cloud. For the following fine segmentation, geometric knowledge in the 3D domain is used for post-processing and filtering via geometric reasoning. Using this method, a 3D semantic segmentation is achieved which does not require any 3D point cloud training data and only a low amount of image training data.

Building Information Modeling is growing more relevant as digital models are not only used during the construction phase but also throughout the building’s life cycle. The digital representation of geometric, physical and functional properties enables new methods for planning, execution and operation. Digital models of existing buildings are commonly derived from surveying data such as laser scanning which needs to be processed either manually or automatically throughout various steps. Aligning point clouds along the coordinate system’s main axes (also commonly known as pose normalization) is a task benefitting any point cloud processing workflow, be it manual or automated. With the goal of automating this task, we compare various existing methods and present our own approach based on point density histograms. We conclude this paper by comparing and discussing all methods in terms of speed and robustness.