3D city-scale models are useful in a number of applications, including education, city planning, navigation systems, artificial intelligence training, and simulations. However, final models need to be immersive and interactive, which requires a mixed reality (XR) environment design that combines e.g., a Cave Automatic Virtual Environment (CAVE) VR system with the Microsoft Hololens2 in a networked and multimodal setting. In this paper, we propose a pipeline to convert a city-scale point cloud into a finalized city-scale textured mesh in which, a number of XR devices can share the same environment and co-exist in a shared space for model interactions. Specifically, we use input point clouds obtained from wide area motion imagery systems or off-the-shelf drones pertaining to Albuquerque, New Mexico, but the pipeline is generalized so that other input can be used. Using four different traditional algorithms and an additional deep learning method, we create meshes for the model interactions. For each mesh produced, we map high-resolution textures onto them, producing a more accurate city, which is then passed into the shared/networked Unity environment. Ten participants provided their assessment of mesh quality and interactivity of the networked environment during exploration of different city reconstructions with the CAVE and laptop device modalities. Results on the perceptual immersive quality of the Point2Mesh deep learning meshes highlights the need for improvements to handle large city scale point clouds.

Modeling and simulation of large urban regions is beneficial for a range of applications including intelligent transportation, smart cities, infrastructure planning, and training artificial intelligence for autonomous navigation systems including ground vehicles and aerial drones. Immersive environments including virtual reality (VR), augmented reality (AR), mixed reality (MR or XR) can be used to explore city scale regions for planning, design, training and operations. Virtual environments are in the midst of rapid change as innovations in display technologies, graphics processors and game engine software present new opportunities for incorporating modeling and simulation into engineering workflows. Game engine software like Unity with photorealistic rendering and realistic physics have plug-in support for a variety of virtual environments and typically model the scene as meshes. In this paper, we develop an end-to-end workflow for creating urban scale real world accurate synthetic environments that can be visualized in virtual environments including the Microsoft HoloLens head mounted display or the CAVE VR for multi-user interaction. Four meshing algorithms are evaluated for representation accuracy and city-scale meshes imported into Unity for assessing the quality of the immersive experience.