The digital construction of cultural heritage promotes communication and sharing of digital cultural resources across time and space. Digital storage serves as the foundation for the digital construction of cultural artifacts. In the digital storage of Dunhuang murals, image stitching plays a critical role in restoring the complete image of the cave murals. Traditional image stitching methods are constrained by the detection accuracy of feature points and are not fit for stitching low-texture murals. Despite deep learning-based image stitching methods, parallax misalignment and ghosting are still prevalent issues. For this reason, we perform the first Dunhuang mural stitching based on deep learning in this paper. This is in response to the need for digitizing and storing Dunhuang murals. Two mural stitching datasets are constructed, and we design a progressive regression image alignment network and a feature differential reconstruction soft-coded seam stitching network. We also introduce a soft-coded seam quality evaluation method. The algorithm presented in this paper achieves state-of-the-art alignment and stitching performance in the mural stitching task through unsupervised learning with a smaller number of model parameters, which provides technical support for the digitization and preservation of Dunhuang murals.

Abstract In this work, we introduce our project on Dunhuang cultural heritage protection using artificial intelligence. The Dunhuang Mogao Grottoes in China, also known as the Grottoes of the Thousand Buddhas, is a religious and cultural heritage located on the Silk Road. The grottoes were built from the 4th century to the 14th century. After thousands of years, the in grottoes decaying is serious. In addition, numerous historical records were destroyed throughout the years, making it difficult for archaeologists to reconstruct history. We aim to use modern computer vision and machine learning technologies to solve such challenges. First, we propose to use deep networks to automatically perform the restoration. Through out experiments, we find the automated restoration can provide comparable quality as those manually restored from an archaeologist. This can significantly speed up the restoration given the enormous size of the historical paintings. Second, we propose to use detection and retrieval for further analyzing the tremendously large amount of objects because it is unreasonable to manually label and analyze them. Several state-of-the-art methods are rigorously tested and quantitatively compared in different criteria and categorically. In this work, we created a new dataset, namely, AI for Dunhuang, to facilitate the research. Version v1.0 of the dataset comprises of data and label for the restoration, style transfer, detection, and retrieval. Specifically, the dataset has 10,000 images for restoration, 3455 for style transfer, and 6147 for property retrieval. Lastly, we propose to use style transfer to link and analyze the styles over time, given that the grottoes were build over 1000 years by numerous artists. This enables the possibly to analyze and study the art styles over 1000 years and further enable future researches on cross-era style analysis. We benchmark representative methods and conduct a comparative study on the results for our solution. The dataset will be publicly available along with this paper.