We present PuzzleSorter, a certainty-aware visual analytics system for cultural relic fragment restoration. Restoring cultural objects from broken fragments is a fundamental task in geometry and archaeology. Prior research proposes automatic models to classify fragments by types and assemble matched pairs successively. However, eroded fragments lead to erroneous results, posing two challenges for restorers to correct: (1) numerous fragments conceal errors within an overwhelming number of object appearances, and (2) the unknown difficulty of restoration hinders correction strategy development. To address these challenges, PuzzleSorter provides multi-criteria analysis that helps users identify certainties of current solutions and alternatives at the type, object, and fragment levels. Moreover, our system visualizes these certainties through a relation graph, which implies alternative assembly solutions with geometric context and indicates correction difficulties through neighbor proximity, number of neighbors, and path length. We demonstrate the feasibility and utility of our system through two case studies and expert interviews.

Under the perspective projection assumption, non-Lambertian photometric stereo is a highly non-linear problem. In this study, we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection. By decomposing the images into diffuse and specular components, we compute the surface normal and reflectance simultaneously. We also propose a variational formulation that is robust and useful for surface reconstruction. The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.

Generally, the distributed bundle adjustment (DBA) method uses multiple worker nodes to solve the bundle adjustment problems and overcomes the computation and memory storage limitations of a single computer. However, the performance considerably degrades owing to the overhead introduced by the additional block partitioning step and synchronous waiting. Therefore, we propose a low-overhead consensus framework. A partial barrier based asynchronous method is proposed to early achieve consensus with respect to the faster worker nodes to avoid waiting for the slower ones. A scene summarization procedure is designed and integrated into the block partitioning step to ensure that clustering can be performed on the small summarized scene. Experiments conducted on public datasets show that our method can improve the worker node utilization rate and reduce the block partitioning time. Also, sample applications are demonstrated using our large-scale culture heritage datasets.

We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo (MVPS) technique that works for general isotropic materials. Our algorithm is suitable for perspective cameras and nearby point light sources. Our data capture setup is simple, which consists of only a digital camera, some LED lights, and an optional automatic turntable. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distribution function (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. In experiments, we demonstrate our algorithm with two different setups: a studio setup for highest precision and a desktop setup for best usability. According to our experiments, under the studio setting, the captured shapes are accurate to 0.5 millimeters and the captured reflectance has a relative root-mean-square error (RMSE) of 9%. We also quantitatively evaluate state-of-the-art MVPS on a newly collected benchmark dataset, which is publicly available for inspiring future research.

To speed up the reconstruction of 3D dynamic scenes in an ordinary hardware platform, we propose an efficient framework to reconstruct 3D dynamic objects using a multiscale-contour-based interpolation from multi-view videos. Our framework takes full advantage of spatio-temporal-contour consistency. It exploits the property to interpolate single contours, two neighboring contours which belong to the same model, and two contours which belong to the same view at different times, corresponding to point-, contour-, and model-level interpolations, respectively. The framework formulates the interpolation of two models as point cloud transport rather than non-rigid surface deformation. Our framework speeds up the reconstruction of a dynamic scene while improving the accuracy of point-pairing which is used to perform the interpolation. We obtain a higher frame rate, spatio-temporal-coherence, and a quasi-dense point cloud sequence with color information. Experiments with real data were conducted to test the efficiency of the framework.

The surface-based approaches modeling from silhouettes are a popular and useful topic. Intersecting operator of line segments is an essential and crucial operation in these approaches. It consists of two phases: intersecting operator in 2D and overlap testing of the set of line segments in real domain. The former has made considerable progresses while the latter remains insignificant status when there are complex objects in the scene. In order to accelerate the speed of reconstructing visual hulls, this paper addresses the problem and proposes a novel approach based on contribution weights which reflect viewing ray's contributions to the visual hulls. Experimental results demonstrate that the novel approach can attain the promise.