Under the perspective projection assumption, non-Lambertian photometric stereo is a highly non-linear problem. In this study, we present an optimized framework for reconstructing the surface normal and depth with non-Lambertian reflection models under perspective projection. By decomposing the images into diffuse and specular components, we compute the surface normal and reflectance simultaneously. We also propose a variational formulation that is robust and useful for surface reconstruction. The experiments show that our method accurately reconstructs both the surface shape and reflectance of colorful objects with non-Lambertian surfaces.

We present a method to capture both 3D shape and spatially varying reflectance with a multi-view photometric stereo (MVPS) technique that works for general isotropic materials. Our algorithm is suitable for perspective cameras and nearby point light sources. Our data capture setup is simple, which consists of only a digital camera, some LED lights, and an optional automatic turntable. From a single viewpoint, we use a set of photometric stereo images to identify surface points with the same distance to the camera. We collect this information from multiple viewpoints and combine it with structure-from-motion to obtain a precise reconstruction of the complete 3D shape. The spatially varying isotropic bidirectional reflectance distribution function (BRDF) is captured by simultaneously inferring a set of basis BRDFs and their mixing weights at each surface point. In experiments, we demonstrate our algorithm with two different setups: a studio setup for highest precision and a desktop setup for best usability. According to our experiments, under the studio setting, the captured shapes are accurate to 0.5 millimeters and the captured reflectance has a relative root-mean-square error (RMSE) of 9%. We also quantitatively evaluate state-of-the-art MVPS on a newly collected benchmark dataset, which is publicly available for inspiring future research.