Absolute testing is a key technique for achieving nanoscale measurements in Fizeau interferometry. Traditional methods use Zernike polynomials to fit the test surface form and indirectly infer the reference surface form, which can lead to error accumulation. In this article, we propose a neural absolute testing method that uses an advanced neural radiance field to directly model the reference surface form, thereby reducing error accumulation. The proposed framework includes an invariant surface branch and a variant surface branch; the sum of these two branches is expected to closely approximate the measured data under varying rotations and shifts. Consequently, a self-supervised training strategy has been developed to optimize the model parameters, eliminating the need for extensive training datasets with ground truth labels. Synthetic experiments demonstrate that the proposed method achieves superior absolute calibration accuracy compared to the traditional Zernike polynomial approach. Furthermore, real-world experiments performed using a Fizeau interferometer confirm that the proposed method produces a more robust reference surface form than the classical Zernike polynomial technique.

Photometric stereo faces challenges from non-Lambertian reflectance in real-world scenarios. Systematically measuring the reliability of photometric stereo methods in handling such complex reflectance necessitates a real-world dataset with quantitatively controlled reflectances. This paper introduces DiLiGenRT, the first real-world dataset for evaluating photometric stereo methods under quantified reflectances by manufacturing 54 hemispheres with varying degrees of two reflectance properties: Roughness and Transluency, Unlike qualitative and semantic labels, such as “diffuse” and “specular,” that have been used in previous datasets, our quantified dataset allows comprehensive and systematic benchmark evaluations. In addition, it facilitates selecting best-fit photometric stereo methods based on the quantitative reflectance properties. Our dataset and benchmark results are available at https://photometricstereo.github.io/diligentrt.html.

Surface texture measurement helps control the quality of large workpieces produced by machine systems. Current optical measurement methods, e.g., fringe projection profilometry and coherent scanning interferometry, are difficult to perform full-field measurements of microtextures on large-sized surfaces. Photometric stereo could potentially address this challenge; however, machined metal surfaces exhibit highly reflective non-Lambertian reflectance that dramatically decreases its effectiveness. To solve this problem, a task-specific near-field photometric stereo approach is proposed to dramatically enhance the accuracy of the surface normal estimation on machined metal surfaces. First, a near-field photometric stereo network is designed for efficient industrial applications, which fully employs the pixelwise information under a small number of lights to achieve surface normal estimation. Then, a task-specific training strategy is proposed to train the proposed network, where a task-specific real dataset is established for each specific combination of material and machine processing to optimize the network parameters initially trained by a synthetic dataset. Experiments on synthetic sinusoidal surfaces and real machined surfaces validated the superiority of the proposed method for metal reflectance compared with the state-of-the-art photometric stereo methods and the sub/micrometer-level sensitivity to surface height variations. Two case studies on tool marks on a large freeform surface and defects on a stamping surface are presented, demonstrating their potential for industrial applications.

The highlight reflectance on machined metal surfaces dramatically influences the performance of optical measurement sensors, such as fringe projector profilometers. Previous research addressed such an issue through fusion of images under different camera exposure times or modulation of the lighting intensity of the projector. Differently, this article provides a new research perspective that resorts to the modeling of complicated anisotropic reflectance on machined metal surfaces, which essentially causes the decrease in the quality of fringe images. In this article, an anisotropic neural reflectance model is proposed for the effective description of machined metal reflectance. Then, a facility with one camera, one projector, and several lighting emittance didoes is designed for the application of the proposed model in the measurement of curved machined metal surfaces. A complete and accurate point cloud is finally obtained through fitting of the real reflectance using the proposed model in a self-supervision way. Synthetic experiments illustrate that the capability of the proposed model to describe the machined metal reflectance is considerably enhanced compared with the traditional parameterized reflectance model and isotropic neural reflectance model. Real experiments prove that the proposed method can achieve measurement accuracy of 23 μm on machined metal surfaces with repeated accuracy of 0.15 μm, where the ground truth is given by the coordinate measuring machine.

To ensure the machining quality of micro/nano scale structural units for meter scale workpieces, relay turning with multiple single-point diamond tools has been broadly required. However, the existing tool setting methods have the problems of long tool setting time and low tool setting accuracy. To address the above issues, a novel normalized variational auto-encoder model with an adaptive activation function (NVAE-AAF) is proposed in this article. The batch normalization and the adaptive activation function are introduced into the variational auto-encoder model to learn robust features of force signals at the tool idle move state. Then, the reconstruction error threshold is constructed according to the kernel density estimation method to realize the nanoscale tool setting. In the ultraprecision tool setting experiments based on piezoelectric ceramic force sensing, the reconstruction error of the force signals at the tool idle move state is less than 0.07%, and the contact detection accuracy reached 92%. Compared to the traditional trial cutting for tool setting method, the proposed method significantly improves tool setting accuracy by 75%–85%, reaching a level of 75 nm.

Photometric stereo aims to recover detailed surface shapes from images captured under varying illuminations. However, existing real-world datasets primarily focus on evaluating photometric stereo for general non-Lambertian reflectances and feature bulgy shapes that have a certain height. As shape detail recovery is the key strength of photometric stereo over other 3D reconstruction techniques, and the near-planar surfaces widely exist in cultural relics and manufacturing workpieces, we present a new real-world dataset DiLiGenT-Π containing 30 near-planar scenes with rich surface details. This dataset enables us to evaluate recent photometric stereo methods specifically for their ability to estimate shape details under diverse materials and to identify open problems such as near-planar surface normal estimation from uncalibrated photometric stereo and surface detail recovery for translucent materials. To inspire future research, this dataset will open soruced at https://photometricstereo.github.io/diligentpi.html.

Evaluating photometric stereo using real-world dataset is important yet difficult. Existing datasets are insufficient due to their limited scale and random distributions in shape and material. This paper presents a new real-world photometric stereo dataset with “ground truth” normal maps, which is 10 times larger than the widely adopted one. More importantly, we propose to control the shape and material variations by fabricating objects from CAD models with carefully selected materials, covering typical aspects of reflectance properties that are distinctive for evaluating photometric stereo methods. By benchmarking recent photometric stereo methods using these 100 sets of images, with a special focus on recent learning based solutions, a 10x 10 shape-material error distribution matrix is visualized to depict a “portrait” for each evaluated method. From such comprehensive analysis, open problems in this field are discussed. To inspire future research, this dataset is available at https://photometricstereo.github.io.

Light field (LF) imaging is an advanced visual perception system, which can record the intensity and direction information of light rays and provide multi-viewpoint images from a single capture. However, there is a trade-off between spatial and angular resolutions due to the restricted sensor size, which limits the wide applications of LF cameras. To address this problem, we propose a cooperative network to super-resolve LF sub-aperture images based on the multi-modality fusion. Specifically, in order to fully explore the LF information, we adopt various modalities and extract corresponding features to emphasise diverse LF characteristics. Then, we design a multi-scale fusion module to effectively integrate global and local LF features and apply frequency-aware attention mechanism to adaptively reinforce fused features. Extensive experiments demonstrate the superiority of our method on both qualitative and quantitative evaluations, with competitive execution efficiency.

One of the key issues in photometric stereo is the extension of its application in real world objects which shows non-Lambertian reflectance. This letter proposes a multi-scale weighted convolutional fusion network with deep learning architecture to realize high-precision perception of non-Lambertian surfaces under arbitrary illumination conditions. A multi-scale convolutional fusion module is designed to strengthen the photometric physics and the utilization of the neighborhood features at the same time so as to overcome shadows and distinguish multiple materials. In order to further deal with the problem of arbitrary illumination conditions, a multi-resolution polar coordinate division method is proposed to integrate the input image information and fully utilizes the multi-scale convolution. Both syntheses and real-world experiments verifies the performance of the proposed method in recovery accuracy and computational efficiency.

Current non-Lambertian photometric stereo methods generally require a large number of images to ensure accurate surface normal estimation. To achieve accurate surface normal recovery under a sparse set of lights, this paper proposes a non-Lambertian photometric stereo network based on a derived inverse reflectance model with collocated light. The model is deduced using monotonicity of isotropic reflectance and the univariate property of collocated light to decouple the surface normal from the reflectance function. Thus, the surface normal can be estimated by three steps, i.e., model fitting, shadow rejection, and normal estimation. We leverage a supervised deep learning technique to enhance the shadow rejection ability and the flexibility of the inverse reflectance model. Shadows are handled through max-pooling. Information from a neighborhood image patch is utilized to improve the flexibility to various reflectances. Experiments using both synthetic and real images demonstrate that the proposed method achieves state-of-the-art accuracy in surface normal estimation.