Automated visual quality inspection is a core topic of robotics and computer vision. In industrial applications, the CAD model of the object to be inspected is often known and can be used to generate appropriate sensor poses (viewpoints) from which to inspect the object’s surface and assure the quality of its geometry. Current approaches in this field generate optimal viewpoints by evaluating the geometric coverage but the photometric appearance of the object is usually not considered. This lack of photometric information results in a loss of crucial cues to establish actual visibility, especially when the object to inspect presents specular highlights (e.g., polished metal parts) and a complex geometry. In this paper, we propose integrating photometric information into the viewpoint evaluation to consider the object’s appearance. To achieve this, we embed a bidirectional reflectance distribution function (BRDF) within the evaluation of the viewpoint candidates. We benchmark different BRDFs with increasingly realistic rendering to prove the concept of our approach. Specifically, we consider the Blinn-Phong and Cook-Torrance reflectance models. Our simulation results demonstrate the suitability and importance of using a photometric approach that considers material properties and selects optimal viewpoints for specific materials.

Real-time applications for autonomous operations depend largely on fast and robust vision-based localization systems. Since image processing tasks require processing large amounts of data, the computational resources often limit the performance of other processes. To overcome this limitation, traditional marker-based localization systems are widely used since they are easy to integrate and achieve reliable accuracy. However, classical marker-based localization systems significantly depend on standard cameras with low frame rates, which often lack accuracy due to motion blur. In contrast, event-based cameras provide high temporal resolution and a high dynamic range, which can be utilized for fast localization tasks, even under challenging visual conditions. This paper proposes a simple but effective event-based pose estimation system using active LED markers (ALM) for fast and accurate pose estimation. The proposed algorithm is able to operate in real time with a latency below 0.5 ms while maintaining output rates of 3 kHz. Experimental results in static and dynamic scenarios are presented to demonstrate the performance of the proposed approach in terms of computational speed and absolute accuracy, using the OptiTrack system as the basis for measurement. Moreover, we demonstrate the feasibility of the proposed approach by deploying the hardware, i.e., the event-based camera and ALM, and the software in a real quadcopter application. Our project page is available at: almpose.github.io

Retrieving accurate 3D reconstructions of objects from the way they reflect light is a very challenging task in computer vision. Despite more than four decades since the definition of the Photometric Stereo problem, most of the literature has had limited success when global illumination effects such as cast shadows, self-reflections and ambient light come into play, especially for specular surfaces. Recent approaches have leveraged the capabilities of deep learning in conjunction with computer graphics in order to cope with the need of a vast number of training data to invert the image irradiance equation and retrieve the geometry of the object. However, rendering global illumination effects is a slow process which can limit the amount of training data that can be generated.In this work we propose a novel pixel-wise training procedure for normal prediction by replacing the training data (observation maps) of globally rendered images with independent per-pixel generated data. We show that global physical effects can be approximated on the observation map domain and this simplifies and speeds up the data creation procedure. Our network, PX-NET, achieves state-of-the-art performance compared to other pixelwise methods on synthetic datasets, as well as the DiLiGenT real dataset on both dense and sparse light settings.

Highly accurate 3D volumetric reconstruction is still an open research topic where the main difficulty is usually related to merging some rough estimations with high frequency details. One of the most promising methods is the fusion between multi-view stereo and photometric stereo images. Beside the intrinsic difficulties that multi-view stereo and photometric stereo in order to work reliably, supplementary problems arise when considered together. In this work, we present a volumetric approach to the multi-view photometric stereo problem. The key point of our method is the signed distance field parameterisation and its relation to the surface normal. This is exploited in order to obtain a linear partial differential equation which is solved in a variational framework, that combines multiple images from multiple points of view in a single system. In addition, the volumetric approach is naturally implemented on an octree, which allows for fast ray-tracing that reliably alleviates occlusions and cast shadows. Our approach is evaluated on synthetic and real data-sets and achieves state-of-the-art results.

Despite the longtime research aimed at retrieving geometrical information of an object from polarimetric imaging, physical limitations in the polarisation phenomena constrain current approaches to provide ambiguous depth estimation. As an additional constraint, polarimetric imaging formulation differs when light is reflected off the object specularly or diffusively. This introduces another source of ambiguity that current formulations cannot overcome. With the aim of deriving a formulation capable of dealing with as many heterogeneous effects as possible, we propose a differential formulation of the Shape from Polarisation problem that depends only on polarimetric images. This allows the direct geometrical characterisation of the level-set of the object keeping consistent mathematical formulation for diffuse and specular reflection. We show via synthetic and real-world experiments that diffuse and specular reflection can be easily distinguished in order to extract meaningful geometrical features from just polarimetric imaging. The inherent ambiguity of the Shape from Polarization problem becomes evident through the impossibility of reconstructing the whole surface with this differential approach. To overcome this limitation, we consider shading information elegantly embedding this new formulation into a two-light calibrated photometric stereo approach..

3D Reconstruction from shading information through Photometric Stereo is considered a very challenging problem in Computer Vision. Although this technique can potentially provide highly detailed shape recovery, its accuracy is critically dependent on a numerous set of factors among them the reliability of the light sources in emitting a constant amount of light. In this work, we propose a novel variational approach to solve the so called semi-calibrated near field Photometric Stereo problem, where the positions but not the brightness of the light sources are known. Additionally, we take into account realistic modeling features such as perspective viewing geometry and heterogeneous scene composition, containing both diffuse and specular objects. Furthermore, we also relax the point light source assumption that usually constraints the near field formulation by explicitly calculating the light attenuation maps. Synthetic experiments are performed for quantitative evaluation for a wide range of cases whilst real experiments provide comparisons, qualitatively outperforming the state of the art.

3D shape recovery using photometric stereo (PS) gained increasing attention in the computer vision community in the last three decades due to its ability to recover the thinnest geometric structures. Yet, the reliability of PS for color images is difficult to guarantee, because existing methods are usually formulated as the sequential estimation of the colored albedos, the normals and the depth. Hence, the overall reliability depends on that of each subtask. In this work we propose a new formulation of color photometric stereo, based on image ratios, that makes the technique independent from the albedos. This allows the unbiased 3D-reconstruction of colored surfaces in a single step, by solving a system of linear PDEs using a variational approach.

After thirty years of researching, the photometric stereo technique for 3D shape recovery still does not provide reliable results if it is not constrained into very well-controlled scenarios. In fact, dealing with realistic materials and lightings yields a non-linear bidirectional reflectance distribution function which is primarily difficult to parametrize and then arduous to solve. With the aim to let the photometric stereo approach face more realistic assumptions, in this work we firstly introduce a unified irradiance equation describing both diffuse and specular reflection components in a general lighting setting. After that, we define a new equation we call unifying due to its basic features modeling the photometric stereo problem for heterogeneous materials. It is provided by making the ratio of irradiance equations holding both diffuse and specular reflections as well as non-linear light propagation features simultaneously. Performing a wide range of experiments, we show that this new approach overcomes state-of-the-art since it leads to a system of unifying equations which can be solved in a very robust manner using an efficient variational approach.

Shape from shading with multiple light sources is an active research area, and a diverse range of approaches have been proposed in recent decades. However, devising a robust reconstruction technique still remains a challenging goal, as the image acquisition process is highly nonlinear. Recent Photometric Stereo variants rely on simplifying assumptions in order to make the problem solvable: light propagation is still commonly assumed to be uniform, and the Bidirectional Reflectance Distribution Function is assumed to be diffuse, with limited interest for specular materials. In this work, we introduce a well-posed formulation based on partial differential equations (PDEs) for a unified reflectance function that can model both diffuse and specular reflections. We base our derivation on ratio of images, which makes the model independent from photometric invariants and yields a well-posed differential problem based on a system of quasi-linear PDEs with discontinuous coefficients. In addition, we directly solve a differential problem for the unknown depth, thus avoiding the intermediate step of approximating the normal field. A variational approach is presented ensuring robustness to noise and outliers (such as black shadows), and this is confirmed with a wide range of experiments on both synthetic and real data, where we compare favorably to the state of the art.

Shape recovery based on shading variations of a lighted object was recently revisited with improvements that allow for the photometric stereo approach to serve as a competitive alternative for other shape reconstruction methods. However, most efforts of using photometric stereo tend to ignore some factors that are relevant in practical applications. The approach we consider tackles the photometric stereo reconstruction in the case of near-field imaging which means that both camera and light sources are close to the imaged object. The known challenges that characterize the problem involve perspective viewing geometry, attenuation of light and possibly missing regions. Here, we pay special attention to the question of how to faithfully model these aspects and by the same token design an efficient and robust numerical solver. We present a well-posed mathematical representation that integrates the above assumptions into a single coherent model. The surface reconstruction in our near-field scenario can then be executed efficiently in linear time. The merging strategy of the irradiance equations provided for each light source allows us to consider a characteristic expansion model which enables the direct computation of the surface. We evaluate several types of light attenuation models with nonuniform albedo and noise on synthetic data using four virtual sources. We also demonstrate the proposed method on surface reconstruction of real data using three images, each one taken with a different light source.

Shape from shading and photometric stereo are two fundamental problems in computer vision aimed at reconstructing surface depth given either a single image taken under a known light source or multiple images taken under different illuminations from the same viewing angle. Whereas the former uses partial differential equation techniques to solve the image irradiance equation, the latter can be expressed as a linear system of equations in surface derivatives when three or more images are given. Therefore, it seems that current photometric stereo techniques do not extract all possible depth information from each image by itself. Extending our previous results on this problem, we consider the more realistic perspective projection of surfaces during the photographic process. Under this assumption, there is a unique weak solution (Lipschitz continuous) to the problem at hand, solving the well-known convex/concave ambiguity of the shape from shading problem. The main contribution of this paper is based on a new differential approach for multi-image photometric stereo. Most of the existing works on this topic do not directly address this problem. The common approach is to estimate the gradient field of the surface by minimizing some functional and integrate it afterwards to find the depth and hence the geometry of the object. Our new differential approach allows us to solve the problem directly, while dealing with images having missing parts. The mathematical well-posedness of the new formulation allows a fast numerical algorithm based on a combination of fast marching and fast sweeping methods.

Shape recovery of an object based on shading variations resulting from different light sources has recently been reconsidered. Improvements have been made that allow for the photometric stereo approach to serve as a competitive alternative to other shape reconstruction methods. However, most photometric stereo methods tend to ignore factors that are relevant in practical applications. The setup considered in this paper tackles photometric stereo reconstruction in the case of a specific near-field imaging. This means that both the camera and the light sources are close to the imaged object, where close can be loosely considered as a setup having similar distances between lights, camera, and object. The known challenges that characterize the problem involve perspective viewing geometry, point light sources, and images that may include shadowed regions. Here, we pay special attention to the question of how to faithfully model these aspects and at the same time design an efficient and robust numerical solver. We present a mathematical formulation that integrates the above assumptions into a single coherent model based on quasi-linear PDEs. The well-posedness is proved showing uniqueness of a weak (i.e., Lipschitz continuous) solution. The surface reconstruction in our near-field scenario can then be executed efficiently in linear time. The merging strategy of the irradiance equations provided for each light source allows us to consider a characteristic expansion model which enables the direct computation of the surface. We evaluate several types of light attenuation models with a nonuniform albedo and noise on synthetic data. We also demonstrate the proposed method on surface reconstruction of real data using three images, each one taken with a different light source by a working prototype. We demonstrate the accuracy of the proposed method compared to other methods that ignore the near-field setup and assume distant, parallel beam light sources.

Reconstruction of 3D objects Based on images is useful in many applications. One of the methods Based on multi-image data is the Photometric Stereo technique relying on several photographs of the observed object from the same point of view, each one taken under a different illumination condition. The common approach is to estimate the gradient field of the surface by minimizing a functional, integrating the distance from the camera and thereby obtaining the geometry of the observed object. We propose an alternative method that consists of a novel differential approach for multi-image Photometric Stereo and permits a direct solution of a novel PDE Based model without going through the gradient field while naturally dealing with shadowed regions. The mathematical well-posed ness of the problem in terms of numerical stability yields a fast algorithm that efficiently converges, even for pictures of sizes in the order of several mega pixels affected by noise.

We deal with an inverse problem where we want to determine the surface of an object using the information contained in two or more pictures which correspond to different light conditions. In particular, we will examine the case where the light source direction varies between the pictures, and we will show how this additional information allows us to obtain a uniqueness result solving the well-known convex/concave ambiguity of the shape-from-shading problem. We will prove a uniqueness result for weak (Lipschitz continuous) solutions that improves previous results in [R. Kozera, Appl. Math. Comput., 44 (1991), pp. 1--103] and [R. Onn and A. Bruckstein, Int. J. Comput. Vision, 5 (1990), pp. 105--113]. We also propose some approximation schemes for the numerical solution of this problem and analyze the properties of two approximation schemes: an upwind finite difference scheme and a semi-Lagrangian scheme. Finally, we present some numerical tests on smooth and nonsmooth surfaces coming from virtual and real images.

In this paper, a general framework based on fractional-order partial differential equations allows to solve image reconstruction problems. The algorithm presented in this work combines two previous notions: a fractional derivative implementation by Discrete Fourier Transform and the edge detection by topological gradient. The purpose of the paper is to extend some existing results in image denoising problem with fractional-order diffusion equations and presents new results in image inpainting. The results emphasize the importance of particular fractional-orders.

We deal with the inverse problem of reconstructing a surface with photometric stereo technique, i.e. using two or more pictures of the surface lighted under different light sources. The new model studied in this paper allows us to extend previous results [1, 2] obtaining a uniqueness result and to solve the classical convex/concave ambiguity of the Shape from Shading (SfS) problem. Finally, we propose an approximation scheme for the solution of the problem testing it on real and synthetic images.