Reflectance Transformation Imaging (RTI) is a computational photography technique in which an object is acquired from a fixed point-of-view with different light directions. The aim is to estimate the light transport function at each point so that the object can be interactively relighted in a physically-accurate way, revealing its surface characteristics. In this paper, we propose a novel RTI approach describing surface reflectance as an implicit neural representation acting as a ”relightable image” for a specific object. We propose to represent the light transport function with a Neural Reflectance Field (NRF) model, feeding it with pixel coordinates, light direction, and a latent vector encoding the per-pixel reflectance in a neighbourhood. These vectors, computed during training, allow a more accurate relighting than a pure implicit representation (i.e., relying only on positional encoding) enabling the NRF to handle complex surface shadings. Moreover, they can be efficiently stored with the learned NRF for compression and transmission. As an additional contribution, we propose a novel synthetic dataset containing objects of various shapes and materials created with a physically based rendering software. An extensive experimental section shows that the proposed NRF accurately models the light transport function for challenging datasets in synthetic and real-world scenarios.

Whitecaps are formed at the ocean surface when breaking waves entrain sufficient quantities of air to produce an optically distinct signal at the water surface. Whitecaps drive energy transfer from the surface wave field to the upper ocean and the entrained air drives key bubble-mediated processes such as air-sea gas exchange and aerosol production flux. By exploiting the broadband scattering of light by the surface whitecaps, this study develops an algorithm for automated whitecap detection and tracking (AWDAT) from fixed image systems, in order to detect and track individual whitecaps. AWDAT introduces new image processing and computer vision techniques that handle temporal developments of whitecaps and complex behaviors of whitecap foam patches such as splitting and merging. We teach a learning-based model to filter tracked whitecaps based on the AWDAT-measured foam area, breaking speed, and direction time series. The algorithm is tested on three different image datasets to assess its performance with different camera systems in different geographic locations—the Adriatic Sea (AS), Black Sea (BS), and Yellow Sea (YS). Applications of AWDAT are demonstrated by aggregating whitecap statistics from geometric, kinematic, and dynamic measurements of individual breaking waves, which are then evaluated within the volume-time-integral (VTI) method and Phillips (1985)$\Lambda (c)$spectral framework.

Structured-light 3D reconstruction techniques are employed in a wide range of applications for industrial inspection. In particular, some tasks require micrometric precision for the identification of microscopic surface irregularities. We propose a novel calibration technique for structured-light systems adopting telecentric lenses for both camera and projector. The device exploits a fixed light pattern (striped-based) to perform accurate microscopic surface reconstruction and measurements. Our method employs a sphere with a known radius as calibration target and takes advantage of the orthographic projection model of the telecentric lenses to recover the bundle of planes originated by the projector. Once the sheaf of parallel planes is properly described in the camera reference frame, the triangulation of the surface’s object hit by the light stripes is immediate. Moreover, we tested our technique in a real-world scenario for industrial surface inspection by implementing a complete pipeline to recover the intersections between the projected planes and the surface. Experimental analysis shows the robustness of the proposed approach against synthetic and real-world test data.

Industrial manufacturing processes often involve a visual control system to detect possible product defects during production. Such inspection devices usually include one or more cameras and several light sources designed to highlight surface imperfections under different illumination conditions (e.g. bumps, scratches, holes). In such scenarios, a preliminary calibration procedure of each component is a mandatory step to recover the system’s geometrical configuration and thus ensure a good process accuracy. In this paper we propose a procedure to estimate the position of each light source with respect to a camera network using an inexpensive Lambertian spherical target. For each light source, the target is acquired at different positions from different cameras, and an initial guess of the corresponding light vector is recovered from the analysis of the collected intensity isocurves. Then, an energy minimization process based on the Lambertian shading model refines the result for a pr ecise 3D localization. We tested our approach in an industrial setup, performing extensive experiments on synthetic and real-world data to demonstrate the accuracy of the proposed approach.

Polarisation Filter Array (PFA) cameras allow the analysis of light polarisation state in a simple and cost-effective manner. Such filter arrays work as the Bayer pattern for colour cameras, sharing similar advantages and drawbacks. Among the others, the raw image must be demosaiced considering the local variations of the PFA and the characteristics of the imaged scene. Non-linear effects, like the cross-talk among neighbouring pixels, are difficult to explicitly model and suggest the potential advantage of a data-driven learning approach. However, the PFA cannot be removed from the sensor, making it difficult to acquire the ground-truth polarization state for training. In this work we propose a novel CNN-based model which directly demosaics the raw camera image to a per-pixel Stokes vector. Our contribution is twofold. First, we propose a network architecture composed by a sequence ofMosaiced Convolutionsoperating coherently with the local arrangement of the different filters. Second, we introduce a new method, employing a consumer LCD screen, to effectively acquire real-world data for training. The process is designed to be invariant by monitor gamma and external lighting conditions. We extensively compared our method against algorithmic and learning-based demosaicing techniques, obtaining a consistently lower error especially in terms of polarisation angle.

Phase-shift is one of the most effective techniques in 3D structured-light scanning for its accuracy and noise resilience. However, the periodic nature of the signal causes a spatial ambiguity when the fringe periods are shorter than the projector resolution. To solve this, many techniques exploit multiple combined signals to unwrap the phases and thus recovering a unique consistent code. In this paper, we study the phase estimation and unwrapping problem in a stochastic context. Assuming the acquired fringe signal to be affected by additive white Gaussian noise, we start by modelling each estimated phase as a zero-mean Wrapped Normal distribution with variance σ ̄ 2 . Then, our contributions are twofolds. First, we show how to recover the best projector code given multiple phase observations by means of a ML estimation over the combined fringe distributions. Second, we exploit the Cramér-Rao bounds to relate the phase variance σ ̄ 2 to the variance of the observed signal, that can be easily estimated online during the fringe acquisition. An extensive set of experiments demonstrate that our approach outperforms other methods in terms of code recovery accuracy and ratio of faulty unwrappings.

The ubiquitous presence of cylindrical shapes in both natural and man-made environments makes their automated extraction a pivotal task for a broad range of applications such as robot manipulation, reverse engineering and automated industrial inspection. Albeit conceptually simple, the task of fitting cylinders from 3D data can quickly become challenging if performed "in-the-wild", when no prior is given to the number of primitives to find or when the point cloud is noisy and not oriented. In this paper we introduce a new robust approach to iteratively extract cylindrical primitives from a 3D point cloud by exploiting mutual consensus of different cylinder candidates. First, a set of possible axes is generated by slicing the point cloud with multiple random planes. Then, a game-theoretic inlier selection process is performed to extract a subset of axes maximizing the fitness against a payoff function based on the shortest geodesic path in SE(3) between pairs of corresponding 3D lines. Finally, the probability distribution resulting from the previous selection step is used to weight the input candidates and robustly obtain the final cylinder coefficients. Compared to other methods, our approach does not require point normals, offers superior resilience to noise and does not depend on delicate tuning of multiple parameters.

Within modern Deep Learning setups, data augmentation is the weapon of choice when dealing with narrow datasets or with a poor range of different samples. However, the benefits of data augmentation are abysmal when applied to a dataset which is inherently unable to cover all the categories to be classified with a significant number of samples. To deal with such desperate scenarios, we propose a possible last resort: Cross-Dataset Data Augmentation. That is, the creation of new samples by morphing observations from a different source into credible specimens for the training dataset. Of course specific and strict conditions must be satisfied for this trick to work. In this paper we propose a general set of strategies and rules for Cross-Dataset Data Augmentation and we demonstrate its feasibility over a concrete case study. Even without defining any new formal approach, we think that the preliminary results of our paper are worth to produce a broader discussion on this topic.

Among several structured light approaches, phase shift is the most widely adopted in real-world 3D reconstruction devices. This is mainly due to its high accuracy, strong resilience to noise and straightforward implementation. However, Phase shift also exhibits an inherent weakness, that is the spatial ambiguity resulting from the periodicity of the sinusoidal wave adopted. Of course many phase unwrapping methods have been proposed to solve such ambiguity. One of the most promising methods exploits additional signals of mutually prime periods, in order to observe a distinct combination of phases for each spatial point. Unfortunately, for such combination to be properly recognized, a very high accuracy in phase recovery must be attained for each signal. In fact, even modest errors could lead to unwrapping faults, making the overall approach much less resilient to noise than plain phase shift. With this paper we introduce a feasible and effective fault recovery method that can be directly applied to multi-period phase shift. The combined pipeline offers an optimal accuracy and coverage even with high noise conditions, overcoming the setbacks of the original method. The performance of such pipeline is established by means of an in depth set of experimental evaluations and comparison, both with real and synthetically generated data.

Among structured light strategies, the ones based on phase shift are considered to be the most adaptive with respect to the features of the objects to be captured. Inter alia, the theoretical invariance to signal strength and the absence of discontinuities in intensity, make phase shift an ideal candidate to deal with complex surfaces of unknown geometry, color and texture. However, in practical scenarios, unexpected artifacts could still result due to the characteristics of real cameras. This is the case, for instance, with high contrast areas resulting from abrupt changes in the albedo of the captured objects. In fact, the not negligible size of pixels and the presence of blur can produce a mix of signal integration from adjacent areas with different albedo. This, in turn, would result in a bias in the phase recovery and, consequentially, in an inaccurate 3D reconstruction of the surface. While this problem affects most structure light methods based on phase shift or derived techniques, little effort has been put in addressing it. With this paper we propose a model for the phase corruption and a theoretically sound correction step to be adopted to compensate the bias. The practical effectiveness of our approach is well demonstrated by a complete set of experimental evaluations.

A number of critical factors arises when a complex 3D scene is to be reconstructed by means of a large sequence of different views. Some of them are related to the ability to recover the correct identity and the accurate projection of each observed feature. Other sources of error are tied to the reliability of the orientation estimate for each view. With this paper we propose a method that tries to solve both problems at the same time, while being also inherently resilient to outliers. At the core of the approach stands a widely adopted game-theoretical selection technique, which has already been successfully embraced to address similar tasks. The original inception, however, has been further refined to address a wider range of scenarios, as well as to offer a reduced memory consumption and computation complexity. By exploiting these enhancements, we were able to apply our technique to a large-scale setup involving several hundreds of view points and tens of thousands of independent observations.