Reconstructing the geometry and material properties of translucent objects from images is a challenging problem due to the complex light propagation of translucent media and the inherent ambiguity of inverse rendering. Therefore, previous works often make the assumption that the objects are opaque or use a simplified model to describe translucent objects, which significantly affects the reconstruction quality and limits the downstream tasks such as relighting or material editing. We present a novel framework that tackles this challenge through a combination of physically grounded and data-driven strategies. At the core of our approach is a hybrid rendering supervision scheme that fuses a differentiable physical renderer with a learned neural renderer to guide reconstruction. To further enhance supervision, we introduce an augmented loss tailored to the neural renderer. Our system takes as input a flash/no-flash image pair, enabling it to disambiguate complex light propagation that happens inside translucent objects. We train our model on a large-scale synthetic dataset of 117 K scenes and evaluate across both synthetic benchmarks and real-world captures. To mitigate the domain gap between synthetic and real data, we contribute a new real-world dataset with ground-truth surface normals and fine-tune our model accordingly. Extensive experiments validate the robustness and accuracy of our method across diverse scenarios. • We tackle the inverse rendering of translucent objects by jointly estimating shape, spatially-varying surface reflectance, SSS parameters, and illumination from a single-view flash/no-flash pair. • We propose a hybrid rendering architecture combining physically accurate surface modeling with a learned neural renderer for SSS. • We introduce a novel augmented loss strategy to improve the supervision of SSS effects. • We build a large-scale synthetic dataset and a real-world benchmark to train and validate our model. • We demonstrate state-of-the-art performance on both synthetic and real translucent objects, supported by fine-tuning to address domain shift.

In this work, we propose an inverse rendering model that estimates 3D shape, spatially-varying reflectance, homogeneous subsurface scattering parameters, and an environment illumination jointly from only a pair of captured images of a translucent object. In order to solve the ambiguity problem of inverse rendering, we use a physically-based renderer and a neural renderer for scene reconstruction and material editing. Because two renderers are differentiable, we can compute a reconstruction loss to assist parameter estimation. To enhance the supervision of the proposed neural renderer, we also propose an augmented loss. In addition, we use a flash and no-flash image pair as the input. To supervise the training, we constructed a large-scale synthetic dataset of translucent objects, which consists of 117K scenes. Qualitative and quantitative results on both synthetic and real-world datasets demonstrated the effectiveness of the proposed model. Code and Data are available at https://github.com/ligoudaner377/homo_translucent

We introduce a method of recovering the shape of a smooth dielectric object using diffuse polarization images taken with different directional light sources. We present two constraints on shading and polarization and use both in a single optimization scheme. This integration is motivated by photometric stereo and polarization-based methods having complementary abilities. Polarization gives strong cues for the surface orientation and refractive index, which are independent of the light direction. However, employing polarization leads to ambiguities in selecting between two ambiguous choices of the surface orientation, in the relationship between the refractive index and zenith angle (observing angle). Moreover, polarization-based methods for surface points with small zenith angles perform poorly owing to the weak polarization. In contrast, the photometric stereo method with multiple light sources disambiguates the surface normals and gives a strong relationship between surface normals and light directions. However, the method has limited performance for large zenith angles and refractive index estimation and faces strong ambiguity when light directions are unknown. Taking the advantages of these methods, our proposed method recovers surface normals for small and large zenith angles, light directions, and refractive indexes of the object. The proposed method is positively evaluated in simulations and real-world experiments.

We introduce a method to recover the shape of a smooth dielectric object from polarization images taken with a light source from different directions. We present two constraints on shading and polarization and use both in a single optimization scheme. This integration is motivated by the fact that photometric stereo and polarization-based methods have complementary abilities. The polarization-based method can give strong cues for the surface orientation and refractive index, which are independent of the light direction. However, it has ambiguities in selecting between two ambiguous choices of the surface orientation, in the relationship between refractive index and zenith angle (observing angle), and limited performance for surface points with small zenith angles, where the polarization effect is weak. In contrast, photometric stereo method with multiple light sources can disambiguate the surface orientation and give a strong relationship between the surface normals and light directions. However, it has limited performance for large zenith angles, refractive index estimation, and faces the ambiguity in case the light direction is unknown. Taking their advantages, our proposed method can recover the surface normals for both small and large zenith angles, the light directions, and the refractive indexes of the object. The proposed method is successfully evaluated by simulation and real-world experiments.

This paper describes a method for score-level fusion in multi-cue two-class classification problems. Fusion based on the probability density function (PDF) of multiple scores given for each class is a promising approach because it guarantees optimality as long as the estimated PDFs are correct. Instead of lattice-type control points used in previous non-parametric density-based approaches, floating control points (FCPs) are introduced to improve scalability and the whole posterior distribution is represented by interpolation or extrapolation using generalized Delaunay triangulation. Given a set of FCPs obtained by k-means, posteriors on the FCPs are estimated by an energy minimization framework using training samples. The experiments, using both simulation data as well as several types of real data from three publicly available score databases for multi-cue biometric authentication, demonstrate the effectiveness of the proposed method.

In this paper, we propose a method of inertial sensor-based gait authentication by inter-period phase registration of an owner's gallery. In spite of the importance for gait authentication of constructing a gallery of phase-registered gait patterns, previous implementations just relied on simple methods of period detection based on heuristic knowledge such as local peaks/valleys or local auto-correlation of the gait signals. Consequently, we propose to improve a gait gallery by incorporating a phase registration technique which globally optimizes inter-period phase consistency in an energy minimization framework. However, the previous phase registration technique suffers from a phase distortion problem due to ambiguities in the combination of a periodic signal function and a phase evolution function. We present a linear phase evolution prior to constructing an undistorted gait signal for better matching performance. Experiments using real gait signals from 32 subjects show that the proposed methods outperform the latest methods in the field.

Although RANSAC is the most widely used robust estimator in computer vision, it has certain limitations making it ineffective in some situations, such as the motion estimation problem, in which uncertainty on the image features changes according to the capturing conditions. The greatest problem is that the threshold used by RANSAC to detect inliers cannot be changed adaptively; instead it is fixed by the user. An adaptive scale algorithm must therefore be applied in such cases. In this paper, we propose a new adaptive scale robust estimator that adaptively finds the best solution with the best scale to fit the inliers, without the need for predefined information. Our new adaptive scale estimator matches the residual probability density from an estimate and the standard Gaussian probability density function to find the best inlier scale. Our algorithm is evaluated in several motion estimation experiments under varying conditions and the results are compared with several of the latest adaptive-scale robust estimators.

We propose a new egomotion estimation algorithm for a compound omnidirectional camera. Image features are detected by a conventional feature detector and then quickly classified into near and far features by checking infinity on the omnidirectional image of the compound omnidirectional sensor. Egomotion estimation is performed in two steps: first, rotation is recovered using far features; then translation is estimated from near features using the estimated rotation. RANSAC is used for estimations of both rotation and translation. Experiments in various environments show that our approach is robust and provides good accuracy in real-time for large motions.

Camera ego-motion consists of translation and rotation, in which rotation can be described simply by distant features. We present a robust rotation estimation using distant features given by our compound omnidirectional sensor. Features are detected by a conventional feature detector, and then distant features are identified by checking the infinity on the omnidirectional image of the compound sensor. The rotation matrix is estimated between consecutive video frames using RANSAC with only distant features. Experiments with various environments show that our approach is robust and also gives reasonable accuracy in real-time.