It is well known in the photometric stereo literature that uncalibrated photometric stereo, where light source strength and direction are unknown, can recover the surface geometry of a Lambertian object up to a 3-parameter linear transform known as the generalized bas relief (GBR) ambiguity. Many techniques have been proposed for resolving the GBR ambiguity, typically by exploiting prior knowledge of the light sources, the object geometry, or non-Lambertian effects such as specularities. A less celebrated consequence of the GBR transformation is that the albedo at each surface point is transformed along with the geometry. Thus, it should be possible to resolve the GBR ambiguity by exploiting priors on the albedo distribution. To the best of our knowledge, the only time the albedo distribution has been used to resolve the GBR is in the case of uniform albedo. We propose a new prior on the albedo distribution : that the entropy of the distribution should be low. This prior is justified by the fact that many objects in the real-world are composed of a small finite set of albedo values.

A set of images of a Lambertian surface under varying lighting directions defines its shape up to a three-parameter generalized bas-relief (GBR) ambiguity. In this paper, we examine this ambiguity in the context of surfaces having an additive non-Lambertian reflectance component, and we show that the GBR ambiguity is resolved by any non-Lambertian reflectance function that is isotropic and spatially invariant. The key observation is that each point on a curved surface under directional illumination is a member of a family of points that are in isotropic or reciprocal configurations. We show that the GBR can be resolved in closed form by identifying members of these families in two or more images. Based on this idea, we present an algorithm for recovering full Euclidean geometry from a set of uncalibrated photometric stereo images, and we evaluate it empirically on a number of examples.

Single particle reconstruction using Cryo-Electron Microscopy (cryo-EM) is an emerging technique in structural biology for estimating the 3-D structure (density) of protein macromolecules. Unlike tomography where a large number of images of a specimen can be acquired, the number of images of an individual particle is limited because of radiation damage. Instead, the specimen consists of identical copies of the same protein macro-molecule embedded in vitreous ice at random and unknown 3-D orientations. Because the images are extremely noisy, thousands to hundreds-of-thousands of projections are needed to achieve the desired resolution of 5A. Along with differences of the imaging modality compared to photographs, single particle reconstruction provides a unique set of challenges to existing computer vision algorithms. Here, we introduce the challenge and opportunity of reconstruction from transmission electron micrographs, and briefly describe our contributions in areas of particle detection, contrast transfer function (CTF) estimation, and initial 3-D model construction. Reconstructing the Structure of Protein Macromolecules One of the most exciting challenges for biology today is understanding the molecular machinery of the cell as a working, dynamic system. Critical to this understanding is determining the 3-D structure of protein macromolecules, a task that is often accomplished using x-ray crystallography. The technique of cryo electron microscopy (cryo-EM) has a unique role to play in addressing this challenge as it can provide structural information of large macromolecular complexes in a variety of conformational and compositional states while preserved under close to physiological conditions. Traditionally the methods for cryo-EM have been time consuming and labor intensive, involving data acquisition, analysis and averaging of thousands to hundreds of thousands of images (views) of the individual macro-molecular complexes. Thus, over the last few years there has been considerable interest and substantial effort devoted to developing automated methods to improve the accuracy, robustness, ease of use, and throughput of cryo-EM [1, 2, 3, 12, 15, 17], and this abstract considers three aspects originally presented in [9, 10, 11]. 1

This paper addresses the problem of reconstructing the density of a scene from multiple projection images produced by modalities such as x-ray, electron microscopy, etc. where an image value is related to the integral of the scene density along a 3D line segment between a radiation source and a point on the image plane. While computed tomography (CT) addresses this problem when the absolute orientation of the image plane and radiation source directions are known, this paper addresses the problem when the orientations are unknown - it is akin to the structure-from-motion (SFM) problem when the extrinsic camera parameters are unknown. We study the problem within the context of reconstructing the density of protein macro-molecules in Cryogenic Electron Microscopy (cryo-EM), where images are very noisy and existing techniques use several thousands of images. In a non-degenerate configuration, the viewing planes corresponding to two projections, intersect in a line in 3D. Using the geometry of the imaging setup, it is possible to determine the projections of this 3D line on the two image planes. In turn, the problem can be formulated as a type of orthographic structure from motion from line correspondences where the line correspondences between two views are unreliable due to image noise. We formulate the task as the problem of denoising a correspondence matrix and present a Bayesian solution to it. Subsequently, the absolute orientation of each projection is determined followed by density reconstruction. We show results on cryo-EM images of proteins and compare our results to that of Electron Micrograph Analysis (EMAN) - a widely used reconstruction tool in cryo-EM.

Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe two such invariants" one invariant to specular reflections, and the other invariant to both specular reflections and diffuse shading" that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from subspaces of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, material-based segmentation, and motion estimation.

We present a photometric stereo method for non-diffuse materials that does not require an explicit reflectance model or reference object. By computing a data-dependent rotation of RGB color space, we show that the specular reflection effects can be separated from the much simpler, diffuse (approximately Lambertian) reflection effects for surfaces that can be modeled with dichromatic reflectance. Images in this transformed color space are used to obtain photometric reconstructions that are independent of the specular reflectance. In contrast to other methods for highlight removal based on dichromatic color separation (e.g., color histogram analysis and/or polarization), we do not explicitly recover the specular and diffuse components of an image. Instead, we simply find a transformation of color space that yields more direct access to shape information. The method is purely local and is able to handle surfaces with arbitrary texture.

In this paper, we present a fast algorithm for automatically generating a 3D model of a human face and synthesizing affects on it. This work is a step towards low bandwidth virtual tele-presence using avatars. The idea is to generate a 3D model of a person's face and animate different emotions on the face at a remote observer's end, thus eliminating the need to send raw video. This research is part of a larger project, which deals with development of novel sensory systems and interfaces to enhance safety of a driver. The proposed algorithm uses a stereo pair of images of a driver's face and creates a 3D mesh model by deforming and texture mapping a predefined 3D mesh.