In this paper, we propose a convex optimization-based method for the visual saliency prediction of high dynamic range (HDR) images, which allows straightforward reuse of any existing saliency estimation methods for typical images with low dynamic range (LDR). First, the proposed method decomposes a given HDR image into multiple LDR images with different levels of intensity using a tone-mapping-based synthesis of imaginary multiexposure images. For each decomposed image, a standard saliency estimation method is then applied for typical LDR images. Finally, the saliency map of each decomposed image is integrated into a single map by solving convex optimization problems. The proposed method is applied to actual HDR images and its effectiveness is demonstrated.

The performance of image denoising algorithms generally depends largely on the selection of the parameters. We address the problem of optimizing the denoising parameters to achieve maximum denoising performance. Most existing methods for no-reference denoising parameter optimization either use the estimated image noise or individual no-reference image quality evaluation metrics. In the paper, we introduce a natural image statistics based on the generalized Gaussian distribution and an elastic net regularization regression model. Consequently, we propose their use for performing no-reference parameter optimization in the state-of-the-art BM3D denoising algorithm, adaptively depending on which of the following most widely used full-reference image quality evaluation metrics is optimized: FSIM, SSIM, MS-SSIM, without any knowledge of the noise distribution.

Omnidirectional images with a 360-degree field of view have much smaller foreground regions compared to background regions and thus are difficult to visually recognize, especially when displaying on a small size display. In this paper, we propose a method for enhancing visibility of omnidirectional images by enlarging the user specified foreground regions while compressing the other background regions without the sense of visual incompatibility. The proposed method first transforms the omnidirectional image into a 3D spherical image, and then relocates the spherical nodes, which are assigned pixel values, from their initial position to a preferable position based on point correspondence between spherical polygons. The experiments demonstrate that our approach produces satisfactory omnidirectional images in which the foreground regions are enlarged analogously.

We propose a framework to extend corner feature detection in standard rectangular images with less distortion to distorted circular images captured with fisheye lenses. To solve two problems of nonuniformity of spatial resolution and spherical polar coordinates singularity, our approach makes use of a modification in the Yin-Yang grid, which is an overset grid consisting of two latitude/longitude coordinate systems. The main contribution is to keep the use of existing corner detection programs. Experimental results on synthetic and real images demonstrate the effectiveness of our method.

We propose a new weight optimization method for image fusion to obtain enhanced images. Given as input a set of images of a static scene captured under different photographic conditions such as exposure time and depth of focus, the algorithm modifies the input images based on visual saliency and then searches for a linear combination of the images that maximizes the total amount of gradient magnitudes. The search is performed by approximating a non-differentiable Lagrangian with the log-sum-exp function and then iteratively updating the closed-form analytical solution until convergence. The simple algorithm has converged fast and has demonstrated significant improvement in image quality over several conventional techniques.

We propose a new error diffusion method based on integral images. The proposed method has only 3 error diffusion coefficients, and therefore is computationally efficient compared to the conventional error diffusion methods. Experimental results show that the proposed method is fast while keeping the halftone quality.

Iterative approaches have been mostly used for blind source separation. The success of iterative methods generally depends upon appropriate selection of the values of the input parameters such as initial conditions and control parameters. We propose a direct non-iterative method for blindly separating linearly mixed images using the boundary values of the determinant of the relative transformation matrix between the mixtures. The algorithm is a non-trivial extension of the mixture ratio based algorithm originally proposed by Diamantaras and Papadimitriou (2005) and allows for a finite arbitrary number of source images. We demonstrate the performance of our method on synthetic images.

Recently, the generalized low rank approximations of matrices (GLRAM) have been proposed for dimensionality reduction of matrices such as images. However, in GLRAM, it is necessary for users to specify the numbers of rows and columns in low rank matrices. In this paper, we propose a method for determining them semiautomatically by symmetrizing GLRAM. Experimental results show that the proposed method can determine the optimal ranks of matrices while achieving competitive approximation performance.

As it is becoming more common for various handheld devices to be equipped with a camera, applications that use image recognition are increasing. Therefore, improving the usability of these devices by enhancing their image capture characteristics is becoming more important. We present ClearPlate - a new optical unit for image capture. It has a physically transparent viewfinder through which the user can observe the target directly without offset; the framed image is captured by the built-in camera. We also present a user study that evaluates ClearPlate performance in the 2D barcode acquisition task. Results show that ClearPlate significantly outperforms the camera-phone-based approach with regard to 2D barcode acquisition.

Estimating the illumination and the reflectance properties of an object surface from a sparse set of images is an important but inherently ill-posed problem. The problem becomes even harder if we wish to account for the spatial variation of material properties on the surface. In this paper, we derive a novel method for estimating the spatially varying specular reflectance properties, of a surface of known geometry, as well as the illumination distribution from a specular-only image, for instance, captured using polarization to separate reflection components. Unlike previous work, we do not assume the illumination to be a single point light source. We model specular reflection with a spherical statistical distribution and encode the spatial variation with radial basis functions of its parameters. This allows us to formulate the simultaneous estimation of spatially varying specular reflectance and illumination as a sound probabilistic inference problem, in particular, using Csiszar's I-divergence measure. To solve it, we derive an iterative algorithm similar to expectation maximization. We demonstrate the effectiveness of the method on synthetic and real-world scenes.

In the paper, we propose a new method for blind separation of an arbitrary number of images from a set of their linear mixtures with unknown coefficients. This approach is as follows. We first introduce a novel multiple correlation between one image and a set of multiple images. Then this multiple correlation leads us to provide a set of simultaneous linear equations for updating each mixture of images. Finally, source images are recovered by iterating between solving the sets of equations and cyclically permuting the mixtures of images. The technique can be applied for extracting multiple layers from images containing multiple reflections and transparency.

We propose two methods for robustifying multilinear principal component analysis (MPCA) which is an extension of the conventional PCA for reducing the dimensions of vectors to higher-order tensors. For two kinds of outliers, i.e., sample outliers and intra-sample outliers, we derive iterative algorithms on the basis of the Lagrange multipliers. We also demonstrate that the proposed methods outperform the original MPCA when datasets contain such outliers experimentally.

This paper presents a parallel algorithm of the Level Set Method named the Parallel Fast Level Set Method, and its application for real-time 3D reconstruction of human shape and motion. The Fast Level Set Method is an efficient implementation algorithm of the Level Set Method and has been applied to several applications such as object tracking in video images and 3D shape reconstruction using multiple stereo cameras. In this paper, we implement the Fast Level Set Method on a PC cluster and develop a real-time motion capture system for arbitrary viewpoint image synthesis. To obtain high performance on a PC cluster, efficient load-balancing and resource allocation algorithms are crucial problems. We develop a novel optimization technique of load distribution based on the estimation of moving direction of object boundaries. In this technique, the boundary motion is estimated in the framework of the Fast Level Set Method, and the optimum load distribution is predicted and performed according to the estimated boundary motion and the current load balance. Experiments of human shape reconstruction and arbitrary viewpoint image synthesis using the proposed system are successfully carried out.

We present a method for simultaneously estimating the illumination of a scene and the reflectance property of an object from single view images - a single image or a small number of images taken from the same viewpoint. We assume that the illumination consists of multiple point light sources and the shape of the object is known. First, we represent the illumination on the surface of a unit sphere as a finite mixture of von Mises-Fisher distributions based on a novel spherical specular reflection model that well approximates the Torrance-Sparrow reflection model. Next, we estimate the parameters of this mixture model including the number of its component distributions and the standard deviation of them, which correspond to the number of light sources and the surface roughness, respectively. Finally, using these results as the initial estimates, we iteratively refine the estimates based on the original Torrance-Sparrow reflection model. The final estimates can be used to relight single-view images such as altering the intensities and directions of the individual light sources. The proposed method provides a unified framework based on directional statistics for simultaneously estimating the intensities and directions of an unknown number of light sources as well as the specular reflection parameter of the object in the scene.

The Chan-Vese level set algorithm has been successfully applied to segmentation of images on Cartesian coordinate meshes, including ordinary planar images. In this paper we present a Chan-Vese model for segmentation of images on polar coordinate meshes, such as topography and remote sensing images. The image segmentation is accomplished by formulating the associated evolution equation in the polar coordinate system and then numerically solving the partial differential equation on an overset grid system called the Yin-Yang grid, which is free from the problem of singularity at the poles. We include examples of segmentations of real earth data that demonstrate the performance of our method.

This paper introduces a new motion capture system for recovering 3D models of multiple persons separately and robustly against occlusion in real-time. Various markerless motion capture systems using video cameras have been proposed so far. However, in case that there are multiple persons in the scene at the same time, it is quite difficult to reconstruct a precise 3D model of each person separately due to the occlusion between them. To deal with this problem, the fast level set method is utilized in the proposed system for integrating stereo range data which is captured by multiple stereo cameras located around the target people. To reconstruct precise 3D models in real-time, the proposed system is implemented on a PC cluster with seven PCs and four stereo cameras. Tracking experiment of multiple persons and real-time reconstruction of 3D human models using the proposed system are successfully carried out

In tins paper we propose a new method for simultaneously estimating the illumination of the scene and the reflectance property of an object from a single image. We assume that the illumination consists of multiple point sources and the shape of the object is known. Unlike previous methods, we will recover not only the direction and intensity of the light sources, but also the number of light sources and the specular reflection parameter of the object. First, we represent the illumination on the surface of a unit sphere as a finite mixture of von Mises-Fisher distributions by deriving a spherical specular reflection model. Next, we estimate this mixture and the number of distributions. Finally, using this result as initial estimates, we refine the estimates using the original specular reflection model. We can use the results to render the object under novel lighting conditions

Several techniques have been developed for recovering reflectance properties of real surfaces under unknown illumination. However, in most cases, those techniques assume that the light sources are located at inifinity, which cannot be applied safely to, for example, reflectance modeling of indoor environments. In this paper, we propose two types of methods to estimate the surface reflectance property of an object, as well as the position of a light source from a single view without the distant illumination assumption, thus relaxing the conditions in the previous methods. Given a real image and a 3D geometric model of an object with specular reflection as inputs, the first method estimates the light source position by fitting to the Lambertian diffuse component, while separating the specular and diffuse components by using an iterative relaxation scheme. Our second method extends that first method by using as input a specular component image, which is acquired by analyzing multiple polarization images taken from a single view, thus removing its constraints on the diffuse reflectance property. This method simultaneously recovers the reflectance properties and the light source positions by optimizing the linearity of a log-transformed Torrance-Sparrow model. By estimating the object's reflectance property and the light source position, we can freely generate synthetic images of the target object under arbitrary lighting conditions with not only source direction modification but also source-surface distance modification. Experimental results show the accuracy of our estimation framework.

Several techniques have been developed for recovering reflectance properties of real surfaces under unknown illumination conditions. However, in most cases, those techniques assume that the light sources are located at infinity, which cannot be applied to, for example, photometric modelling of indoor environments. We propose two methods to estimate the surface reflectance property of an object, as well as the position of a light source from a single image without the distant illumination assumption. Given a color image of an object with specular reflection as an input, the first method estimates the light source position by fitting to the Lambertian diffuse component, while separating the specular and diffuse components by using an iterative relaxation scheme. Moreover, we extend the above method by using a single specular image as an input, thus removing its constraints on the diffuse reflectance property and the number of light sources. This method simultaneously recovers the reflectance properties and the light source positions by optimizing the linearity of a log-transformed Torrance-Sparrow model. By estimating the object's reflectance property and the light source position, we can freely generate synthetic images of the target object under arbitrary source directions and source-surface distances.

This paper presents a method to estimate geometrical, photometrical, and environmental information of a single-viewed object in one integrated framework under fixed viewing position and fixed illumination direction. These three types of information are important to render a photorealistic image of a real object. Photometrical information represents the texture and the surface roughness of an object, while geometrical and environmental information represent the 3D shape of an object and the illumination distribution, respectively. The proposed method estimates the 3D shape by computing the surface normal from polarization data, calculates the texture of the object from the diffuse only reflection component, determines the illumination directions from the position of the brightest intensity in the specular reflection component, and finally computes the surface roughness of the object by using the estimated illumination distribution.

The paper proposes a novel method of modeling multiobject scenes or objects of arbitrary topology by a marching method based on mathematical morphology operations with a variable-size structuring element. We assume that 3D data on the whole object surface has been acquired. A narrow-band stopping region is built by merging the neighborhood of every data point. A single close-surface surrounding the data point set is given as an initial shape. The surface shrinks and splits smoothly by repeated erosions with a ball structuring element, whose size increases continuously as the curvature of the surface increases. Every surface point stops moving if it lies in the stopping region. After this surface fitting to the stopping region, the region is narrowed, and the fitting-and-narrowing cycle is repeated until the model is consistent with the object surface on the basis of a topological check. After the above rough estimation, for model refinement, we extract a quadrangular mesh from the resultant surface, and then perform precise fitting to the 3D data by using an energy minimisation.

Level set methods were proposed mainly by mathematicians for extracting bounding contours of multiple regions or shapes with holes. In the paper, we propose a new method of constructing a surface/solid model of a 3-D object of arbitrary topology on the basis of the approach. Given sensor data covering the whole object surface, the method begins with an initial approximation of the object by evolving a closed-surface into a model typologically equivalent to the real object. The refined approximation is then performed by deforming it at an optimally tuned speed. By numerically solving the level set equations for tracking the evolving surfaces, it becomes possible to handle 3-D objects of arbitrary topology while maintaining high accuracy in the shape fitting.

Several methods have been proposed for building 3D object models from multiple range images mainly on the basis of the deformable models and pseudo-dynamics. However, the patches are usually placed crossing over the edge regions, and the surface orientation discontinuities cannot be properly preserved as a result. This paper proposes a new 3D object modeling method that generates refined triangular meshes while preserving the surface discontinuities. We formulate the process into a problem of minimizing an energy functional by using the idea of line processes and Markov random fields. Moreover, we show that near-optimal solutions can be obtained by utilizing the continuation method.