There is widespread interest in estimating the fluorescence properties of natural materials in an image. However, the separation between reflected and fluoresced components is difficult, because it is impossible to distinguish reflected and fluoresced photons without controlling the illuminant spectrum. We show how to jointly estimate the reflectance and fluorescence from a single set of images acquired under multiple illuminants. We present a framework based on a linear approximation to the physical equations describing image formation in terms of surface spectral reflectance and fluorescence due to multiple fluorophores. We relax the non-convex, inverse estimation problem in order to jointly estimate the reflectance and fluorescence properties in a single optimization step. We provide a software implementation of the solver for our method and prior methods. We evaluate the accuracy and reliability of the method using both simulations and experimental data. To evaluate the methods experimentally we built a custom imaging system using a monochrome camera, a filter wheel with bandpass transmissive filters and a small number of light emitting diodes. We compared the methods based upon our framework with the ground truth as well as with prior methods.

There are many scientific, medical and industrial imaging applications where users have full control of the scene illumination and color reproduction is not the primary objective For example, it is possible to co-design sensors and spectral illumination in order to classify and detect changes in biological tissues, organic and inorganic materials, and object surface properties. In this paper, we propose two different approaches to illuminant spectrum selection for surface classification. In the supervised framework we formulate a biconvex optimization problem where we alternate between optimizing support vector classifier weights and optimal illuminants. We also describe a sparse Principal Component Analysis (PCA) dimensionality reduction approach that can be used with unlabeled data. We efficiently solve the non-convex PCA problem using a convex relaxation and Alternating Direction Method of Multipliers (ADMM). We compare the classification accuracy of a monochrome imaging sensor with optimized illuminants to the classification accuracy of conventional RGB cameras with natural broadband illumination.

Illumination plays an important role in the image capture process. Too little or too much energy in particular wavelengths can impact the scene appearance in a way that is difficult to manage by color constancy post processing methods. We use an adjustable multispectral flash to modify the spectral illumination of a scene. The flash is composed of a small number of narrowband lights, and the imaging system takes a sequence of images of the scene under each of those lights. Pixel data is used to estimate the spectral power distribution of the ambient light, and to adjust the flash spectrum either to match or to complement the ambient illuminant. The optimized flash spectrum can be used in subsequent captures, or a synthetic image can be computationally rendered from the available data. Under extreme illumination conditions images captured with the matching flash have no color cast, and the complementary flash produces more balanced colors. The proposed system also improves the quality of images captured in underwater environments.

Many creative ideas are being proposed for image sensor designs, and these may be useful in applications ranging from consumer photography to computer vision. To understand and evaluate each new design, we must create a corresponding image processing pipeline that transforms the sensor data into a form, that is appropriate for the application. The need to design and optimize these pipelines is time-consuming and costly. We explain a method that combines machine learning and image systems simulation that automates the pipeline design. The approach is based on a new way of thinking of the image processing pipeline as a large collection of local linear filters. We illustrate how the method has been used to design pipelines for novel sensor architectures in consumer photography applications.

Many multispectral imaging systems are computational in nature and require processing of raw data in order to obtain radiance spectra. In this paper, we derive a fast and scalable spectral estimation algorithm based on the Alternating Direction Method of Multipliers (ADMM). Using this approach we solve for the unknown surface spectral reflectance simultaneously for all pixels in the image. This global formulation allows us to incorporate spatial as well as spectral regularizers, such as total variation penalty or non-negativity. We show that the estimates derived with our solver are more accurate and more robust in the presence of noise.

This paper presents an underwater image formation model incorporating scatter and spectral absorption curves of pure water and three other particle classes; phytoplankton, non-algal particles and colored dissolved organic matter. We also describe an algorithm allowing to derive these depth dependent curves from a sequence of RGB images of a known target. Underwater images generated using the model closely resemble the experimental ones acquired in a variety of conditions. Similarly, the estimated absorption curves, and relative particle concentrations agree with common sense expectations.

The appearance of rendered text is a compromise between the designer's intent and the display capabilities. The ClearType rendering method is designed to enhance rendered text by exploiting the subpixel resolution available on color displays. ClearType represents the high-resolution font outline at the full subpixel resolution of the display and then filters the image to enhance contrast and reduce color artifacts. The filter choice influences text appearance, and people have clear preferences between the renderings with different filters. In this paper, we predict these preferences using S-CIELAB, a spatial extension to the perceptual color metric CIELAB. We calculate the S-CIELAB difference between designed and rendered fonts for various filters. We compare the size of these differences with preference data obtained from individual subjects.

Provides an overview of the technical articles and features presented in this issue.

This paper describes the design and performance of an image capture simulator. The general model underlying the simulator assumes that the image capture device contains multiple classes of sensors with different spectral sensitivities and that each sensor responds in a known way to irradiance over most of its operating range. The input to the simulator is a set of narrow-band images of the scene taken with a custom-designed hyperspectral camera system. The parameters for the simulator are the number of sensor classes, the sensor spectral sensitivities, the noise statistics and number of quantization levels for each sensor class, the spatial arrangement of the sensors and the exposure duration. The output of the simulator is the raw image data that would have been acquired by the simulated image capture device. To test the simulator, we acquired images of the same scene both with the hyperspectral camera and with a calibrated Kodak DCS-200 digital color camera. We used the simulator to predict the DCS-200 output from the hyperspectral data. The agreement between simulated and acquired images validated the image capture response model and our simulator implementation. We believe the simulator will provide a useful tool for understanding the effect of varying the design parameters of an image capture device.

Image fidelity (inferred by the ability to discriminate between two images) and image quality (inferred by the preference for one image over another) are often assumed to be directly related. We investigated the relationship between the perceived image fidelity and image quality of halftone textures. Subjects were asked to rank order a set of printed halftone swatches on the basis of smoothness. They were then asked to reduce the contrast of each pattern until it was at threshold, thus providing an estimate of the pattern's perceptual strength and its discriminability from a non-textured swatch. We found only a moderate correlation between image fidelity and image quality.