Efficient hardware implementations routinely approximate mathematical functions with look-up tables, while keeping the error of the approximation under control. For a certain class of commonly occurring 1D functions, namely monotonically increasing or decreasing functions, we found that it is possible to approximate such functions by repeated application of a very low resolution 1D look-up table. There are many advantages to cascading multiple identical LUTs, including the promise of a very simple hardware design and the use of standard linear interpolation. Further, the complexity associated with unequal bin sizes can be avoided. We show that for realistic applications, including gamma correction, high dynamic range encoding and decoding curves, as well as tone mapping and inverse tone mapping applications, multiple cascaded look-up tables can reduce the approximation error by more than 50% compared to a single look-up table with the same total memory footprint.

Abstract The objective of colour mapping or colour transfer methods is to recolour a given image or video by deriving a mapping between that image and another image serving as a reference. These methods have received considerable attention in recent years, both in academic literature and industrial applications. Methods for recolouring images have often appeared under the labels of colour correction, colour transfer or colour balancing, to name a few, but their goal is always the same: mapping the colours of one image to another. In this paper, we present a comprehensive overview of these methods and offer a classification of current solutions depending not only on their algorithmic formulation but also their range of applications. We also provide a new dataset and a novel evaluation technique called ‘evaluation by colour mapping roundtrip’. We discuss the relative merit of each class of techniques through examples and show how colour mapping solutions can have been applied to a diverse range of problems.

We address the problem of compensating color differences in image stitching. We explicitly color correct input images before image stitching by leveraging sparse correspondences. Our approach is twofold. First, for each geometric correspondence, we locally collect and process color information to be robust against low accuracy in geometric correspondences. Second, for all collected colors, we fit a global model that compensates complex color changes. Despite complex cases between input images (changes of illumination condition and imaging devices) qualitative experiments show good stitching of our method compared to recent methods in the literature.

The variability among human observers is a challenge to the calibration of modern displays based on Light Emitting Diodes (LED) and lasers. The spectra of the displayed colors are so peaky that slight differences in the cone sensitivities in human color vision are sufficient to make two observers perceiving different colors on the same screen. Recently published results give evidence to the existence of a small number of classes of human observers. In this paper, we present first results on the development of a prototype for a lightweight, inexpensive, temporally stable and easy to calibrate and easy to use instrument that allows classification of an observer with normal color vision in a small number of categories. The instrument employs a set of LEDs having specific wavelengths and is controlled by a computer interface. The use of such an instrument will allow adapting displays and viewing conditions to individual observers in color-critical applications.

The paper discusses the integration of augmented reality techniques into interactive television (ITV) and presents a new method for ensuring photometric realism. ITV services known today are email and World Wide Web (WWW) access. This article focuses on a future option for interactive television: the integration of games and virtual 3D scenes. For example, while watching football, you may be invited by a joint WWW commercial to have a look at a virtual 3D model of a sporting shoe. Displaying the sporting shoe on the screen in front of the football scene is in fact a case of augmented reality. The paper proposes a fast and simple method to extract a number of light sources from the football video that ensure a dynamic illumination of the virtual object depending on the content of the TV screen. The method works for all kinds of video content since only the light source intensities are estimated from the video signal while the light source positions are set by a fixed rule.

In this paper, a non-complex estimator is developed for the tilt angle of the rotation axis of an object that is illuminated by a point light source. The tilt angle defines the orientation of the 2D projection of the 3D object rotation axis in the image plane and is a strong clue for image understanding. The estimator evaluates two images of a video image sequence showing a moving object. Additionally, only a displacement vector field and the 2D object silhouette are required. No 3D information is required. Therefore, the object is assumed to be rigid, to be matte, to have equally distributed surface normals and to be illuminated by a distant point light source and ambient light. For estimation, the displaced frame ratio (DFR), i.e. the frame ratio after motion compensation, is evaluated statistically. The DFR depends only on the photometric effect of temporally changing object shading. Experimental results with real images show the proper performance of the derived estimator for real objects.

This paper proposes a motion-based video sequence indexing method for scenes with non-uniform illumination, i.e. object shading and cast shadows may be present. The motion feature used for indexing is the 3D object rotation axis projected into the image plane that is defined by a tilt angle. As previous work shows, the tilt angle can be easily estimated from a video sequence evaluating only temporally changing shading on object surfaces-an effect that is often neglected in computer vision. A new indexing method is proposed that cuts the video sequence into temporal segments that are characterized by a tilt angle following the homogeneous motion model of a temporal Kalman filter. Experiments with real video sequences of simple objects like a ball or a persons head show that the video is cut into semantically meaningful segments.

In applications of augmented reality like virtual studio TV production, multisite video conference applications using a virtual meeting room and synthetic/natural hybrid coding according to the new ISO/MPEG-4 standard, a synthetic scene is mixed into a natural scene to generate a synthetic/natural hybrid image sequence. For realism, the illumination in both scenes should be identical. In this paper, the illumination of the natural scene is estimated automatically and applied to the synthetic scene. The natural scenes are restricted to scenes with nonoccluding, simple, moving, mainly rigid objects. For illumination estimation, these natural objects are automatically segmented in the natural image sequence and three-dimensionally (3-D) modeled using ellipsoid-like models. The 3-D shape, 3-D motion, and the displaced frame difference between two succeeding images are evaluated to estimate three illumination parameters. The parameters describe a distant point light source and ambient light. Using the estimated illumination parameters, the synthetic scene is rendered and mixed to the natural image sequence. Experimental results with a moving virtual object mixed into real video telephone sequences show that the virtual object appears naturally having the same shading and shadows as the real objects. Further, shading and shadow allows the viewer to understand the motion trajectory of the objects much better.

To prevent moving shadows being misclassified as moving objects or parts of moving objects, this paper presents an explicit method for detection of moving cast shadows on a dominating scene background. Those shadows are generated by objects moving between a light source and the background. Moving cast shadows cause a frame difference between two succeeding images of a monocular video image sequence. For shadow detection, these frame differences are detected and classified into regions covered and regions uncovered by a moving shadow. The detection and classification assume plane background and a nonnegligible size and intensity of the light sources. A cast shadow is detected by temporal integration of the covered background regions while subtracting the uncovered background regions. The shadow detection method is integrated into an algorithm for two-dimensional (2-D) shape estimation of moving objects from the informative part of the description of the international standard ISO/MPEG-4. The extended segmentation algorithm compensates first apparent camera motion. Then, a spatially adaptive relaxation scheme estimates a change detection mask for two consecutive images. An object mask is derived from the change detection mask by elimination of changes due to background uncovered by moving objects and by elimination of changes due to background covered or uncovered by moving cast shadows. Results obtained with MPEG-4 test sequences and additional sequences show that the accuracy of object segmentation is substantially improved in presence of moving cast shadows. Objects and shadows are detected and tracked separately.

For virtual studio TV production, multi-site videoconference applications using a virtual meeting room and for synthetic/natural hybrid coding according to the forthcoming ISO/MPEG-4 standard, a synthetic and a natural scene are mixed to generate a synthetic/natural hybrid image sequence. For realism, the illumination in both scenes should be identical. The illumination of the natural scene is estimated automatically and applied to the synthetic scene. For illumination estimation, moving real objects are segmented in the natural image sequence and 3D modeled. The 3D shape, 3D motion and the displaced frame difference between two succeeding images are evaluated to estimate three illumination parameters. The parameters describe a distant point light source and ambient light. Using the estimated illumination parameters, the synthetic scene is rendered and mixed to the natural image sequence. Experimental results with a moving virtual ball mixed automatically into a real video telephone sequence show that virtual objects appear naturally having the same shading and shadows as the real person.