Local feature descriptors in combination with bag of (visual) words have recently become the state of the art in writer identification. In this work we propose the use of Zernike moments evaluated at the contours of the script as local descriptor. We then form a global descriptor by encoding the extracted Zernike moments into Vectors of Locally Aggregated Descriptors (VLAD). This local / global descriptor combination outperforms existing methods: on the ICDAR 2013 benchmark database our Zernike / VLAD method yields 0.880 mAP, a 31% improvement over the 0.671 mAP of the state of the art. We also set a new performance standard on the CVL dataset.

In this paper we propose a method for learning the materials in a scene in an unsupervised manner making use of imaging spectroscopy data. Here, we view the input image spectra as a data point on a manifold which corresponds to a node in a graph whose vertices correspond to a set of parameters that should be inferred using the Expectation Maximisation (EM) algorithm. In this manner, we can pose the problem as a statistical unsupervised learning one where the aim of computation becomes the recovery of the set of parameters that allow for the image spectra to be projected onto a set of graph vertices defined a priori. Moreover, as a result of this treatment, the scene material prototypes can be recovered making use of a clustering algorithm applied to the parameter-set. This setting also allows, in a straightforward manner, for the visualisation of the spectra. We discuss the links between our method and self-organizing maps and illustrate the utility of the method as compared to other alternatives elsewhere in the literature.

This paper proposes a new system for offline writer identification and writer verification. The proposed method uses GMM supervectors to encode the feature distribution of individual writers. Each supervector originates from an individual GMM which has been adapted from a background model via a maximum-a-posteriori step followed by mixing the new statistics with the background model. We show that this approach improves the TOP-1 accuracy of the current best ranked methods evaluated at the ICDAR-2013 competition dataset from 95.1% [13] to 97.1%, and from 97.9% [11] to 99.2% at the CVL dataset, respectively. Additionally, we compare the GMM supervector encoding with other encoding schemes, namely Fisher vectors and Vectors of Locally Aggregated Descriptors.

Most existing color constancy algorithms assume uniform illumination. However, in real-world scenes, this is not often the case. Thus, we propose a novel framework for estimating the colors of multiple illuminants and their spatial distribution in the scene. We formulate this problem as an energy minimization task within a conditional random field over a set of local illuminant estimates. In order to quantitatively evaluate the proposed method, we created a novel data set of two-dominant-illuminant images comprised of laboratory, indoor, and outdoor scenes. Unlike prior work, our database includes accurate pixel-wise ground truth illuminant information. The performance of our method is evaluated on multiple data sets. Experimental results show that our framework clearly outperforms single illuminant estimators as well as a recently proposed multi-illuminant estimation approach.

Mean shift clustering and its recent variants are a viable and popular image segmentation tool. In this paper we investigate mean shift segmentation on multispectral and hyperspectral images and propose three new algorithms. First, we improve segmentation performance by running mean shift on the spectral gradient. At the same time, we adapt a popular superpixel segmentation method to the multispectral domain using modified similarity measures from spectral mapping. Based on superpixels, we design two mean shift variants that both obtain competitive segmentation results in significantly reduced running time. For one variant, the speedup in our benchmark is over 100 times. This enables mean shift clustering in an interactive setting.

Automatic assessment of retinal vessels plays an important role in the diagnosis of various eye, as well as systemic diseases. A public screening is highly desirable for prompt and effective treatment, since such diseases need to be diagnosed at an early stage. Automated and accurate segmentation of the retinal blood vessel tree is one of the challenging tasks in the computer‐aided analysis of fundus images today. We improve the concept of matched filtering, and propose a novel and accurate method for segmenting retinal vessels. Our goal is to be able to segment blood vessels with varying vessel diameters in high‐resolution colour fundus images. All recent authors compare their vessel segmentation results to each other using only low‐resolution retinal image databases. Consequently, we provide a new publicly available high‐resolution fundus image database of healthy and pathological retinas. Our performance evaluation shows that the proposed blood vessel segmentation approach is at least comparable with recent state‐of‐the‐art methods. It outperforms most of them with an accuracy of 95% evaluated on the new database.

For decades, photographs have been used to document space-time events and they have often served as evidence in courts. Although photographers are able to create composites of analog pictures, this process is very time consuming and requires expert knowledge. Today, however, powerful digital image editing software makes image modifications straightforward. This undermines our trust in photographs and, in particular, questions pictures as evidence for real-world events. In this paper, we analyze one of the most common forms of photographic manipulation, known as image composition or splicing. We propose a forgery detection method that exploits subtle inconsistencies in the color of the illumination of images. Our approach is machine-learning-based and requires minimal user interaction. The technique is applicable to images containing two or more people and requires no expert interaction for the tampering decision. To achieve this, we incorporate information from physics- and statistical-based illuminant estimators on image regions of similar material. From these illuminant estimates, we extract texture- and edge-based features which are then provided to a machine-learning approach for automatic decision-making. The classification performance using an SVM meta-fusion classifier is promising. It yields detection rates of 86% on a new benchmark dataset consisting of 200 images, and 83% on 50 images that were collected from the Internet.

The log-chromaticity space (LCS) is a color space with excellent illumination-invariant properties. When converting RGB colors to LCS, there exist four different options for choosing the normalizing channel. For classification applications, we analyze the impact of the normalizer on the distribution of colors in LCS. Based on synthetic and real image data we show that the geometric mean does not introduce a bias to the color clusters and always results in an intermediate clustering performance. However, data-specific selection of the normalizing channel can further improve the results. For instance, for skin classification we show that using the blue channel as denominator results in a recognition improvement of about 25.9% compared to the red channel (worst result).In comparison to the geometric mean and the green channel, the two most popular denominators, the performance increase is 12.9% and 2.9%.

In recent years, graph-based methods have had a significant impact on image segmentation. They are especially noteworthy for supervised segmentation, where the user provides task-specific foreground and background seeds. We adapt the power watershed framework to multispectral and hyperspectral image data and incorporate similarity measures from the field of spectral matching. We also propose a new data-driven graph edge weighting. Our weights are computed by the topological information of a self-organizing map. We show that graph weights based on a simple Lp-norm, as used in other modalities, do not give satisfactory segmentation results for multispectral data, while similarity measures that were specifically designed for this domain perform better. Our new approach is competitive and has an advantage in some of the tested scenarios.

A copy-move forgery is created by copying and pasting content within the same image, and potentially postprocessing it. In recent years, the detection of copy-move forgeries has become one of the most actively researched topics in blind image forensics. A considerable number of different algorithms have been proposed focusing on different types of postprocessed copies. In this paper, we aim to answer which copy-move forgery detection algorithms and processing steps (e.g., matching, filtering, outlier detection, affine transformation estimation) perform best in various postprocessing scenarios. The focus of our analysis is to evaluate the performance of previously proposed feature sets. We achieve this by casting existing algorithms in a common pipeline. In this paper, we examined the 15 most prominent feature sets. We analyzed the detection performance on a per-image basis and on a per-pixel basis. We created a challenging real-world copy-move dataset, and a software framework for systematic image manipulation. Experiments show, that the keypoint-based features Sift and Surf, as well as the block-based DCT, DWT, KPCA, PCA, and Zernike features perform very well. These feature sets exhibit the best robustness against various noise sources and downsampling, while reliably identifying the copied regions.

We propose a new method for performing edge detection in multi-spectral images based on the self-organizing map (SOM) concept. Previously, 1-dimensional or 2-dimensional SOMs were trained to provide a linear mapping of high-dimensional multispectral vectors. Then, edge detection was applied on that mapping. However, the 1-dimensional SOM may not converge on a suitable global order for images with rich content. Likewise, the 2-dimensional SOM introduces false edges due to linearization artifacts. Our method feeds the edge detector without linearization. Instead, it exploits directly the distances of SOM neurons. This avoids the aforementioned drawbacks and is more general, as a SOM of arbitrary dimensionality can be used. We show that our method achieves significantly better edge detection results than previous work on a high-resolution multispectral image database.

The neutral interface reflection (NIR) assumption is a widely accepted theory in computer vision. According to the NIR the color of specularities of dielectric materials is the color of the incident illumination and the influence of the Fresnel reflectance is neglected. We show, that there is a material- and geometry-dependent shift between the color of the specularity and the color of the incident light due to the Fresnel effect which for human skin can be up to approximately 5.8%. As the NIR concept is often the core idea of specularity-based illuminant-color estimation techniques, the ignored Fresnel effect introduces a systematic error in the estimation result. We thus propose a material-dependent rectification method for correcting this color shift. Our experiments on human skin regions show an average improvement of the illuminant color estimation of about 30%.

Most algorithms for extracting illuminant chromaticity from arbitrary images, such as the images found on the web, are based on machine learning techniques. We will show how a physics-based methodology can be adapted to provide relative illumination information on real images. More specifically, we use the inverse-intensity chromaticity representation and show how the analysis of the histograms of illumination-chromaticity candidates provides information about the type of illumination(s) present in a scene. Experiments indicate that the estimate is quite robust towards noise, and that simple measurements on the histogram peak can be used to counter-check the reliability of the estimate.

In visual 3-D reconstruction tasks with mobile cameras, one wishes to move the cameras so that they provide the views that lead to the best reconstruction result. When the camera motion is adapted during the reconstruction, the view of interest is the next best view for the current shape estimate. We present such a next best view planning approach for visual 3-D reconstruction. The reconstruction is based on a probabilistic state estimation with sensor actions. The next best view is determined by a metric of the state estimation's uncertainty. We compare three metrics: D-optimality, which is based on the entropy and corresponds to the (D)eterminant of the covariance matrix of a Gaussian distribution, E-optimality, and T-optimality, which are based on (E)igenvalues or on the (T)race of this matrix, respectively. We show the validity of our approach with a simulation as well as real-world experiments, and compare reconstruction accuracy and computation time for the optimality criteria.

Reliable specularity detection can affect the accuracy of further image analysis. The majority of specularity detection algorithms are based on the chromaticity of the regions of specular highlights. They assume that the color of specular highlights of dielectrics is approximately the color of the incident light. We will show how this assumption is inaccurate especially in multispectral images. We propose a new, physics-based specularity detection method, which depends on the Fresnel term of the specular highlight, instead of assumptions on chromaticity space. We compute at each pixel an approximation to the Fresnel term at various wavelengths. We then use mean-shift analysis to segment the image based on the Fresnel information. Our experiments with multispectral, as well as traditional RGB images, show improved specularity detection and higher robustness in chromaticity space noise.

The automated detection of humans in computer vision as well as the realistic rendering of people in computer graphics necessitates improved modeling of the human skin color. We describe the acquisition and modeling of skin reflectance data densely sampled over the entire visible spectrum. The data collected through a spectrograph allows us to explain skin color (and its variations) and to discriminate between human skin and dyes designed to mimic human skin. We study the approximation of these data using several sets of basis functions. Our study shows that skin reflectance data can best be approximated by a linear combination of Gaussians or their first derivatives. This result has a significant practical impact on optical acquisition devices: the entire visible spectrum of skin reflectance can now be captured with a few filters of optimally chosen central wavelengths and bandwidth.

The light reflected from a surface depends on the scene geometry, the incident illumination and the surface material. A novel methodology is presented which extracts reflectivity information of the various materials in the scene independent of incident light and scene geometry. A scene is captured under different narrow-band color filters and the spectral derivatives of the scene are computed. The resulting spectral derivatives form a spectral gradient at each pixel. This spectral gradient is a material descriptor which is invariant to scene geometry and incident illumination for smooth diffuse surfaces. Spectral gradients can discriminate among smooth dielectrics with different reflectance properties independent of viewing conditions.

Methods for the analysis of images of the same scene taken under three different lighting conditions are illustrated. A technique that separates the effects of geometry and surface coloration/texture in this "tri-luminal" environment is developed and experimental results are shown. Exploiting this technique to isolate geometric information, two methods which extract differential geometric properties of surfaces (the sign of Gaussian curvature and its magnitude to within a multiplicative factor) directly from tri-luminal photometric data are derived and demonstrated.

We compute the sign of Gaussian curvature using a purely geometric definition. Consider a point p on a smooth surface S and a closed curve /spl gamma/ on S which encloses p. The image of /spl gamma/ on the unit normal Gaussian sphere is a new curve /spl beta/. The Gaussian curvature at p is defined as the ratio of the area enclosed by /spl gamma/ over the area enclosed by /spl beta/ as /spl gamma/ contracts to p. The sign of Gaussian curvature at p is determined by the relative orientations of the closed curves /spl gamma/ and /spl beta/. We directly compute the relative orientation of two such curves from intensity data. We employ three unknown illumination conditions to create a photometric scatter plot. This plot is in one-to-one correspondence with the subset of the unit Gaussian sphere containing the mutually illuminated surface normal. This permits direct computation of the sign of Gaussian curvature without the recovery of surface normals. Our method is albedo invariant. We assume diffuse reflectance, but the nature of the diffuse reflectance can be general and unknown. Error analysis on simulated images shows the accuracy of our technique. We also demonstrate the performance of this methodology on empirical data.

The authors compute the sign of Gaussian curvature using a purely geometric definition. Consider a point p on a smooth surface S and a closed curve /spl gamma/ on S which encloses p. The image of /spl gamma/ on the unit normal Gaussian sphere is a new curve /spl beta/. The Gaussian curvature at p is defined as the ratio of the area enclosed by /spl gamma/ over the area enclosed by /spl beta/ as /spl gamma/ contracts to p. The sign of Gaussian curvature at p is determined by the relative orientations of the closed curves /spl gamma/ and /spl beta/. They directly compare the relative orientation of two such curves from intensity data. They employ three unknown illumination conditions to create a photometric scatter plot. This plot is in one-to-one correspondence with the subset of the unit Gaussian sphere containing the mutually illuminated surface normals. This permits direct computation of the sign of Gaussian curvature without the recovery of surface normals. Their method is albedo invariant. They assume diffuse reflectance, but the nature of the diffuse reflectance can be general and unknown. Simulations, as well as empirical results, demonstrate the accuracy of the technique.

Gaussian curvature is an invariant local descriptor of smooth surfaces. We present an object signature which is a condensed representation of the distribution of Gaussian curvature information at visible object points. An invariant related to Gaussian curvature at a point is derived from the covariance matrix of the photometric values in a neighborhood about that point. In addition, we introduce an albedo-normalization method that is capable of cancelling albedo on Lambertian surfaces. We use three illumination conditions, two of which are unknown. The three-tuple of intensity values at a point is related via a one-to-one mapping to the surface normal at that point. The determinant of the covariance matrix of the local three-tuples is invariant to albedo, rotation and translation. The collection of determinants over mutually illuminated object points is combined into a signature distribution which is albedo, rotation, translation, and scale invariant. An object recognition methodology using these signatures is proposed.