Reflectance Confocal Microscopy (RCM) is used for evaluation of human skin disorders and the effects of skin treatments by imaging the skin layers at different depths. Traditionally, clinical experts manually categorize the images captured into different skin layers. This time-consuming labeling task impedes the convenient analysis of skin image datasets. In recent automated image recognition tasks, deep learning with convolutional neural nets (CNN) has achieved remarkable results. However in many clinical settings, training data is often limited and insufficient for CNN training. For recognition of RCM skin images, we demonstrate that a CNN trained on a moderate size dataset leads to low accuracy. We introduce a hybrid deep learning approach which uses traditional texton-based feature vectors as input to train a deep neural network. This hybrid method uses fixed filters in the input layer instead of tuned filters, yet superior performance is achieved. Our dataset consists of 1500 images from 15 RCM stacks belonging to six different categories of skin layers. We show that our hybrid deep learning approach performs with a test accuracy of 82% compared with 51% for CNN. We also compare the results with additional proposed methods for RCM image recognition and show improved accuracy.

Polarization has been used in numerous prior studies for separating diffuse and specular reflectance components, but in this work we show that it also can be used to separate surface reflectance contributions from individual light sources. Our approach is called polarization multiplexing and it has a significant impact in appearance modeling where the image as a function of illumination direction is needed. Multiple unknown light sources can illuminate the scene simultaneously, and the individual contributions to the overall surface reflectance are estimated. Polarization multiplexing relies on the relationship between the light source direction and the intensity modulation. Inverting this transformation enables the individual intensity contributions to be estimated. In addition to polarization multiplexing, we show that phase histograms from the intensity modulations can be used to estimate scene properties including the number of light sources.

Our goal is to incorporate polarization in appearance-based modeling in an efficient and meaningful way. Polarization has been used in numerous prior studies for separating diffuse and specular reflectance components, but in this work we show that it also can be used to separate surface reflectance contributions from individual light sources. Our approach is called polarization multiplexing and it has significant impact in appearance modeling and bidirectional imaging where the image as a function of illumination direction is needed. Multiple unknown light sources can illuminate the scene simultaneously, and the individual contributions to the overall surface reflectance can be estimated. To develop the method of polarization multiplexing, we use a relationship between light source direction and intensity modulation. Inverting this transformation enables the individual intensity contributions to be estimated. In addition to polarization multiplexing, we show that phase histograms from the intensity modulations can be used to estimate scene properties including the number of light sources.

A bidirectional texture function (BTF) describes image texture as it varies with viewing and illumination direction. Many real world surfaces such as skin, fur, gravel, etc. exhibit fine-scale geometric surface detail. Accordingly, variations in appearance with viewing and illumination direction may be quite complex due to local foreshortening, masking and shadowing. Representations of surface texture that support robust recognition must account for these effects. We construct a representation which captures the underlying statistical distribution of features in the image texture as well as the variations in this distribution with viewing and illumination direction. The representation combines clustering to learn characteristic image features and principle components analysis to reduce the space of feature histograms. This representation is based on a core image set as determined by a quantitative evaluation of importance of individual images in the overall representation. The result is a compact representation and a recognition method where a single novel image of unknown viewing and illumination direction can be classified efficiently. The CUReT (Columbia-Utrecht reflectance and texture) database is used as a test set for evaluation of these methods.