Deep Neural Networks (DNN) are now the state-of-the-art for many image and object recognition tasks, as illustrated by their performance on standard benchmarks. The success of DNNs is attributed to their ability to learn rich mid-level image representations, as opposed to hand-designed low-level features used in other image analysis methods. Typically a large dataset of unlabeled images is used for unsupervised feature learning, and then standard classifiers are trained on the features extracted from the images in a labeled set. In this paper, we show that clustering the images using the features from the DNN allows more accurate per-cluster classifiers to be learned, which improves the overall classification accuracy. We demonstrate the effectiveness of our approach on a scene recognition task.

Process models are widely used tools, both for studying fundamental processes themselves and as elements of larger system studies. A radiative transfer model (RTM) simulates the interaction of light with a medium. We are interested in RTMs that model light reflected from a vegetated region. Such an RTM takes as input various biospheric and illumination parameters and computes the upwelling radiation at the top of the canopy. The question we address is as follows: Which of the inputs to the RTM has the greatest impact on the computed observation? We study the leaf canopy model (LCM) RTM, which was designed to study the feasibility of observing leaf chemistry remotely. Its inputs are leaf chemistry variables (chlorophyll, water, lignin, and cellulose) and canopy structural parameters (leaf area index, leaf angle distribution, soil reflectance, and sun angle). We present a statistical approach to the sensitivity analysis of RTMs to answer the question previously posed. The focus is on global sensitivity analysis, studying how the RTM output changes as the inputs vary continuously according to a probability distribution over the input space. The influence of each input variable is captured through the “main effects” and “sensitivity indices.” Direct computation requires extensive computationally expensive runs of the RTM. We develop a Gaussian process approximation to the RTM output to enable efficient computation. We illustrate how the approach can effectively determine the inputs that are vital for accurate prediction. The methods are applied to the LCM with seven inputs and output obtained at eight wavelengths associated with Moderate-resolution Imaging Spectroradiometer bands that are sensitive to vegetation.

This paper presents and compares two methods of tracking the beat in musical performances, one based on a Bayesian decision framework and the other a gradient strategy. The techniques can be applied directly to a digitized performance (i.e., a soundfile) and do not require a musical score or a MIDI transcription. In both cases, the raw audio is first processed into a collection of "rhythm tracks" which represent the time evolution of various low-level features. The Bayesian approach chooses a set of parameters that represent the beat by modeling the rhythm tracks as a concatenation of random variables with a patterned structure of variances. The output of the estimator is a trio of parameters that represent the interval between beats, its change (derivative), and the position of the starting beat. Recursive (and potentially real time) approximations to the method are derived using particle filters, and their behavior is investigated via simulation on a variety of musical sources. The simpler method, which performs a gradient descent over a window of beats, tends to converge more slowly and to undulate about the desired answer. Several examples are presented that highlight both the strengths and weaknesses of the approaches.

Bayesian inference has been used successfully for many problems where the aim is to infer the parameters of a model of interest. In this paper we formulate the three dimensional reconstruction problem as the problem of inferring the parameters of a surface model from image data, and show how Bayesian methods can be used to estimate the parameters of this model given the image data. Thus we recover the three dimensional description of the scene. This approach also gives great flexibility. We can specify the geometrical properties of the model to suit our purpose, and can also use different models for how the surface reflects the light incident upon it. In common with other Bayesian inference problems, the estimation methodology requires that we can simulate the data that would have been recorded for any values of the model parameters. In this application this means that if we have image data we must be able to render the surface model. However it also means that we can infer the parameters of a model whose resolution can be chosen irrespective of the resolution of the images, and may be super-resolved. We present results of the inference of surface models from simulated aerial photographs for the case of super-resolution, where many surface elements project into a single pixel in the low-resolution images.

Recent developments in statistics now allow maximum likelihood estimators for the parameters of Markov random fields (MRFs) to be constructed. We detail the theory required, and present an algorithm that is easily implemented and practical in terms of computation time. We demonstrate this algorithm on three MRF models--the standard Potts model, an inhomogeneous variation of the Potts model, and a long-range interaction model, better adapted to modeling real-world images. We estimate the parameters from a synthetic and a real image, and then resynthesize the models to demonstrate which features of the image have been captured by the model. Segmentations are computed based on the estimated parameters and conclusions drawn.

Developments in Markov chain Monte Carlo procedures have made it possible to perform fully Bayesian image segmentation. By this we mean that all the parameters are treated identically, be they the segmentation labels, the class parameters or the Markov random field prior parameters. We perform the analysis by sampling from the posterior distribution of all the parameters. Sampling from the MRF parameters has traditionally been considered if not intractable then at least computationally prohibitive. In the statistics literature there are descriptions of experiments showing that the MRF parameters may be sampled by approximating the partition function. These experiments are all, however, on 'toy' problems; for the typical size of image encountered in engineering applications the phase transition behaviour of the models becomes a major limiting factor in the estimation of the partition function. Nevertheless, we show that, with some care, fully Bayesian segmentation can be performed on realistic sized images. We also compare the fully Bayesian approach with the approximate pseudolikelihood method.

We address the problem of detecting, and subsequently removing, 'line scratch' distortion in motion picture frames. A model for the lines' interaction with the image data is constructed. A sampling based algorithm based on the reversible jump Markov chain Monte Carlo framework is developed which enables automatic determination of both the unknown number of lines present, together with the lines' parameters. Previous work has not attempted to automatically determine the number of lines present. Our approach is widely applicable in many object recognition problems, where the number of objects is unknown.

An algorithm is presented for determining the subset of the basis functions of a generalized single-layer network (GSLN) needed to solve the classification problem defined by the training data. A Markov chain Monte Carlo sampling technique is used to traverse the space of models having a low sum squared error (SSE). The frequency of a term's inclusion is an indication of its importance to the classifier. Fast, iterative updates can be used for the matrix calculations needed. Theoretical results for the required length of the chain needed to obtain good discrimination between functions fitting the data and those modeling the added noise are given, and these are confirmed by experiment.

Bright and dark flashes are typical artifacts in degraded motion picture material. The distortion is referred to as "dirt and sparkle" in the motion picture industry. This is caused either by dirt becoming attached to the frames of the film, or by the film material being abraded. The visual result is random patches of the frames having grey level values totally unrelated to the initial information at those sites. To restore the film without causing distortion to areas of the frames that are not affected, the locations of the blotches must be identified. Heuristic and model-based methods for the detection of these missing data regions are presented in this paper, and their action on simulated and real sequences is compared.

This paper presents a number of model based interpolation schemes tailored to the problem of interpolating missing regions in image sequences. These missing regions may be of arbitrary size and of random, but known, location. This problem occurs regularly with archived film material. The film is abraded or obscured in patches, giving rise to bright and dark flashes, known as "dirt and sparkle" in the motion picture industry. Both 3-D autoregressive models and 3-D Markov random fields are considered in the formulation of the different reconstruction processes. The models act along motion directions estimated using a multiresolution block matching scheme. It is possible to address this sort of impulsive noise suppression problem with median filters, and comparisons with earlier work using multilevel median filters are performed. These comparisons demonstrate the higher reconstruction fidelity of the new interpolators.

Many archived motion pictures suffer from what we term replacement noise, that is various degradations such as dirt, scratches, fingerprints, etc., where the original picture is replaced by some unrelated information. We use Markov random field based motion field segmentation to detect these areas, and then interpolate into the gaps using a motion-compensated interpolation scheme to restore the frame.>