Distinguishing epistemic from aleatoric uncertainty is a central idea to out-of-distribution (OOD) detection. By interpreting adversarial and OOD inputs from this perspective, we can collect them into a single unclassifiable group. Rejecting such inputs mid-inference will reduce resource usage. To achieve this, we apply k-nearest neighbour (KNN) classifiers to the embedding space of branched neural networks. This introduces a novel means of additional power savings, through an early-exit reject. Our technique works out-of-the-box on any branched neural network and can be competitive on OOD benchmarks, achieving an area under receiver operator characteristic (AUROC) of over 0.9 in most datasets, and scores of 0.95+ when identifying perturbed inputs. A mixed input test set is introduced, we show how OOD inputs can be identified up to 50% of the time, and adversarial inputs up to 85% of the time. In a balanced test environment, this equates to power savings of up to 18% in the OOD scenario and 40% in the adversarial scenario. This allows a more stringent in-distribution (ID) classification policy, leading to accuracy improvements of 15% and 20% on the OOD and adversarial tests, respectively, when compared to conventional exit policies operating under the same conditions.

Deep learning is becoming more widespread due to its power in solving complex classification problems. However, deep learning models often require large memory and energy consumption, which may prevent them from being deployed effectively on embedded platforms, limiting their application. This work addresses the problem of memory requirements by proposing a regularization approach to compress the memory footprint of the models. It is shown that the sparsity-inducing regularization problem can be solved effectively using an enhanced stochastic variance reduced gradient optimization approach. Experimental evaluation of our approach shows that it can reduce the memory requirements both in the convolutional and fully connected layers by up to 300X without affecting overall test accuracy.

In this paper, we present a novel coarse-grained technique for monitoring online the bias temperature instability (BTI) aging of circuits by exploiting their power gating infrastructure. The proposed technique relies on monitoring the discharge time of the virtual-power-network during standby operations, the value of which depends on the threshold voltage of the CMOS devices in a power-gated design (PGD). It does not require any distributed sensors, because the virtual-power-network is already distributed in a PGD. It consists of a hardware block for measuring the discharge time concurrently with normal standby operations and a processing block for estimating the BTI aging status of the PGD according to collected measurements. Through SPICE simulation, we demonstrate that the BTI aging estimation error of the proposed technique is less than 1% and 6.2% for PGDs with static operating frequency and dynamic voltage and frequency scaling, respectively. Its area cost is also found negligible. The power gating minimum idle time (MIT) cost induced by the energy consumed for monitoring the discharge time is evaluated on two scalar machine models using either x86 or ARM instruction sets. It is found less than 1.3× and 1.45× the original power gating MIT, respectively. We validate the proposed technique through accelerated aging experiments conducted with five actual chips that contain an ARM cortex M0 processor, manufactured with a 65 nm CMOS technology.

Biology demonstrates powerful information processing capabilities. Of particular interest are enzymes, which process information in highly complex dynamic environments. Exploring the information processing characteristics of an enzyme by selectively altering its environment may lead to the discovery of new modes of computation. The physical experiments required to perform such exploration are combinatorial in nature. Thus resource consumption, both time and money, poses major limiting factors on any exploratory work. New tools are required to mitigate these factors. One such tool is lab-on-chip based autonomous experimentation system, where a microfluidic experimentation platform is driven by machine learning algorithms. The lab-on-chip approach provides an automated platform that can perform complex protocols, which is also capable of reducing the resource cost of experimentation. The machine learning algorithms provide intelligent experiment selection that reduces the number of experiments required for discovery. Here we discuss development of the experimentation platform and machine learning software that will lead to fully autonomous characterisation of enzymes.

Active learning and experimental data acquisition address the same problems, understanding a system under investigation with as few resources as possible. However there are few instances where the theoretically principled techniques in active learning or sequential experimental design have been applied to managing data acquisition in physical experimentation. Partly this is due to fundamental differences between the problems investigated within active learning and the issues faced in much physical experimentation. From a previous study we conducted into autonomous experimentation, where we developed a system capable of automatically designing experiments and proposing potential hypotheses, we aim to investigate and highlight the differences between theoretical active learning and the requirements of experimentalists. We also propose an update of the multi-armed bandit problem that provides a theoretical problem more closely aligned to that found in physical experimentation. We believe that for active learning techniques to be used more widely as tools within physical experimentation, a greater focus of research has to be placed on theoretical problems that have assumptions more closely aligned to those found commonly within physical experimentation. Assumptions such as extremely limited resources, more so than typically considered in active learning problems, along with erroneous observations or noisy oracles, should become standard features of active learning problems, as in experimentation there are rarely enough resources available to be certain about the validity of the data obtained and the quality of the hypotheses produced.

A functional for unsupervised texture segmentation is investigated in this paper. An auto-normal model based on Markov Random Fields is employed here to represent textures. The functional investigated here is optimized with respect to the auto-normal model parameters and the evolving contour to simultaneously estimate auto-normal model parameters and find the evolving contour. Experimental results applied on the textures of the Brodatz album demonstrate the higher speed of convergence of this algorithm in comparison with a traditional stochastic algorithm in the literature.

The presence of irrelevant features in training data is a significant obstacle for many machine learning tasks. One approach to this problem is to extract appropriate features and, often, one selects a feature extraction method based on the inference algorithm. Here, we formalize a general framework for feature extraction, based on Partial Least Squares, in which one can select a user-defined criterion to compute projection directions. The framework draws together a number of existing results and provides additional insights into several popular feature extraction methods. Two new sparse kernel feature extraction methods are derived under the framework, called Sparse Maximal Alignment (SMA) and Sparse Maximal Covariance (SMC), respectively. Key advantages of these approaches include simple implementation and a training time which scales linearly in the number of examples. Furthermore, one can project a new test example using only k kernel evaluations, where k is the output dimensionality. Computational results on several real-world data sets show that SMA and SMC extract features which are as predictive as those found using other popular feature extraction methods. Additionally, on large text retrieval and face detection data sets, they produce features which match the performance of the original ones in conjunction with a Support Vector Machine.

Previous research applying kernel methods such as support vector machines (SVMs) to hyperspectral image classification has achieved performance competitive with the best available algorithms. However, few efforts have been made to extend SVMs to cover the specific requirements of hyperspectral image classification, for example, by building tailor-made kernels. Observation of real-life spectral imagery from the AVIRIS hyperspectral sensor shows that the useful information for classification is not equally distributed across bands, which provides potential to enhance the SVM's performance through exploring different kernel functions. Spectrally weighted kernels are, therefore, proposed, and a set of particular weights is chosen by either optimizing an estimate of generalization error or evaluating each band's utility level. To assess the effectiveness of the proposed method, experiments are carried out on the publicly available 92AV3C dataset collected from the 220-dimensional AVIRIS hyperspectral sensor. Results indicate that the method is generally effective in improving performance: spectral weighting based on learning weights by gradient descent is found to be slightly better than an alternative method based on estimating "relevance" between band information and ground truth.

Spectral band selection is a fundamental problem in hyperspectral data processing. In this letter, a new band-selection method based on mutual information (MI) is proposed. MI measures the statistical dependence between two random variables and can therefore be used to evaluate the relative utility of each band to classification. A new strategy is described to estimate the MI using a priori knowledge of the scene, reducing reliance on a "ground truth" reference map, by retaining bands with high associated MI values (subject to the so-called "complementary" conditions). Simulations of classification performance on 16 classes of vegetation from the AVIRIS 92AV3C data set show the effectiveness of the method, which outperforms an MI-based method using the associated reference map, an entropy-based method, and a correlation-based method. It is also competitive with the steepest ascent algorithm at much lower computational cost

The NIPS 2003 workshops included a feature selection competi- tion organized by the authors. We provided participants with five datasets from different application domains and called for classifica- tion results using a minimal number of features. The competition took place over a period of 13 weeks and attracted 78 research groups. Participants were asked to make on-line submissions on the validation and test sets, with performance on the validation set being presented immediately to the participant and performance on the test set presented to the participants at the workshop. In total 1863 entries were made on the validation sets during the development period and 135 entries on all test sets for the final competition. The winners used a combination of Bayesian neu- ral networks with ARD priors and Dirichlet diffusion trees. Other top entries used a variety of methods for feature selection, which combined filters and/or wrapper or embedded methods using Ran- dom Forests, kernel methods, or neural networks as a classification engine. The results of the benchmark (including the predictions made by the participants and the features they selected) and the scoring software are publicly available. The benchmark is available at www.nipsfsc.ecs.soton.ac.uk for post-challenge submissions to stimulate further research. 1 Introduction Recently, the quality of research in Machine Learning has been raised by the sus- tained data sharing efforts of the community. Data repositories include the well known UCI Machine Learning repository [13], and dozens of other sites [10]. Yet, this has not diminished the importance of organized competitions. In fact, the proliferation of datasets combined with the creativity of researchers in designing experiments makes it hardly possible to compare one paper with another [12]. A number of large conferences have regularly organized competitions (e.g. KDD, CAMDA, ICDAR, TREC, ICPR, and CASP). The NIPS workshops offer an ideal forum for organizing such competitions. In 2003, we organized a competition on the theme of feature selection, the results of which were presented at a workshop on feature extraction, which attracted 98 participants. We are presently preparing a book combining tutorial chapters and papers from the proceedings of that work- shop [9]. In this paper, we present to the NIPS community a concise summary of our challenge design and the findings of the result analysis. 2 Benchmark design We formatted five datasets (Table 1) from various application domains. All datasets are two-class classification problems. The data were split into three subsets: a training set, a validation set, and a test set. All three subsets were made available at the beginning of the benchmark, on September 8, 2003. The class labels for the validation set and the test set were withheld. The identity of the datasets and of the features (some of which were random features artificially generated) were kept secret. The participants could submit prediction results on the validation set and get their performance results and ranking on-line for a period of 12 weeks. By December 1st, 2003, which marked the end of the development period, the participants had to turn in their results on the test set. Immediately after that, the validation set labels were revealed. On December 8th, 2003, the participants could make submissions of test set predictions, after having trained on both the training and the validation set. Some details on the benchmark design are provided in this Section.

Foreground/background segmentation is an active research area for moving object analysis. We combine two probabilistic approaches one of which estimates foreground/background probabilistic density and the other uses prior knowledge to decompose the colour space. The observed performance advantages are associated with the fusion of operators with completely different basis. Tests on outdoor and indoor sequences confirm the efficacy of this approach. The new algorithms can successfully identify and remove shadows and highlights with improved moving-object segmentation. A particular advantage of our evaluation is that it is the first approach that compares foreground/background labelling with results obtained from labelling by broadcast techniques. 1

This paper proposes active handwriting models, in which kernel principal component analysis is applied to capture nonlinear handwriting variations. In the recognition phase, the chamfer distance transform and a dynamic tunneling algorithm (DTA) are employed to search for the optimal shape parameters. The proposed methodology is successfully applied to a novel radical decomposition approach to the challenging problem of handwritten Chinese character recognition.

Handwritten Chinese characters can be recognized by first extracting the basic shapes (radicals) of which they are composed. Radicals are described by nonlinear active shape models and optimal parameters found using the chamfer distance transform and a dynamic tunneling algorithm. The radical recognition rate is 96.5 percent correct (writer-independent) on 280,000 characters containing 98 radical classes.

Offline handwritten Chinese character recognition is a very hard pattern-recognition problem of considerable practical importance. Two popular approaches are to extract features holistically from the character image or to decompose characters structurally into component parts---usually strokes. Here we take a novel approach, that of decomposing into radicals on the basis of image information (i.e., without first decomposing into strokes). During training, 60 examples of each radical were represented by "landmark" points, labeled semiautomatically, with radicals in different characteristic positions treated as distinctly different radicals. Kernel principal-component analysis then captured the main (nonlinear) variations around the mean radical. During the recognition, the dynamic tunneling algorithm was used to search for optimal shape parameters in terms of chamfer distance minimization. Considering character composition as a Markov process in which up to four radicals are combined in some assumed sequential order, we can recognize complete, hierarchically-composed characters by using the Viterbi algorithm. This gave a character recognition rate of 93.5% characters correct (writer-independent) on a test set of 430,800 characters from 2,154 character classes composed of 200 radical categories, which is comparable to the best reported results in the literature. Although the initial semiautomatic landmark labeling is time consuming, the decomposition approach is theoretically well-motivated and allows the different sources of variability in Chinese handwriting to be handled separately and by the most appropriate means--either learned from example data or incorporated as prior knowledge. Hence, high generalizability is obtained from small amounts of training data, and only simple prior knowledge needs to be incorporated, thus promising robust recognition performance. As such, there is very considerable potential for further development and improvement in the direction of larger character sets and less constrained writing conditions.

Prospective authors are requested to submit new, unpublished manuscripts for inclusion in the upcoming event described in this call for papers.

This paper applies active handwriting models (AHM) to handwritten Chinese character recognition. Exploiting active shape models (ASM), the AHM can capture the handwriting variation from character skeletons. The AHM has the following characteristics: principal component analysis is applied to capture variations caused by handwriting, an energy functional on the basis of chamfer distance transform is introduced as a criterion to fit the model to a target character skeleton, and the dynamic tunneling algorithm (DTA) is incorporated with gradient descent to search for shape parameters. The AHM is used within a radical approach to handwritten Chinese characters recognition, which converts the complex pattern recognition problem to recognizing a small set of primitive structures-radicals. Our initial experiments are conducted on 98 radicals covering 1400 loosely-constrained Chinese character categories written by 200 different writers. The correct matching rate is 94.2% on these 2.8/spl times/10/sup 5/ characters. Comparison with existing radical approaches shows that our method achieves superior performance.

Handwritten Chinese character recognition is one of the most difficult problems of pattern recognition. Since the majority of Chinese characters are made up from just a small set of primitive structures (radicals), this paper describes an approach to active radical modeling for such handwritten characters. The most significant characteristic of our method is that radicals can be found robustly without stroke extraction, and the principal variations of the radical can be encoded in a small number of parameters. In the training phase, the example radicals are represented by manually-labeled 'landmark' points. Then a small number of principal components of the eigenvectors are calculated to capture the main variation of the training examples from the mean radical. In the matching phase, each radical model is fitted to the image evidence by adjusting the shape parameters in terms of chamfer distance minimization. Initial experiments are conducted on 1,100 loosely-constrained Chinese character categories written by 200 different writers. The correct matching rate is 95.8%, showing that our radical modeling is effective and capable of forming a sound basis for handwritten Chinese character recognition.

There has been an increasing interest in kernel-based techniques, such as support vector techniques, regularization networks, and gaussian processes. There are inner relationships among those techniques, with the kernel function playing a central role. This article discusses a new class of kernel functions derived from the so-called frames in a function Hilbert space.

The relevance vector machine (RVM) technique is applied to communication channel equalization. It is demonstrated that the RVM equalizer can closely match the optimal performance of the Bayesian equalizer, with a much sparser kernel representation than that is achievable by the state-of-art support vector machine (SVM) technique.

Mixture modeling is becoming an increasingly important tool in the remote sensing community as researchers attempt to resolve subpixel, area information. This paper compares a well-established technique, linear spectral mixture models (LSMM), with a much newer idea based on data selection, support vector machines (SVM). It is shown that the constrained least squares LSMM is equivalent to the linear SVM, which relies on proving that the LSMM algorithm possesses the "maximum margin" property. This in turn shows that the LSMM algorithm can be derived from the same optimality conditions as the linear SVM, which provides important insights about the role of the bias term and rank deficiency in the pure pixel matrix within the LSMM algorithm. It also highlights one of the main advantages for using the linear SVM algorithm in that it performs automatic "pure pixel" selection from a much larger database. In addition, extensions to the basic SVM algorithm allow the technique to be applied to data sets that exhibit spectral confusion (overlapping sets of pure pixels) and to data sets that have nonlinear mixture regions. Several illustrative examples, based on an area-labeled Landsat dataset, are used to demonstrate the potential of this approach.

Previous work has shown that connectionist learning systems can simulate important aspects of the categorization of speech sounds by human and animal listeners.Training is on representations of synthetic, exemplar voiced and unvoiced stop consonants passed through a computational model of the auditory periphery.In this work, we use the modern inductive inference technique of support vector machines (SVMs) as the learning system.Visualization of the SVM's weight vector reveals what has been learned about the voiced/unvoiced distinction.

This paper addresses the problem of determining suitable mask values and sizes for the discrete implementation of the Laplacian of Gaussian edge operator. This is achieved by providing a theoretical connection between the mask size and the edge detection error introduced by truncation of the operator. Additionally, the importance of introducing a bias correction within the finite approximation is developed. Experimental results are shown to confirm the theoretical predictions.

Magnetic Resonance Imaging(MRI) has been used to measure the shape of the vocal tract during speech in several recent studies. Its safety to the subject, high quality imaging of soft tissue, and the ability to select relatively thin imaging planes at any angle are significant advantages over other imaging methods used for speech research. The most significant disadvantage is the long exposure time. As a result most studies have focused on obtaining high-resolution images of the vocal tract volume for static sounds, such as vowels[1], fricatives[5,6], nasals, the closed phase of plosives[7] and liquids[3,7]. In this paper we will describe our method of obtaining MR images of a moving vocal tract in which we post-synchronize the MR data using a recorded speech signal and thus reconstruct the images without using the MR machine's built-in processing.

A conventional active contour formulation suffers difficulty in appropriate choice of an initial contour and values of parameters. Recent approaches have aimed to resolve these problems but can compromise other performance aspects. To relieve the problem in initialization, we use a dual active contour, which is combined with a local shape model to improve the parameterization. One contour expands from inside the target feature, the other contracts from the outside. The two contours are interlinked to provide a balanced technique with an ability to reject "weak" local energy minima.

Snakes are now a very popular technique for shape extraction by minimising a suitably formulated energy functional. A dual snake configuration using dynamic programming has been developed to locate a global energy minimum. This complements recent approaches to global energy minimisation via simulated annealing and genetic algorithms. These differ from a conventional evolutionary snake approach, where an energy function is minimised according to a local optimisation strategy and may not converge to extract the target shape, in contrast with the guaranteed convergence of a global approach. The new technique employing dynamic programming is deployed to extract the inner face boundary, along with a conventional normal-driven technique to extract the outer face boundary. Application to a database of 75 subjects showed that the outer contour was extracted successfully for 96% of the subjects and the inner contour was successful for 82%. The results demonstrated the benefits that could accrue from inclusion of face features, giving an appropriate avenue for future research.

Active contours are now established as a technique for extracting salient contours from an image. Unfortunately the original technique suffers from many problems. A novel model-based dual active contour, a method of integrating global shape information with two active contours, has been developed to overcome the primary problems; sensitivity to initialisation and undesirable attractions by insignificant localised or regionalised features. The model guides the technique to avoid insignificant minima and is relinquished when the energy minimum is sufficiently compatible. The technique then finally operates as a pair of conventional active contours, ensuring that only image information is extracted, consistent with the original technique. 1