In computational sensing and imaging, optics can perform some of the processing tasks traditionally performed by digital signal processing and, conversely, digital signal processing can perform some of the processing tasks traditionally performed by optics. This approach thus provides a new framework for reducing electrical power dissipated by sensors: for a criterion end-to-end functional performance, design the optics such that the digital computation cost is as low as possible. There are two such strategies to designing optics to reduce electrical power dissipation: creating optics that produce an optical signal that either 1) requires less digital processing for the criterion functional performance or 2) requires fewer pixels and hence less read, A/D conversion, and signal transfer power. A special-purpose diffractive optical element integrated with a CMOS image sensor photodetector array with simple on-chip processing reveal power reduction in one operational mode roughly two orders of magnitude compared to that in a functionally equivalent traditional lens-based sensors.

Images captured under low-light conditions are noisy as a result of photon statistics and quantization error, among other reasons. Such statistical limitations can be reduced by using pixels with larger areas, but this approach leads to aliasing artifacts. We propose a maximum-likelihood version of super-resolution for low-light conditions in which Fourier image coefficients and unknown spatial shifts between captured frames are estimated iteratively, all in order to produce the single image with high expected fidelity. We illustrate the power of our method on both one-dimensional synthetic data and on two-dimensional medical images.

Still-image processing algorithms are tailored to and depend crucially upon the properties of the class of images to which they are applied, for instance natural images in consumer digital cameras, medical images in fMRI machines, and binary text images in some photocopiers. We describe a new and possibly very important class of images and tasks for which traditional algorithms seem ill-suited, and for which new algorithms and general methods and concepts are required. This new class of images arises in imaging systems designed through new, joint optimization methods where the optics and the image processing are designed simultaneously in order to yield a high-quality digital image. These new design methods yield intermediate optical images that have unusual spatial, noise and chromatic properties ill-served by traditional image methods. Moreover, these new images present a number of novel challenges in image processing hardware implementations such as constrained space-variance. We describe these briefly new, joint methods for designing digital-optical imaging systems, characterize the intermediate optical images they yield, and some of the digital image processing challenges for producing high-quality still images from these sensed optical images.

An important halftoning problem faced in the design of many color printers and copiers is to represent a 24-bit color image by a small number of preset output colors, generally at a higher spatial resolution. We propose dividing the input and output image into corresponding small blocks, calculating each input block's color average, and then determining a set of the printing colors for the corresponding output block to best render the input block's average color. Our method exploits the higher spatial resolution of the printing process and yields constrained local color optimality. Artifacts common with error diffusion methods do not occur. In an experimental comparison to vector color error diffusion halftoning, block color quantization yields superior rendering of graphics. For natural images, block color quantization may occasionally generate false contours, but due to the lack of texture artifacts, block color quantization halftones may be preferable to error diffusion halftones.

We describe the Open Mind Initiative, a framework for large scale collaborative efforts in building components of "intelligent" systems that address common sense reasoning, document and language understanding, speech and character recognition, and so on. Based on the Open Source methodology, the Open Mind Initiative allows domain specialists to contribute algorithms, tool developers to provide software infrastructure and tools, and non specialist "e-citizens" to contribute training data and information to large databases. An important challenge is to make it easy and rewarding for e-citizens to provide such information. The paper illustrates the initiative through several demonstration projects of modest scale, including some related to character and document problems, and identifies general challenges and opportunities.

No abstract available.

We give an overview of speechreading systems from the perspective of the face and gesture recognition community, paying particular attention to approaches to key design decisions and the benefits and drawbacks. We discuss the central issue of sensory integration how much processing of the acoustic and the visual information should go on before integration how should it be integrated. We describe several possible practical applications, and conclude with a list of important outstanding problems that seem amenable to attack using techniques developed in the face and gesture recognition community.

"Error-Confidence" measures the probability that the proportion of errors made by a classifier will be within epsilon of EB, the optimal (Bayes) error. Probably Almost Bayes (PAB) theory attempts to quantify how this confidence increases with the number of training samples. We investigate the relationship empirically by comparing average error versus number of training patterns (m) for linear and neural network classifiers. On Gaussian problems, the resulting EC curves demonstrate that the PAB bounds are extremely conservative. Asymptotic statistics predicts a linear relationship between the logarithms of the average error and the number of training patterns. For low Bayes error rates we found excellent agreement between the prediction and the linear discriminant performance. At higher Bayes error rates we still found a linear relationship, but with a shallower slope than the predicted-1. When the underlying true model is a three-layer network, the EC curves show a greater dependence on classifier capacity, and the linear predictions no longer seem to hold.

We present a special greedy algorithm for assigning codebook bits in order to improve the context-dependent compression of palettized images. Our method provides superior compression to that of other bit-plane compression schemes, such as a "random" method based on the palettization process itself, and binary sequence coding; the improvement is quite large for the full M-ary Markov model. Our method avoids the high computational costs of exhaustive codebook search, and can be tailored to specific context models and hardware constraints.

We present a graph-based method for rapid, accurate search through prototypes for transformation-invariant pattern classifica(cid:173) tion. Our method has in theory the same recognition accuracy as other recent methods based on ''tangent distance" [Simard et al., 1994], since it uses the same categorization rule. Nevertheless ours is significantly faster during classification because far fewer tan(cid:173) gent distances need be computed. Crucial to the success of our system are 1) a novel graph architecture in which transformation constraints and geometric relationships among prototypes are en(cid:173) coded during learning, and 2) an improved graph search criterion, used during classification. These architectural insights are applica(cid:173) ble to a wide range of problem domains. Here we demonstrate that on a handwriting recognition task, a basic implementation of our system requires less than half the computation of the Euclidean sorting method.

Describes a special purpose, very high speed, digital deterministic Boltzmann neural network VLSI chip. Each chip has 32 physical neural processors, which can be apportioned into an arbitrary topology (input, multiple hidden and output layers) of up to 160 virtual neurons total. Under typical conditions, the chip learns at approximately 5*10/sup 8/ connection updates/second (CUPS). Through relatively minor (subsequent) modifications, the authors' chips can be 'tiled' in multi-chip modules, to make multi-layer networks of arbitrary size suffering only slight communications delays and overhead. In this way, the number of CUPS can be made arbitrarily large, limited only by the number of chips tiled. The chip's high speed is due to massively parallel array computation of the inner products of connection weights and neural activations, limited (but adequate) precision for weights and activations (5 bits), high clock rate (180 MHz), as well as several algorithmic and design insights.>

Because of the complexity and high dimensionality of the problem, speech recognition—perhaps more than any other problem of current interest in network research—will profit from human neurophysiology, psychoacoustics and psycholinguistics: approaches based exclusively on engineering principles will provide only limited benefits. Despite the great power of current learning algorithms in homogeneous or unstructured networks, a number of difficulties in speech recognition seem to indicate that homogeneous networks taken alone will be insufficient for the task, and that structure—representing constraints—will also be required. In the biological system, the sources of such structure include developmental and evolutionary effects. Recent considerations of the evolutionary sources of neural structure in the human speech and language systems, including models of the interrelationship between speech motor system and auditory system, are analyzed with special reference to neural network approaches.