Associative memory (AM) robustly retrieves information from given partial data. Compared to artificial neural network (ANN)-based AM, spiking neural network (SNN)-based AM offers greater bio-plausibility, sparsity, and message storage capacity. Recently, an ANN-based multidirectional associative memory neural network (MAMNN) for handling multiple associations was implemented by extending an ANN-based bidirectional associative memory (BAM) neural network. In comparison, this study implements SpiMAM, a more bio-plausible MAMNN based on SNN with a winner-take-all mechanism. The circuit design of spiking MAMNN (SpiMAM) employing in-situ synaptic training was proposed for the first time. Instead of a memristor device or memristor model, a CMOS circuit of a memristive synapse featuring spike-timing-dependent-plasticity (STDP) is used to incorporate the CMOS integrated circuit challenges. The synaptic weights in the crossbar for storage and association of patterns were trained on-chip without requiring additional computing platforms and digital circuitry attached to the synapse. The entire circuit of the spiking MAMNN was implemented at the transistor-level in 180 nm standard CMOS technology to demonstrate pattern recognition applications. The robustness of the proposed circuit of SpiMAM was evaluated through post-layout simulations for PVT, mismatch variation, pixel flip, hard faults, memristive drifts, and Gaussian noise. Compared to the previous work, this work uses 86 % fewer synapses and 70 % fewer neurons for the pattern recognition of nine binary images of$5\times 3$pixel size.

The investigation of creeping waves around the torso combined with classification using a three-dimensional memristive neural network can be used for developing new smart clothes. In this work, analysis is performed to study the electromagnetic wave propagation around the torso. Two crossslot antennas are used in the study. #Antl is fixed and #Ant2 is moved around the body. The path gain is experimentally noted at various positions. An exponential decay is obtained when #Ant2 moves around the body, and this varies from person to person as the dimensions and dielectric constant value change. The effect of the hand near the #Ant2 while experimenting is also studied. The S parameter values obtained here can be fed to a threedimensional memristor crossbar array to perform classification using an Artificial Neural Network. Based on these S parameter values, the neural network with effectively low power and area requirements predicts the cloth size required for the individual.

The importance of intelligent transformers in the contemporary society cannot be overstated. Timely detection of damage or failure in transformers is crucial. This study evaluates many factors through the use of sensors that include oil level indicators, all of which are connected to the transformers. The data collected from these sensors is subsequently processed using a memristive Echo state network framework to monitor for errors. Two distinct ESN topologies, namely ‘sparsely connected’ and ‘stochasticity improved reservoir connections’, are utilized to ensure a precise prediction of the oil level in the transformer. By proactively detecting possible difficulties with the transformers, necessary steps may be swiftly initiated. Empirical findings demonstrate that the real-time forecasts for oil level and associated variables surpass alternative methods, exhibiting reduced error rates and enhanced precision in predictions.

Early detection and diagnosis of brain tumors are of great significance, as they can be life saving. Current state-of-the-art methods, including X-ray and magnetic resonance imaging (MRI) require more resources and advanced medical facilities, and cannot be used for continuous or long-term monitoring. The importance of this contribution lies in the timely detection of these medical conditions. In our work, we propose a method for identifying brain tumors that overcomes these shortcomings. Two antennas, Ant1 and Ant2 were used around the head, and changes in the transmission coefficients (S21) were monitored. Experiments are conducted on a human head-shaped container, and the transmission data obtained were transferred to a memristor crossbar array using a Voltage Threshold Adaptive Memristor (VTEAM) model for the prediction of cancer. The proposed crossbar is used for implementing echo state networks that detects the presence of cancer with an accuracy of 77.5% after incorporating compensation for signal integrity influences.

The human mind does not recognize absolute distances. Instead, it seeks comparisons based on similarity, often called psychometric metrics. While many psychometric metrics have been used in cognitive studies, they are seldom used for machine learning or neural computing studies. In this paper, we present a case of Shepherd's similarity metric that can be effective in naturally removing outliers in natural language classification problems. Natural language processing uses multiple cognitive regions of the human brain, investigations of which can help with developmental studies of the human mind. The proposed similarity metric can help understand the causal links of language processing, giving a sense of human mind functions. A comparison with other similarity metrics indicates that Shepherd's similarity shows unusual tolerance to noise changes and the ability to reject outliers naturally.

Variability analysis is an essential step in the design of robust memristive systems. The memristor devices often are faced with device to device, and cycle to cycle variability, which can be analysed using different types of memristor model parameters. Different memristor devices fit best in behaviour to different memristor models, and there is no universal model that fits all memristor-like devices. Therefore, it is important to develop a library of memristor models to cross validate, fit and analyse the behavior of realistic devices connecting memristive systems implementation. In this paper, we propose a library that contains different models of memristor, that can be programmed, and analysed for studying variability, and sensitivity of model parameters. The focus is on those parameters that are practically relevant to the design of realistic memristive systems. This library provides the options for parameter tuning, noise analysis and device fitting to like the right models to the right devices.

Electrocardiogram (ECG) analysis is widely used in the diagnosis of cardiovascular diseases. This paper proposes an explainable rule-mining strategy for prioritizing abnormal class detection in ECG data. The proposed method utilizes a biased-trained Artificial Neural Network (ANN) with input features derived from an ECG beat sequence and formulates a set of rules at each node of an on-demand tree-like search algorithm. The rule base at each node is derived from a linear combination of the most impactful features identified using gradient analysis in an ANN. The final derived model is an explainable rule-based system that detects abnormal heartbeats based on statistical and morphological features from ECG. The model achieves the target sensitivity, and accuracy with a low run-time complexity through a comprehensive offline rule mining process and is trained using the MIT-BIH Arrhythmia Database. The system achieves an accuracy of 93% with only nine nodes and a test sensitivity of 90% and 80% respectively for VEB and SVEB beat types, when tested on previously unseen ECG data from the INCART database. The model performance and complexity can be easily adjusted based on the real-time resource constraints of a wearable sensor. The model was deployed on an ARM Cortex M4-based embedded device and is shown to achieve a >50% reduction in sensor power consumption when only abnormal beats are wirelessly transmitted. i.e RF transmission is gated using the model output and transmission is disabled when the subject’s ECG is normal. The proposed technique is highly suited for healthcare applications because of its explainability, lower complexity, and real-time flexibility when deployed in the Internet of Things (IoT) enabled wearable edge sensors.

Pruning is a process of removing unwanted neurons from neural network computations. In Neural Networks, pruning creates sparse information processing, which can improve the overall generalisation and energy efficiency of the network. By excluding redundant weight values which are not contributing significantly to the system performance, hardware complexity can be reduced while maintaining the recognition accuracy. This paper evaluates the effectiveness of unstructured pruning on memristive crossbar based neural nodes in Artificial Neural Network (ANN) and Binary Weighted Neural Network (BWNN) architectures. The impact of pruning is analysed in terms of inference accuracy, energy consumption and area efficiency. The robustness of the pruned system is validated under the influence of conductance variability, bit errors, and multiply and accumulate errors.

The increasing demand for a high density of transistors in the chip with high yield often requires the manufacturing process to be mature. Defects are common when a new process technology is introduced. Manual testing for each wafer during a separate fabrication stage is a time-consuming process. To improve the production quality, the wafer map obtained after the electrical probe test should be categorized into corresponding defect classes. In this paper, we propose an automated wafer defect classification system using a Convolutional neural network (CNN)-memristor crossbar structure. The weights extracted from a pretrained neural network model are deployed into a crossbar structure. The output probability values from the softmax layer determine the class to which the wafer input belongs. The performance of CNN architecture is compared with other deep learning models. The area and power requirements of the proposed hardware architecture are also presented.

In super-resolution magnetic resonance imaging (SR-MRI), the low-resolution scans are acquired keeping the central low-frequency components intact. The scan-time for a given matrix size is shortened by acquiring only the central low-frequency part of the k-space (Fourier space), and filling the unacquired portion with zeroes. While transforming the zero-padded low-resolution acquired k-space to the image domain through inverse Fourier transformation, an inherent blur and high-frequency oscillatory artifacts are manifested in the reconstructed image. State-of-the-art methods including reconstruction-based and deep-learning-based approaches have addressed this problem to a large extent; however, it remains a challenging problem to completely eliminate the remnant effects of blur or ringing in some of the subtle clinical features. In the proposed SR method, we have implemented an interleaved hybrid domain convolutional neural network (CNN) model consisting of a k-space and spatial domain network together with local residual connections. This helps to improve the accuracy of hyper-parameter estimation and effectively reduce the loss function, with the resulting advantage of attaining improved peak-signal-to-noise-ratio (PSNR), structural similarity index measure (SSIM) and normalized root mean square error (NRMSE) as compared to the state-of-the-art methods.

Noise in image sensors led to the development of a whole range of denoising filters. A noisy image can become hard to recognize and often require several types of post-processing compensation circuits. This paper proposes an adaptive denoising system implemented using analog in-memory neural computing network. The proposed method can learn new noises and can be integrated into or alone with CMOS image sensors. Three denoising network configurations are implemented, namely, (1) single layer network, (2) convolution network, and (3) fusion network. The single layer network shows the processing time, energy consumption and on-chip area of 3.2$\mu$s, 21n J per image and 0.3mm2respectively, meanwhile, convolution denoising network correspondingly shows 72m s, 236$\mu$J and 0.48mm2. Among all the implemented networks, it is observed that performance metrics SSIM, MSE and PSNR show a maximum improvement of 3.61, 21.7 and 7.7 times respectively.

Memristor crossbar arrays are used in a wide range of in-memory and neuromorphic computing applications. However, memristor devices suffer from non-idealities that result in the variability of conductive states, making programming them to a desired analog conductance value extremely difficult as the device ages. In theory, memristors can be a nonlinear programmable analog resistor with memory properties that can take infinite resistive states. In practice, such memristors are hard to make, and in a crossbar, it is confined to a limited set of stable conductance values. The number of conductance levels available for a node in the crossbar is defined as the crossbar’s resolution. This paper presents a technique to improve the resolution by building a super-resolution memristor crossbar with nodes having multiple memristors to generate$r$-simplicial sequence of unique conductance values. The wider the range and number of conductance values, the higher the crossbar’s resolution. This is particularly useful in building analog neural network (ANN) layers, which are proven to be one of the go-to approaches for forming a neural network layer in implementing neuromorphic computations.

The operation of stochastic neural networks with randomly disconnected or blanked out synaptic connections can optimize the power consumption of its circuit implementations in traditional crossbars. However, the permanent disconnection of some synaptic weights can lead to the poor performance of the network, which will require further optimization of the remaining active synapses. In this work, we propose a learning scheme for such stochastically blanked out neural networks. The architecture of the neural network is implemented with standard 0.18u CMOS circuits, in 1T1M crossbar configuration for controlling the blank out rate. The results of the retraining the network with up to 50% of disconnected synapses are reported for the standard image classification problem such as MNIST handwritten digits recognition.

Applied AI chips with neural networks fail to capture and scale different forms of human intelligence. In this study, the definition of a strong AI system in hardware and architecture for building neuro-memristive strong AI chips is proposed. The architecture unit consists of loop and hoop networks that are built on recurrent and feedforward information propagation concepts. Applying the principle that `every brain is different', we build a strong network that can take different structural and functional forms. The strong networks are build using hybrids of loop and hoop networks having generalisation abilities, with higher levels of randomness incorporated to introduce greater flexibility in creating different neural architectures.

The selection of hyperparameters and circuit components for optimum hardware implementation of a neural network is a challenging task, which has not been automated yet. This work proposes the method for the selection of optimum neural network architecture and hyperparameters using genetic algorithm based on the hardware-related performance metrics, such an on-chip area, power consumption, processing time and robustness to hardware non-idealities, and focus on memristor-based analog network architecture. The experimental results show that the proposed approach allows to select the optimum architecture based on the designers' preferences.

The security of personal consumer electronic devices requires reading biometric signatures of individuals such as iris, fingerprints, and face recognition. However, they are often subject to attacks on hardware and software. To address this issue, we propose to develop biometric-aware pixels and use neuro-memristive processing crossbar arrays to implement biometric pattern matching within the pixels. In this work, we use a neuro-memristor array computing to implement a nearest similarity search with a spatio-temporal encoding scheme.

The volume, veracity, variability, and velocity of data produced from the ever increasing network of sensors connected to Internet pose challenges for power management, scalability, and sustainability of cloud computing infrastructure. Increasing the data processing capability of edge computing devices at lower power requirements can reduce several overheads for cloud computing solutions. This paper provides the review of neuromorphic CMOS-memristive architectures that can be integrated into edge computing devices. We discuss why the neuromorphic architectures are useful for edge devices and show the advantages, drawbacks, and open problems in the field of neuromemristive circuits for edge computing.

The scalability and non-ideality issues of the memristor circuits poses several challenges to the implementation of analog memristive probabilistic neural networks in hardware. To meet the emerging challenges of faster edge AI computing devices, the integration of neural networks within or near to the sensor can improve the data processing times, reduce bandwidth requirements, and reduce data transfer errors. The fast learning in probabilistic neural network (PNN) make it an attractive solution for energy efficient computing in edge devices. The PNN estimates the density function of the categories and classifies the input based on the Bayes decision rule. It avoids backpropagation, since weights are derived from training samples directly and set in the first initialization stage. The proposed hardware realization of the PNN is based on a memristor crosssbar architecture. The simulations demonstrate that the accuracy of the hardware realization of the PNN can be as high as 93.3% for the MNIST dataset if a proper smoothing parameter is selected.

A generalized bell-shaped function is an essential building block for a neuro-fuzzy systems and radial basis function neural networks. With the advent of edge computing, analog neuro-chips can potentially speed-up the near sensor computing. In this paper, we present a generalized bell function generator circuit for machine learning architectures. The circuit design of generalized bell-shaped function as the hybrid CMOS-Memristor that is compatible with typical memristive crossbar architecture is presented, where it uses three memristors to control the output current shape. Designed circuit occupies 10 μm2and consumes less than 4.1 μW. The proposed circuit could be used as a standalone neuron for radial basis function neural network, as demonstrated in this work.

The impact of device-to-device, cycle-to-cycle, and parasitic variations in memristor devices on the performance of neural network architectures is not a fully understood topic. In this paper, we present an explicit analysis of memristor variabilities and non-idealities of memristive crossbar based learning architectures. The measurements of real devices and their effects on dot product operation in a memristive crossbar is reported. The effect of these non-idealities, limited resistive levels and variabilities on the performance and reliability of two-layer Artificial Neural Network (ANN), Convolutional Neural Network (CNN) and Binary Neural Network (BNN) is analyzed and presented.

The implementation of backpropagation algorithm using gradient descent operation with analog circuits is an open problem. In this paper, we present the analog learning circuits for realizing backpropagation algorithm for use with neural networks in memristive crossbar arrays. The circuits are simulated in SPICE using TSMC 180nm CMOS process models, and HP memristor models. The gradient descent operations are validated comprehensively using the relevant transfer characteristics and transient response of individual circuit modules.

The memristor crossbar array architecture can find a wide range of applications in the design of neuromorphic computing systems. The scalability of the arrays is important to extend the use in complex cognitive tasks. However, the creation of large-sized arrays is limited by a sneak-path problem reducing noise margins and accuracy. In this paper, we perform a large scale analysis of a sneak path problem in crossbar arrays using HSPICE simulation models. This allows for developing a realistic mathematical model for simulating large scale crossbar arrays. The performance analysis and impact of sneak paths for neural network implemented on a crossbar array is tested using the MNIST character recognition database. Also, in this work we provide a possible solution to suppress the influence of the sneak path current on the network. The suppressing effect is achieved by dividing a large memristive crossbar array into smaller arrays. These crossbars are simulated in HSPICE as well, examined and compared to the originally constructed crossbar.

In this study, a post‐processing filter to enhance diffusion sensitivity, resulting in larger intensity changes in regions with the abrupt transition of local diffusivity in raw diffusion weighted image (DWI) volumes. Weights computed using a non‐linear three‐dimensional neighbourhood operation are assigned to each voxel within the neighbourhood, with the weighted average representative of the enhanced DWI. The processed images exhibit better distinction among regions with differing levels of physical diffusion. While the resulting improvements in diffusion sensitivity are highlighted with the help of colour maps, parametric maps, and tractography, implications of the filtering process to recover missing information is illustrated in terms of ability to restore portions of fibre tracts which are otherwise absent in the unprocessed diffusion tensor imaging. Quantitative evaluation of the filtering process is performed using a metric representative of the estimated b ‐value, which is the consolidation machine parameters used for DWI acquisition.

Hierarchical temporal memory (HTM) is a machine learning algorithm inspired by the information processing mechanisms of the human neocortex and consists of a spatial pooler (SP) and temporal memory (TM). In this paper, we develop circuits and systems to achieve the optimized design of an HTM SP, an HTM TM, and a memristive analog pattern matcher for pattern recognition applications. The HTM SP realizes an optimized hardware design through the introduction of mean overlap calculations and by replacing the threshold determination in the inhibition stage with a weighted summation operator over the neighborhood of the pixel under consideration. HTM TM is based on discrete analog memristive memory arrays and a weight update procedure. The operation of the proposed system is demonstrated for a face recognition problem, using the standard AR, ORL, and Yale databases, and for speech recognition, using the TIMIT database, with achieved accuracies of 87.21% and approximately 90%, respectively, given an SNR of 10 dB. Visual data processing using binary HTM SP features requires less storage and processing memory than required by the traditional processing methods, with the area and power requirements for its implementation being 0.096 mm2and 1756 mW, respectively. The design of the TM circuit for a single pixel requires 23.85 μm2of area and 442.26 μW of power.

Main challenge with denoising applications using conventional image filtering with fixed size windows is the smearing or blurring of various region boundaries. This blurring is the visual representation of deviation in estimated denoised value (region mean) due to the presence of heterogeneous (or dissimilar pixels) regions in region-averaging along with the homogenous (or similar pixels) regions. Thus an adaptive window shape selection using only the most similar pixels (G-Neighbor) within the fixed window can avoid the presence of such undesired heterogeneous regions from mean intensity calculation. This Variable Pixel G-Neighbor filter is implemented using CMOS circuits and further simulated in MATLAB. This analog hardware approach demonstrates the possibilities in visual quality enhancement in image acquisition stages itself. Also the near-real time response of this pre-processor offers a practical solution to image computing problems caused by image quality and processing resources limitations. The improvement in image quality is evaluated by comparing results from conventional Mean Filter and its modified version using Variable Pixel G-Neighbor Filter. Metrics evaluates the signal strength enhancement (Peak Signal-to-Noise Ratio, PSNR), amount of noise removal (Mean Square Error, MSE), and structural preservation which indicates minimal deformation or blurring (Structural Similarity Index Measure, SSIM). From sample result it is shown that proposed approach offers PSNR = 41.25, MSE = 3.9, SSIM = 0.85 against conventional mean filter having values 38.64, 7.4, and 0.71 respectively.

In this paper, we review different memristive threshold logic (MTL) circuits that are inspired from the synaptic action of the flow of neurotransmitters in the biological brain. The brainlike generalization ability and the area minimization of these threshold logic circuits aim toward crossing Moore's law boundaries at device, circuits, and systems levels. Fast switching memory, signal processing, control systems, programmable logic, image processing, reconfigurable computing, and pattern recognition are identified as some of the potential applications of MTL systems. The physical realization of nanoscale devices with memristive behavior from materials, such as TiO2, ferroelectrics, silicon, and polymers, has accelerated research effort in these application areas, inspiring the scientific community to pursue the design of high-speed, low-cost, low-power, and high-density neuromorphic architectures.

Hierarchical Temporal Memory (HTM) is a machine learning algorithm that is inspired from the working principles of the neocortex, capable of learning, inference, and prediction for bit-encoded inputs. Spatial pooler is an integral part of HTM that is capable of learning and classifying visual data such as objects in images. In this paper, we propose a memristor-CMOS circuit design of spatial pooler and exploit memristors capabilities for emulating the synapses, where the strength of the weights is represented by the state of the memristor. The proposed design is validated on a challenging application of single image per person face recognition problem using AR database resulting in a recognition accuracy of 80%.

MOTIVATION: The biomarker discovery process in high-throughput genomic profiles has presented the statistical learning community with a challenging problem, namely learning when the number of variables is comparable or exceeding the sample size. In these settings, many classical techniques including linear discriminant analysis (LDA) falter. Poor performance of LDA is attributed to the ill-conditioned nature of sample covariance matrix when the dimension and sample size are comparable. To alleviate this problem, regularized LDA (RLDA) has been classically proposed in which the sample covariance matrix is replaced by its ridge estimate. However, the performance of RLDA depends heavily on the regularization parameter used in the ridge estimate of sample covariance matrix. RESULTS: We propose a range-search technique for efficient estimation of the optimum regularization parameter. Using an extensive set of simulations based on synthetic and gene expression microarray data, we demonstrate the robustness of the proposed technique to Gaussianity, an assumption used in developing the core estimator. We compare the performance of the technique in terms of accuracy and efficiency with classical techniques for estimating the regularization parameter. In terms of accuracy, the results indicate that the proposed method vastly improves on similar techniques that use classical plug-in estimator. In that respect, it is better or comparable to cross-validation-based search strategies while, depending on the sample size and dimensionality, being tens to hundreds of times faster to compute. AVAILABILITY AND IMPLEMENTATION: The source code is available at https://github.com/danik0411/optimum-rlda CONTACT: [email protected] information: Supplementary materials are available at Bioinformatics online.

The inability of automated edge detection methods inspired from primal sketch models to accurately calculate object edges under the influence of pixel noise is an open problem. Extending the principles of image perception i.e. Weber-Fechner law, and Sheperd similarity law, we propose a new edge detection method and formulation that use perceived brightness and neighbourhood similarity calculations in the determination of robust object edges. The robustness of the detected edges is benchmark against Sobel, SIS, Kirsch, and Prewitt edge detection methods in an example face recognition problem showing statistically significant improvement in recognition accuracy and pixel noise tolerance.

Brain-inspired circuits can provide an alternative solution to implement computing architectures taking advantage of fault tolerance and generalization ability of logic gates. In this brief, we advance over the memristive threshold circuit configuration consisting of memristive averaging circuit in combination with operational amplifier and/or CMOS inverters in application to realizing complex computing circuits. The developed memristive threshold logic gates are used for designing fast Fourier transform and multiplication circuits useful for modern microprocessors. Overall, the proposed threshold logic outperforms previous memristive-CMOS logic cells on every aspect, however, they indicate a lower chip area, lower total harmonic distortion, and controllable leakage power, but a higher power dissipation with respect to CMOS logic.

Real-time detection of moving objects involves memorization of features in the template image and their comparison with those in the test image. At high sampling rates, such techniques face the problems of high algorithmic complexity and component delays. We present a new resistive switching-based threshold logic cell which encodes the pixels of a template image. The cell comprises a voltage divider circuit that programs the resistances of the memristors arranged in a single-node threshold logic network, and the output is encoded as a binary value by using a CMOS inverter gate. When a test image is applied to the template-programmed cell, a mismatch in the respective pixels is seen as a change in the output voltage of the cell. The proposed cell when compared with CMOS equivalent implementation shows improved performance in area, leakage power, power dissipation, and delay.

We report a resistance-based threshold logic family useful for mimicking brain-like large variable logic functions in VLSI. A universal boolean logic cell based on an analog resistive divider and threshold logic circuit is presented. The resistive divider is implemented using memristors, and provides output voltage as a summation of weighted product of input voltages. The output of the resistive divider is converted into a binary value by a threshold operation implemented by CMOS inverter and/or Opamp. A universal cell structure is presented to decrease the overall implementation complexity and number of components. When the number of input variables becomes very high, the proposed cell offers advantages of smaller area and design simplicity in comparison with CMOS-based logic circuits.

High feature dimensionality of realistic datasets adversely affects the recognition accuracy of nearest neighbor (NN) classifiers. To address this issue, we introduce a nearest feature classifier that shifts the NN concept from the global-decision level to the level of individual features. Performance comparisons with 12 instance-based classifiers on 13 benchmark University of California Irvine classification datasets show average improvements of 6 and 3.5% in recognition accuracy and area under curve performance measures, respectively. The statistical significance of the observed performance improvements is verified by the Friedman test and by the post hoc Bonferroni–Dunn test. In addition, the application of the classifier is demonstrated on face recognition databases, a character recognition database and medical diagnosis problems for binary and multi-class diagnosis on databases including morphological and gene expression features.

The human brain exhibits robustness against natural variability occurring in face images, yet the commonly attempted algorithms for face recognition are not modular and do not apply the principle of binary decisions made by the firing of neurons. We present a biologically inspired modular unit implemented as an algorithm for face recognition that applies pixel-wise local binary decisions on similarity of spatial-intensity change features. The results obtained with a single gallery image per person show a robust and high recognition performance: 94% on AR, 98% on Yale, 97% on ORL, 97% on FERET (fb), 92% on FERET (fc), and 96% on Caltech face image databases.