We address both recognition of true classes and rejection of unseen false classes inputs, as occurs in many realistic pattern recognition problems. we advance a hierarchical binary-decision classifier and produce analog outputs at each node, with yields a new soft-decision hierarchical is designed by our new support vector clustering method, which selects the classes to be separated at each node in the hierarchy. Use of our SVRDM (support vector representation and discrimination machine) classifiers at each node provides generalization and rejection ability. The soft-decision SVRDM output allows use of the confidence score for each class at each node; this is shown to improve classification (for true classes) and rejection (for false classes) performance. New aspects of this paper are that we provide remarks on our hierarchical design method, including our hierarchical clustering rule, and discuss the meaning and the use of probabilities in our soft-decision hierarchical SVRDM classifiers. We also provide initial tests results on a new database (COIL) that allows large class problem to be addressed. No prior work considered rejection of false classes on this database.

In this paper, we address both recognition of true object classes and rejection of false (non-object) classes as occurs in many realistic pattern recognition problems. We modified our hierarchical binary-decision classifier to produce analog outputs at each node, with values proportional to the class conditional probabilities at that node. This yields a new soft-decision hierarchical system. The hierarchical classification structure is designed by our weighted support vector k-means clustering method, which selects the classes to be separated at each node in the hierarchy. Use of our SVRDM (support vector representation and discrimination machine) classifiers at each node provides generalization and rejection ability. Compared to the standard SVM, use of the Gaussian kernel function and a looser constraint in the classifier design give our SVRDM an improved rejection ability. The soft-decision SVRDM output allows us to use the confidence level of each class to improve the classification (for true class inputs) and rejection (for false class inputs) performance of the hierarchical classifier. False class rejection is a major new aspect of this work. It is not present in most prior work. Excellent test results on a real infra-red (IR) database are presented.

We propose a binary hierarchical classification structure to address the multi-class classification problem with a new hierarchical design method, weighted support vector k-means clustering, which automatically separates a set of classes into two smaller groups at each node in the hierarchy. This method is able to visualize and cluster high-dimensional support vector data; therefore, it greatly improves upon prior hierarchical classifier design. At each node in the hierarchy, we apply an SVRDM (support vector representation and discrimination machine) classifier, which offers generalization and good rejection of unseen false objects, which is not achieved by the standard SVM classifier. We provide a new theoretical basis for the good SVRDM rejection obtained, due to its looser constrained optimization problem, compared to that of an SVM. New classification and rejection test results are presented on a real IR (infra-red) database.

We propose a binary hierarchical classification structure to address the multi-class classification problem with a new hierarchical design method, k-means SVRM (support vector representation machine) clustering. This greatly improves upon our prior IJCNN hierarchical design. At each node in the hierarchy, we apply the SVRDM (support vector representation and discrimination machine) classifier, which offers generalization and good rejection ability. We also provide new theoretical bases and methods for our choice of the kernel function and new SVRDM parameter selection rules. Classification and rejection test results are presented on new databases of both simulated and real infra-red (IR) data.

A binary hierarchical classifier is proposed to solve the multi-class classification problem. We also require rejection of non-target inputs, which produces a very difficult problem. The SVRDM (support vector representation and discrimination machine) classifier is considered at each node in the hierarchy, since it offers good generalization and rejection ability. Using this hierarchical SVRDM classifier with magnitude Fourier transform features, initial recognition and rejection test results on simulated infrared data are excellent.

A hierarchical classifier using a new SVRDM (support vector representation and discrimination machine) is proposed for automatic target recognition. An accuracy and distance-based method is used to design a hierarchical classifier. Our SVRDM hierarchical classifier has the ability to reject unseen non-object classes and clutter inputs. Uses of both iconic and spatial frequency domain features are considered. Initial recognition and rejection test results on infrared (IR) data are excellent.

In many practical applications, learning from imbalanced data poses a significant challenge that is increasingly faced by the machine learning community. The class imbalance problem raises issues that are either nonexistent or less severe compared to balanced class cases. This paper presents a new method for imbalanced data classification. The proposed method is based on support vector machine classifiers and backward pruning technique. The experimental results obtained on two data sets demonstrate the effectiveness of the new algorithm.

Support vector machines (SVMs) have been successfully used in many classification fields. However, conventional SVMs do not consider rejecting inputs and thus suffer from false alarms. The first reason for this is that every input is assumed to belong to one of the object classes and is accepted in some class. In this paper, we show that the second reason is that conventional SVMs do not describe each object class well. Thus, use of an output threshold does not solve this problem. We present a new support vector representation and discrimination machine (SVRDM), which has a discrimination capability comparable to that of the conventional SVM, and also offers good rejection ability. False alarm rates are greatly reduced. We analyze the properties of these two classifiers (SVM and SVRDM) in transformed feature space and compare their performances using both synthetic and real data.

We propose a novel technique for the design of radial basis function (RBF) neural networks (NNs). To select various RBF parameters, the class membership information of training samples is utilized to produce a new cluster classes. This allows us to control performance as desired and approximate Neyman-Pearson classification. We show that by properly choosing the desired output neuron levels, then the RBF hidden to output layer performs Fisher discrimination analysis, and the full system performs a nonlinear Fisher analysis. Data on an agricultural product inspection problem and on synthetic data confirm the effectiveness of these methods.

Distinguishing one object class from others is the main task of many classification systems. However, often a classifier must also be able to reject non-object inputs and must thus achieve both rejection and classification. We address this problem with a novel support vector representation and discrimination machine (SVRDM). The support-vector-based nature allows the SVRDM to exhibit good generalization. The SVRDM allows rejection of non-object data, while the standard SVMs do not do well at this. We present results on synthetic data and on the pose, illumination and expression (PIE) database that demonstrate that the SVRDM outperforms popular classifiers.

We advance new active object recognition algorithms that classify rigid objects and estimate their pose from intensity images. Our algorithms automatically detect if the class or pose of an object is ambiguous in a given image, reposition the sensor as needed, and incorporate data from multiple object views in determining the final object class and pose estimate. A probabilistic feature space trajectory (FST) in a global eigenspace is used to represent 3D distorted views of an object and to estimate the class and pose of an input object. Confidence measures for the class and pose estimates, derived using the probabilistic FST object representation, determine when additional observations are required as well as where the sensor should be positioned to provide the most useful information. We demonstrate the ability to use FSTs constructed from images rendered from computer-aided design models to recognize real objects in real images and present test results for a set of metal machined parts.

We present a new class of quadratic filters that are capable of creating spherical, elliptical, hyperbolic and linear decision surfaces which result in better detection and classification capabilities than the linear decision surfaces obtained from correlation filters. Each filter comprises of a number of separately designed linear basis filters. These filters are linearly combined into several macro filters; the output from these macro filters are passed through a magnitude square operation and are then linearly combined using real weights to achieve the quadratic decision surface. For detection, the creation of macro filters (linear combinations of multiple single filters) allows for a substantial computational saving by reducing the number of correlation operations required. In this work, we consider the use of Gabor basis filters; the Gabor filter parameters are separately optimized. The fusion parameters to combine the Gabor filter outputs are optimized using an extended piecewise quadratic neural network (E-PQNN). We demonstrate methods for selecting the number of macro Gabor filters, the filter parameters and the linear and nonlinear combination coefficients. We present preliminary results obtained for an infrared (IR) vehicle detection problem.

An efficient two-stage algorithm to compute nonlinear features is described. Its implementation on a neural net with adaptive activation functions that raise the input data to an arbitrary power is described. Its use in face recognition with unknown input poses is presented.

We present new test results for our active object recognition algorithms which are based on the feature space trajectory (FST) representation of objects and a neural network processor for computation of distances in global feature space. The algorithms are used to classify, and estimate the pose of objects in different stable rest positions and automatically re-position the camera if the class or pose of an object is ambiguous in a given image. Multiple object views are used in determining both the final object class and pose estimate. An FST in eigenspace is used to represent 3D distorted views of an object. FSTs are constructed using images rendered from solid models. The FSTs are analyzed to determine the camera positions that best resolve ambiguities in class or pose. Real objects are then recognized from intensity images using the FST representations derived from rendered imagery.

A new technique is discussed to recognize human faces under varying aspect views (pose). We first estimate the pose of an unknown human face from a 2D gray-scale image and then transform the unknown face to a reference pose using a feature extraction procedure. A different set of features for discriminating between different individuals are then extracted from these reconstructed faces for recognition. The feature extraction scheme used is known as the maximum representation and discrimination feature method. The advantage of our procedure is that it inherently removes distortions due to pose variations, and therefore requires only single training and/or test face images, which could be at different aspect views. For transformation, it does not require the face to be in the database during training. For recognition, only one aspect view at any pose is necessary.

Automatic target recognition (ATR) generally refers to the autonomous or aided target detection and recognition by computer processing of data from a variety of sensors such as forward looking infrared (FLIR), synthetic aperture radar(SAR), inverse synthetic aperture radar (ISAR), laser radar (LADAR), millimeter wave (MMW) radar, multispectral/hyperspectral sensors, low-light television (LLTV), video, ete. It is an extremely important capability for targeting and surveillance missions of defense weapon systems operating from a variety of platforms.

Detection involves locating all candidate regions of interest (objects) in a scene independent of the object class with object distortions and contrast differences, etc., present. It is one of the most formidable problems in automatic target recognition, since it involves analysis of every local scene region. We consider new detection algorithms and the fusion of their outputs to reduce the probability of false alarm P(FA) while maintaining high probability of detection P(D). Emphasis is given to detecting obscured targets in infrared imagery.

Advanced correlation filter synthesis algorithms to achieve rotation invariance are described. We use a specified form for the filter as the rotation invariance constraint and derive a general closed-form solution for a multiclass rotation-invariant filter that can recognize a number of different objects. By requiring the filter to minimize the average correlation plane energy, we produce a multiclass rotation invariant (RI) RI-MACE filter, which controls correlation plane sidelobes and improves discrimination against false targets. To improve noise performance, we require the filter to minimize a weighted sum of correlation plane signal and noise energy. Initial test results of all filters are provided.>

A general-purpose programmable optical image processor architecture is described. A correlator architecture is used with different filter functions employed to implement a variety of different operations required in the different levels of computer vision. We consider input scenes with a number of objects present in clutter with a number of different object classes, distortions, and contrasts. The system locates all objects and identifies the class of each. The optical image processing operations performed include morphological low-level nonlinear functions, detection of candidate regions of interest, fusion of correlation outputs to reduce false alarms, image enhancement, and feature extraction. The optical correlation filters to realize each operations, examples of each, and real-time optical correlator hardware are described.>

A new approximate method is proposed for updating robust pseudoinverse and Ho-Kashyap associative processors. The method can both add and delete vectors. It is faster than existing methods for updating the standard pseudoinverse associative processor, in addition to operating on a preferable robust associative processor. The new method is based on the matrix inversion lemma. Update algorithms are also noted that are suitable for reduced accuracy (analog) processors and for pipelined array processors.>

Application of neural nets to invariant pattern recognition is considered. The authors study various techniques for obtaining this invariance with neural net classifiers and identify the invariant-feature technique as the most suitable for current neural classifiers. A novel formulation of invariance in terms of constraints on the feature values leads to a general method for transforming any given feature space so that it becomes invariant to specified transformations. A case study using range imagery is used to exemplify these ideas, and good performance is obtained.

A three-layer adaptive clustering neural net is described for distortion-invariant multiclass object recognition in difficult problems requiring piecewise nonlinear discriminant surfaces. The number of hidden-layer neurons is determined by an organized procedure (several neurons are used per class as prototypes of each class). These are chosen by clustering techniques. The vector description of each prototype in the multidimensional input feature space specifies a set of linear discriminant functions that are the initial input to the hidden-layer weights used. These weights are then refined by a neural net algorithm using conjugate gradient techniques to produce the final weights. A neural net (NN) that marries pattern-recognition and NN techniques is thus obtained. Various multiclass distortion-invariant classification results are presented

The authors compare the storage capacity and other properties of various neural associative processors (APs) and find that the Ho-Kashyap (H-K) AP has the largest storage capacity and can handle linearly dependent keys. General memory (random keys) and distortion-invariant pattern-recognition APs are considered. A discussion is presented of a content-addressable structure that further improves recall accuracy and noise performance and decreases the size of the memory matrix. Results from the new H-K content-addressable AP are given. In all cases, the APs use only one pass (no iterations) in recall

A technique for determining the distortion parameters (location and orientation) of general three-dimensional objects from a single range image view is introduced. The technique is based on an extension of the straight-line Hough transform to three-dimensional space. It is very efficient and robust, since the dimensionality of the feature space is low and since it uses range images directly (with no preprocessing such as segmentation and edge or gradient detection). Because the feature space separates the translation and rotation effects, a hierarchical algorithm to detect object rotation and translation is possible. The new Hough space can also be used as a feature space for discriminating among three-dimensional objects.>

The error rates of linear classifiers that utilize various criterion functions are investigated for the case of two normal distributions with different variances and a priori probabilities. It is found that the classifier based on the least mean squares (LMS) criterion often performs considerably worse than the Bayes rate. The perceptron criterion (with suitable safety margin) and the linearized sigmoid generally lead to lower error rates than the LMS criterion, with the sigmoid usually the better of the two. Also investigated are the exceptions to the general trends: only if one class is known to have much larger a priori probability or variance than the other should one expect the LMS or perceptron criteria to be slightly preferable as far as error rate is concerned. The analysis is related to the performance of the back-propagation (BP) classifier, giving some understanding of the success of BP. A neural-net classifier, the adaptive-clustering classifier, suggested by this analysis is compared with BP (modified by using a conjugate-gradient optimization technique) for two problems. It is found that BP usually takes significantly longer to train than the adaptive-clustering technique.>

The software and digital sections of a hybrid optical/digital processor are described with emphasis on the required operations, their realization, and the hierarchy of software and hardware used. Several applications of this interface are also included.

A real-time two-dimensional hybrid processor consisting of a coherent optical system, an optical/digital interface, and a PDP-11/15 control minicomputer is described. The input electrical-to-optical transducer is an electron-beam addressed potassium dideuterium phosphate (KD2PO4) light valve. The requirements and hardware for the output optical-to-digital interface, which is constructed from modular computer building blocks, are presented. Initial experimental results demonstrating the operation of this hybrid processor in phased array radar data processing, synthetic aperture image correlation, and text correlation are included. The applications chosen emphasize the role of the interface in the analysis of data from an optical processor and possible extensions to the digital feedback control of an optical processor.

An investigation of the various cache schemes that are practical for a minicomputer has been found to provide considerable insight into cache organization. Simulations are used to obtain data on the performance and sensitivity of organizational parameters of various writeback and lookahead schemes. Hardware considerations in the construction of the actual cache-minicomputer are also noted and a simple cost/performance analysis is presented.

A brief review of the principles of optical data processing (ODP) with its high throughput data rate and parallel processing potential is given, followed by a description of a viable on-line electron-beam addressed electrical-to-optical input transducer. A proposed on-line hybrid digital/optical two-dimensional processing system is then described. An output plane optical-to-digital interface and several applications of the system to high bit-rate on-line data processing situations are included.