This paper presents our research in multi-class pattern learning from imbalanced data. In many real world applications, the data among different pattern classes are imbalanced; some classes may have far more training data than the others. Typically a neural network classifier has troubles to learn from the imbalanced data distribution among different pattern classes. In this paper we propose a new pattern classification algorithm, One-Against-Higher-Order (OAHO), that effectively learn multi-class patterns from the imbalanced data, and a theoretical analysis of data imbalance problem related to other popular multi-class pattern classification approaches. We have conducted experiments on the two highly imbalanced data sets posted at the UCI site, and the results show that the neural network system trained with the proposed OAHO algorithm gives better performances on minority pattern classes over the neural network systems trained with the two other popular multi-class classification methods: OAO and OAA.

We propose a new technique to improve the confidence in results of repeated neural network training runs under the practical constraint of a fixed computational budget. Our technique is applicable to problems for which there is a correlation between results early in the training process and results near the end of training. Targeting well-studied training problems, the technique may be most valuable when the computational time required for thorough training makes impractical performing a large number of differently initialized training sessions.

We demonstrate that a fixed-weight neural network can be trained with Kalman filter methods to exhibit input-output behavior that depends on which of two conditioning tasks had been performed a substantial number of time steps in the past. This behavior can also be made to survive an intervening interference task.

Support vector machines have been gaining popularity in the research community of pattern classification. In this paper, we investigate efficient and effective algorithms for training SVMs on large data collections. We decompose SVM learning problem into two stages. At the first stage we developed an algorithm that uses a sequence of small subsets of training data to select the parameters /spl gamma/ and C. At the second stage, we developed an algorithm that generates a reliable set of support vectors using a small subset of the training data. Experiments are conducted on about 850K data samples for automotive engine misfire detection.

This paper proposes a novel technique to approximate Lyapunov functions for discrete time autonomous systems using a special form of support vector machine (SVM). We assume that a Lyapunov function can be accurately approximated by a polynomial of arbitrary degree on a finite set of points from trajectories of the closed-loop system. We transform the original problem of linearly constrained quadratic optimization into an equivalent dual problem. For computational tractability, we apply an iterative decomposition of the dual problem. We illustrate our technique on two examples.

Singhal and Wu (1989) introduced the extended Kalman filter (EKF) neural network training algorithm. Since then, many modifications, simplifications and improvements of the EKF procedure have been developed and applied to difficult problems in pattern classification, signal processing and control. Recently, a number of enhancements related to learning rates and square root filtering for EKF training have been proposed, and performance superior to previously reported methods has been claimed. This paper serves to clarify issues related to the proper treatment of learning rates, artificial process noise and square root filtering for EKF training.

We propose an approach to analyze trained neurocontrollers numerically for achieving Lyapunov stability of the closed-loop system. Our approach is based on training an adaptive critic to approximate a Lyapunov function. The trained critic is analyzed numerically to establish regions of stability of the closed-loop system. A simple example illustrates our approach. Our setting is the most relevant for neurocontrollers, but conclusions are applicable to other forms of controllers as well.

We present a coherent neural net based framework for solving various signal processing problems. It relies on the assertion that time-lagged recurrent networks possess the necessary representational capabilities to act as universal approximators of nonlinear dynamical systems. This applies to system identification, time-series prediction, nonlinear filtering, adaptive filtering, and temporal pattern classification. We address the development of models of nonlinear dynamical systems, in the form of time-lagged recurrent neural nets, which can be used without further training. We employ a weight update procedure based on the extended Kalman filter (EKF). Against the tendency for a net to forget earlier learning as it processes new examples, we develop a technique called multistream training. We demonstrate our framework by applying it to 4 problems. First, we show that a single time-lagged recurrent net can be trained to produce excellent one-time-step predictions for two different time series and also to be robust to severe errors in the input sequence. Second, we model stably a complex system containing significant process noise. The remaining two problems are drawn from real-world automotive applications. One involves input-output modeling of the dynamic behavior of a catalyst-sensor system which is exposed to an operating engine's exhaust stream, the other the real-time and continuous detection of engine misfire.

The application of neural network techniques to the control of nonlinear dynamical systems has been the subject of substantial interest and research in recent years. In our own work, we have concentrated on extending the dynamic gradient formalism as established by Narendra and Parthasarathy (1990, 1991), and on employing it for applications in the control of nonlinear systems, with specific emphasis on automotive subsystems. The results we have reported to date, however, have been based exclusively upon simulation studies. In this paper, we establish that dynamic gradient training methods can be successfully used for synthesizing neural network controllers directly on instances of real systems. In particular we describe the application of dynamic gradient methods for training a time-lagged recurrent neural network feedback controller for the problem of engine idle speed control on an actual vehicle, discuss hardware and software issues, and provide representative experimental results.

Although the potential of the powerful mapping and representational capabilities of recurrent network architectures is generally recognized by the neural network research community, recurrent neural networks have not been widely used for the control of nonlinear dynamical systems, possibly due to the relative ineffectiveness of simple gradient descent training algorithms. Developments in the use of parameter-based extended Kalman filter algorithms for training recurrent networks may provide a mechanism by which these architectures will prove to be of practical value. This paper presents a decoupled extended Kalman filter (DEKF) algorithm for training of recurrent networks with special emphasis on application to control problems. We demonstrate in simulation the application of the DEKF algorithm to a series of example control problems ranging from the well-known cart-pole and bioreactor benchmark problems to an automotive subsystem, engine idle speed control. These simulations suggest that recurrent controller networks trained by Kalman filter methods can combine the traditional features of state-space controllers and observers in a homogeneous architecture for nonlinear dynamical systems, while simultaneously exhibiting less sensitivity than do purely feedforward controller networks to changes in plant parameters and measurement noise.

An analysis is presented of the requirements for developing a practical trainable classifier to detect and identify faults in vehicle powertrain systems. An examination is made of requirements on the data sets used for training and testing and the criteria needed to select the most appropriate classifier for a particular family of problems. Empirical results supporting the authors' hypothesis are presented based on an analysis of two data sets drawn under rather different circumstances from test vehicles with faults introduced. Several different classifier forms are applied to these data sets, and their performance is evaluated. Despite similar performance on simple statistical tests, the classifies exhibit significant performance variations on more rigorous tests, and therefore viable criteria for selecting the most appropriate classifiers can be established

A vision guided robot system consisting of a stereo-pair of CCD array cameras mounted on the end-effector of a six-axis revolute robot arm is presented. The accuracy of the vision system, the robot, and the manufacturing requirements is considered from a systems point of view. The development of automated calibration methodologies for local and global work volumes of the robot/vision system is discussed. The accuracy of local calibration which is on the order of 1.5 mm is sufficient for many automotive assembly applications. The assembly strategy is described, and multiple-component assembly and robotic fastening with the vision guided robot is reported.>

The camera model which uses homogeneous coordinates, assumes the laws of Gaussian optics, and an extension for the aberration of geometric radial distortion is included. Features that collectively demonstrate the methodology's superiority over existing techniques are: (1) the simultaneous and accurate inference of all camera model parameters using only iterative linear least squares; (2) no requirement of a priori knowledge of camera model parameters; (3) utilization with a wide range of optical sensing devices; (4) adaptability to include camera model parameters that are departures from the assumptions of Gaussian optics; and (5) computational efficiency. Absolute stereo calibration accuracy of 0.084 mm in a 2000 cm/sup 3/ volume is achieved for a pair of miniaturized CCD array cameras.>

The success of industrial robotic applications involving off-line programming and sensory-based guidance will depend significantly upon the positioning accuracy of robots, the accuracy of sensing devices and the software coupling between the robot controller and the sensors. These accuracy and coupling issues are addressed by a methodology developed for the automatic global calibration of a robot/vision system. The methodology employs a stereo-pair of CCD array cameras, which are mounted to the end-effector of a six-axis revolute robot arm. With an automatic procedure,three-dimensional coordinate measurements are made, relative to the robot's base frame, of a single spherical point in space at numerous and widely varying joint-angle configurations. Based upon a modified Denavit-Hartenburg robot kinematic model, both geometric and nongeometric robotic errors are inferred simultaneously with the geometric errors of the vision system using an iterative least-squares algorithm. Preliminary results indicate an approximately threefold improvement in positioning accuracy of the robot arm.