Matrix-variable optimization is a generalization of vector-variable optimization and has been found to have many important applications. To reduce computation time and storage requirement, this article presents two matrix-form recurrent neural networks (RNNs), one continuous-time model and another discrete-time model, for solving matrix-variable optimization problems with linear constraints. The two proposed matrix-form RNNs have low complexity and are suitable for parallel implementation in terms of matrix state space. The proposed continuous-time matrix-form RNN can significantly generalize existing continuous-time vector-form RNN. The proposed discrete-time matrix-form RNN can be effectively used in blind image restoration, where the storage requirement and computational cost are largely reduced. Theoretically, the two proposed matrix-form RNNs are guaranteed to be globally convergent to the optimal solution under mild conditions. Computed results show that the proposed matrix-form RNN-based algorithm is superior to related vector-form RNN and matrix-form RNN-based algorithms, in terms of computation time.

Because of the robustness and sparsity performance of least absolute deviation (LAD or$l_{1}$) optimization, developing effective solution methods becomes an important topic. Recurrent neural networks (RNNs) are reported to be capable of effectively solving constrained$l_{1}$-norm optimization problems, but their convergence speed is limited. To accelerate the convergence, this article introduces two RNNs, in form of continuous- and discrete-time systems, for solving$l_{1}$-norm optimization problems with linear equality and inequality constraints. The RNNs are theoretically proven to be globally convergent to optimal solutions without any condition. With reduced model complexity, the two RNNs can significantly expedite constrained$l_{1}$-norm optimization. Numerical simulation results show that the two RNNs spend much less computational time than related RNNs and numerical optimization algorithms for linearly constrained$l_{1}$-norm optimization.

In this article, a continuous-time complex-valued projection neural network (CCPNN) in a matrix state space is first proposed for a general complex-variable basis pursuit problem. The proposed CCPNN is proved to be stable in the sense of Lyapunov and to be globally convergent to the optimal solution under the condition that the sensing matrix is not row full rank. Furthermore, an improved discrete-time complex projection neural network (IDCPNN) is proposed by discretizing the CCPNN model. The proposed IDCPNN consists of a two-step stop strategy to reduce the calculational cost. The proposed IDCPNN is theoretically guaranteed to be global convergent to the optimal solution. Finally, the proposed IDCPNN is applied to the reconstruction of sparse signals based on compressed sensing. Computed results show that the proposed IDCPNN is superior to related complex-valued neural networks and conventional basis pursuit algorithms in terms of solution quality and computation time.

Summary With the application of computed tomography (CT) technology to the clinic diagnosis and image‐guided surgeries, various CT image reconstruction algorithms were presented for enhancing CT image quality based on different dose acquisitions. For wide and effective uses, this paper is going to provide a comprehensive survey on representative CT image reconstruction algorithms, in the light of low‐dose and high‐dose acquisitions and algorithm performance, including their merits and limitations. The algorithm performance and complexity are discussed under Gaussian and Poisson noise environments, respectively. Finally, experimental results analyze the efficiency and performance of several popular CT image reconstruction algorithms in terms of computation time and accuracy.

Super resolution image reconstruction under unknown Gaussian blur has been a challenging topic. Advanced optimization-based works for blind image super-resolution (SR) were reported to be effective, but there exist both large data space storage and time consuming due to vector-variable optimization. This paper proposes a matrix-variable optimization method for fast blind image SR. We first present an accurate blur kernel estimation-based matrix decomposition method. Then we propose minimizing a matrix-variable optimization problem with sparse representation and TV regularization terms. The proposed method can exactly estimate the unknown blur kernel and blur matrix. Compared with vector-variable optimization based methods for blind image SR, the proposed method can greatly reduce their data space storage and computation time. Compared with deep learning methods, the proposed method can directly deal with multiframe SR problem without training and learning task. Experimental results show that the proposed algorithm is superior to conventional optimization-based method in terms of solution quality and computation time. Moreover, the proposed method can obtain higher reconstruction quality than the deep learning methods, specially in the case of large blur kernels.

Blind image deblurring has been a challenging issue due to the unknown blur and computation problem. Recently, the matrix-variable optimization method successfully demonstrates its potential advantages in computation. This paper proposes an effective matrix-variable optimization method for blind image deblurring. Blur kernel matrix is exactly decomposed by a direct SVD technique. The blur kernel and original image are well estimated by minimizing a matrix-variable optimization problem with blur kernel constraints. A matrix-type alternative iterative algorithm is proposed to solve the matrix-variable optimization problem. Finally, experimental results show that the proposed blind image deblurring method is much superior to the state-of-the-art blind image deblurring algorithms in terms of image quality and computation time.

Recent reports show that projection neural networks with a low-dimensional state space can enhance computation speed obviously. This paper proposes two projection neural networks with reduced model dimension and complexity (RDPNNs) for solving nonlinear programming (NP) problems. Compared with existing projection neural networks for solving NP, the proposed two RDPNNs have a low-dimensional state space and low model complexity. Under the condition that the Hessian matrix of the associated Lagrangian function is positive semi-definite and positive definite at each Karush-Kuhn-Tucker point, the proposed two RDPNNs are proven to be globally stable in the sense of Lyapunov and converge globally to a point satisfying the reduced optimality condition of NP. Therefore, the proposed two RDPNNs are theoretically guaranteed to solve convex NP problems and a class of nonconvex NP problems. Computed results show that the proposed two RDPNNs have a faster computation speed than the existing projection neural networks for solving NP problems.

Much research has been devoted to complex-variable optimization problems due to their engineering applications. However, the complex-valued optimization method for solving complex-variable optimization problems is still an active research area. This paper proposes two efficient complex-valued optimization methods for solving constrained nonlinear optimization problems of real functions in complex variables, respectively. One solves the complex-valued nonlinear programming problem with linear equality constraints. Another solves the complex-valued nonlinear programming problem with both linear equality constraints and an -norm constraint. Theoretically, we prove the global convergence of the proposed two complex-valued optimization algorithms under mild conditions. The proposed two algorithms can solve the complex-valued optimization problem completely in the complex domain and significantly extend existing complex-valued optimization algorithms. Numerical results further show that the proposed two algorithms have a faster speed than several conventional real-valued optimization algorithms.

The robust Huber's M-estimator is widely used in signal and image processing, classification, and regression. From an optimization point of view, Huber's M-estimation problem is often formulated as a large-sized quadratic programming (QP) problem in view of its nonsmooth cost function. This paper presents a generalized regression estimator which minimizes a reduced-sized QP problem. The generalized regression estimator may be viewed as a significant generalization of several robust regression estimators including Huber's M-estimator. The performance of the generalized regression estimator is analyzed in terms of robustness and approximation accuracy. Furthermore, two low-dimensional recurrent neural networks (RNNs) are introduced for robust estimation. The two RNNs have low model complexity and enhanced computational efficiency. Finally, the experimental results of two examples and an application to image restoration are presented to substantiate superior performance of the proposed method over conventional algorithms for robust regression estimation in terms of approximation accuracy and convergence rate.

This paper proposes a novel blind hyperspectral image super resolution method. The proposed method can estimate simultaneously the unknown hyperspectral image and the blur kernel based on the linear spectral unmixing technique. The total variation term is used for the blur kernel regularization and simultaneous total variation and sparse representation are used for abundance regularization terms. Because the image and blur kernel are simultaneously estimated with the double reg-ularization terms introduced for abundance, the estimation error can be minimized so that the performance of the proposed method can be improved. Finally, the proposed optimization formulation is effectively solved by block coordinate descent method. Experimental results show that the proposed method is effective and superior to existing blind hyperspectral image super resolution approach in terms of reconstruction quality.

This letter proposes a novel double regularization unmixing-based method for hyperspectral image (HSI) superresolution. The proposed cost function contains two data-fidelity terms, the endmember regularization term and the abundance regularization term. Since the double regularization unmixing terms are able to exploit the spatial structure information of endmember and abundance, respectively, the nonnegative factorization (spectral unmixing) error is minimized. As a result, the performance of the proposed HSI superresolution method can be enhanced in terms of noise suppression and the special structure information preservation of reconstruction images. Finally, the associated optimization problem is effectively solved by an alternating direction optimization algorithm. Simulation results illustrate that the proposed method has a better performance than the state-of-the-art methods in terms of both visual effectiveness and quality indices.

In this paper, we propose two fast complex-valued optimization algorithms for solving complex quadratic programming problems: 1) with linear equality constraints and 2) with both an l1-norm constraint and linear equality constraints. By using Brandwood's analytic theory, we prove the convergence of the two proposed algorithms under mild assumptions. The two proposed algorithms significantly generalize the existing complex-valued optimization algorithms for solving complex quadratic programming problems with an l1-norm constraint only and unconstrained complex quadratic programming problems, respectively. Numerical simulations are presented to show that the two proposed algorithms have a faster speed than conventional real-valued optimization algorithms.

In this paper, a bi-projection neural network for solving a class of constrained quadratic optimization problems is proposed. It is proved that the proposed neural network is globally stable in the sense of Lyapunov, and the output trajectory of the proposed neural network will converge globally to an optimal solution. Compared with existing projection neural networks (PNNs), the proposed neural network has a very small model size owing to its bi-projection structure. Furthermore, an application to data fusion shows that the proposed neural network is very effective. Numerical results demonstrate that the proposed neural network is much faster than the existing PNNs.

Recurrent neural networks, as neurodynamic systems, are a class of connectionist models that capture the dynamics of sequences via cycles in artificial neurons. Since the invention of Hopfield neural network, recurrent neural networks have attracted considerable attention, which marks the beginning of the modern age of neural network studies. Thanks to their inherent nature of parallel and distributed information processing, many computationally intensive applications can be solved by recurrent neural networks in the real-time environment.

The problem of identification of autoregressive (AR) signals with noisy measurements is considered. A new algorithm is proposed to estimate the AR parameters. To cope with the effect of the measurement noise that causes a bias in the least-squares estimate of the AR parameters, an efficient procedure is developed for estimating the measurement noise variance. The proposed identification algorithm is implemented via the Newton iterative scheme and is able to produce better parameter estimates. A numerical example is presented to show the efficiency of the new identification algorithm for noisy AR signals.

In this paper, we present a complex-valued projection neural network for solving constrained convex optimization problems of real functions with complex variables, as an extension of real-valued projection neural networks. Theoretically, by developing results on complex-valued optimization techniques, we prove that the complex-valued projection neural network is globally stable and convergent to the optimal solution. Obtained results are completely established in the complex domain and thus significantly generalize existing results of the real-valued projection neural networks. Numerical simulations are presented to confirm the obtained results and effectiveness of the proposed complex-valued projection neural network.

This paper proposes a fast discrete-time learning algorithm for speech enhancement of single-channel noisy speech signal, based on a noise constrained least squares estimate. Unlike existing learning algorithms for the noise constrained estimate, the proposed discrete-time learning algorithm has a low complexity and fast speed. Simulation results show that the proposed discrete-time learning algorithm has a faster speed than the existing learning algorithms for speech enhancement. Moreover, the proposed discrete-time learning algorithm has a good performance in having a significant gain in SNR at colored noise.

In this letter, we propose a novel iterative method for computing generalized inverse, based on a novel KKT formulation. The proposed iterative algorithm requires making four matrix and vector multiplications at each iteration and thus has low computational complexity. The proposed method is proved to be globally convergent without any condition. Furthermore, for fast computing generalized inverse, we present an acceleration scheme based on the proposed iterative method. The global convergence of the proposed acceleration algorithm is also proved. Finally, the effectiveness of the proposed iterative algorithm is evaluated numerically.

There is growing interest in solving linear L1 estimation problems for sparsity of the solution and robustness against non-Gaussian noise. This paper proposes a discrete-time neural network which can calculate large linear L1 estimation problems fast. The proposed neural network has a fixed computational step length and is proved to be globally convergent to an optimal solution. Then, the proposed neural network is efficiently applied to image restoration. Numerical results show that the proposed neural network is not only efficient in solving degenerate problems resulting from the nonunique solutions of the linear L1 estimation problems but also needs much less computational time than the related algorithms in solving both linear L1 estimation and image restoration problems.

The constrained L(1) estimation is an attractive alternative to both the unconstrained L(1) estimation and the least square estimation. In this letter, we propose a cooperative recurrent neural network (CRNN) for solving L(1) estimation problems with general linear constraints. The proposed CRNN model combines four individual neural network models automatically and is suitable for parallel implementation. As a special case, the proposed CRNN includes two existing neural networks for solving unconstrained and constrained L(1) estimation problems, respectively. Unlike existing neural networks, with penalty parameters, for solving the constrained L(1) estimation problem, the proposed CRNN is guaranteed to converge globally to the exact optimal solution without any additional condition. Compared with conventional numerical algorithms, the proposed CRNN has a low computational complexity and can deal with the L(1) estimation problem with degeneracy. Several applied examples show that the proposed CRNN can obtain more accurate estimates than several existing algorithms.

Recently the extended projection neural network was proposed to solve constrained monotone variational inequality problems and a class of constrained nonmonotontic variational inequality problems. Its exponential convergence was developed under the positive definiteness condition of the Jacobian matrix of the nonlinear mapping. This note proposes new results on the exponential convergence of the output trajectory of the extended projection neural network under the weak conditions that the Jacobian matrix of the nonlinear mapping may be positive semidefinite or not. Therefore, new results further demonstrate that the extended projection neural network has a fast convergence rate when solving a class of constrained monotone variational inequality problems and nonmonotonic variational inequality problems. Illustrative examples show the significance of the obtained results.

This paper presents a novel recurrent neural network for solving nonlinear optimization problems with inequality constraints. Under the condition that the Hessian matrix of the associated Lagrangian function is positive semidefinite, it is shown that the proposed neural network is stable at a Karush-Kuhn-Tucker point in the sense of Lyapunov and its output trajectory is globally convergent to a minimum solution. Compared with variety of the existing projection neural networks, including their extensions and modification, for solving such nonlinearly constrained optimization problems, it is shown that the proposed neural network can solve constrained convex optimization problems and a class of constrained nonconvex optimization problems and there is no restriction on the initial point. Simulation results show the effectiveness of the proposed neural network in solving nonlinearly constrained optimization problems.

This paper proposes a generalized least absolute deviation (GLAD) method for parameter estimation of autoregressive (AR) signals under non-Gaussian noise environments. The proposed GLAD method can improve the accuracy of the estimation of the conventional least absolute deviation (LAD) method by minimizing a new cost function with parameter variables and noise error variables. Compared with second- and high-order statistical methods, the proposed GLAD method can obtain robustly an optimal AR parameter estimation without requiring the measurement noise to be Gaussian. Moreover, the proposed GLAD method can be implemented by a cooperative neural network (NN) which is shown to converge globally to the optimal AR parameter estimation within a finite time. Simulation results show that the proposed GLAD method can obtain more accurate estimates than several well-known estimation methods in the presence of different noise distributions.

Identification of a general nonlinear noisy system viewed as an estimation of a predictor function is studied in this article. A measurement fusion method for the predictor function estimate is proposed. In the proposed scheme, observed data are first fused by using an optimal fusion technique, and then the optimal fused data are incorporated in a nonlinear function estimator based on a robust least squares support vector machine (LS-SVM). A cooperative learning algorithm is proposed to implement the proposed measurement fusion method. Compared with related identification methods, the proposed method can minimize both the approximation error and the noise error. The performance analysis shows that the proposed optimal measurement fusion function estimate has a smaller mean square error than the LS-SVM function estimate. Moreover, the proposed cooperative learning algorithm can converge globally to the optimal measurement fusion function estimate. Finally, the proposed measurement fusion method is applied to ARMA signal and spatial temporal signal modeling. Experimental results show that the proposed measurement fusion method can provide a more accurate model.

To deal with the problem of restoring degraded images with non-Gaussian noise, this paper proposes a novel cooperative neural fusion regularization (CNFR) algorithm for image restoration. Compared with conventional regularization algorithms for image restoration, the proposed CNFR algorithm can relax need of the optimal regularization parameter to be estimated. Furthermore, to enhance the quality of restored images, this paper presents a cooperative neural fusion (CNF) algorithm for image fusion. Compared with existing signal-level image fusion algorithms, the proposed CNF algorithm can greatly reduce the loss of contrast information under blind Gaussian noise environments. The performance analysis shows that the proposed two neural fusion algorithms can converge globally to the robust and optimal image estimate. Simulation results confirm that in different noise environments, the proposed two neural fusion algorithms can obtain a better image estimate than several well known image restoration and image fusion methods.

A cooperative recurrent neural network (CRNN) algorithm for parameter estimation of autoregressive (AR) signals is proposed in this paper. The proposed CRNN algorithm is based on a generalized least absolute deviation (GLAD) method, which generalizes significantly the conventional least absolute deviation method. Compared with second-order and high-order statistic algorithms, the proposed CRNN algorithm can obtain robustly an optimal AR parameter estimation without requiring measurement Gaussian noise. Unlike existing cooperative neural network algorithms, the proposed CRNN algorithm has a global convergence and a novel weighting cooperation scheme to integrate single neural network output automatically. Simulation results shows that the more accurate estimates can be attained by the proposed CRNN algorithm in the presence of non-Gaussian colored noise.

A novel cooperative neural learning (CNL) algorithm based on a new linearly constrained least absolute deviation (LCLAD) method for data fusion is proposed in this paper. The state model of the proposed CNL algorithm combines adaptively three recurrent modular neural networks and is sample for implementation using both software and hardware. Unlike the conventional LAD approach, the propose LCLAD method can obtain the optimal fusion solution. Compared with the minimum variance method and linearly constrained least square method, the proposed LCLAD method can minimize an augmented least absolute deviation energy of the linearly fused information and has the robustness performance in non-Gaussian noise environments. Illustrative examples of signal and image fusion show that the quality of the solution can be more enhanced by the proposed CNL algorithm.

The problem of spatial-temporal signal processing and modeling has been of great interest in recent years. A new spatial-temporal prediction method is presented in this paper. An optimal fusion scheme based on fourth-order statistic is first employed to combine the received signals at different spatial domains. The fused signal is then used to construct a spatial-temporal predictor by a support vector machine. It is shown theoretically that the proposed method has an improved performance even in non-Gaussian environments. To demonstrate the practicality of this spatial-temporal predictor, we apply it to model real-life radar sea scattered signals. Experimental results show that the proposed method can provide a more accurate model for sea clutter than the conventional methods.

The output trajectory convergence of an extended projection neural network was developed under the positive definiteness condition of the Jacobian matrix of nonlinear mapping. This note offers several new convergence results. The state trajectory convergence and the output trajectory convergence of the extended projection neural network are obtained under the positive semidefiniteness condition of the Jacobian matrix. Comparison and illustrative examples demonstrate applied significance of these new results.

In this paper, we present a delayed neural network approach to solve linear projection equations. The Lyapunov-Krasovskii theory for functional differential equations and the linear matrix inequality (LMI) approach are employed to analyze the global asymptotic stability and global exponential stability of the delayed neural network. Compared with the existing linear projection neural network, theoretical results and illustrative examples show that the delayed neural network can effectively solve a class of linear projection equations and some quadratic programming problems.

In this paper, we propose a recurrent neural network for solving nonlinear convex programming problems with linear constraints. The proposed neural network has a simpler structure and a lower complexity for implementation than the existing neural networks for solving such problems. It is shown here that the proposed neural network is stable in the sense of Lyapunov and globally convergent to an optimal solution within a finite time under the condition that the objective function is strictly convex. Compared with the existing convergence results, the present results do not require Lipschitz continuity condition on the objective function. Finally, examples are provided to show the applicability of the proposed neural network.

This paper proposes a primal-dual neural network with a one-layer structure for online resolution of constrained kinematic redundancy in robot motion control. Unlike the Lagrangian network, the proposed neural network can handle physical constraints, such as joint limits and joint velocity limits. Compared with the existing primal-dual neural network, the proposed neural network has a low complexity for implementation. Compared with the existing dual neural network, the proposed neural network has no computation of matrix inversion. More importantly, the proposed neural network is theoretically proved to have not only a finite time convergence, but also an exponential convergence rate without any additional assumption. Simulation results show that the proposed neural network has a faster convergence rate than the dual neural network in effectively tracking for the motion control of kinematically redundant manipulators.

Variational inequalities with nonlinear inequality constraints are widely used in optimization and engineering problems. This paper present a recurrent neural network for solving variational inequalities with nonlinear inequality constraints in real time. The proposed neural network has one-layer structure and is amenable to parallel implementation. The proposed neural network is a significant generalization of several existing neural networks for optimization. Moreover, the proposed neural network is stable in the sense of Lyapunov and globally convergent to an optimal solution under a strictly monotone condition of the mapping. The simulation shows that the proposed neural network is effective for solving this class of variational inequality problems.

Recently, a projection neural network has been shown to be a promising computational model for solving variational inequality problems with box constraints. This letter presents an extended projection neural network for solving monotone variational inequality problems with linear and nonlinear constraints. In particular, the proposed neural network can include the projection neural network as a special case. Compared with the modified projection-type methods for solving constrained monotone variational inequality problems, the proposed neural network has a lower complexity and is suitable for parallel implementation. Furthermore, the proposed neural network is theoretically proven to be exponentially convergent to an exact solution without a Lipschitz condition. Illustrative examples show that the extended projection neural network can be used to solve constrained monotone variational inequality problems.

Recently, a projection neural network for solving monotone variational inequalities and constrained optimization problems was developed. In this paper, we propose a general projection neural network for solving a wider class of variational inequalities and related optimization problems. In addition to its simple structure and low complexity, the proposed neural network includes existing neural networks for optimization, such as the projection neural network, the primal-dual neural network, and the dual neural network, as special cases. Under various mild conditions, the proposed general projection neural network is shown to be globally convergent, globally asymptotically stable, and globally exponentially stable. Furthermore, several improved stability criteria on two special cases of the general projection neural network are obtained under weaker conditions. Simulation results demonstrate the effectiveness and characteristics of the proposed neural network.

Grasping-force optimization of multifingered robotic hands can be formulated as a problem for minimizing an objective function subject to form-closure constraints and balance constraints of external force. This paper presents a novel recurrent neural network for real-time dextrous hand-grasping force optimization. The proposed neural network is shown to be globally convergent to the optimal grasping force. Compared with existing approaches to grasping-force optimization, the proposed neural-network approach has the advantages that the complexity for implementation is reduced, and the solution accuracy is increased, by avoiding the linearization of quadratic friction constraints. Simulation results show that the proposed neural network can achieve optimal grasping force in real time.

This paper presents a one-layer recurrent neural network for support vector machine (SVM) learning in pattern classification and regression. The SVM learning problem is first converted into an equivalent formulation, and then a one-layer recurrent neural network for SVM learning is proposed. The proposed neural network is guaranteed to obtain the optimal solution of support vector classification and regression. Compared with the existing two-layer neural network for the SVM classification, the proposed neural network has a low complexity for implementation. Moreover, the proposed neural network can converge exponentially to the optimal solution of SVM learning. The rate of the exponential convergence can be made arbitrarily high by simply turning up a scaling parameter. Simulation examples based on benchmark problems are discussed to show the good performance of the proposed neural network for SVM learning.

In this paper, we propose a general projection neural network for solving a wider class of optimization and related problems. In addition to its simple structure and low complexity, the proposed neural network include existing neural networks for optimization, such as the projection neural network, the primal-dual neural network, and the dual neural network, as special cases. Under various mild conditions, the proposed general projection neural network is shown to be globally convergent, globally asymptotically stable, and globally exponentially stable. Furthermore, several improved stability criteria on two special cases of the general projection neural network are obtained under weaker conditions. Simulation results demonstrate the effectiveness and characteristics of the proposed neural network.

In this paper, a recurrent neural network called the dual neural network is proposed for online redundancy resolution of kinematically redundant manipulators. Physical constraints such as joint limits and joint velocity limits, together with the drift-free criterion as a secondary task, are incorporated into the problem formulation of redundancy resolution. Compared to other recurrent neural networks, the dual neural network is piecewise linear and has much simpler architecture with only one layer of neurons. The dual neural network is shown to be globally (exponentially) convergent to optimal solutions. The dual neural network is simulated to control the PA10 robot manipulator with effectiveness demonstrated.

Two novel neural data fusion algorithms based on a linearly constrained least square (LCLS) method are proposed. While the LCLS method is used to minimize the energy of the linearly fused information, two neural-network algorithms are developed to overcome the ill-conditioned and singular problems of the sample covariance matrix arisen in the LCLS method. The proposed neural fusion algorithms are samples for implementation using both software and hardware. Compared with the existing fusion methods, the proposed neural data fusion method has an unbiased statistical property and does not require any a priori knowledge about the noise covariance. It is shown that the proposed neural fusion algorithms converge globally to the optimal fusion solution when the sample covariance matrix is singular, and converge globally with exponential rate when the sample covariance matrix is nonsingular. We apply the proposed neural fusion method to image and signal fusion, and it is shown that the quality of the solution can be greatly enhanced by the proposed technique.

The inverse kinematics problem in robotics can be formulated as a time-varying quadratic optimization problem. A new recurrent neural network, called the dual network, is presented in this paper. The proposed neural network is composed of a single layer of neurons, and the number of neurons is equal to the dimensionality of the workspace. The proposed dual network is proven to be globally exponentially stable. The proposed dual network is also shown to be capable of asymptotic tracking for the motion control of kinematically redundant manipulators.

We apply a recurrent neural network to support vector machine (SVM) training for pattern recognition. Specifically, a primal-dual neural network is exploited to solve the quadratic programming problem encountered in training SVMs. The properties of the network allow one to design SVMs without adjustable network parameters and give a better solution for ill-posed problems.

In this paper, we are concerned with the nonlinear projection equations of the following form P/sub /spl chi//(u-F(u))=u. Despite the particular structure of the feasible set /spl chi/, the problem is still a very general problem in mathematics programming. Moreover, there are a number of important applications which lead to this special class of variational inequalities such as equilibrium models arising in fields of economics and transportation science, etc. Various numerical solution procedures for the problem have been investigated over decades. Because of the nature of digital computers, conventional algorithms are time-consuming for large-scale optimization problems. It is well-known that one promising approach to optimization problems in real time is to employ artificial neural networks implemented in hardware. Recurrent neural networks for solving optimization problems are readily hardware-implementable. Thus, neural networks are a top choice of real-time solvers for optimization problems. Since the seminal work of Hopfield and Tank (1985), the neural network approach to optimization has been investigated and many neural networks for optimization problems have been proposed.

Global exponential stability is a desirable property for dynamic systems. This paper studies the global exponential stability of several existing recurrent neural networks for solving linear programming problems, convex programming problems with interval constraints, convex programming problems with nonlinear constraints, and monotone variational inequalities. In contrast to the existing results on global exponential stability, the present results do not require additional conditions on the weight matrices of recurrent neural networks and improve some existing conditions for global exponential stability. Therefore, the stability results in this paper further demonstrate the superior convergence properties of the existing neural networks for optimization.

We propose a dual neural network with globally exponential stability for solving quadratic programming problems with unique solutions. Compared with the Bouzerdoum-Pattison network (1993), there is no need for choosing the self-feedback or lateral connection matrices in the present network. Moreover, the size of the dual network is less than that of the original problem.

We propose two primal neural networks with globally exponential stability for solving quadratic programming problems. Both the self-feedback and lateral connection matrices in the present network are compared with the Bonzerdoum-Pattison network (1993). Moreover, the size of the proposed networks is same as that of the original problem, smaller than that of primal-dual networks.

The assignment problem is an archetypical combinatorial optimization problem having widespread applications. This paper presents two recurrent neural networks, a continuous-time one and a discrete-time one, for solving the assignment problem. Because the proposed recurrent neural networks solve the primal and dual assignment problems simultaneously, they are called primal-dual assignment networks. The primal-dual assignment networks are guaranteed to make optimal assignment regardless of initial conditions. Unlike the primal or dual assignment network, there is no time-varying design parameter in the primal-dual assignment networks. Therefore, they are more suitable for hardware implementation. The performance and operating characteristics of the primal-dual assignment networks are demonstrated by means of illustrative examples.

In this paper, we present a general methodology for designing optimization neural networks. We prove that the neural networks constructed by using the proposed method are guaranteed to be globally convergent to solutions of problems with bounded or unbounded solution sets, in contrast with the gradient methods whose convergence is not guaranteed. We show that the proposed method contains both the gradient methods and nongradient methods employed in existing optimization neural networks as special cases. Based on the theoretical results of the proposed method, we study the convergence and stability of general gradient models in case of unisolated solutions. Using the proposed method, we derive some new neural network models for a very large class of optimization problems, in which the equilibrium points correspond to exact solutions and there is no variable parameter. Finally, some numerical examples show the effectiveness of the method.

Presents a new neural network which improves existing neural networks for solving general linear programming problems. The network, without setting parameter, uses only simple hardware in which no analog multipliers are required, and is proved to be completely stable to the exact solutions. Moreover, using this network the author can solve linear programming problems and its dual simultaneously, and cope with problems with nonunique solutions whose set is allowed to be unbounded.

A new neural network for solving linear and quadratic programming problems is presented and is shown to be globally convergent. The new neural network improves existing neural networks for solving these problems: it avoids the parameter turning problem, it is capable of achieving the exact solutions, and it uses only simple hardware in which no analog multipliers for variables are required. Furthermore, the network solves both the primal problems and their dual problems simultaneously.

A new neural network for solving linear programming problems with bounded variables is presented. The network is shown to be completely stable and globally convergent to the solutions to the linear programming problems. The proposed new network is capable of achieving the exact solutions, in contrast to existing optimization neural networks which need a suitable choice of the network parameters and thus can obtain only approximate solutions. Furthermore, both the primal problems and their dual problems are solved simultaneously by the new network.