This article addresses a collective heterogeneous multiagent pursuit-evasion (MPE) game problem where pursuers cooperatively capture escaping evaders. The analytical challenge of the present design lies in solving the associated coupled Hamilton-Jacobi-Isaacs (HJI) equations induced by the additional interacting roles in the MPE game while ensuring the achievement of the Nash equilibrium. To tackle this issue, a gaming framework is accordingly proposed to solve the coupled HJI equations. Sufficient conditions are derived to guarantee both the capturability and Nash equilibrium of the proposed collective MPE gaming scheme. Finally, numerical simulations are conducted to verify the effectiveness of the present MPE gaming strategy.

A hybrid Koopman deep learning algorithm is developed to predict the final synchronization of networked nonlinear dynamics with different topologies merely using neighboring state information. This algorithm introduces a nonlinear encoder as an observable function that maps the nonlinear state into a high-dimensional Hilbert space. By this means, a networked linear model is established to predict the future state of multiple transformed linear systems in the lifted space. Meanwhile, a nonlinear decoder is constructed, as the inverse of the lifting function, to retrieve the original nonlinear states. The virtue of the present algorithm lies in distilling and merging the linear features of multiple different topologies solely from the individual and/or neighboring state series. Therefore, the final synchronization states are calculated within the encoded linear space and subsequently decoded to recover the synchronization of the original nonlinear systems. Compared to most existing relevant algorithms that could only predict consensus values for linear networks, the present method could predict the final synchronization state of networked nonlinear dynamics with varying backbones. Sufficient conditions are derived to guarantee the prediction capability of the distributed final synchronization prediction (DFSP). Extensive numerical simulations verify its effectiveness.

Time-series images with rich spatiotemporal features contain comprehensive and accurate context information for image segmentation. Due to the variability of time-series images, a random offset phenomenon may occur in targets, interfering with the continuity of temporal features. Although windowed attention mechanisms are adopted to capture the complete image information, they are prone to triggering the edge-jagged phenomenon. To address the above issues, this article presents a Swin transformer with spatiotemporal feature correction (SwinTSFC) for the semantic segmentation of time-series images. A convolutional long-short-term memory (ConvLSTM) module with dynamic correction is proposed to adjust the target deviation of temporal data by capturing the offset relationship among sequences. It learns image semantic association and maintains object alignment among dynamic data. A global-to-local learning strategy is adopted to extract spatial features. Swin transformer blocks are adopted to capture the long-range dependencies of images by strengthening interaction capabilities among windows and to improve the overall recognition ability of SwinTSFC. Self-calibrated convolution (SCConv) adaptively extracts fine-grained information to optimize edge continuity features and overcome the phenomenon of edge-jagged. The superiority of the SwinTSFC to state-of-the-art algorithms is demonstrated via experimentation. The code is available at: https://github.com/fjc1575/Marine-Aquaculture/tree/main/SwinTSFC

In this article, we propose a constrained optimization approach to portfolio selection by maximizing nine risk-adjusted return metrics, subject to second-order stochastic dominance (SSD) constraints. The SSD constraints ensure that the portfolio returns are no less than an amplified proportion of returns from a benchmark in the sense of SSD. Because the number of SSD constraints is extremely large, the resulting constrained optimization problems are computationally challenging. To reduce computational complexity, we develop an efficient algorithm to solve the problems iteratively by incrementally adding SSD constraints. We experimentally demonstrate the superiority of the proposed approaches to several baselines in terms of out-of-sample performance criteria based on financial data from major world stock markets.

As public awareness of privacy protection increases and data become more valuable, the applications of federated learning (FL) in the emerging field of Artificial Internet of Things (AIoT) has received widespread attention. Meanwhile, differential privacy (DP), providing strict privacy guarantees, has been introduced to meet users’ stringent privacy protection needs and increasingly sound laws and regulations. However, the implementations of DP in multiple iterations and rounds of FL training, as well as adding noise to all parameters without differentiation, will cause noise accumulation, resulting in the FL system to decline in performance or even fail to converge. To address the issue, FL with denoising DP and sparsification (DDPS-FL) is proposed in this article. First, a local denoising mechanism (LDM) suitable for DP with arbitrary noise adding mechanism is proposed. By removing the previously added noise from global models, LDM achieves direct noise reduction for clients. Second, sparsification based on parameter variation (SPV) is proposed to reduce noise indirectly by deleting nonsignificant parameters without compromising the level of privacy protection. Besides, SPV is able to multiple beneficial effects, such as saving privacy budget, stimulating the dynamism of FL training, amplifying privacy protection effect, and improving communication efficiency. Third, theoretical analysis is performed to prove that DDPS-FL can guarantee user privacy and has ideal convergence, and to analyze the impact of parameters, such as the number of training rounds and iterations on the system performance. Evaluation experiments based on four real-world datasets are elaborated to show that DDPS-FL outperforms state-of-the-art schemes in terms of training stability, model accuracy, and communication efficiency, and its performance becomes relatively better when more noise is added.

In this article, the classic portfolio selection problem is reformulated as nine convex optimization problems to maximize nine risk-adjusted performance indexes based on nine different risk measures in Markowitz's return-risk framework. The exact convex reformulations facilitate a decision maker to optimize portfolios efficiently by maximizing one of the nine risk-adjusted performance criteria using widely available convex optimization problem solvers, without compromising the portfolio optimality. The superior performances of the proposed approaches to the state-of-the-art methods, in terms of out-of-sample risk-adjusted returns, annualized returns, and portfolio sparsity, are demonstrated through extensive experimentation on 13 datasets from major world stock markets.

Meta-learning provides a promising solution to the issue of insufficient training samples in Pumi spectrogram recognition. However, capturing model uncertainty remains a critical challenge, particularly for tasks influenced by lexical ambiguities. To overcome this problem, we propose a novel method, Synthetic Gradient Optimization-Based Implicit Amortized Bayesian Meta-Learning (SGO-IABML), which captures model uncertainty by evaluating posterior distributions within a hierarchical Bayesian framework, thereby facilitating few-shot Pumi spectrogram recognition. Specifically, SGO-IABML reformulates meta-learning as a bi-level variational inference problem, leveraging information bottleneck principles. At the lower level, a generative inference module is developed to implicitly model task-specific variational posteriors, thereby enhancing the model’s expressiveness. Given the lack of analytical forms for implicit distributions, we derive the Fenchel-Bayesian Bound Theorem to measure the divergence between arbitrary distributions. For the meta-learning of variational parameters, SGO-IABML constructs a synthetic gradient optimizer, integrating prior gradient information to facilitate rapid adaptation to new tasks. At the upper level, the model is calibrated by estimating the local geometry of the posterior distribution, utilizing the Generalized Gauss-Newton Matrix to capture the directional sensitivity of the loss function. Comprehensive experimental results on Pumi spectrograms demonstrate that SGO-IABML achieves state-of-the-art performance in generalization, calibration, expressiveness, versatility, and cross-domain adaptability. Furthermore, ablation studies confirm the contribution of each component to the overall performance improvement.

Index tracking is a primary passive investment strategy. Many existing methods, such as cardinality-constrained and regularized regressions, need to prespecify parameters to generate sparse portfolios to track indices, which complicates the tracking procedure and may compromise tracking performance. This article addresses index tracking and enhanced index tracking via Bayesian learning and collaborative neurodynamic optimization. Specifically, we formulate a sparse Bayesian regression problem for index tracking. Furthermore, we reformulate the problem for enhanced index tracking by adding constraints based on a second-order stochastic domination rule. To overcome the nonconvexity of the objective function in the formulated problems, we propose a sparse Bayesian regression algorithm based on multiple recurrent neural networks in the collaborative neurodynamic optimization framework. We demonstrate the superiority of the proposed methods to mainstream baselines in terms of predictability, consistency, sparsity, and profitability via experimentation on the data from seven major stock markets.

As a primary passive investment strategy, index tracking replicates the performance of a specific financial market index by minimizing tracking errors. Most existing index tracking methods are developed based on the assumption that all historical data are equally important. As a result, the importance of different historical data may be overlooked. This article addresses index tracking via temporally weighted least-squares regression. The weight for each time period except for the latest one is defined as the reciprocal of the largest absolute residual of the returns between the index currently and all the selected stocks in the subsequent periods. The weight for the latest period is inferred from the weights in the preceding periods via Gaussian process regression. The tracking accuracy and consistency of the proposed approach are demonstrated via experimentation on historical data from seven major stock markets.

Meta-learning aims to leverage prior knowledge from related tasks to enable a base learner to quickly adapt to new tasks with limited labeled samples. However, traditional meta-learning methods have limitations as they provide an optimal initialization for all new tasks, disregarding the inherent uncertainty induced by few-shot tasks and impeding task-specific self-adaptation initialization. In response to this challenge, this article proposes a novel probabilistic meta-learning approach called prototype Bayesian meta-learning (PBML). PBML focuses on meta-learning variational posteriors within a Bayesian framework, guided by prototype-conditioned prior information. Specifically, to capture model uncertainty, PBML treats both meta- and task-specific parameters as random variables and integrates their posterior estimates into hierarchical Bayesian modeling through variational inference (VI). During model inference, PBML employs Laplacian estimation to approximate the integral term over the likelihood loss, deriving a rigorous upper-bound for generalization errors. To enhance the model's expressiveness and enable task-specific adaptive initialization, PBML proposes a data-driven approach to model the task-specific variational posteriors. This is achieved by designing a generative model structure that incorporates prototype-conditioned task-dependent priors into the random generation of task-specific variational posteriors. Additionally, by performing latent embedding optimization, PBML decouples the gradient-based meta-learning from the high-dimensional variational parameter space. Experimental results on benchmark datasets for few-shot image classification illustrate that PBML attains state-of-the-art or competitive performance when compared to other related works. Versatility studies demonstrate the adaptability and applicability of PBML in addressing diverse and challenging few-shot tasks. Furthermore, ablation studies validate the performance gains attributed to the inference and model components.

The trust-region method and projection neural networks are two branches of optimization approaches with different operational principles and characteristics. In this article, a trust-region projection neural network (TRPNN) is proposed by integrating the trust-region method and projection neural networks. TRPNN is a discrete-time neurodynamic optimization model that inherits the exploration-exploitation capability of the trust-region method and the local search capability of projection neural networks. TRPNN is theoretically proven to be convergent to a Karush-Kuhn-Tuchker (KKT) point of nonlinear programming problems. The efficacy of TRPNNs leveraged in a collaborative neurodynamic framework is numerically demonstrated for global optimization in the presence of nonconvexity in objective functions or constraints.

Identifying structures of complex networks based on time series of nodal data is of considerable interest and significance in many fields of science and engineering. This article presents a sparse Bayesian learning (SBL) method for identifying structures of community-bridge networks, where nodes are grouped to form communities connected via bridges. Using the structural information of such networks with unknown nodal dynamics and community formations, network structure identification is tackled similar to sparse signal reconstruction with mixed sparsity mode. The proposed method is theoretically proved to be convergent. Its superiority to mainstream baselines is demonstrated via extensive experiments without the need for manual adjustment of regularization parameters.

This article proposes an adaptive intermittent control and an adaptive optimal control for quarter-vehicle active suspension systems. This article leverages integral-type barrier Lyapunov functions (BLFs) to ensure that the vertical displacement and vertical displacement velocity always satisfy state-related constraints. In order to stabilize the vehicle’s attitude and improve passenger comfort, an adaptive intermittent control method is designed to seek the dwell-time condition, achieving a balance between controllable and uncontrollable subsystems. In addition, to reduce the power consumption of the control input, an adaptive optimal control method is designed by designing optimal cost functions and employing the backstepping algorithm under the framework of actor–critic neural networks (critic NNs). The stability of the quarter-vehicle active suspension system is analyzed based on the Lyapunov theory. Finally, simulation results demonstrate significant effects on passenger comfort and reduced control input power for both control methods.

Fully convolutional encoder-decoder networks have been developed for the segmentation of sensing synthetic aperture radar (SAR) images. A recent one called the multiscaled attention U-net with dilated convolution and offset convolution (MDOAU-net) has been proposed for SAR image segmentation in aquaculture raft monitoring. Despite its excellent performance, its hyperparameters have to be handcrafted based on human experience, consuming a significant amount of time to tune. In this letter, a swarm intelligence algorithm is leveraged to optimize the hyperparameters of fully convolutional encoder-decoder networks (particularly MDOAU-net), including their kernel size, dilation rate, learning rate, batch size, and activation function indicator. Based on segmentation performance, early-stop termination criteria are introduced into a particle swarm optimization (PSO) algorithm to avoid overusing computing resources to train the networks. Specifically, the hyperparameters are optimized using the PSO algorithm with early-stop termination criteria. Experimental results show that the segmentation accuracy of the proposed method reaches 91.49%, which statistically outperforms other methods.

This article addresses the safety-critical receding-horizon planning and formation control of autonomous surface vehicles (ASVs) in the presence of model uncertainties, environmental disturbances, as well as stationary and moving obstacles. A three-level formation control architecture is proposed with a safety-critical formation trajectory generation module at its high level, a collision-free guidance module at its middle level, and an anti-disturbance control module at its low level. Specifically, a safety-critical formation trajectory generator is designed by leveraging collaborative neurodynamic optimization to plan safe formation trajectories to track a given trajectory and avoid stationary obstacles in a receding-horizon manner. Based on control barrier functions, a collision-free line-of-sight guidance law is developed to generate safe guidance commands to avoid collision with moving obstacles and other vehicles. An anti-disturbance control law is customized with a finite-time convergent observer for a vehicle to follow the guidance command signals. Simulation and hardware-in-the-loop experimental results are elaborated to validate the efficacy of the proposed method for the receding-horizon planning and formation control of ASVs.

In this article, we propose a collaborative neurodynamic optimization (CNO) method for the distributed seeking of generalized Nash equilibriums (GNEs) in multicluster games with nonconvex functions. Based on an augmented Lagrangian function, we develop a projection neural network for the local search of GNEs, and its convergence to a local GNE is proven. We formulate a global optimization problem to which a global optimal solution is a high-quality local GNE, and we adopt a CNO approach consisting of multiple recurrent neural networks for scattering searches and a metaheuristic rule for reinitializing states. We elaborate on an example of a price-bidding problem in an electricity market to demonstrate the viability of the proposed approach.

Learning dynamical networks based on time series of nodal states is of significant interest in systems science, computer science, and control engineering. Despite recent progress in network identification, most research focuses on static structures rather than switching ones. Therefore, this article develops a method for identifying the structures of switching networks by exploring and leveraging both temporal and spatial structural information that characterizes the switching process. The proposed method employs a new sparse Bayesian learning algorithm based on coupled hyperblocks to estimate unknown switching instants. Experimental results on benchmark artificial and real networks are elaborated to demonstrate the effectiveness and superiority of the proposed method.

The fuzzy c-means clustering algorithm is the most widely used soft clustering algorithm. In contrast to hard clustering, the cluster membership of data generated using the fuzzy c-means algorithm is ambiguous. Similar to hard clustering algorithms, the clustering results of the fuzzy c-means clustering algorithm are also suboptimal with varied performance depending on initial solutions. In this paper, a collaborative annealing fuzzy c-means algorithm is presented. To address the issue of ambiguity, the proposed algorithm leverages an annealing procedure to phase out the fuzzy cluster membership degree toward a crispy one by reducing the exponent gradually according to a cooling schedule. To address the issue of suboptimality, the proposed algorithm employs multiple fuzzy c-means modules to generate alternative clusters based on memberships repeatedly reinitialized using a metaheuristic rule. Experimental results on eight benchmark datasets are elaborated to demonstrate the superiority of the proposed algorithm to thirteen prevailing hard and soft algorithms in terms of internal and external cluster validity indices.

In this paper, we present a terminal slidingmode control method for the projective synchronization of unmodeled hyperjerk systems subject to parameter perturbation and external disturbances. We leverage fuzzy recurrent neural networks to identify unknown hyperjerk systems. We propose a control law for projective synchronization via the adaptive estimation of the unknown bounds of parameter perturbation and external disturbances. We theoretically prove that the proposed control law is able to achieve chattering-free projective synchronization in finite time. Finally, we elaborate on simulation results to demonstrate the efficacy of the methods.

This article addresses the hybrid model predictive control of chiller systems via collaborative neurodynamic optimization. A mixed-integer optimization problem is formulated for the model predictive control of chiller systems to minimize power consumption, subject to various constraints including thermodynamic and energy-conservation constraints. It is then decomposed into a global and a binary optimization subproblem. A collaborative neurodynamic optimization approach is proposed to solve the subproblems sequentially. The approach is based on multiple pairs of projection neural networks and discrete Hopfield networks, assisted with a metaheuristic rule. The effectiveness of the approach is demonstrated through experiments based on the parameters and specifications of a chiller system.

In this article, we present a collaborative neurodynamic optimization approach to distributed chiller loading in the presence of nonconvex power consumption functions and binary variables associated with cardinality constraints. We formulate a cardinality-constrained distributed optimization problem with nonconvex objective functions and discrete feasible regions, based on an augmented Lagrangian function. To overcome the difficulty caused by the nonconvexity in the formulated distributed optimization problem, we develop a collaborative neurodynamic optimization method based on multiple coupled recurrent neural networks reinitialized repeatedly using a meta-heuristic rule. We elaborate on experimental results based on two multi-chiller systems with the parameters from the chiller manufacturers to demonstrate the efficacy of the proposed approach in comparison to several baselines.

Exploring the mechanism of hysteresis dynamics may facilitate the analysis and controller design to alleviate detrimental effects. Conventional models, such as the Bouc-Wen and Preisach models consist of complicated nonlinear structures, limiting the applications of hysteresis systems for high-speed and high-precision positioning, detection, execution, and other operations. In this article, a Bayesian Koopman (B-Koopman) learning algorithm is therefore developed to characterize hysteresis dynamics. Essentially, the proposed scheme establishes a simplified linear representation with time delay for hysteresis dynamics, where the properties of the original nonlinear system are preserved. Furthermore, model parameters are optimized via sparse Bayesian learning together with an iterative strategy, which simplifies the identification procedure and reduces modeling errors. Extensive experimental results on piezoelectric positioning are elaborated to substantiate the effectiveness and superiority of the proposed B-Koopman algorithm for learning hysteresis dynamics.

This paper addresses capacitated clustering based on majorization-minimization and collaborative neurodynamic optimization (CNO). Capacitated clustering is formulated as a combinatorial optimization problem. Its objective function consists of fractional terms with intra-cluster similarities in their numerators and cluster cardinalities in their denominators as normalized cluster compactness measures. To obviate the difficulty in optimizing the objective function with factional terms, the combinatorial optimization problem is reformulated as an iteratively reweighted quadratic unconstrained binary optimization problem with a surrogate function and a penalty function in a majorization-minimization framework. A clustering algorithm is developed based on CNO for solving the reformulated problem. It employs multiple Boltzmann machines operating concurrently for local searches and a particle swarm optimization rule for repositioning neuronal states upon their local convergence. Experimental results on ten benchmark datasets are elaborated to demonstrate the superior clustering performance of the proposed approaches against seven baseline algorithms in terms of 21 internal cluster validity criteria.

Since the last decade, deep neural networks have shown remarkable capability in learning representations. The recently proposed neural ordinary differential equations (NODEs) can be viewed as the continuous-time equivalence of residual neural networks. It has been shown that NODEs have a tremendous advantage over the conventional counterparts in terms of spatial complexity for modeling continuous-time processes. However, existing NODEs methods entail their final time to be specified in advance, precluding the models from choosing a desirable final time and limiting their expressive capabilities. In this article, we propose learnable final-time (LFT) NODEs to overcome this limitation. LFT rebuilds the NODEs learning process as a final-time-free optimal control problem and employs the calculus of variations to derive the learning algorithm of NODEs. In contrast to existing NODEs methods, the new approach empowers the NODEs models to choose their suitable final time, thus being more flexible in adjusting the model depth for given tasks. Additionally, we analyze the gradient estimation errors caused by numerical ordinary differential equations (ODEs) solvers and employ checkpoint-based methods to obtain accurate gradients. We demonstrate the effectiveness of the proposed method with experimental results on continuous normalizing flows (CNFs) and feedforward models.

The strengthening and the weakening of synaptic strength in existing Bienenstock-Cooper-Munro (BCM) learning rule are determined by a long-term potentiation (LTP) sliding modification threshold and the afferent synaptic activities. However, synaptic long-term depression (LTD) even affects low-active synapses during the induction of synaptic plasticity, which may lead to information loss. Biological experiments have found another LTD threshold that can induce either potentiation or depression or no change, even at the activated synapses. In addition, existing BCM learning rules can only select a set of fixed rule parameters, which is biologically implausible and practically inflexible to learn the structural information of input signals. In this article, an evolved dual-threshold BCM learning rule is proposed to regulate the reservoir internal connection weights of the echo-state-network (ESN), which can contribute to alleviating information loss and enhancing learning performance by introducing different optimal LTD thresholds for different postsynaptic neurons. Our experimental results show that the evolved dual-threshold BCM learning rule can result in the synergistic learning of different plasticity rules, effectively improving the learning performance of an ESN in comparison with existing neural plasticity learning rules and some state-of-the-art ESN variants on three widely used benchmark tasks and the prediction of an esterification process.

As a crucial part of machine learning and pattern recognition, feature selection aims at selecting a subset of the most informative features from the set of all available features. In this article, supervised feature selection is at first formulated as a mixed-integer optimization problem with an objective function of weighted feature redundancy and relevancy subject to a cardinality constraint on the number of selected features. It is equivalently reformulated as a bound-constrained mixed-integer optimization problem by augmenting the objective function with a penalty function for realizing the cardinality constraint. With additional bilinear and linear equality constraints for realizing the integrality constraints, it is further reformulated as a bound-constrained biconvex optimization problem with two more penalty terms. Two collaborative neurodynamic optimization (CNO) approaches are proposed for solving the formulated and reformulated feature selection problems. One of the proposed CNO approaches uses a population of discrete-time recurrent neural networks (RNNs), and the other use a pair of continuous-time projection networks operating concurrently on two timescales. Experimental results on 13 benchmark datasets are elaborated to substantiate the superiority of the CNO approaches to several mainstream methods in terms of average classification accuracy with three commonly used classifiers.

In the operation planning of heating, ventilation, and air conditioning systems, optimal chiller loading assigns cooling loads to chillers with minimized power consumption. In this article, a mixed-integer optimization problem is formulated for distributed chiller loading and is then decomposed into two optimization subproblems with binary and continuous variables. A collaborative neurodynamic optimization approach is proposed for distributed chiller loading by solving the formulated subproblems. In the collaborative neurodynamic optimization framework, multiple projection neural networks and discrete Hopfield networks are used for scattered searches and a metaheuristic rule is adopted for reinitializing neuronal states upon their local convergence. Experimental results based on the specifications and parameters of three actual chiller systems are elaborated to substantiate the high performance of the approach.

Data-based discovery of the underlying dynamics of nonlinear systems is of great importance to the prediction and control of engineering systems. This article presents a duplex neurodynamic learning (DNL) approach to the identification of discrete-time nonlinear systems subjected to both external disturbances and measurement noise. A neurodynamic learning method is proposed based on two-timescale recurrent neural networks (RNNs) for system identification. Truncated singular value decomposition is adopted to purify the data contaminated by external disturbances and measurement noises. Two RNNs are employed to cooperatively search for a global optimal solution, and the particle swarm optimization rule is used to reinitialize the RNNs upon the local convergence of the RNNs. The effectiveness and superiority of the proposed DNL method are demonstrated via simulations on benchmark chaotic and NARMAX systems.

This article addresses multitarget circumnavigation with autonomous surface vehicles (ASVs) subject to model nonlinearities, environmental disturbances, and physical constraints in the presence of stationary/moving obstacles. A neurodynamics-driven distributed optimization method is proposed to achieve safety-certified cooperative circumnavigation guided by multiple targets. Specifically, a cooperative circumnavigation guidance law based on a finite-time distributed observer is designed for surrounding multiple targets. Based on the geometric characteristics of multitarget circumnavigation, three collision-avoidance rules are formulated with respect to obstacles, targets, and ASVs; and three types of control barrier functions are derived to encode the coupled safety constraints into state constraints. A distributed command governor optimization problem is formulated to generate optimal commanded guidance signals within the globally coupled state constraints. To compute optimal commands in real time, multiple recurrent neural networks (RNNs) are employed to solve a distributed optimization problem. An event-triggered communication scheme is designed for the communication among RNNs with reduced communication burden. A predictor-based fuzzy control law is designed to track safe velocity commands. The closed-loop system is proven to be input-to-state stable. Simulation results are elaborated to demonstrate the effectiveness of the safety-certified control method for ASVs to circumnavigate multiple targets with guaranteed safety.

Although cooperative coevolutionary algorithms are developed for large-scale dynamic optimization via subspace decomposition, they still face difficulties in reacting to environmental changes, in the presence of multiple peaks in the fitness functions and unevenness of subproblems. The resource allocation mechanisms among subproblems in the existing algorithms rely mainly on the fitness improvements already made but not potential ones. On the one hand, there is a lack of sufficient computing resources to achieve potential fitness improvements for some hard subproblems. On the other hand, the existing algorithms waste computing resources aiming to find most of the local optima of problems. In this article, we propose a cooperative particle swarm optimization algorithm to address these issues by introducing a bilevel balanceable resource allocation mechanism. A search strategy in the lower level is introduced to select some promising solutions from an archive based on solution diversity and quality to identify new peaks in every subproblem. A resource allocation strategy in the upper level is introduced to balance the coevolution of multiple subproblems by referring to their historical improvements and more computing resources are allocated for solving the subproblems that perform poorly but are expected to make great fitness improvements. Experimental results demonstrate that the proposed algorithm is competitive with the state-of-the-art algorithms in terms of objective function values and response efficiency with respect to environmental changes.

This article addresses the cooperative multitarget encircling control of underactuated autonomous surface vehicles with unknown kinetics subject to velocity and input constraints. A distributed observer is designed for the vehicles to estimate the geometric center of the area covered by multiple moving targets. Based on the target center estimate, a multitarget encircling guidance law is developed to form encircling trajectories around the targets. A data-driven fuzzy predictor is designed for learning the vehicle kinetics, including model input gains, with available data. Based on the learned model, a nominal control law is developed to track reference guidance signals. In order to satisfy the velocity and input constraints, a feasibility condition for velocities is derived based on a control barrier function, and a neurodynamics-based optimal control law is developed based on the feasibility condition and input constraint. The bounded input-to-state stability of the closed-loop control system is theoretically proved. Simulation results are elaborated to substantiate the effectiveness of the proposed control approach for circumnavigating multiple moving targets.

This paper addresses the multistability of switched fuzzy neural networks with a general class of activation functions under state-dependent switching. The existence, stability, and attraction basins of equilibria are analyzed via state-space decomposition based on Brouwer fixed point theorem and M-matrix properties. It is shown that there exist$5^{k_{1}}3^{k_{2}}$equilibria, and$3^{k_{1}}2^{k_{2}}$of them are locally exponentially stable under four sets of sufficient conditions for an$n$-neuron switched network, where$k_{1}$and$k_{2}$are nonnegative integers such that$0< k_{1}+k_{2}\leq n$. The results reveal that the switched fuzzy neural networks have much more equilibria than conventional fuzzy neural networks. Four numerical examples with simulation results are discussed to substantiate the theoretical results.

This article presents theoretical results on the multistability of fuzzy neural networks with rectified linear units and a state-dependent switching rule. Because of the boundlessness of state activation and multifariousness of state-dependent switching, such fuzzy neural networks exhibit very rich and complex dynamics. We show that there are up to$3^{n}-2^{n}-1$stable equilibria in an$n$-neuron switched fuzzy neural network, substantially more than recurrent neural networks without switching. Based on the properties of positive invariant set, we derive seven sets of sufficient conditions to ensure the multistability of switched fuzzy neural networks with rectified linear units. We elaborate on three numerical examples to illustrate the theoretical results and a potential application in associative memories.

The rapid development of the marine aquaculture industry has brought about a series of environmental problems that need to be monitored and planned. There is abundant marine aquaculture data obtained through synthetic aperture radar (SAR) remote sensing over a long period. With a large amount of unlabeled data, self-supervised learning can describe the feature representation of targets. However, when self-supervised learning meets big data, it often leads to semantic information loss, such as inter-class misjudgment and intra-class discontinuity. To address this issue, this paper proposes a self-supervised transformer with feature fusion (STFF) for the semantic segmentation of SAR images in marine aquaculture monitoring. STFF consists mainly of a self-attention encoding module with a hybrid loss function and a semantic segmentation decoding module with feature fusion. For encoding, the transformer is pretrained via self-supervised learning based on a hybrid loss function to enrich local, global and edge information for dealing with semantic information loss and data imbalance in whole-scene SAR images. For decoding, the features extracted from transformer blocks are fused to enhance semantic characteristics, improve the intra-class continuity of segmentation, and reduce the occurrence of inter-class misjudgment. The superiority of the proposed method to state-of-the-art algorithms is demonstrated via experimentation on GaoFen-3 and Radarsat-2 SAR datasets. The code has been available at https://github.com/fjc1575/Marine-Aquaculture/tree/main/STFF-code for the sake of reproducibility.

Chillers are indispensable machines for heat removal and the primary sources of power consumption in heating, ventilation, and air conditioning systems. In this paper, a cardinality-constrained global optimization problem is formulated to minimize power consumption for optimal chiller loading. The formulated problem is solved using a collaborative neurodynamic optimization method based on multiple neurodynamic models. Experimental results based on available actual chiller parameters are elaborated to demonstrate the superiority of the proposed approach to many baseline methods for optimal chiller loading.

In this article, sparse nonnegative matrix factorization (SNMF) is formulated as a mixed-integer bicriteria optimization problem for minimizing matrix factorization errors and maximizing factorized matrix sparsity based on an exact binary representation of$l_{0}$matrix norm. The binary constraints of the problem are then equivalently replaced with bilinear constraints to convert the problem to a biconvex problem. The reformulated biconvex problem is finally solved by using a two-timescale duplex neurodynamic approach consisting of two recurrent neural networks (RNNs) operating collaboratively at two timescales. A Gaussian score (GS) is defined as to integrate the bicriteria of factorization errors and sparsity of resulting matrices. The performance of the proposed neurodynamic approach is substantiated in terms of low factorization errors, high sparsity, and high GS on four benchmark datasets.

This article addresses event-triggered optimal load dispatching based on collaborative neurodynamic optimization. Two cardinality-constrained global optimization problems are formulated and two event-triggering functions are defined for event-triggered load dispatching in thermal energy and electric power systems. An event-triggered dispatching method is developed in the collaborative neurodynamic optimization framework with multiple projection neural networks and a meta-heuristic updating rule. Experimental results are elaborated to demonstrate the efficacy and superiority of the approach against many existing methods for optimal load dispatching in air conditioning systems and electric power generation systems.

This article addresses the safety-critical containment maneuvering of multiple underactuated autonomous surface vehicles (ASVs) in the presence of multiple stationary/moving obstacles. In a complex marine environment, every ASV suffers from model uncertainties, external disturbances, and input constraints. A safety-critical control method is proposed for achieving a collision-free containment formation. Specifically, a fixed-time extended state observer is employed for estimating the model uncertainties and external disturbances. By estimating lumped disturbances in fixed time, nominal containment maneuvering control laws are designed in an Earth-fixed reference frame. Input-to-state safe control barrier functions (ISSf-CBFs) are constructed for mapping safety constraints on states to constraints on control inputs. A distributed quadratic optimization problem with the norm of control inputs as the objective function and ISSf-CBFs as constraints is formulated. A recurrent neural network-based neurodynamic optimization approach is adopted to solve the quadratic optimization problem for computing the forces and moments within the safety and input constraints in real time. It is proven that the error signals in the closed-loop control system are uniformly ultimately bounded and the multi-ASVs system is guaranteed for input-to-state safety. Simulation results are elaborated to substantiate the effectiveness of the proposed safety-critical control method for ASVs based on neurodynamic optimization with control barrier functions.

The stability analysis of recurrent neural networks (RNNs) with multiple equilibria has received extensive interest since it is a prerequisite for successful applications of RNNs. With the increasing theoretical results on this topic, it is desirable to review the results for a systematical understanding of the state of the art. This article provides an overview of the stability results of RNNs with multiple equilibria including complete stability and multistability. First, preliminaries on the complete stability and multistability analysis of RNNs are introduced. Second, the complete stability results of RNNs are summarized. Third, the multistability results of various RNNs are reviewed in detail. Finally, future directions in these interesting topics are suggested.

This paper addresses the synchronization of multiple fractional-order recurrent neural networks (RNNs) with time-varying delays under event-triggered communications. Based on the assumption of the existence of strong connectivity or a spanning tree in the communication digraph, two sets of sufficient conditions are derived for achieving event-triggered synchronization. Moreover, an additional condition is derived to preclude Zeno behaviors. As a generalization of existing results, the criteria herein are also applicable to the event-triggered synchronization of multiple integer-order RNNs with or without delays. Two numerical examples are elaborated to illustrate the new results.

In this paper, we address the Clifford-valued distributed optimization subject to linear equality and inequality constraints. The objective function of the optimization problems is composed of the sum of convex functions defined in the Clifford domain. Based on the generalized Clifford gradient, a system of multiple Clifford-valued recurrent neural networks (RNNs) is proposed for solving the distributed optimization problems. Each Clifford-valued RNN minimizes a local objective function individually, with local interactions with others. The convergence of the neural system is rigorously proved based on the Lyapunov theory. Two illustrative examples are delineated to demonstrate the viability of the results in this article.

Because of the robustness and sparsity performance of least absolute deviation (LAD or$l_{1}$) optimization, developing effective solution methods becomes an important topic. Recurrent neural networks (RNNs) are reported to be capable of effectively solving constrained$l_{1}$-norm optimization problems, but their convergence speed is limited. To accelerate the convergence, this article introduces two RNNs, in form of continuous- and discrete-time systems, for solving$l_{1}$-norm optimization problems with linear equality and inequality constraints. The RNNs are theoretically proven to be globally convergent to optimal solutions without any condition. With reduced model complexity, the two RNNs can significantly expedite constrained$l_{1}$-norm optimization. Numerical simulation results show that the two RNNs spend much less computational time than related RNNs and numerical optimization algorithms for linearly constrained$l_{1}$-norm optimization.

A chiller plant is an essential part of a heating, ventilation, and air conditioning system. Chiller plant operation planning is to determine the throughput of active chillers, pumps, and fans in a chiller plant to meet cooling load demands with minimized power consumption. Existing planning methods are limited to chiller plant operation with homogeneous devices subject to constraints for the conservation of energy or with heterogeneous devices without considering the conservation of energy. In this article, a mixed-integer optimization problem is formulated for chiller plant operation planning with heterogeneous devices to minimize power consumption subject to various constraints, including the constraints for the conservation of energy. The formulated problem is reformulated as a global optimization problem and solved via collaborative neurodynamic optimization with multiple projection neural networks. Experimental results based on equipment manufacturers’ specifications are elaborated to demonstrate the significantly higher performance of the proposed approach than four mainstream methods in terms of power consumption wattage.

Autonomous marine vehicles (AMVs), including autonomous surface and underwater vehicles, are versatile means to explore, exploit, monitor, and protect marine resources and environments. Motion control is a fundamental enabling technique for state-of-the-art AMV development. Especially, guidance is a critical component in AMV motion control. In recent years, line-of-sight (LOS) guidance, as an efficient guidance method, has attracted tremendous interest from both theoretical and practical perspectives. In this paper, an overview of recent advances in LOS guidance for AMV path following is provided. First, a control objective for the path following of an AMV with a kinematic model is specified. Next, major LOS guidance laws for path following are reviewed in detail. Then, LOS guidance laws applicable to coordinated path following of multiple AMVs are elaborated. Finally, six challenging issues for future research are addressed.

The Halanay inequality is widely used in various time-delayed dynamical systems analyses and its vectorial form has become available recently. In this article, the integer-order vectorial Halanay-type inequality is further extended to fractional-order ones in both time-invariant and time-varying forms. It is shown that the fractional-order vectorial Halanay-type inequalities hold under the derived conditions in the form of$M$-matrices. In addition, the time-invariant inequalities are applied to analyzing the stability and synchronization of fractional-order systems with two numerical examples to substantiate the theoretical results.

In dynamic multiobjective optimization, the Pareto front (PF) or Pareto set varies over time as the problem environment changes. In such scenarios, optimization algorithms are required to efficiently find and continuously track a set of Pareto-optimal and diverse solutions. However, existing algorithms often result in the imbalanced approximation of PFs since some objectives are usually harder to optimize than others. In addition, the prediction strategies of the existing algorithms usually entail additional parameters (e.g., reference points, weights, and clustering parameters) to match available Pareto-optimal solutions for prediction in dynamically changing environments. This article presents a cooperative differential evolution algorithm with an attention-based prediction strategy. Multiple populations are adopted to optimize multiple objectives in search of subparts of PFs. Every population adopts a new fusion-based mutation strategy for coevolution. In addition, an expanding procedure is proposed on archived solutions to further expand the objective space covered by the populations to the entire PF. Specifically, once an environment change is detected, the populations are updated by using a new attention-based prediction strategy according to the historical variation of the objective functions. In this way, every population is adapted to the change of its attentive objective in the dynamic environment. Experimental results on a recent test suite of scalable dynamic multiobjective optimization problems are elaborated to demonstrate the superiority of the proposed method to state-of-the-art algorithms.

This article addresses the distributed formation control of multiple under-actuated autonomous surface vehicles (ASVs) in a receding-horizon setting. The ASVs are subject to physical constraints, in addition to stationary and moving obstacles. A barrier-certified distributed model predictive control method is proposed with the capability of avoiding collision with stationary and moving obstacles and neighboring ASVs. Specifically, a data-driven neural predictor is used to learn unknown functions in ASV kinetics. A nominal distributed receding-horizon position control law is developed based on the learned unknown function to achieve the desired formation within physical constraints. To ensure the safety requirement, a barrier-certified control law is designed based on control barrier functions to generate the signals of optimal surge force and heading angle within the safety constraints. A receding-horizon heading control law is designed based on the data-driven neural predictor to track the desired heading signals. Constrained quadratic programming problems are formulated based on barrier functions for barrier-certified distributed formation control and solved via neurodynamic optimization using one-layer recurrent neural networks. Thus, the proposed control method is able to ensure obstacle avoidance in the formation control of multiple ASVs in the presence of stationary and moving obstacles. Simulation results are elaborated to validate the efficacy of the proposed barrier-certified distributed model predictive control method for ASV formation.

In this article, we present a collaborative neurodynamic approach to distributed optimization with nonconvex functions. We develop a recurrent neural network (RNN) group by connecting individual projection neural networks through a communication network. We prove the convergence of the RNN group to the local optimal solutions of a given distributed optimization problem. We propose a collaborative neurodynamic optimization system with multiple RNN groups for scattered searches and a metaheuristic rule for reinitializing the neuronal states upon their local convergence. We elaborate on three numerical examples to demonstrate the efficacy of the proposed approach to distributed global optimization in the presence of nonconvexity.

Offshore aquaculture raft information extraction from synthetic aperture radar (SAR) images is essential for large-scale marine resource exploitation and protection. In this letter, a deep learning model called multi-scaled attention U-net with dilated convolution and offset convolution (MDOAU-net) is proposed for aquaculture raft monitoring via SAR image segmentation. The U-net backbone and attention gate of the Attention U-net are used in the MDOAU-net model. In addition, the MDOAU-net model consists of three distinctive parts. First, a multi-scale feature-fusion block is adopted in its input to extract features from raw images. Moreover, adapted from the Attention U-net for SAR image segmentation, fewer channels are used in each convolution layer of the MDOAU-net to match latent features in SAR images. Furthermore, nine dilated convolution blocks are adopted in the encoder–decoder structure to extract semantic features in the presence of speckle noises. In addition, offset convolution blocks are developed to convert spatial information into channel information for the precise segmentation of blurry boundaries. Four skip connections of the U-net backbone are replaced by four offset convolution blocks. Experimental results are elaborated to demonstrate the superior performance of the MDOAU-net model to seven existing methods in terms of overall accuracy (OA) and number of parameters.

This article addresses decentralized robust portfolio optimization based on multiagent systems. Decentralized robust portfolio optimization is first formulated as two distributed minimax optimization problems in a Markowitz return-risk framework. Cooperative-competitive multiagent systems are developed and applied for solving the formulated problems. The multiagent systems are shown to be able to reach consensuses in the expected stock prices and convergence in investment allocations through both intergroup and intragroup interactions. Experimental results of the multiagent systems with stock data from four major markets are elaborated to substantiate the efficacy of multiagent systems for decentralized robust portfolio optimization.

Hash bit selection determines an optimal subset of hash bits from a candidate bit pool. It is formulated as a zero-one quadratic programming problem subject to binary and cardinality constraints. In this article, the problem is equivalently reformulated as a global optimization problem. A collaborative neurodynamic optimization (CNO) approach is applied to solve the problem by using a group of neurodynamic models initialized with particle swarm optimization iteratively in the CNO. Lévy mutation is used in the CNO to avoid premature convergence by ensuring initial state diversity. A theoretical proof is given to show that the CNO with the Lévy mutation operator is almost surely convergent to global optima. Experimental results are discussed to substantiate the efficacy and superiority of the CNO-based hash bit selection method to the existing methods on three benchmarks.

In this article, a continuous-time complex-valued projection neural network (CCPNN) in a matrix state space is first proposed for a general complex-variable basis pursuit problem. The proposed CCPNN is proved to be stable in the sense of Lyapunov and to be globally convergent to the optimal solution under the condition that the sensing matrix is not row full rank. Furthermore, an improved discrete-time complex projection neural network (IDCPNN) is proposed by discretizing the CCPNN model. The proposed IDCPNN consists of a two-step stop strategy to reduce the calculational cost. The proposed IDCPNN is theoretically guaranteed to be global convergent to the optimal solution. Finally, the proposed IDCPNN is applied to the reconstruction of sparse signals based on compressed sensing. Computed results show that the proposed IDCPNN is superior to related complex-valued neural networks and conventional basis pursuit algorithms in terms of solution quality and computation time.

Regression in a sparse Bayesian learning (SBL) framework is usually formulated as a global optimization problem with a nonconvex objective function and solved in a majorization-minimization framework where the solution quality and consistency depend heavily on the initial values of the used algorithm. In view of the shortcomings, this article presents an SBL algorithm based on collaborative neurodynamic optimization (CNO) for searching global optimal solutions to the global optimization problem. The CNO system consists of a population of recurrent neural networks (RNNs) where each RNN is convergent to a local optimum to the global optimization problem. Reinitialized repetitively via particle swarm optimization with exchanged local optima information, the RNNs iteratively improve their searching performance until reaching global convergence. The proposed CNO-based SBL algorithm is almost surely convergent to a global optimal solution to the formulated global optimization problem. Two applications with experimental results on sparse signal reconstruction and partial differential equation identification are elaborated to substantiate the superiority and efficacy of the proposed method in terms of solution optimality and consistency.

Sparse Bayesian learning (SBL) is a popular machine learning approach with a superior generalization capability due to the sparsity of its adopted model. However, it entails a matrix inversion at each iteration, hindering its practical applications with large-scale data sets. To overcome this bottleneck, we propose an efficient SBL algorithm with$\mathcal {O}(n^{2})$computational complexity per iteration based on a Gaussian-scale mixture prior model. By specifying two different hyperpriors, the proposed efficient SBL algorithm can meet two different requirements, such as high efficiency and high sparsity. A surrogate function is introduced herein to approximate the posterior density of model parameters and thereby to avoid matrix inversions. Using a data-dependent term, a joint cost function with separate penalty terms is reformulated in a joint space of model parameters and hyperparameters. The resulting nonconvex optimization problem is solved using a block coordinate descent method in a majorization–minimization framework. Finally, the results of extensive experiments for sparse signal recovery and sparse image reconstruction on benchmark problems are elaborated to substantiate the effectiveness and superiority of the proposed approach in terms of computational time and estimation error.

This article presents theoretical results on the multistability of switched neural networks with Gaussian activation functions under state-dependent switching. It is shown herein that the number and location of the equilibrium points of the switched neural networks can be characterized by making use of the geometrical properties of Gaussian functions and local linearization based on the Brouwer fixed-point theorem. Four sets of sufficient conditions are derived to ascertain the existence of$7^{p_{1}}5^{p_{2}}3^{p_{3}}$equilibrium points, and$4^{p_{1}}3^{p_{2}}2^{p_{3}}$of them are locally stable, wherein$p_{1}$,$p_{2}$, and$p_{3}$are nonnegative integers satisfying$0\leq p_{1}+p_{2}+p_{3}\leq n$and$n$is the number of neurons. It implies that there exist up to$7^{n}$equilibria, and up to$4^{n}$of them are locally stable when$p_{1}=n$. It also implies that properly selecting$p_{1}$,$p_{2}$, and$p_{3}$can engender a desirable number of stable equilibria. Two numerical examples are elaborated to substantiate the theoretical results.

Hash bit selection (HBS) aims to find the most discriminative and informative hash bits from a hash pool generated by using different hashing algorithms. It is usually formulated as a binary quadratic programming problem with an information-theoretic objective function and a string-length constraint. In this article, it is equivalently reformulated in the form of a quadratic unconstrained binary optimization problem by augmenting the objective function with a penalty function. The reformulated problem is solved via collaborative neurodynamic optimization (CNO) with a population of classic discrete Hopfield networks. The two most important hyperparameters of the CNO approach are determined based on Monte Carlo test results. Experimental results on three benchmark data sets are elaborated to substantiate the superiority of the collaborative neurodynamic approach to several existing methods for HBS.

This article presents a neurodynamic approach to nonlinear programming. Motivated by the idea of sequential quadratic programming, a class of two-timescale multilayer recurrent neural networks is presented with neuronal dynamics in their output layer operating at a bigger timescale than in their hidden layers. In the two-timescale multilayer recurrent neural networks, the transient states in the hidden layer(s) undergo faster dynamics than those in the output layer. Sufficient conditions are derived on the convergence of the two-timescale multilayer recurrent neural networks to local optima of nonlinear programming problems. Simulation results of collaborative neurodynamic optimization based on the two-timescale neurodynamic approach on global optimization problems with nonconvex objective functions or constraints are discussed to substantiate the efficacy of the two-timescale neurodynamic approach.

Dropout and DropConnect are two techniques to facilitate the regularization of neural network models, having achieved the state-of-the-art results in several benchmarks. In this paper, to improve the generalization capability of spiking neural networks (SNNs), the two drop techniques are first applied to the state-of-the-art SpikeProp learning algorithm resulting in two improved learning algorithms called SPDO (SpikeProp with Dropout) and SPDC (SpikeProp with DropConnect). In view that a higher membrane potential of a biological neuron implies a higher probability of neural activation, three adaptive drop algorithms, SpikeProp with Adaptive Dropout (SPADO), SpikeProp with Adaptive DropConnect (SPADC), and SpikeProp with Group Adaptive Drop (SPGAD), are proposed by adaptively adjusting the keep probability for training SNNs. A convergence theorem for SPDC is proven under the assumptions of the bounded norm of connection weights and a finite number of equilibria. In addition, the five proposed algorithms are carried out in a collaborative neurodynamic optimization framework to improve the learning performance of SNNs. The experimental results on the four benchmark data sets demonstrate that the three adaptive algorithms converge faster than SpikeProp, SPDO, and SPDC, and the generalization errors of the five proposed algorithms are significantly smaller than that of SpikeProp. Furthermore, the experimental results also show that the five algorithms based on collaborative neurodynamic optimization can be improved in terms of several measures.

In this paper, we address the finite-time and fixed-time synchronization of a general class of switched neural networks (SNNs) with time delays subject to stochastic disturbances. Considering two types of switching in this class of SNNs: 1) intra-SNN state-dependent switching and 2) inter-SNN Markovian switching, we develop three control laws and derive three sets of sufficient conditions for both finite-time and fixed-time synchronization of SNNs subject to stochastic disturbances. We make two remarks on the effects of control-law parameters on synchronization settling time. Moreover, we derive several upper bounds of synchronization settling time and evaluate their pros and cons. Finally, we elaborate on two numerical examples to illustrate the viability of the theoretical results.

Inspired by the human learning principle that learning easier concepts first and then gradually paying more attention to harder ones, curriculum learning uses the nonuniform sampling of mini-batches according to the order of examples’ difficulty. Just as a teacher adjusts the curriculum according to the learning progress of each student, a proper curriculum should be adapted to the current state of the model. Therefore, in contrast to recent works using a fixed curriculum, we devise a new curriculum learning method, Adaptive Curriculum Learning (Adaptive CL), adapting the difficulty of examples to the current state of the model. Specifically, we make use of the loss of the current model to adjust the difficulty score while retaining previous useful learned knowledge by KL divergence. Moreover, under a non-linear model and binary classification, we theoretically prove that the expected convergence rate of curriculum learning monotonically decreases with respect to the loss of a point regarding the optimal hypothesis, and monotonically increases with respect to the loss of a point regarding the current hypothesis. The analyses indicate that Adaptive CL could improve the convergence properties during the early stages of learning. Extensive experimental results demonstrate the superiority of the proposed approach over existing competitive curriculum learning methods.

This article addresses an output-feedback flocking control problem for a swarm of autonomous surface vehicles (ASVs) to follow a leading ASV guided via a parameterized path. The leading and following ASVs are subject to completely unknown model parameters, external disturbances, and unmeasured velocities. A data-driven adaptive anti-disturbance control method is proposed for establishing a flocking behavior without any prior knowledge of model parameters. Specifically, a data-driven adaptive extended state observer (ESO) is proposed such that unknown input gains, unmeasured velocities, and total disturbance are simultaneously estimated. For the leading ASV, an output-feedback path-following control law is developed to follow a predefined parameterized path. For following ASVs, an output-feedback flocking control law is developed based on an artificial potential function for collision avoidance and connectivity preservation, in addition to a distributed ESO for estimating the velocity of the leading ASV through a cooperative estimation network. The simulation results are discussed to substantiate the efficacy of the proposed path-guided output-feedback ASV flocking control based on data-driven adaptive ESOs without measured velocity information.

Autonomous surface vehicles (ASVs) are marine vessels capable of performing various marine operations without a crew in a variety of cluttered and hostile water/ocean environments. For complex missions, there are increasing needs for deploying a fleet of ASVs instead of a single one to complete difficult tasks. Cooperative operations with a fleet of ASVs offer great advantages with enhanced capability and efficacy. Despite various application potentials, coordinated motion control of ASVs pose great challenges due to the multiplicity of ASVs, complexity of intravehicle interactions and fleet formation with collision avoidance requirements, and scarcity of communication bandwidths in sea environments. Coordinated control of multiple ASVs has received considerable attention in the last decade. This article provides an overview of recent advances in coordinated control of multiple ASVs. First, some challenging issues and scenarios in motion control of ASVs are presented. Next, coordinated control architecture and methods of multiple ASVs are briefly discussed. Then, recent results on trajectory-guided, path-guided, and target-guided coordinated control of multiple ASVs are reviewed in detail. Finally, several theoretical and technical issues are suggested to direct future investigations including network-based coordination, event-triggered coordination, collision-free coordination, optimization-based coordination, data-driven coordination of ASVs, and task-region-oriented coordination of multiple ASVs and autonomous underwater vehicles.

This article presents a two-timescale duplex neurodynamic approach to mixed-integer optimization, based on a biconvex optimization problem reformulation with additional bilinear equality or inequality constraints. The proposed approach employs two recurrent neural networks operating concurrently at two timescales. In addition, particle swarm optimization is used to update the initial neuronal states iteratively to escape from local minima toward better initial states. In spite of its minimal system complexity, the approach is proven to be almost surely convergent to optimal solutions. Its superior performance is substantiated via solving five benchmark problems.

This article presents new theoretical results on global exponential synchronization of nonlinear coupled delayed memristive neural networks with reaction-diffusion terms and Dirichlet boundary conditions. First, a state-dependent memristive neural network model is introduced in terms of coupled partial differential equations. Next, two control schemes are introduced: distributed state feedback pinning control and distributed impulsive pinning control. A salient feature of these two pinning control schemes is that only partial information on the neighbors of pinned nodes is needed. By utilizing the Lyapunov stability theorem and Divergence theorem, sufficient criteria are derived to ascertain the global exponential synchronization of coupled neural networks via the two pining control schemes. Finally, two illustrative examples are elaborated to substantiate the theoretical results and demonstrate the advantages and disadvantages of the two control schemes.

Portfolio selection is one of the important issues in financial investments. This article is concerned with portfolio selection based on collaborative neurodynamic optimization. The classic Markowitz mean-variance (MV) framework and its variant mean conditional value-at-risk (CVaR) are formulated as minimax and biobjective portfolio selection problems. Neurodynamic approaches are then applied for solving these optimization problems. For each of the problems, multiple neural networks work collaboratively to characterize the efficient frontier by means of particle swarm optimization (PSO)-based weight optimization. Experimental results with stock data from four major markets show the performance and characteristics of the collaborative neurodynamic approaches to the portfolio optimization problems.

This article presents new theoretical results on multistability and complete stability of recurrent neural networks with a sinusoidal activation function. Sufficient criteria are provided for ascertaining the stability of recurrent neural networks with various numbers of equilibria, such as a unique equilibrium, finite, and countably infinite numbers of equilibria. Multiple exponential stability criteria of equilibria are derived, and the attraction basins of equilibria are estimated. Furthermore, criteria for complete stability and instability of equilibria are derived for recurrent neural networks without time delay. In contrast to the existing stability results with a finite number of equilibria, the new criteria, herein, are applicable for both finite and countably infinite numbers of equilibria. Two illustrative examples with finite and countably infinite numbers of equilibria are elaborated to substantiate the results.

In this article, we address the disturbance/ uncertainty estimation of maritime autonomous surface ships (MASSs) with unknown internal dynamics, unknown external disturbances, and unknown input gains. In contrast to existing disturbance observers where some prior knowledge on kinetic model parameters such as the control input gains is available in advance, reduced- and full-order data-driven adaptive disturbance observers (DADOs) are proposed for estimating unknown input gains, as well as total disturbance composed of unknown internal dynamics and external disturbances. An advantage of the proposed DADOs is that the total disturbance and input gains can be simultaneously estimated with guaranteed convergence via data-driven adaption. We apply the proposed full-order DADO for the trajectory tracking control of an MASS without kinetic modeling and present a model-free trajectory tracking control law for the ship based on the DADO and a backstepping technique. We report the simulation results to substantiate the efficacy of the proposed DADO approach to model-free trajectory tracking control of an autonomous surface ship without knowing its dynamics.

This article presents a collaborative neurodynamic optimization (CNO) approach to multivehicle task assignments (TAs). The original combinatorial quadratic optimization problem for TA is reformulated as a quadratic unconstrained binary optimization (QUBO) problem with a quadratic utility function and a penalty function for handling load capacity and cooperation constraints. In the framework of CNO with a population of discrete Hopfield networks (DHNs), a TA algorithm is proposed for solving the formulated QUBO problem. Superior experimental results in four typical multivehicle operation scenarios are reported to substantiate the efficacy of the proposed neurodynamics-based TA approach.

Cyber-physical systems (CPSs) empower the integration of physical processes and cyber infrastructure with the aid of ubiquitous computation resources and communication capabilities. CPSs have permeated modern society and found extensive applications in a wide variety of areas, including energy, transportation, advanced manufacturing, and medical health. The security of CPSs against cyberattacks has been regarded as a long-standing concern. However, CPSs suffer from extendable vulnerabilities that are beyond classical networked systems due to the tight integration of cyber and physical components. Sophisticated and malicious cyberattacks continue to emerge to adversely impact CPS operation, resulting in performance degradation, service interruption, and system failure. Secure state estimation and control technologies play a vital role in warranting reliable monitoring and operation of safety-critical CPSs. This article provides a review of the state-of-the-art results for secure state estimation and control of CPSs. Specifically, the latest development of secure state estimation is summarized in light of different performance indicators and defense strategies. Then, the recent results on secure control are discussed and classified into three categories: 1) centralized secure control; 2) distributed secure control; and 3) resource-aware secure control. Furthermore, two specific application examples of water supply distribution systems and wide-area power systems are presented to demonstrate the applicability of secure state estimation and control approaches. Finally, several challenging issues are discussed to direct future research.

Sparsity is a desirable property in many nonnegative matrix factorization (NMF) applications. Although some level of sparseness of NMF solutions can be achieved by using regularization, the resulting sparsity depends highly on the regularization parameter to be valued in an ad hoc way. In this letter we formulate sparse NMF as a mixed-integer optimization problem with sparsity as binary constraints. A discrete-time projection neural network is developed for solving the formulated problem. Sufficient conditions for its stability and convergence are analytically characterized by using Lyapunov's method. Experimental results on sparse feature extraction are discussed to substantiate the superiority of this approach to extracting highly sparse features.

This article is devoted to the cluster synchronization issue of coupled fractional-order neural networks. By introducing the stability theory of fractional-order differential systems and the framework of Filippov regularization, some sufficient conditions are derived for ascertaining the asymptotic and finite-time cluster synchronization of coupled fractional-order neural networks, respectively. In addition, the upper bound of the settling time for finite-time cluster synchronization is estimated. Compared with the existing works, the results herein are applicable for fractional-order systems, which could be regarded as an extension of integer-order ones. A numerical example with different cases is presented to illustrate the validity of theoretical results.

This paper addresses task assignment (TA) for multivehicle systems. Multivehicle TA problems are formulated as a combinatorial optimization problem and further as a global optimization problem. To fulfill heterogeneous tasks, cooperation among heterogeneous vehicles is incorporated in the problem formulations. A collaborative neurodynamic optimization approach is developed for solving the TA problems. Experimental results on four types of TA problems are discussed to substantiate the efficacy of the approach.

Recent reports show that projection neural networks with a low-dimensional state space can enhance computation speed obviously. This paper proposes two projection neural networks with reduced model dimension and complexity (RDPNNs) for solving nonlinear programming (NP) problems. Compared with existing projection neural networks for solving NP, the proposed two RDPNNs have a low-dimensional state space and low model complexity. Under the condition that the Hessian matrix of the associated Lagrangian function is positive semi-definite and positive definite at each Karush-Kuhn-Tucker point, the proposed two RDPNNs are proven to be globally stable in the sense of Lyapunov and converge globally to a point satisfying the reduced optimality condition of NP. Therefore, the proposed two RDPNNs are theoretically guaranteed to solve convex NP problems and a class of nonconvex NP problems. Computed results show that the proposed two RDPNNs have a faster computation speed than the existing projection neural networks for solving NP problems.

This paper presents new theoretical results on the multistability of switched recurrent neural networks with radial basis functions and state-dependent switching. By partitioning state space, applying Brouwer fixed-point theorem and constructing a Lyapunov function, the number of the equilibria and their locations are estimated and their stability/instability are analyzed under some reasonable assumptions on the decomposition of index set and switching threshold. It is shown that the switching threshold plays an important role in increasing the number of stable equilibria and different multistability results can be obtained under different ranges of switching threshold. The results suggest that switched recurrent neural networks would be superior to conventional ones in terms of increased storage capacity when used as associative memories. Two examples are discussed in detail to substantiate the effectiveness of the theoretical analysis.

This paper presents a two-timescale duplex neurodynamic system for constrained biconvex optimization. The two-timescale duplex neurodynamic system consists of two recurrent neural networks (RNNs) operating collaboratively at two timescales. By operating on two timescales, RNNs are able to avoid instability. In addition, based on the convergent states of the two RNNs, particle swarm optimization is used to optimize initial states of the RNNs to avoid local minima. It is proven that the proposed system is globally convergent to the global optimum with probability one. The performance of the two-timescale duplex neurodynamic system is substantiated based on the benchmark problems. Furthermore, the proposed system is applied for L1-constrained nonnegative matrix factorization.

This paper is concerned with the multistability of switched neural networks with piecewise linear activation functions under state-dependent switching. Under some reasonable assumptions on the switching threshold and activation functions, by using the state-space decomposition method, contraction mapping theorem, and strictly diagonally dominant matrix theory, we can characterize the number of equilibria as well as analyze the stability/instability of the equilibria. More interesting, we can find that the switching threshold plays an important role for stable equilibria in the unsaturation regions of activation functions, and the number of stable equilibria of an n-neuron switched neural network with state-dependent parameters increases to 3nfrom 2nin the conventional one. Furthermore, for two-neuron switched neural networks, the precise attraction basin of each stable equilibrium point can be figured out, and its boundary is composed of the stable manifolds of unstable equilibrium points and the switching lines. Two simulation examples are discussed in detail to substantiate the effectiveness of the theoretical analysis.

This paper presents new theoretical results on the global synchronization of coupled fractional-order recurrent neural networks. Under the assumptions that the coupled fractional-order recurrent neural networks are sequentially connected in form of a single spanning tree or multiple spanning trees, two sets of sufficient conditions are derived for ascertaining the global synchronization by using the properties of Mittag-Leffler function and stochastic matrices. Compared with existing works, the results herein are applicable for fractional-order systems, which could be viewed as an extension of integer-order ones. Two numerical examples are presented to illustrate the effectiveness and characteristics of the theoretical results.

The past three decades witnessed the birth and growth of neurodynamic optimization which has emerged and matured as a powerful approach to real-time optimization due to its inherent nature of parallel and distributed information processing and the hardware realizability. Despite the success, almost all existing neurodynamic approaches work well only for convex and generalized-convex optimization problems with unimodal objective functions. Effective neurodynamic approach to constrained global optimization with multimodal objective functions is rarely available. In this talk, starting with the idea and motivation of neurodynamic optimization, I will review the historic review and present the state of the art of neurodynamic optimization with many individual models for convex and generalized convex optimization. In addition, I will present a multiple-time-scale neurodynamic approach to selected constrained optimization. Finally, I will introduce population-based collaborative neurodynamic approaches to constrained distributed and global optimization. By deploying a population of individual neurodynamic models with diversified initial states at a lower level coordinated by using some global search and information exchange rules (such as PSO or DE) at a upper level, it will be shown that distributed, global, and multi-objective optimization problems can be solved effectively and efficiently.

Land cover classification using polarimetric synthetic aperture radar (PolSAR) images is an important tool for remote sensing analysis. In view that PolSAR image effective interpretation is commonly affected by the absence of discriminative features and the presence of speckle noises, this letter proposes an adaptive nonlocal stacked sparse autoencoder (ANSSAE) to achieve PolSAR image classification. It extracts the adaptive nonlocal spatial information by adaptively calculating weighted average values of each pixel from nonlocal regions, which can reduce the influence of speckle noises and retain edge details. In the first layer of the ANSSAE, the adaptive nonlocal spatial information is introduced into the objective function to obtain the robust feature representation, whose effects would transfer to the rest of layers. Therefore, the ANSSAE can automatically capture spatial-related, robust, and distinguishable features, which can suppress speckle noises and gain accurate classification results. Experimental results on two real PolSAR images demonstrate that the proposed approach can significantly improve the classification accuracy.

This paper presents a two-phase fuzzy clustering algorithm based on neurodynamic optimization with its application for polarimetric synthetic aperture radar (PolSAR) remote sensing image segmentation. The two-phase clustering algorithm starts with the linear-assignment initialization phase with the least similar cluster representatives to remedy the inconsistency of clustering results from random initialization and is, then, followed with multiple-kernel fuzzy C-means clustering. By incorporating multiple kernels in the clustering framework, various features are incorporated cohesively. A winner-takes-all neural network is employed to acquire the highest kernel weights and associated cluster centers and membership matrices, which enables better characterization and adaptability in each individual cluster. Simulation results for UCI benchmark datasets and PolSAR remote sensing image segmentation are reported to substantiate the effectiveness and the superiority of the proposed clustering algorithm.

There are two ultimate goals in multiobjective optimization. The primary goal is to obtain a set of Pareto-optimal solutions while the secondary goal is to obtain evenly distributed solutions to characterize the efficient frontier. In this paper, a collaborative neurodynamic approach to multiobjective optimization is presented to attain both goals of Pareto optimality and solution diversity. The multiple objectives are first scalarized using a weighted Chebyshev function. Multiple projection neural networks are employed to search for Pareto-optimal solutions with the help of a particle swarm optimization (PSO) algorithm in reintialization. To diversify the Pareto-optimal solutions, a holistic approach is proposed by maximizing the hypervolume (HV) using again a PSO algorithm. The experimental results show that the proposed approach outperforms three other state-of-the-art multiobjective algorithms (i.e., HMOEA/D, MOEA/DD, and NSGAIII) most of times on 37 benchmark datasets in terms of HV and inverted generational distance.

This paper is concerned with the coexistence of multiple equilibrium points and dynamical behaviors of recurrent neural networks with nonmonotonic activation functions and unbounded time-varying delays. Based on a state space partition by using the geometrical properties of the activation functions, it is revealed that an -neuron neural network can exhibit equilibrium points with . In particular, several sufficient criteria are proposed to ascertain the asymptotical stability of equilibrium points for recurrent neural networks. These theoretical results cover both monostability and multistability. Furthermore, the attraction basins of asymptotically stable equilibrium points are estimated. It is shown that the attraction basins of the stable equilibrium points can be larger than their originally partitioned subsets. Finally, the results are illustrated by using the simulation results of four examples.

The robust Huber's M-estimator is widely used in signal and image processing, classification, and regression. From an optimization point of view, Huber's M-estimation problem is often formulated as a large-sized quadratic programming (QP) problem in view of its nonsmooth cost function. This paper presents a generalized regression estimator which minimizes a reduced-sized QP problem. The generalized regression estimator may be viewed as a significant generalization of several robust regression estimators including Huber's M-estimator. The performance of the generalized regression estimator is analyzed in terms of robustness and approximation accuracy. Furthermore, two low-dimensional recurrent neural networks (RNNs) are introduced for robust estimation. The two RNNs have low model complexity and enhanced computational efficiency. Finally, the experimental results of two examples and an application to image restoration are presented to substantiate superior performance of the proposed method over conventional algorithms for robust regression estimation in terms of approximation accuracy and convergence rate.

This paper is concerned with multiple-objective distributed optimization. Based on objective weighting and decision space decomposition, a collaborative neurodynamic approach to multiobjective distributed optimization is presented. In the approach, a system of collaborative neural networks is developed to search for Pareto optimal solutions, where each neural network is associated with one objective function and given constraints. Sufficient conditions are derived for ascertaining the convergence to a Pareto optimal solution of the collaborative neurodynamic system. In addition, it is proved that each connected subsystem can generate a Pareto optimal solution when the communication topology is disconnected. Then, a switching-topology-based method is proposed to compute multiple Pareto optimal solutions for discretized approximation of Pareto front. Finally, simulation results are discussed to substantiate the performance of the collaborative neurodynamic approach. A portfolio selection application is also given.

This paper is concerned with global asymptotic stability of a neural network with a time-varying delay, where the delay function is differentiable uniformly bounded with delay-derivative bounded from above. First, a general reciprocally convex inequality is presented by introducing some slack vectors with flexible dimensions. This inequality provides a tighter bound in the form of a convex combination than some existing ones. Second, by constructing proper Lyapunov-Krasovskii functional, global asymptotic stability of the neural network is analyzed for two types of the time-varying delays depending on whether or not the lower bound of the delay derivative is known. Third, noticing that sufficient conditions on stability from estimation on the derivative of some Lyapunov-Krasovskii functional are affine both on the delay function and its derivative, allowable delay sets can be refined to produce less conservative stability criteria for the neural network under study. Finally, two numerical examples are given to substantiate the effectiveness of the proposed method.

This paper presents a design method for output-feedback path-following control of under-actuated autonomous underwater vehicles moving in a vertical plane without using surge, heave, and pitch velocities. Specifically, an extended state observer (ESO) is developed to recover the unmeasured velocities as well as to estimate total uncertainty induced by internal model uncertainty and external disturbance. At the kinematic level, a commanded guidance law is developed based on a vertical line-of-sight guidance scheme and the observed velocities. To optimize guidance signals, optimization-based reference governors are formulated as bound-constrained quadratic programming problems for computing optimal reference signals. Two globally convergent recurrent neural networks called projection neural networks are used to solve the optimization problems in real-time. Based on the optimal reference signals and ESO, a kinetic control law with disturbance rejection capability is constructed at the kinetic level. It is proved that all error signals in the closed-loop system are uniformly and ultimately bounded. Simulation results substantiate the efficacy of the proposed method for output-feedback path-following of under-actuated autonomous underwater vehicles.

This letter studies the multistability analysis of delayed recurrent neural networks with Mexican hat activation function. Some sufficient conditions are obtained to ensure that an [Formula: see text]-dimensional recurrent neural network can have [Formula: see text] equilibrium points with [Formula: see text], and [Formula: see text] of them are locally exponentially stable. Furthermore, the attraction basins of these stable equilibrium points are estimated. We show that the attraction basins of these stable equilibrium points can be larger than their originally partitioned subsets. The results of this letter improve and extend the existing stability results in the literature. Finally, a numerical example containing different cases is given to illustrate the theoretical results.

Nonnegative matrix factorization (NMF) is an advanced method for nonnegative feature extraction, with widespread applications. However, the NMF solution often entails to solve a global optimization problem with a nonconvex objective function and nonnegativity constraints. This paper presents a collective neurodynamic optimization (CNO) approach to this challenging problem. The proposed collective neurodynamic system consists of a population of recurrent neural networks (RNNs) at the lower level and a particle swarm optimization (PSO) algorithm with wavelet mutation at the upper level. The RNNs act as search agents carrying out precise local searches according to their neurodynamics and initial conditions. The PSO algorithm coordinates and guides the RNNs with updated initial states toward global optimal solution(s). A wavelet mutation operator is added to enhance PSO exploration diversity. Through iterative interaction and improvement of the locally best solutions of RNNs and global best positions of the whole population, the population-based neurodynamic systems are almost sure able to achieve the global optimality for the NMF problem. It is proved that the convergence of the group-best state to the global optimal solution with probability one. The experimental results substantiate the efficacy and superiority of the CNO approach to bound-constrained global optimization with several benchmark nonconvex functions and NMF-based clustering with benchmark data sets in comparison with the state-of-the-art algorithms.

This paper presents a two-time-scale neurodynamic approach to constrained minimax optimization using two coupled neural networks. One of the recurrent neural networks is used for minimizing the objective function and another is used for maximization. It is shown that the coupled neurodynamic systems operating in two different time scales work well for minimax optimization. The effectiveness and characteristics of the proposed approach are illustrated using several examples. Furthermore, the proposed approach is applied for H∞model predictive control.

This paper presents a collective neurodynamic approach with multiple interconnected recurrent neural networks (RNNs) for distributed constrained optimization. The objective function of the distributed optimization problems to be solved is a sum of local convex objective functions, which may be nonsmooth. Subject to its local constraints, each local objective function is minimized individually by using an RNN, with consensus among others. In contrast to existing continuous-time distributed optimization methods, the proposed collective neurodynamic approach is capable of solving more general distributed optimization problems. Simulation results on three numerical examples are discussed to substantiate the effectiveness and characteristics of the proposed approach. In addition, an application to the optimal placement problem is delineated to demonstrate the viability of the approach.

This paper presents a predictor-based neural dynamic surface control (PNDSC) design method for a class of uncertain nonlinear systems in a strict-feedback form. In contrast to existing NDSC approaches where the tracking errors are commonly used to update neural network weights, a predictor is proposed for every subsystem, and the prediction errors are employed to update the neural adaptation laws. The proposed scheme enables smooth and fast identification of system dynamics without incurring high-frequency oscillations, which are unavoidable using classical NDSC methods. Furthermore, the result is extended to the PNDSC with observer feedback, and its robustness against measurement noise is analyzed. Numerical and experimental results are given to demonstrate the efficacy of the proposed PNDSC architecture.

This paper presents a neurodynamic optimization approach to bilevel quadratic programming (BQP). Based on the Karush-Kuhn-Tucker (KKT) theorem, the BQP problem is reduced to a one-level mathematical program subject to complementarity constraints (MPCC). It is proved that the global solution of the MPCC is the minimal one of the optimal solutions to multiple convex optimization subproblems. A recurrent neural network is developed for solving these convex optimization subproblems. From any initial state, the state of the proposed neural network is convergent to an equilibrium point of the neural network, which is just the optimal solution of the convex optimization subproblem. Compared with existing recurrent neural networks for BQP, the proposed neural network is guaranteed for delivering the exact optimal solutions to any convex BQP problems. Moreover, it is proved that the proposed neural network for bilevel linear programming is convergent to an equilibrium point in finite time. Finally, three numerical examples are elaborated to substantiate the efficacy of the proposed approach.

Global optimization is a long-lasting research topic in the field of optimization, posting many challenging theoretic and computational issues. This paper presents a novel collective neurodynamic method for solving constrained global optimization problems. At first, a one-layer recurrent neural network (RNN) is presented for searching the Karush-Kuhn-Tucker points of the optimization problem under study. Next, a collective neuroydnamic optimization approach is developed by emulating the paradigm of brainstorming. Multiple RNNs are exploited cooperatively to search for the global optimal solutions in a framework of particle swarm optimization. Each RNN carries out a precise local search and converges to a candidate solution according to its own neurodynamics. The neuronal state of each neural network is repetitively reset by exchanging historical information of each individual network and the entire group. Wavelet mutation is performed to avoid prematurity, add diversity, and promote global convergence. It is proved in the framework of stochastic optimization that the proposed collective neurodynamic approach is capable of computing the global optimal solutions with probability one provided that a sufficiently large number of neural networks are utilized. The essence of the collective neurodynamic optimization approach lies in its potential to solve constrained global optimization problems in real time. The effectiveness and characteristics of the proposed approach are illustrated by using benchmark optimization problems.

In this paper, new results on the global synchronization of multiple recurrent neural networks (NNs) with time delays via impulsive interactions are presented. Impulsive interaction means that a number of NNs communicate with each other at impulse instants only, while they are independent at the remaining time. The communication topology among NNs is not required to be always connected and can switch ON and OFF at different impulse instants. By using the concept of sequential connectivity and the properties of stochastic matrices, a set of sufficient conditions depending on time delays is derived to ascertain global synchronization of multiple continuous-time recurrent NNs. In addition, a counterpart on the global synchronization of multiple discrete-time NNs is also discussed. Finally, two examples are presented to illustrate the results.

In this paper, coexistence and stability of multiple equilibrium points of fractional-order recurrent neural networks are addressed. Several sufficient conditions are derived for ascertaining the existence of Πi=1n(2Ki+ 1) equilibrium points (Ki≥ 0) and the local Mittage - Leffler stability Πi=1n(Ki+ 1) equilibrium points of them by using the geometrical properties of activation functions and algebraic properties of nonsingular M-matrix. In contrast with many existing results, the derived results cover both mono-stability and multistability, and the activation functions herein could be nonmonotonic and nonlinear in any open interval. In addition, three numerical examples are elaborated to substantiate the efficacy and characteristics of the theoretical results.

In this paper, distributed optimization is addressed based on a continuous-time multiagent system in the presence of time-varying communication delays. First, the relationship between optimal solutions and the equilibrium points of the multiagent system with time delay is revealed. Next, delay-dependent and delay-independent sufficient conditions in form of linear matrix inequality are derived for ascertaining convergence to optimal solutions, in the cases of slow-varying delay and fast-varying delay. Furthermore, a set of conditions are also obtained for the delay-free case. In addition, a sampled-data communication scheme is presented based on the conditions for the fast varying delay systems. Simulation results are presented to substantiate the theoretical results. An application for distributed parameter estimation is also given.

This paper proposes a novel superpixel-based method for the classification of hyperspectral image. A superpixel segmentation algorithm called entropy rate superpixel is applied to extract the spatial contextual information in the hyperspectral image, which can change the size and shape of the superpixel adaptively according to spatial structures. Then, a joint sparse representation model is applied to approximate the pixels within each superpixel using a certain number of common samples from a given dictionary in the form of sparse linear combination. Here we use a greedy algorithm called simultaneous orthogonal matching pursuit to pursue the optimal sparse coefficients matrix and a new kind of classification criterion is tested and used to determine the classification results. Experimental results on the Indian Pines hyperspsectral image demonstrate that the proposed method can explore the spatial information effectively and give promising performance when compared with several state-of-art classification methods.

In this paper, an EEG-based brain-computer interface (BCI) system used for emotion recognition is proposed to detect two basic emotional states (happiness and sadness). Selection of frequency bands plays a vital role in distinguishing brain patterns associated with emotions. This paper explores a new method to select suitable subject-specific frequency bands instead of using fixed frequency bands for the emotion recognition. Common spatial pattern and support vector machine were employed to classify two emotional states. Two experiments involving six subjects were conducted to validate our method and BCI system. An average online accuracy of 74.17% for two classes was achieved. The data analysis results demonstrated that the proposed method based on subject-specific frequency bands outperformed the method based on the fixed frequency bands in terms of accuracy.

Indoor localization has been actively researched in recent years due to the increasing demand for location-awareness services. However, to balance localization accuracy and system cost is always a challenge for indoor localization systems. Radio frequency identification (RFID) is a promising technology to achieve both goals, because of its reasonable cost and reliability. In this paper, we propose a novel RFID indoor localization system based on angle of arrival (AoA) and phase of arrival (PoA) methods. This system leverages RFID's two experimental signal diffusion characteristics to estimate AoA. One is that the interrogation zone is constrained in a lobe, and only in this area the tag can be queried. The second is that there exits a stable pattern of received signal strength (RSS) on angle changes. We use the two features to find a general area and to pinpoint the AoA consecutively. This effectively narrows the sampling zone (where signal needs to be sampled), and helps to reduce computational complexity. In addition, we reduce the multipath effect on range estimation by determining the AoA and rotating the reader into the direction of the target. Moreover, we exploit two signals with a slightly different frequency to eliminate the phase ambiguity issue. Our system takes only one reader and achieves mean accuracy of 23 cm. The simplicity and effectiveness of our system make it convenient to be used in practice.

This paper presents several L1-minimization algorithms for sparse signal reconstruction based on a continuous-time projection neural network (PNN). First, a one-layer projection neural network is designed based on a projection operator and a projection matrix. The stability and global convergence of the proposed neural network are proved. Then, based on a discrete-time version of the PNN, several L1-minimization algorithms for sparse signal reconstruction are developed and analyzed. Experimental results based on random Gaussian sparse signals show the effectiveness and performance of the proposed algorithms. Moreover, experimental results based on two face image databases are presented that reveal the influence of sparsity to the recognition rate. The algorithms are shown to be robust to the amplitude and sparsity level of signals as well as efficient with high convergence rate compared with several existing L1-minimization algorithms.

In this paper, a bi-projection neural network for solving a class of constrained quadratic optimization problems is proposed. It is proved that the proposed neural network is globally stable in the sense of Lyapunov, and the output trajectory of the proposed neural network will converge globally to an optimal solution. Compared with existing projection neural networks (PNNs), the proposed neural network has a very small model size owing to its bi-projection structure. Furthermore, an application to data fusion shows that the proposed neural network is very effective. Numerical results demonstrate that the proposed neural network is much faster than the existing PNNs.

This paper presents new theoretical results on the multistability analysis of a class of recurrent neural networks with nonmonotonic activation functions and mixed time delays. Several sufficient conditions are derived for ascertaining the existence of 3nequilibrium points and the exponential stability of 2nequilibrium points via state space partition by using the geometrical properties of activation functions and algebraic properties of nonsingular M-matrix. Compared with existing results, the conditions herein are much more computable with one order less linear matrix inequalities. Furthermore, the attraction basins of these exponentially stable equilibrium points are estimated. It is revealed that the attraction basins of the 2nequilibrium points can be larger than their originally partitioned subspaces. Three numerical examples are elaborated with typical nonmonotonic activation functions to substantiate the efficacy and characteristics of the theoretical results.

The robust stability of nonlinear systems has been studied extensively. It is well known that time delays and additive noises may derail the stability of nonlinear systems. This paper presents theoretical results on the robustness of the exponential stability of nonlinear systems in the presence of time delays and random disturbances. For a given exponentially stable (ES) nonlinear system, it is interesting to know how much time delay and noise intensity there are so that the system may remain to be ES when the system is subject to delay and noise. Upper bounds of allowable delays and noise intensities are derived for nonlinear systems to keep their exponential stability. It is proven that if the noise and delay of ES nonlinear systems are lower than the upper bounds derived herein, the nonlinear systems infected by noises and delays are ensured to be ES. Three numerical examples are given to substantiate the efficacy of the results.

This paper presents a neurodynamic optimization approach with two coupled recurrent neural networks for the synthesis of linear systems with fault detection via robust pole assignment. The proposed approach is shown to be capable of synthesizing control systems with robust state estimators and fault detection with parameter perturbation. The operating characteristics of the recurrent neural networks for state estimation and fault detection are demonstrated by using an illustrative example.

This paper presents new results on neurodynamic optimization approach to solve bilevel linear programming problems (BLPPs) with linear inequality constraints. A sub-gradient recurrent neural network is proposed for solving the BLPPs. It is proved that the state convergence time period is finite and can be quantitatively estimated. Compared with existing recurrent neural networks for BLPPs, the proposed neural network does not have any design parameter and can solve the BLPPs in finite time. Some numerical examples are introduced to show the effectiveness of the proposed neural network.

This paper presents theoretical results on the global exponential synchronization of multiple memristive neural networks with time delays. A novel coupling scheme is introduced, in a general topological structure described by a directed or undirected graph, with a linear diffusive term and discontinuous sign term. Several criteria are derived based on the Lyapunov stability theory to ascertain the global exponential stability of synchronization manifold in the coupling scheme. Simulation results for several examples are given to substantiate the effectiveness of the theoretical results.

In this paper, a neurodynamic optimization approach is proposed for synthesizing high-order descriptor linear systems with state feedback control via robust pole assignment. With a new robustness measure serving as the objective function, the robust eigenstructure assignment problem is formulated as a pseudoconvex optimization problem. A neurodynamic optimization approach is applied and shown to be capable of maximizing the robust stability margin for high-order singular systems with guaranteed optimality and exact pole assignment. Two numerical examples and vehicle vibration control application are discussed to substantiate the efficacy of the proposed approach.

This paper is concerned with the passivity analysis for switched neural networks subject to stochastic disturbances and time-varying delays. First, using the multiple Lyapunov functions method, a state-dependent switching law is designed to present a stochastic passivity condition. Second, a hysteresis switching law involving both the current state and the previous value of the switching signal are presented to avoid chattering resulted from the state-dependent switching. Third, based on the average dwell-time approach, a class of switching signals is determined to guarantee the switched neural network stochastically passive. Finally, three numerical examples are provided to illustrate the characteristics of three proposed switching laws.

This paper presents a projection neural network described by a dynamic system for solving constrained quadratic minimax programming problems. Sufficient conditions based on a linear matrix inequality are provided for global convergence of the proposed neural network. Compared with some of the existing neural networks for quadratic minimax optimization, the proposed neural network in this paper is capable of solving more general constrained quadratic minimax optimization problems, and the designed neural network does not include any parameter. Moreover, the neural network has lower model complexities, the number of state variables of which is equal to that of the dimension of the optimization problems. The simulation results on numerical examples are discussed to demonstrate the effectiveness and characteristics of the proposed neural network.

In general, nonlinear model predictive control (NMPC) entails solving a sequential global optimization problem with a nonconvex cost function or constraints. This paper presents a novel collective neurodynamic optimization approach to NMPC without linearization. Utilizing a group of recurrent neural networks (RNNs), the proposed collective neurodynamic optimization approach searches for optimal solutions to global optimization problems by emulating brainstorming. Each RNN is guaranteed to converge to a candidate solution by performing constrained local search. By exchanging information and iteratively improving the starting and restarting points of each RNN using the information of local and global best known solutions in a framework of particle swarm optimization, the group of RNNs is able to reach global optimal solutions to global optimization problems. The essence of the proposed collective neurodynamic optimization approach lies in the integration of capabilities of global search and precise local search. The simulation results of many cases are discussed to substantiate the effectiveness and the characteristics of the proposed approach.

In this paper, we present a complex-valued projection neural network for solving constrained convex optimization problems of real functions with complex variables, as an extension of real-valued projection neural networks. Theoretically, by developing results on complex-valued optimization techniques, we prove that the complex-valued projection neural network is globally stable and convergent to the optimal solution. Obtained results are completely established in the complex domain and thus significantly generalize existing results of the real-valued projection neural networks. Numerical simulations are presented to confirm the obtained results and effectiveness of the proposed complex-valued projection neural network.

This paper is concerned with the global exponential synchronization of two memristor-based recurrent neural networks (MRNNs) with time delays via static or dynamic coupling. First, four coupling rules (i.e., static state coupling, static output coupling, dynamic state coupling, and dynamic output coupling) are designed for the exponential synchronization of drive-response pair of MRNNs. Then, several global exponential synchronization criteria are derived by constructing suitable Lyapunov-Krasovskii functionals based on the Lyapunov stability theory. Compared with existing results on synchronization of MRNNs, the conditions herein are easy to be verified. Moreover, the designed dynamic state coupling and output coupling rules have good anti-interference capacity. Finally, two illustrative examples are presented to substantiate the effectiveness and characteristics of the presented theoretical results.

This paper is concerned with the global robust synchronization of multiple memristive neural networks (MMNNs) with nonidentical uncertain parameters. A coupling scheme is introduced, in a general topological structure described by a direct or undirect graph, with a linear diffusive term and a discontinuous sign term. First, a set of sufficient conditions are derived based on the Lyapunov stability theory for ascertaining global robust synchronization of coupled MMNNs. Second, a pinning adaptive coupling method is proposed to ensure global synchronization without knowing the bound of parameter uncertainties. Two illustrative examples are discussed to substantiate the theoretical results.

To improve self-made satellites in the marine oil spill monitoring accuracy, it is presented that a Gao Fen (GF-1) satellite marine oil spill remote sensing (RS) image classification algorithm based on a novel evolutionary neural network. First, a non-negative matrix factorization (NMF) algorithm is employed to extract the image features. Compared with basic features, such as the image spectrum and texture, structuring more targeted oil spill image localization non-negative character fits better for the physical significance of remote sensing images. Furthermore, on the basis of the new features, a new feedforward neural network structure with particle swarm optimization (PSO) algorithm is proposed for GF-1 RS image segmentation. Simulation results of the oil spill event substantiate the effectiveness of the proposed approach to GF-1 satellite image segmentation.

In this paper, a neurodynamic optimization approach is proposed for robust eigenstructure assignment problem of second-order descriptor systems via state feedback control. With a novel robustness measure serving as the objective function, the robust eigenstructure assignment problem is formulated as a pseudoconvex optimization problem. Two coupled recurrent neural networks are applied for solving the optimization problem with guaranteed optimality and exact pole assignment. Simulation results are included to substantiate the effectiveness of the proposed approach.

This paper presents a model predictive control (MPC) method based on a recurrent neural network for control of autonomous underwater vehicles (AUVs) in a vertical plane. Both kinematic and dynamic models are considered in the set-point control of the AUV. A one-layer recurrent neural network called the general projection neural network is applied for real-time optimization to compute optimal control vaiables. Simulation results are discussed to demonstrate the effectiveness and characteristics of the proposed model predictive control method.

This paper is concerned with formation control problems of multi-robot systems in framework of model predictive control. The formation control of robots herein is based on the leader-follower scheme. The followers are controlled by torques to track the desired trajectories to form and keep a formation. A model predictive control approach is proposed for solving the formation control problem, where the control problem is formulated as a dynamic quadratic optimization problem. A one-layer recurrent neural network called the simplified dual network is applied for computing the optimal control input in real time. Simulation results substantiate that the formation of robots can be well controlled by the proposed approach.

A neurodynamic optimization approach is proposed for robust pole assignment problem of second-order control systems via output feedback. With a suitable robustness measure serving as the objective function, the robust pole assignment problem is formulated as a quasi-convex optimization problem with linear constraints. Next, the problem further is reformulated as a convex feasibility problem. Two coupled recurrent neural networks are applied for solving the optimization problem with guaranteed optimality and exact pole assignment. Simulation results are included to substantiate the effectiveness of the proposed approach.

In view of the high energy consumption and low response speed of the traditional hydraulic system for an injection molding machine, a servo motor driven constant pump hydraulic system is designed for a precision injection molding process, which uses a servo motor, a constant pump, and a pressure sensor, instead of a common motor, a constant pump, a pressure proportion valve, and a flow proportion valve. A model predictive control strategy based on neurodynamic optimization is proposed to control this new hydraulic system in the injection molding process. Simulation results showed that this control method has good control precision and quick response.

Large-scale optimization arises in a variety of scientific and engineering applications. In this paper, a particle swarm optimization (PSO) approach with dynamic neighborhood that is based on kernel fuzzy clustering and variable trust region methods (called FT-DNPSO) is proposed for large-scale optimization. The cooperative coevolution incorporated with a kernel fuzzy C-means clustering strategy is introduced to divide high-dimensional problems in to subproblems, and explore their search spaces. Furthermore, the independent variable ranges change adaptably by using the variable trust region learning method, which expedites the convergence process and explores in the effective space. In addition, the dynamic neighborhood topology assists the PSO algorithm in cooperating with neighbor particles and avoids the problem of premature convergence. Simulation results substantiate the effectiveness of the proposed algorithm to solve large-scale optimization problems with many well-known benchmark functions.

This paper presents new theoretical results on the invariance and attractivity of memristor-based cellular neural networks (MCNNs) with time-varying delays. First, sufficient conditions to assure the boundedness and global attractivity of the networks are derived. Using state-space decomposition and some analytic techniques, it is shown that the number of equilibria located in the saturation regions of the piecewise-linear activation functions of an n-neuron MCNN with time-varying delays increases significantly from 2(n) to 2(2n2)+n) (2(2n2) times) compared with that without a memristor. In addition, sufficient conditions for the invariance and local or global attractivity of equilibria or attractive sets in any designated region are derived. Finally, two illustrative examples are given to elaborate the characteristics of the results in detail.

This paper presents new theoretical results on the passivity and passification of a class of memristor-based recurrent neural networks (MRNNs) with time-varying delays. The casual assumptions on the boundedness and Lipschitz continuity of neuronal activation functions are relaxed. By constructing appropriate Lyapunov-Krasovskii functionals and using the characteristic function technique, passivity conditions are cast in the form of linear matrix inequalities (LMIs), which can be checked numerically using an LMI toolbox. Based on these conditions, two procedures for designing passification controllers are proposed, which guarantee that MRNNs with time-varying delays are passive. Finally, two illustrative examples are presented to show the characteristics of the main results in detail.

This paper presents a neurodynamic optimization approach to robust pole assignment for synthesizing linear control systems via state and output feedback. The problem is formulated as a pseudoconvex optimization problem with robustness measure: i.e., the spectral condition number as the objective function and linear matrix equality constraints for exact pole assignment. Two coupled recurrent neural networks are applied for solving the formulated problem in real time. In contrast to existing approaches, the exponential convergence of the proposed neurodynamics to global optimal solutions can be guaranteed even with lower model complexity in terms of the number of variables. Simulation results of the proposed neurodynamic approach for 11 benchmark problems are reported to demonstrate its superiority.

This paper presents one-layer projection neural networks based on projection operators for solving constrained variational inequalities and related optimization problems. Sufficient conditions for global convergence of the proposed neural networks are provided based on Lyapunov stability. Compared with the existing neural networks for variational inequalities and optimization, the proposed neural networks have lower model complexities. In addition, some improved criteria for global convergence are given. Compared with our previous work, a design parameter has been added in the projection neural network models, and it results in some improved performance. The simulation results on numerical examples are discussed to demonstrate the effectiveness and characteristics of the proposed neural networks.

This paper presents a neural network approach to robust model predictive control (MPC) for constrained discrete-time nonlinear systems with unmodeled dynamics affected by bounded uncertainties. The exact nonlinear model of underlying process is not precisely known, but a partially known nominal model is available. This partially known nonlinear model is first decomposed to an affine term plus an unknown high-order term via Jacobian linearization. The linearization residue combined with unmodeled dynamics is then modeled using an extreme learning machine via supervised learning. The minimax methodology is exploited to deal with bounded uncertainties. The minimax optimization problem is reformulated as a convex minimization problem and is iteratively solved by a two-layer recurrent neural network. The proposed neurodynamic approach to nonlinear MPC improves the computational efficiency and sheds a light for real-time implementability of MPC technology. Simulation results are provided to substantiate the effectiveness and characteristics of the proposed approach.

This paper presents new results on neurodynamic optimization approaches to robust pole assignment based on four alternative robustness measures. One or two recurrent neural networks are utilized to optimize these measures while making exact pole assignment. Compared with existing approaches, the present neurodynamic approaches can result in optimal robustness in most cases with one of the robustness measures. Simulation results of the proposed approaches for many benchmark problems are reported to demonstrate their performances.

This paper presents a one-layer projection neural network for solving nonsmooth optimization problems with generalized convex objective functions and subject to linear equalities and bound constraints. The proposed neural network is designed based on two projection operators: linear equality constraints, and bound constraints. The objective function in the optimization problem can be any nonsmooth function which is not restricted to be convex but is required to be convex (pseudoconvex) on a set defined by the constraints. Compared with existing recurrent neural networks for nonsmooth optimization, the proposed model does not have any design parameter, which is more convenient for design and implementation. It is proved that the output variables of the proposed neural network are globally convergent to the optimal solutions provided that the objective function is at least pseudoconvex. Simulation results of numerical examples are discussed to demonstrate the effectiveness and characteristics of the proposed neural network.

This paper presents a neurodynamic approach to bicriteria model predictive control (MPC) of nonlinear affine systems based on a goal programming formulation. Bicriteria MPC refers to finding optimal control inputs that minimizes two performance indexes corresponding to tracking errors and control efforts. The bicriteria MPC is formulated as the solution to a nonlinear optimization problem via goal programming technique and is solved by using a two-layer recurrent neural network. Simulation results are included to illustrate the effectiveness of the proposed approach.

Based on a recurrent neural network, a model predictive control (MPC) method for control of a class of autonomous underwater vehicles (AUVs) is presented. A coupled nonlinear kinematic model with constrains is considered. The model predictive control problem of AUVs is formulated as a time-varying quadratic programming problem, and a one-layer recurrent neural network called the simplified dual network is applied for real-time optimization. It is able to converge to the global optimal solution of the constrained optimization problem. Simulation results are discussed to demonstrate the effectiveness and characteristics of the proposed model predictive control method.

This paper presents new results on a neural network approach to nonlinear model predictive control. At first, a nonlinear system with unmodeled dynamics is decomposed by means of Jacobian linearization to an affine part and a higher-order unknown term. The unknown higher-order term resulted from the decomposition, together with the unmodeled dynamics of the original plant, are modeled by using a feedforward neural network via supervised learning. The optimization problem for nonlinear model predictive control is then formulated as a quadratic programming problem based on successive Jacobian linearization about varying operating points and iteratively solved by using a recurrent neural network called the simplified dual network. Simulation results are included to substantiate the effectiveness and illustrate the performance of the proposed approach.

The assignment problem is an archetypal combinatorial optimization problem. In this brief, we present a continuous-time version and a discrete-time version of the improved dual neural network (IDNN) for solving the assignment problem. Compared with most assignment networks in the literature, the two versions of IDNNs are advantageous in circuit implementation due to their simple structures. Both of them are theoretically guaranteed to be globally convergent to a solution of the assignment problem if only the solution is unique.

Since the last decade, several complex-valued neural networks have been developed and applied in various research areas. As an extension of real-valued recurrent neural networks, complex-valued recurrent neural networks use complex-valued states, connection weights, or activation functions with much more complicated properties than real-valued ones. This paper presents several sufficient conditions derived to ascertain the existence of unique equilibrium, global asymptotic stability, and global exponential stability of delayed complex-valued recurrent neural networks with two classes of complex-valued activation functions. Simulation results of three numerical examples are also delineated to substantiate the effectiveness of the theoretical results.

In this paper, a robust recurrent neural network is presented in a Bayesian framework based on echo state mechanisms. Since the new model is capable of handling outliers in the training data set, it is termed as a robust echo state network (RESN). The RESN inherits the basic idea of ESN learning in a Bayesian framework, but replaces the commonly used Gaussian distribution with a Laplace one, which is more robust to outliers, as the likelihood function of the model output. Moreover, the training of the RESN is facilitated by employing a bound optimization algorithm, based on which, a proper surrogate function is derived and the Laplace likelihood function is approximated by a Gaussian one, while remaining robust to outliers. It leads to an efficient method for estimating model parameters, which can be solved by using a Bayesian evidence procedure in a fully autonomous way. Experimental results show that the proposed method is robust in the presence of outliers and is superior to existing methods.

In recent years, the global stability of recurrent neural networks (RNNs) has been investigated extensively. It is well known that time delays and external disturbances can derail the stability of RNNs. In this paper, we analyze the robustness of global stability of RNNs subject to time delays and random disturbances. Given a globally exponentially stable neural network, the problem to be addressed here is how much time delay and noise the RNN can withstand to be globally exponentially stable in the presence of delay and noise. The upper bounds of the time delay and noise intensity are characterized by using transcendental equations for the RNNs to sustain global exponential stability. Moreover, we prove theoretically that, for any globally exponentially stable RNNs, if additive noises and time delays are smaller than the derived lower bounds arrived at here, then the perturbed RNNs are guaranteed to also be globally exponentially stable. Three numerical examples are provided to substantiate the theoretical results.

In this paper, we propose an efficient algorithm for computing the Euclidean distance transform of two-dimensional binary image, called PBEDT (Perpendicular Bisector Euclidean Distance Transform). PBEDT is a two-stage independent scan algorithm. In the first stage, PBEDT computes the distance from each point to its closest feature point in the same column using one time column-wise scan. In the second stage, PBEDT computes the distance transform for each point by row with intermediate results of the previous stage. By using the geometric properties of the perpendicular bisector, PBEDT directly computes the segmentation by feature points for each row and each segment corresponding to one feature point. Furthermore, by using integer arithmetic to avoid time consuming float operations, PBEDT still achieves exact results. All these methods reduce the computational complexity significantly. Consequently, an efficient and exact linear time Euclidean distance transform algorithm is implemented. Detailed comparison with state-of-the-art linear time Euclidean distance transform algorithms shows that PBEDT is the fastest on most cases, and also the most stable one with respect to image contents.

In this paper, we propose a genetic programming algorithm to design the morphological image enhancement procedure. Given a group of morphological operations and logical operations as function set, this algorithm evolves to produce a rational procedure which can enhance the input images. A novel mechanism which combines the ground truth method and feature significance is brought forward to evaluate the performance of images enhanced by generated procedures. In each generation, the best fitted individuals are selected on the basis of fitness values, and some individuals participate in crossover or mutation with a probability. After each generation, this algorithm outputs the best individual. Seven morphological operations and five logical operations are used in this algorithm. Furthermore, the structuring elements of morphological operations are randomly generated and varied in the whole pattern space. These methods promote the expressive ability of generated procedures. Examined by the binary image feature extraction, the procedure generated by this algorithm is more accurate and intelligible than previous work. In the task of gray scale image enhancement, the generated procedure is applied to infrared finger vein images to enhance the region of interest. More accurate features are extracted and the accuracy of authentication is promoted.

A robust model predictive control (MPC) method is proposed for nonlinear affine systems with bounded disturbances. The robust MPC technique requires on-line solution of a minimax optimal control problem. The minimax strategy means that worst-case performance with respect to uncertainties is optimized. The minimax optimization problem involved in robust MPC is reformulated to a minimization problem and then is solved by using a two-layer recurrent neural network. Simulation examples are included to illustrate the effectiveness of the proposed method.

Recently, a continuous-time k-winners-take-all (kWTA) network with a single state variable and a hard limiting activation function and its discrete-time counterpart were developed. These kWTA networks have proven properties of finite-time global convergence and simple architectures. In this paper, the kWTA networks are applied for information retrieval, such as web search. The weights or scores of pages in two real world data sets are calculated with the PageRank algorithm, based on which experimental results of kWTA networks are provided. The results show that the kWTA networks converge faster as the size of the problem grows, which renders them as a promising approach to large-scale data set information retrieval problems.

In this paper, a one-layer recurrent neural network is presented for solving single-ration linear fractional programming problems subject to linear equality and box bound constraints. The convergence condition is derived to guarantee the solution optimality to the fractional programming problems if the design parameters in the neural network are larger than the derived lower bounds. Two numerical examples with simulation results show that the proposed neural network is efficient and accurate for solving constrained linear fractional programming problems.

In this paper, a one-layer recurrent neural network is presented for solving pseudoconvex optimization problems subject to linear equality constraints. The global convergence of the neural network can be guaranteed even though the objective function is pseudoconvex. The finite-time state convergence to the feasible region defined by the equality constraints is also proved. In addition, global exponential convergence is proved when the objective function is strongly pseudoconvex on the feasible region. Simulation results on illustrative examples and application on chemical process data reconciliation are provided to demonstrate the effectiveness and characteristics of the neural network.

A dynamic feedforward neural network (DFNN) is proposed for predictive control, whose adaptive parameters are adjusted by using Gaussian particle swarm optimization (GPSO) in the training process. Adaptive time-delay operators are added in the DFNN to improve its generalization for poorly known nonlinear dynamic systems with long time delays. Furthermore, GPSO adopts a chaotic map with Gaussian function to balance the exploration and exploitation capabilities of particles, which improves the computational efficiency without compromising the performance of the DFNN. The stability of the particle dynamics is analyzed, based on the robust stability theory, without any restrictive assumption. A stability condition for the GPSO+DFNN model is derived, which ensures a satisfactory global search and quick convergence, without the need for gradients. The particle velocity ranges could change adaptively during the optimization process. The results of a comparative study show that the performance of the proposed algorithm can compete with selected algorithms on benchmark problems. Additional simulation results demonstrate the effectiveness and accuracy of the proposed combination algorithm in identifying and controlling nonlinear systems with long time delays.

This paper presents a one-layer recurrent neural network for solving a class of constrained nonsmooth optimization problems with piecewise-linear objective functions. The proposed neural network is guaranteed to be globally convergent in finite time to the optimal solutions under a mild condition on a derived lower bound of a single gain parameter in the model. The number of neurons in the neural network is the same as the number of decision variables of the optimization problem. Compared with existing neural networks for optimization, the proposed neural network has a couple of salient features such as finite-time convergence and a low model complexity. Specific models for two important special cases, namely, linear programming and nonsmooth optimization, are also presented. In addition, applications to the shortest path problem and constrained least absolute deviation problem are discussed with simulation results to demonstrate the effectiveness and characteristics of the proposed neural network.

In this paper, a one-layer recurrent neural network is proposed for solving nonconvex optimization problems subject to general inequality constraints, designed based on an exact penalty function method. It is proved herein that any neuron state of the proposed neural network is convergent to the feasible region in finite time and stays there thereafter, provided that the penalty parameter is sufficiently large. The lower bounds of the penalty parameter and convergence time are also estimated. In addition, any neural state of the proposed neural network is convergent to its equilibrium point set which satisfies the Karush-Kuhn-Tucker conditions of the optimization problem. Moreover, the equilibrium point set is equivalent to the optimal solution to the nonconvex optimization problem if the objective function and constraints satisfy given conditions. Four numerical examples are provided to illustrate the performances of the proposed neural network.

This paper gives an applicable genetic programming(GP) approach to solve the binary image analysis and gray scale image enhancement problems. By showing a section of binary image and the corresponding goal image, this algorithm automatically produces a mathematic morphological operation sequence to transform the target into the goal. While the operation sequence is applied to the whole image, the objective of image analysis is achieved. With well-defined chromosome structure and evolution strategy, the effectiveness of evolution is promoted and more complex morphological operations can be composed in a short sequence. In addition, this algorithm is also applied to infrared finger vein gray scale images to enhance the region of interest. Whose effect is examined by an application of identity authentication, and the accuracy of authentication is promoted.

In recent years, constrained sparsity maximization problems received tremendous attention in the context of compressive sensing. Because the formulated constrained L0norm minimization problem is NP-hard, constrained L1norm minimization is usually used to compute approximate sparse solutions. In this paper, we introduce several alternative objective functions, such as weighted L1norm, Laplacian, hyperbolic secant, and Gaussian functions, as approximations of the L0norm. A one-layer recurrent neural network is applied to compute the optimal solutions to the reformulated constrained minimization problems subject to equality constraints. Simulation results in terms of time responses, phase diagrams, and tabular data are provided to demonstrate the superior performance of the proposed neurodynamic optimization approach to constrained sparsity maximization based on the problem reformulations.

Memristor is a newly prototyped nonlinear circuit device. Its value is not unique and changes according to the value of the magnitude and polarity of the voltage applied to it. In this paper, a simplified mathematical model is proposed to characterize the pinched hysteretic feature of the memristor, a memristor-based recurrent neural network model is given, and its global stability is studied. Using differential inclusion, two sufficient conditions for the global uniform asymptotic stability of memristor-based recurrent neural networks are obtained.

This paper presents a recurrent neural network (RNN) approach to nonlinear model predictive control (MPC). By using decomposition, the original optimization associated with nonlinear MPC is reformulated as a quadratic programming problem with unknown parameters. We employ an RNN and develop a learning algorithm for solving the formulated problem. The proposed RNN approach has many desirable properties such as global convergence and low complexity. Finally, we apply the neurodynamic approach to mobile robot navigation to demonstrate its effectiveness and efficiency.

This paper presents a k-winners-take-all (kWTA) neural network with a single state variable and a hard-limiting activation function. First, following several kWTA problem formulations, related existing kWTA networks are reviewed. Then, the kWTA model model with a single state variable and a Heaviside step activation function is described and its global stability and finite-time convergence are proven with derived upper and lower bounds. In addition, the initial state estimation and a discrete-time version of the kWTA model are discussed. Furthermore, two selected applications to parallel sorting and rank-order filtering based on the kWTA model are discussed. Finally, simulation results show the effectiveness and performance of the kWTA model.

This paper presents a new design procedure for the synthesis of associative memories based on cellular neural networks characterized by input and output matrices obtained based on cloning templates by solving a set of inequalities. The design parameters are few even though the dimension of patterns may be very high. The design procedure enables heteroassociative or autoassociative memories to be synthesized with assured global exponential stability and feeding retrieval probes via external inputs rather than initial conditions. Two specific examples are shown to illustrate the applicability of the methodology.

In this paper, a design procedure is presented for synthesizing associative memories based on the brain-state-in-a-box neural network model. The theoretical analysis herein guarantees that the desired memory patterns are stored as asymptotically stable equilibrium points with very few spurious states. In order to avoid extensive computation, learning and forgetting are utilized by adding patterns to be stored as asymptotically stable equilibrium points to an existing set of stored patterns and deleting specified patterns from a given set of stored patterns without affecting the rest in a given network. Furthermore, the number of the memorized patterns in a designed brain-state-in-a-box neural network model can be made much more than that of neurons. Simulation results demonstrate the validity and characteristics of the proposed approach.

Inverse kinematic motion planning of redundant manipulators by using recurrent neural networks in the presence of obstacles and uncertainties is a real-time nonlinear optimization problem. To tackle this problem, two subproblems should be resolved in real time. One is the determination of critical points on a given manipulator closest to obstacles, and the other is the computation of joint velocities of the manipulator which can direct the manipulator following a desired trajectory and away from obstacles if it is getting close to them. Different from our previous approaches where the critical points on the manipulator were assumed to be known, these points are to be computed by using a recurrent neural network in the paper. A time-varying quadratic programming problem is formulated for avoiding polyhedral obstacles. In view that the problem is not strictly convex, an existing recurrent neural network, general projection neural network, is applied for solving it. By introducing a velocity smoothing technique into our previous quadratic programming formulation of the joint velocity assignment problem, a recently developed recurrent neural network, improved dual neural network, is proposed to solve it, which features lower structural complexity compared with existing neural networks. Moreover, The effectiveness of the proposed neural networks is demonstrated by simulations on the Mitsubishi PA10-7C manipulator.

This paper presents new results on blind separation of instantaneously mixed independent sources based on high-order statistics together with their time and frequency non-properties (i.e., the non-stationarity and non-whiteness of sources). Separation criteria of mixtures are established on a set of cumulants at different time instants using the non-stationarity of sources and/or time-delayed cumulants using the non-whiteness of sources. It is shown that cumulants at different time instants and time-delayed cumulants can be used as criteria for blind source separation (BSS). Furthermore, it is proved that the cumulant-based separation criteria are directly related to the separability conditions. Batch-data and online learning rules are developed based on the joint diagonalization of symmetric fourth-order cumulant matrices, and the learning rules are further simplified to correlation-based BSS algorithms. In addition, an initialization strategy is proposed for improving the convergence of the learning rules. Simulation results are given to demonstrate the validity and performance of the algorithms.

This paper presents new stability results for recurrent neural networks with Markovian switching. First, algebraic criteria for the almost sure exponential stability of recurrent neural networks with Markovian switching and without time delays are derived. The results show that the almost sure exponential stability of such a neural network does not require the stability of the neural network at every individual parametric configuration. Next, both delay-dependent and delay-independent criteria for the almost sure exponential stability of recurrent neural networks with time-varying delays and Markovian-switching parameters are derived by means of a generalized stochastic Halanay inequality. The results herein include existing ones for recurrent neural networks without Markovian switching as special cases. Finally, simulation results in three numerical examples are discussed to illustrate the theoretical results.

This paper presents a one-layer recurrent neural network for solving convex programming problems subject to linear equality and nonnegativity constraints. The number of neurons in the neural network is equal to that of decision variables in the optimization problem. Compared with the existing neural networks for optimization, the proposed neural network has lower model complexity. Moreover, the proposed neural network is proved to be globally convergent to the optimal solution(s) under some mild conditions. Simulation results show the effectiveness and performance of the proposed neural network.

As linear model predictive control (MPC) becomes a standard technology, nonlinear MPC (NMPC) approach is debuting both in academia and industry. In this paper, the NMPC problem is formulated as a convex quadratic programming problem based on nonlinear model prediction and linearization. A recurrent neural network for NMPC is then applied for solving the quadratic programming problem. The proposed network is globally convergent to the optimal solution of the NMPC problem. Simulation results are presented to show the effectiveness and performance of the neural network approach.

A one-layer recurrent neural network with a discontinuous activation function is proposed for linear programming. The number of neurons in the neural network is equal to that of decision variables in the linear programming problem. It is proven that the neural network with a sufficiently high gain is globally convergent to the optimal solution. Its application to linear assignment is discussed to demonstrate the utility of the neural network. Several simulation examples are given to show the effectiveness and characteristics of the neural network.

This paper presents a novel recurrent neural network for solving a class of convex quadratic programming (QP) problems, in which the quadratic term in the objective function is the square of the Euclidean norm of the variable. This special structure leads to a set of simple optimality conditions for the problem, based on which the neural network model is formulated. Compared with existing neural networks for general convex QP, the new model is simpler in structure and easier to implement. The new model can be regarded as an improved version of the dual neural network in the literature. Based on the new model, a simple neural network capable of solving the k-winners-take-all ( k-WTA) problem is formulated. The stability and global convergence of the proposed neural network is proved rigorously and substantiated by simulation results.

In this paper, a one-layer recurrent neural network with a discontinuous hard-limiting activation function is proposed for quadratic programming. This neural network is capable of solving a large class of quadratic programming problems. The state variables of the neural network are proven to be globally stable and the output variables are proven to be convergent to optimal solutions as long as the objective function is strictly convex on a set defined by the equality constraints. In addition, a sequential quadratic programming approach based on the proposed recurrent neural network is developed for general nonlinear programming. Simulation results on numerical examples and support vector machine (SVM) learning show the effectiveness and performance of the neural network.

This brief paper presents an M-matrix-based algebraic criterion for the global exponential stability of a class of recurrent neural networks with decreasing time-varying delays. The criterion improves some previous criteria based on M-matrix and is easy to be verified with the connection weights of the recurrent neural networks with decreasing time-varying delays. In addition, the rate of exponential convergence can be estimated via a simple computation based on the criterion herein.

This paper presents a novel recurrent neural network for solving nonlinear optimization problems with inequality constraints. Under the condition that the Hessian matrix of the associated Lagrangian function is positive semidefinite, it is shown that the proposed neural network is stable at a Karush-Kuhn-Tucker point in the sense of Lyapunov and its output trajectory is globally convergent to a minimum solution. Compared with variety of the existing projection neural networks, including their extensions and modification, for solving such nonlinearly constrained optimization problems, it is shown that the proposed neural network can solve constrained convex optimization problems and a class of constrained nonconvex optimization problems and there is no restriction on the initial point. Simulation results show the effectiveness of the proposed neural network in solving nonlinearly constrained optimization problems.

This paper presents a design method for synthesizing associative memories based on discrete-time recurrent neural networks. The proposed procedure enables both hetero- and autoassociative memories to be synthesized with high storage capacity and assured global asymptotic stability. The stored patterns are retrieved by feeding probes via external inputs rather than initial conditions. As typical representatives, discrete-time cellular neural networks (CNNs) designed with space-invariant cloning templates are examined in detail. In particular, it is shown that procedure herein can determine the input matrix of any CNN based on a space-invariant cloning template which involves only a few design parameters. Two specific examples and many experimental results are included to demonstrate the characteristics and performance of the designed associative memories.

In this paper, an improved particle swarm optimization algorithm with momentum (mPSO) is proposed based on inspiration from the back propagation (BP) learning algorithm with momentum in neural networks. The momentum acts as a lowpass filter to relieve excessive oscillation and also extends the PSO velocity updating equation to a second-order difference equation. Experimental results are shown to verify its superiority both in robustness and efficiency.

In this paper, the K-Winners-Take-All (KWTA) problem is formulated equivalently to a linear program. A recurrent neural network for KWTA is then proposed for solving the linear programming problem. The KWTA network is globally convergent to the optimal solution of the KWTA problem. Simulation results are further presented to show the effectiveness and performance of the KWTA network.

This paper presents a one-layer recurrent neural network with a unipolar hard-limiting activation function for k-winners-take-all (kWTA) operation. The kWTA operation is first converted into an equivalent quadratic programming problem. Then a one-layer recurrent neural network is constructed. The neural network is guaranteed to be capable of performing the kWTA operation in real time. The stability and convergence of the neural network are proven by using Lyapunov and nonsmooth analysis methods.

In this letter, some sufficient conditions are obtained to guarantee recurrent neural networks with linear saturation activation functions, and time-varying delays have multiequilibria located in the saturation region and the boundaries of the saturation region. These results on pattern characterization are used to analyze and design autoassociative memories, which are directly based on the parameters of the neural networks. Moreover, a formula for the numbers of spurious equilibria is also derived. Four design procedures for recurrent neural networks with linear saturation activation functions and time-varying delays are developed based on stability results. Two of these procedures allow the neural network to be capable of learning and forgetting. Finally, simulation results demonstrate the validity and characteristics of the proposed approach.

Generalized linear variational inequality (GLVI) is an extension of the canonical linear variational inequality. In recent years, a recurrent neural network (NN) called general projection neural network (GPNN) was developed for solving GLVIs with simple bound (often box-type or sphere-type) constraints. The aim of this paper is twofold. First, some further stability results of the GPNN are presented. Second, the GPNN is extended for solving GLVIs with general linear equality and inequality constraints. A new design methodology for the GPNN is then proposed. Furthermore, in view of different types of constraints, approaches for reducing the number of neurons of the GPNN are discussed, which results in two specific GPNNs. Moreover, some distinct properties of the resulting GPNNs are also explored based on their particular structures. Numerical simulation results are provided to validate the results.

The stabilization of recurrent neural networks with mixed time-varying delays and Markovian-switching parameters by noise is discussed. First, a new result is given for the existence of unique states of recurrent neural networks (NNs) with mixed time-varying delays and Markovian-switching parameters in the presence of noise, without the need to satisfy the linear growth conditions required by general stochastic Markovian-switching systems. Next, a delay-dependent condition for stabilization of concerned recurrent NNs is derived by applying the ltd formula, the Gronwall inequality, the law of large numbers, and the ergodic property of Markovian chain. The results show that there always exists an appropriate white noise such that any recurrent NNs with mixed time-varying delays and Markovian-switching parameters can be exponentially stabilized by noise if the delays are sufficiently small.

This paper presents a recurrent neural-network model for solving a special class of general variational inequalities (GVIs), which includes classical VIs as special cases. It is proved that the proposed neural network (NN) for solving this class of GVIs can be globally convergent, globally asymptotically stable, and globally exponentially stable under different conditions. The proposed NN can be viewed as a modified version of the general projection NN existing in the literature. Several numerical examples are provided to demonstrate the effectiveness and performance of the proposed NN.

Most existing neural networks for solving linear variational inequalities (LVIs) with the mapping Mx + p require positive definiteness (or positive semidefiniteness) of M. In this correspondence, it is revealed that this condition is sufficient but not necessary for an LVI being strictly monotone (or monotone) on its constrained set where equality constraints are present. Then, it is proposed to reformulate monotone LVIs with equality constraints into LVIs with inequality constraints only, which are then possible to be solved by using some existing neural networks. General projection neural networks are designed in this correspondence for solving the transformed LVIs. Compared with existing neural networks, the designed neural networks feature lower model complexity. Moreover, the neural networks are guaranteed to be globally convergent to solutions of the LVI under the condition that the linear mapping Mx + p is monotone on the constrained set. Because quadratic and linear programming problems are special cases of LVI in terms of solutions, the designed neural networks can solve them efficiently as well. In addition, it is discovered that the designed neural network in a specific case turns out to be the primal-dual network for solving quadratic or linear programming problems. The effectiveness of the neural networks is illustrated by several numerical examples.

An existing recurrent neural network for convex optimization is extended to solve nonconvex optimization problems. One of the prominent features of this neural network is the one-to-one correspondence between its equilibria and the Karush-Kuhn-Tucker (KKT) points of the nonconvex optimization problem. The conditions are derived under which the neural network (locally) converges to the KKT points. It is desired that the neural network is stable at minimum solutions, and unstable at maximum solutions or saddle solutions. It is found in the paper that most likely the neural network is unstable at the maximum solutions. Moreover, we found that if the derived conditions are not satisfied at minimum solutions, by transforming the original problem into an equivalent one with the p-power (or partial p-power) method, these conditions can be satisfied. As a result, the neural network will locally converge to a minimum solution. Finally, two illustrative examples are provided to demonstrate the performance of the recurrent neural network.

In this paper, a design procedure is presented for synthizing associative memories based on cellular neural networks with two-dimensional space-invariant templates. The theoretical analysis herein guarantees that the desired memory patterns are stored as asymptotically stable equilibrium points. In addition, it is shown that procedure herein can ensure the designed input matrix to be obtained by using space-invariant cloning templates. Hence, it is very convenient for one to design cellular neural networks for associative desired memory patterns effectively.

In this paper, we present new extractability conditions for blind source extraction of linear instantaneous mixtures. Two general conditions for source extraction of arbitrarily mixed nonzero sources are presented. A sufficient condition is provided to guarantee that sequential extraction can be continued. We also show an important property for inseparable mixtures; that is, any two extracted signals involving the same sources are proportional to each other. For sub-Gaussian or sup-Gaussian source signals with only mutual independence, cost functions based on fourth-order cumulants are introduced to sequentially extract all separable single sources and all inseparable mixtures. By minimizing the cost functions, gradient-based methods are developed. Our algorithms are guaranteed to converge. Finally, simulation results show the operation characteristics and the effectiveness of our methods

Solving variational inequality problems by using neural networks are of great interest in recent years. To date, most work in this direction focus on solving monotone variational inequalities. In this paper, we show that an existing recurrent neural network proposed originally for solving monotone variational inequalities can be used to solve pseudomonotone variational inequalities with proper choice of a system parameter. The global convergence, global asymptotic stability and global exponential stability of the neural network are discussed under various conditions. The existing stability results are thus extended in view of the fact that pseudomonotonicity is a weaker condition than monotonicity

We show that an n-neuron cellular neural network with time-varying delay can have 2n periodic orbits located in saturation regions and these periodic orbits are locally exponentially attractive. In addition, we give some conditions for ascertaining periodic orbits to be locally or globally exponentially attractive and allow them to locate in any designated region. As a special case of exponential periodicity, exponential stability of delayed cellular neural networks is also characterized. These conditions improve and extend the existing results in the literature. To illustrate and compare the results, simulation results are discussed in three numerical examples.

In recent years, a recurrent neural network called projection neural network was proposed for solving monotone variational inequalities and related convex optimization problems. In this paper, we show that the projection neural network can also be used to solve pseudomonotone variational inequalities and related pseudoconvex optimization problems. Under various pseudomonotonicity conditions and other conditions, the projection neural network is proved to be stable in the sense of Lyapunov and globally convergent, globally asymptotically stable, and globally exponentially stable. Since monotonicity is a special case of pseudomononicity, the projection neural network can be applied to solve a broader class of constrained optimization problems related to variational inequalities. Moreover, a new concept, called componentwise pseudomononicity, different from pseudomononicity in general, is introduced. Under this new concept, two stability results of the projection neural network for solving variational inequalities are also obtained. Finally, numerical examples show the effectiveness and performance of the projection neural network.

The design, analysis, and application of a new recurrent neural network for quadratic programming, called simplified dual neural network, are discussed. The analysis mainly concentrates on the convergence property and the computational complexity of the neural network. The simplified dual neural network is shown to be globally convergent to the exact optimal solution. The complexity of the neural network architecture is reduced with the number of neurons equal to the number of inequality constraints. Its application to k-winners-take-all (KWTA) operation is discussed to demonstrate how to solve problems with this neural network.

This paper presents new theoretical results on global exponential stability of recurrent neural networks with bounded activation functions and time-varying delays. The stability conditions depend on external inputs, connection weights, and time delays of recurrent neural networks. Using these results, the global exponential stability of recurrent neural networks can be derived, and the estimated location of the equilibrium point can be obtained. As typical representatives, the Hopfield neural network (HNN) and the cellular neural network (CNN) are examined in detail.

In this paper, the multiperiodicity of a general class of discrete-time delayed neural networks (DTDNNs) is formulated and studied. Several sufficient conditions are obtained to ensure n-neuron DTDNNs can have 2n periodic orbits and these periodic orbits are locally attractive. In addition, we give the conditions for a periodic orbit to be locally or globally attractive when the periodic orbit locates in a designated region. As two typical representatives, the Hopfield neural network and the cellular neural network are examined in detail. These conditions improve and extend the existing stability results in the literature. Simulations results are also discussed in three illustrative examples.

In this paper, the k-winners-take-all (KWTA) operation is converted to an equivalent constrained convex quadratic optimization formulation. A simplified dual neural network, called KWTA network, is further developed for solving the convex quadratic programming (QP) problem. The KWTA network is shown to be globally convergent to the exact optimal solution of the QP problem. Simulation results are presented to show the effectiveness and performance of the KWTA network.

The bi-criteria joint torque optimization of kinematically redundant manipulators balances between the energy consumption and the torque distribution among the joints. In this paper, a simplified dual neural network is proposed to solve this problem. Joint torque limits are incorporated simultaneously into the proposed optimization scheme. The simplified dual network has less numbers of neurons compared with other recurrent neural networks and is proved to be globally convergent to optimal solutions. The control scheme based on the recurrent neural network is simulated with the PUMA 560 robot manipulator to demonstrate effectiveness.

In this paper, we present the analytical results on the global exponential periodicity of a class of recurrent neural networks with oscillating parameters and time-varying delays. Sufficient conditions are derived for ascertaining the existence, uniqueness and global exponential periodicity of the oscillatory solution of such recurrent neural networks by using the comparison principle and mixed monotone operator method. The periodicity results extend or improve existing stability results for the class of recurrent neural networks with and without time delays.

In this paper, we propose a recurrent neural network for solving nonlinear convex programming problems with linear constraints. The proposed neural network has a simpler structure and a lower complexity for implementation than the existing neural networks for solving such problems. It is shown here that the proposed neural network is stable in the sense of Lyapunov and globally convergent to an optimal solution within a finite time under the condition that the objective function is strictly convex. Compared with the existing convergence results, the present results do not require Lipschitz continuity condition on the objective function. Finally, examples are provided to show the applicability of the proposed neural network.

This paper proposes a primal-dual neural network with a one-layer structure for online resolution of constrained kinematic redundancy in robot motion control. Unlike the Lagrangian network, the proposed neural network can handle physical constraints, such as joint limits and joint velocity limits. Compared with the existing primal-dual neural network, the proposed neural network has a low complexity for implementation. Compared with the existing dual neural network, the proposed neural network has no computation of matrix inversion. More importantly, the proposed neural network is theoretically proved to have not only a finite time convergence, but also an exponential convergence rate without any additional assumption. Simulation results show that the proposed neural network has a faster convergence rate than the dual neural network in effectively tracking for the motion control of kinematically redundant manipulators.

Variational inequalities with nonlinear inequality constraints are widely used in optimization and engineering problems. This paper present a recurrent neural network for solving variational inequalities with nonlinear inequality constraints in real time. The proposed neural network has one-layer structure and is amenable to parallel implementation. The proposed neural network is a significant generalization of several existing neural networks for optimization. Moreover, the proposed neural network is stable in the sense of Lyapunov and globally convergent to an optimal solution under a strictly monotone condition of the mapping. The simulation shows that the proposed neural network is effective for solving this class of variational inequality problems.

We present a mechanism to train support vector machines (SVMs) with a hybrid kernel and minimal Vapnik-Chervonenkis (VC) dimension. After describing the VC dimension of sets of separating hyperplanes in a high-dimensional feature space produced by a mapping related to kernels from the input space, we proposed an optimization criterion to design SVMs by minimizing the upper bound of the VC dimension. This method realizes a structural risk minimization and utilizes a flexible kernel function such that a superior generalization over test data can be obtained. In order to obtain a flexible kernel function, we develop a hybrid kernel function and a sufficient condition to be an admissible Mercer kernel based on common Mercer kernels (polynomial, radial basis function, two-layer neural network, etc.). The nonnegative combination coefficients and parameters of the hybrid kernel are determined subject to the minimal upper bound of the VC dimension of the learning machine. The use of the hybrid kernel results in a better performance than those with a single common kernel. Experimental results are discussed to illustrate the proposed method and show that the SVM with the hybrid kernel outperforms that with a single common kernel in terms of generalization power.

Recently, a projection neural network for solving monotone variational inequalities and constrained optimization problems was developed. In this paper, we propose a general projection neural network for solving a wider class of variational inequalities and related optimization problems. In addition to its simple structure and low complexity, the proposed neural network includes existing neural networks for optimization, such as the projection neural network, the primal-dual neural network, and the dual neural network, as special cases. Under various mild conditions, the proposed general projection neural network is shown to be globally convergent, globally asymptotically stable, and globally exponentially stable. Furthermore, several improved stability criteria on two special cases of the general projection neural network are obtained under weaker conditions. Simulation results demonstrate the effectiveness and characteristics of the proposed neural network.

Grasping-force optimization of multifingered robotic hands can be formulated as a problem for minimizing an objective function subject to form-closure constraints and balance constraints of external force. This paper presents a novel recurrent neural network for real-time dextrous hand-grasping force optimization. The proposed neural network is shown to be globally convergent to the optimal grasping force. Compared with existing approaches to grasping-force optimization, the proposed neural-network approach has the advantages that the complexity for implementation is reduced, and the solution accuracy is increased, by avoiding the linearization of quadratic friction constraints. Simulation results show that the proposed neural network can achieve optimal grasping force in real time.

In this paper, a neural network approach is presented for the motion control of constrained flexible manipulators, where both the contact force everted by the flexible manipulator and the position of the end-effector contacting with a surface are controlled. The dynamic equations for vibration of flexible link and constrained force are derived. The developed control, scheme can adaptively estimate the underlying dynamics of the manipulator using recurrent neural networks (RNNs). Based on the error dynamics of a feedback controller, a learning rule for updating the connection weights of the adaptive RNN model is obtained. Local stability properties of the control system are discussed. Simulation results are elaborated on for both position and force trajectory tracking tasks in the presence of varying parameters and unknown dynamics, which show that the designed controller performs remarkably well.

This paper presents a one-layer recurrent neural network for support vector machine (SVM) learning in pattern classification and regression. The SVM learning problem is first converted into an equivalent formulation, and then a one-layer recurrent neural network for SVM learning is proposed. The proposed neural network is guaranteed to obtain the optimal solution of support vector classification and regression. Compared with the existing two-layer neural network for the SVM classification, the proposed neural network has a low complexity for implementation. Moreover, the proposed neural network can converge exponentially to the optimal solution of SVM learning. The rate of the exponential convergence can be made arbitrarily high by simply turning up a scaling parameter. Simulation examples based on benchmark problems are discussed to show the good performance of the proposed neural network for SVM learning.

One important issue in the motion planning and control of kinematically redundant manipulators is the obstacle avoidance. In this paper, a recurrent neural network is developed and applied for kinematic control of redundant manipulators with obstacle avoidance capability. An improved problem formulation is proposed in the sense that the collision-avoidance requirement is represented by dynamically-updated inequality constraints. In addition, physical constraints such as joint physical limits are also incorporated directly into the formulation. Based on the improved problem formulation, a dual neural network is developed for the online solution to collision-free inverse kinematics problem. The neural network is simulated for motion control of the PA10 robot arm in the presence of point and window-shaped obstacle.

One important issue in motion planning and kinematic control of redundant manipulators is the real-time obstacle avoidance. Following the previous researches, a new problem formulation has been proposed in the sense that the collision avoidance scheme is described by dynamically-updated inequality constraints, and that physical constraints such as joint limits are also incorporated in the formulation. For real-time computation, the dual neural network is applied for the online solution of obstacle-avoidance inverse-kinematic control problem, and then simulated based on the PA10 robot manipulator in the presence of obstacles.

In this paper, we propose a general projection neural network for solving a wider class of optimization and related problems. In addition to its simple structure and low complexity, the proposed neural network include existing neural networks for optimization, such as the projection neural network, the primal-dual neural network, and the dual neural network, as special cases. Under various mild conditions, the proposed general projection neural network is shown to be globally convergent, globally asymptotically stable, and globally exponentially stable. Furthermore, several improved stability criteria on two special cases of the general projection neural network are obtained under weaker conditions. Simulation results demonstrate the effectiveness and characteristics of the proposed neural network.

In this paper, a new sufficient condition is given for the global asymptotic stability and global exponential output stability of a unique equilibrium points of delayed cellular neural networks (DCNNs) by using Lyapunov method. This condition imposes constraints on the feedback matrices and delayed feedback matrices of DCNNs and is independent of the delay. The obtained results extend and improve upon those in the earlier literature, and this condition is also less restrictive than those given in the earlier references. Two examples compared with the previous results in the literatures are presented and a simulation result is also given.

In this paper, we investigate the problem of global and robust stability of a class of interval Hopfield neural networks that have time-varying delays. Some criteria for the global and robust stability of such networks are derived, by means of constructing suitable Lyapunov functionals for the networks. As a by-product, for the conventional Hopfield neural networks with time-varying delays, we also obtain some new criteria for their global and asymptotic stability.

This paper presents a new result on absolute exponential stability (AEST) of a class of continuous-time recurrent neural networks with locally Lipschitz continuous and monotone nondecreasing activation functions. The additively diagonally stable connection weight matrices are proven to be able to guarantee AEST of the neural networks. The AEST result extends and improves the existing absolute stability and AEST ones in the literature.

In this paper, a recurrent neural network called the dual neural network is proposed for online redundancy resolution of kinematically redundant manipulators. Physical constraints such as joint limits and joint velocity limits, together with the drift-free criterion as a secondary task, are incorporated into the problem formulation of redundancy resolution. Compared to other recurrent neural networks, the dual neural network is piecewise linear and has much simpler architecture with only one layer of neurons. The dual neural network is shown to be globally (exponentially) convergent to optimal solutions. The dual neural network is simulated to control the PA10 robot manipulator with effectiveness demonstrated.

In this paper, we present a quantitative measure of multifingered grasps. The measure quantifies the capability of a grasp in firmly holding an object while resisting external loads and/or disturbances. It can also be used for qualitative test of closure properties (form closure and force closure). For planar grasps and frictionless three-dimensional (3-D) grasps, the quantitative measure can be computed efficiently by solving a set of linear programs, while for frictional 3-D grasps, it can be computed by solving nonlinear programs without linearization of the friction cone. By using the proposed quantitative measure, an algorithm for grasp synthesis on polygonal objects is developed. Rather than producing a single grasp configuration, the algorithm computes all grasps on a polygon that satisfy quantitative constraints, i.e., the value of the quantitative measure is greater than a predetermined positive constant. The approach has potential application in grasp planning with multiple optimality criteria.

The synthesis of force-closure grasps on three-dimensional (3-D) objects is a fundamental issue in robotic grasping and dextrous manipulation. In this paper, a numerical force-closure test is developed based on the concept of Q distance. With some mild and realistic assumptions, the proposed test criterion is differentiable almost everywhere and its derivative can be calculated exactly. On this basis, we present an algorithm for planning force-closure grasps, which is implemented by applying descent search to the proposed numerical test in the grasp configuration space. The algorithm is generally applicable to planning optimal force-closure grasps on 3-D objects with curved surfaces and with arbitrary number of contact points. The effectiveness and efficiency of the algorithm are demonstrated by using simulation examples.

Presents a recurrent neural network for solving the Sylvester equation with time-varying coefficient matrices. The recurrent neural network with implicit dynamics is deliberately developed in the way that its trajectory is guaranteed to converge exponentially to the time-varying solution of a given Sylvester equation. Theoretical results of convergence and sensitivity analysis are presented to show the desirable properties of the recurrent neural network. Simulation results of time-varying matrix inversion and online nonlinear output regulation via pole assignment for the ball and beam system and the inverted pendulum on a cart system are also included to demonstrate the effectiveness and performance of the proposed neural network.

Global exponential stability is the most desirable stability property of recurrent neural networks. The paper presents new results for recurrent neural networks applied to online computation of feedback gains of linear time-invariant multivariable systems via pole assignment. The theoretical analysis focuses on the global exponential stability, convergence rates, and selection of design parameters. The theoretical results are further substantiated by simulation results conducted for synthesizing linear feedback control systems with different specifications and design requirements.

A dual neural network is presented for the bi-criteria kinematic control of redundant manipulators. To diminish the discontinuity of minimum infinity-norm solutions, the kinematic-control problem is formulated in the bi-criteria of the infinity and Euclidean norms. Physical constraints such as joint limits and joint velocity limits are also incorporated simultaneously into the proposed kinematic control scheme. The single-layer dual neural network model with a simple structure is developed for bi-criteria redundant resolution of redundant manipulators subject to robot physical constraints. The dual neural network is shown to be globally convergent to optimal solutions in the bi-criteria sense, and is demonstrated to be effective in controlling the PA10 robot manipulator.

A dual neural network is presented for the real-time joint torque optimization of kinematically redundant manipulators, which corresponds to global kinetic energy minimization of robot mechanisms. Compared to other computational strategies on inverse kinematics, the dual network is developed at the acceleration level to resolve redundancy of limited-joint-range manipulators. The dual network has a simple architecture with only one layer of neurons and is proved to be globally exponentially convergent to optimal solutions. The dual neural network is simulated with the PUMA 560 robot arm to demonstrate effectiveness.

Demixing independent source signals from their nonlinear mixtures is a very important issue in many scenarios. This paper presents a novel method for blindly separating unobservable independent source signals from their nonlinear mixtures. The demixing system is modeled using a parameterized neural network whose parameters can be determined under the criterion of independence of its outputs. Compared to conventional gradient-based approaches, the GA-based approach for blind source separation is characterized by high accuracy, high robustness, and high convergence rate. Simulation results are discussed to demonstrate that the proposed GA-based approach is capable of separating independent sources from their nonlinear mixtures generated by a parametric separation model.

This paper presents a novel method for blindly separating unobservable independent source signals from their nonlinear mixtures. The demixing system is modeled using a parameterized neural network whose parameters can be determined under the criterion of independence of its outputs. Two cost functions based on higher order statistics are established to measure the statistical dependence of the outputs of the demixing system. The proposed method utilizes a genetic algorithm (GA) to minimize the highly nonlinear and nonconvex cost functions. The GA-based global optimization technique is able to obtain superior separation solutions to the nonlinear blind separation problem from any random initial values. Compared to conventional gradient-based approaches, the GA-based approach for blind source separation is characterized by high accuracy, robustness, and convergence rate. In particular, it is very suitable for the case of limited available data. Simulation results are discussed to demonstrate that the proposed GA-based approach is capable of separating independent sources from their nonlinear mixtures generated by a parametric separation model.

This paper proposes a novel neural-network approach to blind source separation in nonlinear mixture. The approach utilizes a radial basis function (RBF) neural-network to approximate the inverse of the nonlinear mixing mapping which is assumed to exist and able to be approximated using an RBF network. A contrast function which consists of the mutual information and partial moments of the outputs of the separation system, is defined to separate the nonlinear mixture. The minimization of the contrast function results in the independence of the outputs with desirable moments such that the original sources are separated properly. Two learning algorithms for the parametric RBF network are developed by using the stochastic gradient descent method and an unsupervised clustering method. By virtue of the RBF neural network, this proposed approach takes advantage of high learning convergence rate of weights in the hidden layer and output layer, natural unsupervised learning characteristics, modular structure, and universal approximation capability. Simulation results are presented to demonstrate the feasibility, robustness, and computability of the proposed method.

This paper presents an improved neural computation where scheme for kinematic control of redundant manipulators based on infinity-norm joint velocity minimization. Compared with a previous neural network approach to minimum infinity-non kinematic control, the present approach is less complex in terms of cost of architecture. The recurrent neural network explicitly minimizes the maximum component of the joint velocity vector while tracking a desired end-effector trajectory. The end-effector velocity vector for a given task is fed into the neural network from its input and the minimum infinity-norm joint velocity vector is generated at its output instantaneously. Analytical results are given to substantiate the asymptotic stability of the recurrent neural network. The simulation results of a four-degree-of-freedom planar robot arm and a seven-degree-of-freedom industrial robot are presented to show the proposed neural network can effectively compute the minimum infinity-norm solution to redundant manipulators.

The inverse kinematics problem in robotics can be formulated as a time-varying quadratic optimization problem. A new recurrent neural network, called the dual network, is presented in this paper. The proposed neural network is composed of a single layer of neurons, and the number of neurons is equal to the dimensionality of the workspace. The proposed dual network is proven to be globally exponentially stable. The proposed dual network is also shown to be capable of asymptotic tracking for the motion control of kinematically redundant manipulators.

We apply a recurrent neural network to support vector machine (SVM) training for pattern recognition. Specifically, a primal-dual neural network is exploited to solve the quadratic programming problem encountered in training SVMs. The properties of the network allow one to design SVMs without adjustable network parameters and give a better solution for ill-posed problems.

In this paper, we are concerned with the nonlinear projection equations of the following form P/sub /spl chi//(u-F(u))=u. Despite the particular structure of the feasible set /spl chi/, the problem is still a very general problem in mathematics programming. Moreover, there are a number of important applications which lead to this special class of variational inequalities such as equilibrium models arising in fields of economics and transportation science, etc. Various numerical solution procedures for the problem have been investigated over decades. Because of the nature of digital computers, conventional algorithms are time-consuming for large-scale optimization problems. It is well-known that one promising approach to optimization problems in real time is to employ artificial neural networks implemented in hardware. Recurrent neural networks for solving optimization problems are readily hardware-implementable. Thus, neural networks are a top choice of real-time solvers for optimization problems. Since the seminal work of Hopfield and Tank (1985), the neural network approach to optimization has been investigated and many neural networks for optimization problems have been proposed.

A novel neural network approach is developed for nonlinear blind separation using a radial b axis function (RBF) network and an information theoretic criterion. By utilizing the universal approximation ability and local response property of an RBF network the proposed separation method is characterized by fast convergence and strong demixing ability. After its learning process, the RBF network is able to separate independent signals effectively from their nonlinear mixtures by the nonlinear channel model without the prior knowledge of the source signals and mixing channels. Experimental results illustrate the validity and effectiveness of the proposed method.

A discrete-time recurrent neural network which is called the discrete-time Lagrangian network is proposed in this letter for solving convex quadratic programs. It is developed based on the classical Lagrange optimization method and solves quadratic programs without using any penalty parameter. The condition for the neural network to globally converge to the optimal solution of the quadratic program is given. Simulation results are presented to illustrate its performance.

This paper is concerned with the on-line learning of unknown dynamical systems using a recurrent neural network. The unknown dynamic systems to be learned are subject to disturbances and possibly unstable. The neural-network model used has a simple architecture with one layer of adaptive connection weights. Four learning rules are proposed for the cases where the system state is measurable in continuous or discrete time. Some of these learning rules extend the sigma-modification of the standard gradient learning rule. Convergence properties are given to show that the weight parameters of the recurrent neural network are bounded and the state estimation error converges exponentially to a bounded set, which depends on the modeling error and the disturbance bound. The effectiveness of the proposed learning rules for the recurrent neural network is demonstrated using an illustrative example of tracking a Brownian motion.

This paper introduces a novel technique for sequential blind extraction of singularly mixed sources. First, a neural-network model and an adaptive algorithm for single-source blind extraction are introduced. Next, extractability analysis is presented for singular mixing matrix, and two sets of necessary and sufficient extractability conditions are derived. The adaptive algorithm and neural-network model for sequential blind extraction are then presented. The stability of the algorithm is discussed. Simulation results are presented to illustrate the validity of the adaptive algorithm and the stability analysis. The proposed algorithm is suitable for the case of nonsingular mixing matrix as well as for singular mixing matrix.

This paper presents a continuous-time recurrent neural-network model for nonlinear optimization with any continuously differentiable objective function and bound constraints. Quadratic optimization with bound constraints is a special problem which can be solved by the recurrent neural network. The proposed recurrent neural network has the following characteristics. 1) It is regular in the sense that any optimum of the objective function with bound constraints is also an equilibrium point of the neural network. If the objective function to be minimized is convex, then the recurrent neural network is complete in the sense that the set of optima of the function with bound constraints coincides with the set of equilibria of the neural network. 2) The recurrent neural network is primal and quasiconvergent in the sense that its trajectory cannot escape from the feasible region and will converge to the set of equilibria of the neural network for any initial point in the feasible bound region. 3) The recurrent neural network has an attractivity property in the sense that its trajectory will eventually converge to the feasible region for any initial states even at outside of the bounded feasible region. 4) For minimizing any strictly convex quadratic objective function subject to bound constraints, the recurrent neural network is globally exponentially stable for almost any positive network parameters. Simulation results are given to demonstrate the convergence and performance of the proposed recurrent neural network for nonlinear optimization with bound constraints.

Global exponential stability is a desirable property for dynamic systems. This paper studies the global exponential stability of several existing recurrent neural networks for solving linear programming problems, convex programming problems with interval constraints, convex programming problems with nonlinear constraints, and monotone variational inequalities. In contrast to the existing results on global exponential stability, the present results do not require additional conditions on the weight matrices of recurrent neural networks and improve some existing conditions for global exponential stability. Therefore, the stability results in this paper further demonstrate the superior convergence properties of the existing neural networks for optimization.

Wavelet transforms and fuzzy techniques are used to monitor tool breakage and wear conditions in real time according to the measured spindle and feed motor currents, respectively. First, continuous and discrete wavelet transforms are used to decompose the spindle and feed ac servo motor current signals to extract signal features so as to detect the breakage of drills successfully. Next, the models of the relationships between the current signals and the cutting parameters are established under different tool wear states. Subsequently, fuzzy classification methods are used to detect tool wear states based on the above models. Finally, the two methods above are integrated to establish an intelligent tool condition monitoring system for drilling operations. The monitoring system can detect tool breakage and tool wear conditions using very simple current sensors. Experimental results show that the proposed system can reliably detect tool conditions in drilling operations in real time and is viable for industrial applications.

This paper presents two neural network approaches to real-time joint torque optimization for kinematically redundant manipulators. Two recurrent neural networks are proposed for determining the minimum driving joint torques of redundant manipulators for the eases without and with taking the joint torque limits into consideration, respectively. The first neural network is called the Lagrangian network and the second one is called the primal-dual network. In both neural-network-based computation schemes, while the desired accelerations of the end-effector for a specific task are given to the neural networks as their inputs, the signals of the minimum driving joint torques are generated as their outputs to drive the manipulator arm. Both proposed recurrent neural networks are shown to be capable of generating minimum stable driving joint torques. In addition, the driving joint torques computed by the primal-dual network are shown never exceeding the joint torque limits.

This paper presents an improved neural computation scheme for kinematic control of redundant manipulators based on infinity-norm joint velocity minimization. Compared with a preview neural network approach to minimum infinity-norm kinematic control, the presented approach has a less complex architecture. The recurrent neural network explicitly minimizes the maximum component of the joint velocity vector while tracking a desired end-effector trajectory. The end-effector velocity vector for a given task is fed into the neural network from its input and the minimum infinity-norm joint velocity vector is generated at its output instantaneously. Analytical results are given to substantiate the asymptotic stability of the recurrent neural network. The simulation results of a four degree-of-freedom planar robot arm are presented to show the proposed neural network can effectively compute the minimum infinity-norm solution to redundant manipulators in real-time.

We propose a dual neural network with globally exponential stability for solving quadratic programming problems with unique solutions. Compared with the Bouzerdoum-Pattison network (1993), there is no need for choosing the self-feedback or lateral connection matrices in the present network. Moreover, the size of the dual network is less than that of the original problem.

We propose two primal neural networks with globally exponential stability for solving quadratic programming problems. Both the self-feedback and lateral connection matrices in the present network are compared with the Bonzerdoum-Pattison network (1993). Moreover, the size of the proposed networks is same as that of the original problem, smaller than that of primal-dual networks.

Semidefinite programming problem is an important optimization problem that has been extensively investigated. A real-time solution method for solving such a problem, however, is still not yet available. This paper proposes a novel recurrent neural network for this purpose. First, an auxiliary cost function is introduced to minimize the duality gap between the admissible points of the primal problem and the corresponding dual problem. Then a dynamical system is constructed to drive the duality gap to zero exponentially along any trajectory by modifying the gradient of the auxiliary cost function. Furthermore, a subsystem is developed to circumvent in the computation of matrix inverse, so that the resulting overall dynamical system can be realized using a recurrent neural network. The architecture of the resulting neural network is discussed. The operating characteristics and performance of the proposed approach are demonstrated by means of simulation results.

A recurrent neural network, called the Lagrangian network, is presented for the kinematic control of redundant robot manipulators. The optimal redundancy resolution is determined by the Lagrangian network through real-time solution to the inverse kinematics problem formulated as a quadratic optimization problem. While the signal for a desired velocity of the end-effector is fed into the inputs of the Lagrangian network, it generates the joint velocity vector of the manipulator in its outputs along with the associated Lagrange multipliers. The proposed Lagrangian network is shown to be capable of asymptotic tracking for the motion control of kinematically redundant manipulators.

This paper presents two neural network approaches to minimum infinity-norm solution of the velocity inverse kinematics problem for redundant robots. Three recurrent neural networks are applied for determining a joint velocity vector with its maximum absolute value component being minimal among all possible joint velocity vectors corresponding to the desired end-effector velocity. In each proposed neural network approach, two cooperating recurrent neural networks are used. The first approach employs two Tank-Hopfield networks for linear programming. The second approach employs two two-layer recurrent neural networks for quadratic programming and linear programming, respectively. Both the minimal 2-norm and infinity-norm of joint velocity vector can be obtained from the output of the recurrent neural networks. Simulation results demonstrate that the proposed approaches are effective with the second approach being better in terms of accuracy and optimality.

This paper presents a linear assignment algorithm for solving the clustering problem. By using the most dissimilar data as cluster representatives, a linear assignment algorithm is developed based on the linear assignment model for clustering multivariate data. The computational results evaluated using multiple performance criteria show that the clustering algorithm is very effective and efficient, especially for clustering a large number of data with many attributes.

The assignment problem is a classical combinatorial optimization problem arising in numerous design and planning contexts. Solving an assignment problem of large scale is computationally intensive and time consuming. The paper discusses the development of an FPGA based custom computing system that can accelerate the computation by exploiting the intrinsic parallelism of a recently proposed recurrent neural network for solving the assignment problem. The theoretical background of this work has been discussed in other papers. The digital realization of the system, including architecture, design, FPGA implementation and verification are discussed.

A recurrent neural network for kinematic control of redundant robot manipulators with torque minimization is presented. The proposed recurrent neural network is composed of two bidirectionally connected layers of neuron arrays. While the command signals of desired acceleration of the end-effector are fed into the input layer, the output layer generates the joint acceleration vector of the manipulator with joint torques being minimized. The proposed recurrent neural network is shown to be capable of asymptotic tracking of trajectory for the redundant manipulators with minimized joint torques.

The assignment problem is an archetypical combinatorial optimization problem having widespread applications. This paper presents two recurrent neural networks, a continuous-time one and a discrete-time one, for solving the assignment problem. Because the proposed recurrent neural networks solve the primal and dual assignment problems simultaneously, they are called primal-dual assignment networks. The primal-dual assignment networks are guaranteed to make optimal assignment regardless of initial conditions. Unlike the primal or dual assignment network, there is no time-varying design parameter in the primal-dual assignment networks. Therefore, they are more suitable for hardware implementation. The performance and operating characteristics of the primal-dual assignment networks are demonstrated by means of illustrative examples.

In this paper, we present a general methodology for designing optimization neural networks. We prove that the neural networks constructed by using the proposed method are guaranteed to be globally convergent to solutions of problems with bounded or unbounded solution sets, in contrast with the gradient methods whose convergence is not guaranteed. We show that the proposed method contains both the gradient methods and nongradient methods employed in existing optimization neural networks as special cases. Based on the theoretical results of the proposed method, we study the convergence and stability of general gradient models in case of unisolated solutions. Using the proposed method, we derive some new neural network models for a very large class of optimization problems, in which the equilibrium points correspond to exact solutions and there is no variable parameter. Finally, some numerical examples show the effectiveness of the method.

Presents two recurrent neural networks for solving the shortest path problem. Simplifying the architecture of a recurrent neural network based on the primal problem formulation, the first recurrent neural network called the primal routing network has less complex connectivity than its predecessor. Based on the dual problem formulation, the second recurrent neural network called the dual routing network has even simpler architecture. While being simple in architecture, the primal and dual routing networks are capable of shortest-path routing like their predecessor.

This paper presents two recurrent neural networks for solving the assignment problem. Simplifying the architecture of a recurrent neural network based on the primal assignment problem, the first recurrent neural network, called the primal assignment network, has less complex connectivity than its predecessor. The second recurrent neural network, called the dual assignment network, based on the dual assignment problem, is even simpler in architecture than the primal assignment network. The primal and dual assignment networks are guaranteed to make optimal assignment. The applications of the primal and dual assignment networks for sorting and shortest-path routing are discussed. The performance and operating characteristics of the dual assignment network are demonstrated by means of illustrative examples.

Although the family of LVQ algorithms have been widely used for pattern classification and have achieved a great success, the rigorous theoretical studies on the classification performance of LVQ algorithms have seldom been made. In this paper, the asymptotical performance of LVQ1, LVQ2 and LVQ2.1 algorithms have been studied thoroughly, and three significant conclusions have been achieved respectively. Furthermore, a simple modification scheme to LVQ2 algorithm has been developed and analyzed on the asymptotical performance, which can produce the optimal or nearly-optimal classifier in the stable equilibrium state for the classification problems with classes overlapping.

In this paper, two codebook search methods and a coding scheme are proposed for fast and low rate vector quantization using the self-organizing feature maps (SOFM). Based on the topology preservation property of the SOFM, the search methods use the distance between adjacent input vectors to guide the codebook search process and to determine searching sequence of codevectors. The novel coding scheme, which can be considered as a vector version of delta modulation, eliminates the correlation buried in the source sequence and hence reduces the rate. Simulation results demonstrate the effectiveness of proposed methods and better performances than those obtained previously.

An analog sorting neural network is presented. First, existing order representations are discussed and a generalized order representation is introduced. The sorting problem is then formulated as the assignment problem. Based on the assignment problem formulation, the neural network architecture is described. Design principles and an op-amp based circuit realization of the analog neural network are delineated. Three illustrative examples are also discussed to demonstrate the capability and performance of the analog neural network. The proposed analog neural network is shown to be capable of monotonic and bitonic sorting and suitable for hardware implementation.

In this paper, a new competitive learning algorithm based on the partial distortion theorem is proposed for the on-line vector quantizer design. The novel algorithm is called partial-distortion-equivalent competitive learning (PDECL) algorithm, which aims at making the partial distortions for each neuron (code-vector) be uniform to overcome the neuron underuse problem as well as to minimize the average distortion for the designed vector quantizer. Compared with the Kohonen learning algorithm (KLA), the frequency-sensitive competitive learning (FSCL) algorithm and the soft competition scheme (SCS) algorithm, the PDECL consistently shows the better performance than all of them and the LBG algorithm for the design of vector quantizers with different codebook sizes especially when the codebook size is large enough.>

Recurrent neural networks are proposed for synthesizing linear control systems through pole placement. The proposed neural networks approach uses two coupled recurrent neural networks for computing feedback gain matrix. Each neural network consists of two bidirectionally connected layers and each layer consists of an array of neurons. The proposed recurrent neural networks are shown to be capable of synthesizing linear control systems in real time. The operating characteristics of the recurrent neural networks and closed-loop systems are demonstrated by use of two illustrative examples.>

The shortest path problem is the classical combinatorial optimization problem arising in numerous planning and designing contexts. In this paper, a recurrent neural network for solving the shortest path problem is presented. The proposed recurrent neural network is able to generate optimal solutions to the shortest path problem. The performance and operating characteristics of the recurrent neural network are demonstrated by use of illustrative examples.>

Optical flow computation in dynamic image processing can be formulated as a minimization problem by a variational approach. Because solving the problem is computationally intensive, we reformulate the problem suitable for neural computing. In this paper, we propose a recurrent neural network model which may be implemented in hardware with many processing elements (neurons) operating asynchronously in parallel to achieve a possible real-time solution. We derive and prove the properties of the reformulation, as well as analyze the asymptotic stability and convergence rate of the proposed neural network. Experiments using both the test patterns and the real laboratory images are conducted.

Asymptotic properties of recurrent neural networks for optimization are analyzed. Specifically, asymptotic stability of recurrent neural networks with monotonically time-varying penalty parameters for optimization is proven; sufficient conditions of feasibility and optimality of solutions generated by the recurrent neural networks are characterized. Design methodology of the recurrent neural networks for solving optimization problems is discussed. Operating characteristics of the recurrent neural networks are also presented using illustrative examples.