Multivariate time series prediction has aroused widely research interests during decades. However, the spatial heterogeneity and temporal evolution characteristics bring much challenges for high-dimensional time series prediction. In this paper, a novel adaptive graph convolution module is introduced to automatically learn the spatial correlation of multivariate time series and a Koopman-based neural differential equation is proposed to simulate the nonlinear system state evolution. In detail, the correlation between multivariate time series is revealed by the consine similarity of node embedding to infer the potential relationship between nodes and the spatio-temporal feature fusion module is utilized. The LSTM-based network is adopted as Koopman operator to reveal the latent states of spatio-temporal time series and the reversible assumption is imposed on the Koopman operator. Furthermore, the Euler-trapezoidal integration are utilized to simulate the temporal dynamics and multiple-step prediction is carried out in the latent space from the perspective of dynamical differential equation. The proposed model could explicitly discover the spatial correlation by adaptive graph convolution and reveal the temporal dynamics by neural differential equation, which make the modeling more interpretable. Simulation results show the effectiveness on spatio-temporal dynamic discovery and prediction performance.

Evolving fuzzy system (EFS), a special adaptive model with Takagi-Sugeno (TS) fuzzy rules that can adaptively update internal parameters based on data streams, has been widely used in online learning scenarios. However, current EFSs are mainly single-layer models, which cannot adequately capture hidden information in multidimensional chaotic time series. To perform online prediction of multidimensional chaotic time series, a novel evolving fuzzy system, called hierarchical evolving fuzzy system with kernel conjugate gradient (HEFS-KCG), is proposed in this paper. HEFS-KCG excavates and captures latent evolutionary patterns concealed within dynamic systems through a layer-by-layer processing of multidimensional information. HEFS-KCG performs structural evolution based on data distribution in the antecedent part, and combines the sparse learning strategy and kernel conjugate gradient (KCG) to update the consequent parameters. Subsequently, we provide a theoretical analysis of HEFS-KCG, ensuring its convergence when applied to online prediction. The simulation results demonstrate that HEFS-KCG outperforms existing EFSs and other models for multidimensional chaotic time series online prediction.

In this article, a challenging and significant intelligent fault diagnosis task is investigated, in which multisource sensor signals with intense noise and outlier disturbances are jointly analyzed. Such a scenario has hardly been considered in industrial research. To this end, we develop a novel tensor-based nonlinear classifier called a unified pinball loss intuitionistic fuzzy support tensor machine, which can successfully solve the above tasks and improve the performance of fault diagnosis in practical applications. First, the noisy multisource signals are converted into time–frequency images and reconstructed into tensor samples to mine the time-domain, frequency-domain features, and coupled structure information in the spatial domain. Next, we design two nonlinear forms of nonmembership functions in the tensor space and obtain an intuitionistic fuzzy score for each training sample to enhance the robustness of the model. Subsequently, a pinball loss function is introduced to better handle noise sensitivity and resampling instability problems. Note that, we employ the tensor robust principal component analysis method to accurately recover the low-rank tensors corrupted by sparse noise from the original tensors. Finally, two numerical examples are presented to verify the feasibility and validity of the proposed method.

Domain adaptation indeed promotes the progress of intelligent fault diagnosis in industrial scenarios. The abundant labeled samples are not necessary. The identical distribution between the training and testing datasets is not any more the prerequisite for intelligent fault diagnosis working. However, two issues arise subsequently: Feature learning in domain adaptation framework tends to be biased to the source domain, and unreliable pseudolabeling seriously impacts on the conditional domain adaptation. In this article, a new domain adaptation approach with self-supervised learning and feature clustering (DASSL-FC) is proposed, trying to alleviate the issues by unbiased feature learning and pseudolabels updating strategy. Taking different transformation methods as pretext, the transformed data and its pretext train a neural network in an SSL way. As to pseudolabeling, clusters are taken as the auxiliary information to correct the network predicted labels in terms of the "strong cluster" rule. Then, the updated pseudolabels and their confidence are enforced to further estimate the conditional distribution discrepancy and its confidence weight. To verify the effectiveness of the proposed method, the experiments are implemented on intraplatform and interplatforms for simulating the practical scenarios.

More and more time series data appear in various fields, and the prediction of multivariate time series has been the key to solve many industrial problems. Therefore, it is necessary to establish an accurate prediction model. As an efficient recursive neural network, an echo state network (ESN) model has been widely used in time series prediction. However, it usually faces the problem of how to choose suitable reservoir parameters for different applications. In addition, selecting the input feature set is also an important issue, which will affect the accuracy and computational efficiency of the prediction model. To solve these problems, the modified binary salp swarm algorithm-based optimization ESN (MBSSA-ESN) is proposed for multivariate time series prediction, which can simultaneously realize feature subset selection and parameter optimization. In order to verify the effectiveness of the proposed model, Beijing air quality index data are used for simulation and the key index PM2.5 is used as the target variable for experiment. Compared with several related methods, the proposed model achieves the best results in all evaluation indicators, indicating that the MBSSA-ESN model is competitive in the task of multivariate time series prediction.

Recently, multivariate time series (MTS) clustering has gained lots of attention. However, state-of-the-art algorithms suffer from two major issues. First, few existing studies consider correlations and redundancies between variables of MTS data. Second, since different clusters usually exist in different intrinsic variables, how to efficiently enhance the performance by mining the intrinsic variables of a cluster is challenging work. To deal with these issues, we first propose a variable-weighted K-medoids clustering algorithm (VWKM) based on the importance of a variable for a cluster. In VWKM, the proposed variable weighting scheme could identify the important variables for a cluster, which can also provide knowledge and experience to related experts. Then, a Reverse nearest neighborhood-based density Peaks approach (RP) is proposed to handle the problem of initialization sensitivity of VWKM. Next, based on VWKM and the density peaks approach, an ensemble Clustering framework (SSEC) is advanced to further enhance the clustering performance. Experimental results on ten MTS datasets show that our method works well on MTS datasets and outperforms the state-of-the-art clustering ensemble approaches.

Echo state network (ESN), a type of special recurrent neural network with a large-scale randomly fixed hidden layer (called a reservoir) and an adaptable linear output layer, has been widely employed in the field of time series analysis and modeling. However, when tackling the problem of multidimensional chaotic time series prediction, due to the randomly generated rules for input and reservoir weights, not only the representation of valuable variables is enriched but also redundant and irrelevant information is accumulated inevitably. To remove the redundant components, reduce the approximate collinearity among echo-state information, and improve the generalization and stability, a new method called hierarchical ESN with sparse learning (HESN-SL) is proposed. The HESN-SL mines and captures the latent evolution patterns hidden from the dynamic system by means of layer-by-layer processing in stacked reservoirs, and leverage monotone accelerated proximal gradient algorithm to train a sparse output layer with variable selection capability. Meanwhile, we further prove that the HESN-SL satisfies the echo state property, which guarantees the stability and convergence of the proposed model when applied to time series prediction. Experimental results on two synthetic chaotic systems and a real-world meteorological dataset illustrate the proposed HESN-SL outperforms both original ESN and existing hierarchical ESN-based models for multidimensional chaotic time series prediction.

The application of deep learning in time-series prediction has developed gradually. In this paper, we propose a deep generative network model for feature extraction of multivariate time series, namely, mutual information variational autoencoders (MI-VAE). In the architecture of the proposed model, we use the latent space of VAE for feature learning, which can extract the essential features of multivariate time-series data effectively. The latent space employed directly as a feature extractor can avoid poor interpretability of model. In addition, we introduce a mutual information term into the loss function, which improves the expression capability and accuracy of model. The proposed model, combining the merits of VAE and mutual information, extracts features for multivariate time-series data from a new perspective. The Lorenz system and Beijing air quality time series are used to test performance of the proposed model and comparative models. Results show that the proposed model is superior to other similar models in terms of accuracy and expression capability of latent space.

Change detection (CD) is an important task in remote sensing image processing. The main research goal is to identify whether the target area has changed. Recently, the rise of deep learning has provided many novel methods for change detection, and some excellent models have been proposed. However, most of the available methods have a large number of parameters, and the traditional loss function does not perform well when tackling the problem of unbalanced sample number. In order to solve the above problems, we propose a lightweight fully convolution network for change detection, namely LWCDNet. Specifically, LWCDNet is a typical encoder-decoder structure, and it realizes more detailed information transmission and feature extraction by using the artificial padding convolution (APC) module as the convolution unit of the encoder. In addition, a convolutional block attention module (CBAM) is added between encoder and decoder to boost the model’s performance even more by emphasizing critical information. To deal with the sample imbalance in the change detection task, we propose Lov-wce loss. The experimental results on two actual remote sensing datasets show the effectiveness of LWCDNet.

In the real world, multivariate time series from the dynamical system are correlated with deterministic relationships. Analyzing them dividedly instead of utilizing the shared-pattern of the dynamical system is time consuming and cumbersome. Multitask learning (MTL) is an effective inductive bias method to utilize latent shared features and discover the structural relationships from related tasks. Base on this concept, we propose a novel MTL model for multivariate chaotic time-series prediction, which could learn both dynamic-shared and dynamic-specific patterns. We implement the dynamic analysis of multiple time series through a special network structure design. The model could disentangle the complex relationships among multivariate chaotic time series and derive the common evolutionary trend of the multivariate chaotic dynamical system by inductive bias. We also develop an efficient Crank-Nicolson-like curvilinear update algorithm based on the alternating direction method of multipliers (ADMM) for the nonconvex nonsmooth Stiefel optimization problem. Simulation results and analysis demonstrate the effectiveness on dynamic-shared pattern discovery and prediction performance.

Multivariate time-series prediction is a challenging research topic in the field of time-series analysis and modeling, and is continually under research. The echo state network (ESN), a type of efficient recurrent neural network, has been widely used in time-series prediction, but when using ESN, two crucial problems have to be confronted: 1) how to select the optimal subset of input features and 2) how to set the suitable parameters of the model. To solve this problem, the modified biogeography-based optimization ESN (MBBO-ESN) system is proposed for system modeling and multivariate time-series prediction, which can simultaneously achieve feature subset selection and model parameter optimization. The proposed MBBO algorithm is an improved evolutionary algorithm based on biogeography-based optimization (BBO), which utilizes an S -type population migration rate model, a covariance matrix migration strategy, and a Lévy distribution mutation strategy to enhance the rotation invariance and exploration ability. Furthermore, the MBBO algorithm cannot only optimize the key parameters of the ESN model but also uses a hybrid-metric feature selection method to remove the redundancies and distinguish the importance of the input features. Compared with the traditional methods, the proposed MBBO-ESN system can discover the relationship between the input features and the model parameters automatically and make the prediction more accurate. The experimental results on the benchmark and real-world datasets demonstrate that MBBO outperforms the other traditional evolutionary algorithms, and the MBBO-ESN system is more competitive in multivariate time-series prediction than other classic machine-learning models.

Traditional classification learning algorithms have several limitations: 1) they are time consuming for the large-scale training multivariate time-series (MTS) data, and unsuitable for the dynamically added training data; 2) as the number of the training MTS data becomes larger, they could not achieve the desired classification accuracy; 3) most of them do not consider how to make use of the unlabeled samples to enhance the classifier performance; and 4) due to the high dimension of MTS and complex relationship among variables, existing online learning algorithms are not effective to update shapelet-based association rules. Up to now, few work touched online classification learning for dynamically added unlabeled examples. To efficiently address these issues, we propose an online rule-based classifier learning framework on dynamically added unlabeled MTS data (ORCL-U). This framework integrates a confidence-based labeling strategy (CLS) and an online rule-based classifier learning approach (ORBCL). Extensive experiments on ten datasets show the effectiveness and efficiency of our proposed approach.

This paper aims at the monitoring of birds and their ecological environment in the island and coastal wetland ecosystems. A new approach of island bird detection is proposed based on the Faster R-CNN (Regions with Convolutional Neural Networks) model under multiple backgrounds. It includes feature extraction, region proposal, bounding box regression, classification into the whole neural network structure. This key technology can automatically achieve automatic bird species identification and quantitative statistics in the faster computation speed. The details of constructing Faster R-CNN are described. In the end, many actual images are utilized to demonstrate the effectiveness of the proposed models.

We study the exponential stability of discrete-time neural networks (NNs) with a time-varying delay which contains a few intermittent large delays (LDs). By modeling the considered discrete-time NN as a discrete-time switched NN which contains two subsystems and one of them may be unstable over the LD periods (LDPs), switching techniques are employed to analyze the problem. Delay-dependent exponential stability conditions to check the frequency and the length of the LDs allowed for guaranteeing the exponential stability are proposed by applying a novel Lyapunov-Krasovskii functional (LKF) with LDP-based terms, Wirtinger-based summation inequality, and reciprocally convex combination technique. Based on these conditions, associated evaluation algorithms are developed. Finally, two numerical examples are provided to demonstrate the effectiveness of the proposed method.

To improve the retrieval result obtained from a pairwise dissimilarity, many variants of diffusion process have been applied in visual re-ranking. In the framework of diffusion process, various contextual similarities can be obtained by solving an optimization problem, and the objective function consists of a smoothness constraint and a fitting constraint. And many improvements on the smoothness constraint have been made to reveal the underlying manifold structure. However, little attention has been paid to the fitting constraint, and how to build an effective fitting constraint still remains unclear. In this article, by deeply analyzing the role of fitting constraint, we firstly propose a novel variant of diffusion process named Hybrid Regularization of Diffusion Process (HyRDP). In HyRDP, we introduce a hybrid regularization framework containing a two-part fitting constraint, and the contextual dissimilarities can be learned from either a closed-form solution or an iterative solution. Furthermore, this article indicates that the basic idea of HyRDP is closely related to the mechanism behind Generalized Mean First-passage Time (GMFPT). GMFPT denotes the mean time-steps for the state transition from one state to any one in the given state set, and is firstly introduced as the contextual dissimilarity in this article. Finally, based on the semi-supervised learning framework, an iterative re-ranking process is developed. With this approach, the relevant objects on the manifold can be iteratively retrieved and labeled within finite iterations. The proposed algorithms are validated on various challenging databases, and the experimental performances demonstrate that retrieval results obtained from different types of measures can be effectively improved by using our methods.

How to make full use of the evolution information of chaotic systems for time-series prediction is a difficult issue in dynamical system modeling. In this article, we propose a maximum information exploitation broad learning system (MIE-BLS) for extreme information utilization of large-scale chaotic time-series modeling. An improved leaky integrator dynamical reservoir is introduced in order to capture the linear information of chaotic systems effectively. It can not only capture the information of the current state but also achieve the compromise with historical states in the dynamical system. Furthermore, the feature is mapped to the enhancement layer by nonlinear random mapping to exploit nonlinear information. The cascading mechanism promotes the information propagation and achieves feature reactivation in dynamical modeling. Discussions about maximum information exploration and the comparisons with ResNet, DenseNet, and HighwayNet are presented in this article. Simulation results on four large-scale data sets illustrate that MIE-BLS could achieve better performance of information exploration in large-scale dynamical system modeling.

The extensive applications in the Internet of Things (IoT) have inspired a growing network scale. However, due to the resource-limited IoT devices and the numerous cyber attacks against applications, maintaining the robustness and communication capabilities for the applications is increasingly challenging. In this article, we consider IoT network topologies that provide robust communication for heterogeneous networks and study the networking stability of IoT devices and the intelligent evolution computing in network architectures. We explicate the network robustness problem both for the network architecture and the resistance to cyber attacks. For the network architecture, we optimize the robustness of IoT network topology with a scale-free network model which has good performance in random attacks. In the case with the resistance to cyber attacks, a deep deterministic learning policy (DDLP) algorithm is proposed to improve the stability for large-scale IoT applications. Simulations show that the proposed algorithms greatly advance the robustness of IoT network topology compared to other algorithms, with a less computational cost.

In this paper, a cooperative time-varying formation maneuvering problem with connectivity preservation and collision avoidance is investigated for a fleet of autonomous surface vehicles (ASVs) with position-heading measurements. Each vehicle is subject to unknown kinetics induced by internal model uncertainty and external disturbances. At first, a nonlinear state observer is used to recover the unmeasured linear velocity and yaw rate as well as unknown uncertainty and disturbances. Then, observer-based cooperative time-varying formation maneuvering control laws are designed based on artificial potential functions, nonlinear tracking differentiators, and a backstepping technique. The stability of closed-loop distributed formation control system is analyzed based on input-to-state stability and cascade stability. The salient features of the proposed method are as follows. First, cooperative time-varying formation maneuvering with the capability of connectivity preservation and collision avoidance can be achieved in the absence of velocity measurements. Second, the complexity of the cooperative time-varying formation maneuvering control laws is reduced without resorting to dynamic surface control. Third, the uncertainty and disturbance are actively rejected in the presence of position-heading measurements. Simulation results are given to substantiate the proposed output feedback control method for cooperative time-varying formation maneuvering of ASVs with connectivity preservation and collision avoidance.

The broad learning system (BLS) is an emerging approach for effective and efficient modeling of complex systems. The inputs are transferred and placed in the feature nodes, and then sent into the enhancement nodes for nonlinear transformation. The structure of a BLS can be extended in a wide sense. Incremental learning algorithms are designed for fast learning in broad expansion. Based on the typical BLSs, a novel recurrent BLS (RBLS) is proposed in this paper. The nodes in the enhancement units of the BLS are recurrently connected, for the purpose of capturing the dynamic characteristics of a time series. A sparse autoencoder is used to extract the features from the input instead of the randomly initialized weights. In this way, the RBLS retains the merit of fast computing and fits for processing sequential data. Motivated by the idea of "fine-tuning" in deep learning, the weights in the RBLS can be updated by conjugate gradient methods if the prediction errors are large. We exhibit the merits of our proposed model on several chaotic time series. Experimental results substantiate the effectiveness of the RBLS. For chaotic benchmark datasets, the RBLS achieves very small errors, and for the real-world dataset, the performance is satisfactory.

As an outstanding discriminant analysis technique, Fisher discriminant analysis (FDA) gained extensive attention in supervised dimensionality reduction and fault diagnosis fields. However, it typically ignores the multimodality within the measured data, which may cause infeasibility in practice. In addition, it generally incorporates all process variables without emphasizing the key faulty ones when modeling the complex process, thus leading to degraded fault classification capability and poor model interpretability. To ease the above two drawbacks of conventional FDA, this brief presents an advantageously sparse local FDA (SLFDA) model, it first preserves the within-class multimodality by introducing local weighting factors into scatter matrix. Then, the responsible faulty variables are identified automatically through the elastic net algorithm, and the current optimization problem is subsequently settled through the feasible gradient direction method. Since then, the local data structure characteristics are exploited from both the sample dimension and variable dimension so that the fault diagnosis performance and model interpretability are significantly enhanced. In addition, we naturally extend SLFDA model to nonlinear variant (i.e., sparse kernel local FDA) by the kernel trick, which is substantially more resistant to strong nonlinearity. The simulation studies on Tennessee Eastman (TE) benchmark process and real-world diesel engine working process both validate that the novel diagnosis strategy is more accurate and reliable than the existing state-of-the-art methods.

Underwater image processing is an intelligence research field that has great potential to help developers better explore the underwater environment. Underwater image processing has been used in a wide variety of fields, such as underwater microscopic detection, terrain scanning, mine detection, telecommunication cables, and autonomous underwater vehicles. However, underwater imagery suffers from strong absorption, scattering, color distortion, and noise from the artificial light sources, causing image blur, haziness, and a bluish or greenish tone. Therefore, the enhancement of underwater imagery can be divided into two methods: 1) underwater image dehazing and 2) underwater image color restoration. This paper presents the reason for underwater image degradation, surveys the state-of-the-art intelligence algorithms like deep learning methods in underwater image dehazing and restoration, demonstrates the performance of underwater image dehazing and color restoration with different methods, introduces an underwater image color evaluation metric, and provides an overview of the major underwater image applications. Finally, we summarize the application of underwater image processing.

3D shape feature learning plays a pivotal role in both industry and academia. PointCNN is one of excellent neural networks for 3D object databases classification. Instead of selecting representative points arbitrarily in PointCNN, clustering-enhanced PointCNN proposed in this paper can make representative points more logical and efficient for point cloud classification learning. The proposed clustering-based selection approach is able to distinguish more features and catch more details from 3D shapes. Both K-Means and Gaussian-Mixture-Model (GMM) clustering methods are applied during the point selection period. Both methods have been tested on several public data sets, which substantiates the superior classification accuracy with comparable training time.

The research on three-dimensional topology is important for Internet of Thing. Small-world with shorter average path lengths has proven to be an effective model for building evolutional network topologies. In order to build three-dimensional topology in IoT, the ant colony algorithm is used to plan shortcuts in this paper. First, Gaussian integration is used to simulate the ups and downs of terrain in three-dimensional space. Second, a significant number of nodes are randomly deployed on the modeled terrain. Taking into account the information about slope and aspect around the node, the actual sensing range of the node is calculated. Third, a certain percentage of nodes are selected as super sensor nodes. Finally, the ant colony algorithm is used to add shortcuts between super sensor nodes. Extensive experimental results show that an energy-efficient three-dimensional network topology in IoT can be built by the algorithm.

State space reconstruction is the foundation of chaotic system modeling. Selection of reconstructed variables is essential to the analysis and prediction of multivariate chaotic time series. As most existing state space reconstruction theorems deal with univariate time series, we have presented a novel nonuniform state space reconstruction method using information criterion for multivariate chaotic time series. We derived a new criterion based on low dimensional approximation of joint mutual information for time delay selection, which can be solved efficiently through the use of an intelligent optimization algorithm with low computation complexity. The embedding dimension is determined by conditional entropy, after which the reconstructed variables have relatively strong independence and low redundancy. The scheme, which integrates nonuniform embedding and feature selection, results in better reconstructions for multivariate chaotic systems. Moreover, the proposed nonuniform state space reconstruction method shows good performance in forecasting benchmark and actual multivariate chaotic time series.

Chaotic time series widely exists in nature and society (e.g., meteorology, physics, economics, etc.), which usually exhibits seemingly unpredictable features due to its inherent nonstationary and high complexity. Thankfully, multifarious advanced approaches have been developed to tackle the prediction issues, such as statistical methods, artificial neural networks (ANNs), and support vector machines. Among them, the interval type-2 fuzzy neural network (IT2FNN), which is a synergistic integration of fuzzy logic systems and ANNs, has received wide attention in the field of chaotic time series prediction. This paper begins with the structural features and superiorities of IT2FNN. Moreover, chaotic characters identification and phase-space reconstruction matters for prediction are presented. In addition, we also offer a comprehensive review of state-of-the-art applications of IT2FNN, with an emphasis on chaotic time series prediction and summarize their main contributions as well as some hardware implementations for computation speedup. Finally, this paper trends and extensions of this field, along with an outlook of future challenges are revealed. The primary objective of this paper is to serve as a tutorial or referee for interested researchers to have an overall picture on the current developments and identify their potential research direction to further investigation.

Kernel recursive least squares (KRLS) is a kind of kernel methods, which has attracted wide attention in the research of time series online prediction. It has low computational complexity and updates in a recursive form. However, as data size increases, computational complexity of calculating kernel inverse matrix will raise. And it has some difficulties in accommodating time-varying environments. Therefore, we have presented an improved KRLS algorithm for multivariate chaotic time series online prediction. Approximate linear dependency, dynamic adjustment, and coherence criterion are combined with quantization to form our improved KRLS algorithm. In the process of online prediction, it can bring computational efficiency up and adjust weights adaptively in time-varying environments. Moreover, Lorenz chaotic time series, El Nino-Southern Oscillation indexes chaotic time series, yearly sunspots and runoff of the Yellow River chaotic time series online prediction are presented to prove the effectiveness of our proposed algorithm.

Multivariate chaotic time series prediction is a hot research topic, the goal of which is to predict the future of the time series based on past observations. Echo state networks (ESNs) have recently been widely used in time series prediction, but there may be an ill-posed problem for a large number of unknown output weights. To solve this problem, we propose a hybrid regularized ESN, which employs a sparse regression with the L1/2regularization and the L2regularization to compute the output weights. The L1/2penalty shows many attractive properties, such as unbiasedness and sparsity. The L2penalty presents appealing ability on shrinking the amplitude of the output weights. After the output weights are calculated, the input weights, internal weights, and output weights are fine-tuning by a Hessian-free optimization method-conjugate gradient backpropagation algorithm. The fine-tuning helps to bubble up the input information toward the output layer. Besides, the largest Lyapunov exponent is used to calculate the predictable horizon of a chaotic time series. Experimental results on benchmark and real-world datasets show that our proposed method is superior to other ESN-based models, as sparser, smaller-absolute-value, and more informative output weights are obtained. All of the predictions within the predictable horizon of the proposed model are accurate.

High-dimensional and large-scale time series processing has aroused considerable research interests during decades. It is difficult for traditional methods to reveal the evolution state in dynamical systems and discover the relationship among variables automatically. In this paper, we propose a unified framework for nonuniform embedding, dynamical system revealing, and time series prediction, termed as Structured Manifold Broad Learning System (SM-BLS). The structured manifold learning is introduced for nonuniform embedding and unsupervised manifold learning simultaneously. Graph embedding and feature selection are both considered to depict the intrinsic structure connections between chaotic time series and its low-dimensional manifold. Compared with traditional methods, the proposed framework could discover potential deterministic evolution information of dynamical systems and make the modeling more interpretable. It provides us a homogeneous way to recover the chaotic attractor from multivariate and heterogeneous time series. Simulation analysis and results show that SM-BLS has advantages in dynamic discovery and feature extraction of large-scale chaotic time series prediction.

Improving universality and robustness of the control method is one of the most challenging problems in the field of complex networks (CNs) synchronization. In this paper, a special unilateral coupling finite-time synchronization (UCFTS) method for uncertain CNs is proposed for this challenging problem. Multiple influencing factors are considered, so that the proposed method can be applied to a variety of situations. First, two kinds of drive-response CNs with different sizes are introduced, each of which contains two types of nonidentical nodes and time-varying coupling delay. In addition, the node parameters and topological structure are unknown in drive network. Then, an effective UCFTS control technique is proposed to realize the synchronization of drive-response CNs and identify the unknown parameters and topological structure. Second, the UCFTS of uncertain CNs with four types of nonidentical nodes is further studied. Moreover, both the networks are of unknown parameters, time-varying coupling delay and uncertain topological structure. Through designing corresponding adaptive updating laws, the unknown parameters are estimated successfully and the weight of uncertain topology can be automatically adapted to the appropriate value with the proposed UCFTS. Finally, two experimental examples show the correctness of the proposed scheme. Furthermore, the method is compared with the other three synchronization methods, which shows that our method has a better control performance.

Spatio-temporal series prediction has attracted increasing attention in the field of meteorology in recent years. The spatial and temporal joint effect makes predictions challenging. Most of the existing spatio-temporal prediction models are computationally complicated. To develop an accurate but easy-to-implement spatio-temporal prediction model, this paper designs a novel spatio-temporal prediction model based on echo state networks. For real-world observed meteorological data with randomness and large changes, we use a cubic spline method to bridge the gaps between the neighboring points, which results in a pleasingly smooth series. The interpolated series is later input into the spatio-temporal echo state networks, in which the spatial coefficients are computed by the elastic-net algorithm. This approach offers automatic selection and continuous shrinkage of the spatial variables. The proposed model provides an intuitive but effective approach to address the interaction of spatial and temporal effects. To demonstrate the practicality of the proposed model, we apply it to predict two real-world datasets: monthly precipitation series and daily air quality index series. Experimental results demonstrate that the proposed model achieves a normalized root-mean-square error of approximately 0.250 on both datasets. Similar results are achieved on the long short-term memory model, but the computation time of our proposed model is considerably shorter. It can be inferred that our proposed neural network model has advantages on predicting meteorological series over other models.

In multivariate chaotic time series prediction, correlation analysis is important for reducing input dimensions and improving prediction performance. Grey relational analysis (GRA) has proved to be an effective method for data correlation analysis, especially for inexact data and incomplete data. In GRA, points are usually regarded as objects, and the distance between points or the concave and convex degree are mostly used to measure the correlations. However, with discrete variables, correlation analysis results always tend to have some deviations when using prior GRA methods. Furthermore, GRA methods cannot directly use vector datasets. Therefore, in this paper, an improved GRA method is proposed based on vector projections. The input and output variables are expressed as vectors by linking two adjacent points. The vectors, instants of the points, are regarded as the objects, and the projection length of input variables to output variables is used to measure the correlations. The smaller the difference between the projection length and the input variables, the higher the correlation. Then, a hybrid variable selection and prediction model is proposed based on the improved GRA method for multivariate chaotic time series predictions, in order to overcome the negative effects of irrelevant and redundant variables caused by phase-space reconstruction. The experimental results based on the gas furnace dataset and San Francisco river runoff dataset demonstrate that the improved GRA method is effective for data correlation analysis, and the prediction accuracy is better than prior GRA-based methods.

In this paper, a backpropagating constraints-based trajectory tracking control (BCTTC) scheme is addressed for trajectory tracking of a quadrotor with complex unknowns and cascade constraints arising from constrained actuator dynamics, including saturations and dead zones. The entire quadrotor system including actuator dynamics is decomposed into five cascade subsystems connected by intermediate saturated nonlinearities. By virtue of the cascade structure, backpropagating constraints (BCs) on intermediate signals are derived from constrained actuator dynamics suffering from nonreversible rotations and nonnegative squares of rotors, and decouple subsystems with saturated connections. Combining with sliding-mode errors, BC-based virtual controls are individually designed by addressing underactuation and cascade constraints. In order to remove smoothness requirements on intermediate controls, first-order filters are employed, and thereby contributing to backsteppinglike subcontrollers synthesizing in a recursive manner. Moreover, universal adaptive compensators are exclusively devised to dominate intermediate tracking residuals and complex unknowns. Eventually, the closed-loop BCTTC system stability can be ensured by the Lyapunov synthesis, and trajectory tracking errors can be made arbitrarily small. Simulation studies demonstrate the effectiveness and superiority of the proposed BCTTC scheme for a quadrotor with complex constrains and unknowns.

In this paper, aiming at the cumbersome solution of control law in neural network predictive control algorithm, a quasi-linear neural network identification and predictive control algorithm is proposed. The recurrent neural network is embedded into the quasi-linear model, which can be viewed as a quasi-ARX model macroscopically. In the quasi-linear recurrent neural network predictive control, the solution of the control law only need one-step derivation, which can greatly simplify the solution process of control law. At the same time, the quasi-linear recurrent neural network can effectively restrain the over-fitting problem in the identification process. Theoretical analysis and simulations are given to prove the simplicity and effectiveness of the proposed computing method.

Kernel recursive least squares (KRLS) has become a new research hotspot, attracting researchers' attention in the field of time series online prediction. As an online algorithm, its computational form is updating incrementally as a new sample achieved. But as the number of data samples increases, its computational complexity will increase. And there is a lack of ability to track time-varying characteristics in the online prediction process. Therefore, we add a sliding window approach to the sparse KRLS and propose a new improved algorithm, which is sparse kernel recursive least squares based on sliding window (SW-SKRLS). This improved algorithm can limit the computational complexity and perform better online prediction in time-varying environments. To prove the validity of the proposed algorithm, we make simulations on the Lorenz time series and Dalian monthly average temperature and rainfall time series. The simulation results show that this algorithm has good prediction accuracy and efficiency.

With the applications of Internet of Things (IoT) for smart cities, the real-time performance for a large number of network packets is facing serious challenge. Thus, how to improve the emergency response has become a critical issue. However, traditional packet scheduling algorithms cannot meet the requirements of the large-scale IoT system for smart cities. To address this shortcoming, this paper proposes EARS, an efficient data-emergency-aware packet scheduling scheme for smart cities. EARS describes the packet emergency information with the packet priority and deadline. Each source node informs the destination node of the packet emergency information before sending the packets. The destination node determines the packet scheduling sequence and processing sequence according to emergency information. Moreover, this paper compares EARS with a first-come, first-served, multilevel queue algorithm and a dynamic multilevel priority packet scheduling algorithm. Simulation results show that EARS outperforms these previous scheduling algorithms in terms of packet loss rate, average packet waiting time, and average packet end-to-end delay.

Echo state network is a novel kind of recurrent neural networks, with a trainable linear readout layer and a large fixed recurrent connected hidden layer, which can be used to map the rich dynamics of complex real-world data sets. It has been extensively studied in time series prediction. However, there may be an ill-posed problem caused by the number of real-world training samples less than the size of the hidden layer. In this brief, a Laplacian echo state network (LAESN), is proposed to overcome the ill-posed problem and obtain low-dimensional output weights. First, an echo state network is used to map the multivariate time series into a large reservoir. Then, assuming that an unknown underlying manifold is inside the reservoir, we employ the Laplacian eigenmaps to estimate the manifold by constructing an adjacency graph associated with the reservoir states. Finally, the output weights are calculated by the low-dimensional manifold. In addition, some criteria of transient stability, local controllability, and local observability are given. Experimental results based on two real-world data sets substantiate the effectiveness and characteristics of the proposed LAESN model.

In this paper, for a general class of uncertain nonlinear (cascade) systems, including unknown dynamics, which are not feedback linearizable and cannot be solved by existing approaches, an innovative adaptive approximation-based regulation control (AARC) scheme is developed. Within the framework of adding a power integrator (API), by deriving adaptive laws for output weights and prediction error compensation pertaining to single-hidden-layer feedforward network (SLFN) from the Lyapunov synthesis, a series of SLFN-based approximators are explicitly constructed to exactly dominate completely unknown dynamics. By the virtue of significant advancements on the API technique, an adaptive API methodology is eventually established in combination with SLFN-based adaptive approximators, and it contributes to a recursive mechanism for the AARC scheme. As a consequence, the output regulation error can asymptotically converge to the origin, and all other signals of the closed-loop system are uniformly ultimately bounded. Simulation studies and comprehensive comparisons with backstepping- and API-based approaches demonstrate that the proposed AARC scheme achieves remarkable performance and superiority in dealing with unknown dynamics.

For large-scale Internet of Things (IoT), which located in the hostile environment where exists malicious nodes (MNs), the security of time synchronization is a critical and challenging issue. The malicious sensor nodes could decrease the accuracy of the whole network by broadcasting fake timestamp messages. In this paper, we propose a secure time synchronization model for large-scale IoT. In this model, a node utilizes its father node and grandfather node to detect the MN. By employing the model, a spanning tree topology which synchronizes to the reference nodes can be constructed hop by hop. Then a secure time synchronization protocol is developed to against fake timestamps, which adopts the secure model. We use NS2 as the simulation tool to evaluate our protocol, and compare the impact of fake timestamps in various circumstances with the pervious protocols TPSN and STETS. The experiment results show that our protocol is effective to prevent attacks from MNs.

With the widespread applications of Internet of Things (IoT), the emergency response performance for large-scale network packets is facing serious challenge, especially for renewable distributed energy resources monitoring in a smart grid. Therefore, how to improve the real-time performance of the emergency data packets has been a critical issue. Traditional packet scheduling schemes and topology optimization strategies are not suitable for a large-scale IoT-based smart grid. To address this problem, this paper proposes a new packet scheduling scheme named LOES, which first combines the priority-based packet scheduling scheme with local optimization. We exchange local geographic information to reduce the hop counts and distance between distributed source nodes and sink nodes. Each destination node determines the packet scheduling sequence according to the received emergency information. Finally, we compare LOES with first come first serve, multilevel scheme, and dynamic multilevel priority packet scheduling scheme using packet loss rate, packet waiting time, and average packet end-to-end delay as metrics. The simulation results show that LOES outperforms these previous scheduling schemes.

This paper proposes a novel superpixel-based method for the classification of hyperspectral image. A superpixel segmentation algorithm called entropy rate superpixel is applied to extract the spatial contextual information in the hyperspectral image, which can change the size and shape of the superpixel adaptively according to spatial structures. Then, a joint sparse representation model is applied to approximate the pixels within each superpixel using a certain number of common samples from a given dictionary in the form of sparse linear combination. Here we use a greedy algorithm called simultaneous orthogonal matching pursuit to pursue the optimal sparse coefficients matrix and a new kind of classification criterion is tested and used to determine the classification results. Experimental results on the Indian Pines hyperspsectral image demonstrate that the proposed method can explore the spatial information effectively and give promising performance when compared with several state-of-art classification methods.

This paper reviews the performance difference and similarity in modeling system dynamics and the efficiency of error elimination with fewer fluctuation and adaptability to set point variation between the traditional fixed parameter proportional-integral-derivative controller and the model reference neural net controller applied to aircraft pitch attitude control. The designs of both controllers are presented. System identification using neural net was implemented to capture the system dynamics. Finally, the performances of the controllers were tested using a slow and fast varying input signals within the given bandwidth.

We studied finite-time function projective synchronization of unknown delayed Cohen-Grossberg neural networks with stochastic disturbance. A hybrid control scheme is proposed to let the drive-response networks synchronize and have a scaling function relation in finite time with topology identification by using the finite-time stability theory. Furthermore, we estimate the high bounds of the synchronization settling time. Finally, the corresponding numerical simulation and its application in secure communication are provided to verify the correctness of the method we proposed.

It is expensive in time or resources to obtain adequate labeled data for a new remote sensing image to be categorized. The cost of manual interpretation can be reduced if labeled samples collected from previous temporal images can be reused to classify a new image over the same investigated area. However, it is reasonable to consider that the distributions of the target data and the historical data are usually not identical. Therefore, the efficient strategy of transferring the beneficial information from historical images to the target image hits a bottleneck. In order to reuse sufficient historical samples to classify a given image with scarce labeled samples, this letter presents a novel transfer learning algorithm for remote sensing image classification based on extreme learning machine with weighted least square. This algorithm adds a transferring item to an objective function and adjusts historical and target training data with different weight strategies. Experiments on two sets of remote sensing images show that the presented algorithm reduces the requirement for target training samples and improves classification accuracy, timeliness, and integrity.

Echo state network (ESN) is a new kind of recurrent neural network with a randomly generated reservoir structure and an adaptable linear readout layer. It has been widely employed in the field of time series prediction. However, when high-dimensional reservoirs are utilized to predict multivariate time series, there may be a collinearity problem. In this paper, to overcome the collinearity problem and obtain a sparse solution, we propose a new model-adaptive elastic ESN, in which adaptive elastic net algorithm is used to calculate the unknown weights. It combines the strengths of the quadratic regularization and the adaptively weighted lasso shrinkage. Hence, the proposed model can deal with the collinearity problem and enjoy the oracle property with an unbiased estimation. We exhibit the merits of our model on two benchmark multivariate chaotic datasets and two real-world applications. Experimental results substantiate the effectiveness and characteristics of the proposed model.

Post-Classification Comparison(PCC) method is widely used in change detection for remote sensing images, but it is affected by a significant cumulative error caused by single remote sensing image classification during change detection, which leads to the excessive evaluation of changed types and quantity. To solve this problem, this paper proposes a change detection method for remote sensing images based on Adaptive Resonance Theory Mapping (ARTMAP) neural network. Similarity matrix is constructed by spectral feature vectors. Then the threshold value of similarity is obtained, which is used to control the joint-classification classifier based on the ARTMAP neural network. In addition, an adaptive algorithm of vigilance parameter is introduced to the classification process of fuzzy ARTMAP neural network. The experimental results obtained on remote sensing images show that the proposed method not only accurately classifies the unchanged geographical information in different temporal images into the same class, but also reduces the cumulative error and improves the accuracy of change detection compared with other methods.

Magnetic induction tomography (MIT) is a non-invasive technology for visualization of the conductivity distribution inside inhomogeneous media. So far, the resolution of MIT has not been high enough for practical applications in biomedical imaging yet. In this research, we investigate the image reconstruction problem using statistical classification method to enhance the resolution of MIT. First, Tikhonov regularization or iteration Newton-Raphson algorithm is used to recover the initial conductivities of media understudy. Then, by setting a threshold on the basis of Otsu, the recovered conductivities are classified into some groups, whose labels can be obtained by some prior knowledge. Finally, according to the classification results, the conductivity distribution is spatially visualized. Simulation experiments are conducted, and the applicability and effectiveness of the proposed method are shown by compared with some other well developed methods.

In this paper, the motion dynamics of a large tanker is modeled by the generalized ellipsoidal function-based fuzzy neural network (GEBF-FNN). The reference model of tanker motion dynamics in the form of nonlinear difference equations is established to generate training data samples for the GEBF-FNN algorithm which begins with no hidden neuron. In the sequel, fuzzy rules associated with the GEBF-FNN-based model can be online self-constructed by generation criteria and parameter estimation, and can dynamically capture essential motion dynamics of the large tanker with high prediction accuracy. Simulation studies and comprehensive comparisons are conducted on typical zig-zag maneuvers with moderate and extreme steering, and demonstrate that the GEBF-FNN-based model of tanker motion dynamics achieves superior performance in terms of both approximation and prediction.

This paper deals with tanker steering control based on a novel multiple-input multiple-output generalized ellipsoidal-basis-function-based fuzzy neural network (GEBF-FNN) with online updating of system structure and parameters. The main contributions of this paper are as follows. 1) A GEBF-FNN-based nonlinear steering model incorporating the nonlinearity underlying tanker dynamics is proposed. 2) The static local controller (SLC), whose controller gains are locally fixed with the initial forward speed and the desired heading for individual steering commands, is implemented. 3) The dynamic local controller (DLC) is further realized by employing adaptive controller gains pertaining to time-varying forward speed and heading dynamics. 4) The GEBF-FNN-based steering controller is developed by identifying a nonlinear mapping from the heading error, acceleration and forward speed to dynamic controller gains, and thereby contributing to a model-free adaptive control scheme. Simulation results and comprehensive studies on benchmark problems demonstrate that the GEBF-FNN-based model can capture the essential tanker dynamics, and the proposed SLC, DLC, and GEBF-FNN-based schemes achieve superior performance in terms of heading regulation and forward speed loss. In comparison with the SLC and traditional fuzzy controllers, the DLC and GEBF-FNN-based controllers achieve higher accuracy of heading regulation with less rudder efforts and minimal forward speed losses.

In this paper, traditional single-hidden layer feedforward network (SLFN) is extended to novel generalized SLFN (GSLFN) by employing polynomial functions of inputs as output weights connecting randomly generated hidden units with corresponding output nodes. The significant contributions of this paper are as follows: 1) a primal GSLFN (P-GSLFN) is implemented using randomly generated hidden nodes and polynomial output weights whereby the regression matrix is augmented by full or partial input variables and only polynomial coefficients are to be estimated; 2) a simplified GSLFN (S-GSLFN) is realized by decomposing the polynomial output weights of the P-GSLFN into randomly generated polynomial nodes and tunable output weights; 3) both P- and S-GSLFN are able to achieve universal approximation if the output weights are tuned by ridge regression estimators; and 4) by virtue of the developed batch and online sequential ridge ELM (BR-ELM and OSR-ELM) learning algorithms, high performance of the proposed GSLFNs in terms of generalization and learning speed is guaranteed. Comprehensive simulation studies and comparisons with standard SLFNs are carried out on real-world regression benchmark data sets. Simulation results demonstrate that the innovative GSLFNs using BR-ELM and OSR-ELM are superior to standard SLFNs in terms of accuracy, training speed, and structure compactness.

In this paper, an adaptive robust dynamic constructive fuzzy neural control (AR-DCFNC) scheme for trajectory tracking of a surface vehicle with uncertainties and unknown time-varying disturbances is proposed. System uncertainties and unknown dynamics are identified online by a dynamic constructive fuzzy neural network (DCFNN) which is implemented by employing dynamically constructive fuzzy rules according to the structure learning criteria. The entire AR-DCFNC system is globally asymptotical stable.

This paper proposed a robust structure selection method of radial basis function (RBF) networks for nonlinear system identification problems. A greedy algorithm is first employed by combining information criteria with the forward stepwise selection to choose the RBF network structures. Then, a robust selection procedure, which can select a concise and generalized network structure, is developed based on the forward stepwise selection and the subsampling method. Finally, a numerical example is given to illustrate the effectiveness of the proposed method by using the disturbance storm time index data.

In this paper, a vessel maneuvering model (VMM) based on multi-output dynamic radial-basis-function network (MDRBFN) is proposed. Data samples used for training and testing are obtained from the vessel maneuvering dynamics based on a group of nonlinear differential equations. In order to identify the vessel maneuvering model, the differential equations are transformed into nonlinear state-space form. Considering that the desired states are not only dependent on system inputs, i е., rudder defection and propeller revolution, but also previous states, the proposed MDRBFN is focus on the multi-input multi-output (MIMO) case. The structure of traditional fixed-size RBF networks is difficult to determine, so the growing and pruning algorithm is introduced to multi-output RBF networks to realize RBF networks with dynamic structure. The MDRBFN starts with no hidden neurons, and during the learning process, hidden neurons are recruited automatically according to hidden nodes generation criteria and parameters estimation. In addition, insignificant hidden nodes would be deleted if the node significance is lower than the predefined threshold. As a consequence, the proposed MDRBFN-based VMM (MDRBFN-VMM) reasonably captures the essential maneuvering dynamics with a compact structure. Finally, simulation results indicate that the proposed MDRBFN-VMM achieves promising performance in terms of approximation and prediction.

In this paper, an adaptive self-constructing RBF neural control (AS-RBFNC) scheme for trajectory tracking of MIMO uncertain nonlinear systems with unknown time-varying disturbances is proposed. System uncertainties and unknown dynamics can be exactly identified online by a self-constructing RBF neural network (SC-RBFNN) which is implemented by employing dynamically constructive hidden nodes according to the structure learning criteria including hidden node generating and pruning. The globally asymptotical stability of the entire AS-RBFNC control system is derived from Lyapunov approach.

In this paper, a multiple kernel extreme learning machine (MKELM) is proposed for multivariate time series prediction. The multivariate time series is reconstructed in phase space, and a variable selection algorithm is then applied to form the compact and relevant input for the prediction model. On the basis of multiple kernel learning and extreme learning machine with kernels, multi different kernels is used in MKELM to present the dynamics of multivariate time series. A simulation example, prediction of Lorenz chaotic time series is conducted to demonstrate the effectiveness of the proposed method.

Large-scale optimization arises in a variety of scientific and engineering applications. In this paper, a particle swarm optimization (PSO) approach with dynamic neighborhood that is based on kernel fuzzy clustering and variable trust region methods (called FT-DNPSO) is proposed for large-scale optimization. The cooperative coevolution incorporated with a kernel fuzzy C-means clustering strategy is introduced to divide high-dimensional problems in to subproblems, and explore their search spaces. Furthermore, the independent variable ranges change adaptably by using the variable trust region learning method, which expedites the convergence process and explores in the effective space. In addition, the dynamic neighborhood topology assists the PSO algorithm in cooperating with neighbor particles and avoids the problem of premature convergence. Simulation results substantiate the effectiveness of the proposed algorithm to solve large-scale optimization problems with many well-known benchmark functions.

Novel constructive and destructive parsimonious extreme learning machines (CP- and DP-ELM) are proposed in this paper. By virtue of the proposed ELMs, parsimonious structure and excellent generalization of multiinput-multioutput single hidden-layer feedforward networks (SLFNs) are obtained. The proposed ELMs are developed by innovative decomposition of the recursive orthogonal least squares procedure into sequential partial orthogonalization (SPO). The salient features of the proposed approaches are as follows: 1) Initial hidden nodes are randomly generated by the ELM methodology and recursively orthogonalized into an upper triangular matrix with dramatic reduction in matrix size; 2) the constructive SPO in the CP-ELM focuses on the partial matrix with the subcolumn of the selected regressor including nonzeros as the first column while the destructive SPO in the DP-ELM operates on the partial matrix including elements determined by the removed regressor; 3) termination criteria for CP- and DP-ELM are simplified by the additional residual error reduction method; and 4) the output weights of the SLFN need not be solved in the model selection procedure and is derived from the final upper triangular equation by backward substitution. Both single- and multi-output real-world regression data sets are used to verify the effectiveness and superiority of the CP- and DP-ELM in terms of parsimonious architecture and generalization accuracy. Innovative applications to nonlinear time-series modeling demonstrate superior identification results.

A robust neural predictor is designed for noisy chaotic time series prediction in this paper. The main idea is based on the consideration of the bounded uncertainty in predictor input, and it is a typical Errors-in-Variables problem. The robust design is based on the linear-in-parameters ESN (Echo State Network) model. By minimizing the worst-case residual induced by the bounded perturbations in the echo state variables, the robust predictor is obtained in coping with the uncertainty in the noisy time series. In the experiment, the classical Mackey-Glass 84-step benchmark prediction task is investigated. The prediction performance is studied for the nominal and robust design of ESN predictors.

Chaotic time series prediction has received considerable attention in the last few years. Although many studies have been conducted in the field, there is little attention focused on multivariate time series prediction. Considering this problem, a multi reservoir support vector echo state machine(MRSVESM) based on multi kernel learning and echo state networks is proposed in this paper. The single reservoir approach may be ineffective on multivariate time series prediction, as it is not able to character multi time scale dynamics. The MRSVESM use multi different time scale reservoirs to present the dynamics of multivariate time series and replaced the "kernel trick" with "reservoir trick", that is, performed multi kernel learning in the high dimension "reservoir" state space. Two simulation examples, prediction of Lorenz chaotic time series and prediction of sunspots and the Yellow River annual runoff time series are conducted to demonstrate the effectiveness of the proposed method.

This paper presents a new nonlinear model predictive control (NMPC) strategy based on the Gaussian particle swarm optimization (GPSO). Through the Taylor expansion, NMPC transform to a quadratic programming problem with unknown parameters. Hence, for the global convergence character and higher optimization accuracy, GPSO is employed to dynamically perform nonlinear constraint optimization. Finally, the proposed control strategy is applied to Ball-Plate system to verify the effectiveness.

Extreme Learning Machine (ELM) is a new paradigm for using Single-hidden Layer Feedforward Networks (SLFNs) with a much simpler training method. The input weights and the bias of the hidden layer are randomly chosen and output weights are analytically determined. One of the open problems in ELM research is how to automatically determine network architectures for given tasks. In this paper, it is taken as a model selection problem, a modified fast recursive algorithm (MFRA) is introduced to quickly and efficiently estimate the contribution of each hidden layer node to the decrease of the net function, and then a leave one out (LOO) cross validation is used to select the optimal number of hidden layer nodes. Simulation results on both artificial and real world benchmark datasets indicate the effectiveness of the proposed method.

Chaotic time series prediction has received considerable attention in the last few years. Although many studies have been conducted in the field, there is little attention focused on multivariate time series prediction. Considering this problem, the Hierarchic Reservoirs (HR) prediction model is proposed for multivariate chaotic time series prediction in this paper. The basic idea is using multiple reservoirs to predict multivariate chaotic time series directly without using phase space reconstruction. Each single reservoir of the hierarchic reservoirs prediction model extract the features of a time series of the multivariate chaotic time series. Then, the features are composed to represent the target value of the time series. Two simulation examples, prediction of Lorenz chaotic time series and prediction of sunspots and the Yellow River annual runoff time series are conducted to demonstrate the effectiveness of the proposed method.

In this paper, a robust recurrent neural network is presented in a Bayesian framework based on echo state mechanisms. Since the new model is capable of handling outliers in the training data set, it is termed as a robust echo state network (RESN). The RESN inherits the basic idea of ESN learning in a Bayesian framework, but replaces the commonly used Gaussian distribution with a Laplace one, which is more robust to outliers, as the likelihood function of the model output. Moreover, the training of the RESN is facilitated by employing a bound optimization algorithm, based on which, a proper surrogate function is derived and the Laplace likelihood function is approximated by a Gaussian one, while remaining robust to outliers. It leads to an efficient method for estimating model parameters, which can be solved by using a Bayesian evidence procedure in a fully autonomous way. Experimental results show that the proposed method is robust in the presence of outliers and is superior to existing methods.

A dynamic feedforward neural network (DFNN) is proposed for predictive control, whose adaptive parameters are adjusted by using Gaussian particle swarm optimization (GPSO) in the training process. Adaptive time-delay operators are added in the DFNN to improve its generalization for poorly known nonlinear dynamic systems with long time delays. Furthermore, GPSO adopts a chaotic map with Gaussian function to balance the exploration and exploitation capabilities of particles, which improves the computational efficiency without compromising the performance of the DFNN. The stability of the particle dynamics is analyzed, based on the robust stability theory, without any restrictive assumption. A stability condition for the GPSO+DFNN model is derived, which ensures a satisfactory global search and quick convergence, without the need for gradients. The particle velocity ranges could change adaptively during the optimization process. The results of a comparative study show that the performance of the proposed algorithm can compete with selected algorithms on benchmark problems. Additional simulation results demonstrate the effectiveness and accuracy of the proposed combination algorithm in identifying and controlling nonlinear systems with long time delays.

In order to improve the generalization capacity of neural networks for poorly known nonlinear dynamic system with long time-delay, a novel adaptive dynamic feedforward neural network on modified particle swarm optimization (PSO) algorithm is proposed. The adaptive time delay operator is adopted between input layer and the first hidden layer, and also the last hidden layer and output layer. Utilizing these dynamic time delay parameters, the proposed structure can adequately identify different classes of nonlinear systems expressed in the input-output representation form and pure time delay. Otherwise, to overcome the particles' premature convergence, the white noise and logistic mapping are used to enhance the particles' search performance. Furthermore, the parameters in the dynamic feedforward neural network are trained by the modified PSO method. The proposed neural network shows a satisfactory global search and quick convergence capability, avoiding the complexity of gradient calculation. Simulation results demonstrate that the proposed algorithm is effective and accurate in identifying long-time delay nonlinear systems through the comparison with other methods.

This paper presents a novel dynamic neural network (DNN) predictive control strategy based on modified particle swarm optimization (PSO) for long time delay nonlinear process. The proposed dynamic NN structure could approximate to the actual system model and obtain the pure delay time exactly. An improved version of the original PSO is put forward to train the parameters of NN to enhance the convergence and accuracy. The effectiveness of the proposed control scheme is demonstrated by simulation as well as a test on an experiment on the actual pH Neutralization Process.

A two-step approach to enhance the resolution of remote sensing thermal infrared (TIR) images is proposed in this paper. For difference in imaging principles between TIR image and optical images, traditional image fusion techniques, such as component substation and MRA methods will not be proper. In our study, we use extreme learning machine (ELM) to regress the relationship between TIR image and optical images, then pansharpened multi spectral images are inputted to the already trained ELM network to produce TIR image at resolution of the panchromatic image. Since the approach considers directly about radiance values in a TIR image, the result can be conveniently used in physical applications, for example, creating more precise temperature distribution of ground surface.

This paper proposes an improved nonlinear canonical correlation analysis algorithm named radial basis function canonical correlation analysis (RBFCCA) for multivariate chaotic time series analysis and prediction. This algorithm follows the key idea of kernel canonical correlation analysis (KCCA) method to make a nonlinear mapping of the original data sets firstly with a RBF network and a linear neural network. Then linear CCA is performed using the transformed nonlinear data sets, which corresponds to make nonlinear CCA of the original data. A modified cost function of the neural network with Lagrange multipliers and a joint learning rule based on gradient ascent algorithm which maximizes the correlation coefficient of the network outputs is used to extract the maximal correlation pattern between the input and output of a prediction model. Finally, a regression model is constructed to implement the prediction problem. The performance of RBFCCA prediction algorithm is demonstrated via the prediction problem of Lorenz time series and some practical observed time series. The results compared with the traditional neural network method and the KCCA method indicate that the RBFCCA algorithm proposed in this paper is able to capture the dynamics of complex systems and give reliable prediction accuracy.

This paper presents a modified radial basis function (RBF) neural network for pattern recognition problems, which uses a hybrid learning algorithm to adaptively adjust the structure of the network. Two strategies are used to attain the compromise between the network complexity and accuracy, one is a modified "novelty" condition to create a new neuron in the hidden layer; the other is a pruning technique to remove redundant neurons and corresponding connections. To verify the performance of the modified network, two pattern recognition simulations are completed. One is a two-class pattern recognition problem, and the other is a real-world problem, internal component recognition in the field of architecture engineering. Simulation results including final hidden neurons, error, and accuracy using the method proposed in this paper are compared with performance of radial basis functional link network, resource allocating network and RBF neural network with generalized competitive learning algorithm. And it can be concluded that the proposed network has more concise architecture, higher classifier accuracy and fewer running time.

In this letter, a new threshold algorithm based on wavelet analysis is applied to smooth noise for a nonlinear time series. By detailing the signals decomposed onto different scales, we smooth the details by using the updated thresholds to different characters of a noisy nonlinear signal. This method is an improvement of Donoho's wavelet methods to nonlinear signals. The approach has been successfully applied to smoothing the noisy chaotic time series generated by the Lorenz system as well as the observed annual runoff of Yellow River. For the nonlinear dynamical system, an attempt is made to analyze the noise reduced data by using multiresolution analysis, i.e., the false nearest neighbors, correlation integral, and autocorrelation function, to verify the proposed noise smoothing algorithm

A novel chaotic time-series prediction method based on support vector machines (SVMs) and echo-state mechanisms is proposed. The basic idea is replacing "kernel trick" with "reservoir trick" in dealing with nonlinearity, that is, performing linear support vector regression (SVR) in the high-dimension "reservoir" state space, and the solution benefits from the advantages from structural risk minimization principle, and we call it support vector echo-state machines (SVESMs). SVESMs belong to a special kind of recurrent neural networks (RNNs) with convex objective function, and their solution is global, optimal, and unique. SVESMs are especially efficient in dealing with real life nonlinear time series, and its generalization ability and robustness are obtained by regularization operator and robust loss function. The method is tested on the benchmark prediction problem of Mackey-Glass time series and applied to some real life time series such as monthly sunspots time series and runoff time series of the Yellow River, and the prediction results are promising.

Multivariate time series are common in experimental and real systems. According to the embedding theory, in the absence of observational noise only one time series should be needed to recover dynamics. However, for real data, the noise always exist. There may be large advantages in using more measurements. In this paper, we focus on the issue of using multivariate time series to model and predict. The experiments show that by using multivariate time series the influence of noise could be reduced. Since the structure of the embedded time series is complex, the singular value decomposition (SVD) is used to extract feature components in the multivariate time series. Then the neural network (NN) is applied for identification of the dynamic system. The effectiveness of this method is shown by simulation of the real world multivariate time series as well as a well-known chaotic benchmark system.

The paper presents a generalized predictive control (GFC) algorithm based on feedforward neural network to control nonlinear system. In recent years, approximate linearization theory via feedback is used to control nonlinear system, but robustness can not be guaranteed. Considering neural network can accomplish nonlinear mapping from input to output, feedforward neural network is chosen as a nonlinear model of process. Based on such model, GPC is applied to control a second-order nonlinear system. To test the performance of system utilized such control algorithm, different experiments are made. Simulation results demonstrate that the performance of the system controlled by the proposed algorithm is good, and that system essentially responds in the desired manner. It is also demonstrated that the GPC based on neural network is provided with good adaptation and robustness.

For chaotic system prediction, ESNs (echo state networks) are realization of neural state reconstruction, in which the reconstructed state variable is from the internal neurons' activation, rather than the delay vector obtained from delay coordinate reconstruction. In the framework of the neural state reconstruction, some quantitative analyses can be further made on the issues such as the network structure configuration and initial state determination. Based on the simulation study on chaotic data from Chua's circuit, it is shown that the ESN is a non-minimum state space realization of the target time series, and the initial state can be freely chosen in the training process, and in the phase of prediction, ESN needs to know where the prediction begins by being set a proper initial state through a process of teacher forcing.

This work mainly discusses the application of an improved fuzzy ARTMAP network based on fuzzy logic in wetland classification. The main improvement is represented in two phases: first in the category competition phase, fuzzy reasoning replaces the simply similarity criterion; second in studying (resonance) phase, not only the winning neuron's weights but also the fuzzy membership functions are updated according to input characters. The data source is TM images of Zhalong wetland nature reserve in north China, and both spectrum and spatial characters are extracted from it. A comparison of classification performance of three automatic classifiers, namely the improved fuzzy ARTMAP, the normal fuzzy ARTMAP and BPNN, is made in This work. The result indicates that the improved fuzzy ARTMAP is superior to other classifiers in supervised classification of wetland.

This paper applies neural network to extract marsh information. An adaptive back-propagation algorithm based on a robust error function is introduced to build a four-layer neural network, and it is used to classify Thematic Mapper (TM) image of Zhalong Wetland in China and then extract marsh information. Comparing marsh information extraction results of the four-layer neural network with three-layer neural network and the maximum likelihood classifier, conclusion can be drawn as follows: the structure of the four-layer neural network and the adaptive back-propagation algorithm based on the robust error function is effective to extract marsh information. The four-layer neural network adopted in this paper succeeded in building the complex model of TM image, and it avoided the problem of great storage of remotely sensed data, and the adaptive back-propagation algorithm speeded up the descending of error. Above all, the four-layer neural network is superior to the three-layer neural network and the maximum likelihood classifier in the accuracy of the total classification and marsh information extraction.

The purpose of this paper is to apply a neural network to classify aerial photographs. An adaptive backpropagation algorithm is introduced to build a four-layer neural network as the practical model to classify aerial photographs based on pixel-pixel. The practical example is the classification of lake, forest and land, comparing a neural network with the maximum likelihood classification through the analysis of the process of classification and the results. The four-layer neural network introduced in this paper succeeded in building the complex model as an aerial photograph and it solved the problem of the great storage of data. The adaptive back-propagation algorithm adopted in this paper speeded up the learning rate, the general error decreased to a smaller degree, and the generalization ability of the neural network was also improved. The results demonstrated that the neural network is suitable to be used in the classification of remotely sensed data and is superior to the maximum likelihood classifier in the accuracy of the classification and the overall effect.