The increasing demand for accurate EEG-based epileptic seizure detection calls for more sophisticated and semantically informed methodologies. Traditional approaches often struggle to capture the complex spatiotemporal patterns inherent in EEG signals and typically lack high-level contextual understanding, limiting their applicability in real-world clinical settings. In this study, we propose EEG-VL, a novel visionlanguage framework that treats EEG signals as visual patterns and integrates them with large language models to improve seizure detection. Specifically, a pretrained EfficientNet encoder is used to extract abstract visual features from EEG representations, which are embedded into structured prompts and processed by the Qwen language model. This design synergistically combines the spatial modeling capabilities of convolutional networks with the semantic reasoning strengths of large language models. To address the class imbalance commonly present in seizure datasets, we adopt a logit adjustment strategy based on label distribution priors. Extensive experiments on the TUSZ and CHB-MIT datasets demonstrate that EEG-VL achieves state-of-the-art performance. On TUSZ, our model attains an AUPRC of 0.7599 and an AUROC of 0.9466, surpassing previous best results by 8.19 % and 0.80 %, respectively. These findings underscore the potential of the proposed vision-language paradigm for robust, scalable, and clinically applicable EEG-based seizure detection.

Multi-Agent Reinforcement Learning (MARL) has seen significant advances, with the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm standing out for its effectiveness in cooperative settings. However, the theoretical foundation for why MAPPO performs well has not been fully explored. In this paper, we address this gap by providing a theoretical analysis of MAPPO, particularly in cooperative environments. Building on existing policy optimization formulations, we analyze key properties of the algorithm to explain its convergence and performance. Our analysis highlights the mechanisms that enable MAPPO to handle cooperative MARL tasks effectively. Additionally, we enhance the algorithm for continuous control by incorporating trust-region policy optimization, which allows for more stable updates. This improvement eliminates the need for parameter sharing, supports heterogeneous agents, and uses a squashed Gaussian distribution to handle bounded actions effectively. We validate the algorithm through experiments on multi-agent robotic tasks, demonstrating that our theoretical insights not only enhance the understanding of MAPPO but also improve its practical performance.

To enhance deep learning-based automated interictal epileptiform discharge (IED) detection, this study proposes a multimodal method, vEpiNet, that leverages video and electroencephalogram (EEG) data. Datasets comprise 24 931 IED (from 484 patients) and 166 094 non-IED 4-second video-EEG segments. The video data is processed by the proposed patient detection method, with frame difference and Simple Keypoints (SKPS) capturing patients' movements. EEG data is processed with EfficientNetV2. The video and EEG features are fused via a multilayer perceptron. We developed a comparative model, termed nEpiNet, to test the effectiveness of the video feature in vEpiNet. The 10-fold cross-validation was used for testing. The 10-fold cross-validation showed high areas under the receiver operating characteristic curve (AUROC) in both models, with a slightly superior AUROC (0.9902) in vEpiNet compared to nEpiNet (0.9878). Moreover, to test the model performance in real-world scenarios, we set a prospective test dataset, containing 215 h of raw video-EEG data from 50 patients. The result shows that the vEpiNet achieves an area under the precision-recall curve (AUPRC) of 0.8623, surpassing nEpiNet's 0.8316. Incorporating video data raises precision from 70% (95% CI, 69.8%-70.2%) to 76.6% (95% CI, 74.9%-78.2%) at 80% sensitivity and reduces false positives by nearly a third, with vEpiNet processing one-hour video-EEG data in 5.7 min on average. Our findings indicate that video data can significantly improve the performance and precision of IED detection, especially in prospective real clinic testing. It suggests that vEpiNet is a clinically viable and effective tool for IED analysis in real-world applications.

We extend trust region policy optimization (TRPO) to cooperative multiagent reinforcement learning (MARL) for partially observable Markov games (POMGs). We show that the policy update rule in TRPO can be equivalently transformed into a distributed consensus optimization for networked agents when the agents' observation is sufficient. By using a local convexification and trust-region method, we propose a fully decentralized MARL algorithm based on a distributed alternating direction method of multipliers (ADMM). During training, agents only share local policy ratios with neighbors via a peer-to-peer communication network. Compared with traditional centralized training methods in MARL, the proposed algorithm does not need a control center to collect global information, such as global state, collective reward, or shared policy and value network parameters. Experiments on two cooperative environments demonstrate the effectiveness of the proposed method.

In existing graph-based multi-view clustering algorithms, consensus cluster structures are explored by constructing similarity graphs of multiple views and then fusing them into a unified superior graph. However, they overlook consensus information when learning each graph independently, resulting in the undesirable unified graph with biases. To this end, we proposed a framework named bipartite graph based multi-view clustering (BIGMC) in [1] to tackle this challenge. To summarize, the key idea of BIGMC is to employ a small number of uniform anchors to represent the consensus information across views. In this way, BIGMC creates a bipartite graph between data points and anchors for each view, which are then fused to generate a unified bipartite graph. The unified graph would in turn improve each view bipartite graph and the anchor set. Finally, the clusters are formed directly using the unified graph. In this extended abstract, we also summarize the effectiveness of BIGMC as shown in experimental results originally presented in [1].

Reinforcement learning (RL) is a powerful tool for training agents to interact with complex environments. In particular, trust-region methods are widely used for policy optimization in model-free RL. However, these methods suffer from high sample complexity due to their on-policy nature, which requires interactions with the environment for each update. To address this issue, off-policy trust-region methods have been proposed, but they have shown limited success in highdimensional continuous control problems compared to other off-policy DRL methods. To improve the performance and sample efficiency of trust-region policy optimization, we propose an off-policy trust-region RL algorithm. Our algorithm is based on a theoretical result on a closed-form solution to trust-region policy optimization and is effective in optimizing complex nonlinear policies. We demonstrate the superiority of our algorithm over prior trust-region DRL methods and show that it achieves excellent performance on a range of continuous control tasks in the Multi-Joint dynamics with Contact (MuJoCo) environment, comparable to state-of-the-art off-policy algorithms.

Incomplete multi-view clustering (IMC) aims to integrate the complementary information from incomplete views to improve clustering performance. Most existing IMC methods try to fill the incomplete views or directly learn a common representation based on matrix factorization or subspace learning. The former may introduce useless even noisy information especially for data with a large missing ratio. The latter relies on the initialization and ignores the geometric structure of data. To address these issues, we propose a novel Joint Partition and Graph (JPG) learning method for IMC. Specifically, JPG jointly constructs local incomplete graph matrices, generates incomplete base partition matrices, stretches them to produce a unified partition matrix, and employs it to learn a consensus graph matrix. By this means, we transform incomplete multi-view data into a unified partition space and obtain the consensus graph in a mutual reinforcement manner. Moreover, a partition fusion strategy can allocate a large weight to the stretched base partition that is close to the unified matrix. The objective function is optimized in an alternating optimization fashion. Experimental results on several benchmark datasets demonstrate the effectiveness and superiority of JPG than the state-of-the-art baselines

We present an analytical policy update rule that is independent of parametric function approximators. The policy update rule is suitable for optimizing general stochastic policies and has a monotonic improvement guarantee. It is derived from a closed-form solution to trust-region optimization using calculus of variation, following a new theoretical result that tightens existing bounds for policy improvement using trust-region methods. The update rule builds a connection between policy search methods and value function methods. Moreover, off-policy reinforcement learning algorithms can be derived from the update rule since it does not need to compute integration over on-policy states. In addition, the update rule extends immediately to cooperative multi-agent systems when policy updates are performed by one agent at a time.

Recently, graph convolutional networks (GCNs) and their variants have achieved remarkable successes for the graph-based semisupervised node classification problem. With a GCN, node features are locally smoothed based on the information aggregated from their neighborhoods defined by the graph topology. In most of the existing methods, the graph typologies only contain positive links which are deemed as descriptions for the feature similarity of connected nodes. In this article, we develop a novel GCN-based learning framework that improves the node representation inference capability by including negative links in a graph. Negative links in our method define the inverse correlations for the nodes connected by them and are adaptively generated through a neural-network-based generation model. To make the generated negative links beneficial for the classification performance, this negative link generation model is jointly optimized with the GCN used for class inference through our designed training algorithm. Experiment results show that the proposed learning framework achieves better or matched performance compared to the current state-of-the-art methods on several standard benchmark datasets.

This article presents a new command-filtered composite adaptive neural control scheme for uncertain nonlinear systems. Compared with existing works, this approach focuses on achieving finite-time convergent composite adaptive control for the higher-order nonlinear system with unknown nonlinearities, parameter uncertainties, and external disturbances. First, radial basis function neural networks (NNs) are utilized to approximate the unknown functions of the considered uncertain nonlinear system. By constructing the prediction errors from the serial-parallel nonsmooth estimation models, the prediction errors and the tracking errors are fused to update the weights of the NNs. Afterward, the composite adaptive neural backstepping control scheme is proposed via nonsmooth command filter and adaptive disturbance estimation techniques. The proposed control scheme ensures that high-precision tracking performances and NN approximation performances can be achieved simultaneously. Meanwhile, it can avoid the singularity problem in the finite-time backstepping framework. Moreover, it is proved that all signals in the closed-loop control system can be convergent in finite time. Finally, simulation results are given to illustrate the effectiveness of the proposed control scheme.

Recently, deep-learning-based feature extraction (FE) methods have shown great potential in hyperspectral image (HSI) processing. Unfortunately, it also brings a challenge that the training of the deep learning networks always requires large amounts of labeled samples, which is hardly available for HSI data. To address this issue, in this article, a novel unsupervised deep-learning-based FE method is proposed, which is trained in an end-to-end style. The proposed framework consists of an encoder subnetwork and a decoder subnetwork. The structure of the two subnetworks is symmetric for obtaining better downsampling and upsampling representation. Considering both spectral and spatial information, 3-D all convolution nets and deconvolution nets are used to structure the encoder subnetwork and decoder subnetwork, respectively. However, 3-D convolution and deconvolution kernels bring more parameters, which can deteriorate the quality of the obtained features. To alleviate this problem, a novel cost function with a sparse regular term is designed to obtain more robust feature representation. Experimental results on publicly available datasets indicate that the proposed method can obtain robust and effective features for subsequent classification tasks.

Unsupervised domain adaptation (DA) aims to perform classification tasks on the target domain by leveraging rich labeled data in the existing source domain. The key insight of DA is to reduce domain divergence by learning domain-invariant features or transferable instances. Despite its rapid development, there still exist several challenges to explore. At the feature level, aligning both domains only in a single way (i.e., geometrical or statistical) has limited ability to reduce the domain divergence. At the instance level, interfering instances often obstruct learning a discriminant subspace when performing the geometrical alignment. At the classifier level, only minimizing the empirical risk on the source domain may result in a negative transfer. To tackle these challenges, this article proposes a novel DA method, called discriminant geometrical and statistical alignment (DGSA). DGSA first aligns the geometrical structure of both domains by projecting original space into a Grassmann manifold, then matches the statistical distributions of both domains by minimizing their maximum mean discrepancy on the manifold. In the former step, DGSA only selects the density peaks to learn the Grassmann manifold and so to reduce the influences of interfering instances. In addition, DGSA exploits the high-confidence soft labels of target landmarks to learn a more discriminant manifold. In the latter step, a structural risk minimization (SRM) classifier is learned to match the distributions (both marginal and conditional) and predict the target labels at the same time. Extensive experiments on objection recognition and human activity recognition tasks demonstrate that DGSA can achieve better performance than the comparison methods.

For graph-based multi-view clustering, a critical issue is to capture consensus cluster structures via a two-stage learning scheme. Specifically, first learn similarity graph matrices of multiple views and then fuse them into a unified superior graph matrix. Most current methods learn pairwise similarities between data points for each view independently, which is widely used in single view. However, the consensus information contained in multiple views are ignored, and the involved biases lead to an undesirable unified graph matrix. To this end, we propose a bipartite graph based multi-view clustering (BIGMC) approach. The consensus information can be represented by a small number of representative uniform anchor points for different views. A bipartite graph is constructed between data points and the anchor points. BIGMC constructs the bipartite graph matrices of all views and fuses them to produce a unified bipartite graph matrix. The unified bipartite graph matrix in turn improves the bipartite graph similarity matrix of each view and updates the anchor points. The final unified graph matrix forms the final clusters directly. In BIGMC, an adaptive weight is added for each view to avoid outlier views. A low-rank constraint is imposed on the Laplacian matrix of the unified matrix to construct a multi-component unified bipartite graph, where the component number corresponds to the required cluster number. The objective function is optimized in an alternating optimization fashion. Experimental results on synthetic and real-world data sets demonstrate its effectiveness and superiority compared with the state-of-the-art baselines.

This paper concerns interference estimation problem for device-to-device (D2D) communication networks. In the considered system, D2D users share common spectrum resources, such that the D2D links have interference with each other. To achieve effective interference management, it is necessary to have an accurate understanding of the interference relationship of the D2D devices, which is difficult since the locations and the number of mobile devices can vary over time. In this paper, we formulate an interference estimation problem for the D2D communication network where the D2D users change over time. Our objective is to get accurate estimations for the interference suffered by a generic D2D link and for the interference a D2D link introduces on other users. We propose a graph convolutional neural network (GCN) based estimation model which can estimate the interference of a D2D link based on the location information of the corresponding D2D pairs. Simulation results show the performance of our method.

This paper concerns the cooperative spectrum sensing (CSS) for cognitive radio (CR) networks, where the secondary users (SUs) collaborate to detect the presence of the primary users (PUs). With CSS, the information from different SUs is first fused, then, the detection of the PU signal is implemented based on the fused information. Most of the previous works focus on the design of a mapping function that calculates the probability of the existence of the PU signal based on the fused information. In this paper, we study the fusion process which combines the information from all SUs based on the local property of each SU. A graph-based multi-view learning framework for CSS (GMCSS) is designed to better fuse the information from different SUs. In the proposed framework, the information from each SU is considered as a view of the state of the target wireless channel and is fused with the information from other SUs through a graph-based learning process. Simulation results demonstrate the effectiveness of our method.

In this paper, we propose a multi-agent deep reinforcement learning (DRL) based approach to solve the distributed Volt-VAR optimization (VVO) problem in distribution networks by considering the uncertainty of system load and renewable power generation. We formulate the distributed VVO problem as a non-cooperative Markov game. Specifically, we split the distribution network into multiple regions that are controlled by a group of networked agents. We consider the statuses/ratios of switchable capacitor banks (SCBs), the tap position of voltage regulators (VRs), and the reactive power of inverter-based distributed generators (DGs) as the control variables. The objective is to minimize the total system loss while maintaining bus voltages within a normal operating range. To solve the problem, a multi-agent trust region policy optimization (MATRPO) based approach is applied to learn a set of decentralized policies. Simulation results on a modified IEEE-34 test system show that the proposed approach can successfully learn a set of high-quality decentralized policies for the agents to collaboratively reduce the system loss and regulate the bus voltages. The simulation results also demonstrate the superiority of the proposed approach over independent learning and the MADDPG method.

This paper develops an intelligent control method based on reinforcement learning techniques for unknown nonlinear continuous-time systems in an adversarial environment. The developed method can automatically learn the optimal control input for the system and also predict the worst case adversarial input that one adversary can bring into. Besides, we assume that the agent can only observe partial information of the environment during the learning process. Therefore, a neural network-based observer is developed to adaptively reconstruct the hidden states and dynamics. Then, theoretical analysis is provided to show the stability of the developed intelligent control and the accuracy of the established observer. This method has been applied on a torsional pendulum system and the results demonstrate the effectiveness of the designed approach.

Maintaining the orderliness and efficiency of pedestrian flow through an architectural area is critical for the evacuation process. Especially, clogs and jams are easily triggered in width-changing areas. In this article, we consider pedestrian movement in heterogeneous corridors and design an optimal feedback control to regulate pedestrian flow. Flow characteristics are first studied based on microscopic social-force simulations. A Gaussian process describes the relationship between flow variables with the observation data. The macroscopic model for flow in heterogeneous corridors is developed. To avoid jams, discharges among these corridors are balanced with the narrowest corridor as the primary concern. At the equilibrium, a continuous-time nonlinear control system is formulated, and the adaptive dynamic programming learns the optimal feedback controller. Policy iteration (PI) and neural networks are combined together, and the convergence of neural-network-based PI is demonstrated by analyzing its equivalence to the Gauss–Newton method. Batch normalization is introduced to stabilize the learning process. Simulated experiments demonstrate that the control design can effectively regulate pedestrian flow for both macroscopic and microscopic models.Note to Practitioners—The development of video-processing techniques provides a powerful tool to detect human behavior in real time. In crowd events, the pedestrian movement must be regulated; otherwise, it is easy to fall into the faster-is-slower effect. It is especially important for evacuation routes with different widths. In this article, the optimal feedback control is studied to regulate pedestrian flow in heterogeneous corridors. It takes flow densities as state and produces commands that are composed of entrance influx and free-flow velocities. These commands can be executed with the support of speakers, displays, or the recently developed interactive robots. To avoid congestion, discharges of different corridors are balanced, and the system is optimally stabilized at equilibrium. Based on our work, engineers are able to design pedestrian flow control and achieve optimal evacuation in arbitrary heterogeneous corridors.

This article is concerned with the problem of fixed-time (FXT) and preassigned-time (PAT) synchronization for discontinuous dynamic networks by improving FXT stability and developing simple control schemes. First, some more relaxed conditions for FXT stability are established and several more accurate estimates for the settling time (ST) are obtained by means of some special functions. Based on the improved FXT stability, FXT synchronization for discontinuous networks is discussed by designing a simple controller without a linear feedback term. Besides, the PAT synchronization is also explored by developing several nontrivial control protocols with finite control gains, where the synchronized time can be prespecified according to actual needs and is irrelevant with any initial value and any parameter. Finally, the improved FXT stability and the synchronization for complex networks are confirmed by two numerical examples.

Partial domain adaptation (PDA) aims to transfer knowledge from a label-rich source domain to a label-scarce target domain based on an assumption that the source label space subsumes the target label space. The major challenge is to promote positive transfer in the shared label space and circumvent negative transfer caused by the large mismatch across different label spaces. In this article, we propose a dual alignment approach for PDA (DAPDA), including three components: 1) a feature extractor extracts source and target features by the Siamese network; 2) a reweighting network produces "hard" labels, class-level weights for source features and "soft" labels, instance-level weights for target features; 3) a dual alignment network aligns intra domain and interdomain distributions. Specifically, the intra domain alignment aims to minimize the intraclass variances to enhance the intraclass compactness in both domains, and interdomain alignment attempts to reduce the discrepancies across domains by domain-wise and class-wise adaptations. The negative transfer can be alleviated by down-weighting source features with nonshared labels. The positive transfer can be enhanced by upweighting source features with shared labels. The adaptation can be achieved by minimizing the discrepancies based on class-weighted source data with hard labels and instance-weighed target data with soft labels. The effectiveness of our method has been demonstrated by outperforming state-of-the-art PDA methods on several benchmark datasets.

This article addresses the team-triggered fixed-time consensus problems for a class of double-integrator agents subject to uncertain disturbance. Compared with the finite-time results, the convergence time of the fixed-time results is independent of the initial conditions. Furthermore, a novel team-triggered control (TTC) strategy is presented. This control strategy incorporates the event-triggered control (ETC) and self-triggered control (STC). The ETC and STC are proposed to achieve the fixed-time consensus of second-order multiagent systems (MASs), and no Zeno behavior occurs. The TTC scheme, derived by combining the ETC scheme and the STC scheme, is able to relax the requirement of continuous communication and thus lowering the energy consumption of communication while ensuring the performance of the system. The effectiveness of the proposed algorithms is validated by numerical simulations.

Multiagent reinforcement learning (MARL) has recently attracted considerable attention from both academics and practitioners. Core issues, e.g., the curse of dimensionality due to the exponential growth of agent interactions and nonstationary environments due to simultaneous learning, hinder the large-scale proliferation of MARL. These problems deteriorate with an increased number of agents. To address these challenges, we propose an adversarial collaborative learning method in a mixed cooperative-competitive environment, exploiting friend-or-foe Q-learning and mean-field theory. We first treat neighbors of agent i as two coalitions ( i 's friend and opponent coalition, respectively), and convert the Markov game into a two-player zero-sum game with an extended action set. By exploiting mean-field theory, this new game simplifies the interactions as those between a single agent and the mean effects of friends and opponents. A neural network is employed to learn the optimal mean effects of these two coalitions, which are trained via adversarial max and min steps. In the max step, with fixed policies of opponents, we optimize the friends' mean action to maximize their rewards. In the min step, the mean action of opponents is trained to minimize the friends' rewards when the policies of friends are frozen. These two steps are proved to converge to a Nash equilibrium. Then, another neural network is applied to learn the best response of each agent toward the mean effects. Finally, the adversarial max and min steps can jointly optimize the two networks. Experiments on two platforms demonstrate the learning effectiveness and strength of our approach, especially with many agents.

In the absence of assuming that all agents know the system matrix of the external system, for heterogeneous linear multiagent systems (MASs), this article resolves the cooperative regulation problem in the presence of communication constraints. Based on the mild assumption on the digraph, two novel adaptive protocols are presented to solve the cooperative output regulation problem (ORP) via state feedback and measurement output feedback subject to snatchy and asynchronous information exchange between agents, unknown time-varying delays, and probable information losses. Finally, some simulations are offered to demonstrate the validity of our results about the problem.

This article addresses two kinds of formation tracking problems, namely: 1) the practical formation tracking (PFT) problem and 2) the zero-error formation tracking (ZEFT) problem for multiple Euler-Lagrange systems with input disturbances and unknown models. In these problems, the bounded input constraint, which can be possibly caused by actuator saturation and power limitations, is taken into consideration. Then, the two classes of model-independent distributed control approaches, in which the prior information (i.e., the structures and features) of the system model is not used, are proposed correspondingly. Based on the nonsmooth analysis and Lyapunov stability theory, several novel criteria for achieving PFT and ZEFT of multiple Euler-Lagrange systems are derived. Finally, numerical simulations and comparisons are presented to verify the validity and effectiveness of the proposed control approaches.

In this article, a novel distributed policy iteration algorithm is established for infinite horizon optimal control problems of continuous-time nonlinear systems. In each iteration of the developed distributed policy iteration algorithm, only one controller's control law is updated and the other controllers' control laws remain unchanged. The main contribution of the present algorithm is to improve the iterative control law one by one, instead of updating all the control laws in each iteration of the traditional policy iteration algorithms, which effectively releases the computational burden in each iteration. The properties of distributed policy iteration algorithm for continuous-time nonlinear systems are analyzed. The admissibility of the present methods has also been analyzed. Monotonicity, convergence, and optimality have been discussed, which show that the iterative value function is nonincreasingly convergent to the solution of the Hamilton-Jacobi-Bellman equation. Finally, numerical simulations are conducted to illustrate the effectiveness of the proposed method.

In this article, we propose a novel decentralized event-triggered control (ETC) scheme for a class of continuous-time nonlinear systems with matched interconnections. The present interconnected systems differ from most of the existing interconnected plants in that their equilibrium points are no longer assumed to be zero. Initially, we establish a theorem to indicate that the decentralized ETC law for the overall system can be represented by an array of optimal ETC laws for nominal subsystems. Then, to obtain these optimal ETC laws, we develop a reinforcement learning (RL)-based method to solve the Hamilton-Jacobi-Bellman equations arising in the discounted-cost optimal ETC problems of the nominal subsystems. Meanwhile, we only use critic networks to implement the RL-based approach and tune the critic network weight vectors by using the gradient descent method and the concurrent learning technique together. With the proposed weight vectors tuning rule, we are able to not only relax the persistence of the excitation condition but also ensure the critic network weight vectors to be uniformly ultimately bounded. Moreover, by utilizing the Lyapunov method, we prove that the obtained decentralized ETC law can force the entire system to be stable in the sense of uniform ultimate boundedness. Finally, we validate the proposed decentralized ETC strategy through simulations of the nonlinear-interconnected systems derived from two inverted pendulums connected via a spring.

This article considers an event-driven$H_{\infty }$control problem of continuous-time nonlinear systems with asymmetric input constraints. Initially, the$H_{\infty }$-constrained control problem is converted into a two-person zero-sum game with the discounted nonquadratic cost function. Then, we present the event-driven Hamilton–Jacobi–Isaacs equation (HJIE) associated with the two-person zero-sum game. Meanwhile, we develop a novel event-triggering condition making Zeno behavior excluded. The present event-triggering condition differs from the existing literature in that it can make the triggering threshold non-negative without the requirement of properly selecting the prescribed level of disturbance attenuation. After that, under the framework of adaptive critic learning, we use a single critic network to solve the event-driven HJIE and tune its weight parameters by using historical and instantaneous state data simultaneously. Based on the Lyapunov approach, we demonstrate that the uniform ultimate boundedness of all the signals in the closed-loop system is guaranteed. Finally, simulations of a nonlinear plant are presented to validate the developed event-driven$H_{\infty }$control strategy.

In this paper, we study the constrained optimization problem of a class of uncertain nonlinear interconnected systems. First, we prove that the solution of the constrained optimization problem can be obtained through solving an array of optimal control problems of constrained auxiliary subsystems. Then, under the framework of approximate dynamic programming, we present a simultaneous policy iteration (SPI) algorithm to solve the Hamilton-Jacobi-Bellman equations corresponding to the constrained auxiliary subsystems. By building an equivalence relationship, we demonstrate the convergence of the SPI algorithm. Meanwhile, we implement the SPI algorithm via an actor-critic structure, where actor networks are used to approximate optimal control policies and critic networks are applied to estimate optimal value functions. By using the least squares method and the Monte Carlo integration technique together, we are able to determine the weight vectors of actor and critic networks. Finally, we validate the developed control method through the simulation of a nonlinear interconnected plant.

This article investigates the problem of small fault detection (sFD) for discrete-time nonlinear systems with uncertain dynamics. The faults are considered to be "small" in the sense that the system trajectories in the faulty mode always remain close to those in the normal mode, and the magnitude of fault can be smaller than that of the system's uncertain dynamics. A novel adaptive dynamics learning-based sFD framework is proposed. Specifically, an adaptive dynamics learning approach using radial basis function neural networks (RBF NNs) is first developed to achieve locally accurate identification of the system uncertain dynamics, where the obtained knowledge can be stored and represented in terms of constant RBF NNs. Based on this, a novel residual system is designed by incorporating a newmechanism of absolute measurement of system dynamics changes induced by small faults. An adaptive threshold is then developed for real-time sFD decision making. Rigorous analysis is performed to derive the detectability condition and the analytical upper bound for sFD time. Simulation studies, including an application to a three-tank benchmark engineering system, are conducted to demonstrate the effectiveness and advantages of the proposed approach.

This article investigates the global exponential stability and stabilization problems for a class of Takagi-Sugeno (T-S) fuzzy competitive neural networks (NNs). In the considered model, we introduce the T-S fuzzy rule to describe the parametric switching causing by complexity and the vagueness in practical environment. Besides, the effects of reaction diffusions and distributed delays, which inherently exist in circuits of NNs, are also taken into consideration. By using the Lyapunov functional theory and Green formula, several stability criteria in terms of \mathbb p-norm are established for the uncompensated fuzzy competitive NNs. Moreover, by designing a fuzzy intermittent controller, the corresponding stabilizability criteria in terms of \mathbb p-norm are derived. We also carry out some discussions and comparisons to further show the less conservativeness and wide applicability of the main theorems. Finally, several examples are presented to verify the obtained results.

Sensor-based human activity recognition (HAR) aims to recognize a human's physical actions by using sensors attached to different body parts. As a user-specific application, HAR often suffers poor generalization from training on an individual to testing on another individual, or from one body part to another body part. To tackle this cross-domain HAR problem, this article proposes a domain adaptation (DA) method called local domain adaptation (LDA), whose core is to align cluster-to-cluster distributions between the source domain and the target domain. On the one hand, LDA differs from existing set-to-set alignment by reducing the distribution discrepancy at a finer granularity. On the other hand, LDA is superior to the class-to-class alignment because it can provide more accurate soft labels for the target domain. Specifically, LDA contains three main steps: 1) groups the activity class into several high-level abstract clusters; 2) maps the original data of each cluster in both domains into the same low-dimension subspace to align the intracluster data distribution; 3) predicts the class labels for target domain in the low-dimension subspace. Experimental results on two public HAR benchmark datasets show that LDA outperforms state-of-the-art DA methods for the cross-domain HAR.

In this article, we investigate the coordinated topology attacks in smart grid, which combine a physical topology attack and a cyber-topology attack. The physical attack first trips a transmission line. In order to deceive the control center, the attacker masks the outage signal of the tripped line in the cyber layer and then creates a fake outage signal for another transmission line. The goal of coordinated topology attacks is to overload a critical line (different from the physical tripped line and the fake outage line) by misleading the control center into making improper dispatch. In order to determine the attack strategy, we propose a deep-reinforcement-learning-based method to identify the physical tripped line and the fake outage line. Besides, in order to block the outage signal of the tripped line and create the fake outage signal with limited attack resources, we propose a deep-reinforcement-learning-based approach to determine the minimal attack resources. Numerical simulations verify the effectiveness of the proposed method.

Object instance segmentation in traffic scenes is an important research topic. For training instance segmentation models, synthetic data can potentially complement real data, alleviating manual effort on annotating real images. However, the data distribution discrepancy between synthetic data and real data hampers the wide applications of synthetic data. In light of that, we propose a virtual-real interaction method for object instance segmentation. This method works over synthetic images with accurate annotations and real images without any labels. The virtual-real interaction guides the model to learn useful information from synthetic data while keeping consistent with real data. We first analyze the data distribution discrepancy from a probabilistic perspective, and divide it into image-level and instance-level discrepancies. Then, we design two components to align these discrepancies, i.e., global-level alignment and local-level alignment. Furthermore, a consistency alignment component is proposed to encourage the consistency between the global-level and the local-level alignment components. We evaluate the proposed approach on the real Cityscapes dataset by adapting from virtual SYNTHIA, Virtual KITTI, and VIPER datasets. The experimental results demonstrate that it achieves significantly better performance than state-of-the-art methods.

“Happy New Year!” As you open this January 2021 issue of IEEE Transactions onNeuralNetworks andLearningSystems(IEEE TNNLS), I hope everyone enjoyed a healthy and happy holiday season!

“6 – 72 – 27,846”: Six volumes, seventy-two issues, twenty-seven thousand and eight hundred forty-six pages: How time flies! As I mark down these numbers, this December 2021 issue of IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS) also marks the last issue for me as the Editor-in-Chief.

This paper studies the consensus in fractional-order multiagent systems over directed graph via sampled-data control method. A distributed control protocol using the sampled position and velocity data is designed. By virtue of the Mittag-Leffler function, Laplace transform, and matrix theory, some necessary and sufficient conditions associated with the sampling period, the fractional order, the coupling strengths, and the network structure to obtain consensus of the systems are obtained. Then, some detailed discussions are presented about how to select the sampling period and how to design the coupling strengths to attain the consensus of the systems, respectively. Lastly, some numerical simulation results are illustrated to reflect the availability of the theoretical analysis.

This paper studies quasi-synchronization in networked heterogeneous harmonic oscillators. By introducing a leader, two distributed synchronization protocols are first proposed for heterogeneous networks by utilizing continuous real-time information and aperiodic sampled-data information. Then, the sufficient conditions on quasi-synchronization are established for heterogeneous networks coupled with nonidentical harmonic oscillators. It is found that each follower oscillator can converge to a bounded region of the leader by adopting either a continuous-time protocol or sampled-data protocol. The upper bound of the region is solved for networked heterogeneous harmonic oscillators. Finally, an electrical network is provided to illustrate the applicability of the theoretical results, and two examples are provided to illustrate the effectiveness of the sufficient criteria.

This article investigates the privacy-preserving average consensus problem for the general continuous-time multiagent network systems (MANSs) with time delay via the event-triggered communication scheme. In order to avoid disclosing the initial states of the network agents and at the same time achieve the agents' consensus for MANSs with time delay, a novel consensus control algorithm based on a new privacy-preserving approach is proposed. The new privacy-preserving approach is that we construct an output mask to make agents' internal states indiscernible by others, which is different from the existing privacy-preserving methods adding random noises to the update law of agents' states. Compared with the existing privacy-preserving methods, our approach makes all agents in MANSs exactly converge to the average value of initial states instead of its mean square value. Based on the proposed algorithm, we carry out the detailed theoretical consensus analysis of the network agents, from which, it is shown that the upper bound of the communication time delay between neighbors' agents can be estimated approximately. Moreover, the Zeno-behavior of event-triggered time sequences for each agent is excluded. Finally, two simulation examples are performed to demonstrate the effectiveness of our theoretical results.

In this paper, an optimal risk-averse configuration framework for hybrid energy storage system (HESS) is proposed in planning utility-scale photovoltaic (PV) plants with conditional generative adversarial networks (CGANs)-based PV scenario generation. Other than most existing economy-based methods, we focus on frequency-based method to size a battery-supercapacitor HESS for mitigating the PV generation fluctuations in two time scales. We explore for the first time the potential of CGANs to generate sufficient PV scenarios through learning for experimental data preparation. For satisfying the fluctuation restrictions strictly, a data-driven frequency-based batteries optimization is developed, combining the flexible low-pass filter with wavelet package transform to guide the behaviors of both battery and supercapacitor for every individual scenario. Moreover, to hedge against risk exposure imposed by uncertain PV resource, we employ conditional value-at-risk to perform the optimal risk-averse configuration to meet the fluctuation mitigating requirements and minimize the expected configuration as well as the risk. Case studies are provided to verify the reasonableness and the efficiency of the proposed method.

As a newly emerging multivariate statistical process monitoring method, non-negative matrix factorization (NMF) and its variants avoid the positive and negative cancellations between the extracted features because of the purely additive combination of non-negative components. Thus, they make a good match with this reality that the negative values of both observations and decomposed components are physically meaningless in many kinds of industrial processes. However, these methods are effective only for linearly separable problems and are not suitable for dealing with nonlinear process monitoring. In this article, the kernel-based method is integrated into the projective NMF (KPNMF) to improve the accuracy of fault detection, and the appropriate multiplicative update method is proven to be convergent. Furthermore, inspired by the false nearest neighbors (FNNs) method, a new dimensionality reduction approach (KPNMF-FNN) is presented to further reduce the original variables for determining the smallest dimension regression vector needed. Compared with the traditional methods, the proposed approach can greatly reduce the time and storage space required on the premise of maintaining a high fault detection rate and low false alarm rate. The experimental results on the Tennessee Eastman benchmark process and the pumping unit system show that the proposed algorithms have excellent performance and can effectively detect faults under the circumstances of retaining only the top 60% of the original variables.

By using the terminal sliding-mode control (TSMC) and the pinning control methods, the fully distributed finite-time consensus problems are investigated for second-order multiagent systems (MASs) and multiquadcopter systems (MQSs) with directed topology. For the second-order MASs, a pinning control scheme is designed by analyzing the outdegree and indegree of nodes, and a TSMC protocol with the local information is proposed to achieve the finite-time consensus. Then, as an application of the MASs, the model of MQSs is constructed and its finite-time attitude consensus is discussed. Finally, the effectiveness of the proposed method is validated by two numerical examples.

In this paper, we introduce a deep spiking delayed feedback reservoir (DFR) model to combine DFR with spiking neuros: DFRs are a new type of recurrent neural networks (RNNs) that are able to capture the temporal correlations in time series while spiking neurons are energy-efficient and biologically plausible neurons models. The introduced deep spiking DFR model is energy-efficient and has the capability of analyzing time series signals. The corresponding field programmable gate arrays (FPGA)-based hardware implementation of such deep spiking DFR model is introduced and the underlying energy-efficiency and recourse utilization are evaluated. Various spike encoding schemes are explored and the optimal spike encoding scheme to analyze the time series has been identified. To be specific, we evaluate the performance of the introduced model using the spectrum occupancy time series data in MIMO-OFDM based cognitive radio (CR) in dynamic spectrum sharing (DSS) networks. In a MIMO-OFDM DSS system, available spectrum is very scarce and efficient utilization of spectrum is very essential. To improve the spectrum efficiency, the first step is to identify the frequency bands that are not utilized by the existing users so that a secondary user (SU) can use them for transmission. Due to the channel correlation as well as users' activities, there is a significant temporal correlation in the spectrum occupancy behavior of the frequency bands in different time slots. The introduced deep spiking DFR model is used to capture the temporal correlation of the spectrum occupancy time series and predict the idle/busy subcarriers in future time slots for potential spectrum access. Evaluation results suggest that our introduced model achieves higher area under curve (AUC) in the receiver operating characteristic (ROC) curve compared with the traditional energy detection-based strategies and the learning-based support vector machines (SVMs).

In this paper, inspired from the nuclei in brain, we propose a nucleus neural network (NNN) and corresponding connecting architecture learning method. In a nucleus, the neurons are not assigned as regular layers, i.e., a neuron may connect to any neurons in the nucleus and the connections are self-organized according to data distribution. This type of architecture gets rid of layer limitation and makes full use of processing capability of each neuron. It is crucial to assign connections between all the neuron pairs. To address the time demanding of objectives in traditional architecture learning methods, we propose an efficient architecture learning model for the nucleus based on the principle that more relevant input and output neuron pair deserves higher connecting density. The new objective measures the information flow through the network architecture without involvement of weights and biases which greatly reduces the computational complexity. We find that this novel architecture is robust to irrelevant components in test data. So we reconstruct a new dataset based on the MNIST dataset where the types of digital backgrounds in training and test sets are different. The new dataset is a great challenge for learners because training data and test data are not only independent but also follow different distributions. Experiments demonstrate that NNN achieves significant improvement over architectures with regular layers on the reconstructed dataset.

The existing unsupervised domain adaptation (UDA) methods require not only labeled source samples but also a large number of unlabeled target samples for domain adaptation. Collecting these target samples is generally time-consuming, which hinders the rapid deployment of these UDA methods in new domains. Besides, most of these UDA methods are developed for image classification. In this paper, we address a new problem called one-shot unsupervised domain adaptation for object detection, where only one unlabeled target sample is available. To the best of our knowledge, this is the first time this problem is investigated. To solve this problem, a one-shot feature alignment (OSFA) algorithm is proposed to align the low-level features of the source domain and the target domain. Specifically, the domain shift is reduced by aligning the average activation of the feature maps in the lower layer of CNN. The proposed OSFA is evaluated under two scenarios: adapting from clear weather to foggy weather; adapting from synthetic images to real-world images. Experimental results show that the proposed OSFA can significantly improve the object detection performance in target domain compared to the baseline model without domain adaptation.

Unsupervised anomaly detection for time series is of great importance for various applications, such as Web monitoring, medical monitoring, and device fault diagnosis. Time series anomaly detection (TSAD) aims to find the observations that most different from others in a sequence of observations. With the development of deep learning, deep-autoencoder-based methods achieve state-of-the-art performance. These methods are usually able to find single anomaly points but fail to detect the anomaly segment and the change point. To tackle this problem, this paper proposes a novel TSAD method, which consists of a bidirectional LSTM (BiLSTM) autoencoder and a subsequent Gaussian segmentation model. BiLSTM encodes a time series in a predictive format from both positive and negative time directions, then outputs the latent feature vectors and restructured errors. After that, the latent features are used to find anomaly segments by the Gaussian segment model; the restructured errors are used to find change points and extreme single anomaly by a scoring function. In this way, our method can find all three kinds of anomaly points. Experiments on two real-world datasets demonstrate the effectiveness of the proposed method.

This paper develops an event-triggered multi-agent control method based on adaptive dynamic programming (ADP) techniques. Different from the traditional ADP-based multi-agent control with fixed sampling period, our method designs an adaptive controller only based on the efficiently reduced samples. The sampling instants are decided by an adaptive triggering condition to guarantee the stability of the event-triggered learning process. The theoretical analysis of the proposed method is also provided in this paper. It is proved that the designed event-triggered ADP controller can make all the agents synchronize to the leader's dynamics with reduced sampled data, and also reach Nash equilibrium at the same time. Therefore, the proposed method can save the computational resources in the learning process. Finally, the simulation results verify the theoretical analysis and also demonstrate the performance of the developed method.

Cyber-physical systems (CPS) are characterized by deep and complex intertwining among cyber components and physical components. Due to the fast increase in system complexities, the operations of CPS involve sensing, processing and storage of massive amount of data. This nature of “big data” imposes fundamental challenges on the design and management of CPS in multiple aspects such as performance, energy efficiency, security, privacy, reliability, sustainability, fault tolerance, scalability and flexibility. Tackling these challenges necessitates innovative big data techniques for handling massive data in CPS. The articles in this special section include a few selected state-of-the-art research results on the topic of big data sensing, processing and storage for CPS, and stimulates a broad range of researchers to participate in the interdisciplinary CPS research in the future. This special issue has received a significant number of submissions while only a small portion of them are selected for publications. The selected papers showcase how interesting data analytics techniques can be leveraged to optimize different metrics in CPS, such as timing, efficiency, schedulability, power, reliability, and security, etc.

The graph embedding (GE) methods have been widely applied for dimensionality reduction of hyperspectral imagery (HSI). However, a major challenge of GE is how to choose the proper neighbors for graph construction and explore the spatial information of HSI data. In this paper, we proposed an unsupervised dimensionality reduction algorithm called spatial-spectral manifold reconstruction preserving embedding (SSMRPE) for HSI classification. At first, a weighted mean filter (WMF) is employed to preprocess the image, which aims to reduce the influence of background noise. According to the spatial consistency property of HSI, SSMRPE utilizes a new spatial-spectral combined distance (SSCD) to fuse the spatial structure and spectral information for selecting effective spatial-spectral neighbors of HSI pixels. Then, it explores the spatial relationship between each point and its neighbors to adjust the reconstruction weights to improve the efficiency of manifold reconstruction. As a result, the proposed method can extract the discriminant features and subsequently improve the classification performance of HSI. The experimental results on the PaviaU and Salinas hyperspectral data sets indicate that SSMRPE can achieve better classification results in comparison with some state-of-the-art methods.

Pedestrian regulation can prevent crowd accidents and improve crowd safety in densely populated areas. Recent studies use mobile robots to regulate pedestrian flows for desired collective motion through the effect of passive human-robot interaction (HRI). This paper formulates a robot motion planning problem for the optimization of two merging pedestrian flows moving through a bottleneck exit. To address the challenge of feature representation of complex human motion dynamics under the effect of HRI, we propose using a deep neural network to model the mapping from the image input of pedestrian environments to the output of robot motion decisions. The robot motion planner is trained end-to-end using a deep reinforcement learning algorithm, which avoids hand-crafted feature detection and extraction, thus improving the learning capability for complex dynamic problems. Our proposed approach is validated in simulated experiments, and its performance is evaluated. The results demonstrate that the robot is able to find optimal motion decisions that maximize the pedestrian outflow in different flow conditions, and the pedestrian-accumulated outflow increases significantly compared to cases without robot regulation and with random robot motion.

This article investigates the global stabilization problem of Takagi-Sugeno fuzzy memristor-based neural networks with reaction-diffusion terms and distributed time-varying delays. By using the Green formula and proposing fuzzy feedback controllers, several algebraic criteria dependent on the diffusion coefficients are established to guarantee the global exponential stability of the addressed networks. Moreover, a simpler stability criterion is obtained by designing an adaptive fuzzy controller. The results derived in this article are generalized and include some existing ones as special cases. Finally, the validity of the theoretical results is verified by two examples.

Various real-world applications with interval uncertainty, such as the path planning of mobile robot, layout of radio frequency identification readers and solar desalination, can be formulated as an interval multiobjective optimization problem (IMOOP), which is usually transformed into one or a series of certain problems to solve by using evolutionary algorithms. However, a definite characteristic among them is that only a single optimization task can be catched up at a time. Inspired by the multifactorial evolutionary algorithm (MFEA), a novel interval MFEA (IMFEA) is proposed to solve IMOOPs simultaneously using a single population of evolving individuals. In the proposed method, the potential interdependency across related problems can be explored in the unified genotype space, and multitasks of multiobjective interval optimization problems are solved at once by promoting knowledge transfer for the greater synergistic search to improve the convergence speed and the quality of the optimal solution set. Specifically, an interval crowding distance based on shape evaluation is calculated to evaluate the interval solutions more comprehensively. In addition, an interval dominance relationship based on the evolutionary state of the population is designed to obtain the interval confidence level, which considers the difference of average convergence levels and the relative size of the potential possibility between individuals. Correspondingly, the strict transitivity proof of the presented dominance relationship is given. The efficacy of the associated evolutionary algorithm is validated on a series of benchmark test functions, as well as a real-world case of robot path planning with many terrains that provides insight into the performance of the method in the face of IMOOPs.

This paper investigates the synchronization problem of Takagi-Sugeno fuzzy memristive neural networks (FMNNs) with mixed delays, in which the bounded distributed and unbounded discrete time-varying delays are involved. Then, under the nonsmooth analysis and Lyapunov stability theory, several easily verified algebraic criteria are established to guarantee the global synchronization of FMNNs via a designed fuzzy feedback controller. Moreover, to show the superiority of the theoretical results, several discussions and comparisons with existing work are provided, indicating that derived results in this paper are general and include several existing ones as special cases. Finally, two numerical examples and two applications in psuedorandom number generation and image encryption are presented to show the validity and practicability of the theoretical results.

A key challenge in hyperspectral image (HSI) classification is how to effectively utilize the spectral and spatial information of limited labeled training samples in the data set. In this article, a new spatial-spectral combined classification method, termed local linear spatial-spectral probabilistic distribution (LSPD), has been proposed on the basis of local geometric structure and spatial consistency of HSI. LSPD extracts discriminating spatial-spectral information from limited labeled training samples and their spatial-spectral neighbors. Then, it constructs a multiclass probability map by exploiting the local linear representation and spatial information of HSI. Finally, the spatial-spectral weighted reconstruction has been performed on the probability map, and the class of test sample can be predicted by the maximum value of LSPD. LSPD not only exploits spectral information to discover more intrinsic properties of the labeled training data but also utilizes the spatial relationship between samples to effectively improve discriminating power for classification. Experimental results on the Indian Pines, PaviaU, and HoustonU hyperspectral data sets demonstrate that the proposed LSPD method possesses better classification performance by comparing with some state-of-the-art classifiers.

This article investigates distributed coordinated tracking problems of networked heterogeneous systems. Based on asynchronous sampling information, distributed sampled-data protocols are employed to realize leader-following synchronization and containment tracking in networked heterogeneous systems. In asynchronous sampled-data protocols, each node has different sampling instants with other nodes and only samples itself information at its own sampling instants. By utilizing the input-delay approach and Lyapunove-Krasovskii functional approach, some sufficient conditions for guaranteeing the coordinated tracking are presented. First, quasi-synchronization criteria are obtained for networked heterogeneous oscillator systems with a dynamic leader over the directed graph. Second, in the presence of multiple heterogeneous leaders for networked heterogeneous systems, sufficient conditions of quasi-containment tracking are derived. In a word, all followers can converge into a bounded level of convex hull spanned by the leader(s). The upper bounds of tracking errors are estimated for both quasi-synchronization and quasi-containment tracking. Finally, two numerical examples are given to verify the theoretical results.

Controller synthesis is a challenging problem in cooperative adaptive cruise control (CACC). Especially the requirement of string stability makes it even harder to choose appropriate control parameters. This paper applies a time-domain definition to string stability and converts the problem to the H∞control of a time-delay system. Based on the proposed control structure, the H∞norm and stability criteria of CACC are satisfied by a set of constraints in terms of a Lyapunov-Krasovskii functional candidate. These constraints are further reduced to linear matrix inequalities so that feasible solutions can be easily and efficiently computed. Simulations on an identified model validate the performance of our method in both frequency and time domains.

In data mining, large differences between multi-class distributions regarded as class imbalance issues have been known to hinder the classification performance. Unfortunately, existing sampling methods have shown their deficiencies such as causing the problems of over-generation and over-lapping by oversampling techniques, or the excessive loss of significant information by undersampling techniques. This paper presents three proposed sampling approaches for imbalanced learning: the first one is the entropy-based oversampling (EOS) approach; the second one is the entropy-based undersampling (EUS) approach; the third one is the entropy-based hybrid sampling (EHS) approach combined by both oversampling and undersampling approaches. These three approaches are based on a new class imbalance metric, termed entropy-based imbalance degree (EID), considering the differences of information contents between classes instead of traditional imbalance-ratio. Specifically, to balance a data set after evaluating the information influence degree of each instance, EOS generates new instances around difficult-to-learn instances and only remains the informative ones. EUS removes easy-to-learn instances. While EHS can do both simultaneously. Finally, we use all the generated and remaining instances to train several classifiers. Extensive experiments over synthetic and real-world data sets demonstrate the effectiveness of our approaches.

“Happy New Year!” As you open this January issue of the IEEE Transactions on Neural Networks and Learning Systems (TNNLS), I hope everyone enjoyed a great holiday season and is excited for the new year of 2020.

This paper analyzes the consensus problem in heterogenous nonlinear multiagent systems. The multiagent systems not only have nonidentical nonlinear dynamics for all agents, but also have different network topologies for position and velocity interactions. An asynchronous sampled-data control without any input delays is first proposed, the information of each agent is only sampled at its own sampling instants and need not be sampled at other sampling instants. Then, quasi-consensus in heterogenous multiagent systems is proved by Lyapunov stability theory. When asynchronous sampled-data control has nonuniform input delays, sufficient conditions for quasi-consensus in heterogenous multiagent systems are further obtained. The upper bound of quasi-consensus errors is estimated. Finally, numerical simulations are provided to verify the effectiveness of theoretical results.

In this paper, a deep reinforcement learning-based robust control strategy for quadrotor helicopters is proposed. The quadrotor is controlled by a learned neural network which directly maps the system states to control commands in an end-to-end style. The learning algorithm is developed based on the deterministic policy gradient algorithm. By introducing an integral compensator to the actor-critic structure, the tracking accuracy and robustness have been greatly enhanced. Moreover, a two-phase learning protocol which includes both offline and online learning phase is proposed for practical implementation. An offline policy is first learned based on a simplified quadrotor model. Then, the policy is online optimized in actual flight. The proposed approach is evaluated in the flight simulator. The results demonstrate that the offline learned policy is highly robust to model errors and external disturbances. It also shows that the online learning could significantly improve the control performance.

This paper presents a novel decentralized control strategy for a class of uncertain nonlinear large-scale systems with mismatched interconnections. First, it is shown that the decentralized controller for the overall system can be represented by an array of optimal control policies of auxiliary subsystems. Then, within the framework of adaptive dynamic programming, a simultaneous policy iteration (SPI) algorithm is developed to solve the Hamilton-Jacobi-Bellman equations associated with auxiliary subsystem optimal control policies. The convergence of the SPI algorithm is guaranteed by an equivalence relationship. To implement the present SPI algorithm, actor and critic neural networks are applied to approximate the optimal control policies and the optimal value functions, respectively. Meanwhile, both the least squares method and the Monte Carlo integration technique are employed to derive the unknown weight parameters. Furthermore, by using Lyapunov's direct method, the overall system with the obtained decentralized controller is proved to be asymptotically stable. Finally, the effectiveness of the proposed decentralized control scheme is illustrated via simulations for nonlinear plants and unstable power systems.

In this paper, we study the event-triggered robust stabilization problem of nonlinear systems subject to mismatched perturbations and input constraints. First, with the introduction of an infinite-horizon cost function for the auxiliary system, we transform the robust stabilization problem into a constrained optimal control problem. Then, we prove that the solution of the event-triggered Hamilton-Jacobi-Bellman (ETHJB) equation, which arises in the constrained optimal control problem, guarantees original system states to be uniformly ultimately bounded (UUB). To solve the ETHJB equation, we present a single network adaptive critic design (SN-ACD). The critic network used in the SN-ACD is tuned through the gradient descent method. By using Lyapunov method, we demonstrate that all the signals in the closed-loop auxiliary system are UUB. Finally, we provide two examples, including the pendulum system, to validate the proposed event-triggered control strategy.

In this paper, we develop a decentralized event-triggered control (ETC) strategy for a class of nonlinear systems with uncertain interconnections. To begin with, we show that the decentralized ETC policy for the whole system can be represented by a group of optimal ETC laws of auxiliary subsystems. Then, under the framework of adaptive critic learning, we construct the critic networks to solve the event-triggered Hamilton-Jacobi-Bellman equations related to these optimal ETC laws. The weight vectors used in the critic networks are updated by using the gradient descent approach and the experience replay (ER) technique together. With the aid of the ER technique, we can conquer the difficulty arising in the persistence of excitation condition. Meanwhile, by using classic Lyapunov approaches, we prove that the estimated weight vectors used in the critic networks are uniformly ultimately bounded. Moreover, we demonstrate that the obtained decentralized ETC can force the overall system to be asymptotically stable. Finally, we present an interconnected nonlinear plant to validate the proposed decentralized ETC scheme.

Blockchain technique, with the novelties of decentralization, smart contract, security and cooperative autonomy, is expected to play great effects on promoting the development of energy local networks (ELNs). This paper presents an automated demand response (ADR) framework for decentralized scheduling and secure peer-to-peer (P2P) trading among energy storage systems in ELNs. Different from most existing works that trade electricity over long distances and through complex meshes, this proposed work performs decentralized and automated demand response through energy sharing of P2P executors. We explore for the first time the benefits of a promising blockchain to conduct the overall ADR framework and increase the P2P trading security. To achieve decentralized scheduling without relying on a central entity, a price-incentive noncooperative game theoretic model is introduced to produce equilibrium solutions for energy storage systems. Moreover, we develop a schedulable ability evaluation system to match trading pairs involving buying and selling nodes. On this basis, a state-machine-driven smart contract mechanism is built to realize P2P trading without reliance on a trusted third party. To illustrate the implementation details of the ADR method, a distributed algorithm is designed. Case studies are provided to verify the effectiveness of the proposed method.

This paper develops a new online learning consensus control scheme for multiagent discrete-time systems by goal representation heuristic dynamic programming (GrHDP) techniques. The agents in the whole system are interacted with each other through a communication graph structure. Therefore, each agent can only receive the information from itself and its neighbors. Our goal is to design the GrHDP method to achieve consensus control which makes all the agents track the desired dynamics and simultaneously makes the performance indices reach Nash equilibrium. The new local internal reinforcement signals and local performance indices are provided for each agent and the corresponding distributed control laws are designed. Then, GrHDP algorithm is developed to solve the multiagent consensus control problem with the proof of convergence. It is shown that the designed local internal reinforcement signals are bounded signals and the local performance indices can monotonically converge to their optimal values. Moreover, the desired distributed control laws can also achieve optimal. Two simulation studies, including one with four agents and another with ten agents, are applied to validate the theoretical analysis and also demonstrate the effectiveness of the proposed method.

For systems that can only be locally stabilized, control laws and their effective regions are both important. In this paper, invariant policy iteration is proposed to solve the optimal control of discrete-time systems. At each iteration, a given policy is evaluated in its invariantly admissible region, and a new policy and a new region are updated for the next iteration. Theoretical analysis shows the method is regionally convergent to the optimal value and the optimal policy. Combined with sum-of-squares polynomials, the method is able to achieve the near-optimal control of a class of discrete-time systems. An invariant adaptive dynamic programming algorithm is developed to extend the method to scenarios where system dynamics is not available. Online data are utilized to learn the near-optimal policy and the invariantly admissible region. Simulated experiments verify the effectiveness of our method.

We propose a novel cooperative spectrum sensing (CSS) framework for cognitive radio networks based on imbalanced learning techniques, which aims to resolve the skewed category distribution problems of signal data. For a radio channel shared by primary users (PUs) and secondary users (SUs), the signal data composed of energy vectors, in which each energy level is estimated by SU, can be used to detect the channel availability via a classifier. However, due to the nature of this application, the existing category-imbalance problem hinders the detection performance since the trained classifier has a better effect on the dominated category. To enhance the performance, sampling (e.g., oversampling, under-sampling, and combination) algorithms are employed to balance the training data set based on the imbalance degree metric of imbalance-ratio. The balanced training set then can be used to train classifiers with initial parameters, and the validation set can be utilized to tune as well as evaluate the classifiers. In the testing phase, the actual desired performance on unseen signal data can be determined based on the testing set, i.e., whether the channel is available or not. The performance of each sampling algorithm is measured in terms of receiver operating characteristic (ROC) curve and area under the ROC curve (AUC). The simulation results demonstrate the effectiveness of our proposed framework compared to traditional CSS methods.

Adversarial domain adaptation has achieved some success in learning transferable feature representations and reducing distribution discrepancy between source and target domains. However, existing approaches mainly focus on alignment of global source and target distributions without considering complex structures in categories underlying different distributions, resulting in domain confusion and the mix of distinguishable structures. In this paper, we propose an adversarial domain adaptation via category transfer (ADACT) approach for unsupervised domain adaptation (UDA). ADACT first captures multi-category information through training source and target feature generators as well as a label predictor. Secondly, it uses multi-category domain critic networks to category-wisely estimate Wasserstein distances across domains. Then it learns category-invariant feature representations by finely-grained matching different data distributions with the estimated Wasserstein distances. The adaptation can be achieved by the standard back-propagation training approach with this two-step iteration. The effectiveness of ADACT is demonstrated since it outperforms several state-of-the-art UDA methods on common domain adaptation datasets.

Object instance segmentation can achieve preferable results, powered with sufficient labeled training data. However, it is time-consuming for manually labeling, leading to the lack of large-scale diversified datasets with accurate instance segmentation annotations. Exploiting the synthetic data is a very promising solution except for domain distribution mismatch between synthetic dataset and real dataset. In this paper, we propose a synthetic-to-real domain adaptation method for object instance segmentation. At first, this approach is trained to generate object detection and segmentation using annotated data from synthetic dataset. Then, a feature adaptation module (FAM) is applied to reduce data distribution mismatch between synthetic dataset and real dataset. The FAM performs domain adaptation from three different aspects: global-level base feature adaptation module, local-level instance feature adaptation module, and subtle-level mask feature adaptation module. It is implemented based on novel discriminator networks with adversarial learning. The three modules of FAM have positive effects on improving the performance when adapting from synthetic to real scenes. We evaluate the proposed approach on Cityscapes dataset by adapting from Virtual KITTI and SYNTHIA datasets. The results show that it achieves a significantly better performance over the state-of-the-art methods.

Fast and efficient evacuation of pedestrians from an enclosed area is a difficult but crucial issue in modern society. In this paper, the optimization of evacuation from a building is studied. A graph is adopted to describe the building layout with nodes representing areas and edges representing connections. The dynamics of the evacuation process in the graph is formulated by a nonlinear discrete-time model at a macroscopic level. To find the optimal evacuation plan, a consecutive differential dynamic programming is developed. It inherits the differential dynamic programming property that solves the value and optimal policy locally. Additionally, it consecutively executes actions for multiple steps in the trajectory, which is beneficial to reduce computational burden and lower optimization difficulty. Simulations on a four-storey building layout demonstrates our method is efficient and suitable for on-site evacuation plan making.

Dynamic spectrum access (DSA) is regarded as an effective and efficient technology to share radio spectrum among different networks. As a secondary user (SU), a DSA device will face two critical problems: 1) avoiding causing harmful interference to primary users (PUs) and 2) conducting effective interference coordination with other SUs. These two problems become even more challenging for a distributed DSA network where there is no centralized controllers for SUs. In this paper, we investigate communication strategies of a distributive DSA network under the presence of spectrum sensing errors. To be specific, we apply the powerful machine learning tool, deep reinforcement learning (DRL), for SUs to learn “appropriate” spectrum access strategies in a distributed fashion assuming NO knowledge of the underlying system statistics. Furthermore, a special type of recurrent neural network, called the reservoir computing (RC), is utilized to realize DRL by taking advantage of the underlying temporal correlation of the DSA network. Using the introduced machine learning-based strategy, SUs could make spectrum access decisions distributedly relying only on their own current and past spectrum sensing outcomes. Through extensive experiments, our results suggest that the RC-based spectrum access strategy can help the SU to significantly reduce the chances of collision with PUs and other SUs. We also show that our scheme outperforms the myopic method which assumes the knowledge of system statistics, and converges faster than the Q-learning method when the number of channels is large.

Although sentiment analysis on microblog posts has been studied in depth, sentiment analysis of posts is still challenging because of the limited contextual information that they normally contain. In microblog environments, emoticons are frequently used and they have clear emotional meanings. They are important emotional signals for microblog sentimental analysis. Existing studies typically use emoticons as noisy sentiment labels or similar sentiment indicators to effectively train classifier but overlook their emotional potentiality. We address this issue by constructing an emotional space as a feature representation matrix and projecting emoticons and words into the emotional space based on the semantic composition. To improve the performance of sentimental analysis, we propose a new emotion-semantic-enhanced convolutional neural network (ECNN) model. ECNN can use emoticon embedding as an emotional space projection operator. By projecting emoticons and words into an emoticon space, it can help identify subjectivity, polarity, and emotion in microblog environments. It is more capable of capturing emotion semantic than other models, so it can improve the sentiment analysis performance. The experimental results show that this model consistently outperforms other models on the dataset of several sentiment tasks. This paper provides insights on the design of ECNN for sentimental analysis in other natural language processing tasks.

Multi-modal tasks like visual question answering (VQA) are an important step towards human-level artificial intelligence. In general, the input of the VQA task consists of an image and a related question. In order to correctly answer the question, a model needs to extract and integrate useful information from both the image and the question. In this paper, we propose a model named AnswerNet to tackle this task. In the proposed model, discriminative features are extracted from both the image and the question. Specifically, high-level image features are extracted by the state-of-the-art convolutional neural network, i.e., Deep Residual Net. For question features, the semantic representations of the question and the term frequencies of the distinct words are captured by long short-term memory network and bag-of-words model, respectively. Then, a hierarchical fusion network is proposed to effectively fuse the image features with the question features. Experimental results on three large-scale datasets, VQA, COCO-QA, and VQA2, demonstrate the effectiveness of the proposed AnswerNet.

This paper develops a novel event-triggered robust control strategy for continuous-time nonlinear systems with unknown dynamics. To begin with, the event-triggered robust nonlinear control problem is transformed into an event-triggered nonlinear optimal control problem by introducing an infinite-horizon integral cost for the nominal system. Then, a recurrent neural network (RNN) and adaptive critic designs (ACDs) are employed to solve the derived event-triggered nonlinear optimal control problem. The RNN is applied to reconstruct the system dynamics based on collected system data. After acquiring the knowledge of system dynamics, a unique critic network is proposed to obtain the approximate solution of the event-triggered Hamilton-Jacobi-Bellman equation within the framework of ACDs. The critic network is updated by using simultaneously historical and instantaneous state data. An advantage of the present critic network update law is that it can relax the persistence of excitation condition. Meanwhile, under a newly developed event-triggering condition, the proposed critic network tuning rule not only guarantees the critic network weights to converge to optimums but also ensures nominal system states to be uniformly ultimately bounded. Moreover, by using Lyapunov method, it is proved that the derived optimal event-triggered control (ETC) guarantees uniform ultimate boundedness of all the signals in the original system. Finally, a nonlinear oscillator and an unstable power system are provided to validate the developed robust ETC scheme.

This paper presents a functional model predictive control (MPC) approach based on an adaptive dynamic programming (ADP) algorithm with the abilities of handling control constraints and disturbances for the optimal control of nonlinear discrete-time systems. In the proposed ADP-based nonlinear MPC (NMPC) structure, a neural-network-based identification is established first to reconstruct the unknown system dynamics. Then, the actor-critic scheme is adopted with a critic network to estimate the index performance function and an action network to approximate the optimal control input. Meanwhile, as the MPC strategy can effectively determine the current control by solving a finite horizon open-loop optimal control problem, in the proposed algorithm, the infinite horizon is decomposed into a series of finite horizons to obtain the optimal control. In each finite horizon, the finite ADP algorithm solves the optimal control problem subject to the terminal constraint, the control constraint, and the disturbance. The uniform ultimate boundedness of the closed-loop system is verified by the Lyapunov approach. Finally, the ADP-based NMPC is conducted on two different cases and the simulation results demonstrate the quick response and strong robustness of the proposed method.

This paper is concerned with a class of optimal output consensus control problems for discrete linear multiagent systems with the partially observable system state. Since the optimal control policy depends on the full system state which is not accessible for a partially observable system, traditionally, distributed observers are employed to recover the system state. However, in many situations, the accurate model of a real-world dynamical system might be difficult to obtain, which makes the observer design infeasible. Furthermore, the optimal consensus control policy cannot be analytically solved without system functions. To overcome these challenges, we propose a data-driven adaptive dynamic programming approach that does not require the complete system inner state. The key idea is to use the input and output sequence as an equivalent representation of the underlying state. Based on this representation, an adaptive dynamic programming algorithm is developed to generate the optimal control policy. For the implementation of this algorithm, we design a neural network-based actor-critic structure to approximate the local performance indices and the control polices. Two numerical simulations are used to demonstrate the effectiveness of our method.

Finding the overall Pareto optimal front while addressing the effect of an increasing number of objectives has become an essential and challenging issue for multiobjective optimization in real-world applications. Preference information provided by a decision maker can guide the search for preferred regions of the Pareto front and accelerate the convergence of the population. In this paper, a new variant of the Pareto dominance relation, called preference angle and reference information-based dominance, is proposed to create a stricter partial order among nondominated solutions. In the proposed method, the Euclidean distance and angle information between candidate solutions and reference points are calculated to evaluate the degree of convergence and population diversity, respectively. In addition, an adaptive threshold is designed to adjust the judgment condition of ar-dominance using an iterative process in a prespecified interval. The proposed algorithm increases the convergence speed of the population and reduces the number of solutions in the nonpreferred region. Comparative evaluation experiments are presented with respect to two performance metrics for a variety of benchmark test problems and real-world aluminum electrolytic production cases. The results demonstrate that the proposed approach is effective for highly complex, multiobjective optimization problems when compared with five state-of-the-art evolutionary algorithms.

Gear safety factors are conducive to assessing the reliability of vehicle transmission gears. Due to the insufficiency and high coupling of existing gear data, the prediction of gear safety factors has long been a challenging issue in vehicle transmission industry through learning meaningful representations of high-dimensional gear parameters. This paper presents a framework to find a high-quality solution that improves prediction accuracy of gear safety factors. In the framework, to cope with the insufficiency of gear data, a generative model based on generative adversarial networks is established and proven to generate acceptable data with Adam optimizer. Then, eigen-error principal component analysis is proposed to extract features of gear parameters by reconstructing error function with eigenvectors and eigenvalues. Finally, particle swarm optimization and back propagation are applied to predict safety factors with these extracted features. Experimental results on real-world gear data of vehicle transmissions have validated the effectiveness of our proposed framework.

Fault diagnosis of a thermal system under varying operating conditions is of great importance for the safe and reliable operation of a power plant involved in peak shaving. However, it is a difficult task due to the lack of sufficient labeled data under some operating conditions. In practical applications, the model built on the labeled data under one operating condition will be extended to such operating conditions. Data distribution discrepancy can be triggered by variation of operating conditions and may degenerate the performance of the model. Considering the fact that data distributions are different but related under different operating conditions, this paper proposes a hierarchical deep domain adaptation (HDDA) approach to transfer a classifier trained on labeled data under one loading condition to identify faults with unlabeled data under another loading condition. In HDDA, a hierarchical structure is developed to reveal the effective information for final diagnosis by layerwisely capturing representative features. HDDA learns domain-invariant and discriminative features with the hierarchical structure by reducing distribution discrepancy and preserving discriminative information hidden in raw process data. For practical applications, the Taguchi method is used to obtain the optimized model parameters. Experimental results and comprehensive comparison analysis demonstrate its superiority.

Networked control systems (NCSs) provide many benefits, such as higher control accuracy and better robustness with the successively increasing computational complexity and communication burden. This results in the traditional adaptive dynamic programming control method having difficulty meeting the real-time requirements of industrial systems. In this paper, a novel event-triggered globalized dual heuristic programming method is proposed to reduce the required samples while guaranteeing the stability of the system. In the proposed method, the NCSs can communicate and update the control law only when the designed event-triggered condition is violated. Furthermore, the Elman neural network, which is a dynamic feedback network with a memory function is implemented to reconstruct the state variables as an approximator, and it depends only on the input and output data. To obtain fewer event-triggered times, two optimization methods, i.e., the unscented Kalman filter and the multiobjective quantum particle swarm optimization, are used to optimize the initial weights of the networks and the positive constant in the event-triggered condition, respectively. The simulation results on industrial system of aluminum electrolysis production are included to verify the performance of the controller.

This paper addresses the formation control problem for a group of mechanical systems with nonlinear uncertain dynamics under the virtual leader-following framework. New cooperative deterministic learning-based adaptive formation control algorithms are proposed. Specifically, the virtual leader dynamics is constructed as a linear system subject to unknown bounded inputs, so as to produce more diverse reference signals for formation tracking control. A cooperative discontinuous nonlinear estimation protocol is first proposed to estimate the leader's state information. Based on this, a cooperative deterministic learning formation control protocol is developed using artificial neural networks, such that formation tracking control and locally-accurate nonlinear identification with learning knowledge consensus can be achieved simultaneously. Finally, by utilizing the learned knowledge represented by constant neural networks, an experience-based distributed control protocol is further proposed to enable position-swappable formation control. Numerical simulations using a group of autonomous underwater vehicles have been conducted to demonstrate the effectiveness and usefulness of the proposed results.

This paper proposes a novel data-based optimal control algorithm for continuous stirred tank reactor (CSTR) system based on adaptive dynamic programming (ADP). To overcome the challenge of establishing an accurate mathematical model for the CSTR system, neural networks are employed to reconstruct the dynamics of the CSTR system using the production data of the system. A new nonquadratic form performance index function is provided, where the control input is constrained in order not to exceed the bound of the actuator. Then, the operational optimal control problem of CSTR is formulated. Furthermore, an iterative ADP (IADP) algorithm is developed to obtain the optimal tracking controller for the CSTR system with control constraints. In particular, the convergence analysis of the IADP algorithm is developed. The proposed IADP algorithm is implemented via the dual heuristic dynamic programming structure. Finally, the proposed approach is applied to the real CSTR system to verify the effectiveness and performance.

Fault detection and diagnosis in the pumping unit is a challenging industrial problem for the system that exhibits nonlinearity, coupled parameters, and time-varying noise. This paper proposes a novel combined unscented Kalman filter (UKF) and radial basis function (RBF) method based on an adaptive noise factor for fault diagnosis in the pumping unit. First, to reduce computation and complexity of the diagnosis model, the Fourier descriptor method based on an approximate polygon is presented to extract the features of the indicator diagram. RBF neural network is adopted to establish the fault diagnosis model based on indicator diagram data and production data. In particular, UKF is used to train the weights (wm,l), the center (cm), and the width (bm) of the RBF model. Furthermore, the adaptive noise factor method is proposed to address the adaptive filtering issue in the fault diagnosis model. The proposed method is applied to the pumping unit system, and experimental results show the effectiveness and favorable recognition rate in classifying multiple faults.

As you open this January issue of the IEEE Transactions on Neural Networks and Learning Systems (TNNLS), I hope everyone enjoyed a great holiday season and is excited for the new year of 2019. I am very delighted and honored to report several key metrics of IEEE TNNLS to the community.

This paper presents a parameterized batch reinforcement learning algorithm for near-optimal longitudinal control of autonomous land vehicles (ALVs). The proposed approach uses an actor-critic architecture, where parameterized feature vectors based on kernels are learned from collected samples for approximating the value functions and policies. One difference between the parameterized batch actor-critic (PBAC) algorithm and previous actor-critic learning approaches is that the critic and actor in PBAC share the same linear features, which has been theoretically proved to be a beneficial property for the convergence of actor-critic learning approaches. In order to obtain better learning efficiency, least-squares-based batch updating rules are designed for the critic and actor, respectively. Based on the PBAC learning algorithm, a data-driven longitudinal control method is presented for ALVs to obtain near-optimal control policies which adaptively tune the fuel/brake control signals to track different speeds. A multiobjective reward function is designed so that both tracking precision and driving smoothness are considered. Extensive experiments were conducted on a real ALV platform while driving on flat, slippery, sloping, and bumpy roads. The experimental results illustrate the superiority of the PBAC-based self-learning controller over conventional longitudinal control methods such as proportional-integral (PI) control and learning-based PI control.

Due to the fast-is-slower phenomenon in emergency escape, it is desirable to regulate pedestrian flows at the exit or a bottleneck. We propose a new robot-assisted pedestrian regulation and study passive human-robot interaction (HRI). A learning-based motion control approach is presented for a robot to efficiently interact with pedestrians for desirable collective motion. We first formulate the problem into an optimal control framework using the pedestrian dynamics description based on existing social force models with embedded HRI forces. To solve the defined optimal control problem, we propose an adaptive dynamic programming (ADP) approach to provide adjustable motion parameters of the robot to efficiently interact with pedestrians so that the regulated pedestrian flow tracks a desired velocity. The ADP control process only uses observed flow information rather than the models of pedestrians, and the ADP method provides feedback control with online learning and control capability. Simulation results demonstrate that the proposed approach can regulate pedestrian flows to desirable speeds by online learning.

This paper develops an optimal control scheme for continuous-time unknown nonlinear systems using the event-triggering mechanism. Different from designing controllers using the time-triggering mechanism, the event-triggered controller is updated only when the system state deviates more than a certain threshold from a prescribed value. To obtain the event-triggered optimal controller, we develop an identifier-critic architecture under the framework of reinforcement learning. The identifier network, composed of a feedforward neural network (FNN), aims to derive the knowledge of unknown system dynamics, and the critic network, constituted of an FNN, intends to derive the event-triggered optimal controller. The identifier network is tuned via the combination of a standard back-propagation algorithm and an e-modification method, and the critic network is updated using a modification of the gradient descent method. By introducing an additional stability term to update the critic network, the initial admissible control is no longer required. Meanwhile, by using historical and instantaneous state data together, the persistence of excitation condition is relaxed. A stability analysis of the closed-loop system is provided based on the Lyapunov method. The effectiveness of the proposed designs is illustrated through simulations of a nonlinear example and a single link robot arm system.

This paper investigates the channel access problem of cognitive radio networks. In the cognitive radio network, communication channels are assigned to primary users with priority while secondary users are able to detect the spectrum holes and switch among the channels for data transmission opportunities. The channel access problem of this kind of system can be formulated as a non-cooperative game. However, in prior works, the secondary users are usually assumed to be able to switch to any channel instantaneously, which is not possible in reality because the channel switching will incur transmission delays. In this paper, we formulate the channel access problem as a non-cooperative game where each channel can be used by only one user at a time. Moreover, considering the transmission delays, we limit the channel switching distance of the secondary users to a certain scope. In this case, the optimal channel access policy of each secondary user will depend on the long-term behaviors of primary users as well as the actions of other secondary users. For this non-cooperative game, we propose a multiagent Q-learning algorithm which requires neither the prior knowledge of channel dynamics nor the negotiations among players. Simulation examples are provided to demonstrate the effectiveness of the algorithm.

A large number of datasets in various applications are imbalanced in which majority samples dominate minority samples. The skewed distribution poses a difficulty for existing learning approaches. Oversampling techniques address this concern by replicating original samples or adding new synthetic samples of minority class. Even with success, they raise the problems of over-generation and overlapping. In this paper, we propose an entropy difference-based oversampling approach (EDOS) for imbalanced learning using a novel metric, termed entropy difference (ED). First, given a dataset, EDOS measures the imbalance degree between the majority and the minority with ED. Second, EDOS creates synthetic minority samples. For each synthetic sample, EDOS evaluates its retention capability and remains the informative sample. Third, original and qualified synthetic samples are combined to train the classifiers. In the experiments, we demonstrate the effectiveness of the proposed EDOS method on several UCI datasets.

A smart home with battery energy storage can take part in the demand response program. With proper energy management, consumers can purchase more energy at off-peak hours than at on-peak hours, which can reduce the electricity costs and help to balance the electricity demand and supply. However, it is hard to determine an optimal energy management strategy because of the uncertainty of the electricity consumption and the real-time electricity price. In this paper, a deep reinforcement learning based approach has been proposed to solve this residential energy management problem. The proposed approach does not require any knowledge about the uncertainty and can directly learn the optimal energy management strategy based on reinforcement learning. Simulation results demonstrate the effectiveness of the proposed approach.

Sparse autoencoder is an unsupervised feature extractor and has been widely used in the machine learning and data mining community. However, a sparse hyperparameter has to be determined to balance the trade-off between the reconstruction error and the sparsity of sparse autoencoder. Traditional sparse hyperparameter determination method is time-consuming, especially when the dataset is large. In this paper, we derive a generative model for sparse autoencoder. Based on this model, we derive a formulation to determine the sparse hyperparameter effectively and efficiently. The relationship between the sparse hyperparameter and the average activation of sparse autoencoder hidden units is also presented in this paper. Experimental results and comparative studies over numerous datasets demonstrate the effectiveness of our method to determine the sparse hyperparameter.

In this paper, we propose an end-to-end modular reinforcement learning architecture for a navigation task in complex dynamic environments with rapidly moving obstacles. In this architecture, the main task is divided into two subtasks: local obstacle avoidance and global navigation. For obstacle avoidance, we develop a two-stream Q-network, which processes spatial and temporal information separately and generates action values. The global navigation subtask is resolved by a conventional Q-network framework. An online learning network and an action scheduler are introduced to first combine two pretrained policies, and then continue exploring and optimizing until a stable policy is obtained. The two-stream Q-network obtains better performance than the conventional deep Q-learning approach in the obstacle avoidance subtask. Experiments on the main task demonstrate that the proposed architecture can efficiently avoid moving obstacles and complete the navigation task at a high success rate. The modular architecture enables parallel training and also demonstrates good generalization capability in different environments.

This paper presents a data-based finite-horizon optimal control approach for discrete-time nonlinear affine systems. The iterative adaptive dynamic programming (ADP) is used to approximately solve Hamilton-Jacobi-Bellman equation by minimizing the cost function in finite time. The idea is implemented with the heuristic dynamic programming (HDP) involved the model network, which makes the iterative control at the first step can be obtained without the system function, meanwhile the action network is used to obtain the approximate optimal control law and the critic network is utilized for approximating the optimal cost function. The convergence of the iterative ADP algorithm and the stability of the weight estimation errors based on the HDP structure are intensively analyzed. Finally, two simulation examples are provided to demonstrate the theoretical results and show the performance of the proposed method.

In this paper, a new concept of formation learning control is introduced to the field of formation control of multiple autonomous underwater vehicles (AUVs), which specifies a joint objective of distributed formation tracking control and learning/identification of nonlinear uncertain AUV dynamics. A novel two-layer distributed formation learning control scheme is proposed, which consists of an upper-layer distributed adaptive observer and a lower-layer decentralized deterministic learning controller. This new formation learning control scheme advances existing techniques in three important ways: 1) the multi-AUV system under consideration has heterogeneous nonlinear uncertain dynamics; 2) the formation learning control protocol can be designed and implemented by each local AUV agent in a fully distributed fashion without using any global information; and 3) in addition to the formation control performance, the distributed control protocol is also capable of accurately identifying the AUVs' heterogeneous nonlinear uncertain dynamics and utilizing experiences to improve formation control performance. Extensive simulations have been conducted to demonstrate the effectiveness of the proposed results.

In this paper, we present a new model-free globalized dual heuristic dynamic programming (GDHP) approach for the discrete-time nonlinear zero-sum game problems. First, the online learning algorithm is proposed based on the GDHP method to solve the Hamilton-Jacobi-Isaacs equation associated with optimal regulation control problem. By setting backward one step of the definition of performance index, the requirement of system dynamics, or an identifier is relaxed in the proposed method. Then, three neural networks are established to approximate the optimal saddle point feedback control law, the disturbance law, and the performance index, respectively. The explicit updating rules for these three neural networks are provided based on the data generated during the online learning along the system trajectories. The stability analysis in terms of the neural network approximation errors is discussed based on the Lyapunov approach. Finally, two simulation examples are provided to show the effectiveness of the proposed method.

In this paper, for achieving the discounted optimal feedback stabilization of a nonlinear overhead crane system, we establish an intelligent control strategy to obtain the solution of the corresponding Hamilton-Jacobi-Bellman equation. Specifically, neural networks are employed to serve as a necessary component to the control system, which exhibits strong online learning ability. A novel updating rule compared to the traditional adaptive critic algorithms is developed, which eliminates the requirement of the initial stabilizing controller and brings in unique advantages to the adaptive critic control design. Stability analysis of the closed-loop system based on the well-known Lyapunov approach and experimental simulation considering the nonlinear overhead dynamics with different case studies are performed to verify the effectiveness of the present control method both in theory and applications.

Improvements in the production and energy consumption of the aluminum electrolysis process (AEP) directly depend on the operating parameters of the electrolytic cell. To balance the conflicting goals of efficiency and productivity with reduced energy consumption and emissions, AEP operating parameter optimization is formulated as a constrained multiobjective optimization problem with competing objectives of current efficiency and cell voltage. Then, the improved multiobjective quantum-behaved particle swarm optimization (IMQPSO) algorithm is proposed. The application of an adaptive opposition-based learning strategy and a piecewise Gauss mutation operator can increase the diversity of the population and enhance the global search ability of the IMQPSO. To expand the creativity of the particles, two iterative methods of the mean best position with weighting and the attractor position are redesigned. Experimental analyses are conducted for the benchmark problems and a real case to verify the effectiveness of the proposed method.

This paper develops a new dimensionality reduction method, named Biomimetic Uncorrelated Locality Discriminant Projection (BULDP), for face recognition. It is based on unsupervised discriminant projection and two human bionic characteristics: principle of homology continuity and principle of heterogeneous similarity. With these two human bionic characteristics, we propose a novel adjacency coefficient representation, which does not only capture the category information between different samples, but also reflects the continuity between similar samples and the similarity between different samples. By applying this new adjacency coefficient into the unsupervised discriminant projection, it can be shown that we can transform the original data space into an uncorrelated discriminant subspace. A detailed solution of the proposed BULDP is given based on singular value decomposition. Moreover, we also develop a nonlinear version of our BULDP using kernel functions for nonlinear dimensionality reduction. The performance of the proposed algorithms is evaluated and compared with the state-of-the-art methods on four public benchmarks for face recognition. Experimental results show that the proposed BULDP method and its nonlinear version achieve much competitive recognition performance.

Clustering of data with high dimension and variable densities poses a remarkable challenge to the traditional density-based clustering methods. Recently, entropy, a numerical measure of the uncertainty of information, can be used to measure the border degree of samples in data space and also select significant features in feature set. It was used in our new framework based on the sparsity-density entropy (SDE) to cluster the data with high dimension and variable densities. First, SDE conducts high-quality sampling for multidimensional data and selects the representative features using sparsity score entropy (SSE). Second, the clustering results and noises are obtained adopting a new density-variable clustering method called density entropy (DE). DE automatically determines the border set based on the global minimum of border degrees and then adaptively performs cluster analysis for each local cluster based on the local minimum of border degrees. The effectiveness and efficiency of the proposed SDE framework are validated on synthetic and real data sets in comparison with several clustering algorithms. The results showed that the proposed SDE framework concurrently detected the noises and processed the data with high dimension and various densities.

Supervised object localization requires detailed image annotation, such as bounding box, to indicate the location of the object. However, labeling image with bounding box is labor-intensive. Besides, the labeling process may involve ambiguous decisions. Weakly supervised object localization only needs category annotation which is available in large amounts. Recently, many weakly supervised object localization methods, based on global pooling, have been proposed. However, these methods only localize part of the object. This paper proposes a deep convolutional neural network with spatial pyramid saliency map to localize the full extent of the object. The experimental result on Cub-200 dataset shows that our method outperforms the traditional ones.

In this paper, the distributed tracking problem for heterogeneous second-order multiagent systems with nonlinear dynamics is studied. Both the dynamics among the followers and the dynamics between the leader and each follower are heterogeneous. A sampled-data-based consensus protocol is proposed. In addition, the communication delays are considered. Based on Lyapunov stability theory and linear matrix inequality (LMI) method, sufficient conditions for quasi-consensus in heterogeneous leader-following multiagent systems are established. The results show that all the followers can track the leader within a bounded range even if in the presence of heterogeneity of dynamics and sampled-data information. The effectiveness of the theoretical results is verified by numerical simulations.

Energy flow control (EFC) of residential microgrids (RMGs) equipped with renewable generations (RGs) is an essential component for the future smart grid that contributes to enhance renewable energy consumption and reduce cost. Different from most existing papers that devote to offline EFC to against the uncertainties caused by RGs and local load demand in RMGs, this paper focuses on online EFC framework for achieving optimal operations of a RMG. This framework is based on an event-driven approach that maximizes RGs utilization and maintains supply-demand balance considering the schedulable ability of active loads and the uncertainties of RMG. An event-driven EFC architecture for RMG is developed, and the events analysis are presented. Based on this architecture, the state machine is adopted to trigger the execution of the online EFC. Furthermore, an online algorithm is designed for communal energy server platform to determine scheduling plans for active loads. Finally, the performance analysis of the online algorithm is evaluated. Simulation results illustrate the basic characteristics and the advantages of the proposed approach.

State space reconstruction is usually the first step of nonlinear time series analysis. Among many state space reconstruction approaches, the method of delays (MOD) has been a popular method in noise-free situations. Unfortunately, many real-world time series are usually noisy so that the reconstruction performance can be of low quality. In this paper, we propose an autoencoder-based approach that aims to reconstruct a high-quality state space from noisy nonlinear time series. We present the approach in detail and applied it to several typical nonlinear time series. The simulation results demonstrate that our method can generate better reconstructions than other popular approaches including MOD and principal component analysis (PCA).

In this paper, coordinated sliding mode control (SMC) and adaptive dynamic programming (ADP) strategy is proposed for near-space aerospace vehicle (NSASV) adaptive attitude tracking control. In this design, the NSASV attitude angle control is implemented as classical cascade control scheme with two control loops in the model. The outer one is a slow control loop for the attitude angle tracking, and the inner one is a fast control loop for the attitude angular rate tracking. Both of these two control loops are designed by using SMC, which can provide exact control performance near the operating point. To improve the control performance and robustness under parameter variations and external disturbances, ADP based supplementary control is introduced and incorporated into the inner fast control loop to provide adaptive compensation for the reference signal. Simulation study is carried out in Matlab/Simulink environment, and the results demonstrate that the proposed cooperative control could provide quite satisfied tracking performance in terms of overshoot and oscillation.

Using dictionary atoms to reconstruct input vectors is of great interest in spare representation. However, a key challenge is how to find a proper dictionary. In this paper, we introduce an active dictionary learning (ADL) method which incorporates active learning criteria to select atoms for dictionary construction with the consideration of both classification and reconstruction errors. Specifically, we apply a sparse representation based classification (SRC) method to calculate the learned dictionary and use the classification accuracy and the reconstruction error to evaluate the proposed dictionary learning method. In our experiments, we compare the performance of our proposed dictionary learning method with many other methods, including unsupervised dictionary learning and whole-training-data dictionary, on several UCI data sets and the Extended Yale B face data set. The superior performance demonstrates the effectiveness of the proposed method.

Understanding human behavior is crucial for planning evacuation strategies when an emergency occurs. The social force model, which is a successful quantitative model, has been widely used in investigating human behavior. In this paper, we propose a gradient descent based parameter optimization method to learn the parameters of the social force model from experimental data. Although the original social force model has achieved great success, it does not consider the fact that the response of humans to that happening in front of them is stronger than that happening behind them. In order to model the directional dependency of the interactive force, we propose a modified social force model. Experimental results demonstrate the effectiveness of the modified model.

A data-driven adaptive tracking control approach is proposed for a class of continuous-time nonlinear systems using a recent developed goal representation heuristic dynamic programming (GrHDP) architecture. The major focus of this paper is on designing a multivariable tracking scheme, including the filter-based action network (FAN) architecture, and the stability analysis in continuous-time fashion. In this design, the FAN is used to observe the system function, and then generates the corresponding control action together with the reference signals. The goal network will provide an internal reward signal adaptively based on the current system states and the control action. This internal reward signal is assigned as the input for the critic network, which approximates the cost function over time. We demonstrate its improved tracking performance in comparison with the existing heuristic dynamic programming (HDP) approach under the same parameter and environment settings. The simulation results of the multivariable tracking control on two examples have been presented to show that the proposed scheme can achieve better control in terms of learning speed and overall performance.

control problem is regarded as a two-player zero-sum game and the adaptive critic mechanism is used to achieve the minimax optimization under event-based environment. Then, based on an improved updating rule, the event-based optimal control law and the time-based worst-case disturbance law are obtained approximately by training a single critic neural network. The initial stabilizing control is no longer required during the implementation process of the new algorithm. Next, the closed-loop system is formulated as an impulsive model and its stability issue is handled by incorporating the improved learning criterion. The infamous Zeno behavior of the present event-based design is also avoided through theoretical analysis on the lower bound of the minimal intersample time. Finally, the applications to an aircraft dynamics and a robot arm plant are carried out to verify the efficient performance of the present novel design method.

Adaptive dynamic programming (ADP) and reinforcement learning are quite relevant to each other when performing intelligent optimization. They are both regarded as promising methods involving important components of evaluation and improvement, at the background of information technology, such as artificial intelligence, big data, and deep learning. Although great progresses have been achieved and surveyed when addressing nonlinear optimal control problems, the research on robustness of ADP-based control strategies under uncertain environment has not been fully summarized. Hence, this survey reviews the recent main results of adaptive-critic-based robust control design of continuous-time nonlinear systems. The ADP-based nonlinear optimal regulation is reviewed, followed by robust stabilization of nonlinear systems with matched uncertainties, guaranteed cost control design of unmatched plants, and decentralized stabilization of interconnected systems. Additionally, further comprehensive discussions are presented, including event-based robust control design, improvement of the critic learning rule, nonlinear H∞control design, and several notes on future perspectives. By applying the ADP-based optimal and robust control methods to a practical power system and an overhead crane plant, two typical examples are provided to verify the effectiveness of theoretical results. Overall, this survey is beneficial to promote the development of adaptive critic control methods with robustness guarantee and the construction of higher level intelligent systems.

This paper proposes a novel event-triggered adaptive dynamic programming (ADP) control method for nonlinear continuous-time system with unknown internal states. Comparing with the traditional ADP design with a fixed sample period, the event-triggered method samples the state and updates the controller only when it is necessary. Therefore, the computation cost and transmission load are reduced. Usually, the event-triggered method is based on the system entire state which is either infeasible or very difficult to obtain in practice applications. This paper integrates a neural-network-based observer to recover the system internal states from the measurable feedback. Both the proposed observer and the controller are aperiodically updated according to the designed triggering condition. Neural network techniques are applied to estimate the performance index and help calculate the control action. The stability analysis of the proposed method is also demonstrated by Lyapunov construct for both the continuous and jump dynamics. The simulation results verify the theoretical analysis and justify the efficiency of the proposed method.

Goal representation globalized dual heuristic dynamic programming (Gr-GDHP) method is proposed in this paper. A goal neural network is integrated into the traditional GDHP method providing an internal reinforcement signal and its derivatives to help the control and learning process. From the proposed architecture, it is shown that the obtained internal reinforcement signal and its derivatives can be able to adjust themselves online over time rather than a fixed or predefined function in literature. Furthermore, the obtained derivatives can directly contribute to the objective function of the critic network, whose learning process is thus simplified. Numerical simulation studies are applied to show the performance of the proposed Gr-GDHP method and compare the results with other existing adaptive dynamic programming designs. We also investigate this method on a ball-and-beam balancing system. The statistical simulation results are presented for both the Gr-GDHP and the GDHP methods to demonstrate the improved learning and controlling performance.

Recent studies on sequential attack schemes revealed new smart grid vulnerability that can be exploited by attacks on the network topology. Traditional power systems contingency analysis needs to be expanded to handle the complex risk of cyber-physical attacks. To analyze the transmission grid vulnerability under sequential topology attacks, this paper proposes a Q-learning-based approach to identify critical attack sequences with consideration of physical system behaviors. A realistic power flow cascading outage model is used to simulate the system behavior, where attacker can use the Q-learning to improve the damage of sequential topology attack toward system failures with the least attack efforts. Case studies based on three IEEE test systems have demonstrated the learning ability and effectiveness of Q-learning-based vulnerability analysis.

Data intensive analysis is the major challenge in smart cities because of the ubiquitous deployment of various kinds of sensors. The natural characteristic of geodistribution requires a new computing paradigm to offer location-awareness and latency-sensitive monitoring and intelligent control. Fog Computing that extends the computing to the edge of network, fits this need. In this paper, we introduce a hierarchical distributed Fog Computing architecture to support the integration of massive number of infrastructure components and services in future smart cities. To secure future communities, it is necessary to integrate intelligence in our Fog Computing architecture, e.g., to perform data representation and feature extraction, to identify anomalous and hazardous events, and to offer optimal responses and controls. We analyze case studies using a smart pipeline monitoring system based on fiber optic sensors and sequential learning algorithms to detect events threatening pipeline safety. A working prototype was constructed to experimentally evaluate event detection performance of the recognition of 12 distinct events. These experimental results demonstrate the feasibility of the system's city-wide implementation in the future.

This paper presents the design of a novel adaptive event-triggered control method based on the heuristic dynamic programming (HDP) technique for nonlinear discrete-time systems with unknown system dynamics. In the proposed method, the control law is only updated when the event-triggered condition is violated. Compared with the periodic updates in the traditional adaptive dynamic programming (ADP) control, the proposed method can reduce the computation and transmission cost. An actor-critic framework is used to learn the optimal event-triggered control law and the value function. Furthermore, a model network is designed to estimate the system state vector. The main contribution of this paper is to design a new trigger threshold for discrete-time systems. A detailed Lyapunov stability analysis shows that our proposed event-triggered controller can asymptotically stabilize the discrete-time systems. Finally, we test our method on two different discrete-time systems, and the simulation results are included.

In this paper, an event-triggered near optimal control structure is developed for nonlinear continuous-time systems with control constraints. Due to the saturating actuators, a nonquadratic cost function is introduced and the Hamilton-Jacobi-Bellman (HJB) equation for constrained nonlinear continuous-time systems is formulated. In order to solve the HJB equation, an actor-critic framework is presented. The critic network is used to approximate the cost function and the action network is used to estimate the optimal control law. In addition, in the proposed method, the control signal is transmitted in an aperiodic manner to reduce the computational and the transmission cost. Both the networks are only updated at the trigger instants decided by the event-triggered condition. Detailed Lyapunov analysis is provided to guarantee that the closed-loop event-triggered system is ultimately bounded. Three case studies are used to demonstrate the effectiveness of the proposed method.

This issue marks the first anniversary issue since I was honored to serve as the Editor-in-Chief (EiC) of the IEEE Transactions on Neural Networks and Learning Systems (TNNLS). I am happy to report that we had a very successful year and here are a few highlights that I would like to share with the community.•The latest impact factor of TNNLS is 4.854 according to the Journal Citation Reports. This marks a record high impact factor for our journal and places TNNLS as the number one scholarly publication in Computer Science (Hardware & Architecture), number three in Computer Science (Theory & Methods), and number ten in Electrical and Electronic Engineering journals.

In this paper, we propose a data-driven supplementary control approach with adaptive learning capability for air-breathing hypersonic vehicle tracking control based on action-dependent heuristic dynamic programming (ADHDP). The control action is generated by the combination of sliding mode control (SMC) and the ADHDP controller to track the desired velocity and the desired altitude. In particular, the ADHDP controller observes the differences between the actual velocity/altitude and the desired velocity/altitude, and then provides a supplementary control action accordingly. The ADHDP controller does not rely on the accurate mathematical model function and is data driven. Meanwhile, it is capable to adjust its parameters online over time under various working conditions, which is very suitable for hypersonic vehicle system with parameter uncertainties and disturbances. We verify the adaptive supplementary control approach versus the traditional SMC in the cruising flight, and provide three simulation studies to illustrate the improved performance with the proposed approach.

Feature representation is critical not only for pattern recognition tasks but also for reinforcement learning (RL) methods to solve learning control problems under uncertainties. In this paper, a manifold-based RL approach using the principle of locally linear reconstruction (LLR) is proposed for Markov decision processes with large or continuous state spaces. In the proposed approach, an LLR-based feature learning scheme is developed for value function approximation in RL, where a set of smooth feature vectors is generated by preserving the local approximation properties of neighboring points in the original state space. By using the proposed feature learning scheme, an LLR-based approximate policy iteration (API) algorithm is designed for learning control problems with large or continuous state spaces. The relationship between the value approximation error of a new data point and the estimated values of its nearest neighbors is analyzed. In order to compare different feature representation and learning approaches for RL, a comprehensive simulation and experimental study was conducted on three benchmark learning control problems. It is illustrated that under a wide range of parameter settings, the LLR-based API algorithm can obtain better learning control performance than the previous API methods with different feature representation schemes.

Feature selection aims to gain relevant features for improved classification performance and remove redundant features for reduced computational cost. How to balance these two factors is a problem especially when the categorical labels are costly to obtain. In this paper, we address this problem using semisupervised learning method and propose a max-relevance and min-redundancy criterion based on Pearson's correlation (RRPC) coefficient. This new method uses the incremental search technique to select optimal feature subsets. The new selected features have strong relevance to the labels in supervised manner, and avoid redundancy to the selected feature subsets under unsupervised constraints. Comparative studies are performed on binary data and multicategory data from benchmark data sets. The results show that the RRPC can achieve a good balance between relevance and redundancy in semisupervised feature selection. We also compare the RRPC with classic supervised feature selection criteria (such as mRMR and Fisher score), unsupervised feature selection criteria (such as Laplacian score), and semisupervised feature selection criteria (such as sSelect and locality sensitive). Experimental results demonstrate the effectiveness of our method.

In this paper, we construct an event-driven adaptive robust control approach for continuous-time uncertain nonlinear systems through a neural dynamic programming (NDP) strategy. Through system transformation and theoretical analysis, the robustness of the original uncertain system can be achieved by designing an event-driven optimal controller with respect to the nominal system under a suitable triggering condition. In addition, it is also observed that the event-driven controller has a certain degree of gain margin. Then, the NDP technique is employed to perform the main controller design task, followed by the uniform ultimate boundedness stability proof with the feedback action of the event-driven adaptive control law. The comparative effect of the present control strategy is also illustrated via two simulation examples. The established method provides a new avenue of combining adaptive dynamic programming-based self-learning control, event-triggered adaptive control, and robust control, to investigate the nonlinear adaptive robust feedback design under uncertain environment.

A novel event-triggered approach for a class of nonlinear continuous-time system is proposed in this paper to reduce the computation cost of the dual heuristic dynamic programming (DHP) algorithm. Two neural networks are included in our design. A critic network is used to estimate the partial derivatives of the cost function with respect to its inputs, and an action network is used to approximate the optimal control law. Instead of periodical sampling in the traditional DHP approach, under the event-triggered mechanism, both of the neural networks are only updated at the jump instants, and kept constant during the inter-event time. With the designed trigger threshold, the proposed DHP-based event-triggered approach can save computation time significantly while obtaining competitive control performance when comparing with those of the traditional DHP approach. Two simulation tests are presented to verify the theoretical results.

The modern electric power grid has become highly integrated in order to increase reliability of power transmission from the generating units to end consumers. This integrated nature and its upgrade toward an intelligent smart grid make the power grid vulnerable when facing cyber or physical attacks as well as intentional attacks. Therefore, determining the most vulnerable components (e.g., buses or generators) is critically important for power grid defense. In this paper, a new definition of load is proposed by taking power flow into consideration in comparison with the load definition based on degree or network connectivity. Unsupervised learning techniques (e.g., K-means algorithm and self-organizing map (SOM)) are introduced to cluster the nodes (i.e., buses) in IEEE-39 bus and IEEE-57 bus benchmarks. Then most vulnerable node in each cluster is determined based on their load information to form initial victim set. We use percentage of failure (PoF) to compare the performance of clustering based approach and traditional load based approach during cascading failure process. With the simulation results, the unsupervised learning (clustering based) approaches are more efficient in finding the most vulnerable nodes and our proposed definition of load is relatively useful in studying power grid security.

Understanding human mobility is important for the development of intelligent mobile service robots as it can provide prior knowledge and predictions of human distribution for robot-assisted activities. In this paper, we propose a probabilistic method to model human motion behaviors which is determined by both internal and external factors in an indoor environment. While the internal factors are represented by the individual preferences, aims and interests, the external factors are indicated by the stimulation of the environment. We model the randomness of human macro-level movement, e.g., the probability of visiting a specific place and staying time, under the Bayesian framework, considering the influence of both internal and external variables. We use two case studies in a shopping mall and in a college student dorm building to show the effectiveness of our proposed probabilistic human mobility model. Real surveillance camera data are used to validate the proposed model together with survey data in the case study of student dorm.

The threat of false data injection (FDI) attacks have raised wide interest in the research and development of smart grid security. This paper presents a comparative study on the utilization of supervised learning classifiers to detect direct and stealth FDI attacks in the smart grid. A detailed formulation of the problem for detection with classifiers is first described with proper assumptions and justifications. Three widely used supervised learning (SL) based classifiers are chosen to design corresponding FDI detectors. The performance are tested against false measurement data (direct FDI attack) and false state data (stealth FDI attack) on both balanced and imbalanced cases, with consideration of the influence of FDI resources and magnitudes. Simulations on IEEE 30-bus system have shown that the SL based detectors can effectively detect both direct and stealth FDI attacks, especially for the more severe attacks with large amount or magnitude of compromised measurements.

Adaptive dynamic programming (ADP) has been investigated for its new architectures, algorithms and applications for years. Recently, the goal representation (Gr) design has been demonstrated with promising results to improve ADP control performance from certain perspectives. This paper is focused on the theoretical analysis of the goal representation dual heuristic dynamic programming (GrDHP). Starting from the general formulation of the GrDHP design, we provide the iterative algorithm for this method. The corresponding convergence analysis is showed in terms of the internal reinforcement signal, the performance index, and their derivatives. Our analysis assumes that the system is controllable and stabilizable. Then, neural-network-based implementation of this method is presented. Simulation study validates the theoretical analysis of this paper and also shows the effectiveness of the GrDHP method.

Due to the faster-is-slower phenomenon in emergency escape, it is desirable to regulate pedestrian flow at the exit or a bottleneck. Modification of pedestrian facilities was previously studied to increase the efficiency and safety by the transportation community. We propose a robot-assisted pedestrian regulation scheme and study passive human-robot interaction (HRI), where the robot acts as a dynamic obstacle that interacts with pedestrians. Such a robot-assisted solution replaces expensive infrastructure modification with real-time reconfigurability. In the paper, we first formulate a robot-assisted flow optimization problem based on the social force models of pedestrian dynamics with embedded HRI forces. We then present an online learning algorithm based on adaptive dynamic programming (ADP) to generate motion control so that the robot can replan and adapt its motion to real-time pedestrian flows. The ADP control process uses observed flow information only but not the models of pedestrians, and provides feedback control with online learning and control capability. Simulation results demonstrate efficiency of the proposed method.

In this paper, we present a novel exponentially embedded families (EEF) based classification method, in which the probability density function (PDF) on raw data is estimated from the PDF on features. With the PDF construction, we show that class-specific features can be used in the proposed classification method, instead of a common feature subset for all classes as used in conventional approaches. We apply the proposed EEF classifier for text categorization as a case study and derive an optimal Bayesian classification rule with class-specific feature selection based on the Information Gain score. The promising performance on real-life data sets demonstrates the effectiveness of the proposed approach and indicates its wide potential applications.

Trajectory tracking control of wheeled mobile robots (WMRs) has been an important research topic in control theory and robotics. Although various tracking control methods with stability have been developed for WMRs, it is still difficult to design optimal or near-optimal tracking controller under uncertainties and disturbances. In this paper, a near-optimal tracking control method is presented for WMRs based on receding-horizon dual heuristic programming (RHDHP). In the proposed method, a backstepping kinematic controller is designed to generate desired velocity profiles and the receding horizon strategy is used to decompose the infinite-horizon optimal control problem into a series of finite-horizon optimal control problems. In each horizon, a closed-loop tracking control policy is successively updated using a class of approximate dynamic programming algorithms called finite-horizon dual heuristic programming (DHP). The convergence property of the proposed method is analyzed and it is shown that the tracking control system based on RHDHP is asymptotically stable by using the Lyapunov approach. Simulation results on three tracking control problems demonstrate that the proposed method has improved control performance when compared with conventional model predictive control (MPC) and DHP. It is also illustrated that the proposed method has lower computational burden than conventional MPC, which is very beneficial for real-time tracking control.

In this paper, a novel nonlinear learning controller called fuzzy-based goal representation adaptive dynamic programming (Fuzzy-GrADP) is proposed. In the proposed GrADP method, a goal representation network is introduced to generate an adaptive internal reinforcement signal to the critic network to help the controller provide a general mapping between the input and output actions. Moreover, in the proposed architecture, the action network in the GrADP is improved by using the fuzzy hyperbolic model, which combines the merits of the fuzzy model and the neural network model. Based on the back-propagation technique, the parameters in the membership functions and the fuzzy rules are all undergo training and online adapting. The proposed controller is tested on two numerical benchmarks, and the simulation results show that the proposed controller outperforms the original adaptive dynamic fuzzy controller and the pure neural network-based GrADP controller. In addition, the proposed controller is further applied on a large multimachine power system for static var compensator damping control, where simulation results demonstrate the effectiveness of the proposed approach on real applications. Furthermore, in order to demonstrate the theoretical guarantee of the proposed method, Lyapunov stability analysis to support the proposed Fuzzy-GrADP approach has also been carried out.

A large number of injuries or deaths may occur when an emergency happens in a crowded public place. The congestion at exits may slow down the egress rate due to the effect of “faster-is-slower”. This inspires us to study how human behavior dynamically changes over time at an emergency in a complex indoor environment. In this paper, we refer the panic of evacuees to their perception of the threat and propose a panic propagation model to model how crowd panic changes during evacuation at an emergency. Combined with the existing social force model, our panic model interprets the self-driven force and interactive forces with others in human mobility. To improve evacuation efficiency, robots are introduced to guide evacuees to escape. Using dynamic environment information, we design an evacuation exit selection algorithm where the optimal exit is automatically selected by the robot with the minimum escape time. In our experiments, a real shopping mall is examined, and the dynamic behavior of panicked evacuees is simulated with the proposed panic model. The evacuation performance of using emergency evacuation robots is evaluated. The improvement of evacuation efficiency validates the effectiveness of our robot-assisted evacuation system.

In this paper, we present a Bayesian classification approach for automatic text categorization using class-specific features. Unlike conventional text categorization approaches, our proposed method selects a specific feature subset for each class. To apply these class-specific features for classification, we follow Baggenstoss's PDF Projection Theorem (PPT) to reconstruct the PDFs in raw data space from the class-specific PDFs in low-dimensional feature subspace, and build a Bayesian classification rule. One noticeable significance of our approach is that most feature selection criteria, such as Information Gain (IG) and Maximum Discrimination (MD), can be easily incorporated into our approach. We evaluate our method's classification performance on several real-world benchmarks, compared with the state-of-the-art feature selection approaches. The superior results demonstrate the effectiveness of the proposed approach and further indicate its wide potential applications in data mining.

Automated feature selection is important for text categorization to reduce feature size and to speed up learning process of classifiers. In this paper, we present a novel and efficient feature selection framework based on the Information Theory, which aims to rank the features with their discriminative capacity for classification. We first revisit two information measures: Kullback-Leibler divergence and Jeffreys divergence for binary hypothesis testing, and analyze their asymptotic properties relating to type I and type II errors of a Bayesian classifier. We then introduce a new divergence measure, called Jeffreys-Multi-Hypothesis (JMH) divergence, to measure multi-distribution divergence for multi-class classification. Based on the JMH-divergence, we develop two efficient feature selection methods, termed maximum discrimination ($MD$) and methods, for text categorization. The promising results of extensive experiments demonstrate the effectiveness of the proposed approaches.

“Happy New Year!” At the beginning of 2016, I would like to take this opportunity to wish everyone a very happy, healthy, and prosperous new year! It is my great honor and privilege to serve as the Editor-in-Chief (EiC) of the IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (TNNLS), and I am excited to write this Editorial to start a new journey with you all.

This paper develops an adaptive modulation approach for power system control based on the approximate/adaptive dynamic programming method, namely, the goal representation heuristic dynamic programming (GrHDP). In particular, we focus on the fault recovery problem of a doubly fed induction generator (DFIG)-based wind farm and a static synchronous compensator (STATCOM) with high-voltage direct current (HVDC) transmission. In this design, the online GrHDP-based controller provides three adaptive supplementary control signals to the DFIG controller, STATCOM controller, and HVDC rectifier controller, respectively. The mechanism is to observe the system states and their derivatives and then provides supplementary control to the plant according to the utility function. With the GrHDP design, the controller can adaptively develop an internal goal representation signal according to the observed power system states, therefore, to achieve more effective learning and modulating. Our control approach is validated on a wind power integrated benchmark system with two areas connected by HVDC transmission lines. Compared with the classical direct HDP and proportional integral control, our GrHDP approach demonstrates the improved transient stability under system faults. Moreover, experiments under different system operating conditions with signal transmission delays are also carried out to further verify the effectiveness and robustness of the proposed approach.

Goal representation heuristic dynamic programming (GrHDP) control design has been developed in recent years. The control performance of this design has been demonstrated in several case studies, and also showed applicable to industrial-scale complex control problems. In this paper, we develop the theoretical analysis for the GrHDP design under certain conditions. It has been shown that the internal reinforcement signal is a bounded signal and the performance index can converge to its optimal value monotonically. The existence of the admissible control is also proved. Although the GrHDP control method has been investigated in many areas before, to the best of our knowledge, this is the first study of presenting the theoretical foundation of the internal reinforcement signal and how such an internal reinforcement signal can provide effective information to improve the control performance. Numerous simulation studies are used to validate the theoretical analysis and also demonstrate the effectiveness of the GrHDP design.

In imbalanced learning, most standard classification algorithms usually fail to properly represent data distribution and provide unfavorable classification performance. More specifically, the decision rule of minority class is usually weaker than majority class, leading to many misclassification of expensive minority class data. Motivated by our previous work ADASYN [1], this paper presents a novel kernel based adaptive synthetic over-sampling approach, named KernelADASYN, for imbalanced data classification problems. The idea is to construct an adaptive over-sampling distribution to generate synthetic minority class data. The adaptive over-sampling distribution is first estimated with kernel density estimation methods and is further weighted by the difficulty level for different minority class data. The classification performance of our proposed adaptive over-sampling approach is evaluated on several real-life benchmarks, specifically on medical and healthcare applications. The experimental results show the competitive classification performance for many real-life imbalanced data classification problems.

Recently, the sequential attack, where multiple malignant contingencies are launched by attackers sequentially, has revealed power grid vulnerability under cascading failures. This paper systematically analyzes properties and features of N-k cascaded- based sequential line-switching attacks using a DC power flow based cascading failure simulator (DC- CFS). This paper first explains the key factors behind cascade-based attacks, then compares three adopted metrics with an original line-margin metric to compute vulnerability indexes and design sequential attacks. Two target search schemes, i.e., offline and online target search in sequential attacks, are also presented. Simulation results of N-2 to N-4 line-switching attacks have suggested that the proposed line margin metric produces stronger sequential attacks, and online target search is more effective than offline search. Reasons behind counter-intuitive load loss resulting from different metrics are also analyzed to facilitate future study on the risk of sequential attacks.

We introduce a new parallel computing and communication architecture, Reflex-Tree, with massive sensing, data processing, and control functions suitable for future smart cities. The central feature of the proposed Reflex-Tree architecture is inspired by a fundamental element of the human nervous system: reflex arcs, the neuromuscular reactions and instinctive motions of a part of the body in response to urgent situations. At the bottom level of the Reflex-Tree (layer 4), novel sensing devices are proposed that are controlled by low power processing elements. These "leaf" nodes are then connected to new classification engines based on machine learning techniques, including support vector machines (SVM), to form the third layer. The next layer up consists of servers that provide accurate control decisions via multi-layer adaptive learning and spatial-temporal association, before they are connected to the top level cloud where complex system behavior analysis is performed. Our multi-layered architecture mimics human neural circuits to achieve the high levels of parallelization and scalability required for efficient city-wide monitoring and feedback. To demonstrate the utility of our architecture, we present the design, implementation, and experimental evaluation of a prototype Reflex-Tree. City power supply network and gas pipeline management scenarios are used to drive our prototype as case studies. We show the effectiveness for several levels of the architecture and discuss the feasibility of implementation.

In this paper, a novel predictive event-triggered control method based on heuristic dynamic programming (HDP) algorithm is developed for nonlinear continuous-time systems. A model network is used to estimate the system state vector, so that the event-triggered instant is available to predict one step ahead of time. Furthermore, an actor-critic structure is used to approximate the optimal event-triggered control law and performance index function. Although event-triggered adaptive dynamic programming (ADP) has been investigated in the community before, to our best knowledge, this is the first study of using a “predictive” approach through a model network to design the event-triggered ADP. This is the key contribution of this work. Compared to the existing event-triggered ADP methods, our simulations demonstrate that the predictive event-triggered approach can achieve improved control performance and lower computational cost in comparison with the existing methods.

This paper analyzes advanced reinforcement learning techniques and compares some of them to motivated learning. Motivated learning is briefly discussed indicating its relation to reinforcement learning. A black box scenario for comparative analysis of learning efficiency in autonomous agents is developed and described. This is used to analyze selected algorithms. Reported results demonstrate that in the selected category of problems, motivated learning outperformed all reinforcement learning algorithms we compared with.

A new theoretical analysis towards the goal representation adaptive dynamic programming (GrADP) design proposed in [1], [2] is investigated in this paper. Unlike the proofs of convergence for adaptive dynamic programming (ADP) in literature, here we provide a new insight for the error bound between the estimated value function and the expected value function. Then we employ the critic network in GrADP approach to approximate the Q value function, and use the action network to provide the control policy. The goal network is adopted to provide the internal reinforcement signal for the critic network over time. Finally, we illustrate that the estimated Q value function is close to the expected value function in an arbitrary small bound on the maze navigation example.

In this paper, we propose a novel event-triggered adaptive dynamic programming (ADP) method using only the input-output data. Event-triggered method is widely used for its computational efficiency capacity. Comparing with the traditional method which updates the controller periodically, the event-triggered method only updates the controller when it is necessary and therefore the computation is reduced. Generally, the triggered condition is based on the system current and sampled states. In this paper, we consider a neural-network-based observer to recover the system dynamics using the measured input-output data. The triggered instants are calculated according to the recovered state. Stability analysis of the proposed approach is presented. We verify our proposed method through a robot-arm example.

Power grids are often run near the operational limits because of increasing electricity demand, where even small disturbances could possibly trigger major blackouts. The attacks are the potential threats to trigger large-scale cascading failures in the power grid. In particular, the attacks mean to make substations/transmission lines lose functionality by either physical sabotages or cyber attacks. Previously, the attacks were investigated from substation-only/transmission-line-only perspectives, assuming attacks can occur only on substations/transmission lines. In this paper, we introduce the joint substation-transmission line perspective, which assumes attacks can happen on substations, transmission lines, or both. The introduced perspective is a nature extension to substation-only and transmission-line-only perspectives. Such extension leads to discovering many joint substation-transmission line vulnerabilities. Furthermore, we investigate the joint substation-transmission line attack strategies. In particular, we design a new metric, the component interdependency graph (CIG), and propose the CIG-based attack strategy. In simulations, we adopt IEEE 30 bus system, IEEE 118 bus system, and Bay Area power grid as test benchmarks, and use the extended degree-based and load attack strategies as comparison schemes. Simulation results show the CIG-based attack strategy has stronger attack performance.

Model-based dual heuristic dynamic programming (MB-DHP) is a popular approach in approximating optimal solutions in control problems. Yet, it usually requires offline training for the model network, and thus resulting in extra computational cost. In this brief, we propose a model-free DHP (MF-DHP) design based on finite-difference technique. In particular, we adopt multilayer perceptron with one hidden layer for both the action and the critic networks design, and use delayed objective functions to train both the action and the critic networks online over time. We test both the MF-DHP and MB-DHP approaches with a discrete time example and a continuous time example under the same parameter settings. Our simulation results demonstrate that the MF-DHP approach can obtain a control performance competitive with that of the traditional MB-DHP approach while requiring less computational resources.

A general utility function representation is proposed to provide the required derivable and adjustable utility function for the dual heuristic dynamic programming (DHP) design. Goal representation DHP (GrDHP) is presented with a goal network being on top of the traditional DHP design. This goal network provides a general mapping between the system states and the derivatives of the utility function. With this proposed architecture, we can obtain the required derivatives of the utility function directly from the goal network. In addition, instead of a fixed predefined utility function in literature, we conduct an online learning process for the goal network so that the derivatives of the utility function can be adaptively tuned over time. We provide the control performance of both the proposed GrDHP and the traditional DHP approaches under the same environment and parameter settings. The statistical simulation results and the snapshot of the system variables are presented to demonstrate the improved learning and controlling performance. We also apply both approaches to a power system example to further demonstrate the control capabilities of the GrDHP approach.

In this paper, we extend the exponentially embedded family (EEF), a new approach to model order estimation and probability density function construction originally proposed by Kay in 2005, to multivariate pattern recognition. Specifically, a parametric classifier rule based on the EEF is developed, in which we construct a distribution for each class based on a reference distribution. The proposed method can address different types of classification problems in either a data-driven manner or a model-driven manner. In this paper, we demonstrate its effectiveness with examples of synthetic data classification and real-life data classification in a data-driven manner and the example of power quality disturbance classification in a model-driven manner. To evaluate the classification performance of our approach, the Monte-Carlo method is used in our experiments. The promising experimental results indicate many potential applications of the proposed method.

Adaptive dynamic programming (ADP) has been widely recognized as one of the “core methodologies” to achieve optimal control for intelligent systems in Markov decision process (MDP). Generally, ADP control design requires all the information of the system dynamics. However, in many practical situations, the measured input and output data can only represent part of the system states. This means the complete information of the system cannot be available in many real-world cases, which narrows the range of application of the ADP design. In this paper, we propose a data-driven ADP method to stabilize the system with partially observable dynamics based on neural network techniques. A state network is integrated into the typical actor-critic architecture to provide an estimated state from the measured input/output sequences. The theoretical analysis and the stability discussion of this data-driven ADP method are also provided. Two examples are studied to verify our proposed method.

Previous works on outdoor traffic sign recognition and classification have been demonstrated useful to the driver assistant system and the possibility to the autonomous vehicles. This motivates our research on the assistance for visual impairment or visual disabled pedestrians in the indoor environment. In this paper, we build an indoor sign database and investigate the recognition and classification for the indoor sign problem. We adopt the classical techniques on extracting the features, including the principle component analysis (PCA), dense scale invariant feature transform (DSIFT), histogram of oriented gradients (HOG), and conduct the state-of-art classification techniques, such as the neural network (NN), support vector machine (SVM) and k-nearest neighbors (KNN). We provide the experimental results on this newly built database and also discuss the insight for the possibility of indoor navigation for the blind or visual-disabled people.

Vulnerability analysis on the power grid has been widely conducted from the substation-only and transmission-line-only perspectives. In order words, it is considered that attacks can occur on substations or transmission lines separately. In this paper, we naturally extend existing two perspectives and introduce the joint-substation-transmission-line's perspective, which means attacks can concurrently occur on substations and transmission lines. Vulnerabilities are referred to as these multiple-component combinations that can yield large damage to the power grid. One such combination consists of substations, transmission lines, or both. The new perspective is promising to discover more power grid vulnerabilities. In particular, we conduct the vulnerability analysis on the IEEE 39 bus system. Compared with known substation-only/transmission-line-only vulnerabilities, joint-substation-transmission-line vulnerabilities account for the largest percentage. Referring to three-component vulnerabilities, for instance, joint-substation-transmission-line vulnerabilities account for 76.06%; substation-only and transmission-line-only vulnerabilities account for 10.96% and 12.98%, respectively. In addition, we adopt two existing metrics, degree and load, to study the joint-substation-transmission-line attack strategy. Generally speaking, the joint-substation-transmission-line attack strategy based on the load metric has better attack performance than comparison schemes.

The vulnerability analysis is vital for safely running power grids. The simultaneous attack, which applies multiple failures simultaneously, does not consider the time domain in applying failures, and is limited to find unknown vulnerabilities of power grid networks. In this paper, we discover a new attack scenario, called the sequential attack, in which the failures of multiple network components (i.e., links/nodes) occur at different time. The sequence of such failures can be carefully arranged by attackers in order to maximize attack performances. This attack scenario leads to a new angle to analyze and discover vulnerabilities of grid networks. The IEEE 39 bus system is adopted as test benchmark to compare the proposed attack scenario with the existing simultaneous attack scenario. New vulnerabilities are found. For example, the sequential failure of two links, e.g., links 26 and 39 in the test benchmark, can cause 80% power loss, whereas the simultaneous failure of them causes less than 10% power loss. In addition, the sequential attack is demonstrated to be statistically stronger than the simultaneous attack. Finally, several metrics are compared and discussed in terms of whether they can be used to sharply reduce the search space for identifying strong sequential attacks.

We present a novel online learning control algorithm (OLCPA) which comprises projected gradient temporal difference for action-value function (PGTDAVF) and advanced heuristic dynamic programming with one step delay (AHD-POSD). PGTDAVF can guarantee the convergence of temporal difference(TD)-based policy learning with smooth action-value function approximators, such as neural networks. Meanwhile, AHDPOSD is a specially designed framework for embedding PGTDAVF in to conduct online learning control. It not only coincides with the intention of temporal difference but also enables PGTDAVF to be effective under nonidentical policy environment, which results in more practicality. In this way, the proposed algorithms achieve the stability and practicability simultaneously. Finally, simulation of online learning control on a cart pole benchmark demonstrates practical control capability and efficiency of the presented method.

The parametric classifiers trained with the Bayesian rule are usually more accurate than the non-parametric classifiers such as nearest neighbors, neural network and support vector machine, when the class-conditional densities of distribution models are known except for some of their parameters and the training data is abundant. However, the parametric classifiers would perform poorly if these class-conditional densities are unknown and the assumed distribution models are inaccurate. In this paper, we propose a hybrid classification method for the data with partially known distribution models where only the distribution models of some classes are known. For this partial models case, the proposed hybrid classifier makes the best use of knowledge of known distribution models with Bayesian interference, while both purely parametric and non-parametric classifiers would lose a specific predictive capacity for classification. Theoretical proofs and experimental results show that the proposed hybrid classifier has much better performance than these purely parametric and non-parametric classifiers for the data with partial models.

Due to the intermittent power generation from renewable energy in the smart grid (i.e., photovoltaic (PV) or wind farm), large frequency fluctuation occurs when the load-frequency control (LFC) capacity is not enough to compensate the unbalance of generation and load demand. This problem may become worsen when the system is in island operating. Meanwhile, in the near future, electric vehicles (EVs) will be widely used by customers, where the EV station could be treated as dispersed battery energy storage. Therefore, the vehicle-to-grid (V2G) power control can be applied to compensate for inadequate LFC capacity, thus improving the island smart grid frequency stability. In this paper, an on-line learning method, called goal representation adaptive dynamic programming (GrADP), is adopted to coordinate control of units in an island smart grid. In the controller design, adaptive supplementary control signals are provided to proportional-integral (PI) controllers by online GrADP according to the utility function. Simulations on a benchmark smart grid with micro turbine (MT), EVs and PVs demonstrate the superior control effect and robustness of the proposed coordinate controller over the original PI controller and fuzzy controller.

This paper provides an adaptive event-triggered method using adaptive dynamic programming (ADP) for the nonlinear continuous-time system. Comparing to the traditional method with fixed sampling period, the event-triggered method samples the state only when an event is triggered and therefore the computational cost is reduced. We demonstrate the theoretical analysis on the stability of the event-triggered method, and integrate it with the ADP approach. The system dynamics are assumed unknown. The corresponding ADP algorithm is given and the neural network techniques are applied to implement this method. The simulation results verify the theoretical analysis and justify the efficiency of the proposed event-triggered technique using the ADP approach.

The invention of deep belief network (DBN) provides a powerful tool for data modeling. The key advantage of DBN is that it is driven by training data only, which can alleviate researchers from the routine of devising explicit models or features for data with complicated distributions. However, as the dimensionality and quantity of data increase, the computing load of training a DBN increases rapidly. Prospectively, the remarkable computing power provided by modern GPU devices can reduce the training time of DBN significantly. As highly efficient computational libraries become available, it provides additional support for GPU based parallel computing. Moreover, GPU server is more affordable and accessible compared with computer cluster or supercomputer. In this paper, we implement a variant of the DBNs, called folded-DBN, on NVIDA's Tesla K20 GPU. In our simulations, two sets of database are used to train the folded-DBNs on both CPU and GPU platforms. Comparing execution time of the fine-tuning process, the GPU implementation results 7 to 11 times speedup over the CPU platform.

Faces convey a wealth of social signals, including race, expression, identity, age and gender, all of which have attracted increasing attention from multi-disciplinary research, such as psychology, neuroscience, computer science, to name a few. Gleaned from recent advances in computer vision, computer graphics, and machine learning, computational intelligence based racial face analysis has been particularly popular due to its significant potential and broader impacts in extensive real-world applications, such as security and defense, surveillance, human computer interface (HCI), biometric-based identification, among others. These studies raise an important question: How implicit, non-declarative racial category can be conceptually modeled and quantitatively inferred from the face? Nevertheless, race classification is challenging due to its ambiguity and complexity depending on context and criteria. To address this challenge, recently, significant efforts have been reported toward race detection and categorization in the community. This survey provides a comprehensive and critical review of the state-of-the-art advances in face-race perception, principles, algorithms, and applications. We first discuss race perception problem formulation and motivation, while highlighting the conceptual potentials of racial face processing. Next, taxonomy of feature representational models, algorithms, performance and racial databases are presented with systematic discussions within the unified learning scenario. Finally, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potentially important cross-cutting themes and research directions for the issue of learning race from face.

The security issue of complex networks has drawn significant concerns recently. While pure topological analyzes from a network security perspective provide some effective techniques, their inability to characterize the physical principles requires a more comprehensive model to approximate failure behavior of a complex network in reality. In this paper, based on an extended topological metric, we proposed an approach to examine the vulnerability of a specific type of complex network, i.e., the power system, against cascading failure threats. The proposed approach adopts a model called extended betweenness that combines network structure with electrical characteristics to define the load of power grid components. By using this power transfer distribution factor-based model, we simulated attacks on different components (buses and branches) in the grid and evaluated the vulnerability of the system components with an extended topological cascading failure simulator. Influence of different loading and overloading situations on cascading failures was also evaluated by testing different tolerance factors. Simulation results from a standard IEEE 118-bus test system revealed the vulnerability of network components, which was then validated on a dc power flow simulator with comparisons to other topological measurements. Finally, potential extensions of the approach were also discussed to exhibit both utility and challenge in more complex scenarios and applications.

The modern society increasingly relies on electrical service, which also brings risks of catastrophic consequences, e.g., large-scale blackouts. In the current literature, researchers reveal the vulnerability of power grids under the assumption that substations/transmission lines are removed or attacked synchronously. In reality, however, it is highly possible that such removals can be conducted sequentially. Motivated by this idea, we discover a new attack scenario, called the sequential attack, which assumes that substations/transmission lines can be removed sequentially, not synchronously. In particular, we find that the sequential attack can discover many combinations of substation whose failures can cause large blackout size. Previously, these combinations are ignored by the synchronous attack. In addition, we propose a new metric, called the sequential attack graph (SAG), and a practical attack strategy based on SAG. In simulations, we adopt three test benchmarks and five comparison schemes. Referring to simulation results and complexity analysis, we find that the proposed scheme has strong performance and low complexity.

In this paper, we develop and analyze an optimal control method for a class of discrete-time nonlinear Markov jump systems (MJSs) with unknown system dynamics. Specifically, an identifier is established for the unknown systems to approximate system states, and an optimal control approach for nonlinear MJSs is developed to solve the Hamilton-Jacobi-Bellman equation based on the adaptive dynamic programming technique. We also develop detailed stability analysis of the control approach, including the convergence of the performance index function for nonlinear MJSs and the existence of the corresponding admissible control. Neural network techniques are used to approximate the proposed performance index function and the control law. To demonstrate the effectiveness of our approach, three simulation studies, one linear case, one nonlinear case, and one single link robot arm case, are used to validate the performance of the proposed optimal control method.

Security issues related to power grid networks have attracted the attention of researchers in many fields. Recently, a new network model that combines complex network theories with power flow models was proposed. This model, referred to as the extended model, is suitable for investigating vulnerabilities in power grid networks. In this paper, we study cascading failures of power grids under the extended model. Particularly, we discover that attack strategies that select target nodes (TNs) based on load and degree do not yield the strongest attacks. Instead, we propose a novel metric, called the risk graph, and develop novel attack strategies that are much stronger than the load-based and degree-based attack strategies. The proposed approaches and the comparison approaches are tested on IEEE 57 and 118 bus systems and Polish transmission system. The results demonstrate that the proposed approaches can reveal the power grid vulnerability in terms of causing cascading failures more effectively than the comparison approaches.

In this paper, we proposed a new nonlinear tracking controller based on heuristic dynamic programming (HDP) with the tracking filter. Specifically, we integrate a goal network into the regular HDP design and provide the critic network with detailed internal reward signal to help the value function approximation. The architecture is explicitly explained with the tracking filter, goal network, critic network and action network, respectively. We provide the stability analysis of our proposed controller with Lyapunov approach. It is shown that the filtered tracking errors and the weights estimation errors in neural networks are all uniformly ultimately bounded (UUB) under certain conditions. Finally, we compare our proposed approach with regular HDP approach in virtual reality (VR)/Simulink environment to justify the improved control performance.

In this paper, we propose an optimal control method based on the solution of Hamilton-Jacobi-Bellman (HJB) equation for the continuous-time nonlinear system with bounded unknown perturbation. The robust control system is converted into the corresponding optimal control system with appropriate performance index and the equivalence of the transformation is proved, i.e., the solution of the optimal control problem can globally asymptotically stabilize the robust control system. Adaptive dynamic programming (ADP) based approach is presented to iteratively approximate the optimal performance index and obtain the optimal control policy. A neural network with adaptive weights is applied to implement this approach. An example is given to illustrate the proposed method.

In the study of power grid security, the cascading failure analysis in multi-contingency scenarios has been a challenge due to its topological complexity and computational cost. Both network analyses and load ranking methods have their own limitations. In this paper, based on self-organizing map (SOM), we propose an integrated approach combining spatial feature (distance)-based clustering with electrical characteristics (load) to assess the vulnerability and cascading effect of multiple component sets in the power grid. Using the clustering result from SOM, we choose sets of heavy-loaded initial victims to perform attack schemes and evaluate the subsequent cascading effect of their failures, and this SOM-based approach effectively identifies the more vulnerable sets of substations than those from the traditional load ranking and other clustering methods. As a result, this new approach provides an efficient and reliable technique to study the power system failure behavior in cascading effect of critical component failure.

In this paper, we present a new adaptive dynamic programming approach by integrating a reference network that provides an internal goal representation to help the systems learning and optimization. Specifically, we build the reference network on top of the critic network to form a dual critic network design that contains the detailed internal goal representation to help approximate the value function. This internal goal signal, working as the reinforcement signal for the critic network in our design, is adaptively generated by the reference network and can also be adjusted automatically. In this way, we provide an alternative choice rather than crafting the reinforcement signal manually from prior knowledge. In this paper, we adopt the online action-dependent heuristic dynamic programming (ADHDP) design and provide the detailed design of the dual critic network structure. Detailed Lyapunov stability analysis for our proposed approach is presented to support the proposed structure from a theoretical point of view. Furthermore, we also develop a virtual reality platform to demonstrate the real-time simulation of our approach under different disturbance situations. The overall adaptive learning performance has been tested on two tracking control benchmarks with a tracking filter. For comparative studies, we also present the tracking performance with the typical ADHDP, and the simulation results justify the improved performance with our approach.

Goal representation heuristic dynamic programming (GrHDP) is proposed in this paper to demonstrate online learning in the Markov decision process. In addition to the (external) reinforcement signal in literature, we develop an adaptively internal goal/reward representation for the agent with the proposed goal network. Specifically, we keep the actor-critic design in heuristic dynamic programming (HDP) and include a goal network to represent the internal goal signal, to further help the value function approximation. We evaluate our proposed GrHDP algorithm on two 2-D maze navigation problems, and later on one 3-D maze navigation problem. Compared to the traditional HDP approach, the learning performance of the agent is improved with our proposed GrHDP approach. In addition, we also include the learning performance with two other reinforcement learning algorithms, namely Sarsa(λ) and Q-learning, on the same benchmarks for comparison. Furthermore, in order to demonstrate the theoretical guarantee of our proposed method, we provide the characteristics analysis toward the convergence of weights in neural networks in our GrHDP approach.

In the past decade, adaptive critic designs (ACDs), including heuristic dynamic programming (HDP), dual heuristic programming (DHP), and their action-dependent ones, have been widely studied to realize online learning control of dynamical systems. However, because neural networks with manually designed features are commonly used to deal with continuous state and action spaces, the generalization capability and learning efficiency of previous ACDs still need to be improved. In this paper, a novel framework of ACDs with sparse kernel machines is presented by integrating kernel methods into the critic of ACDs. To improve the generalization capability as well as the computational efficiency of kernel machines, a sparsification method based on the approximately linear dependence analysis is used. Using the sparse kernel machines, two kernel-based ACD algorithms, that is, kernel HDP (KHDP) and kernel DHP (KDHP), are proposed and their performance is analyzed both theoretically and empirically. Because of the representation learning and generalization capability of sparse kernel machines, KHDP and KDHP can obtain much better performance than previous HDP and DHP with manually designed neural networks. Simulation and experimental results of two nonlinear control problems, that is, a continuous-action inverted pendulum problem and a ball and plate control problem, demonstrate the effectiveness of the proposed kernel ACD methods.

The construction of a radial basis function (RBF) network involves the determination of the model size, hidden nodes, and output weights. Least squares-based subset selection methods can determine a RBF model size and its parameters simultaneously. Although these methods are robust, they may not achieve optimal results. Alternatively, gradient methods are widely used to optimize all the parameters. The drawback is that most algorithms may converge slowly as they treat hidden nodes and output weights separately and ignore their correlations. In this paper, a new discrete-continuous algorithm is proposed for the construction of a RBF model. First, the orthogonal least squares (OLS)-based forward stepwise selection constructs an initial model by selecting model terms one by one from a candidate term pool. Then a new Levenberg-Marquardt (LM)-based parameter optimization is proposed to further optimize the hidden nodes and output weights in the continuous space. To speed up the convergence, the proposed parameter optimization method considers the correlation between the hidden nodes and output weights, which is achieved by translating the output weights to dependent parameters using the OLS method. The correlation is also used by the previously proposed continuous forward algorithm (CFA). However, unlike the CFA, the new method optimizes all the parameters simultaneously. In addition, an equivalent recursive sum of squared error is derived to reduce the computation demanding for the first derivatives used in the LM method. Computational complexity is given to confirm the new method is much more computationally efficient than the CFA. Different numerical examples are presented to illustrate the effectiveness of the proposed method. Further, Friedman statistical tests on 13 classification problems are performed, and the results demonstrate that RBF networks built by the new method are very competitive in comparison with some popular classifiers.

Security issues in complex systems such as power grid, communication network, Internet, among others have attracted wide attention from academic, government and industry. In this paper, we investigate the vulnerabilities of power grid under a topology-based network model in the context of cascading failures caused by physical attacks against substations and transmission lines. In particular, we develop attack strategies from the attackers' points of view, aiming to cause severe damage to the network efficiency, as a way to revealing the vulnerability of the system. We propose a new and useful metric, load distribution vector (LDV), to describe the properties of nodes and links. Based on the LDV, we develop a multi-node attack strategy and a multi-link attack strategy, which are proved to be stronger attacks than the traditional load-based attacks using the Western North American power grid data. For example, the removal of only three critical nodes in the grid can reduce more than 30% of the original network efficiency, and the removal of only three critical links can reduce the network efficiency by 23%. In the above cases, the traditional load-based schemes reduce the network efficiency by 23.57% and 18.35%, respectively.

We are interested in developing a multi-goal generator to provide detailed goal representations that help to improve the performance of the adaptive critic design (ACD). In this paper we propose a hierarchical structure of goal generator networks to cascade external reinforcement into more informative internal goal representations in the ACD. This is in contrast with previous designs in which the external reward signal is assigned to the critic network directly. The ACD control system performance is evaluated on the ball-and-beam balancing benchmark under noise-free and various noisy conditions. Simulation results in the form of a comparative study demonstrate effectiveness of our approach.

Feature selection, which aims to obtain valuable feature subsets, has been an active topic for years. How to design an evaluating metric is the key for feature selection. In this paper, we address this problem using imputation quality to search for the meaningful features and propose feature selection via sparse imputation (FSSI) method. The key idea is utilizing sparse representation criterion to test individual feature. The feature based classification is used to evaluate the proposed method. Comparative studies are conducted with classic feature selection methods (such as Fisher score and Laplacian score). Experimental results on benchmark data sets demonstrate the effectiveness of FSSI method.

This paper aims to integrate the fuzzy control with adaptive dynamic programming (ADP) scheme, to provide an optimized fuzzy control performance, together with faster convergence of ADP for the help of the fuzzy prior knowledge. ADP usually consists of two neural networks, one is the Actor as the controller, the other is the Critic as the performance evaluator. A fuzzy controller applied in many fields can be used instead as the Actor to speed up the learning convergence, because of its simplicity and prior information on fuzzy membership and rules. The parameters of the fuzzy rules are learned by ADP scheme to approach optimal control performance. The feature of fuzzy controller makes the system steady and robust to system states and uncertainties. Simulations on under-actuated systems, a cart-pole plant and a pendubot plant, are implemented. It is verified that the proposed scheme is capable of balancing under-actuated systems and has a wider control zone.

In this paper, we propose a novel method SOMKE, for kernel density estimation (KDE) over data streams based on sequences of self-organizing map (SOM). In many stream data mining applications, the traditional KDE methods are infeasible because of the high computational cost, processing time, and memory requirement. To reduce the time and space complexity, we propose a SOM structure in this paper to obtain well-defined data clusters to estimate the underlying probability distributions of incoming data streams. The main idea of this paper is to build a series of SOMs over the data streams via two operations, that is, creating and merging the SOM sequences. The creation phase produces the SOM sequence entries for windows of the data, which obtains clustering information of the incoming data streams. The size of the SOM sequences can be further reduced by combining the consecutive entries in the sequence based on the measure of Kullback-Leibler divergence. Finally, the probability density functions over arbitrary time periods along the data streams can be estimated using such SOM sequences. We compare SOMKE with two other KDE methods for data streams, the M-kernel approach and the cluster kernel approach, in terms of accuracy and processing time for various stationary data streams. Furthermore, we also investigate the use of SOMKE over nonstationary (evolving) data streams, including a synthetic nonstationary data stream, a real-world financial data stream and a group of network traffic data streams. The simulation results illustrate the effectiveness and efficiency of the proposed approach.

We propose a new classifier combination method, the signal strength-based combining (SSC) approach, to combine the outputs of multiple classifiers to support the decision-making process in classification tasks. As ensemble learning methods have attracted growing attention from both academia and industry recently, it is critical to understand the fundamental issues of the combining rule. Motivated by the signal strength concept, our proposed SSC algorithm can effectively integrate the individual vote from different classifiers in an ensemble learning system. Comparative studies of our method with nine major existing combining rules, namely, geometric average rule, arithmetic average rule, median value rule, majority voting rule, Borda count, max and min rule, weighted average, and weighted majority voting rules, is presented. Furthermore, we also discuss the relationship of the proposed method with respect to margin-based classifiers, including the boosting method (AdaBoost.M1 and AdaBoost.M2) and support vector machines by margin analysis. Detailed analyses of margin distribution graphs are presented to discuss the characteristics of the proposed method. Simulation results for various real-world datasets illustrate the effectiveness of the proposed method.

In this paper we propose to integrate the recursive Levenberg-Marquardt method into the adaptive dynamic programming (ADP) design for improved learning and adaptive control performance. Our key motivation is to consider a balanced weight updating strategy with the consideration of both robustness and convergence during the online learning process. Specifically, a modified recursive Levenberg-Marquardt (LM) method is integrated into both the action network and critic network of the ADP design, and a detailed learning algorithm is proposed to implement this approach. We test the performance of our approach based on the triple link inverted pendulum, a popular benchmark in the community, to demonstrate online learning and control strategy. Experimental results and comparative study under different noise conditions demonstrate the effectiveness of this approach.

The power grid network is a complex network which is subjected to attacks and cascading failures. In this paper, we study the vulnerabilities of power grid in terms of cascading failures caused by node failures. Specifically, we define three metrics, the percentage-of-failure, Required Redundancy (RED), and Risk if Failure (RIF), to represent the critical level for each node. Based on these metrics, we can easily find the optimal victim nodes which attackers should choose to attack in order to cause cascading failure. From the defense point of view, these nodes are the weakest components of the power grid system and need more protection. With the results in this paper, we can be more aware of the risk level faced by the system if some nodes are taken down. Simulation results demonstrate the effectiveness of the proposed optimal victim nodes selection strategy.

Multiple Self-Organizing Maps (MSOMs) based classification methods are able to combine the advantages of both unsupervised and supervised learning mechanisms. Specifically, unsupervised SOM can search for similar properties from input data space and generate data clusters within each class, while supervised SOM can be trained from the data via label matching in the global SOM lattice space. In this work, we propose a novel classification method that integrates MSOMs with Genetic Algorithm (GA) to avoid the influence of local minima. Davies-Bouldin Index (DBI) and Mean Square Error (MSE) are adopted as the objective functions for searching the optimal solution space. Experimental results demonstrate the effectiveness and robustness of our proposed approach based on several benchmark data sets from UCI Machine Learning Repository.

This paper focuses on the efficiency improvement of online actor-critic design base on the Levenberg-Marquardt (LM) algorithm rather than traditional chain rule. Over the decades, several generations of adaptive/approximate dynamic programming (ADP) structures have been proposed in the community and demonstrated many successfully applications. Neural network with backpropagation has been one of the most important approaches to tune the parameters in such ADP designs. In this paper, we aim to study the integration of Levenberg-Marquardt method into the regular actor-critic design to improve weights updating and learning for a quadratic convergence under certain condition. Specifically, for the critic network design, we adopt the LM method targeting improved learning performance, while for the action network, we use the neural network with backpropagation to provide an appropriate control action. A detailed learning algorithm is presented, followed by benchmark tests of pendulum swing up and balance and cart-pole balance tasks. Various simulation results and comparative study demonstrated the effectiveness of this approach.

The Sixth International Symposium on Neural Networks (ISNN’09) was held on 26–29 May 2009 in Wuhan, China. The ISNN 2009 was a great success and provided a high-level international forum for scient...

Adaptive dynamic programming (ADP) is a promising research field for design of intelligent controllers, which can both learn on-the-fly and exhibit optimal behavior. Over the past decades, several generations of ADP design have been proposed in the literature, which have demonstrated many successful applications in various benchmarks and industrial applications. While many of the existing researches focus on multiple-inputs-single-output system with steepest descent search, in this paper we investigate a generalized multiple-input-multiple-output (GMIMO) ADP design for online learning and control, which is more applicable to a wide range of practical real-world applications. Furthermore, an improved weight-updating algorithm based on recursive Levenberg-Marquardt methods is presented and embodied in the GMIMO approach to improve its performance. Finally, we test the performance of this approach based on a practical complex system, namely, the learning and control of the tension and height of the looper system in a hot strip mill. Experimental results demonstrate that the proposed approach can achieve effective and robust performance.

Recent years have witnessed an incredibly increasing interest in the topic of incremental learning. Unlike conventional machine learning situations, data flow targeted by incremental learning becomes available continuously over time. Accordingly, it is desirable to be able to abandon the traditional assumption of the availability of representative training data during the training period to develop decision boundaries. Under scenarios of continuous data flow, the challenge is how to transform the vast amount of stream raw data into information and knowledge representation, and accumulate experience over time to support future decision-making process. In this paper, we propose a general adaptive incremental learning framework named ADAIN that is capable of learning from continuous raw data, accumulating experience over time, and using such knowledge to improve future learning and prediction performance. Detailed system level architecture and design strategies are presented in this paper. Simulation results over several real-world data sets are used to validate the effectiveness of this method.

In this paper, an incremental self-organizing map integrated with hierarchical neural network (ISOM-HNN) is proposed as an efficient approach for signal classification in cognitive radio networks. This approach can effectively detect unknown radio signals in the uncertain communication environment. The adaptability of ISOM can improve the real-time learning performance, which provides the advantage of using this approach for on-line learning and control of cognitive radios in many real-world application scenarios. Furthermore, we propose to integrate the ISOM with the hierarchical neural network (HNN) to improve the learning and prediction accuracy. Detailed learning algorithm and simulation results are presented in this work to demonstrate the effectiveness of this approach.

Learning from data streams has inspired considerable interests in recent years due to its wide applications in the fields such as network intrusion detection, credit fraud identification, spam filtering, and many others. Given the fact that most algorithms developed thus far assume the class distribution of the streaming data is relatively balanced, they will inevitably be confronted with severe performance deterioration when handling the imbalanced data streams. Evolved from our previous work SERA (SElectively Recursive Approach), the MuSeRA algorithm is proposed in this paper to deal with the problem of learning from imbalanced data streams. By maintaining an ensemble consisting of hypotheses built upon the coming training data chunks balanced by selectively accommodating previous minority examples, MuSeRA can efficiently learn the target concept of the imbalanced data streams and thus obtain substantial performance improvement compared to our previous work SERA and the existing stream data mining algorithms. Simulation results validate the effectiveness of the proposed MuSeRA algorithm.

The development of an intelligent electric grid of future, a smart grid, has attracted significant amount of attention recently from academia, industry, and government as well. Among many efforts toward this objective, computational intelligence research could provide important technical support to help the society to accomplish this goal. In this paper, I present a high level discussion on the vision of smart grid and how computational intelligence research can provide critical technical support to this vision. Specifically, wide area situational awareness and adaptive dynamic programming (ADP) based intelligent control are used as two examples in this work to illustrate the potential technical contributions of computational intelligence research toward the long-term objective of a smart grid. Numerous recent activities in the society on smart grid as well as future challenges and opportunities in this field are also highlighted and discussed in this paper.

In this paper, we study efficient rate control schemes for delay sensitive communications over wireless fading channels based on reinforcement learning. Our objective is to find a rate control scheme that optimizes the link layer performance, specifically, maximizes the system throughput subject to a fixed bit error rate (BER) constraint and longterm average power constraint. We assume the buffer at the transmitter is finite; hence packet drop happens when the buffer is full. We assume the fading channel under our study can be modeled as a finite state Markov chain, however the transition probability of channel states is not known, and the only information available about the wireless channel is the instantaneous channel gain, which is estimated and fed back from receiver side to the transmitter side on the fly. In this paper, we use reinforcement learning approach to learn the time-varying channel environment and search for the optimal control policy on line. Simulation results show that starting from an arbitrary control policy, the learning agent gradually modifies its estimation about the system model and adjusts the control policy to its optimality.

Co-training is a semi-supervised learning technique used to recover the unlabeled data based on two base learners. The normal co-training approaches use the most confidently recovered unlabeled data to augment the training data. In this paper, we investigate the co-training approaches with a focus on the diversity issue and propose the diversity of class probability estimation (DCPE) co-training approach. The key idea of the DCPE co-training method is to use DCPE between two base learners to choose the recovered unlabeled data. The results are compared with classic co-training, tri-training and self training methods. Our experimental study based on the UCI benchmark data sets shows that the DCPE co-training is robust and efficient in the classification.

In recent years, learning from imbalanced data has attracted growing attention from both academia and industry due to the explosive growth of applications that use and produce imbalanced data. However, because of the complex characteristics of imbalanced data, many real-world solutions struggle to provide robust efficiency in learning-based applications. In an effort to address this problem, this paper presents Ranked Minority Oversampling in Boosting (RAMOBoost), which is a RAMO technique based on the idea of adaptive synthetic data generation in an ensemble learning system. Briefly, RAMOBoost adaptively ranks minority class instances at each learning iteration according to a sampling probability distribution that is based on the underlying data distribution, and can adaptively shift the decision boundary toward difficult-to-learn minority and majority class instances by using a hypothesis assessment procedure. Simulation analysis on 19 real-world datasets assessed over various metrics-including overall accuracy, precision, recall, F-measure, G-mean, and receiver operation characteristic analysis-is used to illustrate the effectiveness of this method.

In this paper, we propose a self-organizing map approach for spatial outlier detection, the SOMSO method. Spatial outliers are abnormal data points which have significantly distinct non-spatial attribute values compared with their neighborhood. Detection of spatial outliers can further discover spatial distribution and attribute information for data mining problems. Self-Organizing map (SOM) is an effective method for visualization and cluster of high dimensional data. It can preserve intrinsic topological and metric relationships in datasets. The SOMSO method can solve high dimensional problems for spatial attributes and accurately detect spatial outliers with irregular features. The experimental results for the dataset based on U.S. population census indicate that SOMSO approach can successfully be applied in complicated spatial datasets with multiple attributes.

In this paper, we present the research of a foreign currency investment framework involving the prediction of the foreign currency exchange rates and the portfolio optimization under certain constrains. We adopt two machine learning methods, support vector machines (SVMs) and neural networks (NNs), as well as the traditional moving average method, to predict the exchange rates for three foreign currencies including Australia Dollars (AUD), European Euro (EUR), and Swiss Francs (CHF). Based on these forecastings, we choose two out of the three currencies listed above and build a portfolio by adopting multi-objective portfolio optimization techniques by maximizing the return and minimizing the risk. Karush-Kuhn-Tucker (KKT) theorem guarantees that the optimal portfolio is reachable. Simulation results show that the optimal portforlio investment can achieve superior return performance compared with three single currency investment benchmarks.

This paper proposes a novel feature ranking method, DensityRank, based on kernel estimation on the feature spaces to improve the classification performance. As the availability of raw data in many of today's applications continues to grow at an explosive rate, it is critical to assess the learning capabilities of different features and select the important subset of features to improve learning accuracy as well as reduce computational cost. In our approach, kernel methods are used to estimate the probability density function for each feature across different class labels. Discrepancy analysis based on the mean integrated square error (MISE) between pairs of such density estimations is used to provide the ranking values. Then, the ranked subspace method is adopted to select subsets of important features that are used to develop the learning models. Comparative study of this method with those of traditional ranking methods related to Fisher's discrimination ratio and information gain theory, as well as the random subspace algorithm and the bootstrap aggregating (bagging), are presented in this paper. Simulation results on various real-world data sets illustrate the effectiveness of the proposed method.

Recent years have witnessed an incredibly increasing interest in the topic of stream data mining. Despite the great success having been achieved, current approaches generally assume that the class distribution of the stream data is relatively balanced. However, in applications such as network intrusion detection, credit fraud detection, spam classification, and many others, the class distribution is mostly imbalanced and the cost for misclassifying a minority example is very expensive. Concept drifts is an unavoidable issue for stream data mining research, which is even more difficult to handle when the classifier has to learn from an imbalanced data stream whose target concept keeps drifting all the time. In this article, we propose a selectively recursive approach (SERA) to deal with the problem of learning from nonstationary imbalanced data streams. By selectively absorbing the previously received minority examples into the current training data chunk and potentially assigning the sampling probabilities proportionally to the majority and minority examples, SERA can alleviate the difficulty confronted by the conventional stream data mining methods when they have to learn from the nonstationary imbalanced data streams. Experiments performed on the synthetic datasets show that compared to the existing approaches, our approach is competitive in the general assessment metrics and is capable of significantly performance improvement in predicting minority instances.

With the continuous expansion of data availability in many large-scale, complex, and networked systems, such as surveillance, security, Internet, and finance, it becomes critical to advance the fundamental understanding of knowledge discovery and analysis from raw data to support decision-making processes. Although existing knowledge discovery and data engineering techniques have shown great success in many real-world applications, the problem of learning from imbalanced data (the imbalanced learning problem) is a relatively new challenge that has attracted growing attention from both academia and industry. The imbalanced learning problem is concerned with the performance of learning algorithms in the presence of underrepresented data and severe class distribution skews. Due to the inherent complex characteristics of imbalanced data sets, learning from such data requires new understandings, principles, algorithms, and tools to transform vast amounts of raw data efficiently into information and knowledge representation. In this paper, we provide a comprehensive review of the development of research in learning from imbalanced data. Our focus is to provide a critical review of the nature of the problem, the state-of-the-art technologies, and the current assessment metrics used to evaluate learning performance under the imbalanced learning scenario. Furthermore, in order to stimulate future research in this field, we also highlight the major opportunities and challenges, as well as potential important research directions for learning from imbalanced data.

Design of artificial neural structures capable of reliable and flexible long-term spatio-temporal memory is of paramount importance in machine intelligence. To this end, we propose a novel, biologically inspired, long-term memory (LTM) architecture. We intend to use it as a building block of a neuron-level architecture that is able to mimic natural intelligence through learning, anticipation, and goal-driven behavior. A mutual input enhancement and blocking structure is proposed, and its operation is discussed in detail. The paper focuses on a hierarchical memory organization, storage, recognition, and recall mechanisms. Simulation results of the proposed memory show its effectiveness, adaptability, and robustness. Accuracy of the proposed method is compared to other methods including Levenshtein distance method and a Markov chain.

In this paper, we propose a novel method for incremental semi-supervised learning. Unlike the traditional way of incremental learning or semi-supervised learning, we try to answer a more challenging question: given inadequate labeled training data, can one use the unlabeled testing data to improve the learning and prediction accuracy? The objective here is to reinforce the learning system trained offline through online incremental semi-supervised learning based on the testing data distribution. To do this, we propose an iterative algorithm that can adaptively recover the labels for testing data based on their confidence levels, and then extend the training population by such recovered data to facilitate learning and prediction. Multiple hypotheses are developed based on different learning capabilities of different recovered data sets, and a voting method is used to integrate the decisions from different hypotheses for the final predicted labels. We compare the proposed algorithm with bootstrap aggregating (bagging) method for performance evaluation. Simulation results on various real-world data sets illustrate the effectiveness of the proposed method.

This paper presents a novel adaptive synthetic (ADASYN) sampling approach for learning from imbalanced data sets. The essential idea of ADASYN is to use a weighted distribution for different minority class examples according to their level of difficulty in learning, where more synthetic data is generated for minority class examples that are harder to learn compared to those minority examples that are easier to learn. As a result, the ADASYN approach improves learning with respect to the data distributions in two ways: (1) reducing the bias introduced by the class imbalance, and (2) adaptively shifting the classification decision boundary toward the difficult examples. Simulation analyses on several machine learning data sets show the effectiveness of this method across five evaluation metrics.

This paper proposes an incremental multiple-object recognition and localization (IMORL) method. The objective of IMORL is to adaptively learn multiple interesting objects in an image. Unlike the conventional multiple-object learning algorithms, the proposed method can automatically and adaptively learn from continuous video streams over the entire learning life. This kind of incremental learning capability enables the proposed approach to accumulate experience and use such knowledge to benefit future learning and the decision making process. Furthermore, IMORL can effectively handle variations in the number of instances in each data chunk over the learning life. Another important aspect analyzed in this paper is the concept drifting issue. In multiple-object learning scenarios, it is a common phenomenon that new interesting objects may be introduced during the learning life. To handle this situation, IMORL uses an adaptive learning principle to autonomously adjust to such new information. The proposed approach is independent of the base learning models, such as decision tree, neural networks, support vector machines, and others, which provide the flexibility of using this method as a general learning methodology in multiple-object learning scenarios. In this paper, we use a neural network with a multilayer perceptron (MLP) structure as the base learning model and test the performance of this method in various video stream data sets. Simulation results show the effectiveness of this method.

This paper presents the research of using boot-strap methods for time-series prediction. Unlike the traditional single model (neural network, support vector machine, or any other types of learning algorithms) based time-series prediction, we propose to use bootstrap methods to construct multiple learning models, and then use a combination function to combine the output of each model for the final predicted output. In this paper, we use the neural network model as the base learning algorithm and applied this approach to the foreign currency exchange rate predictions. Six major foreign currency exchange rates including Australia Dollars (AUD), British Pounds (GBP), Canadian Dollars (CAD), European Euros (EUR), Japanese Yen (JPY) and Swiss Francs (CHF) are used for prediction (base currency is US Dollar). Simulations on the most recently available exchange rate data (January 01, 2003 to December 27, 2006) on both daily prediction and weekly prediction indicate that the proposed method can significantly improve the forecasting performance compared to the traditional single neural network based approach.

Feature selection is an active research area in machine learning for high dimensional dataset analysis. The idea is to perform the learning process solely on the top ranked feature spaces instead of the entire original feature space, and therefore to improve the understanding of the inherent characteristics of such dataset as well as reduce the computational cost. While most of the research efforts are focused on how to select the proper features for machine learning, we studied the following important problem in this paper: can the "unimportant features" (low rank features) also provide useful information to improve the overall learning capability? In this paper, we proposed an adaptive iterative learning mechanism based on feature selection and combination voting (AdaFSCV) to address this issue. Unlike the conventional way of discarding the unselected low rank features, we iteratively build classifiers in those feature spaces as well. Such iterative process will adaptively learn information in different feature spaces, and automatically stop when one classify can not provide better information than a random guess. Finally, a probability voting algorithm is proposed to combine all the votes from different classifiers to provide the final prediction results. Simulation results on the MNIST database of handwritten digits show this method can improve the classification accuracy and robustness with certain levels of trade-off of the computational cost.

Temporal sequence learning is one of the most critical components for human intelligence. In this paper, a novel hierarchical structure for complex temporal sequence learning is proposed. Hierarchical organization, a prediction mechanism, and one-shot learning characterize the model. In the lowest level of the hierarchy, we use a modified Hebbian learning mechanism for pattern recognition. Our model employs both active 0 and active 1 sensory inputs. A winner-take-all (WTA) mechanism is used to select active neurons that become the input for sequence learning at higher hierarchical levels. Prediction is an essential element of our temporal sequence learning model. By correct prediction, the machine indicates it knows the current sequence and does not require additional learning. When the prediction is incorrect, one-shot learning is executed and the machine learns the new input sequence as soon as the sequence is completed. A four-level hierarchical structure that isolates letters, words, sentences, and strophes is used in this paper to illustrate the model.