This article introduces a distributed deep neural network (NN)-based adaptive control framework for cooperative object manipulation in human-robot teams with unknown agent dynamics by using three distinct multilayer NN observers (MNNOs). The first observer, termed the reference point estimator, enables each robotic agent to estimate the object's reference center using consensus-based learning, even without direct access to global reference trajectories. The second observer, referred to as the human force-to-trajectory estimator, uses human-applied forces to infer the intended position, velocity, and acceleration of the object, enabling real-time estimation of human intent. Together, these two observers allow distributed estimation of human-intended motion. In addition, a third observer, the distributed NN dynamics observer, is integrated into the control layer to simultaneously estimate the agent's own state and unknown system dynamics while incorporating the state vector of all other agents. Weight update laws for the multilayer NN observers are developed using singular value decomposition (SVD), enabling stable and efficient parameter tuning in multiagent settings. The framework combines the observer estimates with a distributed online multilayer actor-critic NN controller to compute Pareto game theoretic optimal effort that coordinates robot actions while considering neighborhood interactions. Safety is enforced via barrier Lyapunov functions (BLFs) formulated using Karush-Kuhn-Tucker (KKT) conditions, which dynamically adjust safety constraints based on both the agent's own state and its neighbor state vector. Simulation results demonstrate that the proposed approach achieves accurate intent estimation, robust control, and a 60% reduction in total cost compared to baseline methods.

A scalable output feedback control framework utilizing a multilayer neural network (MNN) observer and a critic network via an integral reinforcement learning (IRL) and adaptive dynamic programming (ADP) approach is proposed for a class of nonlinear systems. The observer and critic MNN weight updates are derived using singular value decomposition (SVD) of the MNN activation gradients, measured outputs, and Bellman errors, respectively. The optimal control input is computed based on the critic network weights and the observer-estimated system dynamics. To enable multitask learning and knowledge retention, an online continual learning mechanism is incorporated by introducing a penalty function into the critic MNN updates, relying solely on estimated states in the output feedback setting. The effectiveness of the proposed MNN-based optimal output feedback tracking control is validated through simulations on a two-link robotic manipulator, achieving a 75% performance improvement compared to recent methods in the literature.

This paper presents an explainable deep-reinforcement learning (DRL)-based safety-aware optimal adaptive tracking (SOAT) scheme for a class of nonlinear discrete-time (DT) affine systems subject to state inequality constraints. The DRL-based SOAT utilizes a multilayer neural network (MNN)-based actor-critic to estimate the cost function and optimal policy while the MNN update laws are tuned both using the singular value decomposition (SVD) of activation function gradient in order to mitigate the vanishing gradient issue and safety-aware Bellman error at each layer. An approximate safety-aware optimal policy is developed using Karush-Kuhn–Tucker (KKT) conditions by incorporating the higher-order control barrier function (HOCBF) into the Hamiltonian through the Lagrangian multiplier. The resulting safety-aware Bellman error helps with safe exploration both during online learning phase and at steady state without any explicit actor-critic MNN update law changes. To study the explainability and gain insights, we employ the Shapley Additive Explanations (SHAP) method to construct an explainer model for the DRL-based SOAT scheme in order to identify the important features in determining the optimal policy. The overall stability is established. Finally, the effectiveness of the proposed method is demonstrated on Shipboard Power Systems (SPS), achieving over a 35% reduction in cumulative cost compared to the existing actor-critic MNN optimal control policy. Note to Practitioners—In practical control systems, meeting safety constraints is often critical since ignoring constraints can lead to degraded performance or damage to equipment. This paper addresses the challenge of a safe DRL-based control approach that not only optimizes performance but also integrates robust safety assurances. Our DRL-based SOAT scheme specifically targets nonlinear discrete-time systems that must satisfy state inequality constraints. The successful proposed control performance in simulations on a Shipboard Power System demonstrates the potential for practical applications. DRL-based SOAT employs an MNN with an actor-critic framework for continuous learning and policy adaptation. Integrating HOCBFs directly into the optimization ensures safe operation, even during online learning, which is critical for real-time applications. The addition of SHAP enhances transparency by identifying key features that influence control decisions. Future work could adapt this framework to other constrained environments, such as autonomous vehicles, robotics, and industrial automation, where safety, optimality, and explainability are essential.

In GPS-denied environments or when GPS signals are unreliable or unavailable, alternative methods of accurate localization with coordinate generation become critical. To address localization, the scale-invariant feature transform (SIFT) algorithm, along with its numerous adaptations, is extensively utilized in computer vision and remote sensing for matching image features to identify objects and perform localization. This article presents a novel approach for estimating the relative altitude of unmanned aerial vehicles (UAVs) using SIFT features’ scale (size), omitting the need for additional data like camera intrinsic parameters, as well as extensive image datasets are also required for training. Furthermore, the approach enhances feature matching through the integration of a Siamese network, leveraging the robustness of SIFT features combined with the discriminative power of convolutional neural network (CNN) features. To further improve the performance of the Siamese network, we applied direct error-driven learning (EDL), a learning method that directly adjusts the network’s weights based on the overall error to enhance its ability to differentiate true from false matches, thereby improving the accuracy of the final altitude estimation results. Altitude estimation is achieved by comparing the SIFT features’ size in the UAV image taken from the current position with those in the preestablished reference, ensuring reliability and computational efficiency. The proposed method is computationally efficient, making it suitable for real-time applications and UAVs with limited processing capabilities. It is also highly adaptable, requiring minimal hardware and eliminating the dependency on external sensors. The robustness and effectiveness of this method were validated across four high-resolution UAV image datasets at varying altitudes.

While deep neural networks achieve remarkable visual perception capabilities for UAV position and orientation estimation, their resilience to different weather conditions still needs improvement. These models often suffer from catastrophic forgetting when adapted to new environments, losing previously acquired knowledge. Lifelong learning methods aim to balance learning flexibility and memory stability. In this paper, we present an image-based approach to estimate the relative altitude of a UAV using 2D images under varying weather conditions, including sunny, sunset, and foggy scenarios. Our experiments demonstrate significant performance degradation when the model is trained sequentially on different weather datasets, especially when new images differ substantially from those in the initial training dataset. However, testing Elastic Weight Consolidation (EWC) and Direct Error-Driven Learning (EDL) separately showed that each method helps maintain stability and performance across various weather conditions. Our results show the feasibility and effectiveness of these methods in diverse environmental conditions.

Unmanned Aerial Vehicles (UAVs) have become indispensable in various applications, including surveillance, urban scene analysis, and agricultural monitoring. Accurate altitude estimation is critical for UAV operations, especially in environments where traditional sensors like GPS, pressure altimeters, and radar may fail. This paper explores the use of infrared and thermal imaging for relative altitude estimation of UAVs, highlighting their significant advantages over traditional RGB images. Infrared and thermal imaging offer superior performance in low-light and adverse weather conditions, providing clearer visibility and more reliable feature detection. By leveraging the Scale-Invariant Feature Transform (SIFT) features, this approach utilizes the inherent benefits of thermal images to estimate altitude changes based on the size variations of matched keypoints in consecutive images. Experimental results on two infrared thermal UAV datasets demonstrate the effectiveness of this approach, showing substantial improvements in estimation accuracy when combined with Siamese networks for enhanced feature matching.

Blockchain systems have been successful in discerning truthful information from interagent interaction amidst possible attackers or conflicts, which is crucial for the completion of nontrivial tasks in distributed networking. However, the state-of-the-art blockchain protocols are limited to resource-rich applications where reliably connected nodes within the network are equipped with significant computing power to run lottery-based Proof-of-Work (PoW) consensus. The purpose of this work is to address these challenges for implementation in a severely resource-constrained distributed network with Internet of Things (IoT) devices. The contribution of this work is a novel lightweight alternative, called weight-based reputation (WBR) scheme, to classify new transactions via modeling blockchain decisions as a distributed machine-learning task. WBR identifies network nodes that are willing to cooperate toward securing ground truth, showing robustness to adversarial subnetworks that are greater than 50% and reducing collaboration error by 50% compared to other similar schemes. This two-step approach of reputation plus transaction classification for generating blockchain data is treated as a novel method of preventing fraud and double-spending attacks in blockchain networks. To capture adversary influence, a Bayesian game is formulated and implemented to show superior performance to the state-of-the-art along with resource consumption metrics.

This article presents a novel lifelong integral reinforcement learning (LIRL)-based optimal trajectory tracking scheme using the multilayer (MNN) or deep neural network (Deep NN) for the uncertain nonlinear continuous-time (CT) affine systems subject to state constraints. A critic MNN, which approximates the value function, and a second NN identifier are together used to generate the optimal control policies. The weights of the critic MNN are tuned online using a novel singular value decomposition (SVD)-based method, which can be extended to MNN with the N-hidden layers. Moreover, an online lifelong learning (LL) scheme is incorporated with the critic MNN to mitigate the problem of catastrophic forgetting in the multitasking systems. Additionally, the proposed optimal framework addresses state constraints by utilizing a time-varying barrier function (TVBF). The uniform ultimate boundedness (UUB) of the overall closed-loop system is shown using the Lyapunov stability analysis. A two-link robotic manipulator that compares to recent literature shows a 47% total cost reduction, demonstrating the effectiveness of the proposed method.

Decision-making and consensus in traditional blockchain protocols is formulated as a repeated Bernoulli trial that solves a computationally-intense lottery puzzle, called Proof-of-Work (PoW) in Bitcoin. This approach has shown robustness through practice, but does not scale with increasing network size and generation of new transactions. Resource constrained Internet of Things (IoT) networks are incompatible with full computation of schemes like Bitcoin's PoW. Our effort proposes a first step towards an alternative consensus using machine learning-based decision-making with prediction of fraud transactions to alleviate need for intense computation. To improve base approval probabilities for fraud detection in an ideal security setting, Vector GAN (VecGAN) is proposed to augment blockchain data in classifier training, which combines error-driven learning with Bayesian estimation to alleviate calculations. This two-step approach with augmentation and classification on new transactions is proposed as a novel approach to blockchain decision-making. Experimental prediction accuracy using VecGAN improved up to 3% on simplistic classifiers compared to other state-of-the-art augmentation techniques. Resource consumption in a realistic blockchain setting was reduced while improving block throughput by 50% compared to PoW. Future work will explore Sybil-spam defensive measures for realistic protocol implementation with this approach.

This article addresses a multilayer neural network (MNN)-based optimal adaptive tracking of partially uncertain nonlinear discrete-time (DT) systems in affine form. By employing an actor-critic neural network (NN) to approximate the value function and optimal control policy, the critic NN is updated via a novel hybrid learning scheme, where its weights are adjusted once at a sampling instant and also in a finite iterative manner within the instants to enhance the convergence rate. Moreover, to deal with the persistency of excitation (PE) condition, a replay buffer is incorporated into the critic update law through concurrent learning. To address the vanishing gradient issue, the actor and critic MNN weights are tuned using control input and temporal difference errors (TDEs), respectively. In addition, a weight consolidation scheme is incorporated into the critic MNN update law to attain lifelong learning and overcome catastrophic forgetting, thus lowering the cumulative cost. The tracking error, and the actor and critic weight estimation errors are shown to be bounded using the Lyapunov analysis. Simulation results using the proposed approach on a two-link robot manipulator show a significant reduction in tracking error by 44% and cumulative cost by 31% in a multitask environment.

In this article, we address two key challenges in deep reinforcement learning (DRL) setting, sample inefficiency and slow learning, with a dual-neural network (NN)-driven learning approach. In the proposed approach, we use two deep NNs with independent initialization to robustly approximate the action-value function in the presence of image inputs. In particular, we develop a temporal difference (TD) error-driven learning (EDL) approach, where we introduce a set of linear transformations of the TD error to directly update the parameters of each layer in the deep NN. We demonstrate theoretically that the cost minimized by the EDL regime is an approximation of the empirical cost, and the approximation error reduces as learning progresses, irrespective of the size of the network. Using simulation analysis, we show that the proposed methods enable faster learning and convergence and require reduced buffer size (thereby increasing the sample efficiency).

Recently introduced deep reinforcement learning (DRL) techniques in discrete-time have resulted in significant advances in online games, robotics, and so on. Inspired from recent developments, we have proposed an approach referred to as Quantile Critic with Spiking Actor and Normalized Ensemble (QC_SANE) for continuous control problems, which uses quantile loss to train critic and a spiking neural network (NN) to train an ensemble of actors. The NN does an internal normalization using a scaled exponential linear unit (SELU) activation function and ensures robustness. The empirical study on multijoint dynamics with contact (MuJoCo)-based environments shows improved training and test results than the state-of-the-art approach: population coded spiking actor network (PopSAN).

In this article, an actor-critic neural network (NN)-based online optimal adaptive regulation of a class of nonlinear continuous-time systems with known state and input delays and uncertain system dynamics is introduced. The temporal difference error (TDE), which is dependent upon state and input delays, is derived using actual and estimated value function and via integral reinforcement learning. The NN weights of the critic are tuned at every sampling instant as a function of the instantaneous integral TDE. A novel identifier, which is introduced to estimate the control coefficient matrices, is utilized to obtain the estimated control policy. The boundedness of the state vector, critic NN weights, identification error, and NN identifier weights are shown through the Lyapunov analysis. Simulation results are provided to illustrate the effectiveness of the proposed approach.

This study provides a novel reinforcement learning-based optimal tracking control of partially uncertain nonlinear discrete-time (DT) systems with state constraints using zero-sum game (ZSG) formulation. To address optimal tracking, a novel augmented system consisting of tracking error and its integral value, along with an uncertain desired trajectory, is constructed. A barrier function (BF) with a tradeoff factor is incorporated into the cost function to keep the state trajectories to remain within a compact set and to balance safety with optimality. Next, by using the modified value functional, the ZSG formulation is introduced wherein an actor-critic neural network (NN) framework is employed to approximate the value functional, optimal control input, and worst disturbance. The critic NN weights are tuned once at the sample instants and then iteratively within sampling instants. Using control input errors, the actor NN weights are adjusted once a sampling instant. The concurrent learning term in the critic weight tuning law overcomes the need for the persistency excitation (PE) condition. Further, a weight consolidation scheme is incorporated into the critic update law to attain lifelong learning by overcoming catastrophic forgetting. Finally, a numerical example supports the analytical claims.

This article proposes an event-triggered optimal adaptive output-feedback control design approach by utilizing integral reinforcement learning (IRL) for linear time-invariant systems with state delay and uncertain internal dynamics. In the proposed approach, the general optimal control problem is formulated into the game-theoretic framework by treating the event-triggering threshold and the optimal control policy as players. A cost function is defined and a value functional, which includes the delayed system output, is considered. First, by using the value functional and applying stationarity conditions using the Hamiltonian function, the output game delay algebraic Riccati equation (OGDARE) and optimal control policy are derived when the internal system dynamics are available. Then to relax the knowledge of internal dynamics, a hybrid learning scheme using measured output is proposed for tuning the value function parameters, which in turn is employed to compute the estimated optimal control policy. The overall closed-loop system is shown to be asymptotically stable by selecting an appropriate event-triggering condition when the dynamics of the system are both known and partially uncertain. A simulation example is given to substantiate the efficacy of the theoretical claims.

The goal of this work is to develop the next generation of coordinated optimal planning and control schemes for real-world robotic applications, with cost-effective intelligent robots that can safely and robustly perform the tasks at hand. This article presents a dual-loop optimal hierarchical control scheme for robotic manipulators consisting of outer and inner loops. The outer kinematic control loop in the operational space provides a joint velocity reference signal to the inner one. The kinematic control is formulated as a closed-loop optimal control approach to trajectory generation where the desired target (possibly time-varying) is obtained by acting upon the feedback from the actual state of the robot. The kinematic control is obtained using a closed-form analytic solution of the Hamilton–Jacobi–Bellman (HJB) equation. The proposed methodology defines the task in terms of the integral cost function, which results in a global optimal solution. The inner dynamic control loop consists of a neural network (NN)-based adaptive critic (AC) optimal tracking control scheme. The online NN approximator-based dynamic controller learns the infinite-horizon cost function related to inner loop error dynamics in continuous time and calculates the corresponding optimal control input to minimize the cost function forward in time. The stability and the performance of the proposed control scheme are shown theoretically via the Lyapunov approach and also verified experimentally using a 7-DOF Barrett WAM robot manipulator. The warehousing applications of the proposed dual-loop control scheme have been demonstrated experimentally in an exact warehouse setting.Note to Practitioners—This article was motivated by our previous experiences in the Amazon Robotics Challenge (ARC) competitions as a participant. Although the major lessons from those events revealed many important issues toward warehouse automation both from robotic vision and grasping aspects, the application aspects of control engineering to solution for these real-world problems could significantly benefit the today’s highly automated society. The intention of this article is to address this situation by designing a cost-effective robot manipulator control scheme by optimizing robot manipulation trajectories in the outer loop along with the actuator input in the inner loop. Therefore, we propose a dual-loop optimal control scheme for robotic manipulations under a cost-optimal framework with guaranteed stability proof via the Lyapunov approach. This results in smoother trajectories with cost-effective control actuator inputs than the state-of-the-art methods. We believe that this work can be beneficial for academic and industrial research to design more advanced and optimized solutions in this domain.

In this paper, a novel approach is presented to mitigate the issue of domain shift observed in big-data classification. Since little information is available about the shift, we introduce a "distortion model", and obtain additional data-samples to represent the shift. Next, a deep neural network (DNN), referred as "classifier," is used to compensate for the shift by learning through these additional samples while maintaining performance on training samples. As the exact magnitude of domain shift is uncertain, we compensate for the optimal expected shift by formulating a zero-sum game. In the proposed game, the distortion model is viewed as the maximizing player which increases the domain shift while the classifier becomes the minimizing player that reduces the impact of domain shift on learning. The Nash solution of the game, which is demonstrated mathematically, provides the domain shift and its optimal adaptation through the classifier. To solve the proposed game for the Nash solution, a direct error-driven learning scheme is introduced where a cost function is derived and solved for each layer in the DNN and the distortion model. Comprehensive mathematical and simulation study is presented to demonstrate the efficacy of the approach.

In this article, we introduce a novel approximate optimal decentralized control scheme for uncertain input-affine nonlinear-interconnected systems. In the proposed scheme, we design a controller and an event-triggering mechanism (ETM) at each subsystem to optimize a local performance index and reduce redundant control updates, respectively. To this end, we formulate a noncooperative dynamic game at every subsystem in which we collectively model the interconnection inputs and the event-triggering error as adversarial players that deteriorate the subsystem performance and model the control policy as the performance optimizer, competing against these adversarial players. To obtain a solution to this game, one has to solve the associated Hamilton-Jacobi-Isaac (HJI) equation, which does not have a closed-form solution even when the subsystem dynamics are accurately known. In this context, we introduce an event-driven off-policy integral reinforcement learning (OIRL) approach to learn an approximate solution to this HJI equation using artificial neural networks (NNs). We then use this NN approximated solution to design the control policy and event-triggering threshold at each subsystem. In the learning framework, we guarantee the Zeno-free behavior of the ETMs at each subsystem using the exploration policies. Finally, we derive sufficient conditions to guarantee uniform ultimate bounded regulation of the controlled system states and demonstrate the efficacy of the proposed framework with numerical examples.

This article investigates adaptive robust controller design for discrete-time (DT) affine nonlinear systems using an adaptive dynamic programming. A novel adaptive interleaved reinforcement learning algorithm is developed for finding a robust controller of DT affine nonlinear systems subject to matched or unmatched uncertainties. To this end, the robust control problem is converted into the optimal control problem for nominal systems by selecting an appropriate utility function. The performance evaluation and control policy update combined with neural networks approximation are alternately implemented at each time step for solving a simplified Hamilton-Jacobi-Bellman (HJB) equation such that the uniformly ultimately bounded (UUB) stability of DT affine nonlinear systems can be guaranteed, allowing for all realization of unknown bounded uncertainties. The rigorously theoretical proofs of convergence of the proposed interleaved RL algorithm and UUB stability of uncertain systems are provided. Simulation results are given to verify the effectiveness of the proposed method.

This article intends to address an online optimal adaptive regulation of nonlinear discrete-time systems in affine form and with partially uncertain dynamics using a multilayer neural network (MNN). The actor-critic framework estimates both the optimal control input and value function. Instantaneous control input error and temporal difference are used to tune the weights of the critic and actor networks, respectively. The selection of the basis functions and their derivatives are not required in the proposed approach. The state vector, critic, and actor NN weights are proven to be bounded using the Lyapunov method. Our approach can be extended to neural networks with an arbitrary number of hidden layers. We have demonstrated our approach via a simulation example.

In this article, a novel learning methodology is introduced for the problem of classification in the context of high-dimensional data. In particular, the challenges introduced by high-dimensional data sets are addressed by formulating a$L_{1}$regularized zero-sum game where optimal sparsity is estimated through a two-player game between the penalty coefficients/sparsity parameters and the deep neural network weights. In order to solve this game, a distributed learning methodology is proposed where additional variables are utilized to derive layerwise cost functions. Finally, an alternating minimization approach developed to solve the problem where the Nash solution provides optimal sparsity and compensation through the classifier. The proposed learning approach is implemented in a parallel and distributed environment through a novel computational algorithm. The efficiency of the approach is demonstrated both theoretically and empirically with nine data sets.

In this paper, a multi-layer neural network (MNN) based online optimal adaptive regulation of a class of nonlinear discrete-time systems in affine form with uncertain internal dynamics is introduced. The multi-layer neural networks (MNN)-based actor-critic framework is utilized to estimate the optimal control input and cost function. The temporal difference (TD) error is derived from the difference between actual and estimated cost function. The MNN weights of both critic and actor are tuned at every sampling instant as a function of the instantaneous temporal difference and control policy errors. The proposed approach does not require the selection of any basis function and its derivatives. The boundedness of the system state vector and actor and critic NN weights are shown through Lyapunov theory. Extension of the proposed approach to MNNs with more hidden layers is discussed. Simulation results are provided to illustrate the effectiveness of the proposed approach.

In this brief, heterogeneity and noise in big data are shown to increase the generalization error for a traditional learning regime utilized for deep neural networks (deep NNs). To reduce this error, while overcoming the issue of vanishing gradients, a direct error-driven learning (EDL) scheme is proposed. First, to reduce the impact of heterogeneity and data noise, the concept of a neighborhood is introduced. Using this neighborhood, an approximation of generalization error is obtained and an overall error, comprised of learning and the approximate generalization errors, is defined. A novel NN weight-tuning law is obtained through a layer-wise performance measure enabling the direct use of overall error for learning. Additional constraints are introduced into the layer-wise performance measure to guide and improve the learning process in the presence of noisy dimensions. The proposed direct EDL scheme effectively addresses the issue of heterogeneity and noise while mitigating vanishing gradients and noisy dimensions. A comprehensive simulation study is presented where the proposed approach is shown to mitigate the vanishing gradient problem while improving generalization by 6%.

In networked control systems (NCS), a certain class of attacks on the communication network is known to raise traffic flows causing delays and packet losses to increase. This paper presents a novel neural network (NN)-based attack detection and estimation scheme that captures the abnormal traffic flow due to a class of attacks on the communication links within the feedback loop of an NCS. By modeling the unknown network flow as a nonlinear function at the bottleneck node and using a NN observer, the network attack detection residual is defined and utilized to determine the onset of an attack in the communication network when the residual exceeds a predefined threshold. Upon detection, another NN is used to estimate the flow injected by the attack. For the physical system, we develop an attack detection scheme by using an adaptive dynamic programming-based optimal event-triggered NN controller in the presence of network delays and packet losses. Attacks on the network as well as on the sensors of the physical system can be detected and estimated with the proposed scheme. The simulation results confirm theoretical conclusions.

This paper presents novel cooperative tracking control for a class of input-constrained multiagent systems with a dynamic leader. Each follower agent is described by a high-order nonlinear dynamics in strict feedback form with input constraints. Our main contribution lies in presenting a system transformation method that can convert the input-constrained state feedback cooperative tracking control of agents into an unconstrained output feedback control of agents with dynamics in Brunovsky normal form. As a result, the original problem is simplified to be a simple stabilization of the transformed system for the agents. Thus, the use of the backstepping scheme is obviated, and the synthesis and computation are extremely simplified. It is strictly proved that all follower agents can synchronize to the leader with bounded synchronization errors, and all other signals in the closed-loop system are semi-global uniformly ultimately bounded. Finally, numerical analysis is carried out to validate the theoretical results and demonstrate the effectiveness of the proposed approach.

In this paper, a novel dimension-reduction approach is presented to overcome challenges such as nonlinear relationships, heterogeneity, and noisy dimensions. Initially, the p attributes in the data are first organized into random groups. Next, to systematically remove redundant and noisy dimensions from the data, each group is independently mapped into a low dimensional space via a parametric mapping. The group-wise transformation parameters are estimated using a low-rank approximation of distance covariance. The transformed attributes are reorganized into groups based on the magnitude of their respective eigenvalues. The group-wise organization and reduction process is performed until a user-defined criterion on eigenvalues is satisfied. In addition, novel procedures are introduced to aggregate the transformation parameters when the data is available in batches. Overall performance is demonstrated with extensive simulation analysis on classification by employing 10 data-sets.

In this paper, adaptive neural networks (NNs) are employed in the event-triggered feedback control framework to enable a robot manipulator to track a predefined trajectory. In the proposed output feedback control scheme, the joint velocities of the robot manipulator are reconstructed using a nonlinear NN observer by using the joint position measurements. Two different configurations are proposed for the implementation of the controller depending on whether the observer is co-located with the sensor or the controller in the feedback control loop. Besides the observer NN, a second NN is utilized to compensate the effects of nonlinearities in the robot dynamics via the feedback control. For both the configurations, by utilizing observer NN and the second NN, torque input is computed by the controller. The Lyapunov stability method is employed to determine the event-triggering condition, weight update rules for the controller, and the observer for both the configurations. The tracking performance of the robot manipulator with the two configurations is analyzed, wherein it is demonstrated that all the signals in the closed-loop system composed of the robotic system, the observer, the event-sampling mechanism, and the controller are locally uniformly ultimately bounded in the presence of bounded disturbance torque. To demonstrate the efficacy of the proposed design, simulation results are presented.

In this paper, approximate optimal distributed control schemes for a class of nonlinear interconnected systems with strong interconnections are presented using continuous and event-sampled feedback information. The optimal control design is formulated as an N -player nonzero-sum game where the control policies of the subsystems act as players. An approximate Nash equilibrium solution to the game, which is the solution to the coupled Hamilton-Jacobi equation, is obtained using the approximate dynamic programming-based approach. A critic neural network (NN) at each subsystem is utilized to approximate the Nash solution and novel event-sampling conditions, that are decentralized, are designed to asynchronously orchestrate the sampling and transmission of state vector at each subsystem. To ensure the local ultimate boundedness of the closed-loop system state and NN parameter estimation errors, a hybrid-learning scheme is introduced and the stability is guaranteed using Lyapunov-based stability analysis. Finally, implementation of the proposed event-based distributed control scheme for linear interconnected systems is discussed. For completeness, Zeno-free behavior of the event-sampled system is shown analytically and a numerical example is included to support the analytical results.

While analyzing high dimensional data-sets using deep neural network (NN), increased sparsity is desirable but requires careful selection of "sparsity parameters." In this paper, a novel distributed learning methodology is proposed to optimize the NN while addressing this challenge. To address this challenge, the optimal sparsity in the NN is estimated via a two player zero-sum game in the paper. In the proposed game, sparsity parameter is the first player with the aim of increasing sparsity in the NN while NN weights is the second player with the goal of improving its performance in the presence of increased sparsity. To solve the game, additional variables are introduced into the optimization problem such that the output at every layer in the NN depends on this variable instead of the previous layer. Using these additional variables, layer wise cost-functions are derived that are then independently optimized to learn the additional variables, NN weights and the sparsity parameters. To implement the proposed learning procedure in a parallelized and distributed environment, a novel computational algorithm is also proposed. The efficiency of the proposed approach is demonstrated using a total of six data-sets.

In this paper, a novel methodology to reduce the generalization errors occurring due to domain shift in big data classification is presented. This reduction is achieved by introducing a suitably selected domain shift to the training data via what is referred to as "distortion model". These distortions are introduced through an affine transformation and additional data-samples are obtained. Next, a deep neural network (NN), referred as "classifier", is used to classify both the original and the additional data samples. By learning from both the original and additional data-samples, the classifier compensates for the domain shift while maintaining its performance on original data. However, as the exact magnitude of the shift one would encounter in real applications is unknown a priori and difficult to predict. The objective is to compensate for the optimal shift that can be introduced by the distortion model without significantly degrading the performance of the model. A two-player zero-sum game is thus designed where the first player is the distortion model with the aim of increasing the domain shift. The classifier then becomes the second player whose aim is to minimize the impact of domain shift. Finally, a direct error-driven learning scheme is utilized to minimize the impact of the classifier while maximizing the domain shift. A comprehensive simulation study is presented where a 12% improvement in the presence of domain shift is demonstrated. The proposed approach is also shown to improve generalization by 6%.

In this paper, a novel adaptive control strategy is presented for the tracking control of a class of multi-input-multioutput uncertain nonlinear systems with external disturbances to place user-defined time-varying constraints on the system state. Our contribution includes a step forward beyond the usual stabilization result to show that the states of the plant converge asymptotically, as well as remain within user-defined time-varying bounds. To achieve the new results, an error transformation technique is first established to generate an equivalent nonlinear system from the original one, whose asymptotic stability guarantees both the satisfaction of the time-varying restrictions and the asymptotic tracking performance of the original system. The uncertainties of the transformed system are overcome by an online neural network (NN) approximator, while the external disturbances and NN reconstruction error are compensated by the robust integral of the sign of the error signal. Via standard Lyapunov method, asymptotic tracking performance is theoretically guaranteed, and all the closed-loop signals are bounded. The requirement for a prior knowledge of bounds of uncertain terms is relaxed. Finally, simulation results demonstrate the merits of the proposed controller.

In this paper, a distributed control scheme for an interconnected system composed of uncertain input affine nonlinear subsystems with event triggered state feedback is presented by using a novel hybrid learning scheme-based approximate dynamic programming with online exploration. First, an approximate solution to the Hamilton-Jacobi-Bellman equation is generated with event sampled neural network (NN) approximation and subsequently, a near optimal control policy for each subsystem is derived. Artificial NNs are utilized as function approximators to develop a suite of identifiers and learn the dynamics of each subsystem. The NN weight tuning rules for the identifier and event-triggering condition are derived using Lyapunov stability theory. Taking into account, the effects of NN approximation of system dynamics and boot-strapping, a novel NN weight update is presented to approximate the optimal value function. Finally, a novel strategy to incorporate exploration in online control framework, using identifiers, is introduced to reduce the overall cost at the expense of additional computations during the initial online learning phase. System states and the NN weight estimation errors are regulated and local uniformly ultimately bounded results are achieved. The analytical results are substantiated using simulation studies.

This paper presents an approximate optimal distributed control scheme for a known interconnected system composed of input affine nonlinear subsystems using event-triggered state and output feedback via a novel hybrid learning scheme. First, the cost function for the overall system is redefined as the sum of cost functions of individual subsystems. A distributed optimal control policy for the interconnected system is developed using the optimal value function of each subsystem. To generate the optimal control policy, forward-in-time, neural networks are employed to reconstruct the unknown optimal value function at each subsystem online. In order to retain the advantages of event-triggered feedback for an adaptive optimal controller, a novel hybrid learning scheme is proposed to reduce the convergence time for the learning algorithm. The development is based on the observation that, in the event-triggered feedback, the sampling instants are dynamic and results in variable interevent time. To relax the requirement of entire state measurements, an extended nonlinear observer is designed at each subsystem to recover the system internal states from the measurable feedback. Using a Lyapunov-based analysis, it is demonstrated that the system states and the observer errors remain locally uniformly ultimately bounded and the control policy converges to a neighborhood of the optimal policy. Simulation results are presented to demonstrate the performance of the developed controller.

In this paper, a novel event-triggered implementation of a tracking controller for an uncertain strict-feedback system is presented. Neural networks (NNs) are utilized in the backstepping approach to design a control input by approximating unknown dynamics of the strict-feedback nonlinear system with event-sampled inputs. The system state vector is assumed to be unknown and an NN observer is used to estimate the state vector. By using the estimated state vector and backstepping design approach, an event-sampled controller is introduced. As part of the controller design, first, input-to-state-like stability for a continuously sampled controller that has been injected with bounded measurement errors is demonstrated, and subsequently, an event-execution control law is derived, such that the measurement errors are guaranteed to remain bounded. Lyapunov theory is used to demonstrate that the tracking errors, the observer estimation errors, and the NN weight estimation errors for each NN are locally uniformly ultimately bounded in the presence bounded disturbances, NN reconstruction errors, as well as errors introduced by event sampling. Simulation results are provided to illustrate the effectiveness of the proposed controllers.

This paper develops a near optimal boundary control method for distributed parameter systems governed by uncertain linear 1-D parabolic partial differential equations (PDE) by using approximate dynamic programming. A quadratic surface integral is proposed to express the optimal cost functional for the infinite-dimensional state space. Accordingly, the Hamilton-Jacobi-Bellman (HJB) equation is formulated in the infinite-dimensional domain without using any model reduction. Subsequently, a neural network identifier is developed to estimate the unknown spatially varying coefficient in PDE dynamics. Novel tuning law is proposed to guarantee the boundedness of identifier approximation error in the PDE domain. A radial basis network (RBN) is subsequently proposed to generate an approximate solution for the optimal surface kernel function online. The tuning law for near optimal RBN weights is created, such that the HJB equation error is minimized while the dynamics are identified and closed-loop system remains stable. Ultimate boundedness (UB) of the closed-loop system is verified by using the Lyapunov theory. The performance of the proposed controller is successfully confirmed by simulation on an unstable diffusion-reaction process.

In this paper, neurodynamic programming-based output feedback boundary control of distributed parameter systems governed by uncertain coupled semilinear parabolic partial differential equations (PDEs) under Neumann or Dirichlet boundary control conditions is introduced. First, Hamilton-Jacobi-Bellman (HJB) equation is formulated in the original PDE domain and the optimal control policy is derived using the value functional as the solution of the HJB equation. Subsequently, a novel observer is developed to estimate the system states given the uncertain nonlinearity in PDE dynamics and measured outputs. Consequently, the suboptimal boundary control policy is obtained by forward-in-time estimation of the value functional using a neural network (NN)-based online approximator and estimated state vector obtained from the NN observer. Novel adaptive tuning laws in continuous time are proposed for learning the value functional online to satisfy the HJB equation along system trajectories while ensuring the closed-loop stability. Local uniformly ultimate boundedness of the closed-loop system is verified by using Lyapunov theory. The performance of the proposed controller is verified via simulation on an unstable coupled diffusion reaction process.

In this paper, an adaptive dynamic programming-based near optimal boundary controller is developed for partial differential equations (PDEs) modeled by the uncertain Burgers' equation under Neumann boundary condition in 2-D. Initially, Hamilton-Jacobi-Bellman equation is derived in infinite-dimensional space. Subsequently, a novel neural network (NN) identifier is introduced to approximate the nonlinear dynamics in the 2-D PDE. The optimal control input is derived by online estimation of the value function through an additional NN-based forward-in-time estimation and approximated dynamic model. Novel update laws are developed for estimation of the identifier and value function online. The designed control policy can be applied using a finite number of actuators at the boundaries. Local ultimate boundedness of the closed-loop system is studied in detail using Lyapunov theory. Simulation results confirm the optimizing performance of the proposed controller on an unstable 2-D Burgers' equation.

In this paper, the relevance of deep neural network (DNN) is studied in big data scenarios, specifically for prognostics applications. It is observed that fault predictions can be performed more efficiently when DNN is used with a pre-processing step. A novel hierarchical dimension reduction (HDR) approach is therefore proposed as a pre-processing step to DNN. This two-step approach is shown to be effective in extracting value from complex and uncertain big data. It is shown that use of HDR prior to DNN improves convergence and allows for the possibility of reduction in model size without any drop in accuracy. A comprehensive methodology is developed to facilitate prognostics using DNN. Simulation results are included to demonstrate the overall methodology using big data-sets.

This paper introduces an online reinforcement learning scheme with exploration for distributed approximate optimal control of uncertain nonlinear interconnected system. The subsystem, interconnection dynamics and input gain matrix are approximated using neural network (NN) identifiers with event-based state feedback. A second NN is designed at each subsystem to construct the mapping of states to future reward prediction via reinforcement signals with which a sequence of approximately optimal distributed control actions are generated. Since the identifiers and the controllers at each subsystem requires the local and other subsystem state vector with non-zero interconnections, a decentralized event-triggering mechanism using Lyapunov theory is developed to dynamically determine the feedback instants so as to reduce the communication overhead. Further, a novel strategy to incorporate exploration in the online control framework using identifiers is proposed to minimize the overall cost during the learning phase. The effects of network delay are discussed and finally, simulation results are presented to verify the effectiveness of the proposed controller.

In this paper, an event-driven stochastic adaptive dynamic programming (ADP)-based technique is introduced for nonlinear systems with a communication network within its feedback loop. A near optimal control policy is designed using an actor-critic framework and ADP with event sampled state vector. First, the system dynamics are approximated by using a novel neural network (NN) identifier with event sampled state vector. The optimal control policy is generated via an actor NN by using the NN identifier and value function approximated by a critic NN through ADP. The stochastic NN identifier, actor, and critic NN weights are tuned at the event sampled instants leading to aperiodic weight tuning laws. Above all, an adaptive event sampling condition based on estimated NN weights is designed by using the Lyapunov technique to ensure ultimate boundedness of all the closed-loop signals along with the approximation accuracy. The net result is event-driven stochastic ADP technique that can significantly reduce the computation and network transmissions. Finally, the analytical design is substantiated with simulation results.

This paper presents an approximate optimal control of nonlinear continuous-time systems in affine form by using the adaptive dynamic programming (ADP) with event-sampled state and input vectors. The knowledge of the system dynamics is relaxed by using a neural network (NN) identifier with event-sampled inputs. The value function, which becomes an approximate solution to the Hamilton-Jacobi-Bellman equation, is generated by using event-sampled NN approximator. Subsequently, the NN identifier and the approximated value function are utilized to obtain the optimal control policy. Both the identifier and value function approximator weights are tuned only at the event-sampled instants leading to an aperiodic update scheme. A novel adaptive event sampling condition is designed to determine the sampling instants, such that the approximation accuracy and the stability are maintained. A positive lower bound on the minimum inter-sample time is guaranteed to avoid accumulation point, and the dependence of inter-sample time upon the NN weight estimates is analyzed. A local ultimate boundedness of the resulting nonlinear impulsive dynamical closed-loop system is shown. Finally, a numerical example is utilized to evaluate the performance of the near-optimal design. The net result is the design of an event-sampled ADP-based controller for nonlinear continuous-time systems.

An adaptive consensus algorithm is proposed for a class of nonlinear multiagent systems with completely unknown agent dynamics. Due to uncertainties in the agent's dynamics, previous consensus approaches usually yield uniformly ultimately bounded consensus error. Our main contribution includes a novel robust consensus algorithm which can guarantee that the consensus error converges to zero asymptotically. In order to address the unknown dynamics, a two-layer neural network (NN) is utilized to learn the unknown dynamics in an online manner, and a robust continuous term is introduced to alleviate effects of the NN residual reconstruction error and external disturbances. The continuousness of the control signal is guaranteed to remove the actuator bandwidth requirement and avoid the caused chattering phenomenon. The proposed consensus algorithm is distributed in the sense that each agent only exchanges information with its neighbor agents. The asymptotic consensus result is achieved via Lyapunov synthesis. Furthermore, the proposed algorithm can also be extended to the case where the agents are required to form a prescribed formation. Finally, simulation studies on a nonlinear multiagent system are provided to demonstrate the performance of the scheme.

Event based sampling of feedback signals and control inputs are shown to reduce computations. In this paper, the design of event-sampled adaptive neural network (NN) state feedback control of robot manipulators is presented in the presence of uncertain robot dynamics. The event-sampled NN approximation property is utilized to represent the uncertain nonlinear dynamics of the robotic manipulator which is subsequently employed to generate the control torque. A novel weight tuning rule is designed using the Lyapunov method. Further, the Lyapunov stability theory is utilized to develop the event-sampling condition and to demonstrate the tracking performance of the robot manipulator. Finally, simulation results are presented to verify the theoretical claims and to demonstrate the reduction in the computations with event-sampled control execution.

The communication links in networked control systems are vulnerable to various malicious attacks. In this paper, we propose a novel network attack detection scheme that is able to capture the abnormality in the traffic flow caused by a class of attacks targeting at the communication links. We model the network traffic flow in the bottleneck as a nonlinear system with unknown dynamics. By utilizing an observer, network attack detection residual is generated which is used to determine the existence of attacks in the networks when the residual exceeds a predefined threshold. We also revisit an optimal event-triggered controller for the physical system and derive the maximum delay and packet loss that the system can tolerate.

Providing security in passive RFID systems has gained significant attention due to their widespread use. Research has focused on providing both location and data privacy through mutual authentication between the readers and tags. In such systems, each party is responsible of verifying the identity of the other party with whom it is communicating. For such a task to succeed, the tags and readers are initialized with shared secret information which is updated after a successful authentication session. Ownership management, which includes transfer and delegation, builds upon mutual authentication. Here, the use of security in RFID is extended to encompass the more practical case where a tagged item is shifted from one owner to another. As such, we propose a new authentication and ownership management protocol that is compliant with the EPC Class-1 Generation-2 Version 2 standard. The protocol is formally analyzed and successfully implemented on hardware. The implementation shows that the use of such protocol adds security with little added overhead in terms of communication and computation.

This paper presents a novel adaptive neural network (NN) control of single-input and single-output uncertain nonlinear discrete-time systems under event sampled NN inputs. In this control scheme, the feedback signals are transmitted, and the NN weights are tuned in an aperiodic manner at the event sampled instants. After reviewing the NN approximation property with event sampled inputs, an adaptive state estimator (SE), consisting of linearly parameterized NNs, is utilized to approximate the unknown system dynamics in an event sampled context. The SE is viewed as a model and its approximated dynamics and the state vector, during any two events, are utilized for the event-triggered controller design. An adaptive event-trigger condition is derived by using both the estimated NN weights and a dead-zone operator to determine the event sampling instants. This condition both facilitates the NN approximation and reduces the transmission of feedback signals. The ultimate boundedness of both the NN weight estimation error and the system state vector is demonstrated through the Lyapunov approach. As expected, during an initial online learning phase, events are observed more frequently. Over time with the convergence of the NN weights, the inter-event times increase, thereby lowering the number of triggered events. These claims are illustrated through the simulation results.

This paper presents a novel approximation-based event-triggered control of multi-input multi-output uncertain nonlinear continuous-time systems in affine form. The controller is approximated using a linearly parameterized neural network (NN) in the context of event-based sampling. After revisiting the NN approximation property in the context of event-based sampling, an event-triggered condition is proposed using the Lyapunov technique to reduce the network resource utilization and to generate the required number of events for the NN approximation. In addition, a novel weight update law for aperiodic tuning of the NN weights at triggered instants is proposed to relax the knowledge of complete system dynamics and to reduce the computation when compared with the traditional NN-based control. Nonetheless, a nonzero positive lower bound for the inter-event times is guaranteed to avoid the accumulation of events or Zeno behavior. For analyzing the stability, the event-triggered system is modeled as a nonlinear impulsive dynamical system and the Lyapunov technique is used to show local ultimate boundedness of all signals. Furthermore, in order to overcome the unnecessary triggered events when the system states are inside the ultimate bound, a dead-zone operator is used to reset the event-trigger errors to zero. Finally, the analytical design is substantiated with numerical results.

This paper presents an event-triggered near optimal control of uncertain nonlinear discrete-time systems. Event-driven neurodynamic programming (NDP) is utilized to design the control policy. A neural network (NN)-based identifier, with event-based state and input vectors, is utilized to learn the system dynamics. An actor-critic framework is used to learn the cost function and the optimal control input. The NN weights of the identifier, the critic, and the actor NNs are tuned aperiodically once every triggered instant. An adaptive event-trigger condition to decide the trigger instants is derived. Thus, a suitable number of events are generated to ensure a desired accuracy of approximation. A near optimal performance is achieved without using value and/or policy iterations. A detailed analysis of nontrivial inter-event times with an explicit formula to show the reduction in computation is also derived. The Lyapunov technique is used in conjunction with the event-trigger condition to guarantee the ultimate boundedness of the closed-loop system. The simulation results are included to verify the performance of the controller. The net result is the development of event-driven NDP.

The stochastic optimal control of nonlinear networked control systems (NNCSs) using neuro-dynamic programming (NDP) over a finite time horizon is a challenging problem due to terminal constraints, system uncertainties, and unknown network imperfections, such as network-induced delays and packet losses. Since the traditional iteration or time-based infinite horizon NDP schemes are unsuitable for NNCS with terminal constraints, a novel time-based NDP scheme is developed to solve finite horizon optimal control of NNCS by mitigating the above-mentioned challenges. First, an online neural network (NN) identifier is introduced to approximate the control coefficient matrix that is subsequently utilized in conjunction with the critic and actor NNs to determine a time-based stochastic optimal control input over finite horizon in a forward-in-time and online manner. Eventually, Lyapunov theory is used to show that all closed-loop signals and NN weights are uniformly ultimately bounded with ultimate bounds being a function of initial conditions and final time. Moreover, the approximated control input converges close to optimal value within finite time. The simulation results are included to show the effectiveness of the proposed scheme.

The output feedback-based near-optimal regulation of uncertain and quantized nonlinear discrete-time systems in affine form with control constraint over finite horizon is addressed in this paper. First, the effect of input constraint is handled using a nonquadratic cost functional. Next, a neural network (NN)-based Luenberger observer is proposed to reconstruct both the system states and the control coefficient matrix so that a separate identifier is not needed. Then, approximate dynamic programming-based actor-critic framework is utilized to approximate the time-varying solution of the Hamilton-Jacobi-Bellman using NNs with constant weights and time-dependent activation functions. A new error term is defined and incorporated in the NN update law so that the terminal constraint error is also minimized over time. Finally, a novel dynamic quantizer for the control inputs with adaptive step size is designed to eliminate the quantization error overtime, thus overcoming the drawback of the traditional uniform quantizer. The proposed scheme functions in a forward-in-time manner without offline training phase. Lyapunov analysis is used to investigate the stability. Simulation results are given to show the effectiveness and feasibility of the proposed method.

This paper presents a novel state-feedback control scheme for the tracking control of a class of multi-input multioutput continuous-time nonlinear systems with unknown dynamics and bounded disturbances. First, the control law consisting of the robust integral of a neural network (NN) output plus sign of the tracking error feedback multiplied with an adaptive gain is introduced. The NN in the control law learns the system dynamics in an online manner, while the NN residual reconstruction errors and the bounded disturbances are overcome by the error sign signal. Since both of the NN output and the error sign signal are included in the integral, the continuity of the control input is ensured. The controller structure and the NN weight update law are novel in contrast with the previous effort, and the semiglobal asymptotic tracking performance is still guaranteed by using the Lyapunov analysis. In addition, the NN weights and all other signals are proved to be bounded simultaneously. The proposed approach also relaxes the need for the upper bounds of certain terms, which are usually required in the previous designs. Finally, the theoretical results are substantiated with simulations.

This paper proposes a novel optimal tracking control scheme for nonlinear continuous-time systems in strict-feedback form with uncertain dynamics. The optimal tracking problem is transformed into an equivalent optimal regulation problem through a feedforward adaptive control input that is generated by modifying the standard backstepping technique. Subsequently, a neural network-based optimal control scheme is introduced to estimate the cost, or value function, over an infinite horizon for the resulting nonlinear continuous-time systems in affine form when the internal dynamics are unknown. The estimated cost function is then used to obtain the optimal feedback control input; therefore, the overall optimal control input for the nonlinear continuous-time system in strict-feedback form includes the feedforward plus the optimal feedback terms. It is shown that the estimated cost function minimizes the Hamilton-Jacobi-Bellman estimation error in a forward-in-time manner without using any value or policy iterations. Finally, optimal output feedback control is introduced through the design of a suitable observer. Lyapunov theory is utilized to show the overall stability of the proposed schemes without requiring an initial admissible controller. Simulation examples are provided to validate the theoretical results.

In this paper, the finite-horizon optimal control design for nonlinear discrete-time systems in affine form is presented. In contrast with the traditional approximate dynamic programming methodology, which requires at least partial knowledge of the system dynamics, in this paper, the complete system dynamics are relaxed utilizing a neural network (NN)-based identifier to learn the control coefficient matrix. The identifier is then used together with the actor-critic-based scheme to learn the time-varying solution, referred to as the value function, of the Hamilton-Jacobi-Bellman (HJB) equation in an online and forward-in-time manner. Since the solution of HJB is time-varying, NNs with constant weights and time-varying activation functions are considered. To properly satisfy the terminal constraint, an additional error term is incorporated in the novel update law such that the terminal constraint error is also minimized over time. Policy and/or value iterations are not needed and the NN weights are updated once a sampling instant. The uniform ultimate boundedness of the closed-loop system is verified by standard Lyapunov stability theory under nonautonomous analysis. Numerical examples are provided to illustrate the effectiveness of the proposed method.

A novel neural network (NN)-based optimal adaptive consensus control scheme is introduced in this paper for networked mobile robots in the presence of unknown robot dynamics. Throughout the paper, two NNs are used. The unknown formation dynamics of each robot is identified by using the first NN. The second NN is utilized to approximate a novel value function derived in this paper as a function of augmented error vector, which is comprised of the regulation and consensus-based formation errors of each robot. A novel near optimal controller is developed by using approximated value function and identified formation dynamics. The Lyapunov stability theorem is employed to derive the NN weight tuning laws and demonstrate the consensus achievement of the overall formation. The simulation results are depicted to show performance of our theoretical claims.

This paper presents a novel stochastic event-based near optimal control strategy to regulate a networked control system (NCS) represented as an uncertain nonlinear continuous time system. An online stochastic actor-critic neural network (NN) based approach is utilized to achieve the near optimal regulation in the presence of network constraints, such as, network induced time-varying delays and random packet losses under event-based transmission of the feedback signals. The transformed nonlinear NCS in discrete-time after the incorporation the delays and packet losses is utilized for the actor-critic NN based controller design. To relax the knowledge of the control coefficient matrix, a NN based identifier is used. Event sampled state vector is utilized as NN inputs and their respective weights are updated non-periodically at the occurrence of events. Further, an event-trigger condition is designed by using the Lyapunov technique to ensure ultimate boundedness of all the closed-loop signals and save network resources and computation. Moreover, policy and value iterations are not utilized for the stochastic optimal regulator design. Finally, the analytical design is verified by using a numerical example by carrying out Monte-Carlo simulations.

In this paper, a novel optimal control over finite horizon has been introduced for linear continuous-time systems by using adaptive dynamic programming (ADP). First, a new time-varying Q-function parameterization and its estimator are introduced. Subsequently, Q-function estimator is tuned online by using both Bellman equation in integral form and terminal cost. Eventually, near optimal control gain is obtained by using the Q-function estimator. All the closed-loop signals are shown to be bounded by using Lyapunov stability analysis where bounds are functions of initial conditions and final time while the estimated control signal converges close to the optimal value. The simulation results illustrate the effectiveness of the proposed scheme.

In this paper, an event-based near optimal control of uncertain nonlinear discrete time systems is presented by using input-output data and approximate dynamic programming (ADP). The nonlinear system dynamics in affine form are transformed into an input-output form. Then, three neural networks (NN) with event sampled input-output vector are used, namely, the identifier NN to relax the knowledge of the system dynamics, a critic NN to approximate the value function which is the solution to the Hamilton-Jacobi Bellman (HJB) equation, and an actor NN to approximate the optimal control policy, in an online manner without utilizing value or policy iterations. In addition, the NN weights of all the three NNs are tuned only at event-triggered instants leading to a novel non-periodic update rule to reduce computation when compared to traditional NN based scheme. Further, an event-trigger condition to decide the trigger instants is derived. Finally, the Lyapunov technique is used in conjunction with the event-trigger condition to guarantee the uniform ultimate boundedness (UUB) of the closed-loop system. The analytical design is substantiated with numerical results via simulation.

In this paper, the finite horizon stochastic optimal control of nonlinear two-player zero-sum games, referred to as Nonlinear Networked Control Systems (NNCS) two-player zero-sum game, between control and disturbance input players in the presence of unknown system dynamics and a communication network with delays and packet losses is addressed by using neuro dynamic programming (NDP). The overall objective being to find the optimal control input while maximizing the disturbance attenuation. First, a novel online neural network (NN) identifier is introduced to estimate the unknown control and disturbance coefficient matrices which are needed in the generation of optimal control input. Then, the critic and two actor NNs have been introduced to learn the time-varying solution to the Hamilton-Jacobi-Isaacs (HJI) equation and determine the stochastic optimal control and disturbance policies in a forward-in-time manner. Eventually, with the proposed novel NN weight update laws, Lyapunov theory is utilized to demonstrate that all closed-loop signals and NN weights are uniformly ultimately bounded (ÜUB) during the finite horizon with ultimate bounds being a function of initial conditions and final time. Further, the approximated control input and disturbance signals tend close to the saddle-point equilibrium within finite-time. Simulation results are included.

A routing protocol for Passive RFID tag-to-tag networks is not reported in contrast with routing protocols for battery powered mobile wireless ad hoc and sensor networks. In this paper, we provide a cross-layer approach for passive RFID tag-to-tag communication. In the data link layer, we designed the medium access control (MAC) protocol which is suitable for passive tag-to-tag communication. In the network layer, we developed the optimal link cost multipath routing (OLCMR) protocol by using modulation depth as the link cost. Simulation results verify that proposed routing protocol consumes less energy when it is compared to optimum link state routing (OLSR). Additionally, when compared to the single path routing, the proposed multipath routing protocol increases the delivery ratio significantly.

Passive RFID tags and readers are initialized with secret keys which are updated after a successful cycle of authenticatio. Ownership transfer builds upon mutual authentication where a tagged item is shifted from one owner to another. Since the available protocols provide limited security for passive RFID systems and are vulnerable to attackers, we propose a novel ultra-lightweight authentication and ownership management protocol which conforms to the EPC Class-1 Generation-2 Version 2 standard while taking into account the storage and computational resources of the tags. The protocol is successfully implemented on hardware to overcome the weaknesses of the available protocols. The experimental results show that the use of such protocol ensures security with little added communication and computation overhead.

In this paper, a novel model-based fault detection (FD) and prediction scheme is developed for a class of Takagi-Sugeno (T-S) fuzzy systems. Unlike other FD schemes, in the proposed design, an FD observer with online fault learning capability is utilized to generate a residual which is obtained by comparing the system output with respect to the observer output. A fault is declared active if the generated residual exceeds an a priori chosen threshold. Subsequently, the fault magnitude is estimated online by using a suitable parameter update law. Upon detection, the online estimate of the fault magnitude is used in a mathematical equation to determine time-to-failure (TTF) or remaining useful life. TTF is determined by projecting the estimated fault magnitude at the current time instant against a failure threshold. Note that the previously reported FD schemes could neither estimate the magnitude of a growing fault in real time nor were they able to predict the remaining useful life of the fuzzy system. In this paper, the stability of the proposed FD and prognostics scheme is verified using the Lyapunov theory. Finally, two different simulation case studies are considered to verify the theoretical conjectures presented in this paper.

Multipath and shadow fading are the primary cause for positioning errors in a Received Signal Strength Indicator (RSSI) based localization scheme. While fading, in general, is detrimental to localization accuracy, cross-correlation and divergence properties of shadow fading residuals may be utilized to improve localization and tracking accuracy of mobile IEEE 802.15.4 transmitters. Therefore, this paper begins by presenting a stochastic filter that models the fast changing multipath fading as a mean reverting Ornstein-Uhlenbeck (OU) process followed by a Generalized Auto Regressive Conditional Heteroskedasticity (GARCH) filtering to isolate the slow changing shadow fading residuals from measured RSSI values. Subsequently, a novel wireless transmitter localization scheme that combines the measured cross-correlation in shadow fading residuals between adjacent receivers using a Student-t Copula likelihood function is proposed. However, the long convergence time for this highly non-convex copula function might render our method unsuitable for tracking applications. Therefore, we present a faster tracking method where the velocity and heading of a mobile transmitter are estimated from α-Divergence between shadow fading signals and an onboard gyroscope respectively. To bind the localization error in this tracking method, the transmitter location estimates are smoothed by a Bayesian particle filter. The performance of our proposed localization and tracking method is validated over simulations and hardware experiments.

Measured data or states for a nonlinear dynamic system is usually contaminated by outliers. Identifying and removing outliers will make the data (or system states) more trustworthy and reliable since outliers in the measured data (or states) can cause missed or false alarms during fault diagnosis. In addition, faults can make the system states nonstationary needing a novel analytical model-based fault detection (FD) framework. In this paper, an online outlier identification and removal (OIR) scheme is proposed for a nonlinear dynamic system. Since the dynamics of the system can experience unknown changes due to faults, traditional observer-based techniques cannot be used to remove the outliers. The OIR scheme uses a neural network (NN) to estimate the actual system states from measured system states involving outliers. With this method, the outlier detection is performed online at each time instant by finding the difference between the estimated and the measured states and comparing its median with its standard deviation over a moving time window. The NN weight update law in OIR is designed such that the detected outliers will have no effect on the state estimation, which is subsequently used for model-based fault diagnosis. In addition, since the OIR estimator cannot distinguish between the faulty or healthy operating conditions, a separate model-based observer is designed for fault diagnosis, which uses the OIR scheme as a preprocessing unit to improve the FD performance. The stability analysis of both OIR and fault diagnosis schemes are introduced. Finally, a three-tank benchmarking system and a simple linear system are used to verify the proposed scheme in simulations, and then the scheme is applied on an axial piston pump testbed. The scheme can be applied to nonlinear systems whose dynamics and underlying distribution of states are subjected to change due to both unknown faults and operating conditions.

In this paper, finite horizon stochastic optimal control issue has been studied for linear networked control system (LNCS) in the presence of network imperfections such as network-induced delays and packet losses by using adaptive dynamic programming (ADP) approach. Due to an uncertainty in system dynamics resulting from network imperfections, the stochastic optimal control design uses a novel adaptive estimator (AE) to solve the optimal regulation of uncertain LNCS in a forward-in-time manner in contrast with backward-in-time Riccati equation-based optimal control with known system dynamics. Tuning law for unknown parameters of AE has been derived. Lyapunov theory is used to show that all the signals are uniformly ultimately bounded (UUB) with ultimate bounds being a function of initial values and final time. In addition, the estimated control input converges to optimal control input within finite horizon. Simulation results are included to show the effectiveness of the proposed scheme.

In this paper, the finite-horizon optimal adaptive control design for linear discrete-time systems with unknown system dynamics by using adaptive dynamic programming (ADP) is presented. In the presence of full state feedback, the terminal state constraint is incorporated in solving the optimal feedback control via the Bellman equation. The optimal regulation of the uncertain linear system is solved in a forward-in-time and online manner without using value and/or policy iterations. Due to the nature of finite horizon, the stability of the closed-loop system is involved but verified by using Lyapunov theory. The effectiveness of the proposed method is verified by simulation results.

Actor-critic reinforcement learning algorithms have shown to be a successful tool in learning the optimal control for a range of (repetitive) tasks on systems with (partially) unknown dynamics, which may or may not be nonlinear. Most of the reinforcement learning literature published up to this point only deals with modeling the task at hand as a Markov decision process with an infinite horizon cost function. In practice, however, it is sometimes desired to have a solution for the case where the cost function is defined over a finite horizon, which means that the optimal control problem will be time-varying and thus harder to solve. This paper adapts two previously introduced actor-critic algorithms from the infinite horizon setting to the finite horizon setting and applies them to learning a task on a nonlinear system, without needing any assumptions or knowledge about the system dynamics, using radial basis function networks. Simulations on a typical nonlinear motion control problem are carried out, showing that actor-critic algorithms are capable of solving the difficult problem of time-varying optimal control. Moreover, the benefit of using a model learning technique is shown.

Existing neuro-dynamic programming (NDP) techniques are not applicable for optimizing real-time NNCS with terminal constraints during the finite horizon. Therefore, a novel time-based NDP scheme is developed in this paper to solve finite horizon optimal control of NNCS. First, an online neural network (NN) identifier is introduced to approximate the control coefficient matrix. Then, the critic and action NNs are utilized to determine time-based finite horizon stochastic optimal control for NNCS in a forward-in-time manner. By incorporating novel NN weight update laws, Lyapunov theory is used to show that all closed-loop signals and NN weights are uniformly ultimately bounded (UUB) with ultimate bounds being a function of initial conditions and final time. Moreover, the approximated control input converges close to target value within finite time. Simulation results are included to show the effectiveness of the proposed scheme.

In this paper, the finite-horizon optimal control design for affine nonlinear continuous-time systems in the presence of known system dynamics is presented. A neural network (NN) is utilized to learn the time-varying solution of the Hamilton-Jacobi-Bellman (HJB) equation in an online and forward in time manner. To handle the time varying nature of the value function, the NN with constant weights and time-varying activation function is considered. The update law for tuning the NN weights is derived based on normalized gradient descent approach. To satisfy the terminal constraint and ensure stability, additional terms, one corresponding to the terminal constraint, and the other to stabilize the nonlinear system are added to the novel updating law. A uniformly ultimately boundedness of the non-autonomous closed-loop system is verified by using standard Lyapunov theory. The effectiveness of the proposed method is verified by simulation results.

In this paper, the nearly optimal solution for discrete-time (DT) affine nonlinear control systems in the presence of partially unknown internal system dynamics and disturbances is considered. The approach is based on successive approximate solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which appears in optimal control. Successive approximation approach for updating control and disturbance inputs for DT nonlinear affine systems are proposed. Moreover, sufficient conditions for the convergence of the approximate HJI solution to the saddle point are derived, and an iterative approach to approximate the HJI equation using a neural network (NN) is presented. Then, the requirement of full knowledge of the internal dynamics of the nonlinear DT system is relaxed by using a second NN online approximator. The result is a closed-loop optimal NN controller via offline learning. A numerical example is provided illustrating the effectiveness of the approach.

This paper presents a novel localization scheme in the 3D wireless domain that employs cross correlation in backscattered signal power from a cluster of radio frequency identification (RFID) tags to estimate their location. Spatially co-located RFID tags, energized by a common tag reader, exhibit correlation in their received signal strength indicator (RSSI) values. Hence, for a cluster of RFID tags, the posterior distribution of their unknown radial separation is derived as a function of the measured RSSI correlations between them. The global maxima of this posterior distribution represent the actual radial separation between the RFID tags. The radial separations are then utilized to obtain location estimates of the tags. However, due to the nonconvex nature of the posterior distribution, deterministic optimization methods that are used to solve true radial separations between tags provide inaccurate results due to local maxima, unless the initial radial separation estimates are within the region of attraction of its global maximum. The proposed RFID localization algorithm called LOCalization Using Stochastic Tunneling (LOCUST) utilizes constrained simulated annealing with tunneling transformation to solve this nonconvex posterior distribution. The tunneling transformation allows the optimization search operation to circumvent or “tunnel” through ill-shaped regions in the posterior distribution resulting in faster convergence to the global maximum. Finally, simulation results of our localization method are presented to demonstrate the theoretical conclusions.

Helicopter unmanned aerial vehicles (UAVs) are widely used for both military and civilian operations. Because the helicopter UAVs are underactuated nonlinear mechanical systems, high-performance controller design for them presents a challenge. This paper introduces an optimal controller design via an output feedback for trajectory tracking of a helicopter UAV, using a neural network (NN). The output-feedback control system utilizes the backstepping methodology, employing kinematic and dynamic controllers and an NN observer. The online approximator-based dynamic controller learns the infinite-horizon Hamilton-Jacobi-Bellman equation in continuous time and calculates the corresponding optimal control input by minimizing a cost function, forward-in-time, without using the value and policy iterations. Optimal tracking is accomplished by using a single NN utilized for the cost function approximation. The overall closed-loop system stability is demonstrated using Lyapunov analysis. Finally, simulation results are provided to demonstrate the effectiveness of the proposed control design for trajectory tracking.

The stochastic optimal controller design for the nonlinear networked control system (NNCS) with uncertain system dynamics is a challenging problem due to the presence of both system nonlinearities and communication network imperfections, such as random delays and packet losses, which are not unknown a priori. In the recent literature, neuro dynamic programming (NDP) techniques, based on value and policy iterations, have been widely reported to solve the optimal control of general affine nonlinear systems. However, for realtime control, value and policy iterations-based methodology are not suitable and time-based NDP techniques are preferred. In addition, output feedback-based controller designs are preferred for implementation. Therefore, in this paper, a novel NNCS representation incorporating the system uncertainties and network imperfections is introduced first by using input and output measurements for facilitating output feedback. Then, an online neural network (NN) identifier is introduced to estimate the control coefficient matrix, which is subsequently utilized for the controller design. Subsequently, the critic and action NNs are employed along with the NN identifier to determine the forward-in-time, time-based stochastic optimal control of NNCS without using value and policy iterations. Here, the value function and control inputs are updated once a sampling instant. By using novel NN weight update laws, Lyapunov theory is used to show that all the closed-loop signals and NN weights are uniformly ultimately bounded in the mean while the approximated control input converges close to its target value with time. Simulation results are included to show the effectiveness of the proposed scheme.

Complex nonlinear systems such as an aircraft, trains, automobiles, power plants and chemical plants are represented as nonlinear interconnected subsystems. Therefore, in this paper a novel decentralized fault diagnosis and prognosis (FDP) methodology is proposed for such large-scale systems. Current FDP approaches require the knowledge of the entire state or its estimated vector. But the main goal in this work is to design a local fault detector (LFD) or observer for each subsystem based on the measured local states of the subsystem alone. A local residual signal is generated via the measured states of the local subsystem and the estimated states provided by the LFD. A fault is detected when this local residual exceeds a predefined threshold. The adaptive online approximator in each LFD is activated upon detection to compensate the fault dynamics due to local and non-local faults. A novel update law for tuning the parameters of the online approximator is derived. Upon detection, faults local to the subsystem and to other subsystems are isolated. In addition, the proposed scheme provides the time to failure (or remaining useful life) information by using local measurements and the parameter update law of the LFD. Simulation results verify the effectiveness of the proposed decentralized FDP scheme.

In this paper, the Bellman equation is used to solve the stochastic optimal control of unknown linear discrete-time system with communication imperfections including random delays, packet losses and quantization. A dynamic quantizer for the sensor measurements is proposed which essentially provides system states to the controller. To eliminate the effect of the quantization error, the dynamics of the quantization error bound and an update law for tuning its range are derived. Subsequently, by using adaptive dynamic programming technique, the infinite horizon optimal regulation of the uncertain NCS is solved in a forward-in-time manner without using value and/or policy iterations by using Q-function and reinforcement learning. The asymptotic stability of the closed-loop system is verified by standard Lyapunov stability theory. Finally, the effectiveness of the proposed method is verified by simulation results.

In Cyber Physical Systems (CPS), multiple real-time dynamic systems are connected to their corresponding controllers through a shared communication network in contrast with a dedicated line. For such CPS, the existing scheduling schemes for instance Centralized or Distributed Scheduling schemes are unsuitable since the behavior of real-time dynamic systems is ignored during the network protocol design. Therefore, in this paper, a novel distributed scheduling protocol design via cross-layer approach is proposed to optimize the performance of CPS by maximizing the utility function which is generated by using the information collected from both the application and network layers. Subsequently, a novel adaptive model based optimal event-triggered control scheme is derived for each real-time dynamic system with unknown system dynamics in the application layer. Compared with traditional scheduling algorithms, the proposed distributed scheduling scheme via cross-layer approach can not only allocates the network resources efficiently but also improves the performance of the overall real-time dynamic system.

In this paper, the Hamilton-Jacobi-Bellman equation is solved forward-in-time for the optimal control of a class of general affine nonlinear discrete-time systems without using value and policy iterations. The proposed approach, referred to as adaptive dynamic programming, uses two neural networks (NNs), to solve the infinite horizon optimal regulation control of affine nonlinear discrete-time systems in the presence of unknown internal dynamics and a known control coefficient matrix. One NN approximates the cost function and is referred to as the critic NN, while the second NN generates the control input and is referred to as the action NN. The cost function and policy are updated once at the sampling instant and thus the proposed approach can be referred to as time-based ADP. Novel update laws for tuning the unknown weights of the NNs online are derived. Lyapunov techniques are used to show that all signals are uniformly ultimately bounded and that the approximated control signal approaches the optimal control input with small bounded error over time. In the absence of disturbances, an optimal control is demonstrated. Simulation results are included to show the effectiveness of the approach. The end result is the systematic design of an optimal controller with guaranteed convergence that is suitable for hardware implementation.

Data aggregation is necessary for extending the network lifetime of wireless sensor nodes with limited processing and power capabilities, since energy expended in transmitting a single data bit would be at least several orders of magnitude higher when compared to that needed for a 32-bit computation. Therefore, in this article, a novel nonlinear adaptive pulse coded modulation-based compression (NADPCMC) scheme is proposed for data aggregation in a wireless sensor network (WSN). The NADPCMC comprises of two estimators—one at the source or transmitter and the second one at the destination node. The estimator at the source node approximates the data value for each sample. The difference between the data sample and its estimate is quantized and transmitted to the next hop node instead of the actual data sample, thus reducing the amount of data transmission and rending energy savings. A similar estimator at the next hop node or base station reconstructs the original data. It is demonstrated that repeated application of the NADPCMC scheme along the route in a WSN results in data aggregation. Satisfactory performance of the proposed scheme in terms of distortion, compression ratio, and energy efficiency and in the presence of estimation and quantization errors for data aggregation is demonstrated using the Lyapunov approach. Then the performance of the proposed scheme is contrasted with the available compression schemes in an NS-2 environment through several benchmarking datasets. Simulation and hardware results demonstrate that almost 50% energy savings with low distortion levels below 5% and low overhead are observed when compared to no compression. Iteratively applying the proposed compression scheme at the cluster head nodes along the routes over the network yields an additional improvement of 20% in energy savings per aggregation with an overall distortion below 8%.

In this paper, reinforcement learning state- and output-feedback-based adaptive critic controller designs are proposed by using the online approximators (OLAs) for a general multi-input and multioutput affine unknown nonlinear discretetime systems in the presence of bounded disturbances. The proposed controller design has two entities, an action network that is designed to produce optimal signal and a critic network that evaluates the performance of the action network. The critic estimates the cost-to-go function which is tuned online using recursive equations derived from heuristic dynamic programming. Here, neural networks (NNs) are used both for the action and critic whereas any OLAs, such as radial basis functions, splines, fuzzy logic, etc., can be utilized. For the output-feedback counterpart, an additional NN is designated as the observer to estimate the unavailable system states, and thus, separation principle is not required. The NN weight tuning laws for the controller schemes are also derived while ensuring uniform ultimate boundedness of the closed-loop system using Lyapunov theory. Finally, the effectiveness of the two controllers is tested in simulation on a pendulum balancing system and a two-link robotic arm system.

In this paper, the infinite horizon optimal tracking control problem is solved online and forward-in-time for leader-follower based formation control of nonholonomic mobile robots. Using the backstepping design approach, the dynamical controller inputs for the robots are approximated from nonlinear optimal control techniques in order to track the control velocities designed to keep the formation. The proposed nonlinear optimal control technique, referred to as adaptive dynamic programming, uses neural networks (NN's) to solve the optimal formation control problem in discrete-time in the presence of unknown internal dynamics and a known control coefficient matrix. All NN's are tuned online using novel weight update laws, and the stability of the entire formation is demonstrated using Lyapunov methods. Simulation results are provided to demonstrate the effectiveness of the proposed approach.

Multi-input and multi-output (MIMO) optimal control of unknown nonaffine nonlinear systems is a challenging problem due to the presence of control inputs inside the unknown nonlinearity. In this paper, the optimal control of MIMO nonlinear nonaffine discrete-time systems in input-output form is considered when the internal dynamics are unknown. First, the nonaffine nonlinear system is converted into an affine-like equivalent nonlinear system under the assumption that the higher-order terms are bounded. Next, a forward-in-time Hamilton-Jaccobi-Bellman (HJB) equation-based optimal approach is developed to control the affine-like nonlinear system using neural network (NN). To overcome the need to know the control gain matrix of the affine-like system for the optimal controller, an online identifier is introduced. Lyapunov stability of the overall system including the online identifier shows that the approximate control input approaches the optimal control with a bounded error. Finally, the optimal control approach is applied to the cycle-by-cycle discrete-time representation of the experimentally validated HCCI engine which is represented as a nonaffine nonlinear system. Simulation results are included to demonstrate the efficacy of the approach in presence of actuator disturbances.

While fading, in general, is detrimental to accurately localizing a target, fading correlation between adjacent receivers may be exploited to improve localization accuracy. Therefore, this paper presents a novel wireless localization scheme that employs a combination of cross-correlation between shadow fading noise and copula technique to recursively estimate the location of a transmitter. A stochastic filter that models multipath fading as an Ornstein-Uhlenbeck process followed by a Generalized Auto Regressive Conditional Heteroskedasticity (GARCH) filtering is proposed to extract shadow fading residuals from measured RSSI values. Subsequently, Student-t Copula function is used to create the log likelihood function, which acts as the cost function for localization, by combining spatial shadow fading correlation arising among adjacent receivers due to pedestrian traffic in the area. Maximum Likelihood Estimate (MLE) is used for position estimation as it inherits the statistical consistency and asymptotic normality. The performance of our proposed localization method is validated over simulations and hardware experiments.

This paper presents a novel wireless localization scheme in the three dimensional domain that employs stochastic optimization with tunneling transformation to recursively estimate the location of wireless tags in a network from pair wise signal strength measurements. Spatially co-located wireless tags, receiving signals from a common transmitter, exhibit correlation in their Received Signal Strength Indicator (RSSI) values. Hence in a network of wireless tags, with pair wise correlation coefficients available, posterior distribution of the unknown tag separation is used to relatively localize them using maximum a posteriori (MAP) Estimator. However, due to the non-convex/non-tractable nature of this posterior distribution, deterministic optimization methods will end in one of the many local maxima unless the initial guess is close to the region of attraction of the global maximum. In this paper, a novel stochastic localization method called LOCalization Using Stochastic Tunneling (LOCUST) is proposed which utilizes constrained simulated annealing with tunneling transformation to solve this non-tractable posterior distribution. The tunneling transformation allows the optimization search operation to circumvent or “tunnel” through ill-shaped regions in the posterior distribution resulting in faster convergence to global maximum. Finally, simulation results of our localization method are presented.

In this paper, the direct neural dynamic programming technique is utilized to solve the Hamilton-Jacobi-Bellman equation forward-in-time for the decentralized near optimal regulation of a class of nonlinear interconnected discrete-time systems with unknown internal subsystem and interconnection dynamics, while the input gain matrix is considered known. Even though the unknown interconnection terms are considered weak and functions of the entire state vector, the decentralized control is attempted under the assumption that only the local state vector is measurable. The decentralized nearly optimal controller design for each subsystem consists of two neural networks (NNs), an action NN that is aimed to provide a nearly optimal control signal, and a critic NN which evaluates the performance of the overall system. All NN parameters are tuned online for both the NNs. By using Lyapunov techniques it is shown that all subsystems signals are uniformly ultimately bounded and that the synthesized subsystems inputs approach their corresponding nearly optimal control inputs with bounded error. Simulation results are included to show the effectiveness of the approach.

A novel neural network (NN)-based nonlinear decentralized adaptive controller is proposed for a class of large-scale, uncertain, interconnected nonlinear systems in strict-feedback form by using the dynamic surface control (DSC) principle, thus, the "explosion of complexity" problem which is observed in the conventional backstepping approach is relaxed in both state and output feedback control designs. The matching condition is not assumed when considering the interconnection terms. Then, NNs are utilized to approximate the uncertainties in both subsystem and interconnected terms. By using novel NN weight update laws with quadratic error terms as well as proposed control inputs, it is demonstrated using Lyapunov stability that the system states errors converge to zero asymptotically with both state and output feedback controllers, even in the presence of NN approximation errors in contrast with the uniform ultimate boundedness result, which is common in the literature with NN-based DSC and backstepping schemes. Simulation results show the effectiveness of the approach.

A novel Mahalanobis Taguchi System (MTS) based fault detection, isolation, and prognostics scheme is presented. The proposed scheme fuses data from multiple sensors into a single system level performance metric using Mahalanobis Distance (MD), and generates fault clusters based on MD values. MD thresholds derived from the clustering analysis are used for fault detection and isolation. When a fault is detected, the prognostics scheme, which monitors the progression of the MD values over time, is initiated. Then, using a linear approximation, time to failure is estimated. The performance of the scheme has been validated via experiments performed on a mono-block centrifugal water pump testbed. The pump has been instrumented with vibration, pressure, temperature, and flow sensors; and experiments involving healthy and various types of faulty operating conditions have been performed. The experiments show that the proposed approach renders satisfactory results for centrifugal water pump fault detection, isolation, and prognostics. Overall, the proposed solution provides a reliable multivariate analysis and real-time decision making tool that 1) fuses data from multiple sensors into a single system level performance metric; 2) extends MTS by providing a single tool for fault detection, isolation, and prognosis, eliminating the need to develop each separately; and 3) offers a systematic way to determine the key parameters, thus reducing analysis overhead. In addition, the MTS-based scheme is process independent, and can easily be implemented on wireless motes1, and deployed for real-time monitoring, diagnostics, and prognostics in a wide variety of industrial environments.

A novel fault diagnostics and prediction scheme in continuous-time is introduced for a class of nonlinear systems. The proposed method uses a novel neural network (NN) based robust integral sign of the error (RISE) observer, or estimator, allowing for semi-global asymptotic stability in the presence of NN approximation errors, disturbances and unmodeled dynamics. This is in comparison to typical results presented in the literature that show only boundedness in the presence of uncertainties. The output of the observer/estimator is compared with that of the nonlinear system and a residual is used for declaring the presence of a fault when the residual exceeds a user defined threshold. The NN weights are tuned online with no offline tuning phase. The output of the RISE observer is utilized for diagnostics. Additionally, a method for time-to-failure (TTF) prediction, a first step in prognostics, is developed by projecting the developed parameter-update law under the assumption that the nonlinear system satisfies a linear-in-the-parameters (LIP) assumption. The TTF method uses known critical values of a system to predict when an estimated parameter will reach a known failure threshold. The performance of the NN/RISE observer system is evaluated on a nonlinear system and a simply supported beam finite element analysis (FEA) simulation based on laboratory experiments. Results show that the proposed method provides as much as 25% increased accuracy while the TTF scheme renders a more accurate prediction.

In this paper, the nearly optimal solution for discrete-time (DT) affine nonlinear control systems in the presence of partially unknown internal system dynamics and disturbances is considered. The approach is based on successive approximate solution of the Hamilton-Jacobi-Isaacs (HJI) equation, which appears in optimal control. Successive approximation approach for updating control input and disturbance for DT nonlinear affine systems are proposed. Moreover, sufficient conditions for the convergence of the approximate HJI solution to the saddle-point are derived, and an iterative approach to approximate the HJI equation using a neural network (NN) is presented. Then, the requirement of full knowledge of the internal dynamics of the nonlinear DT system is relaxed by using a second NN online approximator. The result is a closed-loop optimal NN controller via offline learning. Numerical example is provided illustrating the effectiveness of the approach.

In this paper, the direct neural dynamic programming technique is utilized to solve the Hamilton Jacobi-Bellman (HJB) equation online and forward-in-time for the decentralized nearly optimal control of nonlinear interconnected discrete-time systems in affine form with unknown internal subsystem and interconnection dynamics. Only the state vector of the local subsystem is considered measurable. The decentralized optimal controller design for each subsystem consists of an action neural network (NN) that is aimed to provide a nearly optimal control signal, and a critic NN which approximates the cost function. The NN weights are tuned online for both the NNs. It is shown that all subsystems signals are uniformly ultimately bounded (UUB) and that the subsystem inputs approach their corresponding nearly optimal control inputs with bounded error.

Node mobility causes fading wireless channels, which in turn renders topology changes in emerging wireless ad hoc networks. In this paper, on-line estimators and Markov models are utilized to estimate fading channel conditions. Using the estimated channel conditions as well as queue occupancy, available energy and link delay, approximate dynamic programming (ADP) techniques are utilized to find dynamic routes, while solving discrete-time Hamilton-Jacobi-Bellman (HJB) equation forward-in-time for route cost in multichannel multi-interface networks. The performance of the proposed load balancing method in the presence of fading channels and the performance of the optimal route selection approach for multi-channel multi-interface wireless ad hoc network is evaluated by extensive simulations and comparing it to AODV.

In this paper, a sub-optimal solution to the receiver placement and number of receiver determination problem is introduced. To achieve this overall goal, first, localization error for a received signal strength indicator (RSSI)-based N-receiver system localizing a transmitter is estimated. Subsequently, this estimator error along with the 2D-tessellation techniques such as Delaunay refinement are used to position candidate receivers not only to minimize their number needed to meet the location error threshold but also to reduce the dilution of localization accuracy due to the layout of receivers. Rigorous mathematical analysis indicates that the receiver count generated by our Delaunay refinement-based sub-optimal solution using triangular tiles is indeed bounded from the optimal count by a constant which in turn depends upon the workspace layout. Finally, the sub-optimal scheme is demonstrated by using experimental data.

Wireless sensor nodes typically have limited processing capabilities and are powered by batteries. The amount of energy expended in transmitting a single data bit would be several orders of magnitude higher when compared to the energy needed for a 32 bit computation. Thus, to maximize network lifetime, data transmissions should be minimized without losing vital information. In this paper, a novel adaptive compression scheme using nonlinear estimation theory is proposed for data aggregation. Satisfactory performance of the proposed compression scheme in the presence of noise, distortion, and quantization errors is demonstrated using Lyapunov approach. The proposed scheme is contrasted with existing compression schemes using various metrics applicable to wireless sensor networks such as energy efficiency, distortion and compression ratio. Simulation and hardware experimental results demonstrate almost 50% energy savings with very low distortion (less than 5%) and overhead. By iteratively applying the proposed scheme at the cluster head nodes, higher energy savings are obtained with a tolerable level of distortion.

In this paper, a new nonlinear controller for a quadrotor unmanned aerial vehicle (UAV) is proposed using neural networks (NNs) and output feedback. The assumption on the availability of UAV dynamics is not always practical, especially in an outdoor environment. Therefore, in this work, an NN is introduced to learn the complete dynamics of the UAV online, including uncertain nonlinear terms like aerodynamic friction and blade flapping. Although a quadrotor UAV is underactuated, a novel NN virtual control input scheme is proposed which allows all six degrees of freedom (DOF) of the UAV to be controlled using only four control inputs. Furthermore, an NN observer is introduced to estimate the translational and angular velocities of the UAV, and an output feedback control law is developed in which only the position and the attitude of the UAV are considered measurable. It is shown using Lyapunov theory that the position, orientation, and velocity tracking errors, the virtual control and observer estimation errors, and the NN weight estimation errors for each NN are all semiglobally uniformly ultimately bounded (SGUUB) in the presence of bounded disturbances and NN functional reconstruction errors while simultaneously relaxing the separation principle. The effectiveness of proposed output feedback control scheme is then demonstrated in the presence of unknown nonlinear dynamics and disturbances, and simulation results are included to demonstrate the theoretical conjecture.

In this paper, a novel, unified model-based fault-detection and prediction (FDP) scheme is developed for nonlinear multiple-input-multiple-output (MIMO) discrete-time systems. The proposed scheme addresses both state and output faults by considering separate time profiles. The faults, which could be incipient or abrupt, are modeled using input and output signals of the system. The fault-detection (FD) scheme comprises online approximator in discrete time (OLAD) with a robust adaptive term. An output residual is generated by comparing the FD estimator output with that of the measured system output. A fault is detected when this output residual exceeds a predefined threshold. Upon detecting the fault, the robust adaptive terms and the OLADs are initiated wherein the OLAD approximates the unknown fault dynamics online while the robust adaptive terms help in ensuring asymptotic stability of the FD design. Using the OLAD outputs, a fault diagnosis scheme is introduced. A stable parameter update law is developed not only to tune the OLAD parameters but also to estimate the time to failure (TTF), which is considered as a first step for prognostics. The asymptotic stability of the FDP scheme enhances the detection and TTF accuracy. The effectiveness of the proposed approach is demonstrated using a fourth-order MIMO satellite system.

In this paper, a combined kinematic/torque output feedback control law is developed for leader-follower-based formation control using backstepping to accommodate the dynamics of the robots and the formation in contrast with kinematic-based formation controllers. A neural network (NN) is introduced to approximate the dynamics of the follower and its leader using online weight tuning. Furthermore, a novel NN observer is designed to estimate the linear and angular velocities of both the follower robot and its leader. It is shown, by using the Lyapunov theory, that the errors for the entire formation are uniformly ultimately bounded while relaxing the separation principle. In addition, the stability of the formation in the presence of obstacles, is examined using Lyapunov methods, and by treating other robots in the formation as obstacles, collisions within the formation are prevented. Numerical results are provided to verify the theoretical conjectures.

Discrete time approximate dynamic programming (ADP) techniques have been widely used in the recent literature to determine the optimal or near optimal control policies for nonlinear systems. However, an inherent assumption of ADP requires at least partial knowledge of the system dynamics as well as the value of the controlled plant one step ahead. In this work, a novel approach to ADP is attempted while relaxing the need of the partial knowledge of the nonlinear system. The proposed methodology entails a two part process: online system identification and offline optimal control training. First, in the identification process, a neural network (NN) is tuned online to learn the complete plant dynamics and local asymptotic stability is shown under a mild assumption that the NN functional reconstruction errors lie within a small-gain type norm bounded conic sector. Then, using only the NN system model, offline ADP is attempted resulting in a novel optimal control law. The proposed scheme does not require explicit knowledge of the system dynamics as only the learned NN model is needed. Proof of convergence is demonstrated. Simulation results verify theoretical conjecture.

A novel decentralized controller using the dynamic surface control (DSC) is proposed for a class of uncertain large scale interconnected nonlinear systems in strict-feedback form while relaxing the ldquoexplosion of complexityrdquo problem which is observed in the typical backstepping approach. The matching condition is not assumed when dealing with the interconnection terms. Neural networks (NNs) are utilized to approximate the uncertainties in both subsystem and interconnected terms. By using novel NN weight update laws, it is demonstrated using Lyapunov stability that the closed-loop signals are asymptotically stable in the presence of NN approximation errors in contrast with the uniform ultimate boundedness result that is common in the literature with NN-based DSC and backstepping schemes. Simulation results of the controller performance for a nonlinear decentralized system justify theoretical conclusions.

The fundamental cause of localization error in an indoor environment is fading and spreading of the radio signals due to scattering, diffraction, and reflection. These effects are predominant in regions where there is no-line-of-sight (NLoS) between the transmitter and the receiver. Efficient algorithms are needed to identify the subset of receivers that provide better localization accuracy. This paper introduces a new parameter called the R-Factor to indicate the extent of radial distance estimation error introduced by a receiver and to select a subset of receivers that result in better accuracy in real-time location determination systems (RTLS). In addition, it was demonstrated that location accuracy improves with R-factor reduction. Further, it was shown that for a given R-factor threshold, localization accuracy is enhanced either by increasing the number of receivers that fall below this threshold or by increasing the diversity channel count with appropriate combining of signals from diversity. Therefore, existing localization algorithms can utilize R-factor and diversity through selection combining to improve accuracy. Both analytical and experimental results are included to justify the theoretical results in terms of improvement in accuracy by using R-factor.

Multiple non-interfering channels are available in 802.11 and 802.15.4 based wireless networks. Capacity of such channels can be combined to achieve a better performance thus providing a higher quality of service (QoS) than for a single channel network. However, existing routing protocols often are not suited to fully take advantage of these channels. The proposed multi-interface multi-channel routing (MMCR) protocol considers various QoS parameters such as throughput, end-to-end delay, and energy utilization as a single unified cost metric and identifies the route that optimizes the cost metric and balances the traffic among the channels on a per flow basis. Multipoint relay nodes (MPRs) are first selected using available energy and bandwidth and utilized in routing. A novel load balancing scheme is introduced and analytical performance guarantees are demonstrated. Simulation results using the Ns2 show superior performance of the MMCR over the multi-channel optimal link state routing protocol (m-OLSR) in terms of throughput end-to-end delay, and energy efficiency.

The bandwidth utilization of a single channel-based wireless networks decreases due to congestion and interference from other sources and therefore transmission on multiple channels are needed. In this paper, we propose a distributed dynamic channel allocation scheme for wireless networks using adaptive learning automata whose nodes are equipped with single radio interfaces so that a more suitable channel can be selected. The proposed scheme, adaptive pursuit reward-inaction, runs periodically on the nodes, and adaptively finds the suitable channel allocation in order to attain a desired performance. A novel performance index, which takes into account the throughput and the energy consumption, is considered. The proposed scheme is adaptive in the sense that probabilities in the each step are updated as a function of the error in the performance index. The extensive simulation results in static and mobile environments provide that using the proposed scheme for channel allocation in the multiple channel wireless networks significantly improves the throughput, drop rate, energy consumption per packet and fairness index.

In this paper, an asymptotically stable (AS) combined kinematic/torque control law is developed for leader-follower-based formation control using backstepping in order to accommodate the complete dynamics of the robots and the formation, and a neural network (NN) is introduced along with robust integral of the sign of the error feedback to approximate the dynamics of the follower as well as its leader using online weight tuning. It is shown using Lyapunov theory that the errors for the entire formation are AS and that the NN weights are bounded as opposed to uniformly ultimately bounded stability which is typical with most NN controllers. Additionally, the stability of the formation in the presence of obstacles is examined using Lyapunov methods, and by treating other robots in the formation as obstacles, collisions within the formation do not occur. The asymptotic stability of the follower robots as well as the entire formation during an obstacle avoidance maneuver is demonstrated using Lyapunov methods, and numerical results are provided to verify the theoretical conjectures.

A novel reinforcement-learning-based output adaptive neural network (NN) controller, which is also referred to as the adaptive-critic NN controller, is developed to deliver the desired tracking performance for a class of nonlinear discrete-time systems expressed in nonstrict feedback form in the presence of bounded and unknown disturbances. The adaptive-critic NN controller consists of an observer, a critic, and two action NNs. The observer estimates the states and output, and the two action NNs provide virtual and actual control inputs to the nonlinear discrete-time system. The critic approximates a certain strategic utility function, and the action NNs minimize the strategic utility function and control inputs. All NN weights adapt online toward minimization of a performance index, utilizing the gradient-descent-based rule, in contrast with iteration-based adaptive-critic schemes. Lyapunov functions are used to show the stability of the closed-loop tracking error, weights, and observer estimates. Separation and certainty equivalence principles, persistency of excitation condition, and linearity in the unknown parameter assumption are not needed. Experimental results on a spark ignition (SI) engine operating lean at an equivalence ratio of 0.75 show a significant (25%) reduction in cyclic dispersion in heat release with control, while the average fuel input changes by less than 1% compared with the uncontrolled case. Consequently, oxides of nitrogen (NO(x)) drop by 30%, and unburned hydrocarbons drop by 16% with control. Overall, NO(x)'s are reduced by over 80% compared with stoichiometric levels.

In this work, an inexpensive electric utilities monitoring solution using wireless sensor networks is demonstrated that can easily be installed, deployed, maintained and eliminate unnecessary energy costs and effort. The monitoring solution is designed to support network enabled manufacturing (NEM) program using Missouri University of Science and Technology (MST), formerly the University of Missouri-Rolla (UMR), motes.

A novel adaptive distributed rate and power control (ADRPC) protocol is introduced for wireless networks. The proposed controller contrasts from others by providing nonlinear compensation to the problem of transmission power and bit-rate adaptation. The protocol provides control of both signal-to-interference ratio (SIR) and quality-of-service (QoS) support to bit-rate adaptation. Bit-rate adaptation is performed by local estimation of congestion levels, rendering little packet overhead, using Lyapunov based adaptive control methods. Performance of the proposed control scheme is shown through analytical proof and simulation examples.

FACTS devices have been shown to be powerful in damping power system oscillations caused by faults; however, in the multi machine control using FACTS, the control problem involves solving differential-algebraic equations of a power network which renders the available control schemes ineffective due to heuristic design and lack of know how to incorporate FACTS into the network. A method to generate nonlinear dynamic representation of a power system consisting of differential equations alone with universal power flow controller (UPFC) is introduced since differential equations are typically preferred for controller development. Subsequently, backstepping methodology is utilized to reduce the generator oscillations by using a FACTS device after a fault has occurred. Finally, we use neural networks to approximate the nonlinear network dynamics for controller design. The net result is a representation that could be potentially utilized for studying the placement and number of FACTS devices as well as to design a better control scheme for FACTS given a power network. Simulation results justify theoretical conjectures.

In this paper, a novel robust scheme is developed for detecting faults in nonlinear discrete time multi-input and multi-output systems in contrast with the available schemes that are developed in continuous-time. Both state and output faults are addressed by considering separate time profiles. The faults, which could be incipient or abrupt, are modeled using input and output signals of the system. By using nonlinear estimation techniques, the discrete-time system is monitored online. Once a fault is detected, its dynamics are characterized using an online approximator. A stable parameter update law is developed for the online approximator scheme in discrete-time. The robustness, sensitivity, and performance of the fault detection scheme are demonstrated mathematically. Finally, a continuous stir tank reactor (CSTR) is used as a simulation example to illustrate the performance of the fault detection scheme.

In this paper, we consider the use of nonlinear networks towards obtaining nearly optimal solutions to the control of nonlinear discrete-time (DT) systems. The method is based on least squares successive approximation solution of the generalized Hamilton-Jacobi-Bellman (GHJB) equation which appears in optimization problems. Successive approximation using the GHJB has not been applied for nonlinear DT systems. The proposed recursive method solves the GHJB equation in DT on a well-defined region of attraction. The definition of GHJB, pre-Hamiltonian function, HJB equation, and method of updating the control function for the affine nonlinear DT systems under small perturbation assumption are proposed. A neural network (NN) is used to approximate the GHJB solution. It is shown that the result is a closed-loop control based on an NN that has been tuned a priori in offline mode. Numerical examples show that, for the linear DT system, the updated control laws will converge to the optimal control, and for nonlinear DT systems, the updated control laws will converge to the suboptimal control.

In this paper, a suite of adaptive neural network (NN) controllers is designed to deliver a desired tracking performance for the control of an unknown, second-order, nonlinear discrete-time system expressed in nonstrict feedback form. In the first approach, two feedforward NNs are employed in the controller with tracking error as the feedback variable whereas in the adaptive critic NN architecture, three feedforward NNs are used. In the adaptive critic architecture, two action NNs produce virtual and actual control inputs, respectively, whereas the third critic NN approximates certain strategic utility function and its output is employed for tuning action NN weights in order to attain the near-optimal control action. Both the NN control methods present a well-defined controller design and the noncausal problem in discrete-time backstepping design is avoided via NN approximation. A comparison between the controller methodologies is highlighted. The stability analysis of the closed-loop control schemes is demonstrated. The NN controller schemes do not require an offline learning phase and the NN weights can be initialized at zero or random. Results show that the performance of the proposed controller schemes is highly satisfactory while meeting the closed-loop stability.

A novel reinforcement-learning-based dual-control methodology adaptive neural network (NN) controller is developed to deliver a desired tracking performance for a class of complex feedback nonlinear discrete-time systems, which consists of a second-order nonlinear discrete-time system in nonstrict feedback form and an affine nonlinear discrete-time system, in the presence of bounded and unknown disturbances. For example, the exhaust gas recirculation (EGR) operation of a spark ignition (SI) engine is modeled by using such a complex nonlinear discrete-time system. A dual-controller approach is undertaken where primary adaptive critic NN controller is designed for the nonstrict feedback nonlinear discrete-time system whereas the secondary one for the affine nonlinear discrete-time system but the controllers together offer the desired performance. The primary adaptive critic NN controller includes an NN observer for estimating the states and output, an NN critic, and two action NNs for generating virtual control and actual control inputs for the nonstrict feedback nonlinear discrete-time system, whereas an additional critic NN and an action NN are included for the affine nonlinear discrete-time system by assuming the state availability. All NN weights adapt online towards minimization of a certain performance index, utilizing gradient-descent-based rule. Using Lyapunov theory, the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight estimates, and observer estimates are shown. The adaptive critic NN controller performance is evaluated on an SI engine operating with high EGR levels where the controller objective is to reduce cyclic dispersion in heat release while minimizing fuel intake. Simulation and experimental results indicate that engine out emissions drop significantly at 20% EGR due to reduction in dispersion in heat release thus verifying the dual-control approach.

A suite of novel robust controllers is introduced for the pickup operation of microscale objects in a microelectromechanical system (MEMS). In MEMS, adhesive, surface tension, friction, and van der Waals forces are dominant. Moreover, these forces are typically unknown. The proposed robust controller overcomes the unknown contact dynamics and ensures its performance in the presence of actuator constraints by assuming that the upper bounds on these forces are known. On the other hand, for the robust adaptive critic-based neural network (NN) controller, the unknown dynamic forces are estimated online. It consists of an action NN for compensating the unknown system dynamics and a critic NN for approximating a certain strategic utility function and tuning the action NN weights. By using the Lyapunov approach, the uniform ultimate boundedness of the closed-loop manipulation error is shown for all the controllers for the pickup task. To imitate a practical system, a few system states are considered to be unavailable due to the presence of measurement noise. An output feedback version of the adaptive NN controller is proposed by exploiting the separation principle through a high-gain observer design. The problem of measurement noise is also overcome by constructing a reference system. Simulation results are presented and compared to substantiate the theoretical conclusions.

A nonaffine discrete-time system represented by the nonlinear autoregressive moving average with eXogenous input (NARMAX) representation with unknown nonlinear system dynamics is considered. An equivalent affinelike representation in terms of the tracking error dynamics is first obtained from the original nonaffine nonlinear discrete-time system so that reinforcement-learning-based near-optimal neural network (NN) controller can be developed. The control scheme consists of two linearly parameterized NNs. One NN is designated as the critic NN, which approximates a predefined long-term cost function, and an action NN is employed to derive a near-optimal control signal for the system to track a desired trajectory while minimizing the cost function simultaneously. The NN weights are tuned online. By using the standard Lyapunov approach, the stability of the closed-loop system is shown. The net result is a supervised actor-critic NN controller scheme which can be applied to a general nonaffine nonlinear discrete-time system without needing the affinelike representation. Simulation results demonstrate satisfactory performance of the controller.

In this paper, a combined kinematic/torque control law is developed for leader-follower based formation control using backstepping in order to accommodate the dynamics of the robots and the formation in contrast with kinematic-based formation controllers that are widely reported in the literature. A neural network (NN) is introduced along with robust integral of the sign of the error (RISE) feedback to approximate the dynamics of the follower as well as its leader using online weight tuning. It is shown using Lyapunov theory that the errors for the entire formation are asymptotically stable and the NN weights are bounded as opposed to uniformly ultimately bounded (UUB) stability which is typical with most NN controllers. Theoretical results are demonstrated using numerical simulations.

A novel reinforcement-learning based output-adaptive neural network (NN) controller, also referred as the adaptive-critic NN controller, is developed to track a desired trajectory for a class of complex nonlinear discrete-time systems in the presence of bounded and unknown disturbances. The controller includes an observer for estimating states and the outputs, critic, and two action NNs for generating virtual, and actual control inputs. The critic approximates certain strategic utility function and the action NNs are used to minimize both the strategic utility function and their outputs. All NN weights adapt online towards minimization of a performance index, utilizing gradient-descent based rule. A Lyapunov function proves the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight, and observer estimation. Separation principle and certainty equivalence principles are relaxed; persistency of excitation condition and linear in the unknown parameter assumption is not needed. The performance of this controller is evaluated on a spark ignition (SI) engine operating with high exhaust gas recirculation (EGR) levels and experimental results are demonstrated.

No abstract available.

In this paper, we propose a subnetwork key management strategy in which the heterogeneous security requirements of a wireless sensor network are considered to provide differing levels of security with minimum communication overhead. Additionally, it allows the dynamic creation of high security subnetworks within the wireless sensor network and provides subnetworks with a mechanism for dynamically creating a secure key using a novel and dynamic group key management protocol. The proposed energy-efficient protocol utilizes a combination of pre-deployed group keys and initial trustworthiness of nodes to create a level of trust between neighbors in the network. This trust is later used to allow secure communication between neighbors when creating a dynamic, high security subnetwork within the sensor network. Results of simulations of the protocol in Ns2 are presented and the complexity of the protocol is analyzed. The proposed protocol reduces delay by 50% and energy consumption by 70% over the existing dynamic group key management (DGKM) scheme.

Literature indicates that spatial diversity can be utilized to compensate channel uncertainties such as multipath fading. Therefore, in this paper, spatial diversity is exploited for locating stationary and mobile objects in the indoor environment. First, space diversity technique is introduced for small scale motion and temporal variation compensation of received signal strength and it is demonstrated analytically that it enhances location accuracy. Small scale motion refers to movements of the transmitter and/or the receiver of the order of sub-wavelengths while temporal effects refer to environmental variations with time. A novel metric is introduced for selection combining in order to improve location accuracy through the addition of spatial diversity upon two available location determination schemes. The results are evaluated experimentally against single antenna system for reception by using low cost wireless RF devices such as motes. Alternatively, the impact of the number of location determination devices in a probabilistic WLAN network based on pre-profiling of signal strength is analyzed and it is demonstrated analytically that location accuracy improves with the number of receivers used. Spatial diversity in terms of the antenna spacing of 2lambda is evaluated and shown to provide a reduction in location determination error between 30 and 40% when compared to a single antenna system.

Literature indicates that frequency diversity can be utilized to compensate channel uncertainties such as multipath fading. Therefore, in this paper it is exploited for improving accuracy in locating stationary and mobile objects in the indoor environment. First, frequency diversity technique is introduced for small scale and temporal variation compensation of received signals and demonstrated analytically that it in fact enhances location accuracy. A novel metric is introduced in selection combining in order to achieve location accuracy through the addition of frequency diversity upon two of the available location determination schemes. The results are evaluated experimentally against the case where there is no frequency diversity for reception by using low cost wireless RF devices such as motes. An asset location tracking system is then devised to both improve accuracy and predict asset movement. Frequency diversity in terms of channel spacing of 55 MHz is evaluated and shown to provide a reduction in location determination error between 18 and 23% when compared to a system without frequency diversity. Finally, results from frequency diversity are compared against the spatial diversity techniques in terms of improvement in location accuracy, power consumption of the transmitter, and hardware and software costs.

Past research has shown substantial reductions in the oxides of nitrogen (NOx) concentrations by using 10%-25% exhaust gas recirculation (EGR) in spark ignition (SI) engines (see Dudek and Sain, 1989). However, under high EGR levels, the engine exhibits strong cyclic dispersion in heat release which may lead to instability and unsatisfactory performance preventing commercial engines to operate with high EGR levels. A neural network (NN)-based output feedback controller is developed to reduce cyclic variation in the heat release under high levels of EGR even when the engine dynamics are unknown by using fuel as the control input. A separate control loop was designed for controlling EGR levels. The stability analysis of the closed-loop system is given and the boundedness of the control input is demonstrated by relaxing separation principle, persistency of excitation condition, certainty equivalence principle, and linear in the unknown parameter assumptions. Online training is used for the adaptive NN and no offline training phase is needed. This online learning feature and model-free approach is used to demonstrate the applicability of the controller on a different engine with minimal effort. Simulation results demonstrate that the cyclic dispersion is reduced significantly using the proposed controller when implemented on an engine model that has been validated experimentally. For a single cylinder research engine fitted with a modern four-valve head (Ricardo engine), experimental results at 15% EGR indicate that cyclic dispersion was reduced 33% by the controller, an improvement of fuel efficiency by 2%, and a 90% drop in NOx from stoichiometric operation without EGR was observed. Moreover, unburned hydrocarbons (uHC) drop by 6% due to NN control as compared to the uncontrolled scenario due to the drop in cyclic dispersion. Similar performance was observed with the controller on a different engine.

A novel adaptive-critic-based neural network (NN) controller in discrete time is designed to deliver a desired tracking performance for a class of nonlinear systems in the presence of actuator constraints. The constraints of the actuator are treated in the controller design as the saturation nonlinearity. The adaptive critic NN controller architecture based on state feedback includes two NNs: the critic NN is used to approximate the "strategic" utility function, whereas the action NN is employed to minimize both the strategic utility function and the unknown nonlinear dynamic estimation errors. The critic and action NN weight updates are derived by minimizing certain quadratic performance indexes. Using the Lyapunov approach and with novel weight updates, the uniformly ultimate boundedness of the closed-loop tracking error and weight estimates is shown in the presence of NN approximation errors and bounded unknown disturbances. The proposed NN controller works in the presence of multiple nonlinearities, unlike other schemes that normally approximate one nonlinearity. Moreover, the adaptive critic NN controller does not require an explicit offline training phase, and the NN weights can be initialized at zero or random. Simulation results justify the theoretical analysis.

Available congestion control schemes, for example transport control protocol (TCP), when applied to wireless networks, result in a large number of packet drops, unfair scenarios and low throughputs with a significant amount of wasted energy due to retransmissions. To fully utilize the hop by hop feedback information, this paper presents a novel, decentralized, predictive congestion control (DPCC) for wireless sensor networks (WSN). The DPCC consists of an adaptive flow and adaptive back-off interval selection schemes that work in concert with energy efficient, distributed power control (DPC). The DPCC detects the onset of congestion using queue utilization and the embedded channel estimator algorithm in DPC that predicts the channel quality. Then, an adaptive flow control scheme selects suitable rate which is enforced by the newly proposed adaptive backoff interval selection scheme. An optional adaptive scheduling scheme updates weights associated with each packet to guarantee the weighted fairness during congestion. Closed-loop stability of the proposed hop-by-hop congestion control is demonstrated by using the Lyapunov-based approach. Simulation results show that the DPCC reduces congestion and improves performance over congestion detection and avoidance (CODA) [3] and IEEE 802.11 protocols.

This paper presents a neural network (NN) based decentralized excitation controller design for large scale power systems. The proposed controller design considers not only the dynamics of generators but also the algebraic constraints of the power flow equations. The control signals are calculated using only local signals. The transient stability and the coordination of the subsystem controllers can be guaranteed. NNs are used to approximate the unknown/imprecise dynamics of the local power system and the interconnections. All signals in the closed loop system are guaranteed to be uniformly ultimately bounded (UUB). Simulation results with a 3-machine power system demonstrate the effectiveness of the proposed controller design.

Research has shown substantial reductions in the oxides of nitrogen (NOx) concentrations by using 10% to 25% exhaust gas recirculation (EGR) in spark ignition (SI) engines (Dudek and Sain, 1989). However under high EGR levels the engine exhibits strong cyclic dispersion in heat release which may lead to instability and unsatisfactory performance. A suite of neural network (NN)-based output feedback controllers with and without reinforcement learning is developed to control the SI engine at high levels of EGR even when the engine dynamics are unknown by using fuel as the control input. A separate control loop was designed for controlling EGR levels. The neural network controllers consists of three NN: a) ANN observer to estimate the states of the engine such as total fuel and air; b) a second NN for generating virtual input; and c) a third NN for generating actual control input. For reinforcement learning, an additional NN is used as the critic. The stability analysis of the closed loop system is given and the boundedness of all signals is ensured without separation principle. Online training is used for the adaptive NN and no offline training phase is needed. Experimental results obtained by testing the controller on a research engine indicate an 80% drop of NOxfrom stoichiometric levels using 10% EGR. Moreover, unburned hydrocarbons drop by 25% due to NN control as compared to the uncontrolled scenario.

Resource constraints require that ad hoc wireless networks are energy efficient during transmission and rate adaptation. In this paper we propose a novel cross-layer energy-efficient rate adaptation scheme that employs dynamic programming (DP) principle to analytically select the modulation scheme online. The scheme uses channel state information from the physical layer and congestion information from the scheduling layer to select a modulation rate. This online selection maximizes throughput while saving energy and preventing congestion. The simulation results indicate that an increase in throughput by 96% and energy-efficiency by 131% is observed when compared to the Receiver Based AutoRate (RBAR) protocol

A novel adaptive and distributed fair scheduling (ADFS) scheme for wireless sensor networks is shown through hardware implementation. In contrast to simulation, hardware evaluation provides valuable feedback to protocol and hardware development process. The proposed protocol focuses on quality-of-service (QoS) issues to address flow prioritization. Thus, when nodes access a shared channel, the proposed ADFS allocates the channel bandwidth proportionally to the weight, or priority, of the packet flows. Moreover, ADFS allows for dynamic allocation of network resources with little added overhead. Weights are initially assigned using user specified QoS criteria. These weights are subsequently updated as a function of delay, enqueued packets, flow arrival rate, and the previous packet weight. The back-off interval is also altered using the weight update equation. The weight update and the back-off interval selection ensure that global fairness is attained even with variable service rates. The algorithm is implemented using UMR/SLU motes for an industrial monitoring application. Results the hardware implementation demonstrates improved performance in terms of fairness index, flow rate, and delay.

In this paper, a novel distributed power control (DPC) scheme for cellular network in the presence of radio channel uncertainties such as path loss, shadowing, and Rayleigh fading is presented. Since these uncertainties can attenuate the received signal strength and can cause variations in the received signal-to-interference ratio (SIR), a new DPC scheme, which can estimate the slowly varying channel uncertainty, is proposed so that a target SIR at the receiver can be maintained. Further, the standard assumption of a constant interference during a link's power update used in other works in the literature is relaxed. A CDMA-based cellular network environment has been developed to compare the proposed scheme with earlier approaches. The results show that our DPC scheme can converge faster than others by adapting to the channel variations. In the presence of channel uncertainties, our DPC scheme renders lower outage probability while consuming significantly low power per active mobile user compared with other schemes that are available in the literature.

Resource constraints in ad hoc wireless networks require that they are energy efficient during both transmission and rate adaptation. In this paper, we propose a novel energy-efficient rate adaptation protocol that selects modulation schemes online to maximize throughput based on channel state while saving energy. This protocol uses the distributed power control (DPC) algorithm (M. Zawodniok et al., 2004) to accurately determine the necessary transmission power and to reduce the energy consumption. Additionally, the transmission rate is altered using energy efficiency as a constraint to meet the required throughput, which is estimated with queue fill ratio. Moreover, back-off scheme is incorporated to prevent energy wastage and to avoid retransmissions in the event of congestion. The back-off scheme employs backpressure mechanism to emulate congestion control. Consequently, the nodes will conserve energy when the traffic is low, offer higher throughput when needed and save energy during congestion by limiting transmission.

A novel neural network (NN)-based output feedback controller with magnitude constraints is designed to deliver a desired tracking performance for a class of multi-input and multi-output (MIMO) strict feedback nonlinear discrete-time systems. Reinforcement learning is proposed for the output feedback controller, which uses three NNs: 1) an NN observer to estimate the system states with the input-output data, 2) a critic NN to approximate certain strategic utility function, and 3) an action NN to minimize both the strategic utility function and the unknown dynamics estimation errors. Using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the state estimation errors, the tracking errors and weight estimates is shown.

Power system stabilizers are widely used to generate supplementary control signals for the excitation system in order to damp out the low frequency oscillations. This paper proposes a stable neural network (NN) controller for the stabilization of a single machine infinite bus power system. In the power system control literature, simplified-analytical models are used to represent the power system and the controller designs are not based on rigorous stability analysis. This work overcomes the two major problems by using an accurate analytical model for controller development and presents the closed-loop stability analysis. The NN is used to approximate the complex nonlinear power system online and the weights of which can be set to zero to avoid the time consuming offline training process. Magnitude constraint of the activators is modeled as saturation nonlinearities and is included in the Lyapunov stability analysis. Simulation results demonstrate that the proposed design can successfully damp out oscillations. The control algorithms of This work can also be applied to other similar control problems.

A novel distributed power control (DPC) scheme for cellular networks in the presence of radio channel uncertainties such as path loss, shadowing, and Rayleigh fading is presented. Since these uncertainties can attenuate the received signal strength and can cause variations in the received signal-to-interference ratio (SIR), the proposed DPC scheme maintains a target SIR at the receiver provided the uncertainty is slowly varying with time. The DPC estimates the time varying nature of the channel quickly and uses the information to arrive at a suitable transmitter power value. Further, the standard assumption of a constant interference during a link's power update used in other works in the literature is relaxed. A CDMA-based cellular network environment is used to compare the proposed scheme with earlier approaches. The results show that our DPC scheme can converge faster than others by adapting to the channel variations. The proposed DPC scheme can render outage probability of 5 to 30% in the presence of uncertainties compared with other schemes of 50 to 90% while consuming low power per active mobile user. In other words, the proposed DPC scheme allows significant increase in network capacity while consuming low power values even when the channel is uncertain.

A novel distributed power control (DPC) scheme and a MAC protocol for wireless ad hoc networks in the presence of radio channel uncertainties such as path loss, Shadowing and Rayleigh fading is presented. The DPC quickly estimates the time-varying nature of the channel and uses the information to select a suitable transmitter power value in order to maintain a target signal-to-interference ratio (SIR) at the receiver. The standard assumption of a constant interference during a link's power update used in other works is relaxed. The performance of the proposed DPC is demonstrated analytically. The power used for all RTS-CTS-DATA-ACK frames is selected using the proposed DPC; hence, energy savings and spatial reuse are achieved. The hidden-terminal problem is overcome by periodically increasing the power. The NS simulator is used to compare the proposed scheme with 802.11. The proposed MAC protocol renders a significant increase in throughput in the presence of channel variations compared with 802.11 while consuming low energy per bit.

Grasping of objects has been a challenging task for robots. The complex grasping task can be defined as object contact control and manipulation subtasks. In this paper, object contact control subtask is defined as the ability to follow a trajectory accurately by the fingers of a gripper. The object manipulation subtask is defined in terms of maintaining a predefined applied force by the fingers on the object. A sophisticated controller is necessary since the process of grasping an object without a priori knowledge of the object's size, texture, softness, gripper, and contact dynamics is rather difficult. Moreover, the object has to be secured accurately and considerably fast without damaging it. Since the gripper, contact dynamics, and the object properties are not typically known beforehand, an adaptive critic neural network (NN)-based hybrid position/force control scheme is introduced. The feedforward action generating NN in the adaptive critic NN controller compensates the nonlinear gripper and contact dynamics. The learning of the action generating NN is performed on-line based on a critic NN output signal. The controller ensures that a three-finger gripper tracks a desired trajectory while applying desired forces on the object for manipulation. Novel NN weight tuning updates are derived for the action generating and critic NNs so that Lyapunov-based stability analysis can be shown. Simulation results demonstrate that the proposed scheme successfully allows fingers of a gripper to secure objects without the knowledge of the underlying gripper and contact dynamics of the object compared to conventional schemes.

This paper proposes a fair, rate-based end-to-end congestion control mechanism for multimedia traffic in packet switched networks such as the Internet. The scheme is modeled after a non-linear system. The congestion is controlled by adjusting the transmission rates of the sources in response to the feedback information from destination such as the buffer occupancy, packet arrival rate and service rate, so that a desired quality of service (QoS) can be met. The QoS is defined in terms of packet loss ratio, transmission delay, power and fairness. The performance and stability of the system is analyzed mathematically. The proposed scheme has been implemented in the NS-2 simulator. Simulation results demonstrate the performance of the scheme to be in agreement with mathematical analysis. The proposed scheme is shown to provide considerable improvements in terms of the QoS metrics over New-Reno TCP technique.

A novel neural network (NN) controller is developed to control spark ignition (SI) engines at extreme lean conditions. The purpose of neurocontroller is to reduce the cyclic dispersion at lean operation even when the engine dynamics are unknown. The stability analysis of the closed-loop control system is given and the boundedness of all signals is ensured. Results demonstrate that the cyclic dispersion is reduced significantly using the proposed controller. The neuro controller can also be extended to minimize engine emissions with high EGR levels, where similar complex cyclic dynamics are observed. Further, the proposed approach can be applied to control nonlinear systems that have similar structure as that of the engine dynamics.

Energy efficiency is a measure of performance in wireless networks. Therefore, controlling the transmitter power at a given node increases not only battery operating life, but also overall system capacity by successfully admitting new links. It is essential to find effective means of power control in point-to-point, broadcasting and multicasting scenarios. Wireless networking presents formidable challenges and we consider the problem of unicast or point-to-point (peer-to-peer) communication in wireless networks in the presence of other nodes. We study the feasibility of admitting new links in an wireless network operating area while maintaining quality of service (QoS), in terms of signal-to-interference ratio (SIR), for each link. SIR is maintained by adjusting the transmitter power levels at each source for a given link. Distributed power control (DPC) is a natural choice for this purpose because, unlike centralized power control, DPC should be able to adjust the power levels of each transmitted signal using local measurements, so that in a reasonable time, all nodes/links maintain the desired SIR. We present a suite of DPC schemes using both state space and optimal control methodology in discrete-time. Further, we prove the convergence of the overall network with our algorithm using Lyapunov stability analysis in comparison with a well known DPC scheme (see Bambos, N. et al., IEEE ACM Trans. on Networking, p.583-97, 2000). We present simulation results and comparisons for point to point communications in an overlapping scenario.

A novel real-time discrete-event admission control (AC) scheme for high-speed networks is proposed with the aim of attaining a desired quality of service (QoS) and high network utilization. The AC uses the available capacity from a novel adaptive bandwidth estimation scheme, a congestion indicator derived from a congestion controller, peak bit/cell rate (PBR/PCR) estimate from new sources, along with the desired QoS metrics, and makes decisions whether to 'admit' or 'reject' new sources. The novel aspect of the proposed approach is the application of hybrid system theory to prove the performance of the admission controller, stability and the development of rigorous and repeatable design procedure. The proposed AC is evaluated using the QoS metrics, which are given in terms of service delay, packet/cell losses, and network utilization. Simulation results are presented by streaming ON/OFF and MPEG video data into the network.

This paper proposes an adaptive methodology to prevent congestion in packet switched networks such as the Internet, where the internal network nodes convey very little information to the ingress nodes. Two architectures of preventing the congestion are presented: the first one when the traffic arrival rates and bottleneck queue levels are known and the other when these are unknown. In the latter, the network traffic is estimated online using an adaptive system by measuring the buffer occupancy. In both architectures, the congestion is controlled by adjusting the transmission rates of non real-time and certain real-time sources in response to the feedback information so that a desired quality of service (QoS) can be met instead of using the existing additive increase multiplicative decrease (AIMD) algorithm. The QoS is defined in terms of packet loss, transmission delay, network utilization and fairness. Mathematical analysis is given to demonstrate the stability of the closed-loop system. Studies are included to show the effectiveness of the proposed scheme during simulated congestion. The proposed methodology can be readily applied to network planning, designing routing algorithms and transmission links as well as for real-time video and voice data transfer in unicast networks.

A multilayer neural-network (NN) controller is designed to deliver a desired tracking performance for the control of a class of unknown nonlinear systems in discrete time where the system nonlinearities do not satisfy a matching condition. Using the Lyapunov approach, the uniform ultimate boundedness of the tracking error and the NN weight estimates are shown by using a novel weight updates. Further, a rigorous procedure is provided from this analysis to select the NN controller parameters. The resulting structure consists of several NN function approximation inner loops and an outer proportional derivative tracking loop. Simulation results are then carried out to justify the theoretical conclusions. The net result is the design and development of an NN controller for strict-feedback class of nonlinear discrete-time systems.

The local structure of CMAC neural networks (NN) result in better and faster controllers for nonlinear dynamical systems. A CMAC neural network-based discrete-time controller which linearizes the unknown multiinput and multioutput (MIMO) nonlinear system through feedback is presented. Control action is defined in order to achieve tracking performance for this unknown nonlinear system. An efficient and localized weight addressing scheme for the CMAC NN's is described using an appropriate choice of the B-spline receptive field functions that form a basis. A uniform ultimate boundedness of the closed-loop system is given in the sense of Lyapunov using the persistency of excitation (PE) condition. Simulation results are shown to demonstrate the theoretical conclusions.

This paper presents a practical method of achieving improved performance from existing industrial robot controllers using intelligent control techniques. While there have been many solutions to dealing with the unmodeled dynamics and disturbances in a robotic system, these results have not found much acceptance in the industrial community as they require a complete redesign of the controller. The strategy presented here uses concepts and results from the intelligent control literature, to build on the existing linear controller. It is shown that the existing conventional linear controller can be augmented by a control component that can compensate for the unmodeled dynamics and disturbances. This component can be manufactured by neural networks, fuzzy logic controllers, or adaptive controllers as long as they satisfy key approximation properties. In this paper, this concept is demonstrated by implementing the design using CMAC neural networks (CMAC NN). The adaptation laws for this controller are designed and the closed-loop performance of the overall system is rigorously proved.

The discrete-time implementation of the controllers are of importance, since almost all the implementation of controllers are done on a digital computer. Therefore, this paper attempts to provide a comprehensive treatment of neural network (NN) controller design in discrete-time for the control of a multi-input multi-output robot arm using neural networks. The NN controller exhibits learning-while-functioning-feature instead of learning-then-control and do not need the dynamics of the robotic system apriori. The structure of the NN controller is derived using filtered error notions. A uniform ultimate boundedness of the closed-loop system is given in the sense of Lyapunov. Certainty equivalence is not used, persistency of excitation is not required and regression matrix is not computed, New online tuning algorithms in discrete-time are derived, which are similar to /spl epsiv/-modification for the case of continuous-time systems, and guarantee tracking as well as bounded NN weights in nonideal situations.

This paper demonstrates tracking control of a class of feedback linearizable unknown nonlinear dynamical systems, such as a robotic systems, using a discrete-time fuzzy logic controller (FLC). Designing a discrete-time FLC is significant because almost all FLC's are implemented on digital computers. A repeatable design algorithm and a stability proof are examined for an adaptive fuzzy logic controller that uses fuzzy basis functions based on the fuzzy system, unlike most standard adaptive control approaches which use basis vectors depending on the unknown plant. An /spl epsiv/-modification sort of approach to adapt the fuzzy system parameters is examined. Using this adaptive fuzzy logic controller, uniform ultimate boundedness of the closed-loop signals is presented and that the controller achieves tracking. In fact, the fuzzy system designed is a model-free universal fuzzy controller that works for any system in the given class of systems.

A family of novel multilayer discrete-time neural-net (NN) controllers is presented for the control of a class of multi-input multi-output (MIMO) dynamical systems. The neural net controller includes modified delta rule weight tuning and exhibits a learning while-functioning-features. The structure of the NN controller is derived using a filtered error/passivity approach. Linearity in the parameters is not required and certainty equivalence is not used. This overcomes several limitations of standard adaptive control. The notion of persistency of excitation (PE) for multilayer NN is defined and explored. New online improved tuning algorithms for discrete-time systems are derived, which are similar to sigma or epsilon-modification for the case of continuous-time systems, that include a modification to the learning rate parameter plus a correction term. These algorithms guarantee tracking as well as bounded NN weights in nonideal situations so that PE is not needed. An extension of these novel weight tuning updates to NN with an arbitrary number of hidden layers is discussed. The notions of discrete-time passive NN, dissipative NN, and robust NN are introduced. The NN makes the closed-loop system passive.