Coordination strategies in large-scale systems are mainly based on two principles: interaction prediction and interaction balance. Using these principles, Model coordination and Goal coordination were proposed. The interactions in the first method and the Lagrangian coefficients in the second method were considered as coordination parameters. In this paper, the concept of coordination is introduced within the framework of two-level large-scale systems and a new intelligent approach for Model coordination is introduced. For this purpose, the system is decomposed into several subsystems, and the overall problem is considered as an optimization problem. With the aim of optimization, the control problem is first decomposed into m sub-problems at the first level, where each sub-problem is solved using a fuzzy optimal control strategy. Three-layer neural networks, which are capable of learning and reconstructing nonlinear mappings, are used for modelling each corresponding subsystem. Also, by using a neural network at the second level, which uses the gradient of interaction errors for learning, the change of interactions for each sub-system is predicted so that the overall system can be coordinated. The efficacy and advantages of the proposed approach is shown in an application example.

Customer assessment plays an important role in make to order production systems for ranking of the orders. This paper extents α-cut and interactive Fuzzy Multi-Objective Linear Programming combine approaches for solving the imprecise customer assessment for order acceptance problem under price breaks with fuzzy goals and fuzzy constraints with linear Membership Function. The proposed combined approach aims to simultaneously minimize the storage cost, pay delay time, conflict rate and transportation cost. The fifth objective function maximize the total value of purchasing. This combined approach enables decision maker to interactively modify fuzzy coefficients and constraints until attained both an efficient compromise solution and increase overall decision maker satisfaction degree. Additionally, the proposed combination approach provides a systematic framework that facilitates the fuzzy decision-making process. The fuzzy combined approach was utilized on an industrial case, and findings will be shown.

Energy time series forecasting plays a crucial role in the process of energy planning. This topic has been, and is still attracting vast research activities that are performed by researchers in the academia and energy companies. Various techniques exist for energy time series forecasting, and the selection of the most suitable forecasting algorithm is not an easy process. For a clear application of such techniques in energy time series forecasting, there should be a clear distinction between these techniques. This paper compares the overall performance of the Time Delay Neural Network (TDNN), Neuro Fuzzy Inference System and Support Vector Regression (SVR). The efficiency of these techniques is compared in energy time series forecasting, and the performances of them are tested. The results of our analysis indicate that the Time Delay Neural network (TDNN) shows the best performances overall.

Due to the increasing data volume, it is difficult for direct marketing decision makers to find target customers. Computational intelligence models that computerize human analysis have recognized a testing device for customer response modeling and target customers identification. Widespread research has resulted in numerous direct marketing applications using computational intelligence models; customer response modeling is an important activity of direct marketing. This paper proposes a hybrid model for customer response modeling. The Proposed model evolves neural network primary weights by genetic algorithms and optimizes it by back-propagation. We also utilize data pre-processing methods such as data reduction for improving precision of the model. We test ability of the proposed method by applying it to standard data set. Results show that the proposed approach is capable to deal with the customer response modeling and it also yields superior prediction accuracy.

The System of Systems (SOS) technology is an advanced technology for Intelligent Systems that is developed with multiple intelligent systems. Recently, there has been a growing interest in a class of complex systems (robotic swarm as an example) whose constituents are themselves complex. Performance optimization, robustness and reliability among an emerging group of heterogeneous systems in order to realize a common goal have become the focus of various applications including military, security, aerospace, space, manufacturing, service industry, environmental systems, and disaster management, to name a few. In this paper, we discuss how Fuzzy Reinforcement Learning (FRL) can be used in SOS.

Multi-agent prognostic health and usage monitoring (Multi-PHUM) is proposed in the paper for use in fault diagnosis and prognosis. Georgia Tech has developed a Matlab simulation model of monopropellant propulsion for studying fault diagnosis. Here we apply Multi-PHUM to the fault diagnosis of the monopropellant propulsion system. In particular, regulator failure is discussed here.

Case-based reasoning (CBR) is a mature technology used in help-desk customer services. To implement this technology to the diagnosis and prognosis of dynamic systems, many important factors must be considered. This paper reports on our development of a Matlab system for multi-agent fault diagnosis for predictive health monitoring. In particular, we apply this technology to the SIMULINK simulation of a dynamic chiller system. Our system can detect single component faults and also multiple component faults in the system and advise on how to fix the problem using case based reasoning.

Wavelet Neural Networks have been developed for fault diagnosis and prognosis with unique capabilities in addressing identification and classification problems. A fault diagnostic and prognostic system is presented by using Wavelet Neural Networks and Dynamic Wavelet Neural Networks. We used a Matlab Simulink model of a chiller system and applied the Wavelet Neural Network to detect and recover sensor errors. It has been shown that this method gains good performance in sensor fault diagnostics.

Dynamic case based reasoning (DCBR) is a powerful technique for fault diagnosis and prognosis. DCBR allows the accumulation of experience from inclusion of the new cases and therefore accommodates learning. In this paper, we discuss DCBR and develop a methodology for using it in a dynamic model of a chiller system

We address the issue of power-controlled shared channel access in wireless networks supporting packetized data traffic. We formulate this problem using the dynamic programming framework and present a new distributed fuzzy reinforcement learning algorithm (ACFRL-2) capable of adequately solving a class of problems to which the power control problem belongs. Our experimental results show that the algorithm converges almost deterministically to a neighborhood of optimal parameter values, as opposed to a very noisy stochastic convergence of earlier algorithms. The main tradeoff facing a transmitter is to balance its current power level with future backlog in the presence of stochastically changing interference. Simulation experiments demonstrate that the ACFRL-2 algorithm achieves significant performance gains over the standard power control approach used in CDMA2000. Such a large improvement is explained by the fact that ACFRL-2 allows transmitters to learn implicit coordination policies, which back off under stressful channel conditions as opposed to engaging in escalating "power wars."

Three individual experts have been developed based on extended auto associative neural networks (E-AANN), Kohonen self organizing maps (KSOM), and the radial basis function based clustering (RBFC) algorithms. An integrated method is proposed later to combine the set of individual experts managed by a gated experts algorithm, which assigns the experts based on their best performance regions. We have used a Matlab Simulink model of a chiller system and applied the individual experts and the integrated method to detect and recover sensor errors. It has been shown that the integrated method gets better performance in diagnostics and prognostics compared with each individual expert.

There is a steadily increasing need for autonomous systems that must be able to function with minimal human intervention to detect and isolate faults, and recover from such faults. In this paper we present a novel hybrid Model based and Data Clustering (MDC) architecture for fault monitoring and diagnosis, which is suitable for complex dynamic systems with continuous and discrete variables. The MDC approach allows for adaptation of both structure and parameters of identified models using supervised and reinforcement learning techniques. The MDC approach will be illustrated using the model and data from the Hybrid Combustion Facility (HCF) at the NASA Ames Research Center.

In this paper we study the problem of sensor allocation in Unmanned Aerial Vehicles (UAVs). Each UAV uses perception-based rules for generalizing decision strategy across similar states and reinforcement learning for adapting these rules to the uncertain, dynamic environment. A big challenge for reinforcement learning algorithms in this problem is that UAVs need to learn two complementary policies: how to allocate their individual sensors to appearing targets and how to distribute themselves as a team in space to match the density and importance of targets underneath. We address this problem using a co-evolutionary approach, where the policies are learned separately, but they use a common reward function. The applicability of our approach to the UAV domain is verified using a high-fidelity robotic simulator. Based on our results, we believe that the co-evolutionary reinforcement learning approach to reducing dimensionality of the action space presented in this paper is general enough to be applicable to many other multi-objective optimization problems, particularly those that involve a tradeoff between individual optimality and team-level optimality.

This paper provides the first convergence proof for fuzzy reinforcement learning (FRL) as well as experimental results supporting our analysis. We extend the work of Konda and Tsitsiklis, who presented a convergent actor-critic (AC) algorithm for a general parameterized actor. In our work we prove that a fuzzy rulebase actor satisfies the necessary conditions that guarantee the convergence of its parameters to a local optimum. Our fuzzy rulebase uses Takagi-Sugeno-Kang rules, Gaussian membership functions, and product inference. As an application domain, we chose a difficult task of power control in wireless transmitters, characterized by delayed rewards and a high degree of stochasticity. To the best of our knowledge, no reinforcement learning algorithms have been previously applied to this task. Our simulation results show that the ACFRL algorithm consistently converges in this domain to a locally optimal policy.

Fuzzy set theory was recently shown to be an effective tool for generalizing the learned experience between similar states in reinforcement learning problems with large or continuous state spaces. In our previous work (2001) we presented the first convergence proof for an algorithm combining fuzzy sets and reinforcement learning. In this paper we apply our algorithm to a very challenging wireless power control problem characterized by heavily delayed rewards combined with several sources of randomness. The results show a considerable improvement in performance as compared to the optimal constant power transmission.

We present an adaptive multi-agent coordination algorithm applied to the problem of distributed dynamic load balancing. As a specific example, we consider the problem of dynamic web caching in the Internet. In our general formulation of this problem, each agent represents a mirrored piece of content that tries to move itself closer to areas of the network with a high demand for this item. Each agent in our model uses a fuzzy rule base for choosing the optimal direction of motion and adjusts the parameters of this rule base using reinforcement learning. The resulting architecture for multi-agent coordination among fuzzy reinforcement learning agents (MAC-FRL) allows the team of agents to adaptively redistribute its members in the environment to match the changing pattern of demand. We simulate the performance of MAC-FRL and show that it significantly improves performance over non-coordinating agents.

This paper provides the first convergence proof for fuzzy reinforcement learning. We extend the work of Konda and Tsitsiklis (2000), who presented a convergent actor-critic algorithm for a general parameterized actor. In our work we prove that a fuzzy rule base actor satisfies the necessary conditions that guarantee the convergence of its parameters to a local optimum. Our fuzzy rule base uses the Takagi-Sugeno-Kang rules, Gaussian membership functions and product inference.

Presents the first results in understanding the reasons for cooperative advantage between reinforcement learning agents. We consider a cooperation method which consists of using and updating a common policy. We tested this method on a complex fuzzy reinforcement learning problem and found that cooperation brings larger than expected benefits. More precisely, we found that K cooperative agents each learning for N time steps outperform K independent agents each learning in a separate world for K*N time steps. We explain the observed phenomenon and determine the necessary conditions for its presence in a wide class of reinforcement learning problems.

A method for learning and tuning a fuzzy logic controller based on reinforcements from a dynamic system is presented. It is shown that: the generalized approximate-reasoning-based intelligent control (GARIC) architecture learns and tunes a fuzzy logic controller even when only weak reinforcement, such as a binary failure signal, is available; introduces a new conjunction operator in computing the rule strengths of fuzzy control rules; introduces a new localized mean of maximum (LMOM) method in combining the conclusions of several firing control rules; and learns to produce real-valued control actions. Learning is achieved by integrating fuzzy inference into a feedforward network, which can then adaptively improve performance by using gradient descent methods. The GARIC architecture is applied to a cart-pole balancing system and demonstrates significant improvements in terms of the speed of learning and robustness to changes in the dynamic system's parameters over previous schemes for cart-pole balancing.