This paper addresses the H∞ state feedback stabilisation problem for networked control systems (NCSs) in the presence of time delays and packet losses. By introducing the concept of an effective sensor packet, the NCS is transformed into a new discrete-time switched model, where the parameters have a clear physical meaning and can be easily determined. In this framework, we derive the stability conditions of the closed-loop system in the H∞ sense, and also provide the corresponding H∞ stabilising controller design method. Finally, we give simulation and experimental results to demonstrate the effectiveness of the proposed approaches.

Estimation of distribution algorithms(EDA) are a class of recently-developed evolutionary algorithms in which the probabilistic model are used to explicitly characterize the distribution of the population and to generate new individuals. The polynomial distribution is applied by discrete EDAs and continuous EDAs based on discretization of the domain such as histogram-based EDA. We can unify those kinds of EDA from their distribution and call them PolyEDA. In this paper, we theoretically analyze PolyEDA from a Bayesian analysis view. Our analysis is based on the assumption that the prior distribution of the parameters satisfies a Dirichlet Distribution, because under this assumption the formulation can be analytically solved. Furthermore, we notice that the prior distribution is always overlooked by previous algorithms, so we follow this way and propose some strategies to improve the PolyEDA. The experimental results show that these new strategies can help the polynomial model based estimation of distribution algorithms achieve better convergence and diversity.

Marginal probability distribution has been widely used as the probabilistic model in EDAs because of its simplicity and efficiency. However, the obvious shortcoming of the kind of EDAs lies in its incapability of taking the correlation between variables into account. This paper tries to solve the problem from the point view of space transformation. As we know, it seems a default rule that the probabilistic model is usually constructed directly from the selected samples in the space defined by the problem. In the algorithm CM-MEDA, instead, we first transform the sampled data from the initial coordinate space into the characteristic space of covariance-matrix and then the marginal probabilistic model is constructed in the new space. We find that the marginal probabilistic model in the new space can capture the variable linkages in the initial space quite well. The relationship of CM-MEDA with Covariance-Matrix estimation and principal component analysis is also analyzed in this paper. We implement CM-MEDA in continuous domain based on both Gaussian and histogram models. The experimental results verify the effectiveness of our idea.

This paper proposes a simple trajectory generation method for biped walking. This method needs only three trajectory control parameters, which are the stepping cycle, the length of a step and the max height of swing foot. The trajectory of robot's stance leg is generated by using the 3DLIPM (Three Dimensional Linear Inverted Pendulum Mode), and the combination of sinusoid curves is used to generate the swing leg's trajectory. This method has been applied on an actual biped robot. The result shows that this simple method is suitable for practical application well.

Conventional cost functions of adaptive filtering are usually related to the error’s dispersion, such as error’s moments or error’s entropy, but neglect the shape aspects (peaks, kurtosis, tails, etc.) of the error distribution. In this work, we propose a new notion of filtering (or estimation) in which the error’s probability density function (PDF) is shaped into a desired one. As PDFs contain all the probabilistic information, the proposed method can be used to achieve the desired error variance or error entropy, and is expected to be useful in the complex signal processing and learning systems. In our approach, the information divergence between the actual errors and the desired errors is used as the cost function. By kernel density estimation, we derive the associated stochastic gradient algorithm for the finite impulse response (FIR) filter. Simulation results emphasize the effectiveness of this new algorithm in adaptive system training.

In continuous domain, how to efficiently learn the complex probabilistic graphical model is a bottleneck problem for estimation of distribution algorithms (EDAs). The predominant researches focus on Gaussian probabilistic model instead of histogram distribution model because of its comparative superiority in the computational complexity. In this paper, however, we find that using the histogram model does not necessarily bring into exponential computational complexity. Based on the fact many bins are zero-height, we propose a novel method that can learn the multivariate dependency histogram based probabilistic graphical model with acceptable polynomial computational complexity. Several strategies previously used in the HEDA are combined into the new algorithm to improve the convergence and diversity. Experiments showed the superior performance of the new algorithm on several continuous problems compared with UMDAc, IDEA-G and sur-shr-HEDA.

This paper gives a matrix interpretation of generalized embedded landscape which is a class of additive decomposable functions mapping from high-cardinality alphabets domain to real numbers. Discrete Fourier transform is used to analyze the epistatic structure. We theoretically show the close relationship between Fourier coefficients and the underlying epistatic structure.

The work addresses the problem of identifying the epistatic linkage of a function from high cardinality alphabets to the real numbers. It is a generalization of Heckendorn and Wright's work that restricts problem representation into the binary-string domain. Discrete Fourier transform is used to analyze underlying structure in high-cardinality alphabets space. Boolean operators are replaced with new operators such as ⊕, ⊖, ⊗ and so on in high cardinality alphabets. The "probe" formulation is redesigned to determine epistatic properties of non-binary function. Theoretical analysis shows the close relationship between probe value and problem structure. A deterministic and a stochastic algorithm based on properties of probes are proposed to completely identify the linkage structure and rigourously compute all Fourier coefficients. Some discussion about linkage detection for continuous problems is given.

In this paper, an uncalibrated dynamic visual servoing algorithm is proposed and analyzed. No calibration or robot model is needed. After a brief introduction of the development of uncalibrated visual servoing, the theoretical backgrounds and mathematical requirements of recursive least square (RLS) are stated respectively. Then the core uncalibrated visual servoing algorithm, in RLS form, or more technically, VS-RLS as well as its performance analysis, is investigated. After that, the experimental 6DOF Puma560 simulation of static and moving target tracking is demonstrated. Finally the weak and strength of the algorithm as well as the potential and promising improvements are discussed.

A new estimation of distribution algorithm (EDA) with spline kernel function (EDA_S) is proposed to optimize biped gait for a nine-link humanoid robot. Gait synthesis of the biped locomotion is firstly formulated as a multi-constraint optimization problem with consideration of two objectives, including zero-moment point (ZMP) for dynamically stable locomotion and driving torque for energy efficiency. The parameters to be optimized are joint coordinates at transition poses between three successive phases. Rather than searching in joint angle permutation space directly, the proposed method approximates the probability distribution by Catmull-Rom (CR) cubic spline function and updates them with gradient descent learning strategy. The effectiveness of EDA_S for biped gait optimization has been successfully tested on the simulated model of a humanoid soccer robot. It is shown that the flexible kernel with the updating rule is able to remarkably accelerate the learning speed with comparison to the traditional EDA

A new biped gait generation and optimization method is proposed in the frame of estimation of distribution algorithms (EDAs) with Q-learning method. By formulating the biped gait synthesis as a constrained multi-objective optimization problem, a dynamically stable and low energy cost biped gait is generated by EDAs with Q-learning (EDA_Q), which estimate probability distributions derived from the objective function to be optimized to generate searching points in the highly-coupled and high dimensional working space of biped robots. To get the preferable permutation of the interrelated parameters, Q-learning is combined to build and modify the probability models in EDA autonomously. By making use of the global optimization capability of EDA, the proposed EDA_Q can also solve the local minima problem in traditional Q-learning. On the other hand, with the learning agent, EDA_Q can evaluate the probability distribution model regularly without pre-designed structure and updating rule. The simulation results show that faster and more accurate searching can be achieved to generate preferable biped gait. The gait has been successfully used to drive a soccer-playing humanoid robot called Robo-Erectus which is one of the foremost leading soccer-playing humanoid robots in the RoboCup Humanoid League

Path-constrained optimal trajectory planning for serial-link robot arms has been extensively studied in the past decades. As an interesting topic in the robot motion planning and control, the kinematic singularity problem also attracted much attention in the literatures. In this paper, we present a systematic approach to plan an optimal trajectory passing though kinematic singularities with the end-effector precisely tracking the desired path. The method follows two steps. In the first step, the desired path is parameterized with a geometric variable q0. The robot motion can then be represented as a spatial curve in the joint space augmented by q0. The spatial curve, called augmented path, is obtained by numeric method as a function of another parameter s, which represents the arc-length of the augmented path. In the second step, the optimal trajectory with respect to time can be solved by optimally assigning s(t) along the augmented path. Dynamic programming is adopted in this paper to fulfill the optimal computation. Numerical simulations are carried out to verify the presented approach

Robot curve tracking is needed in some industrial applications, such as automatic welding or incising. Such a robot system is usually equipped with visual sensors, which always require calibration before used. The calibration process is often complicated. In this paper a neural network is used to learn the relationship between the world coordinate information and the image information, instead of computing accurate camera parameters. The neural network is first trained based on sample data by using the 2D and 3D coordinates of some control points on a standard pattern. During the tracking stage, images captured by cameras are firstly changed into binary images. The curve is then thinned and its position on the image is recorded. From the image data, the curve's position in the world coordinate frame can be specified by using the trained neural network. The curve tracked can be arbitrary, open or closed. The experimental results illuminate that the neural network technique is satisfying and it is successfully used in the vision-based robot curve tracking

This paper is concerned with the optimal controller design of networked control systems (NCSs). Aiming at a class of NCSs, in which network-induced delay and data packet dropout exist simultaneously from sensor to controller, this paper proposes a novel optimal controller based on a new adaptive genetic algorithm and simulated anneal algorithm (AGASA), guaranteed cost controller (GCC) and state feedback controller (SFC). The proposed controller is less conservative than GCC and SFC, and can be applied to both linear and nonlinear NCSs. Two examples are presented to illustrate the proposed controller.

A novel scheme is proposed in this paper which combines independent component analysis (ICA) and support vector machines (SVM) to classify MRS. ICA is used to extract features by decomposing MRS into components which correspond to biomedical metabolites. SVM is used to train a classifier based on features extracted by ICA. The new scheme can extract meaningful features and therefore obtain a classifier with good generalization. Experimental results show that the new method has better performance than others previous ones.

In this paper, we investigate the stability analysis and synthesis problems for both continuous-time and discrete-time fuzzy singularly perturbed systems. For continuous-time case, both the stability analysis and synthesis can be parameterized in terms of a set of linear matrix inequalities (LMIs). For discrete-time case, only the analysis problem can be cast in LMIs, while the derived stability conditions for controller design are nonlinear matrix inequalities (NMIs). Furthermore, a two-stage algorithm based on LMI and iterative LMI (ILMI) techniques is developed to solve the resulting NMIs and the stabilizing feedback controller gains can be obtained. For both continuous-time and discrete-time cases, the reduced-control law, which is only dependent on the slow variables, is also discussed. Finally, an illustrated example based on the flexible joint inverted pendulum model is given to illustrate the design procedures.

A stable gait generation algorithm based on T-S type fuzzy learning net is proposed in this paper. Gait generation is divided into model construction and error learning. Reference gait model and dynamic model are firstly constructed with basic gait geometric knowledge. Then reinforcement learning method is introduced into T-S type fuzzy network to learn the gain parameters for hip trajectory adjustment. Few fuzzy rules with ZMP stable knowledge are needed to formulate the nonlinear relation between the ZMP curve and hip trajectory. The problem of finding multi-variables in continuous space is also simplified to searching independent action gains simultaneously. Results of simulation on a biped robot proved the feasibility.

Robot soccer requires vision to be fast, robust and accurate to capture environment situation. In this paper a new method to improve accuracy and stability of vision system in robot soccer is presented. The main topics are camera calibration and robot localization with colored patterns. The lens distortion is successfully compensated and image coordinates is correctly projected to real field, thus achieving a more accurate estimation of object detection. Stability of the position and orientation of a robot is significantly improved with well-chosen colored patterns. Both simulation and experiment results show that our method performs well to provide accurate and stable vision in robot soccer.

This work presents a novel learning market based layered architecture for distributed multi-robot systems. For the market system, a reward function is defined and adapted by some kind of learning algorithm in the dynamical system. A layered architecture combining the market based system with the typical layered robot architecture is also proposed to make the robot team execute smoothly in the dynamic environment. Results illustrate the relationships between task time, task amount and robot amount. The task/robot rate is defined to deeply study their relationships.

A new type of wavelet based linearly parameterized network, dyadic network, is proposed in this paper, with application to an inverse dynamic based adaptive control. The function approximation capability of dyadic networks is supported by dyadic wavelet theory. Dyadic networks are easy to construct and the time cost is limited. Simulation results of a flight control system are presented to illustrate the performance of dyadic networks.

In this paper an efficient Q-learning paradigm implemented on a fuzzy CMAC network is proposed. The fuzzy CMAC network topological architecture is described. The continuous states of the system are partitioned into a number of fuzzy boxes. With the proposed fuzzy CMAC the Q-values of agents in the fired fuzzy boxes are evaluated and the control actions with maximum Q-values can be derived. The proposed hybrid adaptive and learning type of Fuzzy Neural control system based on the Q-learning is applied to the control of a pH-neutralization process.

This paper presents a third-order spline interpolation based trajectory planning method which is aiming to achieve smooth biped swing leg trajectory by reducing the instant velocity change which occurs at the time of collision of the biped swing leg with the ground. We first characterize the bipedal walking cycle and point out some major issues that need to be addressed to plan a continuous swing leg trajectory by using the concept of the spline interpolation and zero moment points (ZMP). We demonstrate that the impact effects can be avoided at the time of the swing foot's heel touching with the ground. The proposed biped trajectory planning method has been tested on our soccer-playing humanoid robot, Robo-Erectus, which got Second Place at the 2002 RoboCup Humanoid Walk competition.

In this paper, the time-delay fuzzy singularly perturbed model is proposed based on the Takagi-Sugeno form, the basic /spl epsiv/-dependent stability conditions are given. Then the H/sub 2/ control problem is investigated in the form of /spl epsiv/-independent linear matrix inequalities. Finally, we show that the obtained controller solve also the corresponding problem for the time-delay fuzzy descriptor system.

A class of fuzzy neural network with dynamic weights is proposed and its corresponding network topological architecture with suitable supervised learning algorithm is presented. Using the proposed network in control system design, a priori knowledge of the control system is not essential and this includes the order of the control system. The proposed network is applied to the control of a highly nonlinear pH-neutralization process. Simulation shows that the proposed dynamic learning control strategy has better dynamic quality, stronger robustness, adaptability and intelligence while comparing to the conventional control techniques, which demand the explicit and quantitative mathematical model of the system under control.

This paper presents a new learning method to attack two significant sub-problems in reinforcement learning at the same time: continuous space and linguistic rewards. Linguistic reward-oriented Takagi-Sugeno fuzzy reinforcement learning (LRTSFRL) is constructed by combining Q-learning with Takagi-Sugeno type fuzzy inference systems. The proposed paradigm is capable of solving complicated learning tasks of continuous domains, also can be used to design Takagi-Sugeno fuzzy logic controllers. Experiments on the double inverted pendulum system demonstrate the performance and applicability of the presented scheme. Finally, the conclusion remark is drawn.

A novel neural networks control structure for manipulators with flexible last link was proposed The manipulator with flexible last link was regarded as two parts: the flexible latter part (FLP) that includes the last joint and the last link, and the rigid former part (RFP), i.e. the rest. The kinematical and dynamic equations of the two parts were derived apart. The proposed control approach combines organically a conventional model-based controller with a neural networks controller. The neural networks approximate the dynamic anti-model of FLP and decompose the desired trajectory of end point into two parts, desired trajectory of axis of the last joint and desired outer corner of the last joint. Those would be realized by conventional controllers. By combining, the approach has not only solved the problem of nonlinearity and calculation inefficiency of the dynamic model of a flexible manipulator but also lessen greatly the size of the neural network and thus improve the convergence speed The proposed approach achieves fine control effect on the emulation with a three links planar manipulator with a flexible last link.

A neural network (NN)-based adaptive controller with an observer is proposed for the trajectory tracking of robotic manipulators with unknown dynamics nonlinearities. It is assumed that the robotic manipulator has only joint angle position measurements. A linear observer is used to estimate the robot joint angle velocity, while NNs are employed to further improve the control performance of the controlled system through approximating the modified robot dynamics function. The adaptive controller for robots with an observer can guarantee the uniform ultimate bounds of the tracking errors and the observer errors as well as the bounds of the NN weights. For performance comparisons, the conventional adaptive algorithm with an observer using linearity in parameters of the robot dynamics is also developed in the same control framework as the NN approach for online approximating unknown nonlinearities of the robot dynamics. Main theoretical results for designing such an observer-based adaptive controller with the NN approach using multilayer NNs with sigmoidal activation functions, as well as with the conventional adaptive approach using linearity in parameters of the robot dynamics are given. The performance comparisons between the NN approach and the conventional adaptation approach with an observer is carried out to show the advantages of the proposed control approaches through simulation studies.

A small power biped robot is designed by the load sharing method of walking gait. Walking can be classified into one-foot and two-foot standing during the common speed smooth walking. In two-foot standing, the weight of robot can be distributed to all joints. However, in one-foot standing, it is burdened to one foot, especially to ankle joint. So bigger actuator or even the passive joint is considered at the ankle joint; but bigger actuator increases the weight of robot and passive joint does not control ankle. When the small power actuator is used at all joints, ankle joint is sometimes overloaded during walking and this makes walking impossible. This problem is solved using the appropriate walking gait, that is, it depresses or distributes the driving power of the ankle joint when the load is applied to the ankle, and drives the ankle joint before the load is applied. By this method, the small power biped robot can be implemented without overloading the ankle joint. Experiments are conducted to prove the proposed method.

On the basis of the Takagi-Sugeno (TS) fuzzy model, this paper discusses in detail the following three problems: (1) output tracking of the nonlinear system; (2) output regulation of the nonlinear system via a state feedback; (3) output regulation of the nonlinear system via a error feedback. Numerical simulations are given to illustrate the soundness of these results and the effectiveness of the new methodology solving the output tracking and regulation problem of the nonlinear system.

An indirect adaptive control approach is developed in this paper for unknown robots using neural networks (NN). A key property of the proposed approach is that the actual joint angle values in the control law is replaced by the desired joint angle, angular speed and acceleration, and the bound on the NN reconstruction error is assumed to be unknown. Main theoretical results for designing such a neuro-adaptive controller are given, and the control performance of the proposed controller is verified with simulation studies.

This paper addresses the analysis and design of a fuzzy controller and a fuzzy observer on the basis of the Takagi-Sugeno (T-S) fuzzy model. The main contribution of the paper is the development of the separation property; that is, the fuzzy controller and the fuzzy observer can be independently designed. A numerical simulation and an experiment on an inverted pendulum system are described to illustrate the performance of the fuzzy controller and the fuzzy observer.

For a class of multiinput-multioutput (MIMO) sampled-data nonlinear systems with unknown dynamic nonlinearities, a stable neural-network (NN)-based adaptive control approach which is an integration of an NN approach and the adaptive implementation of the variable structure control with a sector, is developed. The sampled-data nonlinear system is assumed to be controllable and its state vector is available for measurement. The variable structure control with a sector serves two purposes. One is to force the system state to be within the state region in which the NN's are used when the system goes out of neural control; and the other is to provide an additional control until the system tracking error metric is controlled inside the sector within the network approximation region. The proof of a complete stability and a tracking error convergence is given and the setting of the sector and the NN parameters is discussed. It is demonstrated that the asymptotic error of the system can be made dependent only on inherent network approximation errors and the frequency range of unmodeled dynamics. Simulation studies of a two-link manipulator show the effectiveness of the proposed control approach.

Existing stable adaptive control approaches using neural networks have been developed mostly in continuous time systems for robot trajectory tracking. This paper investigates the discrete time case. A novel scheme for integrating a neural network (NN) approach with an adaptive implementation of the sliding mode control with the sector is developed. The sliding mode control with the sector serves two purposes, one is to provide the global stability of the closed loop system, the other is to improve the tracking performance. The whole system stability and tracking error convergence are proved by Lyapunov techniques which yield a NN weight tuning algorithm.

An automatic programming method for robot system simulation is introduced. The method integrates 'reuse' technology and artificial intelligence to generate the robot simulation program. The overall simulation method for a general system is first proposed. Then a language translator, EXPC, and an expert system, EXPV, are briefly reported. EXPC translates C-based language to C language. EXPV calculates manipulator variables from sensing results. The overall structure of the automatic programming system is presented to show how the system works. Control definition, as an example of a C-based language, is presented.>