Trajectory prediction facilitates effective planning and decision-making, while constrained trajectory prediction integrates regulation into prediction. Recent advances in constrained trajectory prediction focus on structured constraints by constructing optimization objectives. However, handling unstructured constraints is challenging due to the lack of differentiable formal definitions. To address this, we propose a novel method for constrained trajectory prediction using a conditional generative paradigm, named Controllable Trajectory Diffusion (CTD). The key idea is that any trajectory corresponds to a degree of conformity to a constraint. By quantifying this degree and treating it as a condition, a model can implicitly learn to predict trajectories under unstructured constraints. CTD employs a pre-trained scoring model to predict the degree of conformity (i.e., a score), and uses this score as a condition for a conditional diffusion model to generate trajectories. Experimental results demonstrate that CTD achieves high accuracy on the ETH/UCY and SDD benchmarks. Qualitative analysis confirms that CTD ensures adherence to unstructured constraints and can predict trajectories that satisfy combinatorial constraints.

Due to the complexity of football matches, high-quality player policies must exhibit diverse behaviors for effective collaboration. However, in football tasks where online interactions are time-consuming and costly, it is difficult to balance training efficiency and the emergence of diversity when learning policies from scratch. Therefore, this paper proposes a novel diversity-driven offline-to-online (DOTO) multi-player policy learning method for football matches. Specifically, we design a transformer-actor-critic module that is universally applicable to both offline and online stages, enabling seamless adaptation. This module utilizes expert data for offline pre-training, which serves as the initialization for online fine-tuning. Subsequently, we introduce an across offline and online adaptive intention clustering guidance module, which learns intention representations of agents and performs intention-based clustering to construct adaptive loss functions to guide policy diversity. Besides, to mitigate the bootstrapping errors of distribution shift from offline to online, we implement an online partial random initialization mechanism, balancing the inherent conservativeness of offline learning with the flexible exploration of online learning. Extensive experiments in Google Research Football environment show that DOTO not only increases win rate to 50% which surpasses all state-of-the-art methods in the hardest 11 vs. 11 task, but also generates richer policy diversity.

State-of-the-art (SOTA) multiagent reinforcement algorithms distinguish themselves in many ways from their single-agent equivalences. However, most of them still totally inherit the single-agent exploration-exploitation strategy. Naively inheriting this strategy from single-agent algorithms causes potential collaboration failures, in which the agents blindly follow mainstream behaviors and reject taking minority responsibility. We name this problem the responsibility diffusion (RD) as it shares similarities with the same-name social psychology effect. In this work, we start by theoretically analyzing the cause of this RD problem, which can be traced back to the exploration-exploitation dilemma of multiagent systems (especially large-scale multiagent systems). We address this RD problem by proposing a policy resonance (PR) approach which modifies the collaborative exploration strategy of agents by refactoring the joint agent policy while keeping individual policies approximately invariant. Next, we show that SOTA algorithms can equip this approach to promote the collaborative performance of agents in complex cooperative tasks. Experiments are performed in multiple test benchmark tasks to illustrate the effectiveness of this approach.

Inspired by psychological insights into individual behavior, we propose a novel cognition-oriented multiagent reinforcement learning (CORL) framework. CORL equips agents with two distinct types of cognition-situational and self-cognition-derived from local observations. To enhance the informativeness and precision of these cognition types, we introduce two information-theoretical regularizers: one to align situational cognition with the global state and the other to align self-cognition with each agent's identity for improved role differentiation and team coordination. In addition, the centralized training and decentralized execution framework is adopted to train the policy network. Our simulations demonstrate that CORL effectively harnesses local observations for enriched cooperation, leading to pronounced performance improvements, particularly in challenging tasks.

Knowledge transfer from teacher agent to student agent in reinforcement learning addresses the issue of sample inefficiency resulting from random exploration. However, existing approaches usually assume that the policy or state-action pairs (demonstrations) of teacher is accessible, which may not be met in reality due to privacy and other limitations. This paper proposes a novel knowledge transfer method based solely on the state sequences of teacher. Specifically, We encode raw state into general deep feature through learning state evaluation and domain discrimination tasks based on teachers’ experience, and share the encoder among different student agents in various scenarios. We validate our method in Starcraft II and Google Research Football environment. The experimental results show that our method significantly accelerates the learning process of agents and improves their convergence performance, showcasing the "depth" of the feature. Additionally, the feature successfully generalize to teachers’ unseen scenarios and unfamiliar roles, demonstrating the "generality" of the feature.

Learning effective policies is challenging for a multi-agent system in partially observable environments, where agents need to extract relevant features from local observations. Most approaches in multi-agent reinforcement learning (MARL) are limited to feature extraction for homogenous agents. They struggle to deal with local observations in heterogeneous multi-agent scenarios, where agents have different observation spaces and are necessitated to process semantically varied information. To address this issue, we analyze the observational heterogeneity of multi-agent systems, and propose a heterogeneous-graph-based approach for feature extraction in MARL. We model agent observations as heterogeneous graphs, and design a heterogeneous observation aggregation network (HOA-Net) for processing these graph-based observations. HOA-Net is specifically designed to address various forms of observational heterogeneity. It employs class-specific weighting networks and computes across-class attentions for observed entities, effectively reducing the number of learnable parameters. The proposed method is evaluated on SMAC and an Unreal-Engine-based heterogeneous multi-agent testbed. Experimental results demonstrate that our method significantly outperforms other baselines in effectively aggregating an agent’s observation, and finally enhancing the performance of heterogeneous multi-agent systems.

A large proportion of recent studies on cooperative Multi-Agent Reinforcement Learning (MARL) focus on the policylearning process in scenarios with stationary opponents (or without opponents). This paper, instead, investigates a different challenge of achieving team superiority in dynamic competitions among competitors that evolve dynamically with MARL. We aim to enhance the competitiveness of such MARL learners by enabling them to adjust their own learning settings dynamically, so as to take quick counter-measures against the policy shift of competitor learners, or to learn faster to suppress the opponents. We propose a Competitive Auto-Multiagent Learner with Fuzzy Feedback (CALF) with two essential highlights: (1) CALF establishes feedback controllers to achieve real-time adjustments based on fuzzy logic, using human-readable fuzzy rules to provide significant explainability and flexibility; (2) CALF integrates Bayesian Optimization to search and optimize the feedback fuzzy logic rules automatically. CALF can be used to apply real-time adjustments for MARL hyperparameters and intrinsic rewards. We also give solid empirical results to show that CALF significantly promotes team competitiveness in adversarial competitions, spanning from small-scale tasks involving 2 teams to large-scale tasks involving 3 teams and hundreds of agents. Furthermore, CALF exhibits superior competitiveness when engaging in competition with established competitors like Qmix, Qtran, and Qplex in dynamic competitive environments. Moreover, the experiments also demonstrate that the integration of the fuzzy logic with Bayesian Optimization offers considerable transferability and explainability, enabling a CALF-implemented learner optimized from one scenario to be transferred to other distinct scenarios.

Recently, multiagent reinforcement learning (MARL) has shown great potential for learning cooperative policies in multiagent systems (MASs). However, a noticeable drawback of current MARL is the low sample efficiency, which causes a huge amount of interactions with environment. Such amount of interactions greatly hinders the real-world application of MARL. Fortunately, effectively incorporating experience knowledge can assist MARL to quickly find effective solutions, which can significantly alleviate the drawback. In this article, a novel multiexperience-assisted reinforcement learning (MEARL) method is proposed to improve the learning efficiency of MASs. Specifically, monotonicity-constrained reward shaping is innovatively designed using expert experience to provide additional individual rewards to guide multiagent learning efficiently, with the invariance guarantee of the team optimization objective. Furthermore, a reward distribution estimator is specially developed to model an implicated reward distribution of environment by using transition experience from environment, containing collected samples (state-action pair, reward, and next state). This estimator can predict the expectation reward of each agent for the taken action to accurately estimate the state value function and accelerate its convergence. Besides, the performance of MEARL is evaluated on two multiagent environment platforms: our designed unmanned aerial vehicle combat (UAV-C) and StarCraft II Micromanagement (SCII-M). Simulation results demonstrate that the proposed MEARL can greatly improve the learning efficiency and performance of MASs and is superior to the state-of-the-art methods in multiagent tasks.

Sparse reward remains a valuable and challenging problem in multi-agent reinforcement learning (MARL). This paper addresses this issue from a new perspective, i.e., lazy agents. We empirically illustrate how lazy agents damage learning from both exploration and exploitation. Then, we propose a novel MARL framework called Lazy Agents Avoidance through Influencing External States (LAIES). Firstly, we examine the causes and types of lazy agents in MARL using a causal graph of the interaction between agents and their environment. Then, we mathematically define the concept of fully lazy agents and teams by calculating the causal effect of their actions on external states using the do-calculus process. Based on definitions, we provide two intrinsic rewards to motivate agents, i.e., individual diligence intrinsic motivation (IDI) and collaborative diligence intrinsic motivation (CDI). IDI and CDI employ counterfactual reasoning based on the external states transition model (ESTM) we developed. Empirical results demonstrate that our proposed method achieves state-of-the-art performance on various tasks, including the sparse-reward version of StarCraft multi-agent challenge (SMAC) and Google Research Football (GRF). Our code is open-source and available at https://github.com/liuboyin/LAIES.

Automated analysis in football matches is meaningful for player and team evaluation. However, most related works ignore match style and team strength. In this paper, a novel stylized automated analysis framework termed All for Goals (AFG) is proposed for football matches, which considers match style and team strength to better quantify the relationship of all match states and player actions respectively with potential goals. AFG is composed of an automatic labeling module, a potential goal prediction module, and a state and player evaluation module. Specifically, in the automatic labeling module, relevant samples are given the same label to avoid manual labeling. In the potential goal prediction module, we introduce the Pretrain-Finetune paradigm. Based on labeled data, an average model learning to identify scoring difficulty is obtained in the first pre-training procedure, and tuned models learning specific styles are obtained in the second fine-tuning procedure. In the state and player evaluation module, the evaluation mechanisms of state, on-ball action, and off-ball running based on potential goal prediction result are designed for match review and tactics mining. Finally, we validate the rationality and validity of AFG on multiple tasks. On the goal prediction task, the models show high recall rates and remarkable difference in style. On real-time situation analysis, credit assignment for football events, and off-ball running analysis tasks, the evaluation mechanisms give the results consistent with football domain knowledge.

Multi-agent Reinforcement Learning (MARL) has become a powerful tool for addressing multi-agent challenges. Existing studies have explored numerous models to use MARL to solve single-team cooperation (competition) problems and adversarial problems with opponents controlled by static knowledge-based policies. However, most studies in the literature often ignore adversarial multi-team problems involving dynamically evolving opponents. We investigate adversarial multi-team problems where all participating teams use MARL learners to learn policies against each other. Two objectives are achieved in this study. Firstly, we design an adversarial team-versus-team learning framework to generate cooperative multi-agent policies to compete against opponents without preprogrammed opponent partners or any supervision. Secondly, we explore the key factors to achieve win-rate superiority during dynamic competitions. Then we put forward a novel FeedBack MARL (FBMARL) algorithm that takes advantage of feedback loops to adjust optimizer hyper-parameters based on real-time game statistics. Finally, the effectiveness of our FBMARL model is tested in a benchmark environment named Multi-Team Decentralized Collective Assault (MT-DCA). The results demonstrate that our feedback MARL model can achieve superior performance over baseline competitor MARL learners in 2-team and 3-team dynamic competitions.

Football player's decision-making problem is quite challenging because of the essential feature of football matches: many players with different and complex cooperative or competitive relationships. To better leverage these relationships, this paper proposes a player-policy learning method with heterogeneous-graph attention reinforcement learning (PPL-HGARL) to enable an active player closest to the ball to learn effective policies for playing football matches. Specifically, a multi-head feature representation module is designed to reconstruct raw observations using prior expert knowledge. Furthermore, a heterogeneous-graph player-relation attention network is designed by use of graph among different roles, in order to model cooperative and competitive relations among players. The network learns proper state representation for the active player, making the player pay attention to other important players. Besides, an actor-critic algorithm is adopted to train the policies efficiently. Competing with rule-based opponents of different difficulty levels in the Google Research Football environment, the active player achieves excellent results, which validates the effectiveness and superiority of the proposed method.

Deriving efficient cooperative policies in uncertain dynamic environments poses huge challenges for a distributed swarm system due to the limited capability of the agents and the complex dynamics of the environment. In this paper, a novel distributed method based on deep reinforcement learning using observation-level and communication-level graph networks is proposed to learn cooperative policies for the distributed swarm system. Specifically, a relational directed graph attention neural network is designed to model observation-level graphs composed of heterogeneous relational graphs among each agent and each type of entities (e.g., obstacles, other teammates, opponents), for extracting different relational representations. Moreover, a relevant directed graph attention network is presented to cut off the ineffective communication among irrelevant agents, and model a relevant communication topology between each agent and relevant homogeneous neighbor agents as an communication-level graph, for promoting efficient inter-agent interactions. Furthermore, a distributed actor-critic algorithm with full parameter sharing is implemented to learn cooperative swarm policies by using distributed critics, which avoids the curse of dimensionality under a centralized critic. Various simulation results validate the effectiveness and generalization of the proposed method, and demonstrate that the proposed method outperforms existing state-of-the-art methods on coverage and pursuit tasks.

Due to the high complexity of a football match, the opponents' strategies are variable and unknown. Thus predicting the opponents' future intentions accurately based on current situation is crucial for football players' decision-making. To better anticipate the opponents and learn more effective strategies, a deconfounded opponent intention inference (DOII) method for football multi-player policy learning is proposed in this paper. Specifically, opponents' intentions are inferred by an opponent intention supervising module. Furthermore, for some confounders which affect the causal relationship among the players and the opponents, a decon-founded trajectory graph module is designed to mitigate the influence of these confounders and increase the accuracy of the inferences about opponents' intentions. Besides, an opponent-based incentive module is designed to improve the players' sensitivity to the opponents' intentions and further to train reasonable players' strategies. Representative results indicate that DOII can effectively improve the performance of players' strategies in the Google Research Football environment, which validates the superiority of the proposed method.

Applying deep reinforcement learning to football games has recently received extensive attention. However, this remains challenging due to the excessively high complexity of the football environment, such as high-dynamical game states, sparse rewards, and multiple roles with different capabilities. Existing works aim to address these problems without considering abundant domain knowledge of football. In this article, a football knowledge-embedded learning framework is proposed. Specifically, the pitch control concept is innovatively introduced to design a knowledge-embedded state representation. As a result, a novel pitch control model is designed that quantitatively provides space influence values of a single player, the whole team, and the ball. Different from existing models, this model additionally considers each player's various capabilities, including flexibility, explosive force, and stamina. Furthermore, the deformable convolution network is adopted for state representation extracting, which is used to process the geometric transformation of the players' positions and spatial influence values generated by the pitch control model. Then, based on this comprehensive state representation, a proximal policy optimization-based reinforcement learning scheme is adopted to generate the final policy. Finally, extensive simulations, including learning against a fixed opponent and learning from self-play, clearly show the effectiveness and adaptability of our proposed framework.

Many attempts have been made to promote cooperation for multiagent systems. However, several issues that draw less attentions but may dramatically degrade the cooperation performance still exist, such as redundant information interactions among neighbors, and difficulties in understanding complex and dynamic environments from high-level cognition. To address these limitations, a cognition-driven multiagent policy (CDMAP) learning framework is proposed in this article. It includes a cognition difference network (CDN), a coupling cognition network (CCN), and a policy optimization network (PON). CDN is designed based on a variational autoencoder, where a concept of cognition difference is defined to prune redundant interactions among agents for more efficient communication. Based on the pruned topology, CCN captures the hidden representations of the surrounding environment. Several coupling graph attention layers are incorporated in CCN, each layer with different but coupling adjacent matrices, yielding a comprehensive state understanding from multiple representation spaces. Based on the captured hidden states, PON generates the final policies, where QMIX is adopted as a value factorization method to alleviate the credit-assignment problem. At last, CDMAP is evaluated through two representative multiagent games including Google Research Football andStarCraft II. The results demonstrate its superior effectiveness compared with existing methods.

Social learning, especially social incentives, is extremely important for humans to achieve a high level of coordination. Inspired by this, we introduce this concept into cooperative multiagent reinforcement learning (MARL), to implicitly address the credit assignment problem and promote the interagent direct interactions for cooperations among agents in cooperative multiagent games. In this article, we propose a novel intrinsic reward method with peer incentives (IRPI) based on actor–critic policy gradient. This method can enable agents to incentivize each other for their cooperations through using causal influence among them. Specifically, a novel intrinsic reward mechanism is innovatively designed to empower each agent the ability to give positive or negative rewards to other peer agents' actions through considering the causal influence of the other agents on it. The mechanism is realized by a feedforward neural network through utilizing causal influence between the agents. The causal influence of one agent on another is inferred via counterfactual reasoning using the joint action-value function in MARL. The quality of the influence is assessed via counterfactual reasoning using the individual value function in MARL. Simulations are carried out on two popular multiagent game testbeds: Starcraft II Micromanagement and Multiagent Particle Environments. Simulation results demonstrate that the proposed IRPI can enhance cooperations among the agents to achieve better performance compared with a number of state-of-the-art MARL methods in a variety of cooperative multiagent games.

Deriving a distributed, time-efficient, and connectivity-guaranteed coverage policy in multitarget environment poses huge challenges for a multirobot team with limited coverage and limited communication. In particular, the robot team needs to cover multiple targets while preserving connectivity. In this article, a novel deep-reinforcement-learning-based approach is proposed to take both multitarget coverage and connectivity preservation into account simultaneously, which consists of four parts: a hierarchical observation attention representation, an interaction attention representation, a two-stage policy learning, and a connectivity-guaranteed policy filtering. The hierarchical observation attention representation is designed for each robot to extract the latent features of the relations from its neighboring robots and the targets. To promote the cooperation behavior among the robots, the interaction attention representation is designed for each robot to aggregate information from its neighboring robots. Moreover, to speed up the training process and improve the performance of the learned policy, the two-stage policy learning is presented using two reward functions based on algebraic connectivity and coverage rate. Furthermore, the learned policy is filtered to strictly guarantee the connectivity based on a model of connectivity maintenance. Finally, the effectiveness of the proposed method is validated by numerous simulations. Besides, our method is further deployed to an experimental platform based on quadrotor unmanned aerial vehicles and omnidirectional vehicles. The experiments illustrate the practicability of the proposed method.

In this article, a novel method, called attention enhanced reinforcement learning (AERL), is proposed to address issues including complex interaction, limited communication range, and time-varying communication topology for multi agent cooperation. AERL includes a communication enhanced network (CEN), a graph spatiotemporal long short-term memory network (GST-LSTM), and parameters sharing multi-pseudo critic proximal policy optimization (PS-MPC-PPO). Specifically, CEN based on graph attention mechanism is designed to enlarge the agents' communication range and to deal with complex interaction among the agents. GST-LSTM, which replaces the standard fully connected (FC) operator in LSTM with graph attention operator, is designed to capture the temporal dependence while maintaining the spatial structure learned by CEN. PS-MPC-PPO, which extends proximal policy optimization (PPO) in multi agent systems with parameters' sharing to scale to environments with a large number of agents in training, is designed with multi-pseudo critics to mitigate the bias problem in training and accelerate the convergence process. Simulation results for three groups of representative scenarios including formation control, group containment, and predator-prey games demonstrate the effectiveness and robustness of AERL.

Generating collision-free formation control strategy for multiagent systems faces huge challenges in collaborative navigation tasks, especially in a highly dynamic and uncertain environment. Two typical methodologies for solving this problem are the conventional model-based paradigm and the data-driven paradigm, particularly the widely used deep reinforcement learning (DRL) method. However, both the model-based and data-driven paradigms encounter inherent drawbacks. In this paper, we present two novel general schemes that combine these two paradigms together in an online mode. Specifically, the two paradigms are combined in a parallel and a serial structure in these two schemes, respectively. In the parallel scheme, the outputs of the model-based and DRL-based controllers are lumped together. In the serial scheme, the output of the model-based controller is fed as an input of the DRL-based controller. The interpretation of the two combined schemes is suggested from a control-oriented perspective, where the parallel DRL controller is viewed as a complementary uncertainty compensator and the serial DRL controller is taken as an inverse dynamics estimator. Finally, comprehensive simulations are conducted to demonstrate the superiority of the proposed schemes, and the effectiveness is further verified by deploying our schemes to a physical experiment platform based on a set of three-wheeled omnidirectional robots.

When dealing with a series of imminent issues, humans can naturally concentrate on a subset of these concerning issues by prioritizing them according to their contributions to motivational indices, e.g., the probability of winning a game. This idea of concentration offers insights into reinforcement learning of sophisticated Large-scale Multi-Agent Systems (LMAS) participated by hundreds of agents. In such an LMAS, each agent receives a long series of entity observations at each step, which can overwhelm existing aggregation networks such as graph attention networks and cause inefficiency. In this paper, we propose a concentration network called ConcNet. First, ConcNet scores the observed entities considering several motivational indices, e.g., expected survival time and state value of the agents, and then ranks, prunes, and aggregates the encodings of observed entities to extract features. Second, distinct from the well-known attention mechanism, ConcNet has a unique motivational subnetwork to explicitly consider the motivational indices when scoring the observed entities. Furthermore, we present a concentration policy gradient architecture that can learn effective policies in LMAS from scratch. Extensive experiments demonstrate that the presented architecture has excellent scalability and flexibility, and significantly outperforms existing methods on LMAS benchmarks.

Air-to-air close-in combat is based on many basic fighter maneuvers and can be largely modeled as an algorithmic function of inputs. This paper studies autonomous close-in combat, to learn new strategy that can adapt to different circumstances to fight against an opponent. Current methods for learning close-in combat strategy are largely limited to discrete action sets whether in the form of rules, actions or sub-polices. In contrast, we consider one-on-one air combat game with continuous action space and present a deep reinforcement learning method based on proximal policy optimization (PPO) that learns close-in combat strategy from observations in an end-to-end manner. The state space is designed to promote the learning efficiency of PPO. We also design a minimax strategy for the game. Simulation results show that the learned PPO agent is able to defeat the minimax opponent with about 97% win rate.

In this paper, we propose a novel decentralized method based on deep reinforcement learning using robot-level and target-level relational graphs, to solve the problem of multi-target encirclement with collision avoidance (MECA). Specifically, the robot-level relational graphs, composed of three heterogeneous relational graphs between each robot and other robots, targets and obstacles, are modeled and learned through using graph attention networks (GATs) for extracting different spatial relational representations. Moreover, for each target within the observation of each robot, a target-level relational graph is built with GAT to construct spatial relations from the robot. Furthermore, the movement of each target is modeled by the target-level relational graph and learned through supervised learning for predicting the trajectory of the target. In addition, a knowledge-embedded compound reward function is defined to solve the multi-objective problem in MECA, and guide the policy learning for deriving the behavior of MECA. An actor-critic training algorithm based on the centralized training and decentralized execution framework is adopted to train the policy network. Simulation and real-world experiment results demonstrate the effectiveness and generalization of our method.

Multiagent reinforcement learning (MARL) can solve complex cooperative tasks. However, the efficiency of existing MARL methods relies heavily on well-defined reward functions. Multiagent tasks with sparse reward feedback are especially challenging not only because of the credit distribution problem, but also due to the low probability of obtaining positive reward feedback. In this paper, we design a graph network called Cooperation Graph (CG). The Cooperation Graph is the combination of two simple bipartite graphs, namely, the Agent Clustering subgraph (ACG) and the Cluster Designating subgraph (CDG). Next, based on this novel graph structure, we propose a Cooperation Graph Multiagent Reinforcement Learning (CG-MARL) algorithm, which can efficiently deal with the sparse reward problem in multiagent tasks. In CG-MARL, agents are directly controlled by the Cooperation Graph. And a policy neural network is trained to manipulate this Cooperation Graph, guiding agents to achieve cooperation in an implicit way. This hierarchical feature of CG-MARL provides space for customized cluster-actions, an extensible interface for introducing fundamental cooperation knowledge. In experiments, CG-MARL shows state-of-the-art performance in sparse reward multiagent benchmarks, including the anti-invasion interception task and the multi-cargo delivery task.

Learning an effective strategy is challenging for agents in partially observable environment, where the agents can only observe a part of environment information and make decisions based on local information. Hence, how to effectively utilize local information to achieve efficient cooperation among the agents is particularly important. The agents can establish an understanding of themselves and their surrounding environment based on their historical observation information. However, the understanding lacking global information is local or limited, resulting in low performance in some complex tasks. To solve the problem, a situational cognition learning framework is proposed for each agent based on local information, with which each agent can reconstruct a cognition about itself and its surrounding environment and map it into a high dimensional representation space. In particular, situational cognition is modeled as a random variable under the condition of local trajectory. In addition, an information regularizer is introduced to ensure that the situational cognition is complete and accurate through maximizing the mutual information between the situational cognition and global information, conditioned on the local trajectory of the agent. Various simulations are conducted and show that the proposed framework significantly promotes cooperation among the agents and improves performance.

In this paper, we propose a novel Intrinsic Reward method with Peer Incentives (IRPI) to promote the inter-agent direct interactions and implicitly address the credit assignment problem in cooperative multi-agent reinforcement learning (MARL). The IRPI method can build mutual incentives between agents by using their causal effect, to realize their advanced cooperation. Specifically, a new intrinsic reward mechanism is conducted, which equips each agent with the ability to reward other agent by using the causal effect between them. Moreover, the mechanism is built through a neural network and learned by using causal effect between the agents. Furthermore, the counterfactual reasoning is used to infer the causal effect between the agents using the joint action-state value function, and then assess the quality of the effect using individual state value function in MARL. Simulational results in Starcraft II Micromanagement demonstrate that the proposed IRPI can enhance cooperation among the RL agents to achieve better performance than some state-of-the-art MARL methods in various cooperative multi-aaent tasks.

In this paper, we propose a novel distributed method based on attention-based deep reinforcement learning using individual reward shaping, for multiple unmanned aerial vehicles (UAVs) cooperative short-range combat mission. Specifically, a two-level attention distributed policy, composed of observation-level and communication-level attention networks, is designed to enable each UAV to selectively focus on important environmental features and messages, for enhancing the effectiveness of the cooperative policy. Moreover, due to the high complexity and stochasticity of the UAV combat mission, the learning of UAVs is tricky and low efficient. To embed knowledge to accelerate the policy learning, a potential-based individual reward function is constructed by implicitly translating the individual reward into the specific form of dynamic action potentials. In addition, an actor-critic training algorithm based on the centralized training and decentralized execution framework is adopted to train the policy network of UAV maneuver decision. We build a three-dimensional UAV simulation and training platform based on Unity for multi-UAV short-range combat missions. Simulation results demonstrate the effectiveness of the proposed method and the superiority of the attention policy and individual reward shaping.

This article presents a new command-filtered composite adaptive neural control scheme for uncertain nonlinear systems. Compared with existing works, this approach focuses on achieving finite-time convergent composite adaptive control for the higher-order nonlinear system with unknown nonlinearities, parameter uncertainties, and external disturbances. First, radial basis function neural networks (NNs) are utilized to approximate the unknown functions of the considered uncertain nonlinear system. By constructing the prediction errors from the serial-parallel nonsmooth estimation models, the prediction errors and the tracking errors are fused to update the weights of the NNs. Afterward, the composite adaptive neural backstepping control scheme is proposed via nonsmooth command filter and adaptive disturbance estimation techniques. The proposed control scheme ensures that high-precision tracking performances and NN approximation performances can be achieved simultaneously. Meanwhile, it can avoid the singularity problem in the finite-time backstepping framework. Moreover, it is proved that all signals in the closed-loop control system can be convergent in finite time. Finally, simulation results are given to illustrate the effectiveness of the proposed control scheme.

This article investigates the adaptive fuzzy tracking control design for uncertain nonstrict-feedback nonlinear systems with time-varying constraints and asymmetric input saturations. It is known that time-varying output constraint, time-varying error constraints, and input saturations are commonly seen issues in many practical engineering systems due to inherent physical limitations and performance requirements. To deal with these problems and improve the convergence rate of the control system, a novel fixed-time convergent adaptive fuzzy control scheme is proposed for the considered uncertain nonlinear systems via the time-varying barrier Lyapunov function (BLF) technique. First, to deal with the unknown nonlinearities and uncertainties in the system dynamic model, fuzzy logic systems are utilized to approximate the unknown functions of the considered model. Then, the problem of asymmetric input amplitude and rate saturations is handled by constructing a unified smooth characterizing function and an innovative auxiliary design signal system. Subsequently, considering physical limitations and performance requirements, a novel time-varying BLF-based adaptive fuzzy backstepping control scheme is designed for the uncertain nonstrict-feedback nonlinear systems to realize superior tracking performances and keep the states staying in predefined time-varying compact regions during operations. Further, rigorous theoretical analyses have been conducted to show that the proposed control scheme achieves fixed-time convergence of all signals in the closed-loop control system. Finally, representative simulation results verify the effectiveness of the proposed control scheme.

This paper considers a multi-target coverage problem where a robot team aims to efficiently cover multi-targets while maintaining connectivity in a distributed manner. A novel knowledge-incorporated policy framework is proposed to derive a distributed, efficient, and connectivity guaranteed coverage policy. In particular, a knowledge-guided policy network (KGPnet) is designed, which consists of observation attention representation, interaction attention representation, and knowledge-guided policy learning. Giving credit to the KGPnet, the connectivity guaranteed coverage policy can be applied to different number targets. Moreover, based on the knowledge of the algebraic connectivity and coverage rate, a comprehensive reward is designed to guide the training of the behavior of multi-target coverage with connectivity maintenance. Furthermore, since the policy learned through deep reinforcement learning (DRL) can not guarantee the connectivity of the robot team, a knowledge-nested policy filtering is designed to filter dis-connectivity policies to satisfy the connectivity constraint based on the knowledge model of connectivity maintenance. Various simulations are conducted to verify the effectiveness of the proposed method. Besides, numerous real-world experiments with three-wheel omnidirectional cars and a motion capture system are presented to demonstrate the practicability of the proposed method.

Multi-agent cooperation is one of the most attractive research fields in multi-agent systems. There are many attempts made by researchers in this field to promote the cooperation behavior. However, in partially-observable environments, a large number of agents and complex interactions among the agents cause huge difficulty for policy learning. Moreover, redundant communication contents caused by many agents make effective features hard to be extracted, which prevents the policy from converging. To address the limitations above, a novel method called multi-agent cognition difference reinforcement learning (MACD-RL) is proposed in this paper. The key feature of MACD-RL lies in cognition difference network (CDN) and a soft communication network (SCN). CDN is designed to allow each agent to choose its neighbors (communication targets) adaptively with its environment cognition difference. SCN is designed to handle the complex interactions among the agents with soft attention mechanism. The results of simulations including mixed cooperative and competitive tasks demonstrate that the effectiveness and robustness of the proposed model.

This paper proposes a collaborative policy framework via relational graph reasoning for multi-agent systems to accomplish adversarial tasks. A relational graph reasoning module consisting of an agent graph reasoning module and an opponent graph module, is designed to enable each agent to learn mixture state representation to enhance the effectiveness of the policy. In particular, for each agent, the agent graph reasoning module is designed to infer different underlying influences from different opponents and generate agent-level state representation. The opponent graph reasoning module is creatively designed for the opponents to reason relations from their surrounding objects including the agents and the opponents based on their latent features and then predict the future state of the opponents. It forms an opponent-level state representation. Besides, in order to effectively predict the state of the opponents, an intrinsic reward based on prediction error is designed to motivate the policy learning. Furthermore, interactions among agents are utilized to transmit messages and fuse information to promote the cooperative behaviors among the agents. Finally, various representative simulations on two multi-agent adversarial tasks are conducted to demonstrate the superiority and effectiveness of the proposed framework by comparison with existing methods.

This paper presents a novel robust adaptive tracking control method for a hypersonic vehicle in a cruise flight stage based on interval type-2 fuzzy-logic system (IT2-FLS) and small-gain approach. After the input-output linearization, the vehicle model can be decomposed into two uncertain subsystems by considering matching disturbances and parametric uncertainties. For each subsystem, an interval type-2 Takagi-Sugeno-Kang fuzzy logic system (IT2-TSK-FLS) is then employed to approximate the unavailable model information. Following the idea of a small-gain approach, a composite feedback form for each subsystem is constructed, based on which the final robust adaptive tracking control law is developed. Rigorous stability analysis shows that all signals in the derived closed-loop system are kept uniformly ultimately bounded (UUB). The main contribution of this paper is that the proposed control law for the hypersonic vehicle is with only two adaptive parameters in total which can greatly alleviate the computation and storage burden in practice; meanwhile its superiority over the conventional minimal-learning-parameter (MLP)-based one is specifically illustrated. Comparative numerical simulations of three cases demonstrate the effectiveness of our proposed control method with respect to complicated uncertainties.

Generating collision-free, time-efficient paths in an uncertain dynamic environment poses huge challenges for the formation control with collision avoidance (FCCA) problem in a leader-follower structure. In particular, the followers have to take both formation maintenance and collision avoidance into account simultaneously. Unfortunately, most of the existing works are simple combinations of methods dealing with the two problems separately. In this article, a new method based on deep reinforcement learning (RL) is proposed to solve the problem of FCCA. Especially, the learning-based policy is extended to the field of formation control, which involves a two-stage training framework: an imitation learning (IL) and later an RL. In the IL stage, a model-guided method consisting of a consensus theory-based formation controller and an optimal reciprocal collision avoidance strategy is designed to speed up training and increase efficiency. In the RL stage, a compound reward function is presented to guide the training. In addition, we design a formation-oriented network structure to perceive the environment. Long short-term memory is adopted to enable the network structure to perceive the information of obstacles of an uncertain number, and a transfer training approach is adopted to improve the generalization of the network in different scenarios. Numerous representative simulations are conducted, and our method is further deployed to an experimental platform based on a multiomnidirectional-wheeled car system. The effectiveness and practicability of our proposed method are validated through both the simulation and experiment results.

It is commonly believed that observer-based compensation is an effective way for disturbance rejection. A less talked about fact is that such disturbance rejection control technique may also degrade control performance. In this article, we present a typical cross-coupling system to reveal this problem and, more importantly, propose a new design principle of conditional disturbance negation (CDN) to eliminate its potential drawbacks of disturbance observer-based compensation. Qualitative analysis is first given for a general form of such cross-coupling systems, indicating the necessity of CDN. The analysis and control design principle of CDN is then exemplified through two applications. A numerical linear application produces abundant quantitative results through the powerful transfer function and frequency domain tools. A more complex nonlinear flexible air-breathing hypersonic vehicle application shows how conventional compensation deteriorates the couplings between rigid and flexible modes, and validates the effectiveness of CDN through comprehensive model analysis and simulation results. The proposed CDN design principle also arouses awareness of the importance of: 1) understanding the characteristics of the plant to be controlled and 2) recognizing the critical role the information plays in engineering practice.

Making efficient motion decisions for a multi-robot system is a challenging problem in target encirclement with collision avoidance. Specifically, each robot with local communication has to consider cooperative target encirclement and collision avoidance simultaneously. In this paper, a distributed transferable policy network framework based on deep reinforcement learning is proposed to solve the problem of multi-robot cooperative target encirclement with collision avoidance. The proposed policy network framework is able to process the information of uncertain number of robots and obstacles, which is a desirable property for multi-robot systems. In particular, graph attention communication mechanism is adopted to model multi-robot interactions as a graph and extract cooperative information from the graph. Long short-term memory is used to accept the states of uncertain number of obstacles. In addition, a compound reward is designed to lead the training of the behavior of target encirclement with collision avoidance. Curriculum learning is implemented to speed up the process of this training. Simulation results validate the effectiveness of the proposed algorithm. Moreover, we further show that the learned policy can directly transfer to different scenarios along with good generalization.

It is commonly believed that uncertainties are obstacles to the tracking performances of flexible air-breathing hypersonic vehicles (FAHVs). In addition, the estimation of unmeasured states in an FAHV makes this issue much more complicated. To deal with these difficulties, this article explores a novel adaptive fuzzy nonsmooth backstepping output-feedback control scheme for the FAHV with closed-loop finite-time convergence. First, to approximate the unknown dynamics of the FAHV, some function approximators are constructed by utilizing an interval type-2 (IT2) fuzzy logic system. On this basis, a fixed-time convergent adaptive IT2 fuzzy observer is designed to estimate the unmeasured flight path angle and the unmeasured angle of attack of the FAHV accurately and rapidly. Consequently, the estimation errors of the designed observer are convergent in a fixed time independent of their initial estimation errors. Furthermore, based on the estimated states, velocity and altitude tracking controllers are designed by using a finite-time adaptive IT2 fuzzy nonsmooth backstepping control technique, which avoids the problem of “explosion of complexity” in traditional backstepping methods. Subsequently, rigorous Lyapunov stability analysis is conducted to show the finite-time convergence of the closed-loop signals of the FAHV control system. Finally, the robustness and superiority of the proposed control scheme is validated by several simulations in representative scenarios.

Uncertainty and measurement noise are main obstacles that limit the tracking control performances of flexible air-breathing hypersonic vehicles (FAHVs). In this article, we propose a novel fixed-time convergent nonsmooth backstepping control scheme for FAHV via augmented sliding mode observers (ASMOs) to overcome these obstacles. The ASMOs are first designed for the FAHV dynamics by employing the measured states corrupted by noises as inputs. On one hand, the ASMOs can simultaneously estimate the uncertainties and filter out the measurement noises. On the other hand, the observation error of each ASMO can be convergent within a fixed time independent of its initial observation error. Then, based on the estimation results, the altitude and velocity tracking controllers are developed by using fixed-time nonsmooth backstepping technique. Afterwards, a Lyapunov-based stability analysis is given to illustrate the fixed-time convergence of the closed-loop signals of the FAHV control system. Finally, comparative simulations are conducted to illustrate the superiority of the proposed control scheme.

This paper presents a novel fixed-time sliding mode disturbance observer (SMDO)-based robust backstepping cruise tracking control scheme with closed-loop finite-time convergence for flexible air-breathing hypersonic vehicles (FAHVs). In order to enhance the control system's robustness, a fixed-time SMDO is designed to compensate for the flexibility effects, model uncertainties, and external disturbances in FAHVs. In consequence, fixed convergence time of disturbance observation is achieved independently of initial estimation errors. Furthermore, velocity and altitude continuous finite-time tracking controllers are constructed by incorporating the SMDO and nonsmooth backstepping technique. To solve the problem of “explosion of complexity” in the conventional backstepping approach, nonsmooth filters are specifically constructed to generate the derivatives of virtual control laws. A Lyapunov-based stability analysis is conducted to show the finite-time convergence of the closed-loop FAHV control system. Finally, several representative numerical simulations are given to illustrate the effectiveness and superiority of the proposed control strategy.

Generating collision free, time efficient paths for followers is a challenging problem in formation control with collision avoidance. Specifically, the followers have to consider both formation maintenance and collision avoidance at the same time. Recent works have shown the potentialities of deep reinforcement learning (DRL) to learn collision avoidance policies. However, only the collision factor was considered in the previous works. In this paper, we extend the learning-based policy to the area of formation control by learning a comprehensive task. In particular, a two-stage training scheme is adopted including imitation learning and reinforcement learning. A fusion reward function is proposed to lead the training. Besides, a formation-oriented network architecture is presented for environment perception and long short-term memory (LSTM) is applied to perceive the information of an arbitrary number of obstacles. Various simulations are carried out and the results show the proposed algorithm is able to anticipate the dynamic information of the environment and outperforms traditional methods.

This paper investigates the time-varying formation tracking control problem for multi-agent systems with consideration of model uncertainties. For each dimension of an agent, a radial basis function neural network (RBFNN) is first adopted to approximate the model uncertainties online. Taking the square of the norm of the neural network weight vector as a newly developed adaptive parameter, a novel RBFNN-based adaptive control law with minimal learning parameter (MLP) approach is then constructed to tackle the time-varying formation tracking problem. The uniformly ultimately boundedness (UUB) of formation tracking errors is guaranteed through Lyapunov analysis. Compared with other traditional RBFNN-based formation tracking control laws for multi-agent systems, very few parameters need to be updated online in our proposed one, which can greatly lessen the computational burden. Finally, comparative simulation results demonstrate the effectiveness and superiority of the proposed adaptive control law.

In practice, the measurement noise arising from actual state measurement deteriorates the system performance significantly. To alleviate the noise effect on the hypersonic vehicle's pitch angle and pitch rate channels (i.e. attitude subsystem), this paper designs the fuzzy approximation based adaptive controller. Based on the desired compensation technique, a novel adaptive update law is proposed in the fuzzy approximator, in which the desired values are used to replace the actual states. Hence, the corresponding fuzzy approximator is able to estimate the unknown nonlinearity in the system even when measurement noises exist. In addition, the matching error of the fuzzy approximation and residual uncertainties are handled through an additional robust control term in the control law. Stability analysis is conducted by employing the Lyapunov method. Finally, nominal and comparative simulation results show that the control strategies proposed in this paper can guarantee the bounded tracking performance with consideration of the measurement noise and uncertainty.

A parallel deep Q-network (DQN) algorithm is presented for solving multiagent constrained formation path planning, where reaching destination, avoiding obstacles, and maintaining formation are simultaneously considered as independent or interactive tasks. Parallel Q-networks are utilized for each agent to sense different feature information and learn independent behavior policy. Comprehensive reward function is designed in consideration of respective requirements and interaction constraints to correctly guide the training. In order to demonstrate the effectiveness of the algorithm, we build an end-to-end model by designing a pixel game. Both training and testing are carried out in the game with double dueling DQN and the results show that the parallel deep Q-network path planner eventually complete the three tasks very well.

This paper investigates the time-varying formation control problem for multiple unmanned aerial vehicle (multi- UAV) systems with unknown uncertainties and varying time delays. Firstly, a radial basis function neural network (RBFNN) is adopted to estimate the lumped model uncertainties online for compensation. Then, a novel RBFNN-based fully distributed adaptive control scheme consisting of control law to stabilize the system and adaptive law to adjust RBFNN weights is developed to tackle the time-varying formation tracking problem in the presence of varying time delay. The uniformly ultimately boundedness (UUB) of the formation tracking errors is theoretically analyzed through Lyapunov approach. Comparative simulation results demonstrate the effectiveness of the control and adaptive laws proposed in this paper.

The single-input-rule-modules (SIRMs) connected fuzzy inference method can efficiently solve the fuzzy rule explosion phenomenon, which usually occurs in the multivariable modeling and/or control applications. However, the performance of the SIRMs connected fuzzy inference system (SIRM-FIS) is limited due to its simple input-output mapping. In this paper, to further enhance the performance of SIRM-FIS, a functionally weighted SIRM-FIS (FWSIRM-FIS), which adopts multivariable functional weights to measure the important degrees of the SIRMs, is presented. Then, in order to show the fundamental differences of the SIRMs methods, properties of the traditional SIRM-FIS, the type-2 SIRM-FIS (T2SIRM-FIS), the functional SIRM-FIS (FSIRM-FIS), the SIRMs model with single-variable functional weights (SIRM-FW), and FWSIRM-FIS are explored. These properties demonstrate that the proposed FWSIRM-FIS has more general and complex input-output mapping than the existing SIRMs methods. Such properties theoretically guarantee that better performance can be achieved by FWSIRM-FIS. Furthermore, based on the least-squares method, a novel data-driven optimization method is presented for the parameter learning of FWSIRM-FIS. It can also be used to optimize the parameters of SIRM-FIS, T2SIRM-FIS, FSIRM-FIS, and SIRM-FW. Due to the properties of the least-squares method, the proposed parameter learning algorithm can overcome the drawbacks of the gradients-based parameter learning methods and obtain both smallest training errors and smallest parameters. Finally, to show the effectiveness and superiority of FWSIRM-FIS and the proposed optimization method, six examples and detailed comparisons are given. Simulation results show that FWSIRM-FIS can obtain better performance than the other SIRMs methods, and, compared with some well-known methods, FWSIRM-FIS can achieve similar or better performance but has much less parameters and faster training speed.

This paper presents a novel interval type-2 TSK nominal-fuzzy-model-based sliding mode controller (IT2-TSK-NFMSMC) for flexible air-breathing hypersonic vehicle (FAHV) in order to stress robustness of the control system in dealing with data-driven based fuzzy modelling deviations, system uncertainty and disturbances. We adopt backstepping structure decomposing FAHV model into 5 control subsystems and design controllers, respectively. More specifically, two subsystems are designed with integral sliding mode model controllers. Another three subsystems which directly coupling with flexible mode disturbances are designed with 1T2-TSK-NFMSMCs by the following steps: 1) interval type-2 TSK nominal-fuzzy-models (IT2-TSK-NFM) are generated automatically by using type-2 fuzzy self-organizing methods from experiment datasets; 2) nominal model sliding mode controllers are designed based on the IT2-TSK-NFM, respectively; 3) notch filters are adopted in order to decrease the disturbance effects from the flexible modes; 4) sliding mode compensation controllers are designed through Lyapunov synthesis in order to compensate differences between IT2-TSK-NFM and real models of the FAHV. Several scenarios are studied and the simulation results validate the robustness of the proposed controllers when there exist internal flexible vibration and external system disturbances.

In this paper, a novel control scheme for the flexible air-breathing hypersonic vehicle (FAHV) using adaptive interval type-2 fuzzy logic system (AIT2-FLS) is proposed to reduce the side effects of measurement noises in the velocity channel and altitude channel as well as flexible dynamics in real applications. After input-output linearization of the longitudinal model of FAHV, the dynamic inversion controller is formulated to track the reference commands based on state feedback. The AIT2-FLS is further developed to deal with the model uncertainties and input errors. Besides, the state estimator is applied to estimate the true values of the corrupted outputs. The stability characteristics of both the controller and the state estimator are analyzed. The whole control scheme is finally obtained through combining the controller and the state estimator based on the separation principle. Simulation results demonstrate the robustness of our proposed control scheme against measurement noises and flexibilities.

In order to realize adaptive quadruped locomotion on terrains with different properties (such as surface friction or elasticity modulus), we plan to collect the foot-ground contact force and gyroscope information during locomotion on different ground substrates, then classify the ground substrates with the feature vector extracted from the collected data using Support Vector Machine (SVM) algorithm. However, the quadruped walk gait generated by Central Pattern Generators (CPGs) does not perform well on certain ground substrates, e.g., robots may be stuck in the soft ground substrates with small elasticity modulus. Therefore, for one thing, we present a Center of Gravity (COG) adjustment method to eliminate the offset between the control signal generated by CPGs and the actual phase of the quadruped limb, so the limb in theoretical swing phase is able to lift off the ground. For another, we combine CPGs with a foot path planning method to make the lift height controllable. Using these methods, the quadruped robot Biodog realizes the sensor data collection on five different ground substrates. Then we train and classify the sensor data with the SVM. About 99.33% of the five ground substrates can be classified correctly.

This paper presents a novel approach to automatically creating an interval type-2 fuzzy neural network (IT2-FNN) from a type-1 fuzzy TSK system (T1-TSK). The IT2-FNN is constructed in such a way that it takes advantage of the well-behaving T1-TSK. Our approach makes designing the IT2-FNN more efficient and the resulting system is expected to perform better than the T1-TSK due to the footprint of uncertainty of the IT2 fuzzy sets, especially when the system is subject to heavy external or internal uncertainties. There are two automated procedures in the IT2-FNN formation: (1) antecedent structure construction, and (2) learning of the parameters in both the antecedent and consequent. The structure construction is based on antecedent structure of the T1-TSK and consists of three steps - IT2 fuzzy set creation, similarity categorization, and mergence. The IT2 fuzzy sets are directly initialized from the fuzzy sets of the T1-TSK. Then, the IT2 fuzzy sets are classified into different groups based on their similarities. Finally, the IT2 fuzzy sets in each group are merged to create a representative IT2 fuzzy set for each group. The parameter learning procedure uses a hybrid learning algorithm to attain the optimal values for all the parameters. The learning algorithm adopts a new adaptive steepest descent algorithm and a linear least-squares method to adjust the antecedent parameters and consequent parameters, respectively. One benchmark modelling problem is utilized to compare our approach with the T1-TSK systems in the literature under various scenarios. The comparison results show our IT2-FNN performs better than the T1-TSK systems, especially when there are strong uncertainties. In summary, the IT2-FNN can not only achieve better performance but its structure is simpler than that of the similar type-2 fuzzy neural networks in the literature.

A new output feedback control design for robust velocity and altitude tracking of an air-breathing hypersonic vehicle (AHSV) is presented in this paper. The control scheme is performed on the assumption that only partial states of AHSV are measurable. The key idea is to employ the immersion and invariance approach to design globally asymptotically stable observers for the unmeasurable states. For controller design, the whole control architecture is constructed using dynamic surface control, based on the decomposition of the longitudinal dynamics of AHSV into velocity and altitude subsystems. Stability analysis is presented using the Lyapunov theory. Representative simulations are carried out on the high-fidelity model, which illustrate the effectiveness and robustness of the proposed scheme.

In this paper an adaptive interval type-2 fuzzy sliding mode controller, which is applied to flexible air-breathing hypersonic vehicle (FAHV) longitudinal model, is designed based on interval type-2 fuzzy logic systems (IT2-FLS) and sliding mode control (SMC) theory. In order to get FAHV longitudinal model stably controlled, we decouple the model into velocity and altitude channels through output feedback linearization. Moreover, due to the severe uncertainties which mainly come from unpredictable varying aerodynamic interferences and mutual couplings in airframe flexible modes and those difficulties of computing nonlinear functions with high-order derivatives under practical conditions, we design a sliding mode controller to achieve system convergence and adopt IT2-FLS to estimate the nonlinear functions with bounded parameter uncertainties online for counteracting the tracking errors and suppressing flexible vibrations. The adaptive law of interval type-2 fuzzy sliding mode controller is derived through Lyapunov synthesis approach. Furthermore, we adopt tracking differentiator (TD) and nonlinear state observer (NSO) algorithms to generate the real-time derivatives and high-order approximate commands in velocity and altitude channels, respectively. Several comparisons have been done in this paper and the simulation results validate the robustness and effectiveness of the proposed controller.

In modeling, prediction and control applications, the single-input-rule-modules (SIRMs) connected fuzzy inference method can efficiently tackle the rule explosion problem that conventional fuzzy systems always face. In this paper, to improve the learning performance of the SIRMs method, a neural structure is presented. Then, based on the least square method, a novel parameter learning algorithm is proposed for the optimization of the SIRMs connected neural-fuzzy system. Further, the proposed neural-fuzzy system is applied to the cooling and heating loads prediction which is a popular multi-variable problem in the research domain of intelligent buildings. Simulation and comparison results are also given to demonstrate the effectiveness and superiority of the proposed method.

This paper is concerned with the problem of type-2 fuzzy adaptive inverse control for a cable-driven parallel system. Based on the heuristics and prior knowledge of the system, the system is divided into six subsystems. The proposed control scheme for each subsystem contains a forward model and a fuzzy adaptive inverse controller (FAIC), which are expressed by an interval type-2 fuzzy nonlinear autoregressive exogenous (NARX) model, respectively. To construct the antecedents of the interval type-2 fuzzy NARX forward models and FAICs, the monotonic property of the fuzzy NARX model is first proved, and then, their antecedent parameters can be determined by this property. Furthermore, the consequent parameters of the forward models are computed offline via a constrained least squares algorithm, and the consequent parameters of the FAIC are adjusted online via a recursive least squares algorithm. Experiment results are provided to show that the proposed type-2 fuzzy control scheme can realize the control objectives and achieve a good control performance.

Computing with words (CWW) proposed by Zadeh is an useful paradigm to mimic the human decision-making ability in a wide variety of physical and mental tasks. To realize CWW, Mendel proposed a specific architecture called perceptual computer, in which interval type-2 (IT2) fuzzy sets (FSs) and perceptual reasoning (PR) method are adopted. The PR method has been proved to have good properties (e.g. it can output intuitive IT2 FSs) and has found several applications in decision making. In this study, we focus on simplifying this method by avoiding its a-cuts based inference process. We first present a novel property for the inference of the PR method. We observe from the property that, if the IT2 FSs in the consequents of the IF-THEN rules are trapezoidal and have consistent slopes, then the output IT2 FS will be strictly trapezoidal and can be determined easily. In this case, the computation of the PR method can be simplified. To achieve such simplification, the trapezoidal IT2 FSs without consistent slopes should be approximated by the slope-consistent trapezoidal IT2 FSs. This issue is also studied in this paper by solving the constrained linear-quadratic optimization problem. At last, examples are given. The simplified PR method will be useful when the CWW models are utilized in the modeling and/or control problems of complex systems or multivariable dynamic systems.

A robust adaptive type-2 fuzzy logic controller is designed for the longitudinal dynamics of a flexible air-breathing hypersonic vehicle. The aircraft's pitch motion and flexible vibration are strongly coupled explicitly in the dynamic equations. The throttle setting is designed to control the velocity by dynamic inversion control method. The elevator deflection is designed to stabilize the pitch rate and flexible modes and in the end control the altitude in a stepwise manner by backstepping control method. The flexible modes are actively used in the control design in order to counteract both the tracking errors and the flexible vibrations. The virtual control signals in backstepping control as well as their derivatives are obtained by command filters whose magnitudes, bandwidths and rate limit constraints can be set. The transition processes of the velocity and altitude commands are also obtained by command filters. Uncertainties are estimated online by interval type-2 adaptive fuzzy logic system. The adaptive law of the fuzzy logic system is derived by Lyapunov synthesis approach. Simulation results demonstrate the effectiveness and robustness of the proposed controller and also validate type-2 fuzzy logic is more capable of handling uncertainties than type-1 fuzzy logic.

Qualitative knowledge is very useful for system modeling and control problems, especially when specific physical structure knowledge is unavailable and the number of training data points is small. This paper studies the incorporation of one common qualitative knowledge-monotonicity into interval type-2 (IT2) fuzzy logic systems (FLSs). Sufficient conditions on the antecedent and consequent parts of fuzzy rules are derived to guarantee the monotonicity between inputs and outputs. We take into account five type-reduction and defuzzification methods (the Karnik-Mendel method, the Du-Ying method, the Begian-Melek-Mendel method, the Wu-Tan method, and the Nie-Tan method). We show that IT2 FLSs are monotonic if the antecedent and consequents parts of their fuzzy rules are arranged according to the proposed monotonicity conditions. The derived monotonicity conditions are valid for the IT2 FLSs using any kind of IT2 fuzzy sets (FSs) (e.g., Trapezoidal IT2 FSs and Gaussian IT2 FSs) and stand for type-1 FLSs as well. Guidelines for applying the proposed conditions to modeling and control problems are also given. Our results will be useful in the design of monotonic IT2 FLSs for engineering applications when the monotonicity property is desired.

This paper tries to show some important proper ties of the single input rule modules (SIRMs) connected fuzzy inference systems (FIS), including both type-1 (Tl) and inter val type-2 (IT2) FISs. Three kinds of properties continuity, monotonicity and robustness are explored. First, conditions on the parameters are derived to ensure that the SIRMs connected FISs are continuous and monotonic. Then, a methodology for the robustness analysis of the SIRMs connected FISs are presented. At last, an example is given to show the correctness of the theorems on the continuity and monotonicity and to demonstrate the effectiveness of the proposed methodology for robustness analysis. These results can not only deepen our understanding of the SIRMs connected FISs, but also provide us guidelines for the design of the SIRMs connected FISs.

The paper presents the methods of integrating prior knowledge with a first-order Single-Input Single-Output (SISO) Interval Type-2 Takagi-Sugeno-Kang (TSK) Fuzzy Logic System (IT2FLS) for function approximation under noisy circumstances. Firstly, sufficient conditions on the antecedent and the consequent parameters of the IT2FLS are given to ensure that three kinds of prior knowledge monotonicity, symmetry and special points, can be embedded into the IT2FLS. And then, we use three optimization algorithms constrained least squares algorithm, active-set algorithm and hybrid learning algorithm to design the IT2FLS, respectively. The effectiveness of the three algorithms and the comparisons of their performance are demonstrated by simulation examples.

In this paper we propose an effective method to design a Single-Input Single-Output (SISO) Unnormalized Interval Type-2 Takagi-Sugeno-Kang (TSK) Fuzzy Logic System (UIT2FLS) for noisy regression problems based on multi-source knowledge which includes here the information from sample data and the prior knowledge of bounded range, symmetry and monotonicity. The sufficient conditions are given which ensure that the prior knowledge can be embedded into the UIT2FLS, and then the UIT2FLS is designed so that the target function can be approached as accurately as possible via constrained least squares algorithm. The performance of the UIT2FLS is verified through comparisons with unnormalized type-1 Fuzzy Logic Systems (FLSs) and normalized interval type-2 FLSs under three different noisy circumstances. Simulation results verify the correctness of the sufficient conditions, and demonstrate that the UIT2FLS has the best overall performance.

This paper tries to provide a stability analysis approach for the single input rule modules (SIRMs) based type-2 fuzzy logic control systems. First, in the neighbor of the equilibrium point, the closed-form input-output mappings of type-2 SIRMs (T2SIRMs) are explored, and the derivatives of T2SIRMs at the equilibrium point are computed. Then, how to compute the Jacobian matrix of the SIRMs based type-2 fuzzy logic control systems, which is a fundamental step for local stability analysis, is presented. At last, two examples on stabilization control of the TORA system and the inverted pendulum system are given. The results in both examples demonstrate that the stability analysis results agree completely with the control results.

This paper applies prior knowledge - monotonicity and convexity - to a Single-Input-Single-Output (SISO) un-normalized interval type-2 Takagi-Sugeno-Kang (TSK) Fuzzy Logic System (FLS). Sufficient conditions are provided to guarantee its monotonicity and convexity with respect to its input, respectively. The derived monotonic conditions focus on a zeroth-order TSK fuzzy model. Also, the corresponding proofs for the convex conditions of both the zeroth-order and first-order TSK fuzzy models are given, respectively. For the zeroth-order fuzzy systems, simulation examples demonstrate the validity of the theorems.

This paper proposes a new algorithm that employs Adaptive Dynamic Programming(ADP) to solve the distributed control problem of urban traffic with an infinite horizon. Urban traffic congestions lead to a lot of time consumption and exhaust emissions. So alleviating congested situation will have a good impact on both economy and environment. The signal control at urban intersections is an effective and most important way to reduce the traffic jams and collisions. A lot of control theories including traditional mathematical ways and modern artificial intelligent ways have been exploited. ADP is an effective and amiable intelligent control method. We proposed an algorithm to adjust the signal time plan at urban traffic intersections based on ADP theory. Simulations are taken under a microscopic traffic simulation software, TSIS(Traffic Software Integrated System). Several criteria named MOEs(Measures of Effectiveness) are collected to compare with the widely used pre-timed control, actuated control, also with a machine learning method Q-learning control. Results show that ADP control method have a better adaptability to the various traffic simulating real traffic flows.

Ramp metering has been developed as a traffic management strategy to alleviate congestion on freeways. Most ramp metering control algorithms are concerned without queuing consideration, because its still a tough job to deal with the problems of coordinated multiple ramps metering with queuing consideration. In this paper, on the basis of our previous studies, we use action-dependent heuristic dynamic programming based on eligibility traces (ADHDP(lambda)) to solve local ramp metering and multiple ramps metering problems with queuing consideration. First, for the local ramp metering problem, we establish a comprehensive performance index which considers both traffic density and on-ramp queue length. Second, for the multiple ramps metering problem, based on ADHDP(lambda), the coordinated ramps metering and regulating queue lengths are achieved at the same time. Simulation studies on a hypothetical freeway are reported. It is shown that the proposed control scheme is efficient.

The translational oscillations with a rotational proof-mass actuator (TORA) is a well-known benchmark for examining the advantages and limitations of different nonlinear control design techniques. In this paper, a single-input-rule-modules (SIRMs) based type-2 fuzzy logic control scheme is proposed for this nonlinear multivariable system. And, genetic algorithms (GAs) are adopted to determine the parameters and to improve the performance of the SIRMs based type-2 fuzzy logic controller (SIRM-T2FLC). At last, simulations and comparisons are given to demonstrate the effectiveness, robustness and superiority of the proposed controller under three circumstances: normal case, the disturbance existing case, and the parameter varying case. From the design process and comparisons, it can be seen that: 1) this SIRMs based type-2 fuzzy control scheme can alleviate the difficulty to design conventional type-2 fuzzy logic controllers (T2FLCs) for this multivariable TORA system, 2) the SIRM-T2FLC is much easier to design and understand compared with conventional nonlinear control strategies for the TORA system, 3) better performance can be achieved.

The prior knowledge-monotonicity property-is helpful for system analysis, modeling and design, especially when no specific physical structure knowledge about systems is available. This paper presents how to use interval type-2 fuzzy logic systems (IT2FLSs) to incorporate the monotonicity property into system design. First, we present sufficient conditions on the parameters of IT2FLSs to ensure the monotonicity between the inputs and outputs of IT2FLSs. Then, we transform the design of monotonic IT2FLSs to the least squares problem with linear-inequality constraints. At last, simulations are given to show the usefulness of the monotonicity property and the advantages of monotonic IT2FLSs under noisy circumstances.

This paper aims to efficiently deal with the problems of multiple ramps metering. A new method which is called neuro-fuzzy adaptive dynamic programming with eligibility traces (NFADP(lambda)) is proposed. With the introduction of neuro-fuzzy and eligibility traces, the performance of ADP is greatly enhanced. First of all, the expert experience is introduced to ADP, therefore the convergence of ADP is greatly reinforced. Second, with the learning strategy revised, the training of action network is accelerated. In order to achieve multiple ramps metering control, special performance index function is established in NFADP(lambda). Extensive simulation on a hypothetical freeway are carried out with NFADP(lambda), compared to ALINEA as a stand-alone strategy. Simulation results indicate that NFADP(lambda) have good performances in both alleviating stochastic variations of the traffic demand and congestion situations.

To solve the level-adjusting and force-tuning problems of high accurate and costly payloads when loading and unloading, a cable-driven auto-leveling parallel robot is developed. A hierarchical fuzzy controller, which has the ability to deal with the rule explosion problem, is proposed in this paper. After a brief introduction of the architecture of the closed-loop control system for the cable-driven auto-leveling parallel robot, the construction of the hierarchical fuzzy controller is set up, in which the force offsets of the four cables and the angle deviations of the two diagonal inclinations are chosen as input variables, and the output variables are the position changes of the four linear motion units. The hierarchical fuzzy controller contains two layers - the low level layer which generates two outputs for leveling adjustment and force tuning, and the high level layer which is used to coordinate the two outputs from the low level layer. Experimental results have demonstrated that the hierarchical fuzzy controller can achieve the control objectives with high regulation accuracy and short adjusting time, and can be easily applied to practical systems.

A new flight control system design scheme is presented for attitude tracking problem, which integrates hierarchy-structured nonlinear dynamic inversion (NDI) and dynamic control allocation techniques. The hierarchy-structured NDI with two time-scales separation is designed to generate the required aerodynamic moments for given attitude angles command. To avoid the shortcomings in static control allocation approaches, a dynamic control allocation algorithm is adopted to distribute the moments into individual control surfaces. Different from most control allocation methods, the actuator dynamics are considered. The flight control design scheme is evaluated by numerical simulation on a nonlinear six degree-of-freedom aircraft model. Simulation results show the validity and good tracking performance of the flight control system.

This paper addresses the robust trajectory tracking problem for a redundantly actuated omnidirectional mobile manipulator in the presence of uncertainties and disturbances. The development of control algorithms is based on sliding mode control (SMC) technique. First, a dynamic model is derived based on the practical omnidirectional mobile manipulator system. Then, a SMC scheme, based on the fixed large upper boundedness of the system dynamics (FLUBSMC), is designed to ensure trajectory tracking of the closed-loop system. However, the FLUBSMC scheme has inherent deficiency, which needs computing the upper boundedness of the system dynamics, and may cause high noise amplification and high control cost, particularly for the complex dynamics of the omnidirectional mobile manipulator system. Therefore, a robust neural network (NN)-based sliding mode controller (NNSMC), which uses an NN to identify the unstructured system dynamics directly, is further proposed to overcome the disadvantages of FLUBSMC and reduce the online computing burden of conventional NN adaptive controllers. Using learning ability of NN, NNSMC can coordinately control the omnidirectional mobile platform and the mounted manipulator with different dynamics effectively. The stability of the closed-loop system, the convergence of the NN weight-updating process, and the boundedness of the NN weight estimation errors are all strictly guaranteed. Then, in order to accelerate the NN learning efficiency, a partitioned NN structure is applied. Finally, simulation examples are given to demonstrate the proposed NNSMC approach can guarantee the whole system's convergence to the desired manifold with prescribed performance.

This paper presents a control scheme of a class of under-actuated systems with saturation using hierarchical sliding mode. This class with a single input and multiple outputs is made up of several subsystems. Based on this physical structure, the hierarchical structure of the sliding mode surfaces is developed as follows. The sliding surface of every subsystem is defined. Then the sliding surface of one subsystem is selected as the first layer sliding surface. The first layer sliding surface is used to construct the second layer sliding surface with the sliding surface of another subsystem. This process continues till all the subsystem sliding surfaces are included. The hierarchical sliding mode control law is deduced by using Lyapunov theorem. On account of saturation nonlinearity of the single input, asymptotic stability of the control system is proven by nonlinear small gain theorem. Parameter ranges of the subsystem sliding surfaces are also given. In practice, simulation and experimental results show the validity of this control method.

This paper focuses on developing a robust neural network (NN) based sliding mode controller (NNSMC) to solve the trajectory tracking problem of a redundantly-actuated omnidirectional mobile manipulator. The SMC is designed to be robust to disturbances assuring the stability of the system. The NN is used to identify the unstructured uncertainty of system dynamics. The stability of the closed-loop system, the convergence of the NN weight-updating process, and the boundedness of the NN weight estimation errors are all strictly guaranteed. Through theories analysis, we know the controller is also capable of disturbance-rejection in the presence of time varying disturbances. Finally, simulation results demonstrate the proposed NNSMC approach can guarantee the whole system’s convergence to the desired manifold with prescribed performance.

Increasing dependence on car-based travel has led to the daily occurrence of freeway congestions around the world. In order to improve the worse and worse traffic congestion situation and solve the problems brought with it, a new kind of effective, fast, and robust method should be presented. Ramp metering has been developed as a traffic management strategy to alleviate congestion on freeways. But, it doesnpsilat work well in uncertainty situations. In this paper, in order to solve the problems in uncertainty conditions, an on-line learning control method based on the fundamental principle of reinforcement learning is proposed. The method is ADP (adaptive dynamic programming) and in order to expedite the learning rate, the concept about eligibility traces is introduced here. Then eligibility trace and ADP is combined to present a new kind of traffic responsive control method. The new method is called action-dependent heuristic dynamic programming based on eligibility traces (ADHDP (lambda)). ADHDP (lambda) is an approximate optimal ramp metering method. Simulation studies on a hypothetical freeway indicate good control performance of the proposed real-time traffic controller.

This paper aims at developing near optimal traffic signal control for multi-intersection in city. Fuzzy control is widely used in traffic signal control. For improving fuzzy controlpsilas adaptability in fluctuate states, a controller combined with neuro-fuzzy system and adaptive dynamic programming (ADP) is designed. This controller can be used for cooperative control of multi-intersection. The adaptive dynamic programming gives reinforcement for good neuro-fuzzy system behavior and punishment for poor behavior. The neuro-fuzzy system adjusts its parameters according to the reinforcement and punishment. Then, those actions leading to better results tend to be chosen preferentially in the future. Comparing with traditional ADP, this controller uses neuro-fuzzy system as the action network. The neuro-fuzzy system offers some existing knowledge and reduces the randomness of traditional ADP. In this paper, the objective of the controller is to minimize the average vehicular delay. The controller can be trained to adapt fluctuant traffic states by real-time traffic data, and achieves a near optimal control result in a long run. Simulation results show that the trained controller achieves shorter average vehicular delay than the controller with initial membership function.

Based on the methodology of sliding mode, this paper presents a robust controller for a class of under-actuated systems with mismatched uncertainties. Such a system consists of a nominal system and the mismatched uncertainties. The structural characteristic of the nominal system is that it is made up of several subsystems. Based on this characteristic, the hierarchical structure of the sliding mode surfaces is designed for the nominal system as follows. Firstly, the nominal system is divided into several subsystems and the sliding mode surface of every subsystem is defined. Secondly, the sliding mode surface of one subsystem is selected as the first layer sliding mode surface. The first layer sliding mode surface is then to construct the second layer sliding mode surface with the sliding mode surface of another subsystem. This process continues till the sliding mode surfaces of all the subsystems are included. For dealing with the mismatched uncertainties, a lumped sliding mode compensator is designed at the last layer sliding mode surface. The asymptotic stability of every layer sliding mode surface and the sliding mode surface of each subsystem is proven theoretically by Barbalat's lemma. Simulation results show the validity of this robust control method through stabilization control of a double inverted pendulums system with mismatched uncertainties.

Differentiation of syndromes of traditional Chinese medicine (TCM) mainly depends on the information obtained from four diagnosis methods. Now many physicochemical parameters are available in clinic. There exists great correlation between TCM syndromes and physicochemical parameters. The objective of the paper is to analyze the correlation between TCM syndromes and physicochemical parameters quantitatively and find the most informative physicochemical parameter combination which is useful in differentiation of syndromes. A novel definition of correlation degree based on Renyi's entropy is proposed which can measure the correlation between variables efficiently. Feature selection based on the correlation degree is used to find the most informative physicochemical parameters for assisting differentiation of syndromes.

This paper addresses the trajectory tracking problem for an redundantly-actuated omnidirectional mobile manipulator system with uncertainties and disturbances. The proposed algorithm is robust adaptive control strategy and the parameter estimates are tuned online. First, for designing controller, the conservative upper-bounded function of dynamic model of omnidirectional mobile manipulator system is derived based on the dynamic structure properties. Then, a robust adaptive control scheme is presented to ensure trajectory tracking effect of this closed-loop system. The asymptotical stability is verified a Lyapunov method. Finally, simulation examples are given to demonstrate the proposed approach can guarantee the whole system converge to the desired manifold with prescribed performance.

Because of the complexity of the mechanisms of redundantly actuated omni-directional Wheeled Mobile Robots (WMR), its motion regulation is a challenging problem, especially for consideration of the interaction force between the redundantly actuated wheels. The interaction force can be decomposed into motion-induced force and internal force, which are orthogonal between each other. Only the motion-induced force contributes to the motion of the robot, while the internal force abrades the wheels components, and causes the reduction of their life span. So the internal force should be eliminated or minimized. In this paper, kinematic model and dynamic model of redundantly actuated omni-directional WMR considering the interaction force is first established. A proportional differential plus motion regulator is presented. An integral feedback internal force controller is applied to minimize the internal force. Simulation results verify the effectiveness of the proposed control scheme. The robot is regulated successfully, and the internal force is reduced efficiently.

With the development of multicast service in the Internet, much attention has been drawn to multicast congestion control and analysis. Multicast traffic poses new challenges to the design of Internet congestion control protocols and system stability analysis. The rate control problem of feedback-based sessions on the coexistence of both unicast and multirate multicast traffic architecture networks is focused upon in this paper. First, a fairness problem is discussed in detail, and a reasonable consumption strategy is proposed. In the reasonable consumption strategy, scaling functions are adaptively adjusted based on a relationship between the session rates. Second, contraposing the case that available link capacities are changing with time for these feedback-based unicast and multicast sessions, stability analysis of a closed-loop rate control system under the modified rate mechanism is made based on Lyapunov stable theory. Finally, the simulations illustrate the effectiveness and goodness of the reasonable consumption strategy

A new fuzzy control design for way-point tracking control problem of ship autopilot is proposed. To effectively control ships in a designed trajectory is always an important task for ship manipulators. The paper gives the design method of a kind of fuzzy autopilot for way-point tracking control system. The fuzzy control rules are constructed based on human operator's manipulating experience. This control design approach greatly simplifies the control design process and the control algorithm, and is easily applied to practical ship tracking system. Simulation results show that the proposed fuzzy autopilot has desired performance with high accuracy of course keeping, short time of rudder actions and less frequency of changing rudder direction.

This paper proposes a hierarchical sliding mode controller for series double inverted pendulums system. This provides a simple method to control a class of under-actuated systems with three subsystems by sliding mode control. Firstly, the given system is divided into three subsystems according to its structure characteristic. Then, the 1st-level sliding mode surface is defined for every subsystem and the 2nd-level sliding mode surface is constituted by them. Based on the two levels structure, the equivalent control of each subsystem is deduced and the total control law is derived by the Lyapunov stability theorem. The asymptotical stability of the entire sliding mode surfaces is proved theoretically. Finally, simulation results show the validity of this control strategy. And the influence of the controller parameter changes for the performances is also discussed

This paper presents a new feature point pairs based technique for object pose estimation and structure recovery from a single view. It first estimates rotational matrix independently, then computes translation vector and recovers the 3D structure of the object directly. Linear and nonlinear strategies are presented to estimate the rotational matrix. One is for small rotational motion and the other is used to estimate large rotational parameters. When the nonlinear technique is applied, its initial guesses are given automatically by the proposed linear estimation method. On the other hand, the presented structure recovery method is not sensitive to the rotational matrix estimation results. The proposed method is applicable to three, four or more feature points and has no constraints, such as collinear or coplanar, on their relative positions. As the number of feature points increases, the estimation results are improved while the computation cost is almost unchanged. Many experiments are performed on synthetic data and real images to demonstrate the presented technique.

Tracking control problem of mobile manipulators with dynamical uncertainties is addressed in this paper. The controller is designed based on model of mobile manipulators consisting of two cascaded subsystems: a chained-like kinematical model without uncertainties and a dynamical model with uncertainties. The proposed control law can ensure that full states of closed-loop system can track given trajectories in presence of dynamical uncertainties. A globally asymptotic stability is obtained in Lyapunov sense. Simulation studies show feasibility and effectiveness of the proposed approach.

A new stable sliding mode control method for a class of underactuated mechanical systems is proposed in this paper. The controller has the double-layer structure. Firstly, the system states are divided into several different subsystems. For each of these subsystems, a first-layer sliding plane is constructed. From these first-layer sliding planes, then we further construct a second-layer sliding plane. By analyzing the features of the mathematical model of the underactuated mechanical systems, we derive the sliding-mode control law and indicate the ranges of the controller parameters. Using Lyapunov law, the paper proves the stability of all the sliding planes theoretically. The simulation results show the validity of this method for this class of underactuated mechanical systems.

On the basis of sliding mode control, a new cascade sliding-mode controller (CSMC) for a class of large-scale underactuated systems is proposed. The large-scale underactuated systems include several subsystems. Firstly, two states are chosen to construct the first-layer sliding surface. Secondly, the first-layer sliding surface and one of the left states are used to construct the second-layer sliding surface. This process continues till the last-layer sliding surface is obtained. By theoretical analysis, the cascade sliding-mode controller is proved to be globally stable in the sense that all signals involved are bounded. The simulation results show the validity of this method.

In real applications, captured doorplate images often contain unexpected noise from irregular illumination conditions, various imaging angles, different imaging distances, etc. In this paper, a robust doorplate recognition system is presented. First, an efficient method based on Sobel operator, region splitting and merging is applied to extract doorplate. Then, according to the doorplate shape and the character position in the doorplate, the character region is determined. If the candidate of some doorplate characters extracted from the character region cannot be determined at the segmentation step, a speculation based on known knowledge about geometrical relations between each doorplate character is executed. The threshold for character extraction from candidates is adjusted when the corresponding character is rejected after classification. The data set and the images in experiments all come from practical applications. Experimental results indicate that the proposed doorplate recognition system effectively improves the recognition result.

The problem of regulating a nonholonomic system in chained form to an equilibrium state is addressed and solved by a smooth time-varying control law. The proposed scheme based on Lyapunov analysis can guarantee asymptotic stabilization and exponential stabilization, respectively, by simply changing a term in the controller expression. Therefore, the controller can be used for theoretical analysis and practical application. The additional novelty of the proposed approach is that the parameters in our controller have clear physical meanings and are easily implemented and tuned. Simulation results based on a numerical example and a tricycle-type mobile robot are presented to demonstrate the effectiveness of the developed method.

This paper presents a localization method using motion vision. The proposed method locates a mobile robot relative to the object to which the robot moves. Two points are selected from the object as feature points. Consecutive two images containing the feature points are taken before and after the robot moves. Then, the poses of the robot can be determined according to the image coordinates of the feature points. A search algorithm is also presented to enhance the localization precision. It can find more correct image coordinates of the feature points based on the real detected image coordinates. Many experiments are performed on real images to justify this search algorithm. The results show that it is effective.

A new fuzzy controller for anti-swing and positioning control of an overhead traveling crane is proposed based on the SIRMs (single input rule modules) dynamically connected fuzzy inference model. The trolley position and velocity, the rope swing angle and angular velocity are selected as input items, and the trolley acceleration as output item. With a simple structure, the controller can autonomously adjust the influence of each input item. The control system is further proved to be asymptotically stable near destination. Control simulation results show that the controller is robust to different rope lengths and has generalization ability for different initial positions. Compared with linear state feedback controller, the fuzzy controller can drive the crane to destination in short time with small swing angle.

A stability analysis approach for a class of fuzzy controllers based on the single input rule modules (SIRMs) dynamically connected fuzzy inference model is proposed. It is proved that such a fuzzy controller can be approximately changed into a linear state feedback controller near an equilibrium point if each SIRM has a property of linear input-output mapping. Then linear state feedback control theories can be used to solve the stability problem. The proposed approach is applied to stabilization control of inverted pendulum system and the stability analysis results agree completely with the control results.

A fuzzy controller for backing up control of truck-trailer system is proposed based on the single input rule modules dynamically connected fuzzy inference model. The relative angle of the truck with the trailer, trailer angle, and trailer vertical position are selected as the input items. The dynamic importance degrees are set up such that the relative angle takes control priority when it becomes big. Simulation results demonstrate that the fuzzy controller backs up the truck-trailer system successfully even from a jackknife situation.

A systematic method to construct stabilization fuzzy controllers for a single pendulum system and a series-type double pendulum system is presented based on the single input rule modules (SIRMs) dynamically connected fuzzy inference model. The angle and angular velocity of each pendulum and the position and velocity of the cart are selected as the input items. Each input item is given with a SIRM and a dynamic importance degree. All the SIRMs have the same rule setting. The dynamic importance degrees use the absolute value(s) of the angle(s) of the pendulum(s) as the antecedent variable(s). The dynamic importance degrees are designed such that the upper pendulum angular control takes the highest priority and the cart position control takes the lowest priority when the upper pendulum is not balanced upright. The control priority orders are automatically adjusted according to control situations. The simulation results show that the proposed fuzzy controllers have high generalization ability to completely stabilize a wide range of single pendulum systems and series-type double pendulum systems in short time. By extending the architecture, a stabilization fuzzy controller for a series-type triple pendulum system is even possible. © 2001 John Wiley & Sons, Inc.

A systematical approach to constructing stabilization fuzzy controllers for single and series-type double inverted pendulum systems is presented based on the SIRMs (single input rule modules) dynamically connected fuzzy inference model. The common architecture of the controllers is discussed. The simulation results show that the fuzzy controllers have high generalization ability to stabilize a wide range of the single and double pendulum systems in about 6.0 and 10.0 seconds.

A fuzzy controller for stabilizing parallel-type double inverted pendulum system is presented, based on the single input rule modules (SIRMs) dynamically connected fuzzy inference model. By using the SIRMs and the dynamic importance degrees, the angular controls of the two pendulums and the position control of the cart are done entirely in parallel and the priority orders of the three controls are automatically adjusted according to control situations. Simulation results show that the fuzzy controller with a simple and intuitive structure can stabilize completely a parallel-type double inverted pendulum system within 10 seconds. This is the first result for a fuzzy controller to realize complete stabilization of a parallel-type double inverted pendulum system.