Visual trajectory estimation can endow autonomous underwater vehicles (AUVs) with environmental perception capabilities and has broad application prospects in the fields, such as oceanographic surveys, underwater construction, and marine ranching. However, due to featureless images and depth ambiguity issues caused by underwater multiple mediums environments, monocular visual trajectory estimation in complex underwater environments remains a challenging problem. In this article, we propose a monocular visual trajectory estimation method for AUVs, which can address the challenges of featureless and depth ambiguity by leveraging deep image representations and multiview geometry. Specifically, we design a bidirectional optical flow consistency scheme that selects sparse correspondences from monocular dense predictions to deal with the featureless images, and then achieve AUV trajectory estimation through epipolar constraints. Furthermore, we propose an iterative depth-consistent method, which solves the problem of depth ambiguity by aligning geometrically triangulated depths to the scale-consistent deep depths. We also develop a low-cost, agile, and portable AUV picking system with real-time trajectory estimation capabilities, and carry out extensive experiments in the Yellow Sea to test its performance. The experimental results demonstrate the effectiveness of the proposed method.

With the undetectable characteristic, adaptive model poisoning attacks can combine with any other attacks, bypassing the detection and violating the availability of federated learning (FL) systems. Existing defences are vulnerable to adaptive model poisoning attacks, as model poisoning-related features are tailored to these methods and compromise the accuracy of the FL model. We first present a unified reformulation of existing adaptive model poisoning attacks. Analyzing the reformulated attacks, we find that the detectors should reduce the attacker’s optimization cost functions to defeat adaptive attacks. However, existing defences do not consider the causes of model parameters’ high dimensionality and data heterogeneity. We propose a novel robust FL algorithm, FedDet, to tackle the problems. By splitting the local models into layers for robust aggregation, FedDet can overcome the issue with high dimensionality while keeping the functionality of layers. During the robust aggregation, FedDet normalizes every slice of local models by the median norm value instead of excluding some clients, which can avoid deviation from the optimal model. Furthermore, we conduct a comprehensive security analysis of FedDet and an existing robust aggregation method. We propose the upper bounds on the perturbations disturbed by these adaptive attacks. It is found that FedDet can be more robust than Krum with a smaller perturbation upper bound under attacks. We evaluate the performance of FedDet and four baseline methods against these attacks under two classic data sets. It demonstrates that FedDet significantly outperforms the existing compared methods against adaptive attacks. FedDet can achieve 60.72% accuracy against min–max attacks.

Federated learning (FL) is an efficient, scalable, and privacy-preserving technology in which clients collaborate on machine learning or deep learning model training. However, malicious clients can send poisoned model updates to the central server without being identified, which makes FL vulnerable to backdoor attacks. In this work, we propose a novel defence approach, FLSec, to mitigate backdoor attacks caused by adversarial local model updates. FLSec utilizes an original measurement, GradScore, computed from the loss gradient norm of the final layer of the local models for backdoor defence. We show that GradScore is efficient and robust in identifying malicious model updates through analysis and experiments. Our extensive evaluation also demonstrates FLSec is highly effective in mitigating three state-of-the-art backdoor attacks on well-known datasets, MNIST, LOAN, and CIFAR-10. The accuracy on a benign dataset with the proposed defence approach is nearly unchanged, with the accuracy on the backdoor dataset being reduced to 0%. In addition, our experiments show that FLSec significantly outperforms existing backdoor defences in multi-round backdoor attacks.

The visual loop-closure detection for autonomous underwater vehicles (AUVs) is a key component to reduce the drift error accumulated in simultaneous localization and mapping tasks. However, due to viewpoint changes, textureless images, and fast-moving objects, the loop closure detection in dramatically changing underwater environments remains a challenging problem to traditional geometric methods. Inspired by strong feature learning ability of deep neural networks, we propose an underwater loop-closure detection method based on a variational autoencoder network in this article. Our proposed method can learn effective image representations to deal with the challenges caused by dynamic underwater environments. Specifically, the proposed network is an unsupervised method, which avoids the difficulty and cost of labeling a great quantity of underwater data. Also included is a semantic object segmentation module, which is utilized to segment the underwater environments and assign weights to objects in order to alleviate the impact of fast-moving objects. Furthermore, an underwater image description scheme is used to enable efficient access to geometric and object-level semantic information, which helps to build a robust and real-time system in dramatically changing underwater scenarios. Finally, we test the proposed system under complex underwater environments and get a recall rate of 92.31% in the tested environments.

Depth estimation from monocular video plays a crucial role in scene perception. The significant drawback of supervised learning models is the need for vast amounts of manually labeled data (ground truth) for training. To overcome this limitation, unsupervised learning strategies without the requirement for ground truth have achieved extensive attention from researchers in the past few years. This paper presents a novel unsupervised framework for estimating single-view depth and predicting camera motion jointly. Stereo image sequences are used to train the model while monocular images are required for inference. The presented framework is composed of two CNNs (depth CNN and pose CNN) which are trained concurrently and tested independently. The objective function is constructed on the basis of the epipolar geometry constraints between stereo image sequences. To improve the accuracy of the model, a left-right consistency loss is added to the objective function. The use of stereo image sequences enables us to utilize both spatial information between stereo images and temporal photometric warp error from image sequences. Experimental results on the KITTI and Cityscapes datasets show that our model not only outperforms prior unsupervised approaches but also achieving better results comparable with several supervised methods. Moreover, we also train our model on the Euroc dataset which is captured in an indoor environment. Experiments in indoor and outdoor scenes are conducted to test the generalization capability of the model.

Industrial automation has received a considerable attention in the last few years with the rise of Internet of Things (IoT). Specifically, industrial communication network technology such as Profinet has proved to be a major game changer for such automation. However, industrial automation devices often have to exhibit robustness to dynamically changing network conditions and thus, demand a rigorous testing environment to avoid any safety-critical failures. Hence, in this paper, we have proposed an FPGA-based novel framework called “Profi-Load” to generate Profinet traffic with specific intensities for a specified duration of time. The proposed Profi-Load intends to facilitate the performance testing of the industrial automated devices under various network conditions. By using the advantage of inherent hardware parallelism and re-configurable features of FPGA, Profi-Load is able to generate Profinet traffic efficiently. Moreover, it can be reconfigured on the fly as per the specific requirements. We have developed our proposed Profi-Load framework by employing the Xilinx-based “NetJury” device which belongs to Zynq-7000 FPGA family. A series of experiments have been conducted to evaluate the effectiveness of Profi-Load and it has been observed that Profi-Load is able to generate precise load at a constant rate for stringent timing requirements. Furthermore, a suitable Human Machine Interface (HMI) has also been developed for quick access to our framework. The HMI at the client side can directly communicate with the NetJury device and parameters such as, required load amount, number of packet(s) to be sent or desired time duration can be selected using the HMI.

Autonomous gait optimization is an essential survival ability for mobile robots. However, it remains a challenging task for underwater robots. This paper addresses this problem for the locomotion of a bio-inspired robotic fish and aims at identifying fast swimming gait autonomously by the robot. Our approach for learning locomotion controllers mainly uses three components: 1) a biological concept of central pattern generator to obtain specific gaits; 2) an onboard sensory processing center to discover the environment and to evaluate the swimming gait; and 3) an evolutionary algorithm referred to as particle swarm optimization. A key aspect of our approach is the swimming gait of the robot is optimized autonomously, equivalent to that the robot is able to navigate and evaluate its swimming gait in the environment by the onboard sensors, and simultaneously run a built-in evolutionary algorithm to optimize its locomotion all by itself. Forward speed optimization experiments conducted on the robotic fish demonstrate the effectiveness of the developed autonomous optimization system. The latest results show that our robotic fish attained a maximum swimming speed of 1.011 BL/s (40.42 cm/s) through autonomous gait optimization, faster than any of the robot's previously recorded speeds.

We propose a novel monocular visual odometry (VO) system called UnDeepVO in this paper. UnDeepVO is able to estimate the 6-DoF pose of a monocular camera and the depth of its view by using deep neural networks. There are two salient features of the proposed UnDeepVo:one is the unsupervised deep learning scheme, and the other is the absolute scale recovery. Specifically, we train UnDeepVoby using stereo image pairs to recover the scale but test it by using consecutive monocular images. Thus, UnDeepVO is a monocular system. The loss function defined for training the networks is based on spatial and temporal dense information. A system overview is shown in Fig. 1. The experiments on KITTI dataset show our UnDeepVO achieves good performance in terms of pose accuracy.

This paper presents an indoor relocalization system using a dual-stream convolutional neural network (CNN) with both color images and depth images as the network inputs. Aiming at the pose regression problem, a deep neural network architecture for RGB-D images is introduced, a training method by stages for the dual-stream CNN is presented, different depth image encoding methods are discussed, and a novel encoding method is proposed. By introducing the range information into the network through a dual-stream architecture, we not only improved the relocalization accuracy by about 20% compared with the state-of-the-art deep learning method for pose regression, but also greatly enhanced the system robustness in challenging scenes such as large-scale, dynamic, fast movement, and night-time environments. To the best of our knowledge, this is the first work to solve the indoor relocalization problems based on deep CNNs with RGB-D camera. The method is first evaluated on the Microsoft 7-Scenes data set to show its advantage in accuracy compared with other CNNs. Large-scale indoor relocalization is further presented using our method. The experimental results show that 0.3 m in position and 4° in orientation accuracy could be obtained. Finally, this method is evaluated on challenging indoor data sets collected from motion capture system. The results show that the relocalization performance is hardly affected by dynamic objects, motion blur, or night-time environments.

This paper addresses a particular pursuit-evasion game, called as "fishing game" where a faster evader attempts to pass the gap between two pursuers. We are concerned with the conditions under which the evader or pursuers can win the game. This is a game of kind in which an essential aspect, barrier, separates the state space into disjoint parts associated with each player's winning region. We present a method of explicit policy to construct the barrier. This method divides the fishing game into two subgames related to the included angle and the relative distances between the evader and the pursuers, respectively, and then analyzes the possibility of capture or escape for each subgame to ascertain the analytical forms of the barrier. Furthermore, we fuse the games of kind and degree by solving the optimal control strategies in the minimum time for each player when the initial state lies in their winning regions. Along with the optimal strategies, the trajectories of the players are delineated and the upper bounds of their winning times are also derived.

This paper aims to segment similar targets shapes from multiple images by using unsupervised weighted group-similarity active contour model. We first use global contrast based saliency detector to extract the rough regions from the given multiple images group. Then a new algorithm is developed to measure the corresponding weight coefficients according to the similarities between rough regions and their latent common shape. In order to overcome the problem which caused by the trade-off between frame-specific details and group similarity more effectively during the evolution, a novel weighted group-similarity active contour model (WGSAC) is proposed, which reduces the noises generated from saliency detector dynamically and enables the curves to move toward the targets boundaries on different weighted images. Experiments on synthesized and real multiple images demonstrate that our approach is able to yield more stable segmentation results than previous methods.

A potential threat to embedded systems is the execution of unknown or malicious software capable of triggering harmful system behavior, aimed at theft of sensitive data or causing damage to the system. Commercial off-the-shelf embedded devices, such as embedded medical equipment, are more vulnerable as these type of products cannot be amended conventionally or have limited resources to implement protection mechanisms. In this paper, we present a self-organizing map (SOM)-based approach to enhance embedded system security by detecting abnormal program behavior. The proposed method extracts features derived from processor's program counter and cycles per instruction, and then utilises the features to identify abnormal behavior using the SOM. Results achieved in our experiment show that the proposed method can identify unknown program behaviors not included in the training set with over 98.4% accuracy.

This paper presents a novel on-line AdaBoost based discriminative active contour tracking framework (ADACT) using level sets. First we build an on-line AdaBoost based appearance model to track and extract the rough target region, which provides important discriminative clues for our active contour model. Integrating with both edge and discriminative region information, a new active contour model is proposed for obtaining accurate target contour after curve evolution. Experiments on the challenging video sequences demonstrate that the proposed method can achieve more robust deformable target contour tracking under various situations than other competitive contour tracking methods.

This paper studies three-dimensional multi-robot Cooperative Localization (CL) problem. Most of existing CL strategies adopt Extended Kalman Filter (EKF) or Maximum a Posteriori (MAP). In this paper, a novel approach based on Moving Horizon Estimation (MHE) is proposed. The main contribution of this paper is twofold: 1) MHE is integrated with EKF for three-dimensional CL using single mobile beacon, which can bound localization error, impose various constraints on states and noises, and make use of previous range measurements for current estimation. 2) A sufficient condition on observability of multi-robot CL is derived by using Fisher Information Matrix. Simulation is conducted to verify that the proposed MHE based CL algorithm outperforms EKF based method in terms of localization accuracy, and two scenarios where our algorithm is superior to EKF are discussed.

This paper studies the underwater localization problem for a school of robotic fish, i.e., a kind of Autonomous Underwater Vehicles with limited size, power and payload. These robotic fish cannot be equipped with traditional underwater localization sensors that are big and heavy. The proposed localization system is performed by using a single surface mobile beacon which provides range measurement to bound the localization error. The main contribution of this paper lies in twofold: 1) Observability of single beacon based localization is first analyzed in the context of nonlinear discrete time system, deriving a sufficient condition on observability. 2) Moving Horizon Estimation is then integrated with Extended Kalman Filters for three-dimensional localization using single beacon, which can reduce the computational complexity, impose various constraints and make use of previous range measurements for current estimation. Extensive numerical simulations are conducted to verify the observability and high localization accuracy of the proposed underwater localization method.

The lack of a sampling frame (i.e., a complete list of users) for most Online Social Networks (OSNs) makes sampling methods especially difficult. Thus, reliable and efficient sampling methods are essential for practical estimation of OSN properties. Recent work in this area has thus focused on sampling methods that allow precise inference from a relatively large-scale social networks such as Facebook. We propose a sampling method on OSNs, based on a Metropolis-Hastings Random Walk (MHRW) algorithm. In this regard, we have developed a social explorer in order to collect random samples from Facebook. In addition, we address the question whether different probability distributions may be able to alter the behavior of the MHRW and enhance the effectiveness of yielding a representative sample. Thus, in this paper, we seek to understand whether the MHRW algorithm can be exploited by switching the random generator to provide better results. We evaluated the performance of our MHRW algorithm providing a descriptive statistics of the collected data. Moreover, we sketch the collecting procedure carried out on Facebook in real-time. Finally, we provide a formal convergence analysis to evaluate whether the sample of draws has attained an equilibrium state to get a rough estimate of the sample quality.

Providing a visual image of a hazardous substance such as nerve gas or nuclear radiation using multiple robotic agents could be very useful particularly when the substance is invisible. Such visual representation could show where the hazardous substance concentration is highest through the deployment of a higher density of robotic agents to that area enabling humans to avoid such areas. We present an algorithm that is capable of doing the aforementioned with very minimal cost when compared with other techniques such as Voronoi partition methods. Using a mathematical proof, we show that the algorithm would always converge to the distribution of a spatial quantity under investigation. The mathematical model of the bacterium as developed by Berg and Brown is used in this paper, and through simulations and physical experiments, we show that a controller based upon the model is capable of being used to visually represent an invisible spatial hazardous substance using simplistic agents with the future possibility of the same algorithm being used to track a rapidly changing spatiotemporal substance. We believe that the algorithm has this potential because of its low communication and computational needs.

This paper presents a method of using Gaussian process regression to model spatial functions for mobile wireless sensor networks. A distributed Gaussian process regression (DGPR) approach is developed by using a sparse Gaussian process regression method and a compactly supported covariance function. The resultant formulation of the DGPR approach only requires neighbor-to-neighbor communication, which enables each sensor node within a network to produce the regression result independently. The collective motion control is implemented by using a locational optimization algorithm, which utilizes the information entropy from the DGPR result. The collective mobility of sensor networks plus the online learning capability of the DGPR approach also enables the mobile sensor network to adapt to spatiotemporal functions. Simulation results are provided to show the performance of the proposed approach in modeling stationary spatial functions and spatiotemporal functions.

In this paper, we investigate a moving-target tracking problem with sensor networks. Each sensor node has a sensor to observe the target and a processor to estimate the target position. It also has wireless communication capability but with limited range and can only communicate with neighbors. The moving target is assumed to be an intelligent agent, which is "smart" enough to escape from the detection by maximizing the estimation error. This adversary behavior makes the target tracking problem more difficult. We formulate this target estimation problem as a zero-sum game in this paper and use a minimax filter to estimate the target position. The minimax filter is a robust filter that minimizes the estimation error by considering the worst case noise. Furthermore, we develop a distributed version of the minimax filter for multiple sensor nodes. The distributed computation is implemented via modeling the information received from neighbors as measurements in the minimax filter. The simulation results show that the target tracking algorithm proposed in this paper provides a satisfactory result.

In this paper, we investigate a moving target tracking problem with mobile sensor networks. The moving target is assumed to be an intelligent agent, which is ‘smart’ enough to escape from the detection. We formulate this target estimation problem as a zero-sum game in this paper and use a so-called minimax filter to estimate the target position. The minimax filter is a robust filter that minimizes the estimation error by considering the worst case noise. Furthermore we develop a distributed version of the minimax filter for multiple sensor nodes. The distributed computation is implemented via a consensus filter. Finally, the mobile sensor nodes need to control their motions to move towards the estimated target position and avoid collisions with neighbors. A flocking algorithm is developed for this purpose. The simulation results show that the target tracking algorithm proposed in this paper provides a satisfactory result.

This paper presents a distributed algorithm for mobile sensor networks to monitor the environment. With this algorithm, multiple mobile sensor nodes can collectively sample the environmental field and recover the environmental field function via machine learning approaches. The mobile sensor nodes are able to self-organise so that the distribution of mobile sensor nodes matches to the estimated environmental field function. In this way, it is possible to make the next-step sampling more accurate and efficient. The machine learning approach used for function regression is support vector regression (SV R) algorithm. A distributed SV R learning algorithm is used for on-line learning. The self-organised algorithm used for deployment is based on locational optimisation techniques. In particular, Lloyd's algorithm for optimising centroidal Voronoi tessellations (CV T) is used to spread mobile sensor nodes over the monitored environment. The environmental field function is simulated in static and dynamic settings and the demonstration on the simulated environments shows the proposed algorithm is effective.

This paper presents a leader-follower flocking system based on local sensor information. In this system, the leader robot knows global trajectory, tries to track the global trajectory, and has the ability to avoid collisions. The follower robots do not know global trajectory. They only use local sensors to acquire relative information between neighbors. In this paper they use a laser scanner to obtain relative distances and relative angles between neighbors. By using local information, the follower robots maintain the distances between their neighbors and avoid collisions with their neighbors. We prove that such a leader-follower flocking system is stable based on LaSalles invariance principle. To evaluate the performance of the flocking system, we simulate the flocking system tracking a desired trajectory. The flocking algorithm is tested with three Pioneer robots and SICK laser scanners. Both of simulation and real robot experiments successfully show the proposed local sensor based algorithm works in the leader-follower flocking system.

This paper presents a distributed support vector regression (SV R) algorithm for sensor networks. The idea behind this algorithm is to make use of the structure similarity between sensor networks and SV Rs with 2D input data in order to implement SV R in a distributed way. During training stage, each sensor node provides its 2D coordinates as an input pattern and a sensory data as an output to the algorithm. By using local wireless communication with neighbors and kernel function with finite support, each sensor node independently learns its own Lagrange multipliers. During evaluation stage of learned regression function, each sensor node obtains a local result by communicating with local neighbors and estimates a global result by using a consensus algorithm. Simulations are provided to verify the proposed algorithm.

Flocking algorithms essentially consist of three components: alignment, cohesion, and separation. To track a desired trajectory, the flock center should move along the desired trajectory, and thus, the fourth component, navigation, is necessary. The alignment, cohesion, and navigation components are well implemented through consensus protocols and tracking controls, while the separation component is designed through heuristic-based approaches. This paper proposes a fuzzy logic solution to the separation component. The TS rules and Gaussian membership functions are used in fuzzy logic. For fixed network flocking, a standard stability proof by using LaSalle's invariance principle is provided. For dynamic network flocking, a Filipov solution definition is given for nonsmooth dynamics. Then, a LaSalle's invariance principle for nonsmooth dynamics is used to prove the stability. A group of mobile robots with double integrator dynamics is simulated for the flocking algorithms in a 2-D environment.

This paper presents a distributed expectation-maximization (EM) algorithm over sensor networks. In the E-step of this algorithm, each sensor node independently calculates local sufficient statistics by using local observations. A consensus filter is used to diffuse local sufficient statistics to neighbors and estimate global sufficient statistics in each node. By using this consensus filter, each node can gradually diffuse its local information over the entire network and asymptotically the estimate of global sufficient statistics is obtained. In the M-step of this algorithm, each sensor node uses the estimated global sufficient statistics to update model parameters of the Gaussian mixtures, which can maximize the log-likelihood in the same way as in the standard EM algorithm. Because the consensus filter only requires that each node communicate with its neighbors, the distributed EM algorithm is scalable and robust. It is also shown that the distributed EM algorithm is a stochastic approximation to the standard EM algorithm. Thus, it converges to a local maximum of the log-likelihood. Several simulations of sensor networks are given to verify the proposed algorithm.

In this paper, we present an approach to design a flocking algorithm by using fuzzy logic. The design of three basic behaviours in a flocking algorithm is discussed. They are alignment behaviour, separation behaviour, and cohesion behaviour. Navigation control component is used in the design of cohesion behaviour. To avoid becoming crowding or collision, an adaptive navigation gain is used. This gain changes with the number of neighbours. The flocking stability is analyzed and stability conditions are acquired from the stability analysis. These stability conditions can be used as constraints on the design of fuzzy logic control functions. Simulations of 200 agents with double integrator dynamics are given. The three behaviours are evaluated through different performances.

This paper presents a leader-follower flocking system which is consisted of a group of mobile agents with kinematic model. A "position consensus" algorithm is applied to enable the agents to estimate the flocking central position via neighbor-to-neighbor communication. By using the flocking center position, a cohesion controller is developed to enable the mobile agents to move close with each others. To avoid collision, a separation function is designed and fuzzy logic is used to keep this function smooth. The flocking stability is analyzed with LaSalle's invariance principle. Simulations of flocking with 200 agents are given. The flocking system is evaluated through different performances.

This paper investigates target tracking using a distributed particle filter over sensor networks. Gaussian mixture model is adopted to approximate the posterior distribution of weighted particles in this distributed particle filter. The parameters of Gaussian mixture model are exchanged between neighbor sensor nodes. Each node can obtain the Gaussian mixture model representing particle's posterior distribution through the parameter exchange. With the posterior distribution, the distributed particle filter can draw particles from it, predicted particles and observations, update particle weights, and re-sample particles based the predicted weights. The parameter exchange is key to implement the distributed operation. It is implemented by using an average consensus filter. Through this consensus filter, each sensor node can gradually diffuse its local statistics of weighted particles over the entire network and asymptotically obtain the estimated global statistics. The parameters of Gaussian mixture model can be calculated by using the estimated global statistics. Because the average consensus filter only requires that each sensor node communicate with its neighbors, the proposed distributed particle filter is scalable and robust. Simulations of tracking tasks in a sensor network with 100 sensor nodes are given.

This paper presents the design and implementation of a coordination architecture for quadruped walking robots to learn and execute soccer-playing behaviors. A typical hybrid architecture combing reactive behaviors with deliberative reasoning is developed. The reactive behaviors directly map spatial information extracted from sensors into actions. The deliberative reasoning represents temporal constraints of a robot's strategy in terms of finite state machines. In order to achieve real-time and robust control performance in reactive behaviors, fuzzy logic controllers (FLCs) are used to encode the behaviors, and a two-stage learning scheme is adopted to make these FLCs adaptive to complex situations. The experimental results are provided to show the suitability of the architecture and effectiveness of the proposed learning scheme.

This paper presents a hybrid control architecture for autonomous robotic fishes which are able to swim and navigate in unknown or dynamically changing environments. It has a three-layer configuration: cognitive layer, behaviour layer and swim pattern layer. The state-based planning in the cognitive layer provides a good foundation for potential adaptation through machine learning methods such as reinforcement learning (RL). The behaviour layer and the swim pattern layer are specially designed to match the needs for the real-time control of our robotic fish. To test the feasibility and performance of the proposed architecture, the experiment of "tank border exploration" is conducted with Q-learning

This paper presents a fuzzy logic based system for formation control of multiple mobile robots. Two main problems of formation control are investigated - maintaining correct formation position and inter-formation collision avoidance. A leader-follower approach with minimal communication between robots is presented here. Separate fuzzy logic controllers are developed for formation position control and internal collision avoidance with a higher level fuzzy coordinator used to fuse their outputs. A major issue in formation control is the presence of uncertainties in the real world, in the form of noisy sensor data and delay in leader position transmission. Noise is added to the simulated data to prove that the system is capable of tolerating such disturbance. The system has the ability to create any desired geometrical formation shape

A multiagent reinforcement learning algorithm with fuzzy policy is addressed in this paper for dealing with the learning and control issues in cooperative multiagent systems with continuous states and actions, particularly for autonomous robotic formation systems. The parameters of fuzzy policy are finely tuned by the gradient multiagent reinforcement learning algorithm to improve the overall performance of an initial controller (policy). A leader-follower robotic system is chosen as a platform to benchmark the performance of the multiagent fuzzy policy reinforcement learning algorithm. Our simulation results demonstrate that the control performance can be improved in many aspects. This work also can be seen as a scaling up of currently popular multiagent reinforcement learning to the robotic domain with continuous state and action space as well as high dimensionality.

We investigate the leader-following formation control of mobile robots through the model predictive control (MPC) in this paper. We establish its control stability by adding a terminal state penalty to the cost function and a terminal state region to the optimisation constraints. We also design a terminal state region based on an input-output feedback linearisation controller for the MPC. A suboptimal stable solution is sought to reduce the computational time used in the MPC. Simulations on the control are provided to verify the proposed control strategy.

This paper addresses the control of a leader-follower formation where the leader robot has its own target and the follower robots are constrained by the specified formation tasks. The dynamics of the leader robot with nonholonomic constraint is explicitly integrated into the formation system to yield a centralized coordinating controller. As a result there is no need to assume the motion of the leader separatively when we develop cooperative formation controllers for coordinating the robots. The feedback linearization is used to deal with the nonlinear formation control of a team of autonomous mobile robots with nonholonomic constraints. Although the nonlinear formation system under consideration can be exactly linearized by taking advantage of dynamic feedback linearization, there exists structural singularity which may pose serious problems in practice. To solve this singular problem a new formation model for controlling the leader-follower formation in a cooperative manner is developed. This new formation model can be extended to studying other control and learning issues in multi-robot systems for both cooperation and noncooperation. The internal dynamics is derived and proven to be globally stable under the stable linear controller obtained via the partially linearized dynamics. To demonstrate the performance of the developed formation controller, simulation results are provided.

This article presents a hybrid learning architecture for fuzzy control of quadruped walking robots in the RoboCup domain. It combines reactive behaviors with deliberative reasoning to achieve complex goals in uncertain and dynamic environments. To achieve real-time and robust control performance, fuzzy logic controllers (FLCs) are used to encode the behaviors and a two-stage learning scheme is adopted to make these FLCs be adaptive to complex situations. The first stage is called structure learning, in which the rule base of an FLC is generated by a Q-learning scheme. The second stage is called parameter learning, in which the parameters of membership functions in input fuzzy sets are learned by using a real value genetic algorithm. The experimental results are provided to show the suitability of the architecture and effectiveness of the proposed learning scheme. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 131–152, 2005.

This paper presents a receding horizon (RH) controller used for regulating a nonholonomic mobile robot. The RH control stability is guaranteed by adding a terminal-state penalty to the cost function and a terminal-state region to optimization constraints. A suboptimal solution to the optimization problem is sufficient to achieve stability. A new terminal-state penalty and its corresponding terminal-state constraints are found. Implementation and simulation results are provided to verify the proposed control strategy.

This work presents a learning approach to fuzzy classifier systems. Q-learning algorithm is employed to implement credit assignment of the learning. GA operators are used as an action selection mechanism of the learning. The learning approaches can be viewed as a fuzzy learning classifier system or a Q-learning algorithm that adopts fuzzy logic to generalise Q-learning results. Rule accuracies are treated as rule fitness values. The learning algorithm is applied to a control robot behaviour.

Behaviour co-ordination is one of the major problems in behaviour-based robotics. This paper presents a teaching method for mobile robots to learn behaviour coordination. In this method, the sensory information is abstracted into a limited number of feature states that correspond to physical events in the interactive process between a robot and its environment. The continuous motor actions are abstracted into a limited number of behaviours. The goal of the behaviour co-ordination is to map the feature states into the behaviours in the light of environment rewards. The teaching process consists of an imitation stage and an autonomous learning stage. Both stages employ Q-learning algorithms to implement the mapping. The imitation stage serves as a preliminary stage for the teaching method. The learning result is used to bootstrap the autonomous learning stage. Experiments are conducted in the domain of soccer playing by Sony legged robots. Experiment results show that the robots can acquire behaviour coordination ability.

This paper presents a predictive control scheme for mobile robots that possess complexity, non-linearity and uncertainty. A multi-layer back-propagation neural network is employed as a model for nonlinear dynamics of the robot. The control variables are produced by optimizing the performance index on-line using the steepest gradient descent algorithm. The neural network is constructed by the wavelet orthogonal decomposition to form a wavelet neural network that can overcome the problems caused by local minima of optimization. The wavelet network is also helpful to determine the number of the hidden nodes and the initial value of weights. The sparse train data in our path tracking case can reduce the effect of the "curse of dimensionality" on the network size in high dimensional function learning caused by the orthogonal wavelet base function.