The Artificial Internet of Things (AIoT) is growing into a new frontier field with broad development prospects, which essence is the collaborative enhancement of networked heterogeneous agent swarms. The market-based approach is an effective way for the cooperative scheduling of agent swarm, where networked agents need to distributedly select and arrange tasks meeting the spatio-temporal constraints. This paper proposes a new asynchronous consensus method aimed at substantially mitigating the communication traffic and decreasing the message transmission requirements associated with the market-based approach, ultimately leading to a reduction in scheduling time. Firstly, the method innovatively introduces timestamps of agent information updates, which are more informative, thereby reducing inter-agent communication volume to$ n/m$of that in the original protocol (where$ n$represents the number of agents and$ m$denotes the number of tasks, with$ m\gt n$). Secondly, agent-centric asynchronous consensus protocols are designed based on the new timestamps, which can resolve inter-agent task conflicts more rapidly and efficiently. Additionally, a mechanism for avoiding message flooding is proposed to prevent endless broadcasts caused by communication issues such as packet loss, link disruptions, and node withdrawals. Finally, through a self-developed ad-hoc network simulation system, the swarm scheduling under real networking conditions is simulated. The validation results demonstrate that the algorithm can significantly reduce communication traffic and scheduling time.

Automated synthesis of local controllers for multi-agent systems to satisfy complex task specifications has attracted extensive attention. However, it remains unclear how to formally guarantee that their composed behavior satisfies the specified global task. In this paper, we aim to synthesize an automated and distributed control strategy for fleet-wise tasks specified as linear temporal logic formulas, such that the agents act asynchronously and synchronize only on shared actions via local coordination. The proposed method consists of three main steps. First, the set of satisfying global control strategies is computed via parallel composition. Among these strategies the conditions for decomposability are evaluated, based on which the global strategy over a maximum synchronization scheme is found. Then the synchronization scheme is further refined to obtain more efficient local control strategies. It is formally proven that the resulting global behavior along with the synchronization scheme satisfies the specified global task. Also, the synthesized local controllers are reactive to changes in the workspace or in the fleet during online execution, without the need for replanning. Thus, collaborative relations among the agents are adaptive as needed. Numerical simulations and hardware experiments are conducted for nontrivial scenarios.Note to Practitioners—This paper was motivated by the problem of coordinating a fleet of autonomous robots in collaborative search and delivery processes, where local controllers are synthesized for each robot such that the specified global task specification is satisfied. Existing approaches to address such problems often rely on a fully-connected communication topology and fixed collaborative relations among the robots, thus limiting efficiency of the multi-robot execution and yielding difficulty of coordination. Also, most of existing approaches cannot directly deal with the changes in the workspace or in the fleet during task execution unless a task replanning is performed, which leads to longer time of system-wide communication and task completion, yielding failures during online execution. In this paper, we propose a novel distributed control architecture to tackle these issues, where the robots are controlled to operate asynchronously and achieve online local coordination by synchronization on collaborative actions. Moreover, compared with the common solutions, the collaborative relations among the robots are formed and removed dynamically as needed and the robots are robust to uncertainty during task execution. It is formally proven that the resulted global behavior of the robot fleet along with the synchronization scheme is consistent with the specified global task. We have shown that it is particularly useful for complex and coupled multi-robot applications, where the inter-robot collaborations are feasible and local. Experimental results suggest that this approach is applicable to multi-robot systems which greatly improves the concurrency and efficiency of task execution. In the future research, we will draw inspiration from decentralized approaches for task decomposition to alleviate the computational burden as the number of robots increases.

Flocking with multiple unmanned aerial vehicles (UAVs) offers significant potential for diverse applications due to its enhanced maneuverability, improved efficiency, and increased robustness. Collision avoidance is a critical and challenging issue for distributed flocking control with a UAV fleet, especially in dynamic environments with varying numbers of non-cooperative intruders. However, existing reinforcement learning based methods mainly focus on flocking with collision avoidance tasks with static obstacles and a fixed number of UAVs. In this article, we propose a scalable multi-agent reinforcement learning based method to solve the distributed flocking with collision avoidance problem for a scalable fleet of fixed-wing UAVs in dynamic environments. Specifically, we cast this problem in a decentralized partially observable Markov decision process framework and propose a scalable multi-agent reinforcement learning algorithm called spatial-temporal attention multi-agent actor-critic (STAAC). In this algorithm, we design a spatial-temporal attention based population-invariant network architecture to facilitate the representation learning of dynamic dimensional observations. By integrating the local spatial attention and global temporal attention mechanisms, STAAC is able to adapt to the changes in the scale of UAV fleets and the number of intruders. Finally, we empirically demonstrate the effectiveness, scalability, and adaptability of the proposed approach in numerical simulations and hardware-in-the-loop experiments.

Multiagent scheduling has recently been reinvigorated by the burgeoning application of swarm, receiving significant attention due to its new characteristics. The market-based method is a fast distributed scheduling method that is naturally suitable for agent swarm, while its multiround communication is inevitably affected by the environment and the performance deteriorates. This article proposes an idea of computing-for-communication (CFC) with improving or even appropriately increasing computation to reduce communication rounds and improve the performance meanwhile, which does not add additional communication protocols and traffic but may moderately increase the amount of computation and storage. First, a new scoring function and a local optimization method are proposed to improve the agent’s schedule and resolve the conflict among agents in advance. Second, an agent location inference method and task-related agent selection strategy are presented for local optimization, which is expected to avoid the increase of communication in locations and the waste of computation on irrelevant agents. Third, some modifications for removing and adding tasks are proposed to further improve the performance of scheduling. Finally, extensive Monte Carlo experiments demonstrate the commendable performance of the proposed method in comparison with the representative consensus-based bundle algorithm (CBBA) and performance impact algorithm (PI).

Multiple unmanned aerial vehicles (UAVs) are able to efficiently accomplish a variety of tasks in complex scenarios. However, developing a collision-avoiding flocking policy for multiple fixed-wing UAVs is still challenging, especially in obstacle-cluttered environments. In this article, we propose a novel curriculum-based multiagent deep reinforcement learning (MADRL) approach called task-specific curriculum-based MADRL (TSCAL) to learn the decentralized flocking with obstacle avoidance policy for multiple fixed-wing UAVs. The core idea is to decompose the collision-avoiding flocking task into multiple subtasks and progressively increase the number of subtasks to be solved in a staged manner. Meanwhile, TSCAL iteratively alternates between the procedures of online learning and offline transfer. For online learning, we propose a hierarchical recurrent attention multiagent actor-critic (HRAMA) algorithm to learn the policies for the corresponding subtask(s) in each learning stage. For offline transfer, we develop two transfer mechanisms, i.e., model reload and buffer reuse, to transfer knowledge between two neighboring stages. A series of numerical simulations demonstrate the significant advantages of TSCAL in terms of policy optimality, sample efficiency, and learning stability. Finally, the high-fidelity hardware-in-the-loop (HITL) simulation is conducted to verify the adaptability of TSCAL. A video about the numerical and HITL simulations is available at https://youtu.be/R9yLJNYRIqY.

In this paper, predefined-time tracking control of fixed-wing unmanned aerial vehicles (UAV) subject to external disturbance and unmodeled dynamics is studied. First, the nonlinear dynamics of the UAV are converted to the second-order system through feedback linearization. Then, a fixed-time observer is addressed to estimate the disturbance and uncertainty. Finally, a novel predefined-time controller is designed to drive the tracking error to zero within a predefined time. Numerical simulation and comparison demonstrate that the proposed algorithm has a faster convergence rate, and the control input is smoother near the settling time.

In this paper, aimed at improving the cooperative target tracking performance, the optimal cooperative circumnavigation control of multiple robots is studied. To solve the curse of dimensionality in the training process of the optimal cooperative control policy, this paper employs the idea of “Mean Field” and designs a fully distributed optimization algorithm based on the robust optimal control method, which greatly reduces the algorithm complexity and training time. The stability of the system under the designed control policy is analyzed rigorously and the theoretical results are validated by a numerical simulation.

In this article, a novel design scheme is introduced to solve the optimal control problem for nonlinear systems with unsymmetrical and state-dependent input constraints. By introducing an initial stabilizing control policy as the baseline of the constructed optimal control policy, we remove the assumption in the current study for the adaptive optimal control, that is, the internal dynamics should hold zero when the state of the system is in the origin. Particularly, nonlinear control systems with partially unknown dynamics are investigated and the procedure to acquire the corresponding optimal control policy is presented. The stability for the closed-loop dynamics and the optimality of the obtained control policy are both proved. Besides, we apply the proposed control design framework to solve the optimal circumnavigation problem based on the accumulative Fisher information for a fixed-wing unmanned aerial vehicle (UAV). The control performance of our algorithm is compared with that of the existing circumnavigation control policy in a numerical simulation.

A swarm of fixed-wing unmanned aerial vehicles (UAVs) is expected to efficiently accomplish various tasks in complex scenarios. This paper proposes an attention-based population-invariant multi-agent deep reinforcement learning (MADRL) approach to deal with the decentralized collision-free flocking problem for a scalable fixed-wing UAV swarm. First, this problem is modeled as a decentralized partially observable Markov decision process from the perspective of each follower. Then, an improved multi-agent deep deterministic policy gradient (MADDPG) algorithm is presented to efficiently learn the population-invariant flocking policy. In this algorithm, the parameter sharing with ego-centric representation mechanism is incorporated to improve learning efficiency. Besides, the attention-based population-invariant network structure (APINet) is designed by leveraging the self-attention mechanism. With this structure, the learned flocking policy is invariant to the population of the swarm. Finally, both numerical and hardware-in-the-loop simulation results verify the efficiency and scalability of the proposed approach.

This article investigates the problem of coordinated path following for fixed-wing unmanned aerial vehicles (UAVs) with speed constraints in the two-dimensional plane. The objective is to steer a fleet of UAVs along the path(s) while achieving the desired sequenced inter-UAV arc distance. In contrast to the previous coordinated path-following studies, we are able through our proposed hybrid control law to deal with the forward speed and the angular speed constraints of fixed-wing UAVs. More specifically, the hybrid control law makes all the UAVs work at two different levels: 1) those UAVs whose path-following errors are within an invariant set (i.e., the designed coordination set) work at the coordination level and 2) the other UAVs work at the single-agent level. At the coordination level, we prove that even with speed constraints, the proposed control law can make sure the path-following errors reduce to zero, while the inter-UAV arc distances converge to the desired value. At the single-agent level, analysis for the path-following error entering the coordination set is provided. We develop a hardware-in-the-loop simulation testbed of the multi-UAV system by using actual autopilots and the X-Plane simulator. The effectiveness of the proposed approach is corroborated with both numerical simulation and the testbed.

In this article, a multilayered and distributed architecture for mission-oriented miniature fixed-wing UAV swarms is presented. Based on the concept of modularity, the proposed architecture divides the overall system into five layers: 1) low-level control layer; 2) high-level control layer; 3) coordination layer; 4) communication layer; and 5) human interaction layer, and many modules that can be viewed as black boxes with interfaces of inputs and outputs. In this way, not only the complexity of developing a large system can be reduced but also the versatility of supporting diversified missions can be ensured. Furthermore, the proposed architecture is fully distributed that each UAV performs the decision-making procedure autonomously so as to achieve better scalability. Moreover, different kinds of aerial platforms can be feasibly extended by using the control allocation matrices and the integrated hardware box. A prototype swarm system based on the proposed architecture is built and the proposed architecture is evaluated through field experiments with a scale of 21 fixed-wing UAVs. Particularly, to the best of our knowledge, this article is the first work which successfully demonstrates formation flight, target recognition, and tracking missions within an integrated architecture for fixed-wing UAV swarms through field experiments.

In this paper, a curved path following scheme with the aid of the vector field (VF) and the notion of input-to-state stable (ISS) for a fixed-wing unmanned aerial vehicle (UAV) is developed. The VF strategy is a robust and valid guidance method and its stability is proved using ISS properties. Many existing path following algorithms for fixed-wing UAVs are only proposed for straight-lines and orbits. However, the path required to be followed is always in approximate curves rather than straight-lines and orbits in many high-level missions, such as obstacle avoidance, search, and surveillance. The nonlinear-theoretic notion of ISS is playing a central role in the control law design and stability analysis. The error kinematics are converted into two interconnected subsystems with proven ISS properties, which yield the overall system that is globally asymptotically stable, i.e., and the along-track error and the cross-track error asymptotically approach zeros from any initial position in the space. The followed path is defined in terms of the arc-length parameter, and it can be expanded according to the waypoint fitting without the need to obtain a global function representation. The singularity of multiple closest points on the path is eliminated by constructing a speed profile of a virtual point on the path. The scheme is validated with a semi-physical experiment combined by an actual autopilot, ground station and the X-Plane flight simulator. Flight tests using a small fixed-wing UAV show excellent tracking performance of the curved path following.

This paper presents an onboard micro aerial vehicle (MAV) localization algorithm VLO using onboard multi-sensor system consisting of a camera, a laser scanner and an inertial measurement unit. On the basis of onboard processor, the VLO can operate in real time without any prior information and ground assistance. Besides, it shows a strong robustness since it can work in both small and large, indoor and outdoor environment. As the main sensing devices of this system, the camera and laser scanner generate different characteristic data. The VLO fuses these two kinds of data for a more sufficient information about environment. A filter and an optimizer are then designed to estimate the MAV poses with extra onboard sensors data. Finally, an incremental dense map is updated. Different with vision-based or laser-based odometry, this system has no requirements for environments such as strong texture or structured surroundings. The Gazebo-based simulated and real MAV systems are built together for algorithm validation. The simulated and real results show that our onboard odometry VLO performs strong robustness without any prior information and basic assumptions.

This paper presents a study on short-term cooperative conflict detection and resolution (CDR) of unmanned aerial vehicles (UAVs). A two-layered CDR mechanism is proposed, which aims at guaranteeing safe separation, minimizing the overall cost of UAVs, and improving computational efficiency. In the first layer, the information from the environment is processed. In the second layer, conflicts among UAVs are resolved by applying the local centralized optimization method, with consideration given to the dynamic constraints of UAVs. This paper studies the safe separation constraints of pairwise conflicts in virtue of a geometry-based method. A heading change and speed change mixed conflict resolution approach is applied. To meet with the online planning requirements, the vectorized stochastic parallel gradient descent-based method is proposed to find the local optimal heading change solutions. The linear safe separation constraints on speeds are derived. A periodicity feature-based method is used to depart the feasible sub-regions for each pairwise conflict. A mixed integer linear programming model is established to find the optimal speed change solutions. The experiments results show that the proposed heading change algorithm could greatly reduce the summation of additional flight distances of UAVs, and influences on air traffic, compared with other short-term algorithms; the computational efficiency of this algorithm satisfies the online planning requirement. Comparing with the existing algorithm, our speed change algorithm reduces the number of feasible sub-regions to 2nctimes lower, where ncis the number of pairwise conflict, and therefore, it reduces computation time dramatically.

We investigate a decentralized patrolling problem for dynamic environments where information is distributed alongside threats. In this problem, agents obtain information at a location, but may suffer attacks from the threat at that location. In a decentralized fashion, each agent patrols in a designated area of the environment and interacts with a limited number of agents. Therefore, the goal of these agents is to coordinate to gather as much information as possible while limiting the damage incurred. Hence, we model this class of problem as a transition-decoupled partially observable Markov decision process with health constraints. Furthermore, we propose scalable decentralized online algorithms based on Monte Carlo tree search and a factored belief vector. We empirically evaluate our algorithms on decentralized patrolling problems and benchmark them against the state-of-the-art online planning solver. The results show that our approach outperforms the state-of-the-art by more than 56% for six agents patrolling problems and can scale up to 24 agents in reasonable time.

This paper presents a novel method-continuous-time Markov decision process (CTMDP)-to address the uncertainties in pursuit-evasion problem. The primary difference between the CTMDP and the Markov decision process (MDP) is that the former takes into account the influence of the transition time between the states. The policy iteration method-based potential performance for solving the CTMDP and its convergence are also presented. The results obtained by MDP-based method demonstrate that it is a special case of CTMDP-based method involving the identity transition rate matrix. To compare the methods, a well-known pursuit-evasion problem, involving two identical cars, is solved as a benchmark. The CTMDP-based method can provide a discretization solution that is close to the analytical solution obtained by the differential game method. Besides, it shows strong robustness against changes in the transition probability, as compared with the traditional MDP-based method. To the best of our knowledge, this is the first attempt to validate the influence of the transition time between the states in such a pursuit-evasion scenario, or in a similar application, solved by an MDP-related model. The CTMDP-based method offers a new approach to solving the pursuit-evasion problem and can be extended to similar optimization applications.

Switching behavior among different swimming modes of fish is a normal phenomenon in nature. We find that there is the joints' vibration fact for robotic fish in the process of switching. It is believed that the difference might be caused by discontinuous driven signal. This paper analyzes the discontinuous signal's the effect on the robotic fish joints and fin surface. Furthermore, this paper proposes an effective phase compensation method based on sinusoidal model to solve the discontinuous problem that enables robotic fish to mimic this spontaneous mode switching behavior of live fish. Finally, experimental results illustrate that the phase compensation method shows the superiority in saving energy and reducing the complexity of the control tracking algorithm.

The Pan-Tilt Unit (PTTJ) and camera composed a vision system, which can be used in vision measurement. In this paper, a new calibration method for this vision system using differential GPS (global positioning system) is described. The calibration method is an efficient solution for large field of view. The proposed method has advantages in speed, convenience and pertinence. The experimental results show the accuracy is acceptable for the outdoor environment.

This paper presents a new robust and stable controller for attitude of quadrotor. We proposed a combination of feedback linearization and LQR (linear quadratic regulator) control strategy. To our best acknowledge, this method is firstly employed to stabilize the attitude of the quadrotor with disturbance. Numerical simulations demonstrate the stabilizations and robustness of the control system under nominal conditions. Furthermore, a bounded disturbance was added to the system in flight test, and the results also illustrate the successful behavior of the controller proposed.

In this study, this paper focus on the vision-based autonomous helicopter unmanned aerial vehicle (UAV) landing problems. This paper proposed a multisensory fusion to autonomous landing of an UAV. The systems include an infrared camera, an Ultra-wideband radar that measure distance between UAV and Ground-Based system, an PAN-Tilt Unit (PTU). In order to identify all weather UAV targets, we use infrared cameras. To reduce the complexity of the stereovision or one-cameral calculating the target of three-dimensional coordinates, using the ultra-wideband radar distance module provides visual depth information, real-time Image-PTU tracking UAV and calculate the UAV threedimensional coordinates. Compared to the DGPS, the test results show that the paper is effectiveness and robustness.

This paper presents a new ground-based visual approach for guidance and safe landing of an unmanned aerial vehicle (UAV) in Global Navigation Satellite System(GNSS)-denied environments. In our previous work, the old system consists of one pan-tilt unit(PTU) with two cameras, whose detection range is limited by the baseline. To achieve long-range detection and cover wide field of regard, we mounted two separate sets of PTU integrated with visible light camera on both sides of the runway instead of our previous assembled stereo vision system. Then, the well-known AdaBoost method was evaluated with regard to detecting and tracking the target. To achieve the relative position between the UAV and landing area, we used triangulation to calculate the 3D coordinates of the UAV. By combining the estimated position in the closed loop control, we obtain the autonomous landing strategy. Finally, we present several real flights in outdoor environments, and compare its accuracy with ground truth provided by GNSS. The results support the validity and accuracy of the presented system.

Visual guidance has attracted more and more attention in the navigation field thanks to its accuracy and robustness. This paper presents a ground-based visual guidance system for the autonomous Unmanned Aerial Vehicles (UAV) landing. The system consists of two cameras and pan-tilt units (PTU) that mounted on both sides of the runway. In this system, computer vision is adopted for UAV detection and tracking. To be more specific, triangulation, a geometric method in binocular vision, is employed to calculate the 3D coordinates of the UAV in order to provide landing guidance parameters and finally achieve autonomous UAV landing. The 3D positioning principles adopted in ground-based measurement are simulated and verified. The results show that the accuracy can be achieved and relevant requirements are satisfied by ground-based visual guidance.

In this paper, biomimetic undulating fins are considered with the focus on their hysteresis nonlinearity. Hysteresis is confirmed with experimental data on the biorobotic fin prototype, and qualitative modeling on this nonlinear action is then achieved by using Preisach equations. The developed iterative learning control is applied to eliminate hysteresis nonlinearity in biorobotic undulating fins. Both the simulation and experimental results show that the proposed control method is effective and feasible to improve the tracking performance of biorobotic fins by considering the hysteresis effect. Furthermore, the control methods should facilitate biomimetic investigation on propulsive modes and waveforms of fish swimming.

Hysteresis characteristics is highly nonlinear, has memory and is common in engineering systems. Its presence introduces uncertainties and nonlinearity in dynamic modelling and thus difficulties in achieving a good control design. This paper studies the suitability of iterative learning control (ILC) to compensate hysteresis uncertainties for a class of continuous-time dynamic systems. We examine dynamic systems with Preisach model hysteresis nonlinearity. It is shown that this class of systems possess properties of continuity and repeatability which are required for ILC. Furthermore, anticipatory iterative learning control (or A-type ILC) is applied to overcome the uncertainties and nonlinearity introduced by hysteresis. Simulation results are presented to validate the effectiveness of ILC laws to eliminate tracking error due to hysteresis uncertainties.

This paper proposes a novel conceptual underwater bio-robot inspired by Rhinecanthus aculeatus, which belongs to median and/or paired fin (MPF) propulsion fish and impresses researchers with agility by cooperative undulation of the dorsal-and-anal fins. Such a fish-like robot is anticipated to outperform the conventional aquatic robots in maneuverability and stability for oceanic exploitation necessities, e.g. benthonic mineral exploration. To begin with, a specimen of R. aculeatus was filmed in a glass aquarium (150cm times 50cm times 60cm) in which artificial seawater was maintained at 26 degC or so. Afterwards, we analyzed a few characteristics in morphology and locomotion with image processing and other approaches. The morphological and kinematical bionic inspirations were summarized, and in succession, we elaborately delineated the design scheme of our conceptual robotic fish including the schematic architectures, the structural and outside form, and the undulatory multi-fin propulsor

Within median and/or paired fin (MPF) propulsion, many fish routinely use the long-based undulatory fins as the sole means of locomotion. In this paper, the long-based undulatory fin of an Amiiform fish "G. niloticus" was investigated. A simplified physical model was brought forward, which makes up of N equal thin rods and a rectangular elasticmembrane connecting them together. With light mass, small stiffness and low natural vibration frequency, the long-based fin undulating and fluid loading may induce large flexible distortion of their long-based fins when swimming. We established a kinematic model of the long-based undulatory fin on the basis of analyzing the long-based dorsal fin locomotion and considering the fluid-structure interaction. Further, the equilibrium equations of the undulatory fin were obtained by applying the membrane theory of thin shells in which the geometrical non-linearity of the structure is taken into account. Last, we apply the derived the kinematic model and equilibrium equations of the undulatory fin to analyze the thrust and propulsive efficiency varying with the aspect ratio of the fin and the maximum swing amplitude