Mobile robots are increasingly playing a pivotal role in various fields, including environmental monitoring and search-and-rescue tasks. The ultimate development goal for robots is to achieve embodied artificial intelligence (embodied AI), which enables continuous learning and evolution through interactions with the environment. Motion planning and control are fundamental for robots to interact with environments and accomplish complex tasks effectively. With the rapid advancement of AI technologies, machine learning (ML) algorithms have been successfully applied to various robotic tasks. This survey provides a comprehensive overview and classification of learning-based motion planning and control approaches, which can help mobile robots achieve embodied AI. First, the existing approaches are categorized into five system architectures based on the modules applied by the ML algorithms. Second, a detailed review is provided on how reinforcement learning (RL), imitation learning, and the integration of ML with model predictive control (ML-MPC) techniques are applied across the different architectures. Additionally, the current state of research on several critical issues in embodied AI is presented, including safe learning control, Sim2Real transfer, and large language models (LLMs). Finally, the survey highlights some challenges in realizing embodied AI in real-world scenarios and suggests potential directions for future research. Unlike existing reviews that focus on specific tasks or categorize related work by ML algorithms, this survey constructs a taxonomy of existing research from a system architecture perspective. It emphasizes key technologies that facilitate real-world applications. We hope that our work will contribute to the advancement of embodied AI in the field of mobile robots.

Scene graph generation (SGG) is a structured approach to understanding real-world scenes with complex relations, which can enhance UAV autonomy in unfamiliar environments. However, SGG typically has numerous model parameters that require considerable computational resources. This paper proposes a refined SGG model and reduces the model parameters for its UAV applications. First, we use subject-object query pairs to predict triplets directly, eliminating the need for separate entity predictions. Additionally, the cross-attention mechanism enhances the model’s ability to query triplets. We use a single decoder to process subject and object entities simultaneously, enhancing computational speed and reducing the number of parameters. Then, we map the entities to the relational semantic space before performing relations classification, which improves the model performance by adding a small number of parameters. Finally, the set prediction loss function is designed for relation prediction to strengthen the role of relation prediction in triplets. Real-world UAV experiments show that our model can extract more triplets per second with fewer parameters than the benchmarks. Github: https://github.com/SupersPig/myLGTR.

Accurate dynamics modeling is essential for quadrotors to achieve precise trajectory tracking in various applications. Traditional physical knowledge-driven modeling methods face substantial limitations in unknown environments characterized by variable payloads, wind disturbances, and external perturbations. On the other hand, data-driven modeling methods suffer from poor generalization when handling outof-distribution (OoD) data, restricting their effectiveness in unknown scenarios. To address these challenges, we introduce the Physics-Informed Wind-Adaptive Network (PI-WAN), which combines knowledge-driven and data-driven modeling methods by embedding physical constraints directly into the training process for robust quadrotor dynamics learning. Specifically, PI-WAN employs a Temporal Convolutional Network (TCN) architecture that efficiently captures temporal dependencies from historical flight data, while a physics-informed loss function applies physical principles to improve model generalization and robustness across previously unseen conditions. By incorporating real-time prediction results into a model predictive control (MPC) framework, we achieve improvements in closed-loop tracking performance. Comprehensive simulations and real-world flight experiments demonstrate that our approach outperforms baseline methods in terms of prediction accuracy, tracking precision, and robustness to unknown environments.

Distributed task allocation in the UAV swarm is sensitive to excessive communication overhead and frequent transmissions. Combining reinforcement learning and task allocation demonstrates great potential in enhancing algorithm performance and optimizing communication. However, existing studies rely on ideal communication assumptions and the nonphysical environment, making training and validation impractical in applying networked swarms. This paper proposes the Communication-Aware Task Allocation, which aims to train a gating mechanism policy to coordinate the transmission timing, improving robustness and timelessness of the task allocation. First, the policy learning problem is formalized as a POMDP, for which the channel access and other features are designed for observations, actions are inter-agent adaptive gating mechanisms, and the shared reward reflects global task conflicts. Second, to address the asynchronous learning under the CTDE, an asynchronous experience collection and splicing method is proposed to align trajectories. Then, the MOCPPO is proposed, which combines a primal-dual operator with proximal policy optimization, updating the optimal Lagrange multiplier and strategy parameters to simultaneously minimize task conflicts and communication overhead. Finally, sim-to-real experiments are conducted in the HIL environment, and results illustrate the best trade-off optimization of the proposed method over all state-of-the-art approaches.

The proliferation of Internet users has accelerated the spread of fake news on social media, necessitating effective fake news detection systems. However, existing methods often focus solely on the features of claims without considering their uncertainty, limiting their reliability and generalizability. Inspired by Bayesian neural networks, this paper proposes a Bayesian Transformer-based Fake news Detection system with Evidence awareness (BTFDE). To validate the effectiveness of BTFDE, we conducted experiments on several datasets. The results demonstrate that BTFDE outperforms several baseline methods, improving the reliability of fake news detection by quantifying uncertainties and incorporating evidence awareness. This approach improves the generalizability of the model and provides a more rational basis for fake news detection.

In modern aviation and logistics, the professional competence and operational skills of unmanned aerial vehicle (UAV) operators are critical to mission success and overall efficiency. Accordingly, effective UAV training, which ensures high levels of involvement, is paramount in enhancing operators’ learning and operational performance. To address the lack of theoretical research on electroencephalography (EEG)-based involvement prediction for UAV training, this study proposes EEG-IvNet, a deep learning framework designed to predict involvement from EEG signals. The model integrates convolutional neural networks (CNNs), long short-term memory (LSTM) networks, and an attention mechanism to classify involvement states. EEG data were collected from participants performing UAV monitoring tasks in controlled experiments and used as input to the model. The results demonstrate that EEG-IvNet outperformed the benchmark CNN+LSTM model in classifying involvement states. Furthermore, this study offers insights into the neural basis of involvement, providing strong theoretical and algorithmic support for incorporating VR technology in UAV training. These findings not only suggest potential improvements in training outcomes but also highlight broader applicability in intelligent aviation, virtual education, and other related domains.

Complex environments pose significant challenges to the consensus estimation of ground targets by multiple autonomous aerial vehicles (multi-AAVs) with limited sensing capabilities. This paper addresses the design of an$H_{\infty } $consensus fuzzy filter over a finite-time horizon, that is subject to selective network attacks and stochastic incomplete measurements. First, a novel selective-data-based (SDB) network attack model is proposed. Unlike conventional models, this model is constructed from the attacker’s perspective to mimic attacks that target high-value data, thereby maximizing its destructive potential. Second, incomplete measurements, arising from factors such as limited AAV sensing ranges and target motion, are modeled by using a set of random variables to characterize the stochastic nature of data loss. Furthermore, an$H_{\infty } $consensus fuzzy filter is developed to achieve precise consensus estimation of the target with finite-time performance, thereby forming a unified attack-defense architecture. Sufficient conditions for the existence of such a filter are established in the form of linear matrix inequalities (LMIs), from which the filter gains can be derived. Finally, the effectiveness and superiority of the proposed design are validated through both numerical simulations and physical experiments.

Flocking with multiple unmanned aerial vehicles (UAVs) offers significant potential for diverse applications due to its enhanced maneuverability, improved efficiency, and increased robustness. Collision avoidance is a critical and challenging issue for distributed flocking control with a UAV fleet, especially in dynamic environments with varying numbers of non-cooperative intruders. However, existing reinforcement learning based methods mainly focus on flocking with collision avoidance tasks with static obstacles and a fixed number of UAVs. In this article, we propose a scalable multi-agent reinforcement learning based method to solve the distributed flocking with collision avoidance problem for a scalable fleet of fixed-wing UAVs in dynamic environments. Specifically, we cast this problem in a decentralized partially observable Markov decision process framework and propose a scalable multi-agent reinforcement learning algorithm called spatial-temporal attention multi-agent actor-critic (STAAC). In this algorithm, we design a spatial-temporal attention based population-invariant network architecture to facilitate the representation learning of dynamic dimensional observations. By integrating the local spatial attention and global temporal attention mechanisms, STAAC is able to adapt to the changes in the scale of UAV fleets and the number of intruders. Finally, we empirically demonstrate the effectiveness, scalability, and adaptability of the proposed approach in numerical simulations and hardware-in-the-loop experiments.

In recent years, Deep Reinforcement Learning (DRL) has been a key approach to solving Unmanned Aerial Vehicle (UAV) swarm path planning problems. However, traditional DRL methods often face challenges in the initial learning stage and struggle to learn from variable step-size tasks. This paper introduces a novel training framework for large-scale UAV swarm variable step-size path planning: Rapidly-exploring Variable Step-size Deep Reinforcement Learning (RVSDRL). This framework involves common training on the ground local server and decentralized training on distributed UAVs. In the common training stage, we generate rapidly-exploring random graph samples to accelerate the common agent explore environment. In the decentralized training stages, we utilize the priority replay mechanism to improve efficiency. To enhance convergence stability, we restrict the returns of the equivalent paths and propose the Step-size Consistent Markov Decision Process (SCMDP) path planning model. Our method is compared with traditional methods, and the experiments demonstrate its superior performance in complex obstacle environments.

Multiple unmanned aerial vehicles (UAVs) are able to efficiently accomplish a variety of tasks in complex scenarios. However, developing a collision-avoiding flocking policy for multiple fixed-wing UAVs is still challenging, especially in obstacle-cluttered environments. In this article, we propose a novel curriculum-based multiagent deep reinforcement learning (MADRL) approach called task-specific curriculum-based MADRL (TSCAL) to learn the decentralized flocking with obstacle avoidance policy for multiple fixed-wing UAVs. The core idea is to decompose the collision-avoiding flocking task into multiple subtasks and progressively increase the number of subtasks to be solved in a staged manner. Meanwhile, TSCAL iteratively alternates between the procedures of online learning and offline transfer. For online learning, we propose a hierarchical recurrent attention multiagent actor-critic (HRAMA) algorithm to learn the policies for the corresponding subtask(s) in each learning stage. For offline transfer, we develop two transfer mechanisms, i.e., model reload and buffer reuse, to transfer knowledge between two neighboring stages. A series of numerical simulations demonstrate the significant advantages of TSCAL in terms of policy optimality, sample efficiency, and learning stability. Finally, the high-fidelity hardware-in-the-loop (HITL) simulation is conducted to verify the adaptability of TSCAL. A video about the numerical and HITL simulations is available at https://youtu.be/R9yLJNYRIqY.

The evolution of artificial intelligence and Internet of Things (IoT) envision a highly integrated artificial IoT (AIoT) network. Flocking and cooperation with multiple unmanned aerial vehicles (UAVs) are expected to play a vital role in industrial AIoT networks. In this article, we formulate the collision-free flocking problem of fixed-wing UAVs as a Markov decision process and solve it in the deep reinforcement learning (DRL) framework. Our method can deal with a variable number of followers by encoding the dynamic environmental state into a fixed-length embedding tensor. Specifically, each follower constructs a fixed-size local situation map that describes the collision risks with other followers nearby. The local situation maps are used by a proposed DRL algorithm to learn the collision-free flocking behavior. To further improve the learning efficiency, we design a reference-point-based action selection strategy and an adaptive mechanism. We compare the proposed MA2D3QN algorithm with several benchmark DRL algorithms through numerical simulation, and we verify its advantages in learning efficiency and performance. Finally, we demonstrate the scalability and adaptability of MA2D3QN in a semiphysical simulation experiment.

Developing the flocking behavior for a dynamic squad of fixed-wing UAVs is still a challenge due to kinematic complexity and environmental uncertainty. In this paper, we deal with the decentralized flocking and collision avoidance problem through deep reinforcement learning (DRL). Specifically, we formulate a decentralized DRL-based decision making framework from the perspective of every follower, where a collision avoidance mechanism is integrated into the flocking controller. Then, we propose a novel reinforcement learning algorithm PS-CACER for training a shared control policy for all the followers. Besides, we design a plug-n-play embedding module based on convolutional neural networks and the attention mechanism. As a result, the variable-length system state can be encoded into a fixed-length embedding vector, which makes the learned DRL policy independent with the number and the order of followers. Finally, numerical simulation results demonstrate the effectiveness of the proposed method, and the learned policies can be directly transferred to semi-physical simulation without any parameter finetuning.

An affordance is a relation between an object, an action, and the effect of that action in a given environmental context. One key benefit of the concept of affordance is that it provides information about the consequence of an action which can be stored and reused in a range of tasks that a robot needs to learn and perform. In this paper, we address the challenge of the on-line learning and use of affordances simultaneously while performing goal-directed tasks. This requires efficient online performance to ensure the robot is able to achieve its goal fast. By providing conceptual knowledge of action possibilities and desired effects, we show that a humanoid robot NAO can learn and use affordances in two different task settings. We demonstrate the effectiveness of this approach by integrating affordances into an Extended Classifier System for learning general rules in a reinforcement learning framework. Our experimental results show significant speedups in learning how a robot solves a given task.

We present a modified version of Extended Classifier System (XCS) on a humanoid NAO robot. The robot is capable of learning a complete, accurate, and maximally general map of an environment through evolutionary search and reinforcement learning. The standard alternation between explore and exploit trials is revised so that the robot relearns only when necessary. This modification makes the learning more effective and provides the XCS with external memory to evaluate the environmental change. Furthermore, it overcomes the drawbacks of learning rate settings in traditional XCS. A simple object seeking task is presented which demonstrates the desirable adaptivity of LCS for a sequential task on a real robot in dynamic environments.