Mir Feroskhan

dblp:271/3079 · DBLP profile ↗
← Back
25ranked-venue papers
0as first author
25since 2021 · last 2026
0000-0002-2889-7222ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Artificial intelligence and machine learning · 12 · 12 since 2021Applied, interdisciplinary, general and emerging computing · 10 · 10 since 2021Human-computer interaction and ubiquitous computing · 7 · 7 since 2021Systems, architecture and hardware · 3 · 3 since 2021Graphics, computer vision, multimedia, augmented reality and games · 2 · 2 since 2021Computer networks · 1 · 1 since 2021Databases, data management, data science and information retrieval · 1 · 1 since 2021
YearPublicationVenuePosition
2026 TRUST-UP: Trustworthy reinforcement learning using safe techniques for UAV pursuit
Yaosheng Deng, Mengtao Lyu, Jiaping Xiao, Mir Feroskhan
Adv. Eng. Informatics5
2026 Physics-informed and latent-conditioned Fourier neural operators for vector-to-spatial mapping in quadrotor crash area prediction
Anush Kumar Sivakumar, Mir Feroskhan
Eng. Appl. Artif. Intell.2
2026 ParcelDrone: A modular and graph neural network-based decentralized approach to aerial parcel delivery
Jun Kiat Tan, Archit Krishna Kamath, Peng Shi 0001, Mir Feroskhan
Eng. Appl. Artif. Intell.4
2026 Physics-Informed Koopman Neural Operator for Augmented Dynamics Visual Servoing of Multirotors
abstract
This paper introduces a Physics-Informed Koopman Neural Operator (PI-KNO) for augmented dynamics visual servoing of multirotors that integrates Koopman operator theory with neural networks. The proposed method establishes a structured learning framework that effectively captures complex system dynamics while embedding physics-based priors. Unlike fully data-driven approaches, PI-KNO improves generalization and minimizes reliance on extensive real-world training data by employing a hybrid loss function that combines physics-informed constraints with real-time observations. The learned model is incorporated into a monotonically weighted nonlinear model predictive control (NMPC) framework, ensuring precise trajectory tracking while adhering to state and input constraints. Experimental results demonstrate that PI-KNO reduces training time by 16.81% and enhances tracking accuracy by 19.56% compared to conventional Data-Driven Koopman Neural Operators (DD-KNO) and Physics-Informed Neural Networks (PINN). Additionally, under 70.83% uncertainty in camera parameters, PI-KNO achieves 10.6% and 22.2% lower tracking errors than PINN and DD-KNO, respectively. These findings underscore the robustness and efficiency of the proposed approach for real-time multirotor visual servoing applications. Note to Practitioners - This work presents an implementation oriented view of a learning based predictive controller for aerial robots designed to operate within the limits of onboard computation and sensing. The proposed PI-KNO approach combines physical modeling and data driven learning to create an accurate and stable dynamic predictor suitable for real time deployment. The architecture is distributed between an NVIDIA Jetson and a Pixhawk. The Jetson runs the vision front end, state estimator, PI-KNO predictor, and NMPC optimizer. The estimator fuses visual and inertial data to provide position, velocity, and attitude states, while the NMPC computes high level motion commands at about 30Hz using the PI-KNO rollout. The Pixhawk executes inner rate loops at 250Hz through the native autopilot for attitude stabilization and motor mixing. Communication between the Jetson and Pixhawk uses standard MAVLink setpoints, requiring no firmware modification. The training workflow begins with a nominal visual servoing multirotor model and limited flight data, from which the operator is trained offline using a physics-informed regularizer. At runtime, the estimator updates the state, PI-KNO predicts short horizon dynamics, the NMPC optimizes control sequences, and the Pixhawk executes the commands. This setup achieves reliable tracking across trajectories, maintains physical consistency under uncertainty, and fits within the computational limits of embedded hardware.
Archit Krishna Kamath, Bing Yan 0001, Peng Shi 0001, Mir Feroskhan
IEEE Trans Autom. Sci. Eng.4
2026 Physics-Embedded Networks: Improving Convergence and Precision of Physics-Informed Neural Networks for Real-Time Applications
abstract
This article introduces the physics-embedded neural network (PENN), an enhanced physics-informed neural network (PINN) architecture tailored for visual servoing applications of multirotors. Classical PINNs, while interpretable and data-efficient due to their incorporation of physical laws in the training loss function, often suffer from poor convergence and sensitivity to network initialization and activation functions (AFs). To overcome these challenges, this work proposes two improved architectures: the layer-wise PENN (L-PENN) and the neuron-wise PENN (N-PENN). These architectures embed nominal physical dynamics directly into the structure of the network, thereby improving both training efficiency and predictive accuracy. A spectral analysis of the Hessian matrix is conducted to rigorously demonstrate the enhanced convergence behavior of the proposed architectures compared to traditional PINNs. The proposed methods are experimentally validated on a visual servoing task using a multirotor platform, with performance evaluated in terms of tracking performance and training time. The results are also benchmarked against existing literature, confirming that both L-PENN and N-PENN significantly outperform classical PINNs and other learning-based control strategies. The article concludes by outlining selection criteria for choosing between the two architectures based on specific characteristics of the application.
Archit Krishna Kamath, Mir Feroskhan
IEEE Trans. Cybern.2
2025 SaViD: Spectravista Aesthetic Vision Integration for Robust and Discerning 3D Object Detection in Challenging Environments
abstract
The fusion of LiDAR and camera sensors has demonstrated significant effectiveness in achieving accurate detection for short-range tasks in autonomous driving. However, this fusion approach could face challenges when dealing with long-range detection scenarios due to disparity between sparsity of LiDAR and high-resolution camera data. Moreover, sensor corruption introduces complexities that affect the ability to maintain robustness, despite the growing adoption of sensor fusion in this domain. We present SaViD, a novel framework comprised of a three-stage fusion alignment mechanism designed to address long-range detection challenges in the presence of natural corruption. The SaViD framework consists of three key elements: the Global Memory Attention Network (GMAN), which enhances the extraction of image features through offering a deeper understanding of global patterns; the Attentional Sparse Memory Network (ASMN), which enhances the inte-gration of LiDAR and image features; and the KNNnectivity Graph Fusion (KGF), which enables the entire fusion of spatial information. SaViD achieves superior performance on the long-range detection Argoverse-2 (AV2) dataset with a performance improvement of 9.87% in AP value and an improvement of 2.39% in mAPH for L2 difficulties on the Waymo Open dataset (WOD). Comprehensive experiments are carried out to showcase its robustness against 14 natural sensor corruptions. SaViD exhibits a robust performance improvement of 31.43% for AV2 and 16.13% for WOD in RCE value compared to other existing fusion-based methods while considering all the corruptions for both datasets. Our code is available at SaVil).
Tanmoy Dam, Sanjay Bhargav Dharavath, Sameer Alam, Nimrod Lilith, Aniruddha Maiti, Supriyo Chakraborty, Mir Feroskhan
ICRA7
2025 Dragonfly Drone: A Novel Tilt-Rotor Aerial Platform with Body-Morphing Capability
abstract
The development of unmanned aerial vehicles (UAVs) with extended maneuverability has unlocked new applications such as complex inspection tasks at height. In this work, we introduce the Dragonfly drone, a novel tilt-rotor body-morphing UAV, capable of altering its shape and orientation without compromising its position tracking. Unlike most existing UAV designs that only target at decoupling position and orientation control, Dragonfly can also perform unique body-morphing in flight, featuring all six degrees of freedom in every morphology. This enables navigation into tight gaps with irregular shapes, conforming to obstacles of varying geometries, and maintaining physical contact with uneven surfaces. Such capabilities make our design particularly effective for complex inspection tasks at height, such as pipe or bridge inspection. Our contributions include the mechanical design of the system, the modeling and control strategies employed, and the realrobot experiments with a prototype platform. See Dragonfly drone in action: https://youtu.be/YxoV_Qt_5XE.
Syed Waqar Hameed, Alex Liew Jun Jie, Nursultan Imanberdiyev, Efe Camci, Weiyun Yau, Mir Feroskhan
ICRA6
2025 SpaceDet: A Large-scale Space-based Image Dataset and RSO Detection for Space Situational Awareness
abstract
Space situational awareness (SSA) plays an imperative role in maintaining safe space operations, especially given the increasingly congested space traffic around the Earth. Space-based SSA offers a flexible and lightweight solution compared to traditional ground-based SSA. With advanced machine learning approaches, space-based SSA can extract features from high-resolution images in space to detect and track resident space objects (RSOs). However, existing spacecraft image datasets, such as SPARK, fall short of providing realistic camera observations, rendering the derived algorithms unsuitable for real SSA systems. In this work, we introduce SpaceDet, a large-scale realistic space-based image dataset for SSA. We consider accurate space orbit dynamics and a physical camera model with various noise distributions, generating images at the photon level. To extend the available observation window, four overlapping cameras are simulated with a fixed rotation angle. SpaceDet includes images of RSOs observed from 19 km to 63,000 km, captured by a tracker operating in LEO, MEO, and GEO orbits over a period of 5,000 seconds. Each image has a resolution of 4418 x 4418 pixels, providing detailed features for developing advanced SSA approaches. We split the dataset into three subsets: SpaceDet-100, SpaceDet-5000, and SpaceDet-full, catering to various image processing applications. The SpaceDet-full corpus includes a comprehensive dataloader with 781.5 GB of images and 25.9 MB of ground truth labels. Furthermore, we adapted detection and tracking algorithms on the collected dataset using a specified splitting method to accelerate the training process. The trained model can detect RSOs from real-world space observations with zero-shot capability.
Jiaping Xiao, Rangya Zhang, Yuhang Zhang 0014, Lu Bai 0005, Qianlei Jia, Mir Feroskhan
IJCAI6
2025 An Improved Reinforcement Learning-Based UAV Obstacle Avoidance Framework Using PPO-CMA
abstract
In recent years, the widespread adoption of unmanned aerial vehicles (UAVs) has increased the demand for advanced obstacle avoidance capabilities. Traditional path planning algorithms often exhibit low efficiency and poor adaptability, making them unsuitable for dynamic environments. To address these limitations, researchers have explored reinforcement learning (RL)-based approaches, with Proximal Policy Optimization (PPO) widely used for its stable policy updates and improved sample efficiency. However, PPO suffers from slow convergence, which limits its real-time applicability. To overcome this issue, this paper integrates the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) into PPO and proposes PPO-CMA, an enhanced algorithm for end-to-end UAV path planning. The PPO-CMA network, leveraging depth images and UAV pose feedback, generates continuous control actions, combining CMA-ES adaptive search with PPO policy optimization to improve convergence speed and learning efficiency. The proposed method is evaluated in the VisFly simulation environment, demonstrating significantly faster convergence compared to traditional PPO while ensuring accurate target-reaching and obstacle avoidance. The real-world experimental results validate the effectiveness, robustness, and practical applicability of PPO-CMA for UAV navigation and motion planning in real-world scenarios.
Yaosheng Deng, Mir Feroskhan
SMC4
2025 Physics-informed Split Extended Dynamic Mode Decomposition and Real-Time Sequential Action Control of Multirotors with Partially Known Dynamics
abstract
This paper addresses the challenge of real-time control of multirotors subjected to partially known and unmodeled dynamics. A physics-informed Koopman operator framework is proposed, where the known physical dynamics and unknown residual effects are separated using a Strang splitting approach. The continuous-time Koopman operator is trained on physics-derived trajectories, while the discrete-time Koopman operator is learned from real-world trajectory data, enabling a data-efficient and globally linearizable model of the multirotor dynamics. The learned linear model is subsequently used to design a discrete-time Sequential Action Control (SAC) policy for real-time trajectory tracking. Experimental validation on a quadrotor platform tracking a lemniscate trajectory demonstrates that the proposed PI-EDMD-based SAC controller achieves superior tracking accuracy and up to 67% lower control energy consumption compared to baseline nonlinear SAC and LQR controllers. These results highlight the effectiveness of the proposed framework in enhancing both trajectory fidelity and actuation efficiency for multirotors.
Archit Krishna Kamath, Saeid Nahavandi, Sreenatha Anavatti, Mir Feroskhan
SMC4
2025 A Physics-Informed Approach to Intelligent Actuator-Fault Diagnosis in Multirotor UAVs
abstract
Multirotor unmanned aerial vehicles (UAVs) frequently experience degraded control authority due to partial actuator faults, compromising mission reliability and safety. Purely data-driven fault diagnostic methods, although effective, typically demand extensive labelled datasets and lack direct interpretability. Physics-informed neural networks (PINNs), which incorporate physical laws directly into their learning process, offer a promising alternative by enabling data-efficient training and interpretable results. This study proposes a PINN for actuator fault diagnosis in quadrotor UAVs by embedding discrete Newton-Euler residuals within its loss function, ensuring predictions remain consistent with rigid-body dynamics. A quadrotor UAV is modelled in a high-fidelity simulation environment, and flight data from these simulations are collected for analysis. A sensitivity study is subsequently conducted, testing the PINN against varied fault magnitudes, fault intervals, and different training dataset sizes (100%, 50%, and 30%).The proposed PINN outperforms a similarly sized multilayer perceptron (MLP), reducing fault detection delay by approximately 20%, consistently achieving macro F1 scores above 0.90, and improving prediction accuracy (R2) and RMSE, even with limited labelled data.
Thanaraj T, Archit Krishna Kamath, Mir Feroskhan
SMC3
2025 Physics-Informed Neural Network Modelling of Kinematics for Flapping-wing Mechanism in Aerial Robots
abstract
Biological studies have highlighted the critical role of elastic tendons in enabling energy-efficient flapping flight. Inspired by this principle, several robotic prototypes have incorporated elasticity into their wing mechanisms, showing notable improvements in flapping efficiency. However, accurately modeling the kinematics of such elastic-integrated systems remains a significant challenge due to their nonlinear and coupled dynamics. This study presents a data-driven and physics-informed framework for predicting the kinematics of an elastic-incorporated flapping-wing device, an essential step toward the development of accurate and explainable dynamic models. A custom experimental setup was developed to collect time-series kinematic data. A physics-informed neural network (PINN) was trained on this dataset, achieving a normalized RMSE of 0.314 on a holdout test set. In comparison, a multi-layer perceptron (MLP) with the same architecture yielded a higher RMSE of 0.426 and exhibited memorization. The incorporation of physics-based constraints into the PINN enhanced predictive accuracy and improved generalization, particularly in capturing the oscillatory behavior of the flapping system. These results demonstrate the potential of PINNs in modelling elastic-integrated flapping mechanisms and provide a foundation for future dynamic modelling and control of bio-inspired flapping-wing robots.
Clarence Wei Rui Teo, Anush Kumar Sivakumar, Mir Feroskhan
SMC3
2025 Prescribed Performance Finite-Time Observer-based Super-Twisting Controller for Cooperative Aerial Suspended Transport Systems
abstract
This paper presents a prescribed performance finite-time observer-based super-twisting controller (PPFTOST) for cooperative aerial suspended transport systems. The proposed controller integrates a prescribed performance framework, an appointed-time disturbance observer (ATDO) for disturbance estimation, and a fast terminal sliding mode super-twisting controller (FTSMSTC) for rapid convergence with reduced chattering. Through this combination, the closed-loop system achieves finite-time convergence of both the disturbance estimation error and the sliding variables, while ensuring that the tracking errors remain within prescribed performance bounds. In simulations against an ATDO-only baseline, the PPFTOST reduced payload-tracking root mean square error by 8.2%, 6.7%, and 9.3% on the x, y, and z axes, respectively, with all actuator commands remaining within limits. These results demonstrate that the proposed method enables accurate, robust, and smooth multi-UAV load transport under realistic disturbance conditions.
Lu Xiaoqiang, Archit Krishna Kamath, Thanaraj T, Mir Feroskhan
SMC4
2025 Learning Resilient Formation Control of Drones With Graph Attention Network
abstract
Multidrone systems offer notable advantages in various missions, such as search and rescue, environmental surveillance, and industrial inspection, providing enhanced efficiency and redundancy over single-drone operations. However, ensuring resilient multidrone formation in dynamic and adversarial environments, such as during communication loss or cyberattacks, remains a significant challenge. Traditional approaches often struggle with complex modeling requirements and scalability issues. Among them, leader-follower methods rely heavily on predefined hierarchies, making them vulnerable to single-point failures, while distributed methods incur high communication costs and lack efficient mechanisms to dynamically adapt to changing environments. This article proposes a novel learning-based formation control method to enhance the scalability and resilience of multidrone formations. First, a graph attention network (GAT) is leveraged to dynamically model interagent relationships and prioritize critical interactions among variable neighbors via attention mechanisms with bounded communication overhead. Second, a dual-mode control strategy is designed, integrating leader-follower and distributed control approaches to optimize communication costs while maintaining formation performance. Third, deep reinforcement learning is utilized to train the GAT-based controller, achieving objectives, such as maintaining formation tightness, avoiding collisions, and ensuring resilience against Denial-of-Service (DoS) attacks. Extensive simulations demonstrate superior performance of our method over baseline controllers under normal and adversarial conditions. Furthermore, real-world flight experiments validate the effectiveness and generalizability of the trained policy.
Jiaping Xiao, Xu Fang 0001, Qianlei Jia, Mir Feroskhan
IEEE Internet Things J.4
2025 Real-Time Avoidance of Obstacles and Emergent Geo-Fences for Urban Air Mobility Using Deep Reinforcement Learning
abstract
Urban air mobility (UAM) aims to revolutionize transportation in urban airspace by integrating advanced aerial vehicles for efficient, safe, and sustainable short-distance travel within cities. The growth in UAM operations significantly increases collision risks in dense, dynamic urban airspaces filled with buildings, obstacles, and temporary geo-fences triggered by crowded events or severe weather. Addressing the dynamic emergence of geo-fences in environments originally characterized by dense building structures presents significant challenges, and current solutions remain inadequate. In this study, we propose a novel Deep Deterministic Policy Gradient Learning Framework (DDPG), enhanced with Gated Recurrent Units (GRU), introduce a randomized layer after the neural network’s input observation and incorporate Feature Matching (FM) loss. This introduction of the randomized layer boosts the performance of autonomous aerial vehicles in avoiding dynamically emergent geo-fences in environment with dense obstacles and mitigate path deviation. Moreover the similar improvements can be found when AAV operates in unseen environments. We modeled our simulation environment using real spatial data from a residential area in Singapore, dividing it into areas for training and evaluation. Our results demonstrate that employing feature matching gated recurrent unit-deep deterministic policy gradient (FMGRU-DDPG) algorithms enables AAVs to achieve over a 90% success rate in reaching their destinations despite the presence of up to 10 dynamically appearing geo-fences during its flight. Additionally, the algorithm maintains more than a 60% success rate in unseen environments during training which outperforming both GRU-DDPG and DDPG algorithms.
Mingcheng Zhang, Bizhao Pang, Mir Feroskhan, Chen Lv 0001
IEEE Trans. Intell. Transp. Syst.4
2025 Collaborative Target Search With a Visual Drone Swarm: An Adaptive Curriculum Embedded Multistage Reinforcement Learning Approach
abstract
Equipping drones with target search capabilities is highly desirable for applications in disaster rescue and smart warehouse delivery systems. Multiple intelligent drones that can collaborate with each other and maneuver among obstacles show more effectiveness in accomplishing tasks in a shorter amount of time. However, carrying out collaborative target search (CTS) without prior target information is extremely challenging, especially with a visual drone swarm. In this work, we propose a novel data-efficient deep reinforcement learning (DRL) approach called adaptive curriculum embedded multistage learning (ACEMSL) to address these challenges, mainly 3-D sparse reward space exploration with limited visual perception and collaborative behavior requirements. Specifically, we decompose the CTS task into several subtasks including individual obstacle avoidance, target search, and inter-agent collaboration, and progressively train the agents with multistage learning. Meanwhile, an adaptive embedded curriculum (AEC) is designed, where the task difficulty level (TDL) can be adaptively adjusted based on the success rate (SR) achieved in training. ACEMSL allows data-efficient training and individual-team reward allocation for the visual drone swarm. Furthermore, we deploy the trained model over a real visual drone swarm and perform CTS operations without fine-tuning. Extensive simulations and real-world flight tests validate the effectiveness and generalizability of ACEMSL. The project is available at https://github.com/NTU-UAVG/CTS-visual-drone-swarm.git.
Jiaping Xiao, Phumrapee Pisutsin, Mir Feroskhan
IEEE Trans. Neural Networks Learn. Syst.3
2025 Vision-Based Learning for Drones: A Survey
abstract
Drones, as advanced cyber-physical systems (CPSs), are undergoing a transformative shift with the advent of vision-based learning, a field that is rapidly gaining prominence due to its profound impact on drone autonomy and functionality. Unlike existing task-specific surveys, this work offers a comprehensive overview of vision-based learning for drones, emphasizing its pivotal role in enhancing their operational capabilities across various scenarios. First, the fundamental principles of vision-based learning are elucidated, demonstrating how it significantly improves drones' visual perception and decision-making processes. Vision-based control methods are then categorized into indirect, semidirect, and end-to-end approaches from the perception-control perspective. Various applications of vision-based drones with learning capabilities are further explored, ranging from single-agent systems to more complex multiagent and heterogeneous system scenarios, while highlighting the challenges and innovations characterizing each domain. Finally, open questions and potential solutions are discussed to guide future research and development in this dynamic and rapidly evolving field. With the growth of large language models (LLMs) and embodied intelligence, vision-based learning for drones provides a promising yet challenging road toward achieving artificial general intelligence (AGI) in the 3-D physical world.
Jiaping Xiao, Rangya Zhang, Yuhang Zhang 0014, Mir Feroskhan
IEEE Trans. Neural Networks Learn. Syst.4
2025 Practical Robust Formation Control for Nonlinear Multiagent Systems via Generative Adversarial Learning Framework: Theory and Experiment
abstract
Cyber attacks and disturbances greatly impair the performance of formation tasks in multiagent systems (MASs). To achieve robust formation control against these challenges, this article proposes a generative adversarial learning framework that is theoretically transparent and practically applicable. Rather than relying on an end-to-end deep neural networks (DNNs) architecture, our work leverage a double robust structure that combine the representation capabilities of DNNs with established, theoretically grounded linear control theory, ultimately achieving a practical, learning-based robust formation for MASs. Initially, generative adversarial networks (GANs) are used to linearize agent dynamics under false data injection (FDI) attacks and external disturbances. Subsequently, a proportional-integral (PI) protocol is employed to achieve overall robust formation. We present rigorous theoretical analyses of both stages, demonstrating the guaranteed convergence of GANs training and the closed-loop formation errors. Our approach is directly validated through a series of physical experiments involving multi-quadrotors, demonstrating robustness against attacks and disturbances during formation flights, without the sim-to-real gap commonly encountered in learning-based control frameworks.
Nuan Wen, Mir Feroskhan
IEEE Trans. Syst. Man Cybern. Syst.2
2024 Dynamics-Driven Visual Servoing of Over-Actuated Quadrotors
abstract
This study introduces a dynamics-driven visual servoing methodology tailored for an over-actuated quadrotor equipped with tilting rotors. The mathematical framework encompasses both translational and rotational dynamics, incorporating the tilting rotor angles to facilitate autonomous control over attitude and position. The stereo camera model is derived utilizing stacked Jacobians. The proposed dynamics-driven methodology establishes a direct correspondence between the dynamics of image pixel accelerations captured by the stereo cameras and the thrust and torque commands of the over-actuated tilting quadrotor. This obviates the necessity for computationally intensive inverse Jacobian computations typically required in traditional visual servoing methods. By employing an over-actuated tilting rotor configuration instead of a conventional quadrotor setup, the dynamics-driven approach surmounts limitations in independently controlling the pose and attitude. It enables the tracking of not only the 3D position but also the orientation of points of interest using the onboard stereo cameras. Simulation outcomes affirm the efficacy of the approach in achieving precise visual tracking.
Archit Krishna Kamath, Sreenatha Anavatti, Mir Feroskhan
ICARCV3
2024 AYDIV: Adaptable Yielding 3D Object Detection via Integrated Contextual Vision Transformer
abstract
Combining LiDAR and camera data has shown potential in enhancing short-distance object detection in autonomous driving systems. Yet, the fusion encounters difficulties with extended distance detection due to the contrast between LiDAR’s sparse data and the dense resolution of cameras. Besides, discrepancies in the two data representations further complicate fusion methods. We introduce AYDIV, a novel framework integrating a tri-phase alignment process specifically designed to enhance long-distance detection even amidst data discrepancies. AYDIV consists of the Global Contextual Fusion Alignment Transformer (GCFAT), which improves the extraction of camera features and provides a deeper understanding of large-scale patterns; the Sparse Fused Feature Attention (SFFA), which fine-tunes the fusion of LiDAR and camera details; and the Volumetric Grid Attention (VGA) for a comprehensive spatial data fusion. AYDIV’s performance on the Waymo Open Dataset (WOD) with an improvement of 1.24% in mAPH value(L2 difficulty) and the Argoverse2 Dataset with a performance improvement of 7.40% in AP value demonstrates its efficacy in comparison to other existing fusion-based methods. Our code is publicly available at https://github.com/sanjay-810/AYDIV2
Tanmoy Dam, Sanjay Bhargav Dharavath, Sameer Alam, Nimrod Lilith, Supriyo Chakraborty, Mir Feroskhan
ICRA6
2024 Robust Decentralised Control for Modular Aerial Parcel Delivery Using Persistently Excited Physics-Informed Neural Networks
abstract
This paper presents a robust decentralised control approach for modular aerial parcel delivery using persistently excited physics-informed neural networks (PE-PINNs). The proposed method enables each propeller module to independently generate control efforts based solely on its local state information and that of its 1-hop neighbors, without requiring global system knowledge. The PE-PINN is trained to approximate the optimal centralized control policy by incorporating the nominal system dynamics and accounting for modeling uncertainties. Key innovations include estimating the Lipschitz constant to ensure persistent excitation during training, and a decentralised control formulation that minimizes the difference between the learned and optimal control efforts. Experimental results on a modular aerial testbed demonstrate the PE-PINN's ability to achieve high-accuracy fixed-point hover and trajectory tracking performance, outperforming a prior decentralised control approach by 8.57% and 24.17% respectively. The proposed framework enables scalable and robust control of modular aerial systems for parcel delivery applications.
Archit Krishna Kamath, Saeid Nahavandi, Sreenatha Anavatti, Mir Feroskhan
SMC4
2024 A Physics-Informed Neural Network Approach to Augmented Dynamics Visual Servoing of Multirotors
abstract
This article presents a visual servoing strategy that integrates the capabilities of a physics-informed neural network (PINN) to estimate system uncertainties and inaccuracies with a dynamics-centered visual servoing technique for multirotors. The proposed method effectively combines these approaches, eliminating the need for inverse Jacobian calculations to determine multirotor motion by directly relating pixel variations to the multirotor's torque and thrust inputs, while also strengthening the method's robustness through the utilization of the PINN to model and address uncertainties in camera and multirotor parameters, as well as the modeling inaccuracies inherent in the dynamics-centered visual servoing technique. In contrast to existing state-of-the-art data-driven approaches, the proposed PINN approach requires, on average, 65% less labeled data to characterize uncertainties and inaccuracies. To ensure real-time implementation of the visual servoing model, the PINN-learned model is combined with an adaptive horizon monotonically weighted nonlinear model predictive controller (NMPC), capable of processing control efforts at rates 10 times faster than existing Tube MPC and Adaptive MPC strategies. These findings are validated through real-time trajectory tracking experiments, which not only highlight the effectiveness of the proposed approach in approximating modeling inaccuracies but also its capability in handling uncertainties upto 70% in camera parameters.
Archit Krishna Kamath, Sreenatha Anavatti, Mir Feroskhan
IEEE Trans. Cybern.3
2024 Multitarget Assignment Under Uncertain Information Through Decision Support Systems
abstract
Unmanned aerial vehicles (UAVs) play an important role in advancing fire prevention technology. However, existing research usually assumes that the firefighting equipment carried by UAVs, such as fire extinguishing bombs and water, can significantly satisfy actual requirements. However, in many practical scenarios, the firefighting resources available on UAVs are limited, necessitating an assessment and prioritization of affected areas. This article proposes a group decision-making framework for uncertain environments, employing Z-numbers and q-rung orthopair fuzzy sets to address this challenge. Specifically, a polar coordinate system is employed to transform the parameters in Z-numbers into the corresponding membership and nonmembership. Meanwhile, the potential probability distribution of Z-numbers is calculated based on an optimization model. To combine multiple sets of uncertain information into a final overall assessment, we propose an aggregation operator and provide strict proof using mathematical induction. Furthermore, a new weight calculation method is introduced to determine the weights of multiple pieces of information based on the potential probability distribution of Z-numbers and an improved golden rule representative value. Based on the sigmoid function and generalized knowledge measure, a novel score function is defined to rank different Z-information. In addition, a distance measure between Z-information is proposed and rigorously proved. Finally, the proposed method is validated through practical flight tests. The results and comparative analysis illustrate that our proposed method effectively mitigates other existing approaches' computational and informational loss limitations.
Qianlei Jia, Jiaping Xiao, Mir Feroskhan
IEEE Trans. Ind. Informatics3
2024 Design, Modeling, and Control of a Coaxial Drone
abstract
Various quadrotor drones have been developed in recent years, mainly focusing on either improving maximum thrust per platform area or flight maneuverability. Evidently, achieving both advantages simultaneously is a challenging task, since they call for opposing rotor requirements. Specifically, improving the drone's maximum thrust per platform area mainly requires reducing the number of rotors to make way for larger and more powerful rotors. While this can be an effective method to increase overall thrust, improving flight maneuverability requires a greater number of rotors to generate larger rotating torques or to increase the thrust vectoring capability. To address this challenge, we design a novel coaxial drone with two contra-rotating rotors for high thrust efficiency while enabling independent dual-axis rotor rotation to maintain maneuverability along the roll and pitch axes. The thrust vectoring capability is provided by two dedicated servo motors connected vertically in series with the coaxial propellers to produce a compact and elongated fuselage frame. A nonlinear flight model in six degrees of freedom is developed for the underactuated system, incorporating four control inputs from the two propellers and servos respectively. Consequently, a nonlinear control allocation approach is proposed such that the drone can produce a desired control force and yaw torque to stabilize the drone's position and yaw angle. For the uncontrolled roll and pitch dynamics, a damping component is added such that the roll and pitch angular velocities can also be stabilized. Both numerical simulations and real experiments are conducted to validate the design of the drone and the effectiveness of the proposed control strategy.
Liangming Chen, Jiaping Xiao, Yumin Zheng, N. Arun Alagappan, Mir Feroskhan
IEEE Trans. Robotics5
2023 Meta-Heuristics Approach for Arrival Sequencing and Delay Absorption Through Automated Vectoring
abstract
The continuous increase in air traffic compels major airports to optimize their resources and enhance their Terminal Maneuvering Airspace (TMA) operations. The primary constraint to increasing airport capacity is the required separation minima between pairs of aircraft arriving through the same approach routes. RECAT-EU is a revised global wake separation minima scheme released by EUROCONTROL in 2018. This suggested a reduction in separation minima between certain aircraft pairs while maintaining safety levels. However, it increases in separation minima scheme complexity by doubling the number of non-minimum radar separation (MRS) values. In this work, a Meta-Heuristic based optimization model is proposed to sequence arrival flights based on the RECAT-EU separation scheme, which can provide an optimized vectoring to ensure flights absorb their assigned delays before reaching the final approach fix. Findings show that the proposed model is able to generate an optimized landing sequence for 50 arrival flights in a computation time of 16 seconds. It also suggests that the proposed algorithm's computational time increases linearly with increasing the number of flights. Furthermore, trajectory vectoring results demonstrate that 85% of the assigned delays could be absorbed by flying the proposed vectored trajectories.
Imen Dhief, Mir Feroskhan, Sameer Alam, Nimrod Lilith, Daniel Delahaye
CEC2