3D Gaussian splatting (3DGS) has recently demonstrated significant potential in computer vision, enabling high-fidelity 3D scene reconstruction with real-time rendering and fast training times. However, existing methods struggle in large, visually sparse, geometric self-similarity environments due to heavy reliance on image-based feature matching and depth information. In this work, we propose a novel reconstruction pipeline that reduces the dependence on visual features by incorporating IMU and LiDAR data to generate accurate point clouds and robustly localize images within the scene. Global colorization is achieved through 3D-to-2D projections of the localized images, which are then used to supervise 3DGS training. Our results demonstrate that the proposed pipeline significantly enhances the quality of 3D reconstruction for large, sparse scenarios, opening up new opportunities for applications in remote mapping and autonomous inspection.

This paper presents a novel approach to multi-robot collision avoidance that integrates global path planning with local navigation strategies, utilizing attentive graph neural networks to manage dynamic interactions among agents. We introduce a local navigation model that leverages pre-planned global paths, allowing robots to adhere to optimal routes while dynamically adjusting to environmental changes. The model’s robustness is enhanced through the introduction of noise during training, resulting in superior performance in complex, dynamic environments. Our approach is evaluated against established baselines, including NH-ORCA, DRL-NAV, and GA3C-CADRL, across various structurally diverse simulated scenarios. The results demonstrate that our model achieves consistently higher success rates, lower collision rates, and more efficient navigation, particularly in challenging scenarios where baseline models struggle. This work offers an advancement in multi-robot navigation, with implications for robust performance in complex, dynamic environments with varying degrees of complexity, such as those encountered in logistics, where adaptability is essential for accommodating unforeseen obstacles and unpredictable changes.

With increasingly challenging applications for quadrotors, higher requirements are emerging for tracking accuracy and safety. While high accuracy is a prerequisite for complex tasks, safety is ensured through tolerance to actuator faults and resistance to external disturbances. In this article, adaptive robust control (ARC) integrated with Gaussian processes (GPs), i.e., ARC-GP, is proposed to achieve enhanced accuracy, fault tolerance, and anti-disturbance. These three requirements are interrelated and affected by uncertainties. The primary idea of this article is to categorize uncertainties into parametric and nonparametric types, which are then addressed through parameter adaptation and GP, respectively. First, a detailed dynamic model is established, including actuator models that reflect different types of faults corresponding to changes in different physical parameters. Then, parameter adaptation is designed, with direct and indirect methods adopted for different parameters. In particular, the actuator parameters are effectively estimated to achieve targeted fault compensation. Regarding GP for nonparametric uncertainties, its model parameters are also updated via parameter adaptation. The GP thereby also learns parameter estimation errors along with external disturbances. Accordingly, ARC controllers are designed, for which robust feedback terms are constructed to further mitigate uncertainties on the basis of the covariances predicted by GP. The experiments demonstrate that the proposed ARC-GP can actively tolerate various types of actuator faults and better resist wind disturbances.

In autonomous and mobile robotics, a principal challenge is resilient real-time environmental perception, particularly in situations characterized by unknown and dynamic elements, as exemplified in the context of autonomous drone racing. This study introduces a perception technique for detecting drone racing gates under illumination variations, which is common during high-speed drone flights. The proposed technique relies upon a lightweight neural network backbone augmented with capabilities for continual learning. The envisaged approach amalgamates predictions of the gates' positional coordinates, distance, and orientation, encapsulating them into a cohesive pose tuple. A comprehensive number of tests serve to underscore the efficacy of this approach in confronting diverse and challenging scenarios, specifically those involving variable lighting conditions. The proposed methodology exhibits notable robustness in the face of illumination variations, thereby substantiating its effectiveness.

This paper presents MIMIR-UW, a multipurpose underwater synthetic dataset for SLAM, depth estimation, and object segmentation to bridge the gap between theory and application in underwater environments. MIMIR-UW integrates three camera sensors, inertial measurements, and ground truth for robot pose, image depth, and object segmentation. The underwater robot is deployed within a pipe exploration scenario, carrying artificial lights that create uneven lighting, in addition to natural artefacts such as reflections from natural light and backscattering effects. Four environments totalling eleven tracks are provided, with various difficulties regarding light conditions or dynamic elements. Two metrics for dataset evaluation are proposed, allowing MIMIR-UW to be compared with other datasets. State-of-art methods on SLAM, segmentation and depth estimation are deployed and benchmarked on MIMIR-UW. Moreover, the dataset's potential for sim-to-real transfer is demonstrated by leveraging the segmentation and depth estimation models trained on MIMIR-UW in a real pipeline inspection scenario. To the best of the authors' knowledge, this is the first underwater dataset targeted for such a variety of methods. The dataset is publicly available online. https://github.com/remaro-network/MIMIR-UW/

This study presents the conflict-aware multi-agent estimated time of arrival (CAMETA) framework, a novel approach for predicting the arrival times of multiple agents in unstructured environments without predefined road infrastructure. The CAMETA framework consists of three components: a path planning layer generating potential path suggestions, a multi-agent ETA prediction layer predicting the arrival times for all agents based on the paths, and lastly, a path selection layer that calculates the accumulated cost and selects the best path. The novelty of the CAMETA framework lies in the heterogeneous map representation and the heterogeneous graph neural network architecture. As a result of the proposed novel structure, CAMETA improves the generalization capability compared to the state-of-the-art methods that rely on structured road infrastructure and historical data. The simulation results demonstrate the efficiency and efficacy of the multi-agent ETA prediction layer, with a mean average percentage error improvement of 29.5% and 44% when compared to a traditional path planning method (A *) which does not consider conflicts. The performance of the CAMETA framework shows significant improvements in terms of robustness to noise and conflicts as well as determining proficient routes compared to state-of-the-art multi-agent path planners.

Existing Deep Learning (DL) frameworks typically do not provide ready-to-use solutions for robotics, where very specific learning, reasoning, and embodiment problems exist. Their relatively steep learning curve and the different methodologies employed by DL compared to traditional approaches, along with the high complexity of DL models, which often leads to the need of employing specialized hardware accelerators, further increase the effort and cost needed to employ DL models in robotics. Also, most of the existing DL methods follow a static inference paradigm, as inherited by the traditional computer vision pipelines, ignoring active perception, which can be employed to actively interact with the environment in order to increase perception accuracy. In this paper, we present the Open Deep Learning Toolkit for Robotics (OpenDR). OpenDR aims at developing an open, non-proprietary, efficient, and modular toolkit that can be easily used by robotics companies and research institutions to efficiently develop and deploy AI and cognition technologies to robotics applications, providing a solid step towards addressing the aforementioned challenges. We also detail the design choices, along with an abstract interface that was created to overcome these challenges. This interface can describe various robotic tasks, spanning beyond traditional DL cognition and inference, as known by existing frameworks, incorporating openness, homogeneity and robotics-oriented perception e.g., through active perception, as its core design principles.

The detection of contextual anomalies is a challenging task for surveillance since an observation can be considered anomalous or normal in a specific environmental context. An unmanned aerial vehicle (UAV) can utilize its aerial monitoring capability and employ multiple sensors to gather contextual information about the environment and perform contextual anomaly detection. In this work, we introduce a deep neural network-based method (CADNet) to find point anomalies (i.e., single instance anomalous data) and contextual anomalies (i.e., context-specific abnormality) in an environment using a UAV. The method is based on a variational autoencoder (VAE) with a context sub-network. The context sub-network extracts contextual information regarding the environment using GPS and time data, then feeds it to the VAE to predict anomalies conditioned on the context. To the best of our knowledge, our method is the first contextual anomaly detection method for UAV-assisted aerial surveillance. We evaluate our method on the AU-AIR dataset in a traffic surveillance scenario. Quantitative comparisons against several baselines demonstrate the superiority of our approach in the anomaly detection tasks. The codes and data will be available at https://bozcani.github.io/cadnet.

In this paper, we develop an online learning-based visual tracking framework that can optimize the target model and estimate the scale variation for object tracking. We propose a recommender-based tracker, which is capable of selecting the representative convolutional neural network (CNN) layers and feature maps autonomously. In addition, the proposed recommender computes the weights of these layers and feature maps. A discriminative target percept of each recommended layer is reconstructed by the weighted sum of the recommended feature maps. Then the target model of the correlation filter is updated by the weighted sum of the target percepts. Thus, a sub-network is extracted from the pre-trained CNN backbone for the tracking process of a specific target. To deal with scale changes, we propose a spatiotemporal-based min-channel method to estimate the target size variation directly from CNN features. Experimental results on 50 benchmark datasets and video data from a rescue drone demonstrate that the proposed tracker is quite competitive with the state-of-the-art CNN-based trackers in terms of accuracy, scale adaptation, and robustness for UAV-related applications.

In this work we propose a holistic framework for autonomous aerial inspection tasks, using semantically-aware, yet, computationally efficient planning and mapping algorithms. The system leverages state-of-the-art receding horizon exploration techniques for next-best-view (NBV) planning with geometric and semantic segmentation information provided by state-of-the-art deep convolutional neural networks (DCNNs), with the goal of enriching environment representations. The contributions of this article are threefold, first we propose an efficient sensor observation model, and a reward function that encodes the expected information gains from the observations taken from specific view points. Second, we extend the reward function to incorporate not only geometric but also semantic probabilistic information, provided by a DCNN for semantic segmentation that operates in real-time. The incorporation of semantic information in the environment representation allows biasing exploration towards specific objects, while ignoring task-irrelevant ones during planning. Finally, we employ our approaches in an autonomous drone shipyard inspection task. A set of simulations in realistic scenarios demonstrate the efficacy and efficiency of the proposed framework when compared with the state-of-the-art.

Fast and robust gate perception is of great importance in autonomous drone racing. We propose a convolutional neural network-based gate detector (GateNet1) that concurrently detects gate’s center, distance, and orientation with respect to the drone using only images from a single fish-eye RGB camera. GateNet achieves a high inference rate (up to 60 Hz) on an onboard processor (Jetson TX2). Moreover, GateNet is robust to gate pose changes and background disturbances. The proposed perception pipeline leverages a fish-eye lens with a wide field-of-view and thus can detect multiple gates in close range, allowing a longer planning horizon even in tight environments. For benchmarking, we propose a comprehensive dataset (AU-DR) that focuses on gate perception. Throughout the experiments, GateNet shows its superiority when compared to similar methods while being efficient for onboard computers in autonomous drone racing. The effectiveness of the proposed framework is tested on a fully-autonomous drone that flies on previously-unknown track with tight turns and varying gate positions and orientations in each lap.

Unmanned aerial vehicles (UAVs) with mounted cameras have the advantage of capturing aerial (bird-view) images. The availability of aerial visual data and the recent advances in object detection algorithms led the computer vision community to focus on object detection tasks on aerial images. As a result of this, several aerial datasets have been introduced, including visual data with object annotations. UAVs are used solely as flying-cameras in these datasets, discarding different data types regarding the flight (e.g., time, location, internal sensors). In this work, we propose a multi-purpose aerial dataset (AU-AIR) that has multi-modal sensor data (i.e., visual, time, location, altitude, IMU, velocity) collected in real-world outdoor environments. The AU-AIR dataset includes meta-data for extracted frames (i.e., bounding box annotations for traffic-related object category) from recorded RGB videos. Moreover, we emphasize the differences between natural and aerial images in the context of object detection task. For this end, we train and test mobile object detectors (including YOLOv3-Tiny and MobileNetv2-SSDLite) on the AU-AIR dataset, which are applicable for real-time object detection using on-board computers with UAVs. Since our dataset has diversity in recorded data types, it contributes to filling the gap between computer vision and robotics. The dataset is available at https://bozcani.github.io/auairdataset.

This paper presents a nonlinear model predictive control-based trajectory generation with redundancy resolution strategy for a dual-arm aerial manipulator. The proposed approach continuously re-plans the aerial robot motion to track the position of the dual-arm end-effectors, while satisfying other constraints such as avoiding the obstacles and limits of the motion variables. In addition, the minimum manipulability measure is set to ensure a smooth motion and avoid singular configurations of the aerial robot. The performance of the developed framework is shown via simulations and real-time experimental flight tests, involving the dual-arm aerial manipulator performing non-trivial multi-task missions, which require the coordination of both arms, while keeping the configuration of the aerial robot within imposed bounds and preventing collisions with obstacles.

In this work, we introduce a novel, end-to-end motion planner for quadrotor navigation. Informed by a rough path to goal in partially unknown environments, our method creates desirable motion plans using raw depth images from a front-facing camera. It exploits correlations between local spatial portions of these images to generate desirable motion primitive sequences on the fly without conducting explicit sensing-reconstructing-planning. We evaluate our method through an extensive comparison with three competitor algorithms over ten different environments in AirSim simulations. Our method outperforms its competitors in terms of safe navigation distance, navigation time, and crash rate over 50 flights. We also deploy our method for real flight tests with DJI F330 Quadrotor equipped with Intel RealSense D435, and demonstrate its real-time ap-plicability. Our method successfully performs 15 real flights in three different environment settings with increasing complexity. The experiments can be found at https://youtu.be/hw0sxNwliqs.

Drone racing is a recreational sport in which the goal is to pass through a sequence of gates in a minimum amount of time, while avoiding collisions. In autonomous drone racing, one must accomplish this task by flying fully autonomously in an unknown environment by relying only on computer vision methods for detecting the target gates. Due to the challenges such as background objects and varying lighting conditions, traditional object detection algorithms based on colour or geometry tend to fail. Convolutional neural networks offer impressive advances in computer vision, but require an immense amount of data to learn. Collecting this data is a tedious process because the drone has to be flown manually, and the data collected can suffer from sensor failures. In this work, a semi-synthetic dataset generation method is proposed, using a combination of real background images and randomised 3D renders of the gates, to provide a limitless amount of training samples that do not suffer from those drawbacks. Using the detection results, a line-of-sight guidance algorithm is used to cross the gates. In several experimental real-time tests, the proposed framework successfully demonstrates fast and reliable detection and navigation.

Anomaly detection is a key goal of autonomous surveillance systems that should be able to alert unusual observations. In this paper, we propose a holistic anomaly detection system using deep neural networks for surveillance of critical infrastructures (e.g., airports, harbors, warehouses) using an unmanned aerial vehicle (UAV). First, we present a heuristic method for the explicit representation of spatial layouts of objects in bird-view images. Then, we propose a deep neural network architecture for unsupervised anomaly detection (UAV-AdNet), which is trained on environment representations and GPS labels of bird-view images jointly. Unlike studies in the literature, we combine GPS and image data to predict abnormal observations. We evaluate our model against several baselines on our aerial surveillance dataset and show that it performs better in scene reconstruction and several anomaly detection tasks. The codes, trained models, dataset, and video will be available at https://bozcani.github.io/uavadnet.

This article presents an online learning method for improved control of nonlinear systems by combining deep learning and fuzzy logic. Given the ability of deep learning to generalize knowledge from training samples, the proposed method requires minimum amount of information about the system to be controlled. However, in robotics, particularly in aerial robotics where the operating conditions may vary, online learning is required. In this article, fuzzy logic is preferred to provide supervising feedback to the deep model for adapting to variations in the system dynamics as well as new operational conditions. The learning method is divided into two phases: offline pretraining and online posttraining. In the former, the system is controlled by a conventional controller and a deep fuzzy neural network (DFNN) is pretrained based on the recorded input-output dataset, in order to approximate the inverse dynamical model of the system. In the latter, only the pretrained DFNN is used to control the system. In this phase, the fuzzy logic, which encodes the expert knowledge, is utilized to observe the behavior of the system and to correct the action of DFNN instantaneously. The experimental results show that the proposed online learning-based approach improves the trajectory tracking performance of the unmanned aerial vehicle.

This paper presents a novel transition flight mathematical model of tilt-rotor unmanned aerial vehicles and demonstrates an application of a novel learning controller on the developed model. Instead of conventional steady aerodynamic models, an unsteady aerodynamic model capable of representing rapid changes in the air flow is developed for the tilt-rotor transition flight. The vehicle is controlled by a neuro-fuzzy learning controller, consisting of a type-2 fuzzy neural network and a proportional-derivative controller. Its results are compared with the results of proportional-integral-derivative controllers. It is evident from the results that the learning controller is capable of capturing the rapid changes in the aerodynamics and outperforms its nonlearning counterpart under perturbed conditions.

Non-iterative methods for parameter estimation for interval type-2 neuro-fuzzy structure are fast to implement, when compared to online methods, and need no -or a few- parameters to be tuned. In this paper, a novel dynamic cost function, which defines a relationship between the current and past errors, is defined. The minimization of the aforementioned cost function results in a decreasing sequence of error which makes the proposed method numerically more stable when compared to least squares-based methods. It is a well-known phenomenon that a matrix inversion may cause problems if the matrix to be inverted is ill-defined i.e. its condition number is far bigger than one. The use of a dynamic relationship between the current and past error adds more degrees of freedom which makes it possible to improve the condition number of the matrix. Comprehensive simulation studies are presented for the prediction of financial data sets. The simulation results shows the superior numerical stability of the proposed method as the mean value of the condition number is smaller. This finding results in more accurate matrix inversion to be done in the two-step matrix inversion.

This work presents an online learning-based control method for improved trajectory tracking of unmanned aerial vehicles using both deep learning and expert knowledge. The proposed method does not require the exact model of the system to be controlled, and it is robust against variations in system dynamics as well as operational uncertainties. The learning is divided into two phases: offline (pre-)training and online (post-)training. In the former, a conventional controller performs a set of trajectories and, based on the input-output dataset, the deep neural network (DNN)-based controller is trained. In the latter, the trained DNN, which mimics the conventional controller, controls the system. Unlike the existing papers in the literature, the network is still being trained for different sets of trajectories which are not used in the training phase of DNN. Thanks to the rule-base, which contains the expert knowledge, the proposed framework learns the system dynamics and operational uncertainties in real-time. The experimental results show that the proposed online learning-based approach gives better trajectory tracking performance when compared to the only offline trained network.

Aerial filming is constantly gaining importance due to the recent advances in drone technology. It invites many intriguing, unsolved problems at the intersection of aesthetical and scientific challenges. In this work, we propose a deep reinforcement learning agent which supervises motion planning of a filming drone by making desirable shot mode selections based on aesthetical values of video shots. Unlike most of the current state-of-the-art approaches that require explicit guidance by a human expert, our drone learns how to make favorable viewpoint selections by experience. We propose a learning scheme that exploits aesthetical features of retrospective shots in order to extract a desirable policy for better prospective shots. We train our agent in realistic AirSim simulations using both a hand-crafted reward function as well as reward from direct human input. We then deploy the same agent on a real DJI M210 drone in order to test the generalization capability of our approach to real world conditions. To evaluate the success of our approach in the end, we conduct a comprehensive user study in which participants rate the shot quality of our methods. Videos of the system in action can be seen at https://youtu.be/qmVw6mfyEmw.

Quality assessment during postconstruction of buildings is an indispensable procedure in construction industry. This paper describes the design and development of a quality inspection and assessment robot (QuicaBot) that can autonomously scan the entire room using cameras and laser scanners to pick up building defects, such as hollowness, crack, evenness, alignments, and inclination. A robotic system consisting of four types of sensors and a mobile platform as well as the corresponding five types of assessment algorithms is proposed. To the best of our knowledge, this paper is the first attempt to have a complete robotic system for postconstruction quality assessment of buildings. The aim of the developed system is twofold: first, to systematize the manual inspection work through automation resulting in more reliable and objective inspection reports, and then, to speed up the inspection process resulting in a more efficient end product. Based on our experimental on-site tests, the developed novel robot takes only half of the manual inspection time when inspecting the same room. We have also observed that the autonomous assessment results have better inspection accuracy when compared to manual assessments. Last but not least, the results provided by QuicaBot have more consistent measurement accuracy when compared to a manual assessor. Motivated by the initial successful on-site tests and as being a practical mechatronic system illustrating how sensing, sensor fusion, and actuation can be integrated to achieve an intelligent system for building defects assessment, we believe that the QuicaBot-like robots are going to become an integral part of construction industry in the near future.

One of the major challenges of model predictive control (MPC) for robotic applications is the non-trivial weight tuning process while crafting the objective function. This process is often executed using the trial-and-error method by the user. Consequently, the optimality of the weights and the time required for the process become highly dependent on the skill set and experience of the user. In this study, we present a generic and user-independent framework which automates the tuning process by reinforcement learning. The proposed method shows competency in tuning a nonlinear MPC (NMPC) which is employed for trajectory tracking control of aerial robots. It explores the desirable weights within less than an hour in iterative Gazebo simulations running on a standard desktop computer. The real world experiments illustrate that the NMPC weights explored by the proposed method result in a satisfactory trajectory tracking performance.

This paper presents a novel application of a hybrid learning approach to the optimisation of membership and nonmembership functions of a newly developed interval type-2 intuitionistic fuzzy logic system (IT2 IFLS) of a Takagi-Sugeno-Kang (TSK) fuzzy inference system with neural network learning capability. The hybrid algorithms consisting of decoupled extended Kalman filter (DEKF) and gradient descent (GD) are used to tune the parameters of the IT2 IFLS for the first time. The DEKF is used to tune the consequent parameters in the forward pass while the GD method is used to tune the antecedents parts during the backward pass of the hybrid learning. The hybrid algorithm is described and evaluated, prediction and identification results together with the runtime are compared with similar existing studies in the literature. Performance comparison is made among the proposed hybrid learning model of IT2 IFLS, a TSK-type-1 intuitionistic fuzzy logic system (IFLS-TSK), and a TSK-type interval type-2 fuzzy logic system (IT2 FLS-TSK) on two instances of the datasets under investigation. The empirical comparison is made on the designed systems using three artificially generated datasets and three real world datasets. Analysis of results reveal that IT2 IFLS outperforms its type-1 variants, IT2 FLS and most of the existing models in the literature. Moreover, the minimal run time of the proposed hybrid learning model for IT2 IFLS also puts this model forward as a good candidate for application in real time systems.

As non-singleton fuzzy logic controllers (NSFLCs) are capable of capturing input uncertainties, they have been effectively used to control and navigate unmanned aerial vehicles (UAVs) recently. To further enhance the capability to handle the input uncertainty for the UAV applications, a novel NSFLC with the recently introduced similarity-based inference engine, i.e., Sim-NSFLC, is developed. In this paper, a comparative study in a 3D trajectory tracking application has been carried out using the aforementioned Sim-NSFLC and the NSFLCs with the standard as well as centroid composition-based inference engines, i.e., Sta-NSFLC and Cen-NSFLC. All the NSFLCs are developed within the robot operating system (ROS) using the C++ programming language. Extensive ROS Gazebo simulation-based experiments show that the Sim-NSFLCs can achieve better control performance for the UAVs in comparison with the Sta-NSFLCs and Cen-NSFLCs under different input noise levels.

In this paper, our aim is to compare and contrast various ways of modeling uncertainty by using different type-2 fuzzy membership functions available in literature. In particular we focus on a novel type-2 fuzzy membership function, - “Elliptic membership function”. After briefly explaining the motivation behind the suggestion of the elliptic membership function, we analyse the uncertainty distribution along its support, and we compare its uncertainty modeling capability with the existing membership functions. We also show how the elliptic membership functions perform in fuzzy arithmetic. In addition to its extra advantages over the existing type-2 fuzzy membership functions such as having decoupled parameters for its support and width, this novel membership function has some similar features to the Gaussian and triangular membership functions in addition and multiplication operations. Finally, we have tested the prediction capability of elliptic membership functions using interval type-2 fuzzy logic systems on US Dollar/Euro exchange rate prediction problem. Throughout the simulation studies, an extreme learning machine is used to train the interval type-2 fuzzy logic system. The prediction results show that, in addition to their various advantages mentioned above, elliptic membership functions have comparable prediction results when compared to Gaussian and triangular membership functions.

Levenberge-Marquardt (LM) algorithm is a well-known optimization technique which has the advantages of the steepest descent and the Gauss-Newton methods. Unfortunately, LM algorithm-based parameter update rules, regardless of being used to tune the parameters of artificial neural networks or neuro-fuzzy systems, require the calculation of inversion of high dimensional matrices. Matrix inversions are generally computationally expensive, and it is not desired in a real-time application where the computation speed is critical. In this paper, using matrix inversion lemma, LM algorithm is modified to avoid matrix inversion calculations, and therefore lessen its computational burden. The proposed algorithm is compared with the conventional LM algorithm for the training of interval type-2 fuzzy logic systems in terms of its speed. Extensive simulation results demonstrate that that the proposed novel method can increase the speed of LM algorithm by 50% while remaining the same performance.

A significant number of investigations of type-1 and type-2 fuzzy logic controllers have revealed their exceptional ability to capture uncertainties in complex and nonlinear systems, particularly in real-time control applications. However, regardless of being type-1 or type-2, fuzzy logic controller design is still a complicated task due to the lack of a closed form solution of the output and an interpretable relationship between the control output and fuzzy logic controller design parameters, such as center or width of the membership functions. To simplify the design procedure further, we think every attempt to obtain such interpretable relationships is worthwhile. Accordingly, this paper aims to design a double-input interval type-2 fuzzy PID controller and obtain interpretable relationships between the input and the output of the controller. Thereafter, we deploy the novel design for the control of a Y6 coaxial tricopter unmanned aerial vehicle. Simulation results, which are realised in robot operating system (ROS) using C++ and Gazebo environment, are found to tally with the theoretical analysis and claims in the paper.

In this paper, we put forward an interval type-2 fuzzy neural network (IT2FNN) to deal with control issues of nonlinear systems with uncertainties. The fuzzy rules of the IT2FNN use interval type-2 triangular fuzzy sets to account for antecedent parts and adopt crisp numbers for the corresponding consequents. To effectively cope with uncertainties in the systems, a sliding-mode-control theory-based approach with new parameter learning rules is proposed to update the IT2FNN. The overall stability of the proposed methodology is also proved by using appropriate Lyapunov functions. Finally, the proposed method is applied to control the angular position of an inverted pendulum system. Simulation results indicate that, compared to a conventional proportional-derivative controller, the IT2FNN with the proposed learning rules can eliminate the uncertainties in performance and efficiently track the angle trajectory as desired.

This paper describes the design and development of an automated construction quality assessment robot system (QuicaBot) for hollowness, crack, evenness, alignments and inclination problems. To the best of our knowledge, this work is the first attempt to pave the way towards a fully autonomous robotic system for post construction quality assessment of buildings. The main goal of the novel robot is twofold: to systematize the manual inspection work through automation resulting in more reliable and objective inspection reports, and to speed up the inspection process resulting in a cost reduction. Based-on our initial on-site tests, the developed robot increases the overall efficiency in all the aforementioned five problems.

As being one of the most bankable flying objects, quadcopters have already proved their usefulness in both civilian and military applications. On the other hand, their control is still challenging as, unlike from ground robots, they do not have enough friction forces to stabilize their motion. Since they have under-actuated, highly nonlinear and coupled dynamics, and have to operate under noisy conditions, model-free control algorithms are more than welcome. In this paper, type-2 Takagi-Sugeno-Kang fuzzy logic controllers (TSK-FLCs) are tuned by reinforcement learning (RL), and implemented on quadcopters. The controllers are successfully tested on a variety of pursuit-evasion scenarios which provide a suitable basis for the utilization of RL since they consist of conflicting aims. A number of comparative results are presented for several case studies with different quadcopters, different initial points and under noisy conditions.

Fuzzy logic controllers (FLCs) have extensively been used for the autonomous control and guidance of unmanned aerial vehicles (UAVs) due to their capability of handling uncertainties and delivering adequate control without the need for a precise, mathematical system model which is often either unavailable or highly costly to develop. Despite the fact that non-singleton FLCs (NSFLCs) have shown more promising performance in several applications when compared to their singleton counterparts (SFLCs), most of UAV applications are still realized by using SFLCs. In this paper, we explore the potential of both standard and the recently introduced centroid based NSFLCs, i.e., Sta-NSFLC and Cen-NSFLC, for the control of a quadcopter UAV under various input noise conditions using different levels of fuzzifier, and a comparative study has been conducted using the three aforementioned FLCs. We present a series of simulation-based experiments, the simulation results show that the control performances of NSFLCs are better than those of SFLC, and the Cen-NSFLC outperforms the Sta-NSFLC especially under highly noisy conditions.

The aim of this paper is to present experimental validation results to show the design simplicity of single input interval type-2 (IT2) fuzzy PID (FPID) controllers by evaluating their performance on a real-time 3 DOF helicopter testbed. In this study, we briefly show that the presented analytical design approach gives the opportunity to construct the IT2 fuzzy mappings by tuning a single parameter which constructs the footprint of uncertainty (FOU) of the IT2 fuzzy sets. Then, by employing these theoretical analyses, various single input IT2 FPID (SIT2-FPID) controllers are designed to solve the control problem of the 3 DOF helicopter. Through extensive and comparative experimental analysis, we analyze the IT2 fuzzy control system performances and validate the effect of the FOU parameter on the controller characteristics. The experimental results show that, having neither a priori knowledge about the mathematical model of the system nor its parameters, the SIT2-FPID is able to achieve a satisfactory control performance over nonlinear working regions in the presence of noise and unmodelled disturbance dynamics. We believe that the experimental validation of the SIT2-FPID controllers' theoretical analyses will open the door to a wider deployment of SIT2-FPIDs to real world control engineering applications.

This paper is made up of a series of performance evaluations of computer vision algorithms, namely detectors and descriptors. The OpenCV 3.1 implementations of these algorithms were used for these evaluations. The main purpose behind these evaluations was to determine the best algorithms to use for a UAV guidance system.

This paper discusses the formation landing problem of quadrotor UAVs, which is considered as a UAV leader-follower problem, avoiding static obstacles. Rapidly-exploring random tree algorithm is used to generate the path for the leader UAV firstly. In particular, specifics of tree-grow including nodes selection, parent node connection, feasible and optimal path generation are explained. Given the leader UAV position, path finding for the follower UAV is conducted to avoid both static obstacles and the leader quadrotor. Based on the intensive simulations, which are conducted in ROS-Gazebo environment, the proposed framework is considered to be applicable in real-time formation landing of quadrotor UAVs.

Autonomous navigation in an unknown or uncertain environment is one of the challenging tasks for unmanned aerial vehicles (UAVs). In order to address this challenge, it is necessary to have sophisticated high level control methods that can learn and adapt themselves to changing conditions. One of the most promising frameworks for such a purpose is reinforcement learning. In this paper, a novel model-based reinforcement learning algorithm, TEXPLORE, is developed as a high level control method for autonomous navigation of UAVs. The developed approach has been extensively tested with a quadcopter UAV in ROS-Gazebo environment. The experimental results show that our method is able to learn an efficient trajectory in a few iterations and perform actions in real-time. Moreover, we show that our approach significantly outperforms Q-learning based method. To the best of our knowledge, this is the first time that TEXPLORE has been developed to achieve autonomous navigation of UAVs.

Utilizing construction quality standards for almost perfect building projects ensures future marketability of projects, customer satisfaction and maximization of asset value. This paper describes a novel robot system to autonomously assess the post-construction quality of buildings which is currently done manually by using human inspectors. However, manual inspection always has the disadvantages, such as labile inspection accuracy, being time consuming and indistinct recording. As a novel solution to the aforementioned drawbacks, a mobile robot is equipped with a laser scanner, a thermal camera, an inclinometer and a RGB camera to achieve an autonomous assessment system. This proposed system can assess five types of defects: evenness, alignment, cracks, hollowness, and inclination. A movable trolley with different mechanisms are designed to mount and integrate all these sensors. Its mechanical design with four motors and one linear actuator, which are installed to increase the measurement range of sensors, is also presented. The experimental tests show that the proposed system has a great potential in construction quality assessment area in building sector.

This paper deals with the problem of historical feature selection for appearance model update in feature-based tracking. In particular, we convert the feature selection procedure into a ranking process where the top-N keypoint features are ranked based on the tracking histories. To the best of our knowledge, for the first time in this paper, a consensus feature prior (CFP) recommendation system is proposed that allows us to learn and update the appearance model online within a limited model size. Furthermore, the ranking scores obtained from the proposed recommendation system also provide a conviction of recovering the tracking after its failure. Extensive experiments (more than 600,000 frames) have been done by strictly following the Visual Tracking Benchmark v1.0 protocol. The results demonstrate that our method outperforms most of the state-of-art trackers both in terms of speed and accuracy.

Object tracking over image sequences plays an remarkably crucial role in several computer vision applications, interalia, automated video surveillance, unmanned aerial vehicles and 3D reconstruction. In this paper, a novel, accurate, robust and recoverable real-time feature-based tracking framework is presented. The appearance modelling consists of a local and global layer. We propose a recommended keypoint-aware (RKA) tracker, which is fast and accurate, for the former, while the latter employs support vector machine (SVM) to determine the object and background, so that the RKA tracker can be recovered under possible target losing circumstances. Furthermore, the RKA tracker converts the tracking problem into the ranking of samples which provides a score of tracking confidence. Therefore, the priority switching between the local layer and global layer dependent upon the score becomes valid. Extensive experiments have been done by strictly following the visual tracking benchmark v1.0 protocol. The results demonstrate that the proposed novel method outperforms the state-of-the-art trackers in terms of robustness, speed and accuracy.

Differing from previous studies, where sliding mode control theory-based rules are proposed for only the consequent part of the network, the developed algorithm in this paper applies fully sliding mode parameter update rules for both the premise and consequent parts of the interval type-2 fuzzy neural networks. The stability of the proposed learning algorithm has been proved by using an appropriate Lyapunov function. Then, the performance of the proposed learning algorithm is tested on the identification of wing flutter data set available online as a benchmark system and the prediction of Mackey-Glass chaotic system. The simulation results indicate that the proposed algorithm is significantly faster than the gradient-based methods as well as providing a slightly better identification performance. The reason for the fast convergence is that the proposed parameter update rules do not have any matrix manipulations which makes them simple to be implemented in real-time systems. In addition, the responsible parameter for sharing the contributions of the lower and upper parts of the type-2 fuzzy membership functions is also tuned. Another prominent feature of the proposed learning algorithm is to have a closed form which makes it easier to implement than the other existing learning methods, e.g. gradient-based methods.

In this paper, an alternative stability analysis for the Levenberg-Marquardt (LM) algorithm is proposed for the training of Type-2 fuzzy neural networks (T2FNNs). The benefit of the proposed stability analysis is that it does not require any eigenvalues computations, and hence it is simpler when compared to the existing stability analysis studies in literature.

Modeling stage of complex aerial vehicles requires tremendous man power and expertise because of their highly nonlinear dynamics as well as complex inter couplings. In this paper, we investigate a model free controller design which benefits from type-2 fuzzy neural networks with elliptic type-2 fuzzy membership functions to control a 2-DOF helicopter without the need of a priori knowledge about the mathematical model for the system. In order to train the parameters of the consequent part of the type-2 fuzzy neural network, a cost function based on the integral of the square of the sliding surface is defined. The solution of this cost function is an optimal training algorithm for the parameters of the consequent part of the type-2 fuzzy neural network. The simulation results show that having neither a priori knowledge about the mathematical model of the system nor its parameters, the proposed control algorithm is able to track the reference signals for both yaw and pitch angles by eliminating the steady state error. In addition, the simulation results show the superiority of the proposed controller over its type-1 counterpart in the presence of measurement noise in the system.

On one hand, we are aware of the fact that quadrotors have been becoming a part of our daily life day to day; on the other hand, their control is still a challenging task as, unlike from the ground vehicles, they do not have enough friction forces to stabilize their motion. What is more, quadrotor's six DOF motion (three translational and three rotational) is controlled by varying only the speeds of its four independent rotors, resulting in under-actuated, highly nonlinear and coupled dynamics. In this paper, conventional proportional-derivative (PD), Mamdani-type fuzzy and TSK-type fuzzy neural network-based controllers have been designed, and their performance have been compared based on both control accuracy and control effort. A realistic trajectory, which is feasible regarding the input constraints of the quadrotor, is generated to test the accuracy and efficiency of the proposed methods. Realistic uncertainties, such as wind and gust conditions, are also given to the system to demonstrate the robustness of the controllers in real-time operation. The adaptive fuzzy-neural controller gives the most accurate trajectory tracking results for a 4D trajectory reducing the error by a factor of 4 when compared to the conventional PD and fuzzy controller although the control effort increases only by 10%.

A novel type-2 fuzzy membership function (MF) in the form of an ellipse has recently been proposed in literature, the parameters of which that represent uncertainties are de-coupled from its parameters that determine the center and the support. This property has enabled the proposers to make an analytical comparison of the noise rejection capabilities of type-1 fuzzy logic systems with its type-2 counterparts. In this paper, a sliding mode control theory-based learning algorithm is proposed for an interval type-2 fuzzy logic system which benefits from elliptic type-2 fuzzy MFs. The learning is based on the feedback error learning method and not only the stability of the learning is proved but also the stability of the overall system is shown by adding an additional component to the control scheme to ensure robustness. In order to test the efficiency and efficacy of the proposed learning and the control algorithm, the trajectory tracking problem of a magnetic rigid spacecraft is studied. The simulations results show that the proposed control algorithm gives better performance results in terms of a smaller steady state error and a faster transient response as compared to conventional control algorithms.

In this paper, a fully nonlinear aircraft dynamic model in the presence of wind is introduced. Assuming the wind vortex hypothesis, the equations of motion are formulated in a novel nonlinear control system form that is affine in control input for the chosen aircraft model. A sliding mode controller is subsequently developed where the position of the throttle is used to control the air velocity and the aileron, elevator, and rudder deflections are employed to control the rest of the dynamics. It is shown that the full state converges to the desired values even in the presence of the uncertainties imposed by the wind. The applicability of the controller design is illustrated through simulations in the presence of uncertainties.

As a model is only an abstraction of the real system, unmodeled dynamics, parameter variations, and disturbances can result in poor performance of a conventional controller based on this model. In such cases, a conventional controller cannot remain well tuned. This paper presents the control of a spherical rolling robot by using an adaptive neuro-fuzzy controller in combination with a sliding-mode control (SMC)-theory-based learning algorithm. The proposed control structure consists of a neuro-fuzzy network and a conventional controller which is used to guarantee the asymptotic stability of the system in a compact space. The parameter updating rules of the neuro-fuzzy system using SMC theory are derived, and the stability of the learning is proven using a Lyapunov function. The simulation results show that the control scheme with the proposed SMC-theory-based learning algorithm is able to not only eliminate the steady-state error but also improve the transient response performance of the spherical rolling robot without knowing its dynamic equations.

Automatic guidance of agricultural vehicles would lighten the job of the operator, while accuracy is needed to obtain an optimal yield. Accurately navigating a tractor consists of controlling different dynamic subsystems (steering and speed). Instead of modeling the subsystem interaction prior to model-based control, we have developed a control algorithm which learns the interactions on-line from the measured feedback error. In this approach, a PD controller is working in parallel with a type-2 fuzzy neural network. While the former ensures the stability of the related subsystem, the latter learns the system dynamics and becomes the leading controller. In this study, two combinations of a PD controller with a type-2 fuzzy neural network are implemented: one for the yaw dynamics and one for the traction dynamics. The interactions between these subsystems are thus not taken into account explicitly, but considered as disturbances to be handled by the subsystem controllers. A novel sliding mode control theory-based learning algorithm is used to train the type-2 fuzzy neural networks, and the convergence of the parameters is shown by using a Lyapunov function.

Production machines, especially in agriculture, with higher efficiencies will be very important in the future because of the limited agricultural areas in the world and the high energy and labor costs. In order to increase the capacity of agricultural machinery, one can think to further increase the size of the machines. However, the limits in this direction will soon be reached as there is a maximum size to still allow road transport. On the other hand, energy costs are constantly increasing, such that the energy use should be minimized. A better option would be to use advanced learning algorithms, which can learn the system dynamics online, for the control of the production machines in order to increase their effectiveness. In this study, a Takagi-Sugeno-Kang type-2 fuzzy neural network with a sliding mode control theory-based learning algorithm is proposed for the control of the yaw dynamics of an autonomous tractor which includes various uncertainties, disturbances and nonlinearities, especially coming from the hydraulic sub systems. Experimental results show the efficacy and the efficiency of the proposed learning algorithm.

Uncertainty is an inevitable problem in real-time industrial control systems and, to handle this problem and the additional one of possible variations in the parameters of the system, the use of sliding mode control theory-based approaches is frequently suggested. In this paper, instead of using a conventional sliding mode controller, a sliding mode control theory-based learning algorithm is proposed to train the fuzzy neural networks in a feedback-error-learning structure. The parameters of the fuzzy neural network are tuned by the proposed algorithm not to minimize the error function but to ensure that the error satisfies a stable equation. The parameter update rules of the fuzzy neural network are derived, and the proof of the learning algorithm is verified by using the Lyapunov stability method. The proposed method is tested on a real-time servo system with time-varying and nonlinear load conditions.

In this paper, the noise reduction property of type-2 fuzzy logic (FL) systems (FLSs) (T2FLSs) that use a novel type-2 fuzzy membership function is studied. The proposed type-2 membership function has certain values on both ends of the support and the kernel and some uncertain values for the other values of the support. The parameter tuning rules of a T2FLS that uses such a membership function are derived using the gradient descend learning algorithm. There exist a number of papers in the literature that claim that the performance of T2FLSs is better than type-1 FLSs under noisy conditions, and the claim is tried to be justified by simulation studies only for some specific systems. In this paper, a simpler T2FLS is considered with the novel membership function proposed in which the effect of input noise in the rule base is shown numerically in a general way. The proposed type-2 fuzzy neuro structure is tested on different input-output data sets, and it is shown that the T2FLS with the proposed novel membership function has better noise reduction property when compared to the type-1 counterparts.

Type-2 fuzzy logic systems are proposed as an alternative solution in the literature when a system has a large amount of uncertainties and type-1 fuzzy systems come to the limits of their performances. In this study, an adaptive type-2 fuzzy-neuro system is designed for the position control of a servo system with an intelligent sensor. The sensor gives different resistance values with respect to the stretch of it, and it is supposed to be used in an robotic arm position measurement system. These kinds of sensors can be used in human-assistance robots that have soft surfaces in order not to damage the humans. However, these sensors have time-varying gains and uncertainties that are not very easy to handle. Moreover, they generally have a hysteresis on their input-output relations. The simulation results show that the control algorithm developed gives better performances when compared to conventional type-1 fuzzy controllers on such a highly nonlinear, uncertain system.

The control of an antilock braking system (ABS) is a difficult problem due to its strongly nonlinear and uncertain characteristics. To overcome this difficulty, the integration of gray-system theory and sliding-mode control is proposed in this paper. This way, the prediction capabilities of the former and the robustness of the latter are combined to regulate optimal wheel slip depending on the vehicle forward velocity. The design approach described is novel, considering that a point, rather than a line, is used as the sliding control surface. The control algorithm is derived and subsequently tested on a quarter vehicle model. Encouraged by the simulation results indicating the ability to overcome the stated difficulties with fast convergence, experimental results are carried out on a laboratory setup. The results presented indicate the potential of the approach in handling difficult real-time control problems.

In this paper, a grey neuro-adaptive control algorithm is suggested for Antilock Braking Systems (ABS). The concept of grey system theory, which has a certain prediction capability, offers an alternative approach to conventional control methods. A multilayer neural network and a grey predictor, GM(1,1) model, are combined in the approach proposed in the paper. The grey neural network controller is examined under several different operating conditions and it is shown that the proposed control algorithm anticipates the upcoming values of wheel slip and optimal wheel slip, and takes the necessary action to keep the wheel slip at the desired value. The simulation results indicate that the proposed controller has the ability to control the nonlinear system accurately with little oscillations and with no steady-state error.

Antilock braking system (ABS) exhibits strongly nonlinear and uncertain characteristics. To overcome these difficulties, robust control methods should be employed. In this paper, a grey sliding mode controller is proposed to track the reference wheel slip. The concept of grey system theory, which has a certain prediction capability, offers an alternative approach to conventional control methods. The proposed controller anticipates the upcoming values of wheel slip, and takes the necessary action to keep wheel slip at the desired value. The control algorithm is applied to a quarter vehicle model, and it is verified through simulations indicating fast convergence and good performance of the designed controller. Simulated results are validated on real time applications using a laboratory experimental setup.