This paper introduces a prioritized path-planning algorithm for heterogeneous tasks performed by multiple unmanned ground vehicles (UGVs) in agricultural environments. The algorithm considers varying robot priorities, thereby extending the traditional multi-agent path finding (MAPF) approach. The proposed algorithm is evaluated in scenarios occurring during representative agricultural operations: harvesting and transportation. An experimental validation is conducted in agriculture-like settings by using multiple simultaneous localization and mapping systems and navigation systems. The results revealed that the path of agent1, which was assigned the highest priority in both the indoor and outdoor environments, was shortened considerably (3.38 m, 3.6 m, and 5.6 m, respectively). Especially in the face scenario, the sum of changes in distance, calculated using the proposed algorithm was negative, meaning that traffic congestion in the multi-robot system used in the experiment was alleviated without the need for inter-robot communication.

As the interest in ecosystem protection increases, many researchers are focusing on tracking flying insects to preserve biodiversity. Invasive alien species (IAS) such as Vespa velutina nigrithorax require extensive consideration in this regard owing to size and weight limitations. In this paper, we propose and experimentally validate an unmanned aerial vehicle (UAV)-based trilateration system for localization and tracking of flying insects. The proposed tracking algorithm is based on the use of multiple omnidirectional antennas, and it improves the tracking accuracy and shortens the tracking time compared to those of the existing unidirectional antennas. In addition, we reduce the measurement error and noise by applying a finite impulse response (FIR) filter to the received signal strength (RSS) emitted from the radio-telemetry transmitter. A field experiment involving the proposed trilateration system was conducted in an apiary. This experiment was divided into three sub-experiments as follows: 1) behavior experiment, 2) ground truth experiment, and 3) localization and tracking experiment. The proposed system contributes to the research on UAV-based tracking systems that respond to the speed of spreading of invasive V. velutina.

Field robot systems have recently been applied in a wide range of research fields. Further automation, development, and activation of such systems require cooperation among heterogeneous robots. Classical control theory is inefficient in managing large-scale complex dynamic systems. Therefore, a discrete-event system based on the supervisory control theory must be introduced to overcome this limitation. In this article, we propose a hybrid system-based hierarchical control architecture using a supervisory control-based high-level controller and a traditional control-based low-level controller. The hybrid system and its dynamics are modeled through a formal method, called hybrid automata, and the behavior specifications are designed to express the control objectives for cooperation. In addition, modular supervisors that are more scalable and maintainable than a centralized supervisory controller were synthesized. The proposed hybrid system and hierarchical control architecture were implemented, validated, and evaluated for performance through a physics-based simulator and field tests. The experimental results confirmed that the robot team satisfied the given specifications and presented systematic results, validating the efficiency of the proposed control architecture.

This study proposes an intelligent spraying system with semantic segmentation of fruit trees in a pear orchard. A fruit tree detection system was developed using the SegNet model, a semantic segmentation structure. The system is trained with images categorized into five distinct classes. The learned deep learning model performed with an accuracy of 83.79%. Further, we fusion depth data from an RGB-D camera to prevent the tree in the background from being detected. To operate the nozzles, each image captured from the camera is separated lengthwise into quarters and mapped to the nozzles. Then, the nozzle was opened when the area of fruit trees in each zone exceeded 20%. Two types of field experiments were performed in a pear orchard to verify the effectiveness of our system. From the results obtained, we can confirm the satisfactory performance of our deep learning-based intelligent spraying system. It is expected that the introduction of this system to actual farms will signicantly reduce the amount of pesticide used and will make the work environment safer for farmers.

This paper describes a wire-driven parallel robotic system for the maintenance of offshore wind turbines. This robotic system vertically climbs on towers or blades, and performs cleaning and inspection using waterjets and phased array ultrasonic testing (PAUT) devices, respectively. For a detailed mechanical design, mechanisms for (1) climbing towers or blades, (2) transition motion between towers and blades, (3) gripping blades, and (4) contact motion between PAUT devices and blade surfaces are proposed so that the robotic system can maneuver on towers and blades. For the motion analysis of the robotic system, nonholonomic constraints are analyzed and utilized in height and attitude control schemes, which are proposed to adapt varying nonholonomic constraints of blade surface and achieve the climbing motion on towers and blades. To validate these control schemes, we conduct experiment and analyze the performance of the proposed control schemes. The results show that the height and attitude control schemes for the robotic system do achieve the control goals and overcome the control challenges.

Robot has been widely used for industrial application in the flexible manufacturing system. The time-consuming teaching process by skilled engineers is inevitable to teach real robot manipulators for generating the robot program and recording locations which the robot should follow for the given task. The off-line programming (OLP) method is proposed to generate the robot program from CAD format data without the teaching process. It needs a registration process to find the translational and rotational relations between the reference frame of real robot and the reference frame in CAD format data. This paper presents the simple registration method between real robot and workpiece in virtual environment. Also a method to assign position and orientation of TCP (Tool Center Point) in real robot is presented when the target point is selected in the virtual environment by the user. The proposed methods are evaluated by the simulation with the commercial industrial robot for placing tool on the surface of the workpiece which is given in STL format data.

We propose a novel whole-body multi-modal semiautonomous teleoperation framework for mobile manipulator systems, which consists of: 1) Motion capture and whole-body motion mapping to allow the operator to intuitively teleoperate the mobile manipulator without being constrained by the master interface while also fully exploiting whole-body dexterity; 2) Slave robot autonomous control to allow the mobile manipulator to optimally track the operator's whole-body command, while taking into account the user-slave kinematic dissimilarity (slave robot's joint limit, joint velocity limit, and singularity); and 3) Visuo-haptic-vestibular feedback with HMD (Head Mounted Display) for 3D visual information, wearable cutaneous haptic device for manipulation force feedback, and actuated chair for vestibular feedback to reduce HMD-induced motion sickness. Performance of the proposed framework is validated with simulation of a ROV (remotely operated vehicle) manipulator system and some preliminary user studies.

For teleoperation of multiple mobile robots (e.g., unmanned aerial vehicles), a better situational awareness of remote environments is crucial for good performance. Visual and force feedback are the most common ways to perceive the environments accurately using a vision system and a haptic device, respectively. In outdoor environments, however, it is difficult to use adequate/accurate sensors (e.g., global localization system) for haptic feedback of the remote environments as well as the slave robots. In this paper, we propose a visual and force feedback method to enhance a human operator's situational awareness in multi-robot teleoperation by fabricating a global view of the multi-robot system and transmitting velocity information of one, respectively, using only local information of the robots. The proposed visual feedback method, then, is evaluated via two psychophysical experiments: maneuvering and searching tests. Several measures are also proposed to assess the human operator's maneuverability and situational awareness quantitatively.

Recently, offshore plants are demanded to secure natural resources more and more and it is strongly required unmanned ones for safety of human operators and less running costs. In this paper, a practical multi-user teleoperation system is proposed for monitoring, inspection, operation, and maintenance of the unmanned offshore plants. The proposed system is developed to control mobile manipulator in a cooperative way among multiple human operators for better performance. For this, two control schemes, hand-eye coordination and disjoint axes were introduced for easier intuitive control of the mobile manipulator and for better cooperative control among multi-user, respectively. A well-known passivity-based approach, the time-domain passivity approach was, in addition, adopted to maintain system stability. And then, the proposed multi-user teleoperation system was evaluated via a human-centered method with several quantitative metrics regarding task completion time and interaction forces. Experimental results showed that the proposed multi-user teleoperation system has the benefit in tasks requiring less task completion time and interaction forces.

In this paper, we investigate the effect of haptic cueing on a human operator's performance in the field of bilateral teleoperation of multiple mobile robots, particularly multiple unmanned aerial vehicles (UAVs). Two aspects of human performance are deemed important in this area, namely, the maneuverability of mobile robots and the perceptual sensitivity of the remote environment. We introduce metrics that allow us to address these aspects in two psychophysical studies, which are reported here. Three fundamental haptic cue types were evaluated. The Force cue conveys information on the proximity of the commanded trajectory to obstacles in the remote environment. The Velocity cue represents the mismatch between the commanded and actual velocities of the UAVs and can implicitly provide a rich amount of information regarding the actual behavior of the UAVs. Finally, the Velocity+Force cue is a linear combination of the two. Our experimental results show that, while maneuverability is best supported by the Force cue feedback, perceptual sensitivity is best served by the Velocity cue feedback. In addition, we show that large gains in the haptic feedbacks do not always guarantee an enhancement in the teleoperator's performance.

In this paper, a novel decentralized control strategy for bilaterally teleoperating heterogeneous groups of mobile robots from different domains (aerial, ground, marine, and underwater) is proposed. By using a decentralized control architecture, the group of robots, which is treated as the slave side, is made able to navigate in a cluttered environment while avoiding obstacles, interrobot collisions, and following the human motion commands. Simultaneously, the human operator acting on the master side is provided with a suitable force feedback informative of the group response and of the interaction with the surrounding environment. Using passivity-based techniques, we allow the behavior of the group to be as flexible as possible with arbitrary split and join events (e.g., due to interrobot visibility/packet losses or specific task requirements) while guaranteeing the stability of the system. We provide a rigorous analysis of the system stability and steady-state characteristics and validate performance through human/hardware-in-the-loop simulations by considering a heterogeneous fleet of unmanned aerial vehicles (UAVs) and unmanned ground vehicles as a case study. Finally, we also provide an experimental validation with four quadrotor UAVs.

The use of multiple unmanned aerial vehicles (UAVs) is increasingly being incorporated into a wide range of teleoperation applications. To date, relevant research has largely been focused on the development of appropriate control schemes. In this paper, we extend previous research by investigating how control performance could be improved by providing the teleoperator with haptic feedback cues. First, we describe a control scheme that allows a teleoperator to manipulate the flight of multiple UAVs in a remote environment. Next, we present three designs of haptic cue feedback that could increase the teleoperator's environmental awareness of such a remote environment. These cues are based on the UAVs' i) velocity information, ii) proximity to obstacles, and iii) a combination of these two sources of information. Finally, we present an experimental evaluation of these haptic cue designs. Our evaluation is based on the teleoperator's perceptual sensitivity to the physical environment inhabited by the multiple UAVs. We conclude that a teleoperator's perceptual sensitivity is best served by haptic feedback cues that are based on the velocity information of multiple UAVs.

In this paper, a novel distributed control strategy for teleoperating a fleet of Unmanned Aerial Vehicles (UAVs) is proposed. Using passivity based techniques, we allow the behavior of the UAVs to be as flexible as possible with arbitrary split and join decisions while guaranteeing stability of the system. Furthermore, the overall teleoperation system is also made passive and, therefore, characterized by a stable behavior both in free motion and when interacting with unknown passive obstacles. The performance of the system is validated through semi-experiments.

We propose a novel haptic teleoperation control framework for multiple unmanned aerial vehicles (UAVs) over the Internet, consisting of the three control layers: 1) UAV control layer, where each UAV is abstracted by, and is controlled to follow the trajectory of, its own kinematic virtual point (VP); 2) VP control layer, which modulates each VP's motion according to the teleoperation commands and local artificial potentials (for inter-VP/VP-obstacle collision avoidance and inter-VP connectivity preservation); and 3) teleoperation layer, through which a remote human user can command all (or some) of the VPs' velocity while haptically perceiving the state of all (or some) of the UAVs over the Internet. Master passivity/slave-stability and some asymptotic performance measures are proved. Semi-experiment results are presented to validate the theory.

In this paper, we investigate the maneuverability performance of human teleoperators on multi-robots. First, we propose that maneuverability performance can be assessed by a frequency response function that jointly considers the input force of the operator and the position errors of the multi-robot system that is being maneuvered. Doing so allows us to evaluate maneuverability performance in terms of the human teleoperator's interaction with the controlled system. This allowed us to effectively determine the suitability of different haptic cue algorithms in improving teleoperation maneuverability. Performance metrics based on the human teleoperator's frequency response function indicate that maneuverability performance is best supported by a haptic feedback algorithm which is based on an obstacle avoidance force.

This paper focuses on psychophysical evaluation of the control scheme developed to optimize the kinesthetic perception during the scaled teleoperation. The control problem is formulated as a multi-objective constrained optimization. The objective function is a metric which quantifies the detection and discrimination capacity of the human operator. The constraints are position tracking accuracy and absolute stability of the scaled teleoperation. Two popular control architectures, i.e., the position-position and the force-position control architectures are considered in this paper. The method of limits is employed in this paper to conduct the psychophysical experiments and evaluation. Results show that the developed control scheme is more effective in increasing the detection and discrimination capacity of human subjects as compared to the traditional transparency-optimized control laws.