The primary objective of this paper is to introduce a new monocular depth estimation (MDE) model targeting under-represented environments using a novel dataset combining synthetic and real images. The proposed model is small and fast to allow use for UAS navigation and data collection in rainforest environments. Prior works on MDEs target outdoor environments while focusing on urban, ground-level viewpoints due to interest in self-driving or autonomous package delivery applications and data availability. However, under-represented environments, such as rainforests, can benefit from targeted, environment-specific MDEs because existing general MDEs cannot adapt to extreme environmental differences, leading to high error rates. Our model is trained using a distinct rainforest dataset that combines images generated using a synthetic dataset pipeline and depth images collected from aerial robot deployments in the Children’s Eternal Rainforest in Costa Rica. The proposed model will allow for improved rainforest navigation without using expensive LIDAR sensors and can improve the navigation of a UAS in rainforest environments by providing more accurate and useful measurements for object manipulation, such as leaf sampling. Our model outperforms MiDaS across the board and has over a 75% improvement, specifically in the relative error metrics, while maintaining a low runtime. The resulting model matches the performance of state-of-the-art monocular depth estimation models designed for common environments, i.e., urban and indoor environments, and outperforms them when used in a rainforest environment.

The primary objective of this study was to utilize state-of-the-art deep learning-based monocular depth estimation models to assist UAS pilots in rainforest canopy data collection and navigation. Monocular depth estimation models provide a complementary technique to other depth measurement and estimation techniques to extend the range and improve measurements. Several state-of-the-art models were evaluated using a novel dataset composed of data from a simulated rainforest environment. In the evaluation, MiDaS outperformed the other models, and a segmentation pipeline was designed using this model to identify the highest areas of the canopies. The segmentation pipeline was evaluated using 1080p and 360p input videos from the simulated rainforest dataset. It was able to achieve an IoU of 0.848 and 0.826 and an F1 score of 0.915 and 0.902 at each resolution, respectively. We incorporated the proposed depth-estimation-based segmentation pipeline into an example application and deployed it on an edge system. Experimental results display the capabilities of a UAS using the segmentation pipeline for rainforest data collection.

This article reviews the human-machine interaction (HMI) technologies used for telemanipulation by small unmanned systems (SUS) with remote manipulators. SUS, including land, air, and sea vehicles, can perform a wide range of reconnaissance and manipulation tasks with varying levels of autonomy. SUS operations involving physical interactions with the environment require some level of operator involvement, ranging from direct control to goal-oriented supervision. Telemanipulation remains a challenging task for all levels of human interaction because the operator and the vehicle are not colocated, and operators require HMI technologies that facilitate manipulation from a remote location. This article surveys the human operator interfacing for over 70 teleoperated systems, summarizes the effects of physical and visual interface factors on user performance, and discusses these findings in the context of telemanipulating SUS. This article is of importance to SUS researchers and practitioners who will directly benefit from HMI implementations that improve telemanipulation performance.

This paper surveys the human-machine interaction technologies supporting the Mission Specialist role in unmanned aerial systems (UASs). The Mission Specialist role is one of three formal human team member roles extracted from the UAS-related literature (the others are Flight Director and Pilot), but unlike the Pilot role, the interface needs have not been established. The interfaces used by 17 micro, small, medium altitude long endurance (MALE) and high altitude long endurance (HALE) platforms are examined to determine (1) what type of user interface technologies are present and/or available; (2) how the Mission Specialist currently or could interact with the user interface technology; and (3) what are the perceived positive and negative aspects of this user interface technology in the context of the UAS human-robot team roles. Micro and small UAVs pose significant user interface limitations for the Mission Specialist role and may produce unintentional interaction conflicts between the Mission Specialist role and the Pilot, potentially resulting in suboptimal performance and loss of robustness. The survey is expected to serve as a reference for future design and refinement of user interfaces for UAS and a foundation for better understanding human-robot interaction in UAS.

No abstract available.

Military course-of-action (COA) diagrams are used to depict battle scenarios and include thousands of unique symbols, complete with additional textual and designator modifiers. We have created a real-time sketch recognition interface that recognizes 485 freely-drawn military course-of-action sym- bols. When the variations (not allowable by other systems) are factored in, our system is several orders of magnitude larger than the next biggest system. On 5,900 hand-drawn symbols, the system achieves an accuracy of 90% when con- sidering the top 3 interpretations and requiring every aspect of the shape (variations, text, symbol, location, orientation) to be correct.

Freehand drawing on a computer screen allows users to provide input through a natural mode of human interaction. With this freedom of expression, however, there exists a paradoxical limitation: the user is bound through the existing interface to the fixed drawing surface. In this work, we overcome this limitation by presenting a surfaceless pen-based interface with an application in the field of sketch recognition. A pilot study was conducted to examine the usability of the surfaceless pen-based interface. Results indicated that learning to use the device is relatively straightforward, but that interaction difficulty increases in a directly proportionally manner with drawing complexity.

No abstract available.