This paper presents the first AI/ML system for automating building damage assessment in uncrewed aerial systems (sUAS) imagery to be deployed operationally during federally declared disasters (Hurricanes Debby and Helene). In response to major disasters, sUAS teams are dispatched to collect imagery of the affected areas to assess damage; however, at recent disasters, teams collectively delivered between 47GB and 369GB of imagery per day, representing more imagery than can reasonably be transmitted or interpreted by subject matter experts in the disaster scene, thus delaying response efforts. To alleviate this data avalanche encountered in practice, computer vision and machine learning techniques are necessary. While prior work has been deployed to automatically assess damage in satellite imagery, there is no current state of practice for sUAS-based damage assessment systems, as all known work has been confined to academic settings. This work establishes the state of practice via the development and deployment of models for building damage assessment with sUAS imagery. The model development involved training on the largest known dataset of post-disaster sUAS aerial imagery, containing 21,716 building damage labels, and the operational training of 91 disaster practitioners. The best performing model was deployed during the responses to Hurricanes Debby and Helene, where it assessed a combined 415 buildings in approximately 18 minutes. This work contributes documentation of the actual use of AI/ML for damage assessment during a disaster and lessons learned to the benefit of the AI/ML research and user communities.

This survey article investigates human-initiated touch interaction with a touch surface on a robot, reviewing 37 papers, with seven papers involving unprompted touch interaction and 30 papers involving prompted touch interaction. The 37 papers are examined in depth as to whether the touch interaction was unprompted, which of the nine touch surface characteristics (appearance, haptic feedback, light, material, movement, shape, size, texture, and temperature) were examined, and the methods used to measure the effects of the touch surface characteristics. This survey makes three findings. One is that no unprompted touch interaction provides an understanding of what factors influence its initiation. Two, four studies involving prompted touch suggest two methods for inferring influence on causation from attentiveness or engagement. Three, the touch surface characteristics involved in human-initiated touch interaction consist of nine characteristics, and the limited set of four touch surface characteristics presented within the unprompted touch literature should be expanded to the nine characteristics. The survey sets the foundation for future research into unprompted touch, especially for use in disasters and medical intervention, by (a) expanding the demographics, groups, and scenarios by investigating unprompted touch, (b) investigating the triggers and affordances associated with unprompted touch, and (c) investigating the interaction and affect touch surface characteristics have on one another and the touch interaction when combined.

This work presents the first quantitative study of alignment errors between small uncrewed aerial systems (sUAS) georectified imagery and a priori building polygons and finds that alignment errors are non-uniform and irregular, which negatively impacts field robotics systems and human-robot interfaces that rely on geospatial information. There are no efforts that have considered the alignment of a priori spatial data with georectified sUAS imagery, possibly because straight-forward linear transformations often remedy any misalignment in satellite imagery. However, an attempt to develop machine learning models for an sUAS field robotics system for disaster response from nine wide-area disasters using the CRASAR-U-DROIDs dataset uncovered serious translational alignment errors. The analysis considered 21,608 building polygons in 51 orthomosaic images, covering 16787.2 Acres (26.23 square miles), and 7,880 adjustment annotations, averaging 75.36 pixels and an average intersection over union of 0.65. Further analysis found no uniformity among the angle and distance metrics of the building polygon alignments, presenting an average circular variance of 0.28 and an average distance variance of 0.45 pixels2, making it impossible to use the linear transform used to align satellite imagery. The study’s primary contribution is alerting field robotics and human-robot interaction (HRI) communities to the problem of spatial alignment and that a new method will be needed to automate and communicate the alignment of spatial data in sUAS georectified imagery. This paper also contributes a description of the updated CRASAR-U-DROIDs dataset of sUAS imagery, which contains building polygons and human-curated corrections to spatial misalignment for further research in field robotics and HRI.

This paper details four principal challenges encountered with machine learning (ML) damage assessment using small uncrewed aerial systems (sUAS) at Hurricanes Debby and Helene that prevented, degraded, or delayed the delivery of data products during operations and suggests three research directions for future real-world deployments. The presence of these challenges is not surprising given that a review of the literature considering both datasets and proposed ML models suggests this is the first sUAS-based ML system for disaster damage assessment actually deployed as a part of real-world operations. The sUAS-based ML system was applied by the State of Florida to Hurricanes Helene (2 orthomosaics, 3.0 gigapixels collected over 2 sorties by a Wintra WingtraOne sUAS) and Debby (1 orthomosaic, 0.59 gigapixels collected via 1 sortie by a Wintra WingtraOne sUAS) in Florida. The same model was applied to crewed aerial imagery of inland flood damage resulting from post-tropical remnants of Hurricane Debby in Pennsylvania (436 orthophotos, 136.5 gigapixels), providing further insights into the advantages and limitations of sUAS for disaster response. The four challenges (variation in spatial resolution of input imagery, spatial misalignment between imagery and geospatial data, wireless connectivity, and data product format) lead to three recommendations that specify research needed to improve ML model capabilities to accommodate the wide variation of potential spatial resolutions used in practice, handle spatial misalignment, and minimize the dependency on wireless connectivity. These recommendations are expected to improve the effective operational use of sUAS and sUAS-based ML damage assessment systems for disaster response.

This paper introduces and solves a visibility-based escort planning problem. This novel problem, which is closely related to the well-researched family of visibility-based pursuit-evasion problems in robotics, entails an escort agent tasked with escorting a vulnerable agent, called the VIP, in a 2-dimensional environment. The escort protects the VIP from adversaries that pose line-of-sight threats. We describe a correct and complete planning algorithm whose inputs are a simply-connected polygonal map of the environment, starting locations for the escort and the VIP, along with a goal location to which the VIP agent should be safely moved. The algorithm computes trajectories for the escort and VIP which allow the VIP to reach its goal without coming into the line-of-sight of the adversary at any time. During the execution of these trajectories, the adversary is allowed to move along any continuous path that does not enter into the line-of-sight of the escort. The algorithm proceeds by dividing the environment into a collection of conservative regions and planning the escort's movements as a sequence of these regions via breadth-first search over an information graph. The trajectory of the VIP can then be constructed by tracing the ‘safe zones' swept out by the escort's trajectory. We describe an implementation of this algorithm and present computed examples of escort agent strategies in diverse environments.