As Artificial Intelligence makes its way to edge devices, it becomes necessary to understand how Learning Enabled Components (LECs) will perform in resource-limited environments. A two-part evaluation procedure is developed here using a combination of testing data to evaluate the LECs' performance, including but not limited to time-domain performance, independent of the target system, for assessing the LEC within the context of system it will be deployed. The target system for this evaluation is a BlueROV2 device simulated in Gazebo using the UUV Simulator package that has been configured to function as an autonomous underwater vehicle (AUV) for infrastructure inspection. The task of the vehicle was to detect and track an underwater pipeline using a side-scan sonar. Nine LEC architectures were constructed in Tensorflow, trained, quantized, and deployed on a Google Edge Tensor Processing Unit (TPU) LEC accelerator for evaluation using generated data and deployed as a hardware-in-the-loop simulation using the BlueROV2 model. It was found that the Mean IoU metric in the evaluations that used sonar data from (simulated) pipes that were smaller than the ones used in the training set had the strongest correlation with the LECs ability to continue to track an underwater pipe during the mission simulations. The size of the model did not increase the time required to load a sonar image on the edge device and interpret the sonar images suggesting that neither model size, nor inference time affected the results of the evaluation. The architecture of a given LEC does not guarantee its performance within the context of an AUV executing a pipe inspection task and, therefore, must be evaluated within the context of the full system to properly characterize its performance.

We present AircraftVerse, a publicly available aerial vehicle design dataset. Aircraft design encompasses different physics domains and, hence, multiple modalities of representation. The evaluation of these designs requires the use of scientific analytical and simulation models ranging from computer-aided design tools for structural and manufacturing analysis, computational fluid dynamics tools for drag and lift computation, battery models for energy estimation, and simulation models for flight control and dynamics. AircraftVerse contains $27{,}714$ diverse air vehicle designs - the largest corpus of designs with this level of complexity. Each design comprises the following artifacts: a symbolic design tree describing topology, propulsion subsystem, battery subsystem, and other design details; a STandard for the Exchange of Product (STEP) model data; a 3D CAD design using a stereolithography (STL) file format; a 3D point cloud for the shape of the design; and evaluation results from high fidelity state-of-the-art physics models that characterize performance metrics such as maximum flight distance and hover-time. We also present baseline surrogate models that use different modalities of design representation to predict design performance metrics, which we provide as part of our dataset release. Finally, we discuss the potential impact of this dataset on the use of learning in aircraft design, and more generally, in the emerging field of deep learning for scientific design. AircraftVerse is accompanied by a datasheet as suggested in the recent literature, and it is released under Creative Commons Attribution-ShareAlike (CC BY-SA) license. The dataset with baseline models are hosted at http://doi.org/10.5281/zenodo.6525446, code at https://github.com/SRI-CSL/AircraftVerse, and the dataset description at https://uavdesignverse.onrender.com/.