In the realm of Internet of Things (IoT) systems, accurately forecasting the runtime of Inferencing models on heterogeneous devices is paramount for optimizing resource allocation, particularly in the contexts of Model Distributed Inferencing (MDI) and Data Distributed Inferencing (DDI). This paper delves into the application of Gradient Boosting Regression (GBR) as a predictive modeling technique for estimating runtime in both MDI and DDI scenarios. GBR presents an equitable trade-off between interpretability, robustness against noise, and suitability for moderately sized datasets. The study reviews previous research on IoT inference optimization. It accentuates the multifaceted intricacies of device diversity and the significance of model interpretability within MDI and DDI setups. The primary contribution of this research is the novel application of the GBR model to predict machine learning inferencing runtime in MDI and DDI contexts. This approach is invaluable when empirical data is limited and characterizing the behavior of newly introduced devices is imperative. The paper elaborates on the GBR algorithm’s utilization, hyperparameters, and custom loss functions tailored explicitly for MDI and DDI. The results section exemplifies GBR’s performance across various computational regimes, including MDI and DDI. It offers insights into the model’s balance between accuracy and complexity. A performance comparison with prior models underscores GBR’s efficacy in predicting runtime in MDI and DDI. This work contributes to the ongoing discourse on IoT optimization and predictive modeling.

Over the past several years, there has been a large increase to the number of edge sensing devices gathering various types of data, from images to audio. One of the issues however is getting this data to a device that can quickly perform analysis. Limited network and compute are just a few of the issues that will be taken into consideration, when it comes to offloading data for inferencing. Existing solutions to the limits of edge computation includes, local computation w/ compression and pruned models, cloud offloading, and several distributed methods, including data distributed and model distributed inferencing. Previous work has explored the performance of these options using standard HTTP communication between the edge and external devices, now new network communication protocols are explored specifically gRPC. The impact of utilizing a more lightweight communication protocol to efficiently transfer data from the edge to other devices is explored while specifically focusing on potential drawbacks that may occur. gRPC while shown to perform more efficiently than standard HTTP, however does come at the cost of increased CPU utilization which may impact performance at the edge in select cases. Experiments will be conducted using a testbed which was created to allow definition of both edge, mid-range, and cloud computation devices to be defined each with corresponding compute and memory allocations. Additional, control of the network will be managed via the software defined network allowing for multiple bandwidth cases to be tested for performance comparison in various network constrained environments. Results highlight the performance benefits gRPC in each of the cases showing inferencing speed improvements of the previous HTTP implementation. However several of the cases show better improvements than expected.

Alongside advancements in computer related technologies, High Performance Computing (HPC) systems continue to grow in both complexity and scale. As the complexity of hardware components increases so too does the complexity of the software solutions that leverage those resources. It is often difficult or impossible to know whether a distributed application is performing as intended without applying a specific profiling application. Furthermore, errors and adverse performance can go unnoticed until the application is executed at scale. To provide a general mechanism that addresses these types of errors we implement a data analysis pipeline that ingests, stores, and indexes data. This data can then be analyzed and displayed for users to perform analysis at varying levels of granularity. To this end, we have developed multiple hierarchical views that enable workload analysis at varying levels of granularity. The overall approach applies existing tools and leverages monitoring, alerting, ad-hoc analysis, and exploratory analysis to inform parallel HPC workloads.

Alongside advancements in computer related technologies, High Performance Computing (HPC) systems continue to grow in both complexity and scale. As the compute capabilities and space efficiency of these powerful machines continue to improve, there is a correlated increase in complexity which results in increased acquisition costs, node failures, and operational costs. In an effort to address these growing concerns, there have been attempts to improve cost efficiency through the use of data analysis and data monitoring. Facilities use data analysis to understand causes of degraded performance, causes of failure, and requirements for future acquisitions. This information is often obtained through ad-hoc programs. Data monitoring, in turn, is used by HPC facility managers to detect node failures in real-time and decrease downtime, thereby minimizing the impact of failures on operational costs. In this paper we present an application to ingest, store, analyze and display this data at the scale of HPC. Our approach brings monitoring, alerting, ad-hoc analysis, and exploratory analysis into a single integrated solution. Beyond providing support for existing diagnostic data, the analysis pipeline makes it simple to link diagnostic data between multiple data sources. With this linking capability and the features available in the data analysis software stack, the user is able to create interactive, hierarchical views of diagnostic data.