Scientific workflows are increasingly characterized by complex task dependencies and large-scale data exchanges, which place significant pressure on the input/output (I/O) systems of traditional Workflow Engines (WFEs). These challenges are particularly evident in data-intensive and real-time processing contexts, where conventional disk-based I/O mechanisms often become performance bottlenecks. This paper presents an approach to enhancing the DAGonStar scientific workflow engine by integrating CAPIO, a middleware designed to support memory-based streaming I/O. The integration combines DAGonStar’s orchestration capabilities with CAPIO’s efficient data handling to better support workflows operating on continuous or large-scale datasets. We describe the architectural modifications introduced to enable this collaboration and provide an analysis of the resulting system. The proposed solution aims to improve the responsiveness and flexibility of scientific workflows by streamlining data transfers and simplifying task coordination. This work contributes to the evolution of workflow systems toward more efficient and scalable models for scientific computing. • Integration of memory-based streaming I/O into scientific workflow engines. • Automated generation of synchronization rules through workflow dependency analysis of DAGonStar. • Enhanced pipeline’s tasks execution efficiency via system call interception through the usage of CAPIO. • Benchmark evaluation showing up to 33% reduction in execution time with DAGonCAPIO. • Support for both local batch and SLURM-based distributed executions.

Global changes are increasing the frequency and intensity of extreme weather events, posing challenges for forecasting localized phenomena with sub-grid resolution. The Hi-WeFAI project addresses this by combining high-performance computing, Federated Artificial Intelligence, and heterogeneous sensor networks to improve short-term precipitation forecasting and flood nowcasting. In this paper, we present preliminary results using a transformer-based radar prediction model coupled with a flood model to generate high-resolution early warning maps. The results from the Naples pilot site improved accuracy and detail, highlighting the potential of the hybrid AI and HPC approach to support quasi-real-time decision-making and disaster risk reduction.

In this paper, we propose a novel agentic AI system called XPF, which enables users to create "agents" using just natural language, where each agent is capable of executing complex, real-world business workflows in an accurate and reliable manner. XPF provides an interface to develop and iterate over the agent creation process and then deploy the agent in production when satisfactory results are produced consistently. The key components of XPF include: (a) planner, which leverages LLM to generate a step-by-step plan, which can further be edited by a human (b) compiler, which leverages LLM to compile the plan into a flow graph (c) executor, which handles distributed execution of the flow graph (using LLM, tools, RAG, etc.) on an underlying cluster and (d) verifier, which helps in verification of the output (through human generated tests or auto-generated tests using LLM). We develop five different agents using XPF and conduct experiments to evaluate one particular aspect i.e. difference in accuracy and reliability of the five agents with "human-generated" vs "auto-generated" plans. Our experiments show that we can get much more accurate and reliable response for a business workflow when step-by-step instructions (in natural language) are given by a human familiar with the workflow, rather than letting the LLM figure out the execution plan steps. In particular, we observe that "human-generated" plan almost always gives 100% accuracy whereas "auto-generated" plan almost never gives 100% accuracy. In terms of reliability, we observe through Rouge-L, Blue and Meteor scores, that the output from "human-generated" plan is much more reliable than "auto-generated" plan.

Marine litter detection is crucial for environmental monitoring, yet the imbalance in existing datasets limits model performance in identifying various types of waste accurately. This paper presents an efficient data augmentation pipeline that combines generative diffusion models (e.g., Stable Diffusion) and Large Language Models (LLMs) to expand the G-Litter dataset, a marine litter dataset designed for autonomous detection in heterogeneous environments. Leveraging scalable diffusion models for image generation and Alpaca LLMs for diverse prompt generation, our approach augments underrepresented classes by generating over 200 additional images per class, significantly improving the dataset’s balance. Training G-Litter augmented dataset using YOLOv8 for object detection demonstrated an increase in detection performance, improving recall by 7.82% and mAP50 by 3.87% (compared with baseline results). This study emphasizes the potential for combining generative AI with HPC resources to automate data augmentation on large-scale, unstructured datasets, particularly in edge computing contexts for real-time marine monitoring. The models were tested on real videos captured during simulated missions, demonstrating a superior ability to detect submerged objects in dynamic scenarios. These results highlight the potential of generative AI techniques to improve dataset quality and detection model performance, laying the foundation for further expansion in real-time marine monitoring.

In the era of advanced meteorological data platforms such as Copernicus and Climate Data Store, the frontier of weather forecasting has evolved. The primary challenge is no longer the acquisition of accurate and high-resolution data, but rather the effective integration and utilization of diverse observational datasets to enhance localized weather predictions. Crowd sensed weather data through a network of low-cost, widely distributed weather stations can provide the granular data needed for precise local forecasts. However, this approach introduces challenges such as data integration, consistency, and privacy concerns. Federated Learning (FL) addresses these issues by enabling decentralized data processing while maintaining data privacy.This paper introduces an innovative implementation of a federated learning framework integrated with a cluster of Automated Weather Stations (AWS). The primary objective of this study is to leverage federated learning to enhance the predictive accuracy of the Weather Research and Forecasting (WRF) model by using each weather station not only as a data acquisition point but also as a computational node. This decentralized approach maintains data privacy and security while enabling local training of models, such as Crossformer, Autoformer, and DLinear. These models’ locally trained weights are periodically aggregated on the central server, which updates and redistributes the global model.Based on data collected over two years from two automated weather stations, the experimental results analyze the possibility of improving WRF model predictions for temperature and humidity. This research highlights the potential of Federated Learning in meteorological applications, offering a robust solution for enhancing weather forecast accuracy while ensuring data privacy and efficient resource utilization.

Edge computing has emerged as a transformative technology that reduces application latency, improves cost efficiency, enhances security, and enables large-scale deployment of applications across various domains. In environmental monitoring, systems such as MegaSense[49], use low-cost sensors to gather and process real-time air quality data through edge-cloud collaboration, highlighting the critical role of edge computing in enabling scalable, efficient solutions. Similarly, marine science increasingly requires real-time processing and analysis of marine data from remote, resource-constrained environments. In this paper, we extend the power of edge computing by integrating it with Generative Artificial Intelligence(GenAI),specifically large language models (LLMs), to address challenges in marine science applications. We propose DiCE-M (Distributed Code generation and Execution for Marine applications), a robust system that uses LLM to generate distributed code for marine applications and then utilizes a runtime to efficiently execute it on an edge+cloud computing infrastructure. Specifically, DiCE-M leverages edge computing to execute lightweight AI models locally on unmanned surface vehicles(USVs)while offloading complex tasks to the cloud, thus balancing computational load and enabling realtime monitoring in marine environments. We use marine litter identification as an example application to demonstrate the utility of DiCE-M. Our results show that DiCE-M reduces latency by more than 2X when marine litter is not detected and cuts cloud computing costs by more than half compared to traditional cloud-based approaches. By selectively cropping and transmitting relevant image portions, DiCE-M further improves bandwidth efficiency, making it a reliable and cost-effective solution for deploying AI-drivenapplications on resource-constrained USVs in dynamic marine environments.

In the evolving Artificial Intelligence (AI) era, the need for real-time algorithm processing in marine edge en-vironments has become a crucial challenge. Data acquisition, analysis, and processing in complex marine situations require sophisticated and highly efficient platforms. This study optimizes real-time operations on a containerized distributed processing platform designed for Autonomous Surface Vehicles (ASV) to help safeguard the marine environment. The primary objective is to improve the efficiency and speed of data processing by adopting a microservice management system called DataX. DataX leverages containerization to break down operations into modular units, and resource coordination is based on Kubernetes. This combination of technologies enables more efficient resource management and real-time operations optimization, contributing significantly to the success of marine missions. The platform was developed to address the unique challenges of managing data and running advanced algorithms in a marine context, which often involves limited connectivity, high latencies, and energy restrictions. Finally, as a proof of concept to justify this platform's evolution, experiments were carried out using a cluster of single-board computers equipped with GPUs, running an AI-based marine litter detection application and demonstrating the tangible benefits of this solution and its suitability for the needs of maritime missions.

Abstract The request for quickly available forecasts of intense weather and marine events impacting coastal areas is gradually increasing. High-performance computing (HPC) and artificial intelligence techniques are crucial in this application. Risk mitigation and coastal management must design scientific workflow appropriately and maintain them continuously updated and operational. Climate change accelerating increase trend of the past decades impacted on sea-level rise, together with broader factors such as geostatic effects and subsidence, reducing the effectiveness of coastal defenses. Due to this, the support tools, such as Early Warning Systems, have become increasingly more valuable because they can process data promptly and provide valuable indications for mitigation proposals. We developed the Shoreline Alert Model (SAM), an operational Python tool that produces simulation scenarios, ‘what-if’ assumptions, and coastal flooding forecasts to fill this gap in our study area. SAM aims to provide decision-makers, scientists, and engineers with new tools to help forecast significant weather-marine events and support related management or emergency responses. SAM aims to fill the gap between the wind-driven wave models, which produce simulations and forecasts of waves of significant height, period, and direction in deep or mid-water, and the run-up local models, which exstimulate marine ingression in the event of intense weather phenomena. It employs a parallelization scheme that allows users to run it on heterogeneous parallel architectures. It produced results approximately 24 times faster than the baseline when using shared memory with distributed memory, processing roughly 20,000 coastal cross-shore profiles along the coastline of the Campania region (Italy). Increasing the performance of this model and, at the same time, honoring the need for relatively modest HPC resources will enable the local manager and policymakers to enforce fast and effective responses to intense weather phenomena.

Data crowdsourcing is an increasingly pervasive and lifestyle-changing technology due to the flywheel effect that results from the interaction between the Internet of Things and Cloud Computing. This paper presents the Citizen Science for the Sea with Information Technologies (C4Sea-IT) framework. It is an open platform for gathering marine data from leisure boat instruments. C4Sea-IT aims to provide a coastal marine data gathering, moving, processing, exchange, and sharing platform using the existing navigation instruments and sensors for today's leisure and professional vessels. In this work, a use case for the detection and tracking of marine litter is shown. The final goal is weather/ocean forecasts argumentation with Artificial Intelligence prediction models trained with crowdsourced data.

Modern applications are designed as an interacting set of microservices, and these applications are typically deployed on container orchestration platforms like Kubernetes. Several attractive features in Kubernetes make it a popular choice for deploying applications, and automatic scaling is one such feature. The default horizontal scaling technique in Kubernetes is the Horizontal Pod Autoscaler (HPA). It scales each microservice independently while ignoring the interactions among the microservices in an application. In this paper, we show that ignoring such interactions by HPA leads to inefficient scaling, and the optimal scaling of different microservices in the application varies as the stream content changes. To automatically adapt to variations in stream content, we present a novel system called DataX AutoScaler that leverages knowledge of the entire stream processing application pipeline to efficiently auto-scale different microservices by taking into account their complex interactions. Through experiments on real-world video analytics applications, such as face recognition and pose classification, we show that DataX AutoScaler adapts to variations in stream content and achieves up to 43% improvement in overall application performance compared to a baseline system that uses HPA.

Autonomous Surface Vehicles and their management represent one of the significant challenges in coastal and offshore surveying. Although the development of this kind of data acquisition device has skyrocketed in the last few years, line guides and technological solutions still need to come. On the other hand, this kind of robotic vessel's true potential has yet to be explored. This paper presents ArgonautAI, a containerized distributed processing platform for autonomous surface vehicles. The proposed ArgonautAI architecture leverage a cluster of single-board computers with diverse and different characteristics (computing power, CUDA GPUs, FPGAs, GPIOs, PWMs, specialized I/O) orchestrated using Kubernetes and a customized programming interface. Furthermore, the proposed solution introduces two different types of containers: 1) the platform containers hosting the software life support for the platform and 2) the mission containers defined to support the survey mission-specific scopes. The firsts manage the vehicle's instruments (e.g. position, attitude, environment, depth), the data storage, the vessel-to-shore communication, and so on; the latter host mission-specific software components. Finally, as proof of concept of the proposed platform, we present an AI-based marine litter detection application using a hierarchical computer vision approach on heterogenic onboard computing resources.

While using High-Performance Computing (HPC) for precise and accurate air quality forecasts is a common issue, similar services devoted to marine pollution in coastal areas remain challenging. This paper presents Water quality Community Model Plus Plus (WaComM++) leveraging a parallelization schema enabling the users to run it on heterogeneous parallel architectures. We evaluated the proposed model under several execution approaches using a real-world application for pollutants forecast in the Gulf of Napoli (Campania, Italy). As a result, WaComM++ has produced results 657K times faster than the sequential run (taking into account the Particles' Outer Cycle and not considering the particle domain distribution) when using distributed and shared memory with multi-GPUs dealing with about 25 million particles.

In this paper, the Shoreline Alert Model (SAM) is presented as a component of a computation platform based on workflows dedicated to extreme weather/marine event simulation. The model aims to mitigate the effects of global change by providing decision-makers, scientists, and engineers with a novel, next-generation tool set for facing extreme weather events and implementing related management or emergency responses. SAM uses a parallelization schema, allowing users to run it on heterogeneous parallel architectures. As a result, SAM produces approximately 24 times faster results than the baseline when using shared memory with distributed memory and dealing with about 20,000 transects along the Campania coastline. The system is based on the algorithms of the open-source numerical models WRF (Weather Research and Forecasting) and WW3 (Wave-watch III) implemented with refraction and shoaling routines together with run-up equations to form the modeling chain used for coastal flooding assessment.

We present "Application in a Box" (AnB) product concept aimed at simplifying the deployment and operation of remote 5G applications. AnB comes pre-configured with all necessary hardware and software components, including sensors like cameras, hardware and software components for a local 5G wireless network, and 5G-ready apps. Enterprises can easily download additional apps from an App Store. Setting up a 5G infrastructure and running applications on it is a significant challenge, but AnB is designed to make it fast, convenient, and easy, even for those without extensive knowledge of software, computers, wireless networks, or AI-based analytics. With AnB, customers only need to open the box, set up the sensors, turn on the 5G networking and edge computing devices, and start running their applications. Our system software automatically deploys and optimizes the pipeline of microservices in the application on a tiered computing infrastructure that includes device, edge, and cloud computing. Application scalability, dynamic resource management, placement of critical tasks for low-latency response, and dynamic network bandwidth allocation for efficient 5G network usage are all automatically orchestrated.AnB offers cost savings, simplified setup and management, and increased reliability and security. We’ve implemented several real-world applications, such as collision prediction at busy traffic light intersections and remote construction site monitoring using video analytics. With AnB, deployment and optimization effort can be reduced from several months to just a few minutes. This is the first-of-its-kind approach to easing deployment effort and automating self-optimization of the application during system operation.

Summary Cloud‐based services have proved useful in several research fields, such as engineering, health science, and astrophysics, to mention a few examples. The computational environmental science community developed a strong need for cloud facilities to store, process, and manage data from observations and numerical models for simulations and forecasts. Weather forecast models and global sensor networks deal with multidimensional geo‐referenced data∖sets. However, environmental data consumer applications usually require a relatively small amount of multidimensional input data slice to analyze a specific area or time interval. Hence, reducing data dimension for information retrieval is mandatory. This paper presents a twofold solution: a technique to load and retrieve the sliced multidimensional data set on different cloud services such as Amazon Web Service (AWS), Google Cloud Platform, and Microsoft Azure. The experimental results performed on these cloud services highlight that the proposed method can significantly speed up the process of loading and retrieving the data slices compared to working with the entire data set in bulk or OPeNDAP server.

The use of hardware accelerators, based on code and data offloading devoted to overcoming the CPU limitations in cores, is one of the main distinctive trends in high-end computing and related applications in the last decade. However, while code offloading is convenient for performance improvement, becoming a commonly used paradigm, memory access and management are a source of bottlenecks due to the need to interact with different address spaces. In this regard, NVidia introduced the CUDA Unified Memory model to avoid explicit memory copies between the machine hosting the accelerator device and the device itself and vice-versa. This paper shows a novel design and implementation of the support to the CUDA Unified Memory in open-source GPGPU virtualization services. The performance evaluation demonstrates that the overhead due to the virtualization and remoting is acceptable considering the possibility of sharing CUDA-enabled GPUs between various and heterogeneous machines hosted at the edge, in cloud infrastructures, or as accelerator nodes in an HPC scenario. A prototype implementation of the proposed solution is available as open-source.

Monitoring the impact of the pollutants on the sea is a crucial issue for coastal human activities, such as aquaculture. However, leveraging a continuous microbiological laboratory analysis is unfeasible for costs and practical reasons. Here we present a novel methodology finalized to predict water quality as categorized indexes leveraging an integrated approach between computational components and artificial intelligence techniques. As a paradigm demonstrator, we couple WaComM++ with AIQUAM. The use case presented is an application of AIQUAM in the Bay of Naples (Campania Region, Italy) for predicting bacteria contaminants in mussel farms. The results are encouraging as the model reached a correct prediction rate of 93%.