The increasing demand for data processing by new, data-intensive applications is placing significant strain on the performance and capacity of HPC storage systems. Advancements in storage technologies, such as NVMe and persistent memory, have been introduced to address these demands. However, relying exclusively on ultra-fast storage devices is not cost-effective, necessitating multi-tier storage hierarchies to manage data based on its usage. In response, ad-hoc file systems have been proposed as a solution. These systems use the storage resources available in compute nodes, including memory and persistent storage, to create temporary file systems that adapt to application behavior in the HPC environment. This work presents the design, implementation, and evaluation of HERCULES, a distributed ad-hoc in-memory storage system, with a focus on its new metadata and elasticity model. HERCULES takes advantage of the Unified Communication X (UCX) framework, leveraging RDMA protocols such as Infiniband, Omnipath, shared-memory, and zero-copy transfers for data transfer. It includes elasticity features at runtime and fault-tolerant facilities. The elasticity features, together with flexible policies for data allocation, allow HERCULES to migrate data so that the available resources can be efficiently used. Our exhaustive evaluation results demonstrate a better performance than Lustre and BeeGFS, two parallel file systems heavily used in High-Performance Computing systems, and GekkoFS, an ad-hoc state-of-the-art solution.

Summary Cloud‐based services have proved useful in several research fields, such as engineering, health science, and astrophysics, to mention a few examples. The computational environmental science community developed a strong need for cloud facilities to store, process, and manage data from observations and numerical models for simulations and forecasts. Weather forecast models and global sensor networks deal with multidimensional geo‐referenced data∖sets. However, environmental data consumer applications usually require a relatively small amount of multidimensional input data slice to analyze a specific area or time interval. Hence, reducing data dimension for information retrieval is mandatory. This paper presents a twofold solution: a technique to load and retrieve the sliced multidimensional data set on different cloud services such as Amazon Web Service (AWS), Google Cloud Platform, and Microsoft Azure. The experimental results performed on these cloud services highlight that the proposed method can significantly speed up the process of loading and retrieving the data slices compared to working with the entire data set in bulk or OPeNDAP server.

The systems that meet non-functional requirements (NFRs) are key for e-health services to face up events such as service outages and violations of confidentiality. How-ever, traditional NFR systems produce overhead in execution time, which could affect critical decision-making processes. This paper presents a dynamic parallel pattern construction model to design and create efficient NFR self-adaptable sys-tems. The construction of patterns is performed in two design phases: in the first one, the designers build NFR systems by creating pipelines including as many applications as required to meet the NFRs established by healthcare organizations. In the second phase, a pipeline is converted into a worker that auto-matically is added to a dynamic pattern. In a dynamic pattern, the workers can be cloned to be executed by different parallel patterns (e.g., manager/worker, divide&conquer, etc.) to face changes in the incoming workload during execution time, which converts a worker into a self-adaptable NFR system. A proto-type was implemented to create self-adaptable NFR systems, which were used in a case study to manage spirometry studies, tomography images, and electrocardiograms. The evaluation showed the effectiveness of this dynamic pattern model to create self-adaptable systems when processing multiple types of medical data/contents. The evaluation also revealed that the self-adaptable NFR systems built by dynamic patterns yielded significant performance gain in a direct comparison with the implementation of NFR application pipelines built by a traditional framework called Jenkins.