The Internet of Things (IoT) has transformed how everyone interacts with the environment. Over the past few years, this field has experienced exponential growth , which has led to difficulties in efficiently managing the data generated by these devices and has posed new challenges for cloud infrastructures. As the number of devices involved increases, latency and bandwidth issues become increasingly critical in these systems. To address these issues, architectures such as fog and edge computing emerged that proposed bringing information processing and storage closer to the data generators. This reduced the distance the data had to travel, thereby improving latency and bandwidth and reducing potential bottlenecks. These architectures have evolved into a new concept known as the computing continuum. This approach, based on dynamic collaboration between the cloud, fog, and edge, creates a continuous infrastructure of computational resources that optimizes data processing at each network level as needed. The computing continuum presents important challenges that need to be addressed at multiple levels: the application/algorithmic level (programming paradigms), middleware level (deployment, execution, scheduling, monitoring, data storage, transfer, processing, and analysis), and resource management level. The work introduced in this article addresses the challenges related to the efficient storage and transfer of data between the different levels across the computing continuum. We propose a distributed and parallel file system for this kind of infrastructure that can be used transparently at all levels. This file system can be deployed at different levels (fog, edge, and cloud) hierarchically, allowing the execution of applications at each level and efficient data transfer between these levels and the cloud. It also facilitates the development of IoT applications capable of efficiently transferring data by using typical file system calls. The work presents and validates this data storage system at different levels: two IoT devices (Raspberry Pi 1 & 4), an emulated environment, a controlled simulation, and a performance analysis with Amazon Cloud Services.

This article introduces CREATOR-Sail, a RISC-V web simulator that uses Sail as the instruction specification language. This enables binaries written in assembly language to be compiled, executed and debugged while simulating the complete RISC-V architecture and instruction set. The simulator can now simulate both 32-bit and 64-bit RISC-V variants and supports the simulation of the complete instruction set specified in the official standard, including vector and privileged instructions. Both variants include a cache memory module to increase the capacity for architecture simulation, mimicking a real processor. The aim of this work is to provide an industrial-level solution for companies and research groups, offering them a tool with which to customize the environment for their own purposes. The simulator includes an assembly compiler and a debugging module, making it more user-friendly and intuitive than other simulators. All simulator components have been integrated into the web environment using WebAssembly, which enables the execution of native code in a web environment and provides an execution performance similar to that of native applications compared to applications implemented only in JavaScript. The source code of web simulator is available at https://github.com/creatorsim/creator . The web simulator is available at https://creatorsim.github.io/creator/ .

This work introduces an exhaustive evaluation, including both benchmarks and real-world data-intensive applications, performed in MareNostrum 4 and HPC4AI Laboratory supercomputers using the Expand Ad-Hoc file system. Expand Ad-Hoc is an ad-hoc file system that dynamically virtualizes the local storage available on compute nodes (i.e., SSD, SHM, etc.) into a fast storage volume to reduce congestion on parallel file systems used as backends in High-Performance Computing (HPC) environments. The main contributions of this work include the design of a new ad-hoc parallel file system, called Expand Ad-Hoc , compatible with POSIX and MPI-IO, and an exhaustive and comprehensive evaluation of Expand. The evaluation compares the performance obtained using IOR and DLIO benchmarks and Nek5000 and Remote Sensing real-world applications on Expand Ad-Hoc , GekkoFS, GPFS, and BeeGFS, showing satisfactory results proving that Expand Ad-Hoc can be used in HPC environments to improve the I/O performance of data-intensive applications transparently.

Abstract In the last years, applications related to Artificial Intelligence and big data, among others, have been involved. There is a need to improve I/O operations to avoid bottlenecks in accessing a larger amount of data. For this purpose, the Expand Ad-Hoc parallel file system is being designed and developed. Since these applications have very long execution times, fault tolerance mechanisms in the file system are necessary to allow them to continue running in the presence of failures. This work introduces a fault-tolerant design based on data replication for the Expand Ad-Hoc parallel file system and an initial evaluation conducted on the HPC4AI Laboratory supercomputer in Torino. The evaluation of Expand Ad-Hoc with fault-tolerant found that, despite data replication, its performance and scalability are generally better than those of other parallel file systems without fault-tolerant.

An Ad-Hoc File System dynamically virtualizes storage on compute nodes into a fast storage volume to reduce congestion on parallel file systems used as backends in HPC environments and improve data locality. This paper presents Expand Ad-Hoc, a version of the Expand parallel file system, for use as an Ad-Hoc storage system for HPC environments. Such an update seeks to take better advantage of new storage technologies (on SSDs local to nodes, for example) and to adapt to new demands of current parallel applications (e.g., optimizing data access by analyzing data locality). The paper describes the new system’s features and presents a first evaluation comparing the performance of Expand with another Ad-Hoc File System (GekkoFS), and GPFS. The first results are quite satisfactory and demonstrate the good performance of Expand as an Ad-Hoc storage system.

In recent years, the devices that make up the Internet of Things (IoT) paradigm have been increasing in number and complexity as different layers of network computing, such as Cloud, Edge and Fog Computing, have emerged. Thanks to these latest paradigms and their different types of communications, it has been possible to reduce and distribute the computational load on the network. However, to build these infrastructures and reduce the cost, it is necessary to use a simulation platform to model these environments and analyze their behavior (power consumption, CPU usage, bandwidth, etc.) beforehand. An essential aspect of simulators is scalability, the ability to add new components and simulate large infrastructures without compromising the performance of the simulator. Many existing simulators, as will be discussed in this article, cannot scale adequately as new elements are added to the system. Furthermore, in these environments, it is useful to simulate mobile devices and to know the location of these devices for the development and analysis of new algorithms. To solve these problems, this article presents and describes the ENIGMA simulator. ENIGMA is a scalable simulator of Edge, Fog and Cloud computing infrastructures, which allows to efficiently simulate a large number of devices and elements and to analyze different characteristics (CPU usage, power consumption, network bandwidth, application execution time, etc.). Its objective is to analyze new infrastructures and algorithms. ENIGMA also includes support for including mobile devices and an API to integrate their visualization in graphical maps. The article presents an evaluation to compare the scalability of ENIGMA with other state-of-the-art simulators and different use cases that illustrate its operation.

Over the last few years, the number of IoT devices in daily use has increased, as they come in many sizes and of different types. In addition to this, these devices have become cheaper, which has led to many more people being able to use them. These devices are capable of both creating and processing information, thus reducing network overload. However, in Cloud or Edge Computing environments, it is useful to know where these devices are located, in order to better distribute the information among the servers and further reduce the network load, allowing users to get the data faster. Therefore, there are simulators capable of analyzing Cloud infrastructures, but most of them fail to offer the possibility of including mobility in the sensors.For these reasons, in this paper we detail an API extension developed on the SimGrid toolkit to add mobility to IoT sensors and, in addition, it integrates with an API called Folium for the visualization of the mobility of these elements.

This article introduces CREATOR, a new generic simulator for assembly programming, developed by the ARCOS group at the UC3M. CREATOR is a new, highly intuitive, and portable simulator that runs from a web browser (no installation needed). This simulator comes with the MIPS32 and RISC-V (32IMF) instruction set. Nevertheless, CREATOR allows, from the simulator itself, to edit and define other instruction sets (instructions, format, registers, etc.). Even more, CREATOR allows the definition of the parameter passing convention to be used in the instruction set. Once each particular instruction set (MIPS32, ARM, RISCV, etc.) has been defined, students can use CREATOR to edit, compile, execute and debug programs written in the associated assembler. The simulator also allows checking that the developed programs comply with the parameter passing convention defined for the instruction set. CREATOR lets us create subroutine libraries that can be loaded and linked to other assembly programs developed in the simulator. All CREATOR features allows teacher to design and deploy practical laboratories more adapted to the desired teaching goals. That improves the teaching experience of the assembly language frequently used in different subjects such as Computer Architecture or Computer Structure. The experience of its use has been very positive in the past courses for students and teachers in both the Universidad Carlos III de Madrid (UC3M) and the Universidad Castilla la Mancha (UCLM).

In the last few years, deep-learning models are becoming crucial for numerous scientific and industrial applications. Due to the growth and complexity of deep neural networks, researchers have been investigating techniques to train those networks more efficiently. Many efforts have been made to optimize deep-learning models by parallelizing or distributing their training computation across multiple devices. Current state-of-the-art techniques, such as Horovod, have shown to maximize the performance of both the training computation and the inter-node communication of models for different deep-learning frameworks. However, some applications cannot take advantage of the above techniques due to an I/O bottleneck caused by the input data, thus limiting the scalability of the trainings. In this paper, we study an approach based on data locality - that has not been fully studied yet - for those neural networks that cannot benefit from scaling their computation due to a significant bottleneck in the data I/O.

Nowadays, there are two main approaches for dealing with data-intensive applications: parallel file systems in classical High-Performance Computing (HPC) centers and Big Data like parallel file system for ensuring the data centric vision. Furthermore, there is a growing overlap between HPC and Big Data applications, given that Big Data paradigm is a growing consumer of HPC resources. HDFS is one of the most important file systems for data intensive applications while, from the parallel file systems point of view, MPI-IO is the most used interface for parallel I/O. In this paper, we propose a novel solution for taking advantage of HDFS through MPI-based parallel applications. To demonstrate its feasibility, we have included our approach in MIMIR, a MapReduce framework for MPI-based applications. We have optimized MIMIR framework by providing data locality features provided by our approach. The experimental evaluation demonstrates that our solution offers around 25% performance for map phase compared with the MIMIR baseline solution.

Summary The rapid progress of multi/many‐core architectures has caused data‐intensive parallel applications not yet fully optimized to deliver the best performance. In the advent of concurrent programming, frameworks offering structured patterns have alleviated developers' burden adapting such applications to multithreaded architectures. While some of these patterns are implemented using synchronization primitives, others avoid them by means of lock‐free data mechanisms. However, lock‐free programming is not straightforward, ensuring an appropriate use of their interfaces can be challenging, since different memory models plus instruction reordering at compiler/processor levels can interfere in the occurrence of data races. The benefits of race detectors are formidable in this sense; however, they may emit false positives if are unaware of the underlying lock‐free structure semantics. To mitigate this issue, this paper extends ThreadSanitizer, a race detection tool, with the semantics of 2 lock‐free data structures: the single‐producer/single‐consumer and the multiple‐producer/multiple‐consumer queues. With it, we are able to drop false positives and detect potential semantic violations. The experimental evaluation, using different queue implementations on a set ofμbenchmarks and real applications, demonstrates that it is possible to reduce, on average, 60% the number of data race warnings and detect wrong uses of these structures.

Summary Volunteer computing is a type of distributed computing in which ordinary people donate computing resources to scientific projects. BOINC is the main middleware system for this type of distributed computing. The aim of volunteer computing is that organizations be able to attain large computing power thanks to the participation of volunteer clients instead of a high investment in infrastructure. There are projects, like the ATLAS@Home project, in which the number of running jobs has reached a plateau, due to a high load on data servers caused by file transfer. This is why we have designed an alternative, using the same BOINC infrastructure, in order to improve the performance of BOINC projects that have reached their limit due to the I/O bottleneck in data servers. This alternative involves having a percentage of the volunteer clients running as data servers, called data volunteers, that improve the performance of the system by reducing the load on data servers. In addition, our solution takes advantage of data locality, leveraging the low network latencies of closer machines. This paper describes our alternative in detail and shows the performance of the solution, applied to 3 different BOINC projects, using a simulator of our own, ComBoS.

Power dimensioning and energy saving have been traditionally two main issues regarding the deployment of electric grids. Electric railways are also concerned about these issues, and simulators have been traditionally used to test such infrastructure deployments. The main goal of this paper is to present the Railway electric Power Consumption Simulator, a simulation model and tool for the railway energy provisioning problem. This simulator aims to propose electric railway infrastructure deployments, optimizing the quality of the electric flow supplied to train, as well as saving as much energy as possible. The paper describes the simulator structure, as well as the ontology used to translate railway infrastructure elements into an electric circuit. Because these two objectives are conflicting, a multi-objective optimization problem is formulated and solved. Finally, a standard railway scenario is used to illustrate the capabilities of the tool, trying to find the best electric substation placements in order to optimize such objectives. The evaluation shows how the tool can handle hundreds of simulated scenarios using Cloud Computing techniques.

The MPI forum is actively working for a better MPI standard. The results are the new version 3 of the MPI standard, and the efforts for the incoming MPI 3.1/4.0. The technological changes provide many opportunities for improvements and new ideas. This paper introduces two main contributions in this direction: (1) how to improve the MPI_Info object implementation, and (2) a new way of using the former improved MPI_Info object as a storage solution.

The increase of Dynamic Virtual Infrastructures usage brings up some problems. One of them is the efficient deployment, over large-scaled distributed systems, of the Virtual Appliances images. To address this problem two points needs to be faced, which nodes to select and how to transfer the VA images to those nodes. In this paper we propose a function for efficient node selection. This customizable function can be tailored to prioritize distribution time or node performance. We study how to tailor the function in order to balance both factors. We evaluate the performance of this function selection in conjunction with a deployment algorithm called Geometric Propagation obtaining exceptional results. We propose a mechanism that allows deploying a VA image over a large number of nodes within a reasonable period of time.

This paper considers the question of saving energy in the disk drive making advantage of diverse devices in a hybrid storage system employing flash and disk drives. The flash and disk offer different power characteristics, being flash much less power consuming than the disk drive. We propose a technique that uses a flash device as a cache for a single disk device. We examine various options for managing the flash and disk devices in such a hybrid system and show that the proposed method saves energy in diverse scenarios. We implemented a simulator composed of disk and flash devices. This paper gives an overview of the design and evaluation of the proposed approach with the help of realistic workloads.

Data management is one of the most important problems in grid environments. One important challenge facing grid computing is the design of a grid file system. The Global Grid Forum defines a grid file system as a human-readable resource namespace for management of heterogeneous distributed data resources, that can span across multiple autonomous administrative domains. This paper evaluates Expand, a new grid file system according to the Global Grid Forum recommendations that integrates heterogeneous data storage resources in grids using standard grid technologies: GridFTP and the OGSA ByteIO interface defined by the Open Grid Forum.

This paper presents multiple-phase collective I/O, a novel collective I/O technique for distributed memory multiprocessors. Multiple-phase collective I/O is a refinement of two-phase collective I/O technique. The communication phase is structured into several steps, which progressively increase the locality of the data to be written to a file system. Besides the description of multiple-phase collective I/O, our paper addresses two additional objectives. First, the authors target to improve the efficiency of the sulphur transport Eurelian model 2 (STEM-II) application. STEM-II is an air quality model that simulates transport, chemical transformations, emission and deposition processes in a unified framework. Due to the large amount of processed data, I/O becomes a critical factor for the application performance. Multiple-phase collective I/O, considerably enhances the performance of the I/O stage in particular and, consequently, of the whole application in general. Second objective consists of evaluating and comparing the performance of multiple-phase collective I/O with that of other well known parallel I/O techniques

Traditionally, distributed Web servers have used two strategies for allocating files on server nodes: full replication and full distribution. While full replication provides a highly reliable solution, it limits storage capacity to the capacity of the smallest node. On the other hand, full distribution provides higher storage capacity at the cost of lower reliability. A hybrid solution is partial replication where every file is allocated to a small number of nodes. The most promising architecture for a partial replication strategy is the Web cluster architecture. However, Web clusters present a big flaw from reliability perspective as they contain a single point of failure. To correct this flaw, in this paper we present a modified architecture: the Web cluster with distributed Web switch. Reliability of Web clusters is evaluated for different replication strategies. System evaluations show that our proposal leads to a highly reliable solution with high scalability.

This paper presents the design and implementation of the MPI-IO interface for the Clusterfile parallel file system. The approach offers the opportunity of achieving a high correlation between the file access patterns of parallel applications and the physical file distribution. First, any physical file distribution can be expressed by means of MPI data types. Second, mechanisms such as views and collective I/O operations are portably implemented inside the file system, unifying the I/O scheduling strategies of the MPI-IO library and the file system. The experimental section demonstrates performance benefits of more than one order of magnitude.

Currently there is a growing interest in using Java for high performance computing. Java has many advantages for high performance computing: it is based on a high-level and object-oriented programming model with support for multithreading and distributed computing. Furthermore, Java 's virtual machine allows applications to run on multiple heterogeneous platforms. A major problem with the use of Java for high performance computing is the I/O. This problem has been solved traditionally in clusters using parallel file systems and parallel I/O libraries, however there is a lack of parallel file systems for Java applications. In this paper, we present a Java parallel I/O library called jExpand. It provides high performance I/O by using several NFS servers in parallel, as NFS can be found in multiple platforms (Linux, Solaris, Windows 2000, etc), we provide a universal parallel file system that can be used everywhere. jExpand requires no changes in the NFS server as it uses RPC operations to provide parallel access to the same file. The paper describes the design, implementation and evaluation of jExpand.

Caching has been intensively used in memory and traditional file systems to improve system performance. However, the use of caching in parallel file systems and I/O libraries has been limited to I/O nodes to avoid cache coherence problems. We specify an adaptive cache coherence protocol that is very suitable for parallel file systems and parallel I/O libraries. This model exploits the use of caching, both at processing and I/O nodes, providing performance improvement mechanisms such as aggressive prefetching and delayed-write techniques. The cache coherence problem is solved by using a dynamic scheme of cache coherence protocols with different sizes and shapes of granularity. The proposed model is very appropriate for parallel I/O interfaces, such as MPI-IO. Performance results, obtained on an IBM SP2, are presented to demonstrate the advantages offered by the cache management methods proposed.

Distributed filesystems are a typical solution in networked environments as clusters and grids. Parallel filesystems are a typical solution in order to reach high performance I/O distributed environment, but those filesystems have some limitations in heterogeneous storage systems. Usually in distributed systems, load balancing is used as a solution to improve the performance, but typically the distribution is made between peer-to-peer computational resources and from the processor point of view. In heterogeneous systems, like heterogeneous clusters of workstations, the existing solutions do not work so well. However, the utilization of those systems is more extended every day, having an extreme example in the grid environment. In this paper we bring attention to those aspects of heterogeneous distributed data systems presenting a parallel file system that take into account heterogeneity of storage nodes, the dynamic addition of new storage nodes, and an algorithm to group requests in heterogeneous systems.

During the last years, Internet video streaming has experiences a phenomenal growth. This is happening despite the notorious difficulties of transmitting data packets with a deadline over the Internet, due to variability in throughput, delays and losses. These problems arise significantly when using wireless networks where the available bandwidth is low and the losses are important due to its error prone transmission nature. In this paper we propose a fast-forwarding technique that is based on segmenting the movie on different files. Normal movie reproduction requires all the files, but fast-forwarding reproduction only requires one file. Those files can me merged by the client or by the server. The segmentation is frame based, grouping all the frames that can be independently decoded together. The resulting file can be showed with any existing player. This group of frames would be the ones to use in a fast-forward reproduction. Our techniques can also be useful in adaptive environments, like wireless networks, because there is no problem for the fast-forward file to use the same optimizations that exist for full movie files. This method also reduces the storage bandwidth and the storage size needed (there is no extra data for fast-forwarding). We also propose a video server architecture that takes advantage of this technique to achieve full interactive video reproduction. The evaluation results shown in this paper demonstrates that our technique enhances video fast-forwarding operations.

MAPFS is a parallel file system integrated with a multiagent system responsible for the information retrieval [2]. The use of a multiagent system implies coordination among the agents such system consists of. The principal

The paper describes new techniques to increase the performance of collective communication operations in clusters. These techniqnes are based in multithreading operations and on-line data compression. The techniques proposed have been implemented in MiMPI, a thread-safe implementation of MPI. We have evaluated, and compared, the performance of MiMPI with other implementations of MPI available for clusters with Linux and Windows 2000. The benchmark used has been MPBench, a flexible and portable framework to allow benchmarking of MPI implementations.