As Moore's law slows down, developers face difficult choices between low-level HDLs (Verilog, VHDL) offering fine-grained control and higher-level tools (HLS and Chisel) promising improved productivity. While high-level tools accelerate development, performance gaps persist compared to expert HDL implementations. Prior studies emphasize end-to-end performance, offering limited insight into why tools excel or where performance diverges in the design hierarchy. We introduce a hierarchical framework for comparing hardware generation tools by decomposing HPC kernels (FFT, GEMM, QR factorization) into reusable primitives (MAC arrays, butterflies, permutations, reduction trees). Across Verilog, Chisel, and Vivado HLS, we built an automated tool flow and synthesized ~1, 500 variants on AMD Alveo U250, measuring resource utilization and frequency. We derived theoretical bounds for validation. Verilog achieves the highest frequency and lowest resource usage; Chisel performs comparably (5--15% gap), while HLS shows a 20--40% gap. All tools operate within bounds for well-structured designs. Crucially, performance divergence arises during primitive assembly, indicating that high-level tools require better composition optimization. This reproducible framework provides actionable insights and is extensible to other tools, domains, and FPGA architectures.

NET4EXA aims to develop a next-generation high-performance interconnect for HPC and AI systems, addressing the increasing demands of large-scale infrastructures, such as those required for training Large Language Models. Building upon the proven BXI (Bull eXascale Interconnect) European technology used in TOP15 supercomputers, NET4EXA will deliver the new BXI release, BXIv3, a complete hardware and software interconnect solution, including switch and network interface components. The project will integrate a fully functional pilot system at TRL 8, ready for deployment into upcoming exascale and post-exascale systems from 2025 onward. Leveraging prior research from European initiatives like RED-SEA, the previous achievements of consortium partners and over 20 years of expertise from BULL, NET4EXA also lays the groundwork for the future generation of BXI, BXIv4, providing analysis and preliminary design. The project will use a hybrid development and co-design approach, combining commercial switch technology with custom IP and FPGA-based NICs. Performances of NET4EXA BXIv3 interconnect will be evaluated using a broad portfolio of benchmarks, scientific scalable applications, and AI workloads.

We present and evaluate the ExaNeSt prototype, which compactly packages 128 Xilinx ZU9EG MPSoCs, two TBytes of DRAM, and eight TBytes of SSD into a liquid-cooled rack, using a custom interconnection hardware based on 10 GB/s links. We developed this testbed in 2016–2019 in order to leverage the flexibility of FPGAs for experimenting with efficient hardware support for HPC communication among tens of thousands of processors and accelerators in the quest toward Exascale systems and beyond. In the years since then, we carefully studied this system, and we present our key design choices and insights resulting from our measurement and analysis. We developed this testbed, from architecture to the PCBs and the run-time software, within the ExaNeSt project. It is fully operational in configurations with up to 8 × 4 × 4 MPSoC nodes. It achieves high density through tight board design, while also leveraging state-of-the-art liquid cooling technology. In this article, we present a thorough architectural analysis, along with important aspects of our infrastructure development. Our custom interconnect includes a low-cost low-latency network interface, offering user-level, zero-copy RDMA, which we coupled with the ARMv8 processors in the MPSoCs. We further developed the corresponding runtimes that allow us to test real MPI applications on the large-scale testbed. We evaluated our platform through MPI microbenchmarks, mini application, and full MPI applications. Single-hop, one-way latency is 1.3 μs; approximately 0.47 μs out of these are attributed to network interface and the user-space library that exposes its functionality to the runtime. Latency over longer paths increases as expected, reaching 2.55 μs for a five-hop path. Bandwidth tests show that, for single-hop, link utilization reaches \(82\%\) of the theoretical capacity. Microbenchmarks based on MPI collectives reveal that broadcast latency scales as expected when the number of participating ranks increases. We also implemented a custom MPI_Allreduce accelerator in the network interface, which reduces the latency of such collectives by up to \(88\%\) . We assess performance scaling through weak and strong scaling tests for HPCG, LAMMPS, and the miniFE mini application; for all these tests, parallelization efficiency is at least \(69\%\) , or better.

To enable high-performance, programmable, and resilient distributed systems, Distributed Data Persistency (DDP) models provide specific data consistency and persistency guarantees. Since these models target leaderless systems (i.e., systems where any node can initiate requests), they deliver high performance and are scalable. However, they are also more complex. In this paper, we develop detailed distributed algorithms for DDP models. They support Linearizable consistency with five different types of persistency. We call these algorithms MINOSBaseline (MINOS-B) and evaluate them on a 5-node distributed machine. Additionally, to improve performance, we also redesign the algorithms to offload them to a new SmartNIC architecture. The resulting system is called MINOS-Offload (MINOS-O). The MINOS-O SmartNIC introduces optimizations such as selective data coherence in hardware between host and SmartNIC, message batching, and message broadcasting. Our evaluation shows that offloading is very beneficial. It substantially reduces request latency and increases request throughput for various workloads and number of nodes. For example, compared to MINOS-B, MINOS-O reduces the average end-to-end latency of two microservice functions by 35%.

Remote Direct Memory Access (RDMA) is widely used in High-Performance Computing (HPC) while making inroads in datacenters and accelerators. State-of-the-art RDMA engines typically do not endure page faults, therefore users are forced to pin their buffers, which complicates the programming model, limits the memory utilization, and moves the pressure to the Network Interface Cards (NICs). In this article we introduce a mechanism for handling dynamic page faults during RDMA, named PART, suitable for emerging processors that also integrate the Network Interface. PART leverages the IOMMU already present in modern processors for translations. PART avoids the pinning overheads, allows any buffer to be used for communication, and enables overlapping page fault handling with serving subsequent RDMA transfers. We implement and optimize PART for a cluster of ARMv8 cores with tightly-coupled network interfaces. Handling a minor page-fault of a small transfer at the destination takes approximately 38 $\mu$ secs, while there is no performance degradation when running three full MPI applications in 16 nodes and 64 cores. Detailed breakdown uncovers the hardware and system software components of this overhead and was used to further optimize the system. A 4MB RDMA transfer performs 1.46x better over pinning.

State-of-the-art Remote Direct Memory Access (RDMA) engines pin communication buffers, complicating the programming model, limiting the memory utilization, and mandating a separate memory translation subsystem spanning the network interface card and the OS. In this paper, we introduce PART, a page fault handling mechanism suitable for emerging nodes that integrate the NI with the main processor. PART does not need to pin pages, thus any process buffer can be used for communication, and resolves occasional page-faults dynamically, when the network accesses the memory, by reusing the RDMA transport. Additionally, PART leverages the I/O Memory Management Unit (IOMMU) which is next to the processor in order to translate virtual to physical addresses, thus reducing cost and complexity. We implement and evaluate PART in a cluster of 16 nodes and 64 ARM cores. We evaluate the performance of transfers for varying page fault frequency, and examine optimizations that proactively page-in all pages upon the first page fault or ahead of the transfer, providing useful insights that can be used to optimize runtimes. Our results show that PART completes one-page transfers with a minor page-fault at the destination in approximately 38 μsecs, while the slowdown on 1MB transfers that experience faults in all pages is as little as 2.6x compared to the no-page-fault case. Page faults are expected to be rare in HPC setups: the performance of LAMMPS in our cluster is virtually unaffected when pages are handled dynamically using PART.

The increasing amount of data produced in Astronomy by observational studies and the size of theoretical problems to be tackled in the next future pushes the need of HPC (High Performance Computing) resources towards the "Exascale". The HPC sector is undergoing a profound phase of transition, in which one of the toughest challenges to cope with is the energy efficiency that is one of the main blocking factors to the achievement of "Exascale". Since ideal peak-performance is unlikely to be achieved in realistic scenarios, the aim of this work is to give some insights about the energy consumption of contemporary architectures with real scientific applications in a HPC context. We use two state-of-the-art applications from the astrophysical domain, that we optimized in order to fully exploit the underlying hardware: a direct N-body code and a semi-analytical code for Cosmic Structure formation simulations. For these two applications, we quantitatively evaluate the impact of computation on the energy consumption when running on three different systems: one that represents the present of current HPC systems (an Intel-based cluster), one that (possibly) represents the future of HPC systems (a prototype of an Exascale supercomputer) and a micro-cluster based on Arm MPSoC. We provide a comparison of the time-to-solution, energy-to-solution and energy delay product (EDP) metrics, for different software configurations. ARM-based HPC systems have lower energy consumption albeit running ≈10 times slower.

Designing low-latency network topologies of switches is a key objective for next-generation large-scale clusters. Low latency is preconditioned on low hop counts, but existing network topologies have hop counts much larger than theoretical lower bounds. To alleviate this problem, we propose building network topologies based on uni-directional graphs that are known to have hop counts close to theoretical lower bounds. A practical difficulty with uni-directional topologies is switch-by-switch flow control, which we resolve by using hot-potato routing. Cycle-accurate network simulation experiments for various traffic patterns on uni-directional topologies show that hot-potato routing achieves performance comparable to that of conventional deadlock-free routing. Similar experiments are used to compare several uni-directional topologies to bi-directional topologies, showing that the former achieve significantly lower latency and higher throughput. We quantify end-to-end application performance for parallel application benchmarks via discrete-even simulation, showing that uni-directional topologies can lead to large application performance improvements over their bi-directional counterparts. Finally, we discuss practical issues for uni-directional topologies such as cabling complexity and cost, power consumption, and soft-error tolerance. Our results make a compelling case for considering uni-directional topologies for upcoming large-scale clusters.

ExaNest is one of three European projects that support a ground-breaking computing architecture for exascale-class systems built upon power-efficient 64-bit ARM processors. This group of projects share an "everything-close" and "share-anything" paradigm, which trims down the power consumption -- by shortening the distance of signals for most data transfers -- as well as the cost and footprint area of the installation -- by reducing the number of devices needed to meet performance targets. In ExaNeSt, we will design and implement: (i) a physical rack prototype and its liquid-cooling subsystem providing ultra-dense compute packaging, (ii) a storage architecture with distributed (in-node) non-volatile memory (NVM) devices, (iii) a unified, low-latency interconnect, designed to efficiently uphold desired Quality-of-Service guarantees for a mix of storage with inter-processor flows, and (iv) efficient rack-level memory sharing, where each page is cacheable at only a single node. Our target is to test alternative storage and interconnect options on actual hardware, using real-world HPC applications. The ExaNeSt consortium brings together technology, skills, and knowledge across the entire value chain, from computing IP, packaging, and system deployment, all the way up to operating systems, storage, HPC, big data frameworks, and cutting-edge applications.

With each technology improvement, parallel systems get larger, and the impact of interconnection networks becomes more prominent. Random topologies and their variants received more and more attention lately due to their low diameter, low average shortest path length and high scalability. However, existing supercomputers still prefer torus and fat-tree topologies, because a number of existing parallel algorithms are optimized for them and the interconnect implementation is more straight-forward in terms of floor layout. In this paper, we investigate the performance of traditional and emerging parallel workloads on these network topologies, using a event-discrete simulation called SimGrid. We observe that random topology is better for Fourier Transform (FT), Graph500, Himeno benchmarks, and its improvement over the counterpart torus is 18 percent in average. Through this study, our recommendation is to use random topology in current and future supercomputers for these scientific and big-data analysis parallel applications.

This paper studies alternative Network-on-Chip architectures for emerging many-core chip multiprocessors, by exploring the following design options on mesh-based networks: Multiple physical networks (P), cores concentration (C), express channels (X), it widths (W), and virtual channels (V). We exhaustively evaluate all combinations of the afore-mentioned parameters (P, C, X, W, V), using the energy-throughput ratio (ETR) as a metric to classify network congurations. Our experimental results show that, on one hand, with an appropriate selection of parameters (V,W), an optimized baseline 2D mesh offers the best possible ETR for NoCs with up to a few tens of cores (64-core NoC). More complicated networks, using concentration and express channels, can reduce the zero-load latency, but do not necessarily help to improve ETR. On the other hand, for larger CMPs, a 2D mesh with multiple physical networks is a better option: once optimized, this architectural choice can reduce the ETR by up to 46% for 256 cores.

The integration of more and more computing cores into processors drives the adoption of larger and larger Network-on-Chips (NoCs). Concurrently, the decreasing reliability of 1 the latest technologies promotes the utilization of fault-tolerant techniques. Unfortunately, the understanding of fault-tolerant NoCs is increasingly difficult as interconnect scale up, because they require the combination of more and more complex and heterogeneous techniques. In this paper, an high-level model named VOCIS is presented, in order to ease the comprehension and analysis of large unreliable NoCs. This model features a 3D Graphical User Interface (GUI), that offers an effective and in-depth visualization of interconnects. A few analytical measurements provided directly by VOCIS are also presented, in order to assess quantitatively the impact of defects and corresponding fault-tolerant techniques.

System expandability becomes a major concern for highly-parallel computers and datacenters, because their number of nodes gradually increases year by year. In this context we propose a low-degree expandable topology and its floor layout in which a cabinet or node set can be newly inserted by connecting short cables to a single existing cabinet. Our graph analysis shows that the proposed topology has low diameter, low average shortest path length and short aggregate cable length comparable to existing topologies with the same degree. When incrementally adding nodes and cabinets to the proposed topology, its diameter and average shortest path length increase modestly. Flit-level network simulation results show that the proposed topology has lower latency for three synthetic traffic patterns as expected from graph analysis. Our event-driven network simulation results show that the proposed topology provides a comparable performance to 2-D torus even for bandwidth-sensitive parallel applications.

New CMOS processes offer cheaper but less reliable transistors. This trend foreshadows the apparition of processors consisting of hundreds and thousands of cores prone to defects. In this context, the performance of the core interconnect under faults will be critical. In this work, we propose the combination of a novel adaptive routing algorithm and several related router mechanisms, which firstly ensure message transfers under transient and permanent faults and second adapt to congestion in order to improve the interconnect performances. In effect, the widespread 2D mesh topology offers a high redundancy of routes. Hence, a fault-tolerant congestion-aware routing algorithm is able to improve substantially the interconnect efficiency under degraded conditions. Besides, the Virtual Channel allocation is optimized for performance, and a message truncation mechanism is added to cope with dynamic permanent and transient faults.

The coming era of chips consisting of billions of gates foreshadows processors containing thousands of unreliable cores. In this context, high energy efficiency will be available, under the constraint that applications leverage the large amount of computing cores, while masking frequent faults of the chip. In this paper, an high-level method is proposed to map and manage a parallel application on an unreliable many-cores processor System on Chip subject to intra-die variability.

The coming era of chips consisting of billions of gates foreshadows processors containing thousands of unreliable cores. In this context, high energy efficiency will be available, under the constraint that applications leverage the large amount of computing cores, while masking frequent faults of the chip. In this paper, an high-level method is proposed to map and manage a parallel application on an unreliable many-cores processor System on Chip. The approach takes into account versatile constraints relative to these processors (e.g. variability, core-level DVFS) and a generic algorithm is proposed. The distributed mapping process is based on the dynamic search of the best-suited processing node, upon task creation or node defect. An adaptive stop criteria is defined in order to balance the mapping impact and application efficiency gains. The validity of the proposition is assessed with high-level simulations, under different variability and application conditions.

Future applications will require processors with many cores communicating through a regular interconnection network. Meanwhile, the Deep submicron technology foreshadows highly defective chips era. In this context, not only fault-tolerant designs become compulsory, but their performance under failures gains importance. In this paper, we present a deadlock-free fault-tolerant adaptive routing algorithm featuring Explicit Path Routing in order to limit the latency degradation under failures. This is particularly interesting for streaming applications, which transfer huge amount of data between the same source-destination pairs. The proposed routing algorithm is able to route messages in the presence of any set of multiple nodes and links failures, as long as a path exists, and does not use any routing table. It is scalable and can be applied to multicore chips with a 2D mesh core interconnect of any size. The algorithm is deadlock-free and avoids infinite looping in fault-free and faulty 2D meshes. We simulated the proposed algorithm using the worst case scenario, with different failure rates. Experimentation results confirmed that the algorithm tolerates multiple failures even in the most extreme failure patterns. Additionally, we monitored the interconnect traffic and average latency for faulty cases. For 20×20 meshes, the proposed algorithm reduces the average latency by up to 50%.

The advent of the Deep Submicron technology opens the way to many-cores processor chips. However, the variability and reliability of these processes poses new challenges. In particular, the mapping of applications will require specific strategies to leverage the plenty and diversity of the computation cores. In this work, a high-level study of the variability impact on Thousands-core processors is proposed for future technologies, based on the state-of-art VARIUS model. While many crucial details are yet unknown for these technologies, we suggest several scenarios, based on the existing literature. The obtained results are particularily suitable for Embedded Streaming applications, which both require high performance and low energy consumption. In this regard, generic task mapping strategies are proposed to improve the energy efficiency of the applications, and compared for a synthetic application. The Nearest node strategy is used as a baseline, and minimizes the communication overhead. Then, a novel energy criterion is introduced to balance the computation and communication energy consumption. While increasing the communication energy, this strategy reduces the overall consumption by up to 20%. Finally, a mapping strategy based on variability regions improves slightly the energy efficiency of the application in the presence of systematic variations.

In this paper, a Self-Recovering strategy, which is able to "re-map" dynamically application tasks on a multi-core system, is presented. Based on run-time failure aware techniques, this Self-Recovering strategy guarantees seamlessly termination and delivering the expected results despite multiple node and link failures in a 2D mesh topology. It has been demonstrated, based on a statistical analysis, that the proposed technique is able to re-map the tasks of faulty nodes in a bounded number of steps. The theoretical results have been validated by simulations. The proposed technique is allowing to bypass multiple nodes, routers and links failures with a predictable number of hops. It has been demonstrated that the Motion JPEG-2000 application can be parallelized and formally represented as a Directed Acyclic Graph (DAG). It is worth noting that the proposed technique has been validated by the simulation of a 1000 cores system, in the presence of nodes and links failures up to 10%. Therefore, the proposed technique has been shown to be efficient for seamless execution of parallel streaming applications and to provide the Execution Time Reduction Ratio close to ideal.

Future applications will require processors with many cores communicating through a regular interconnection network. Meanwhile, as the Deep submicron technology fore- shadows highly defective chips era, fault-tolerant designs become compulsory. In particular, the fault tolerance of a core interconnect is critical, and inevitably increases its complexity. In this paper, we present a novel adaptive routing algorithm that is able to route messages in the presence of any set of multiple nodes and links failures, as long as a path exists. Compared to the existing solutions, the proposed algorithm provides fault tolerance without using any routing table. It is scalable and can be applied to multicore chips with a 2D mesh core interconnect of any size. The algorithm is deadlock-free and avoids infinite looping in fault-free and faulty 2D meshes, based on Virtual Networks and Virtual Channels. We simulated the proposed algorithm using the worst case scenario, regarding the traffic patterns and the failure rate up to 40%. Experimentation results confirmed that the algorithm tolerates multiple failures even in the most extreme failure patterns. Additionally, we monitored the trade off between the fault tolerance and the average latency for faulty cases, as measurement of the performance degradation. The algorithm detects the interconnects partitioning and enables "preferred paths" for streaming applications.