Abstract This work examines the Halpern’s iterative method as a means to create new neural network optimizers that could surpass many current existing approaches. We introduce HalpernSGD, an innovative network optimizer that leverages Halpern’s technique to enhance the rate of convergence of the Stochastic Gradient Descent (SGD), leading to reduced carbon emissions in neural network training processes. The combination of Halpern’s iterative method and Gradient Descent (GD) has led to an algorithm with a quadratic rate of convergence compared to the simple GD. Experimental comparisons between their stochastic versions show that HalpernSGD achieves greater efficiency than SGD by requiring fewer training epochs, thus reducing energy consumption and carbon footprint while maintaining model accuracy. We also compare HalpernSGD with ADAM, identifying potential improvements to ADAM’s approach in terms of stability and convergence, and suggesting a future direction for the development of combined optimizers. The implementation for HalpernSGD can be accessed at: https://github.com/EttoreRitacco/HalpernSGD.git

Cellular Automata (CA) are extensively utilized for modeling and simulating complex systems due to their simplicity, flexibility, and ability to represent diverse phenomena. However, the computational intensity of CA models, particularly in large-scale applications, demands the development of efficient parallel execution techniques. This is especially critical when targeting shared memory architectures, where resource management and thread coordination play pivotal roles in achieving optimal performance. In this study, we examine a range of parallel execution strategies specifically designed to enhance the efficiency and scalability of CA models. The experimental evaluation highlights good scalability in execution time and resource utilization, making these strategies highly suitable for extensive simulations across various domains. Additionally, the results emphasize that imbalance conditions—often arising during CA computations—can considerably influence the effectiveness of choosing a given strategy.

This work is focused on the transparent execution of Cellular Automata models on a multi-GPU architecture. Although Cellular Automata models can be easily parallelized on a single GPU, the domain size and transition function complexity may require the use of multiple GPUs. Our goal is to allow modellers to be completely unaware of the parallel execution context, i.e., the code implementing the Cellular Automata model remains the same regardless if the execution is performed on CPU, single GPU, or multi-GPU systems. This paper supplies meaningful technical insights on how to ensure both transparency and efficiency in multi-GPU execution of Cellular Automata models. In particular, an object-oriented approach is exploited in which a transparent layer is devised that abstracts the parallelization details and allows a strong “separation of concerns” between the execution parallelism issues and the model implementation. Preliminary experiments have been carried out on the multi-GPU cluster CTE-POWER available at the Barcelona Supercomputing Center (BSC), witnessing good speedups notwithstanding the transparency feature supplied by our approach.

Cellular Automata (CA) models, initially studied by John von Neumann, have been developed by numerous researchers and applied in both academic and scientific fields. Thanks to their local and independent rules, simulations of complex systems can be easily implemented based on CA modelling on parallel machines. However, due to the heterogeneity of the components - from the hardware to the software perspective-the various possible scenarios running parallelism in today's architectures can pose a challenge in such implementations, making it difficult to exploit. This paper presents OpenCAL++, a transparent and efficient object-oriented platform for the parallel execution of cellular automata models. The architecture of OpenCAL++ ensures the modeller a fully transparent parallel execution and a strong “separation of concerns” between the execution parallelism issues and the model implementation. The code implementing the Cellular Automata model remains the same whether the execution performs in a shared-, distributed-memory or a GPGPU context, irrespective of the optimizations adopted. To this aim, the object-oriented paradigm has been intensely exploited. As well as the OpenCAL++ architecture, we present the description of a simple Cellular Automata model implementation for illustrative purposes.

Abstract In this paper, techniques for dynamic load balancing of the cellular automata parallel execution are presented for the case of domain space partitioned along two dimensions. Starting from general closed-form expressions that allow to compute the optimal workload assignment in a dynamic fashion when partitioning takes place along only one dimension, we tailor the procedure to allow partitioning and balancing along both dimensions. Both qualitative and quantitative experiments are carried out that assess performance improvement in applying load balancing for the case of two-dimensional partitioned domain, especially when the load balancing takes place along both dimensions.

The triangular Shepard interpolation method is an extension of the well-known bivariate Shepard’s method for interpolating large sets of scattered data. In particular, the classical point-based weight functions are substituted by basis functions built upon triangulation of the scattered points. As shown in the literature, this method exhibits advantages with respect to other interpolation methods for interpolating scattered bivariate data. Nevertheless, as the size of the data set increases, an efficient implementation of the method becomes more and more necessary. In this paper, we present a parallel implementation of the triangular Shepard interpolation method that beside exploiting benefits due to the parallelization itself, introduces a novel approach for the triangulation of the scattered data.

Load Balancing is generally referred as the technique to properly partition computation among processing elements in order to achieve optimal resource usage and thus reduce computation time. In this paper, we present a dynamic load balancing application in the context of the parallel execution of Cellular Automata where the domain space is partitioned in two dimensional regions that are assigned to different processing elements. Starting from general closed-form expressions that allow to compute the optimal workload assignment in a dynamic fashion when partitioning takes place along only one dimension, we extend the procedure to allow partitioning and balancing along both dimensions. As confirmed by the experimental results, two dimensional partitioning itself enables to speedup the execution, and further improvements are obtained when the load balancing occurs along both dimensions.

The allocation of the computational load across different processing elements is an important issue in parallel computing. Indeed, an unbalanced load distribution can strongly affect the performances of a parallel system caused by an excess of synchronization idle times due to less loaded processes waiting for more loaded ones. In this article, we focus on the load balancing issues in the context of the parallel execution of spatial-related applications where the domain space is partitioned in regions that are assigned to different processing elements. In particular, without loss of generality, we consider the well-known spatial-related Cellular Automata computational paradigm for evaluating the proposed dynamic load balancing approach. The main contribution of this article is the derivation of simple closed-form expressions that allow to compute the optimal workload assignment in a dynamic fashion, with the goal of guaranteeing a fully balanced workload distribution during the parallel execution. Based on these expressions, an algorithm for balanced execution of cellular automata is presented and implemented using the MPI technology. Eventually, an experimental section practically shows the behaviour of the proposed dynamic load balancing approach and proves its performance improvement, compared to the not-balanced version, as witnessed by the appreciable reduction of execution times.

Cellular Automata (CA) are parallel models well-suited for studying complex systems that are based on local rules of evolution. Notable examples of application are found in fluid-dynamics, crowd simulation, flow-simulation and many more. Nevertheless, CA can be fruitfully exploited as a support in numerical approaches, such as finite element and finite volume methods. Though easily parallelizable by domain partitioning among the nodes of a parallel system, the performance and scalability of cellular automata executed on parallel/distributed machines are limited due to the need of synchronizing nodes at each computational step. With the aim of reducing the synchronization burden, we here present a preliminary study on techniques stemmed from the Discrete-Event Simulation field for the optimization of CA on distributed memory architectures. Preliminary results, executed in a distributed memory environment, have shown the usefulness of the considered approach in reducing execution times and therefore in improving the speed up of the parallel execution of the test case.

The Cellular Automata paradigm is an efficient tool to model and study complex systems such as traffic simulation, lava flows and swarm based behaviour. In addition, cellular automata can be profitably used in many mathematical problems such as differential equations and chaos theory. Due to their inherent parallel nature, cellular automata can be efficiently parallelized among a set of computing nodes in order to scale and speed up their execution. This paper presents a preliminary study on different parallelization techniques for structured grid models such as cellular automata on distributed memory architectures. In particular, three strategies are presented and compared in order to evaluate their efficiency in terms of speedup. An experimental section shows the performance achieved by the three strategies when a real-life application, namely the SciddicaT cellular automata model for debris-flows simulation, is adopted.

Here we present different preliminary parallel grid-based implementations of a simple particle system with the purpose to evaluate its performances on multi- and many-core computational devices. The system is modeled by means of the Discrete Element Method and the Extended Cellular Automata formalism, while OpenMP and OpenCL are used for parallelization. In particular, both the 3.1 and 4.5 OpenMP specifications have been considered, the latter also able to run on many-core computational devices like GPUs. The results of a first test simulation performed by considering a cubic domain with about 316,000 particles have shown a clear advantage of OpenCL on the considered Tesla K40 Nvidia GPU, while the OpenMP 3.1 implementation has performed better than the corresponding OpenMP 4.5 on the considered Intel Xeon E5-2650 16-thread CPU.

Special issue on high performance computing in modeling and simulationIn the pure technological era we are living, the need for appropriate tools, methods, and approaches that could boost and skyrocket real world various applications is of paramount importance even for daily life.Toward this direction, in the up-to-date literature, several computational tools are offered, new advanced nearly real-time performing techniques are introduced, almost every day, and powerful computing approaches are promising to tackle the issues of performance, energy efficiency, and computational burden, with many different fruitful ways.Nevertheless, most of these demands, trends, and perspectives would have never met the expected outcome without the help of modern high performance computing systems able to model and simulate computationally intensive scientific applications in the most efficient and appropriate way.Consequently, numerous and various high performance computing approaches like multi-/manycore systems, accelerators, compute clusters, and massively parallel machines, when combined with efficient numerical methods for differential equation systems and native computational paradigms, enable scientists and researchers worldwide to significantly advance the application of computing methodologies in research and industry applications, both in qualitative but mainly in quantitative way.In this aspect, this Special Issue aimed to offer both scientists and engineers in academy and industry an opportunity to express and discuss their views on current trends, challenges, and state-of-the art solutions to various problems in High Performance Computing for Modeling and Simulation.Moreover, it was highly related to the corresponding Special Session on High Performance Computing in Modeling and Simulation (HPCMS), within the 23rd Euromicro International Conference on Parallel, Distributed and network-based Processing (PDP), held in Turku, Finland on March 4-6, 2015, and its relevant topics.Eventually, a major part of these topics is covered by the content of the fore-coming Special Issue of Concurrency and Computation: Practice and Experience through eight (8) finally selected papers, all thoroughly reviewed and revised properly as a detailed major extension of their conference papers earlier published in the PDP 2015 proceedings.More specifically, in this Special Issue, both theoretical aspects of high performance computing systems, like libraries for the reduction of the programming burden of numerical models on heterogeneous parallel architectures, hybrid programming model MPI/OpenMP for tackling the communication load imbalance issues, and applications starting with parallel shared-memory version of the Space Saving algorithm for mining items, approximate and semi-asynchronous parallel model for supporting Parallel and Discrete Event Simulation, parallel execution pipeline of an existing description algorithm capable of characterizing both color and texture information of a given feature point for robotic visual place recognition, and parallel and hardware acceleration of detection of ambiguous objects for surveillance reasons, as well as optimization techniques to be parallelized such as Imperialist Competitive Algorithm, are fully considered in a fruitful and plausible way.In more details, the article by Chakroun et al 1 described ExaShark, 2 an open source library with the aim of reducing the programming burden of numerical models on heterogeneous parallel architectures.The presented library offers a global-array-like interface, whereas its run-time can be configured to use shared memory threading techniques, inter-node distribution techniques, or combinations of both.ExaShark takes advantage of the latest HPC technologies, helping to scale to future generation systems.The article demonstrates the usefulness of the ExaShark library through several experiments, including stencil codes, solvers, and matrix factorization algorithms.Utrera and co-authors 3 analyzed a significant problem in the field of HPC applied to modeling and simulation, that is, the communication load imbalance generated by irregular-data applications running in a multi-node cluster.The study targets, in particular, a hybrid programming model MPI+OpenMP, where several approaches to diminish communication load imbalance are adopted, like computation-communication overlap, issuing communications in parallel, and a new approach based on message fragmentation in order to take advantage of the eager-protocol.The article includes a number of interesting results of experiments, in which the performance of overlapped and non-overlapped approaches are quantified, including the impact due to network latency.The article by Majd et al 4 concerned another relevant aspect often involved in modeling and simulation, that is, optimization.In particular, the authors focus their work on parallelizing a relatively new evolutionary optimization approach, namely, the Imperialist Competitive Algorithm.5 The proposed parallelizations include a master-slave version and a more sophisticated multi-population strategy, both exploiting the well-known Message Passing Interface.The article describes a variety of experiments and comparisons, based on two different computing platforms, and proved that the developed parallel algorithms can achieve significant performances in both optimization and speed of execution.Rousset et al 6 presented nested graphs as an approach to model parallel and distributed multi-agent simulations aiming at facilitating the dynamic distribution of computations among parallel machines.The aforementioned task is successfully achieved due to finer granularity on multiple levels of abstraction.In the proposed approach, a common and generic framework, which represents the agent models, as well as their distribution, is efficiently presented.In addition, the proposed PDMAS framework includes a more graphical method to model parallel and distributed multi-agent

Group emotion recognition in the wild is a challenging problem, due to the unstructured environments in which everyday life pictures are taken. Some of the obstacles for an effective classification are occlusions, variable lighting conditions, and image quality. In this work we present a solution based on a novel combination of deep neural networks and Bayesian classifiers. The neural network works on a bottom-up approach, analyzing emotions expressed by isolated faces. The Bayesian classifier estimates a global emotion integrating top-down features obtained through a scene descriptor. In order to validate the system we tested the framework on the dataset released for the Emotion Recognition in the Wild Challenge 2017. Our method achieved an accuracy of 64.68% on the test set, significantly outperforming the 53.62% competition baseline.

The performance and scalability of cellular automata, when executed on parallel/distributed machines, are limited by the necessity of synchronizing all the nodes at each time step, i.e., a node can execute its code only after all the other nodes have executed the previous step. However, if the code is parallelized by partitioning the space of the automata, these synchronization requirements can be relaxed: indeed, a node that manages a given portion of the cellular automata can execute a new step after synchronizing only with the nodes that manage the adjacent portions, while the remaining nodes can execute different time steps. This can be a notable advantage in many novel and increasingly popular applications of cellular automata, such as smart city applications, simulation of natural phenomena, etc., in which the execution times can be different and variable, due to the heterogeneity of machines and/or of the data and/or of the different functions. Indeed, a longer execution time at a node does not slow down the execution at all the other nodes but only at the neighboring nodes. This is particularly advantageous when the nodes that act as a bottleneck can vary during the execution. The goal of the paper is to analyze the benefits that can be achieved with the described approach when different space partitioning strategies are taken into account: i.e., the mono-and the two-dimensional partitioning. Experiments referred to a well-known cellular automata, namely the SciddicaS3-hex model for landslide simulation, exhibit good scalability and prove that the partitioning scheme adopted can result crucial for improving the overall computational performances.

Particle tracking plays an important role in numerous fields of science. In this paper, we present TraCCA, an algorithm for detecting and tracking particles based on geometrical difference evaluation and centroid displacement analysis to reconstruct the trajectories. This method works for n-dimensional input data provided that particles are represented by at least a centroid space coordinate and a geometrical entity which describe their shape. Since 2-D images are a common source of such data, we also present a framework for image-manipulation based on Extended Cellular Automata (XCA). We have applied and validated TraCCA in investigating the motility of B. subtilis. injected in a microfluidic device using 4100 images taken at 100 frames per second. Results show that the framework is able to reconstruct the trajectories as computed motion parameters are in accordance with the ones reported in the literature.

Birds flocking is an interesting natural phenomenon to study as proven by numerous papers in this field. In this paper, we present a GPGPU model for birds flocking simulation using NVIDIA's CUDA framework. This technology has been widely adopted in computational science and have dramatically increased computation performances. Using the autonomous agent approach with multi-agents and multiple groups for birds flocking modeling, we present the ACIADDRI model both aggregate motion of a large number of birds in virtual environment and other species or predators avoidance in the plane as well. From these experiments we gained significant performance improvements in the terms of speedup. In conclusion, the work shows that the use of the CUDA technology can be effective to cut computational costs also in multi-agent modeling.

Cellular Automata represent a formal frame for dynamical systems which evolve on the base of local interactions. We here present first results of the CUDA parallelization of the SCIDDICA S3-hex Complex Cellular Automata model for simulating debris flows. In particular, a first strategy for the parallelization of the model is based on a straightforward one thread - one cell approach, where each cell in the cellular space is computed by a CUDA thread. A second approach concerns the adoption of a list of CA computational active cells which is handled step by step by an efficient stream compaction algorithm, in order to reduce the excessive use of computationally inactive threads. First results performed on different graphic processors have shown that, by adopting the different CUDA strategies, this kind of hardware can be effective for landslide risk mitigation.

This paper presents the release fv3 of the Complex Cellular Automata model Sciara for simulating lava flows. It is based on a Bingham-like rheology and both flow velocity and the physical time corresponding to a computational step have been made explicit. The model has been preliminary tested with satisfying results by considering the 2006 lava event at Mt Etna (Italy). A HTML5 based Web application with a WebGL 3D interactive visualization system has also been developed as user interface for Sciara-fv3. Eventually, the numerical code has been parallelised by the CUDA GPGPU paradigm, allowing for a considerable reduction of the execution time, despite the numerical code complexity.