To meet the growing demand for cloud computing services, the scale and number of data centers keeps increasing worldwide. This growth comes at the cost of increased electricity consumption, which is directly correlated to CO 2 emissions – the main driver of climate change. Therefore, researching ways to reduce cloud computing emissions is more relevant than ever. However, despite cloud providers are reportedly already working near optimal power efficiency, they fail to provide precise sustainability reporting. This calls for further improvements on the cloud computing consumer’s side. To this end, we propose a user-centric, Kubernetes-based architecture for green cloud computing. We implement a carbon intensity forecaster and we use it to schedule workloads based on the availability of green energy, exploiting both regional and temporal variations to minimize emissions. We evaluate our system using real-world traces of cloud workloads execution comparing the achieved carbon emission savings against those of a baseline round-robin scheduler. Our findings indicate that the proposed system can achieve up to a 13% reduction in emissions in a strict scenario with heavy limitations on the available resources.

Data will be made available on request.

NET4EXA aims to develop a next-generation high-performance interconnect for HPC and AI systems, addressing the increasing demands of large-scale infrastructures, such as those required for training Large Language Models. Building upon the proven BXI (Bull eXascale Interconnect) European technology used in TOP15 supercomputers, NET4EXA will deliver the new BXI release, BXIv3, a complete hardware and software interconnect solution, including switch and network interface components. The project will integrate a fully functional pilot system at TRL 8, ready for deployment into upcoming exascale and post-exascale systems from 2025 onward. Leveraging prior research from European initiatives like RED-SEA, the previous achievements of consortium partners and over 20 years of expertise from BULL, NET4EXA also lays the groundwork for the future generation of BXI, BXIv4, providing analysis and preliminary design. The project will use a hybrid development and co-design approach, combining commercial switch technology with custom IP and FPGA-based NICs. Performances of NET4EXA BXIv3 interconnect will be evaluated using a broad portfolio of benchmarks, scientific scalable applications, and AI workloads.

Capturing the history of operations and activities during a computational workflow is significantly important for Earth Observation (EO). The data provenance helps to collect the metadata that records the lineage of data products, providing information about how data are generated, transferred, manipulated, by whom all these operations are performed and through which processes, parameters, and datasets. This paper presents an approach to improve those aspects, by integrating the data provenance library yProv4WFs within openEO, a platform to let users connect to Earth Observation cloud back-ends in a simple and unified way. In addition, it is demonstrated how the integration of data provenance concepts across EO processing chains enables researchers and stakeholders to better understand the flow, the dependencies, and the transformations involved in analytical workflows.

Provenance in scientific research involves documenting the history of data, focusing on the changes it undergoes and the relationships between various data points throughout its entire lifecycle. By capturing detailed information on how data is processed and transformed, provenance enables researchers to trace, validate and ensure the integrity of their results. This process becomes particularly crucial in complex workflows and high-performance computing environments, where managing intricate tasks and data relationships is essential for maintaining scientific rigor and fostering collaboration. This paper highlights the importance of supporting data reliability and traceability of scientific workflows, providing valuable insights and tools to tracking provenance in Workflow Management Systems.

The earth sciences research community has an unprecedented opportunity to exploit the vast amount of data available from earth observation (EO) satellites and earth system models (ESM). The ascent and application of artificial intelligence foundation models (FM) can be attributed to the availability of large volumes of curated data, access to extensive computing resources and the maturity of deep learning techniques. Vision transformers (ViT) architectures have been adapted for image and image-like data, such as EO data and ESM simulation output. Pretraining foundation models is a compute intensive process, often requiring 105- 107GPU hours for large scale scientific applications. There is a limited body of knowledge on compute optimal methods for pretraining, necessitating a trial and error process. We have performed a series of experiments using ViT backbones at different scales to understand optimal and cost-effective ways to improve scientific throughput. This preliminary benchmark provides an assessment of which architectures and model configurations are favorable in a given scientific context.

Open Science is a vital part in the current and future research agenda worldwide. In order to meet Open Science goals, it is of paramount importance to fully support the research process, which includes also properly addressing provenance and reproducibility of scientific experiments. Indeed, provenance and reproducibility are two key requirements for analytics workflows in Open Science contexts. Handling provenance at different levels of granularity and during the entire experiment lifecycle becomes key to properly and flexibly managing lineage information related to large-scale experiments as well as enabling reproducibility scenarios. To this end, this work introduces the micro-provenance concept, and it provides an in-depth description of its design, implementation and exploitation in the context of a multi-model climate analytics workflow.

The European Network for Earth System Modelling (ENES) Climate Analytics Service (ECAS) is a new service from the EOSC-hub project. It offers a Virtual Research Environment (VRE) to scientific users, combining a Python (Jupyter) work environment with support services for data access, computing and data sharing. ECAS is motivated by providing users with remote access to extensive computing and storage resources beyond what they may have access to locally, reducing the need to conduct costly data transfer, and helping to realize the vision of FAIR data management. ECAS aims at providing a paradigm shift for the ENES community and beyond with a strong focus on data intensive analysis, provenance management, and server-side approaches as opposed to the current ones mostly client-based, sequential and with limited or missing end-to-end analytics workflow and provenance capabilities. Furthermore, the integrated data analytics service enables basic data provenance tracking by establishing a graph of persistent identifiers (PIDs) through the whole chain, and thereby improving reusability, traceability, and reproducibility. ECAS targets multiple user groups, including researchers in lack of local computing and storage resources, researchers with interest in the high-volume climate data pools, and use within education and training scenarios.

Open Science is key to future scientific research and promotes a deep transformation in the whole scientific research process encouraging the adoption of transparent and collaborative scientific approaches aimed at knowledge sharing. Open Science is increasingly gaining attention in the current and future research agenda worldwide. To effectively address Open Science goals, besides Open Access to results and data, it is also paramount to provide tools or environments to support the whole research process, in particular the design, execution and sharing of transparent and reproducible experiments, including data provenance (or lineage) tracking. This work introduces the Climate Analytics-Hub, a new component on top of the Earth System Grid Federation (ESGF), which joins big data approaches and parallel computing paradigms to provide an Open Science environment for reproducible multi-model climate change data analytics experiments at scale. An operational implementation has been set up at the SuperComputing Centre of the Euro- Mediterranean Center on Climate Change, with the main goal of becoming a reference Open Science hub in the climate community regarding the multi-model analysis based on the Coupled Model Intercomparison Project (CMIP).

This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific communities, either locally or through enhanced interfaces. The middleware developed allows to federate hybrid resources, to easily write, port and run scientific applications to the cloud. In particular, we have extended existing PaaS (Platform as a Service) solutions, allowing public and private e-infrastructures, including those provided by EGI, EUDAT, and Helix Nebula, to integrate their existing services and make them available through AAI services compliant with GEANT interfederation policies, thus guaranteeing transparency and trust in the provisioning of such services. Our middleware facilitates the execution of applications using containers on Cloud and Grid based infrastructures, as well as on HPC clusters. Our developments are freely downloadable as open source components, and are already being integrated into many scientific applications.

The need to apply complex algorithms on large volumes of data is boosting the development of technological solutions able to satisfy big data analytics needs in Cloud and HPC environments. In this context Ophidia represents a big data analytics framework for eScience offering a cross-domain solution for managing scientific, multi-dimensional data. It also exploits an in-memory-based distributed data storage and provides support for the submission of complex workflows by means of various interfaces compliant to well-known standards. This paper presents some applications of Ophidia for the computation of climate indicators defined in the CLIPC project, the WPS interface used for the submission and the workflow based approach employed.

A case study on climate models intercomparison data analysis addressing several classes of multi-model experiments is being implemented in the context of the EU H2020 INDIGO-DataCloud project. Such experiments require the availability of large amount of data (multi-terabyte order) related to the output of several climate models simulations as well as the exploitation of scientific data management tools for large-scale data analytics. More specifically, the paper discusses in detail a use case on precipitation trend analysis in terms of requirements, architectural design solution, and infrastructural implementation. The experiment has been tested and validated on CMIP5 datasets, in the context of a large scale distributed testbed across EU and US involving three ESGF sites (LLNL, ORNL, and CMCC) and one central orchestrator site (PSNC).

Recent developments in research and technological studies have shown that High Performance Computing (HPC) will indeed lead to advances in several areas of Science, engineering, and technology, permitting the successful completion of more computationally intensive and data-intensive problems such as those in healthcare, biomedical and biosciences, climate and environmental changes, multimedia processing, design and manufacturing of advanced materials, geology, astronomy, chemistry, physics, and even financial systems. However, further research is required for developing computing infrastructures, models to support newly evolving architectures, programming paradigms, tools to simulate and evaluate new approaches and solutions, and programming languages that are appropriate for the new and emerging domains and applications. The development of the HPC infrastructure has been accelerated by the advances in silicon technology, which permitted the design of complex systems able to incorporate many hardware and software blocks and cores. More precisely, recent rapid advances in technology and design tools enabled engineers to design systems with hundreds of cores, called multi-processor system-on-chip. These systems are composed of several processing elements, that is, dedicated hardware and software components that are interconnected by an on-chip interconnect. According to Moore's law, the number of cores on-chip will double every 18 months; therefore, thousands of cores-on-chip will be integrated in the next 20 years to meet the power and performance requirements of applications. Moreover, current trends on the road to exascale are moving toward the integration of more and more cores into a single chip 1, 2. For example, accelerators and heterogeneous processing offer some opportunities to greatly increase computational performance and to match increasing application requirements 3. Engineering these computing systems is one of the most dynamic fields in modern Science and technology. That said, there will continue to be a growing demand for more powerful HPC in the upcoming years, not just to tackle basic mounting computing needs but also to lay out the foundations for the HPC market that is becoming potentially larger than the desktop/laptop computer market. Furthermore, HPC is turning out to be a major source of hope for future applications development that require greater amounts of computing resources in various modern Science domains such as bioengineering, nanotechnology, and energy where HPC capabilities are mandatory in order to run simulations and perform visualization tasks. At the time of writing this editorial, petaflop computing is well established 4. Several architectures are making major breakthroughs: commodity, accelerators on commodity, and special-purpose cores. All of top 500 systems are based on multicore technologies 5. HPC usage is growing considerably, especially in industry. And significant efforts toward establishing exascale are underway. At the same time, several challenges have been recently identified in order to create large-scale computing systems that meet current and projected application requirements. Most of them are related to system architectures, algorithms, big data processing, and programming models 6. However, energy cost, resilience, Central Processing Unit (CPU) access latency and memory transfers are key challenges to address in the era of exascale. To address these challenges, completely new approaches and technologies and a shift from the current approaches used for application development and execution to adaptive approaches are required. Consequently, further research is required for developing advanced exascale-based computing infrastructures, models, and paradigms to support newly emerging architectures, programming models, tools to simulate and evaluate more elaborate solutions and applications, and programming languages that are appropriate for these new and emerging domains and challenges. This special issue is intended to provide an overview of some key topics and state-of-the-art of recent advances in subjects relevant to High Performance Computing and simulation. The general objectives are to address, explore, and exchange information on the challenges and current state-of-the-art in high-performance and large-scale computing systems, their use in modeling and simulation, their design, performance and use, and their impact in various Science and engineering domains and applications. This special issue contains research papers addressing the state-of-the-art in high-performance and large-scale computing systems. A set of carefully selected works was invited based on the original presentations at the 2013 IEEE International Conference on High Performance Computing and Simulation (HPCS 2013), which was held in Helsinki, Finland, July 01–05, 2013 7. The extended works have been thoroughly reviewed by an international technical reviewing committee, and only thirteen papers covering a wide range of relevant challenges in HPC were selected for this special issue. The manuscripts tackle research on different topics including HPC, distributed, Peer-to-Peer (P2P) systems, data mining, Graphics Processor Unit (GPU), multicore systems as well as real-world simulations related to Computational Fluid Dynamics (CFD), neuroinformatics, bioinformatics and weather forecast performed on large computational infrastructures. The set of accepted papers can be organized under the following key subjects and subsections, and are briefly described in the remaining parts of this Section. Accelerating compute-intensive applications is another recent research in HPC domain. These accelerators are special-purpose processors, which are designed mainly to speed up compute-intensive sections of applications and achieve better performance than CPUs for certain workloads 8. There are two types of accelerators: field-programmable gate arrays (FPGAs) and GPUs. FPGAs are highly customized and designed to be configured, while GPUs provide massive parallel execution resources and high memory bandwidth. GPU is designed to rapidly manipulate and alter memory to accelerate image-processing applications. Generally, GPUs are easier to program and require less hardware resources, while FPGAs provide the best expectation of performance, flexibility, and low overhead. Hardware acceleration using GPUs or FPGAs could potentially improve run times or higher accuracy simulations. Werner et al. 9, in their article Accelerated Join Evaluation in Semantic Web Databases by Using FPGA, provide different FPGA implementations of the join operation in the context of Semantic Web Databases. Authors develop a flexible FPGA-based hardware accelerator to improve the performance of query evaluation in a Semantic Web database. They propose an architecture based on partial reconfiguration to integrate the FPGA in the software system logically and physically optimized Semantic Web Database engines 10 to accelerate the database operations. Thus, hardware architecture considers joining algorithms for query execution implemented on FPGA. Experimental results are compared with a C code software solution for general-purpose CPU and show the efficiency of the hardware-based solution. Another application that could benefit from using accelerators like FPGA and GPU is the Weather Research and Forecasting (WRF) model 11. It is a model designed to serve both atmospheric research and operational forecasting needs. It is a next-generation mesoscale numerical weather prediction system to allow researchers to generate atmospheric simulations based on real data. However, the WRF model requires significant execution time and storage space. The porting to HPC platforms enables the simulations to run faster. GPUs are designed for computationally intensive applications, via a large number of threads on a larger number of processing elements. In their article An Analysis of the Feasibility and Benefits of GPU/Multicore Acceleration of the Weather Research and Forecasting Model, Vanderbauwhede and Takemi 12 show that porting numerical weather prediction model on GPU outperform current multi-core CPUs implementations. First, a simple study is done to evaluate the possible gains of porting a kernel to the GPU. Then, one kernel is selected through profiling and ported to GPU using OpenCL. The performance is then studied both independently and once the kernel integrated into WRF. Porting the code for GPU greatly improves the parallelization, which translates into better scalability for the OpenMP version of the code. HPC infrastructures can also be used for the development, acceleration, and application of bioinformatics applications. Jaziri et al. 13, in their article High Performance Computing of Oligopeptides Complete Backtranslation applied to DNA Microarray Probe Design, tackle the issue of large-scale backtranslation of oligopeptides, a step of generating all possible nucleic acid sequences from a protein sequence, which is needed for the discovery of new organisms. Because back translation is a time-consuming task that can generate very large quantities of data, authors propose an efficient distributed algorithm to compute a complete back translation of several hundreds of oligopeptides for functional Deoxyribose Nucleic Acid microarrays. The proposed algorithm was implemented, and simulations have been conducted on both simulated and real biological datasets. The results reported show a significant computing speedup on different architectures (symmetric multiprocessors, cluster, and grid). Most applications are computationally intensive. Scientists have traditionally attempted to parallelize their algorithms across HPC infrastructures. However, this task requires significant work and effort for learning parallel programming. Developing parallel libraries and high-level and easy to use languages are required to hide parallelization complexity of programs. Another important issue that can influence HPC systems performance is vectorisation. Many scientific codes have vectorisation potential that cannot be exploited due to an algorithm-driven choice of data layouts. In their article Data Layout Inference for Code Vectorisation, Sinkarovs and Scholz 14 propose an interesting approach for automatically generating efficient code for vectorisation by mainly focusing on the evaluation of a family of data layout transformations. The authors demonstrate the effectiveness of their approach by applying it on an N-body simulation. Coullon et al. 15, in their paper Implicit parallelism on 2D meshes using SkelGIS, tackle the issue of overcoming restrictions in parallelization of scientific simulations because of the complexity of functional concepts and specific features. Parallelization of scientific simulations requires a lot of efforts and field-specific knowledge to produce efficient parallel programs. For this reason, the authors introduce SkelGIS as a solution for abstracted and implicit parallelism. They apply SkelGIS to solve heat equations and shallow-water equations and compare both the SkelGIS performance and the SkelGIS programming effort with Message Passing Interface (MPI) solutions counterparts. The DARPA High Productivity Computing Systems program (2002–2011) has defined and released benchmark suits for measuring performance, portability, programmability, robustness, and the productivity in the HPC domain 3, 16. This suite is composed of several performance tests that are required to examine the performance and classify the HPC architectures, languages, and libraries 6: (i) High-Performance Linpack for evaluating the floating point rate of execution for solving a linear system of equations; (ii) Double precision GEneral Matrix Multiply for measuring the floating point rate of execution of double precision real matrix–matrix multiplication; (iii) STREAM for sustainable memory bandwidth evaluation; (iv) PTRANS for testing the total communications capacity of the network; (v) Random Access, which is required for measuring the rate of integer random updates of memory; (vi) Fast Fourier Transform for evaluating the floating point rate of execution of double precision complex one-dimensional Discrete Fourier Transform; and (vii) b-eff for measuring latency and bandwidth of a number of simultaneous communication patterns. Heinecke et al. 17, in their article Data Mining on Vast Datasets as a Cluster System Benchmark, introduce recent situation about benchmark and procurement of supercomputers, and describe the trend from Linpack benchmark to miniapp benchmarks to determine the performance in real usage. Also, it describes the difficulty to optimize the benchmark for new architectures. They discuss a data mining application that is compared on different (accelerated) cluster architectures and the needed optimization to efficiently run on different platforms. The authors in their work demonstrate such an optimization for a data mining algorithm, which solves regression and classification problems on vast datasets. In other terms, the authors propose a data mining application for cluster system benchmark using overlapping of computation and communication to hide latency and overhead reduction. Experiments have been conducted with different datasets and on the SuperMUC machine at the Leibniz-Rechenzentrum, the local CoolMAX AMD GPU cluster in Munich, the Phi-accelerated Beacon at University of Tennessee, and the Todi Cray XK7 at the Swiss National Supercomputing Centre. The performance results show that for strong scaling settings, GPUs and coprocessors suffer from lack of parallelism and do not perform as well at large scale. For weak scaling settings however, they always outperform. Several challenges, as stated previously, have been identified in order to create large-scale computing systems that meet current application requirements. These computing systems may rely on distributed computing mechanisms, implemented often as clusters and clouds, to provide continuous access to a variety of resources, for example, processing cores, large data stores, and information repositories. For example, a computational grid is a distributed computing infrastructure that can provide globally available network resources. These environments have the potential and ability to integrate large-scale computing resources, on demand. User ability to compute will no longer be limited to the resources he has at hand currently or those localized statically on a set of hosts known a priori. However, the main challenge in large-scale computing is how to program and control these distributed systems (Clouds or Exascale) with a billion nodes. Evidently, algorithms and simulators have to be developed to explore these new infrastructures. Distributed peer-to-peer control mechanisms could be devised and used to simulate new algorithms and computing architectures. In their article Flexible Replica Placement for Optimized P2P Backup on Heterogeneous, Unreliable Machines, Skowron and Rzadca 18 tackle the issue of data replication over distributed P2P systems with unreliable machines. They introduce a P2P backup system architecture using an optimal replication strategy for storing data over distributed P2P systems. Furthermore, research works to date have concentrated on static approaches tailored to parallelize existing applications on different HPC systems. However, the rapid growth in the size and complexity of contemporary distributed parallel applications, usually assembled out of a set of interacting software components executed over distributed and heterogeneous platforms, makes such approaches unsuitable for these dynamic environments. Therefore, dynamic approaches are required to allow the system to autonomously adapt its structure and its behavior during the course of its operation. In other words, these approaches allow the system to automatically modify its configuration according to the settings of its computing environment and the properties of its workload. These approaches are mainly motivated by the following issues. The high number of nodes makes the system vulnerable to failures. Consequently, its ability to react autonomously to faults is challenging. For example, the system's nodes should react to the changing environment by taking over pending tasks from faulty nodes. Static and centralized configurations of these systems are difficult or even impossible to use for dynamic and large-scale applications. For instance, for a large system with thousands of nodes, several applications could compete for resources. However, managing the resources at runtime and in a decentralized manner is a challenging task. Mencagli 19 in his article, Adaptive Model Predictive Control of Autonomic Distributed Parallel Computations with Variable Horizons and Switching Costs, addresses the dynamic reconfiguration issue of parallel computations by proposing an automatic method for reconfiguring distributed parallel computing. An autonomic computing approach is applied. It monitors the behavior of parallel modules and adjusts the degree of parallelism in order to achieve a global optimization while balancing the number of reconfigurations in view of performance and efficiency. The approach is based on model predictive control. The paper investigates especially the influence of different horizon lengths and different models for switching costs. The proposed method was evaluated by using a video-streaming application with synthesized workload. A model predictive control-based policy is evaluated with fixed horizon and variable horizon, respectively, and results are reported to show solution's effectiveness in improving the target properties of the adaptation process. Such autonomic control models to automate reconfiguration management based on current system load and application status could be employed in cloud computing platforms in which decision-making strategies are required for the purpose of resource management. Several compute-intensive and emerging applications have also been the subject of extensive research. These applications range from life sciences (e.g., medical imaging and gene sequencing), financial trading, oil and gas exploration, to bioscience, combustion (e.g., complex fluid simulation), astrophysics (e.g., formation of stars, evolution of galaxies), and environment (e.g., modeling world climate). The special issue includes a few such application areas with interesting representative works. Future human brain neuroimaging requires the integration of HPC to achieve high-temporal and high-spatial resolutions. Salman et al. 20, in their article Concurrency in Electrical Neuroinformatics: Parallel Computation for Studying the Volume Conduction of Brain Electrical Fields in Human Head Tissues, highlight the necessity of integrating HPC tools and techniques in order to have a systematic methodology for analyzing the main factors and study the interdependent parameters that affect the accuracy of solutions especially for large, multi-dimensional images. Their paper discusses challenges in human brain neuroimaging, particularly how to achieve high-temporal and high-spatial resolution. They provide two accurate, efficient, and reliable finite difference method-based forward solvers that are parallelized using OpenMP in shared memory and CUDA on GPU in order to show that advances in neuroimaging science and engineering will depend significantly on HPC integration. In their article A Novel Technique for Detecting Suspicious Lesions in Breast Ultrasound Images, Karimi and Krzyzak 21 address an important practical problem of automatic classification of breast lesion images using ultrasound. The problem of automatic classification of suspicious masses in ultrasound images has fundamental importance in oncology. The main advantage of ultrasound is that it is a noninvasive diagnostic tool, its main disadvantage being the heavy presence of acoustic noise. Any progress in automatic breast cancer classification using ultrasound may have significant impact on early detection and treatment of breast cancer. The authors tackle this issue by introducing a novel automatic classification technique of suspicious breast lesions using ultrasound images. The system proposed in the paper is a pipeline, which consists of several functional components. The first component uses the fuzzy logic approach, texture and morphology for de-noising, and segmentation of suspicious lesions. The second component deals with feature extraction and selection. The authors considered geometrical, texture, and morphological features. After applying sequential forward and backward searches, they selected the best features and passed them to the third component, which implements support vector machine classifier categorizing suspicious lesions into benign and malignant classes. The system is validated by a computer experiment on 80 real images. According to the authors, its performance reached a 98% success rate. It was then compared with two other methods, which were significantly outperformed by the proposed system. Pattern recognition is another research field, which focuses on the recognition of patterns and regularities in data. Most approaches used in pattern recognition employ classification methods. Support vector machines (SVM) are considered the most widely used classification technique in the pattern recognition community. It is a supervised learning model with associated learning algorithms that are used for classification and regression analysis needed for recognizing patterns and data analysis. In other words, SVM is mainly a classifier method that performs classification tasks by constructing hyperplanes in a multidimensional space. et al. in their article Support for Pattern propose to improve the SVM classification techniques by using SVM They examine the performance on pattern The authors a on the source with the kernel and its effectiveness on datasets from Deoxyribose Nucleic Experimental results conducted in this paper show that the proposed SVM is in practical pattern recognition applications. et al. in their paper of the Support with the parallelization on multi-core of SVM supervised learning They propose a new SVM algorithm with a optimization algorithm for automatic The authors have two parallelization approaches on GPU using and The results reported demonstrate an important for the proposed approach compared with the CPU and are of compute-intensive applications for complex fluid that have from researchers in computational are for the large of algorithms used to simulate different types of (e.g., and that require and memory requirements. Therefore, them on recent platforms and for different application has been in the 10 In their article and Analysis of et al. the behavior of solvers on modern HPC systems. They first models for both performance and energy In the article, authors also the performance on Message Passing that their models are tools to optimal for large-scale simulations. They highlight the importance of performance and the choice of number of cores used chip to energy The in this special issue provide recent advances in some fields related to High Performance Computing and simulations. In the manuscripts research on different topics including HPC, systems, data mining, systems as well as real-world simulations related to neuroinformatics, and weather forecasting performed on large computational infrastructures. hope that the can benefit from the in this special issue and will to these and research The of this special issue to their to all of the authors their papers to this special issue. are also to the for the work and the to the of this special also to to the for the to this special his during the special issue and for the authors the to their work in the international of Concurrency and and to the for their and the following

High-performance computing (HPC) helps address real-world problems by providing strong environments and support to run data and computational intensive algorithms, complex numerical simulations, and parallel scientific codes. However, the increasing needs to tackle high-resolution exascale, complex systems, and multi-scale data-intensive sciences require dealing with new computing platforms with millions of cores and more complex software and hardware solutions. In such a landscape, key requirements on concurrency, energy, storage, I/O, resiliency, networking, and so on need to be re-examined and investigated at different levels. This special issue contains 10 papers representing recent advances in the area of high-performance computing and simulation. These extended papers were carefully selected from the proceedings of the 2011 International Conference on High Performance Computing and Simulation (HPCS 2011), which was held in Istanbul, Turkey, July 4–8, 2011. The invited papers in this special issue represent fully refereed augmented works originally presented at the conference, which cover some of the main contemporary topics and challenges in various areas of HPC and simulation. The International High Performance Computing and Simulation (HPCS) Conference Series is meant to address, explore, and exchange information on the state-of-the-art in high-performance and large-scale computing systems, their use in modeling and simulation, their design, performance and utilization, and their applications and impact. Typically, the conference includes invited presentations by experts from academia, industry, and government laboratories and institutions as well as contributed paper presentations describing refereed original work on the current state of research in the areas in the preceding text and other related ones such as services computing, cloud and grid computing, and mobile computing. Annually, participation is extended to researchers, designers, educators, and interested parties in all HPCS disciplines and specialties to partake in various functions and contribute to activities such as tutorials, demos, exhibits, posters, panels, and doctoral dissertation colloquia. Over the years, the conference invited some of the top experts and innovators in the field as keynote speakers or for plenary talks and tutorial sessions. Along with the main track, several symposia, workshops, and special sessions are organized every year in conjunction with this meeting. Since 2009, the conference proceedings have been published by IEEE and included in the IEEE Xplore Digital Library 1-3 and indexed accordingly in several major indexing services 4. Prior to that, proceedings were published and indexed by the European Council for Modelling and Simulation (ECMS) (e.g., 5, 6). The conference continues to experience a healthy growth and improved quality contributions. The International HPCS Conference Series started in 2003 as the High Performance & Large Scale Computing (HP&LSC) Track in conjunction with the European Council for Modelling and Simulation 2003 Conference and was initially held in Nottingham, UK. With the first event being a great success, follow-up meetings were held in Magdeburg, Germany (2004); Riga, Latvia (2005); Bonn, Germany (2006); Prague, Czech Republic (2007); Nicosia, Cyprus (2008); Leipzig, Germany (2009); and Caen, France (2010). The Ninth HPCS Conference was held in Istanbul, Turkey, in the summer of 2011 on which this special issue is based. The conference has been organized in technical cooperation with major professional organizations as such Association for Computing Machinery (ACM), The Institute of Electrical and Electronics Engineers (IEEE), and International Federation for Information Processing (IFIP) as well as academic institutions and research centers, and many regional organizations. The first conference-wide special issue was organized based on the HPCS 2009 meeting in Wiley's Concurrency and Computation: Practice and Experience Journal. The set of the final papers published is available from Wiley and also online 7. The second special issue was organized the following year based on the HPCS 2010 meeting in Caen, France. The set of the final papers published is available from Wiley and also online 8. As such, the current conference-wide special issue is the third one, and it is based on selected extended papers from the Ninth HPCS Conference (HPCS 2011), which was held in Istanbul, Turkey, during the days July 4– 8, 2011. In addition to these, tracks, workshops, and special sessions also organized special issues on their respective areas, some of which have been published already 9-12. This conference-wide third special issue consists of selected papers that were compiled from the conference's main track as well as its adjoining workshops and special sessions. The chosen papers embrace several of the contemporary state-of-the-art research issues, such as many-core computing, heterogeneous architectures, cloud computing, self-aware systems, and real-time requirements along with an array of applications with HPC provisions. Authors of 31 papers from the conference proceedings were invited to submit an extended and updated version of their original paper to this special issue. The selection was made by the conference international program committee and workshops and special sessions organizers. In the initial round, we received 16 affirmative responses/proposals, which were reviewed for approval. An international reviewing committee of about 50 experts in various subjects of HPCS was formed. The committee members came from 15 different countries. Each paper was assigned to at least five reviewers. In the first round, 14 manuscripts were submitted for the reviewing process, and 11 papers passed with minor or major revisions required, while three were rejected. The 11 revised manuscripts were revised and again submitted for round two of the reviewing process. After carefully taking the reviewers' remarks and recommendations into account, the outcome this time was 10 papers were passed with minor revisions or accepted, and one was rejected. In round three, the 10 papers were reviewed again, and some minor improvements were requested for most papers. Only two manuscripts required a fourth round of reviews. At the end of this elaborate and thorough review process, we have the 10 manuscripts that you find in this special issue. With the help of our reviewers and a moderate turnaround time in each round, we managed to receive the reviews on time and forward them to the authors. The authors met the strict deadlines we set forth for each cycle and submitted their revised manuscripts in a timely manner as well. This special issue of HPCS papers comprises of 10 contributions from 31 authors from seven different countries, namely, Germany, Italy, Portugal, Romania, Spain, the UK, and the USA, covering research ranging from graph-based approaches, to tuning applications, to state-of-the-art processor architectures, to cloud computing. Basically, the papers in this special issue can be categorized into five main subjects: system- and hardware-oriented papers, out of which category, three papers have been selected for publication here; grid/cloud computing related papers, out of which category, two manuscripts have been selected for publication here; papers dealing with real-time requirements, out of which category, one paper has been selected for publication here; papers on autonomous-reflexive systems, out of which category, two manuscripts have been selected for publication here; and applications-oriented papers, out of which category, two papers have been selected for publication in this special issue. We summarize these next. ‘Finding Near-Perfect Parameters for Hardware and Code Optimizations by Automatic Multi-Objective Design Space Explorations’ by Ralf Jahr, Horia Calborean, Lucian Vintan, and Theo Ungerer 13 introduces FADSE, a design space exploration tool that automatically finds nearly optimal processor design configurations for a given code. Taking into account that performance is no longer the only objective subject to optimization (others comprise power consumption, area, etc.), FADSE tries to find an optimum architecture across multiple objective functions. As an example, the Grid ALU Processor (GAP) and its post-link optimizer GAPtimize are used to demonstrate the feasibility of the approach. ‘Cache-oblivious Matrix Algorithms in the Age of Multi- and Many-Cores’ by Alexander Heinecke and Carsten Trinitis 14 highlights the issue of increasing vector unit width that goes along with increasing core counts on x86 processor architectures. To demonstrate this, a cache-oblivious numerical code has been ported to and optimized on four contemporary x86 architectures representing vector unit widths from 128 to 512 bits. The article discusses the obtained performance results and compares them with the vendors' architecture specific and optimized libraries Math Kernel Library (MKL) and AMD Core Math Library (ACML). A special emphasis is put on providing insights into architectural properties of state-of-the-art processor and accelerator architectures. ‘New System Software for Parallel Programming Models on the Intel SCC Many-core Processor’ by Carsten Clauss, Stefan Lankes, Pablo Reble, and Thomas Bemmerl 15 gives a detailed report on the authors' experiences with implementing parallel programming libraries and tools for the 48-core Intel Single Cloud Chip (SCC) processor. SCC is a prototype of a many-core processor comprising noncoherent memory-coupled cores, a so-called cluster-on-chip architecture. The programming library developed by the authors reflects an SCC-customized Message Passing Interface (MPI) library called SCC-MPICH for distributed memory parallel programming and a shared virtual memory system called MetalSVM for the thread programming. In case of the SCC chip, both approaches are evaluated, and it is shown how these can be optimized for such a novel cluster-on-chip architecture. ‘Cost Optimization of Virtual Infrastructures in Dynamic Multi-Cloud Scenarios’ by Jose Luis Lucas Simarro, Rafael Moreno-Vozmediano, Ruben S. Montero, and Ignacio M. Llorente 16 presents a so-called cloud broker architecture: an architecture that is responsible for deploying virtual resources (virtualized servers) across compute clouds. By taking into account migration overhead costs in a dynamic cloud scenario, several use cases are investigated, demonstrating that using brokering mechanisms in dynamic deployments shows clear advantages over static deployments in cloud environments. From the users' point of view, multiple factors such as pricing schemes, types of instance, or value-added features need to be taken into account, which is why cloud brokering comes into play. ‘Interoperating Grid Infrastructures with the GridWay Metascheduler’ by Ismael Marin Carrion, Eduardo Huedo, and Ignacio M. Llorente 17 describes GridWay, a metascheduler for sharing compute resources within common grid middleware, which was developed by the authors. Latest features comprise enhancements with regard to interoperability and interoperation, which is achieved by introducing a modular architecture design. Two new execution drivers and a new remote interface have been added to GridWay, which is described in detail in the paper. ‘Improved Real-Time Scheduling for Periodic Tasks on Multiprocessors’ by Prapaporn Rattanatamrong and Jose A. B. Fortes 18 presents a novel algorithm for scheduling applications with real-time requirements to supercomputers. Methods to ensure that all resources can be optimally utilized are provided in the paper. This is demonstrated by an application dealing with a human brain-machine interface, a simulation of a prosthetic limb's movement according to activities of input signals. ‘Towards Self-Caring IT Systems: A Study of Performance Penalties under Faults’ by Selvi Kadirvel and José A. B. Fortes 19 is from the area of fault tolerance and utilizes virtualization techniques: taking MapReduce frameworks as an example, it is shown that the performance penalty imposed by fault tolerance mechanisms can not be neglected. Hence, for the open source MapReduce framework Hadoop, this execution time penalty is evaluated by using a simulator. Further investigations are carried out in a virtual environment with varying characteristics regarding hardware, application, data set, and types of fault. The obtained parameter studies show that penalties can be significantly reduced through dynamic resource scaling. ‘AOI-Cast in Distributed Virtual Environments: An Approach based on Delay Tolerant Reverse Compass Routing’ by Laura Ricci, Luca Genovali, Emanuele Carlini, and Massimo Coppola 20 deals with a novel area of interest (AOI)-cast algorithm for distributed environments such as massively multiplayer online games (MMOGs). Through exploiting the mathematical properties of Delaunay Triangulations, a spanning tree supporting event notification within the area of interest can be built. This tree is computed by reverse compass routing. The efficiency of this novel approach is demonstrated through a set of simulations with both artificial and real data from an MMOG. ‘Implementation and Performance Analysis of Efficient Index Structures for DNA Search Algorithms in Parallel Platforms’ by Gustavo Encarnacão, Nuno Sebastião, and Nuno Roma 21 is a paper from the area of bioinformatics. In DNA sequence alignment, it is of crucial importance to choose an appropriate local alignment algorithm in order to achieve reasonable performance. The authors present an analysis of three highly optimized implementations of index-based search algorithms, namely, suffix-trees, suffix-arrays, and hash tables of q-mers. For all three, a performance comparison is carried out on CPU-based and Graphics Processing Unit (GPU) based architectures. On both architectures, it is shown that suffix-trees and suffix-arrays perform significantly better than hash tables of q-mers. ‘Parallel Multigrid on Hierarchical Hybrid Grids: A Performance Study on Current HPC Clusters’ by Björn Gmeiner, Harald Köstler, Markus Stürmer, and Ulrich Rüde 22 investigates the performance of a geometric multigrid solver on up-to-date high-performance computing cluster installations, namely a BlueGene/P cluster run by the Julich Supercompouting center and an Intel Xeon 5650 cluster run by the Erlangen regional computing center (RRZE). The geometric multigrid solver executes inside a software package called hierarchical hybrid grids (HHGs). HHG is a package based on unstructured tetrahedral finite elements. The obtained performance is evaluated and compared with that obtained when using a standard multigrid solver so that an estimate can be given as to whether it is worth using numerical packages like HHG. It is our hope that the collection of manuscripts in this special issue will make a significant contribution to the HPC systems and modeling and simulation fields and their future developments. The guest editors of this special issue on High-performance Computing Systems would like to thank all authors, the special issue reviewing committee, the CPE EIC, Prof. G. C. Fox, and the editorial staff of Wiley for their contributions, efforts, and support in making this special issue possible. It would not have been possible without their support and guidance. The special issue Reviewing Committee members are Giovanni Aloisio (Italy), Andres Avila (Chile), Bruno Bachelet (France), Liz Bacon (UK), Mostafa Bamha (France), Francoise Baude (France), Milan Bradonjic (USA), Ivona Brandic (Austria), Mathieu Chapelle (France), Camille Coti (France), Alfredo Cuzzocrea (Italy), Laurent d'Orazio (France), Luciano Antonio Digiampietri (Brazil), Daniel Etiemble (France), Joel Falcou (France), Bernhard Fechner (Germany), Cecile Germain (France), Alain Giulieri (France), David Gregg (Ireland), Mark Hedges (UK), Alexander Heinecke (Germany), Gonzalo Hernandez (Chile), David Hill (France), Neil Chue Hong (UK), Udo Hönig (Germany), Zhihi Huang (New Zealand), Eric Innocenti (France), Hai Jin (China), David Kaeli (USA), Al Kellie (USA), Harald Köstler (Germany), Dieter Kranzlmüller (Germany), Sébastien Limet (France), Frederic Loulergue (France), Olivier Marin (France), Emmanuel Melin (France), Lizandro Muzy (France), Mariusz Nowostawski (New Zealand), Domenico Potena (Italy), T. K. Prasad (USA), Desh Ranjan (India), Mukesh Singhal (USA), Anna Squicciarini (USA), Domenico Talia (Italy), Lorenzo Verdoscia (Italy), Timothy J. Williams (USA), Chao Tung Yang (Taiwan), Vesna Zeljkovic (China), and Ji Zhang (Australia).

The Ophidia project is a research effort addressing big data analytics requirements, issues, and challenges for eScience. We present here the Ophidia analytics framework, which is responsible for atomically processing, transforming and manipulating array-based data. This framework provides a common way to run on large clusters analytics tasks applied to big datasets. The paper highlights the design principles, algorithm, and most relevant implementation aspects of the Ophidia analytics framework. Some experimental results, related to a couple of data analytics operators in a real cluster environment, are also presented.

Intensive computing and numerical simulation are now essential tools that contribute to the success in systems designs, effectiveness of public policies such as prevention of natural hazards and taking account of climate risks, but also to security and national sovereignty. There are indications that the scientific community is technologically ready for the implementation of private HPC clouds, although a full HPC cloud solution running on virtual machines (VM) may remain application dependent. The possibility of having HPC cloud computing brings fast-compute clusters within the reach of researchers and users for whom traditional HPC facilities are not a choice. For example, with the BiG Grid HPC Cloud, users get access to a virtualized HPC Cluster that they can configure to exactly match their needs. Still, it provides self-service and dynamically scalable high performance computing facilities.

The Earth System Grid Federation (ESGF) is a multi-agency, international collaboration that aims at developing the software infrastructure needed to facilitate and empower the study of climate change on a global scale. The ESGF's architecture employs a system of geographically distributed peer nodes, which are independently administered yet united by the adoption of common federation protocols and application programming interfaces (APIs). The cornerstones of its interoperability are the peer-to-peer messaging that is continuously exchanged among all nodes in the federation; a shared architecture and API for search and discovery; and a security infrastructure based on industry standards (OpenID, SSL, GSI and SAML). The ESGF software is developed collaboratively across institutional boundaries and made available to the community as open source. It has now been adopted by multiple Earth science projects and allows access to petabytes of geophysical data, including the entire model output used for the next international assessment report on climate change (IPCC-AR5) and a suite of satellite observations (obs4MIPs) and reanalysis data sets (ANA4MIPs).

This special issue contains seven papers representing recent advances in the area of high performance computing and simulation.These papers were carefully selected from the 2009 International Conference on High Performance Computing and Simulation, which was held in Leipzig, Germany, June 21-24, 2009.The invited papers in this special issue represent augmented works originally presented at the conference which cover some of the main contemporary topics and challenges.The papers presented in this special issue can be classified in three categories: surveys, performance, and high-performance computing (HPC) for eScience.The first category includes a survey on hardware-aware and heterogeneous computing on multicore processors and accelerators [1] and a contribution on trends and challenges in operating systems [2].The performance category proposes challenging topics such as reliability bottlenecks in integrated parallel fault-tolerant systems [3], input/output in web servers [4], and model checking and code generation for transaction processing software [5].Finally, HPC for eScience presents an application to life sciences simulations based on parallel stochastic simulations with rigorous distribution of pseudo-random numbers [6] and a fast seismic modeling and reverse time migration on a graphics processing unit (GPU) cluster [7].A short description introducing all of the papers of this special issue is presented in the following paragraphs.Buchty et al.[1] present a survey on current multicore and accelerator technologies.The authors outline architectural features and show how these features are exposed to the programmer and how they can be beneficially utilized in the application-mapping process.Moreover, the necessity of hardware-aware computing and the challenges arising from high-performance heterogeneous computing are motivated and summarized.Also investigated is the interaction between hardware and application characteristics for selected applications in numerical simulations.Polze [2] presents major trends and challenges in operating systems, moving from parallel to cloud computing.In his work, the author highlights and discusses roots, design options, and stateof-the-art solutions targeting challenging topics such as virtualization, multicore systems, energy awareness, and cloud computing.In his paper, Fechner [3] tries to answer several questions concerning fault propagation in multithreaded, multicore, and manycore systems.Typically, such integrated systems have a common, shared interface to the outside world, bearing the potential of a single point of failure.A threestage model is also presented and discussed by the author to help identify weaknesses of integrated parallel fault-tolerant systems.Azzedin and Al-Issa [4] present the design of a new, self-adapting Web server architecture that makes decisions on how future input/output operations would be handled based on load conditions.The results obtained from the authors implementation indicate that it is capable of providing competitive performance and better utilization than comparable nonadaptive Web servers at different load levels.In [5], Mentis and Katsaros present a model-driven approach for generating a provably correct implementation of the transaction model of interest.The model is specified by state machines that represent the transaction participants who are synchronized on a set of events.Moreover, the authors checked various possible execution paths of the synchronized state machines for property violations and verified the specification of nested transactions to demonstrate the validity of the adopted approach.

Data grid management systems are becoming increasingly important in the context of the recently adopted service oriented science paradigm. The Grid Relational Catalog (GRelC) project is working towards ubiquitous, integrated, seamless and comprehensive data grid management solutions to fully address application specific requirements. This paper describes a GRelC based environment for bioinformatics and its underlying data grid services allowing scientific users (by means of a customized grid portal) to manage data, handle, share and publish metadata, perform search and discovery activities, etc.

With a growing trend towards grid-based data repositories and data analysis services, scientific data analysis often involves accessing multiple data sources, and analyzing the data using a variety of analysis programs. A strictly related critical challenge is the fact that data sources often hold the same type of data in a number of different formats; moreover, the formats expected and generated by various data analysis services are often distinct. In bioinformatics the data are often stored in flat files, therefore accessing them to retrieve a subset of records determined by constraints, is slower with respect to other approaches such as relational DBMS. We have developed a data grid system, built on top of specific biological data sources in flat file format, which carries out the ingestion into a relational DBMS for data integration reducing the data redundancy present in the biological flat files. In this work, we describe the prototype for the ingestion in a relational DBMS of the Swiss-2D PAGE flat file.

In a growing number of scientific disciplines, large data collections are emerging as important community resources. Data and metadata management exploiting the data grid paradigm is becoming more and more important as the number of involved data sources is continuously increasing and decentralizing. Efficient grid data access services are perceived as mandatory components for data management. In the grid data management area the GRelC Project has been addressing efficiency, transparency, interoperability and security issues, providing grid enabled solutions and proposing a set of data access and integration/federation services. In this paper we present the advances related to the GRelC Data Access, highlighting differences and innovations w.r.t. previous work. Basic foundations about the grid-enabled queries provided by the GRelC DAS and experimental results related to a bioinformatics international testbed on the GILDA t-Infrastructure are also reported and discussed.

Nowadays production grids such as EGEE, Teragrid, DEISA adopt several tools in order to monitor jobs, check the status of the grid, manage accounting information, etc. Anyway, from the end-user perspective, monitoring the global status of the grid taking into account machines, networks, services, databases, job, etc. is not straightforward, uniform, and tightly coupled. What we present in this paper is the iGRelC dashboard, an integrated approach able to retrieve, process and display information coming from different data sources (both relational and non-relational) and published in grid by heterogeneous systems and services.

Grid portals are web gateways aiming at providing a pervasive and ubiquitous access in grid to computational resources, tools, instruments, datasets and metadata via standard web protocols. Moreover, they provide enhanced problem solving capabilities to deal with modern, large scale scientific and engineering problems. Data grid management systems are becoming increasingly important in the context of the recently adopted service oriented paradigm. The Grid Relational Catalog (GRelC) project is working towards ubiquitous, integrated, seamless and comprehensive grid database management solutions. This paper describes the GRelC Portal, a web based grid-enabled solution for grid-database access, management and integration built on top of the GRelC Data Access Service.

Data grids are middleware systems that offer secure shared storage of massive scientific datasets over wide area networks. In this paper we describe the GReIC Data Storage, a novel grid storage service which has been developed within the Grid Relational Catalog (GReIC) Project. The aim of this service is to manage efficiently, securely and transparently collections of bioinformatics data concerning "in silico" experiments on the grid promoting flexible, secure and coordinated storage resource sharing and publication across virtual organizations, taking into account current grid standards and specifications.

Abstract This paper describes the Grid Resource Broker (GRB), a Grid portal built leveraging a set of high‐level, Globus‐Toolkit‐based Grid libraries called GRB libraries. The portal leverages the Liferay framework to provide users with an intuitive, highly customizable Web GUI. The underlying GRB middleware allows trusted users seamless access to their computational Grid environments. Copyright © 2007 John Wiley & Sons, Ltd.

Currently several applications produce huge amount of data making them available for post-processing operations in order to infer new knowledge. Main issues of these applications are the need for efficient mechanisms to access data and high performance computing to obtain the results in an acceptable time. Wrapping the applications as Web services allows interoperability with others tools and in particular with grid computing environments exploiting a large set of resources through a standard interface, to support the requirements of so-called "data intensive" applications that handle large amounts of data. This paper presents the architecture of a complex data managements system leveraging the grid computing paradigm, exploiting existing middleware developed at the University of Lecce within the ProGenGrid, GReIC and GRB projects to support high throughput applications. This architecture has been specialized for a bioinformatics domain and a case study of the use of a biological application will be also described

We describe the ProGenGrid (Proteomics and Genomics Grid) Workflow system, developed by the CACT/ISUFI at the University of Lecce which aims at providing a tool that e-scientists can utilize to simulate biological experiments, compose existing analysis and visualization tools, monitor their execution, store the intermediate and final output and finally, if needed, save the model of the experiment for updating or reproducing it. The tools that we are considering are software components wrapped as Web services and composed through a workflow. Since bioinformatics applications need to use high performance machines or many workstations to reduce the computational time, we are exploiting a Grid infrastructure for interconnecting wide-spread tools and hardware resources.

Given the heterogeneous nature of biological data and their intensive use in many tools, in this paper we propose a semantic data access and integration (DAI) service, based on the grid paradigm, for the bioinformatics domain. This service uses ontologies for correlating different data sets. The DAI proposed in this work is a fundamental component of the ProGenGrid system, a grid-enabled platform, which aims at the design and implementation of a virtual laboratory where e-scientists could simulate complex "in silico" experiments, composing some popular analysis and visualization tools (e.g. Blast and Rasmol) available as Web services, into a workflow. The main goal of the DAI is to provide bioinformatics tools with advanced functionalities and data integration services for heterogeneous biological data banks, such as PDB and Swiss-Prot. A case study of our specialized data access service for locating similar protein sequences is presented.