This paper introduces the Software Quality Assurance as a Service (SQAaaS) concept and it describes an open-source implementation of a comprehensive platform that supports the automated assessment of specific quality metrics for software and services, defined as a set of baseline requirements. The platform is openly accessible, focuses on research software and open science, and promotes best practices by awarding standards-based digital badges to software and services. It provides an easy-to-use web-based graphical user interface which facilitates the interaction with server-side components in charge of automatically creating CI/CD (Continuous Integration/Continuous Delivery) pipelines for automated testing of the baseline criteria. The service is in production and has performed over 2800 assessments, awarding more than 125 digital badges across several scientific disciplines.

Abstract Serverless computing and, in particular, the functions as a service model has become a convincing paradigm for the development and implementation of highly scalable applications in the cloud. This is due to the transparent management of three key functionalities: triggering of functions due to events, automatic provisioning and scalability of resources, and fine‐grained pay‐per‐use. This article presents a serverless web‐based scientific gateway to execute the inference phase of previously trained machine learning and artificial intelligence models. The execution of the models is performed both in Amazon Web Services and in on‐premises clouds with the OSCAR framework for serverless scientific computing. In both cases, the computing infrastructure grows elastically according to the demand adopting scale‐to‐zero approaches to minimize costs. The web interface provides an improved user experience by simplifying the use of the models. The usage of machine learning in a computing platform that can use both on‐premises clouds and public clouds constitutes a step forward in the adoption of serverless computing for scientific applications.

The increased accuracy and exhaustivity of modern Artificial Intelligence techniques in supporting the analysis of complex data, such as medical images, have exponentially increased real-world data collection for research purposes. This fact has led to the development of international repositories and high-performance computing solutions to deal with the computational demand for training models. However, other stages in the development of medical imaging biomarkers do not require such intensive computing resources, which has led to the convenience of integrating different computing backends tailored for the processing demands of the various stages of processing workflows. We present in this article a distributed and federated repository architecture for the development and application of medical image biomarkers that combines multiple cloud storages with cloud and HPC processing backends. The architecture has been deployed to serve the PRIMAGE (H2020 826494) project, aiming to collect and manage data from paediatric cancer. The repository seamlessly integrates distributed storage backends, an elastic Kubernetes cluster on a cloud on-premises and a supercomputer. Processing jobs are handled through a single control platform, synchronising data on demand. The article shows the specification of the different types of applications and a validation through a use case that make use of most of the features of the platform.

The adoption of Artificial intelligence (AI) technologies is steadily increasing. However, to become fully pervasive, AI needs resources at the edge of the network. The cloud can provide the processing power needed for big data, but edge computing is close to where data are produced and therefore crucial to their timely, flexible, and secure management. In this paper, we introduce the AI-SPRINT project, which will provide solutions to seamlessly design, partition, and run AI applications in computing continuum environments. AI-SPRINT will offer novel tools for AI applications development, secure execution, easy deployment, as well as runtime management and optimization: AI-SPRINT design tools will allow trading-off application performance (in terms of end-to-end latency or throughput), energy efficiency, and AI models accuracy while providing security and privacy guarantees. The runtime environment will support live data protection, architecture enhancement, agile delivery, runtime optimization, and continuous adaptation.

This paper introduces an open-source platform to support serverless computing for scientific data-processing workflow-based applications across the Cloud continuum (i.e. simultaneously involving both on-premises and public Cloud platforms to process data captured at the edge). This is achieved via dynamic resource provisioning for FaaS platforms compatible with scale-to-zero approaches that minimise resource usage and cost for dynamic workloads with different elasticity requirements. The platform combines the usage of dynamically deployed auto-scaled Kubernetes clusters on on-premises Clouds and automated Cloud bursting into AWS Lambda to achieve higher levels of elasticity. A use case in public health for smart cities is used to assess the platform, in charge of detecting people not wearing face masks from captured videos. Faces are blurred for enhanced anonymity in the on-premises Cloud and detection via Deep Learning models is performed in AWS Lambda for this data-driven containerised workflow. The results indicate that hybrid workflows across the Cloud continuum can efficiently perform local data processing for enhanced regulations compliance and perform Cloud bursting for increased levels of elasticity.

Despite the many advantages of cloud computing, keeping information in such an environment increases the risk of cyber attacks, as well as the possibility of unauthorized access by cloud provider employees. Another critical concern is privacy protection, since depending on data access control, confidential information may be exposed even through authorized access. To solve these issues we have previously proposed Vallum, a platform that leverages Intel SGX protection to ensure the security, confidentiality, and integrity of data at rest and during processing. It also provides tools for privacy protection, following policies set by the data owner. In this demo we present Vallum-Med, an application of Vallum for the protection of medical patient personal data, including imaging results of their cardiac examinations. We will demonstrate that this system fully supports cloud protection of such sensitive data as well as the definition of privacy policies and ensuring that all results of queries are compliant to these policies. All processing, data storage and network traffic are protected using SCONE, a docker container-based technology for seamlessly incorporating SGX protection for applications, which provides a fully encrypted memory environment.

[EN] This paper describes the development of applications in the frame of the ATMOSPHERE platform. ATMOSPHERE provides means for developing container-based applications over a federated cloud offering measurin he trustworthiness of the applications. In this paper we show the design of a transcontinental application in the frame of medical imaging that keeps the data at one end and uses the processing capabilities of the resources available at the other end. The applications are described using TOSCA blueprints and the federation of IaaS resources is performed by the Fogbow middleware. Privacy guarantees are provided by means of SCONE and intensive computing resources are integrated through the use of GPUs directly mounted on the containers.

[EN] Cloud federation, in the last years, has grown rapidly in literature because to its multiple advantages to coordinate and re-use different services across multiple sites, different geographic locations or cloud providers.\nThis article presents a federated network architecture and focuses on the multi-tenant overlay networks creation across different sites, as well as, the inter-site migration of virtual machines, introducing a framework that allows us to manage our federate cloud environment in three scenarios: distribution of Virtual Machines, resource management, and network management.

Internet of Things (IoT) technologies can facilitate the preventive conservation of cultural heritage (CH) by enabling the management of data collected from electronic sensors. This work presents an IoT architecture for this purpose. Firstly, we discuss the requirements from the artwork standpoint, data acquisition, cloud processing and data visualization to the end user. The results presented in this work focuses on the most critical aspect of the architecture, which are the sensor nodes. We designed a solution based on LoRa and Sigfox technologies to produce the minimum impact in the artwork, achieving a lifespan of more than 10 years. The solution will be capable of scaling the processing and storage resources, deployed either in a public or on-premise cloud, embedding complex predictive models. This combination of technologies can cope with different types of cultural heritage environments.

This paper describes the achievements of the H2020 project INDIGO-DataCloud. The project has provided e-infrastructures with tools, applications and cloud framework enhancements to manage the demanding requirements of scientific communities, either locally or through enhanced interfaces. The middleware developed allows to federate hybrid resources, to easily write, port and run scientific applications to the cloud. In particular, we have extended existing PaaS (Platform as a Service) solutions, allowing public and private e-infrastructures, including those provided by EGI, EUDAT, and Helix Nebula, to integrate their existing services and make them available through AAI services compliant with GEANT interfederation policies, thus guaranteeing transparency and trust in the provisioning of such services. Our middleware facilitates the execution of applications using containers on Cloud and Grid based infrastructures, as well as on HPC clusters. Our developments are freely downloadable as open source components, and are already being integrated into many scientific applications.

After a sequence of creation and destruction of virtual machines (VMs) in an on-premises Cloud computing platform, the scheduling decisions to host the VMs are far from being optimal and the fragmentation of the physical resources may impede the platform to host some VMs despite the free available virtualization resources. This paper describes a Virtual Machine Consolidation Agent that addresses this problem by analyzing the distribution of the VMs in the virtualization platform to migrate some of them among hosts, in order to defragment the physical resources and to enhance the efficiency on their usage. The agent has been validated in a production platform, where it is capable of minimizing the number of servers needed to host the VMs. The algorithms achieve near-optimal values at a very reduced computational cost, thus making it suitable for production platforms.

There is an opportunity for Distributed Computing Infrastructures (DCIs) to embrace container-based virtualisation to support efficient execution of scientific applications without the performance penalty commonly introduced by Virtual Machines (VMs). However, containers (e.g. Docker) and VMs feature different image formats and disparate procedures for deployment and management, thus hindering the adoption of hybrid DCIs (HDCIs) comprised of those kind of resources. This paper describes a workflow based on open-source tools and standards to introduce coherent application delivery on HDCIs in which applications require to be deployed on both VMs and Docker containers. Leveraging and extending the TOSCA standard to describe application requirements, and adopting DevOps practices, resulted in the coherent creation of the artifacts required for the execution of the applications on different platforms. The paper features the adoption of this approach in the INDIGO-DataCloud project.

A case study on climate models intercomparison data analysis addressing several classes of multi-model experiments is being implemented in the context of the EU H2020 INDIGO-DataCloud project. Such experiments require the availability of large amount of data (multi-terabyte order) related to the output of several climate models simulations as well as the exploitation of scientific data management tools for large-scale data analytics. More specifically, the paper discusses in detail a use case on precipitation trend analysis in terms of requirements, architectural design solution, and infrastructural implementation. The experiment has been tested and validated on CMIP5 datasets, in the context of a large scale distributed testbed across EU and US involving three ESGF sites (LLNL, ORNL, and CMCC) and one central orchestrator site (PSNC).

BACKGROUND: Short sequence mapping methods for Next Generation Sequencing consist on a combination of seeding techniques followed by local alignment based on dynamic programming approaches. Most seeding algorithms are based on backward search alignment, using the Burrows Wheeler Transform, the Ferragina and Manzini Index or Suffix Arrays. All these backward search algorithms have excellent performance, but their computational cost highly increases when allowing errors. In this paper, we discuss an inexact mapping algorithm based on pruning strategies for search tree exploration over genomic data. RESULTS: The proposed algorithm achieves a 13x speed-up over similar algorithms when allowing 6 base errors, including insertions, deletions and mismatches. This algorithm can deal with 400 bps reads with up to 9 errors in a high quality Illumina dataset. In this example, the algorithm works as a preprocessor that reduces by 55% the number of reads to be aligned. Depending on the aligner the overall execution time is reduced between 20-40%. CONCLUSIONS: Although not intended as a complete sequence mapping tool, the proposed algorithm could be used as a preprocessing step to modern sequence mappers. This step significantly reduces the number reads to be aligned, accelerating overall alignment time. Furthermore, this algorithm could be used for accelerating the seeding step of already available sequence mappers. In addition, an out-of-core index has been implemented for working with large genomes on systems without expensive memory configurations.

Summary EUBrazilOpenBio is a collaborative initiative addressing strategic barriers in biodiversity research by integrating open access data and user‐friendly tools widely available in Brazil and Europe. The project deploys the EU‐Brazil Hybrid Data Infrastructure that allows the sharing of hardware, software and data on‐demand. This infrastructure provides access to several integrated services and resources to seamlessly aggregate taxonomic, biodiversity and climate data, used by processing services implementing checklist cross‐mapping and ecological niche modelling. A Virtual Research Environment was created to provide users with a single entry point to processing and data resources. This article describes the architecture, demonstration use cases and some experimental results and validation. Copyright © 2014 John Wiley & Sons, Ltd.

Summary This paper presents a software platform to dynamically deploy complex scientific virtual computing infrastructures, on top of Infrastructure as a Service Clouds. The platform orchestrates different services to provision the virtual computing resources. It dynamically installs the appropriate software to satisfy the requirements of a researcher, both on public and on‐premise Clouds. The platform provides a web interface to enable the users to easily manage the life cycle of virtual infrastructures. It enables users to define infrastructures, share them with other users, deploy and relinquish them, add or remove resources dynamically, create and share application recipes, and so on. The paper also describes three case studies to deploy complex infrastructures, namely, a Hadoop cluster, a single‐node to perform Next Generation Sequencing and a gateway for users to access the European Grid Infrastructure. This platform promotes a better use of on‐premise hardware resources of a research center by allocating the computing resources just‐in‐time to the specific life time of the virtual infrastructures as well as the deployment of the very same infrastructures on a public Cloud. Copyright © 2015 John Wiley & Sons, Ltd.

UNLABELLED: HPG Aligner applies suffix arrays for DNA read mapping. This implementation produces a highly sensitive and extremely fast mapping of DNA reads that scales up almost linearly with read length. The approach presented here is faster (over 20× for long reads) and more sensitive (over 98% in a wide range of read lengths) than the current state-of-the-art mappers. HPG Aligner is not only an optimal alternative for current sequencers but also the only solution available to cope with longer reads and growing throughputs produced by forthcoming sequencing technologies. AVAILABILITY AND IMPLEMENTATION: https://github.com/opencb/hpg-aligner.

SUMMARY The main advantage of Grid computing over other distributed computing paradigms is its capability to coordinate the access to data and resources in a virtual multi‐institutional environment. To this end, the information system plays a decisive role in selecting the services that meet the applications' needs. This paper presents an information system for the Grid that provides transparent and scalable group communication services to standard Grid applications, with the objective of supporting dynamic collaborations that could help address problems that involve only some participants of a virtual organization. In particular, it enables more flexible delivery mechanisms, which allows applications to select the appropriate services before sending their data to the information system. This significantly enhances the protection of data from unauthorized access, and avoids the transmission of unnecessary messages over the network. The proposed information system is based on the use of XML technologies and replication. It introduces several new advanced features that are not currently supported as a whole by any Grid middleware, such as: several entry points to the information, persistent capabilities, support for advanced queries based on XQuery, and support for the industrial standard WS‐Policy. The information system has been stress tested under realistic workloads in a Grid infrastructure with 50 sites. Scalability has been evaluated in up to 1000 messages that can be up to 10KB in size each, updated with a frequency of 5min. Copyright © 2012 John Wiley & Sons, Ltd.

General Purpose Graphic Processing Units (GPGPUs) constitute an inexpensive resource for computing-intensive applications that could exploit an intrinsic fine-grain parallelism. This paper presents the design and implementation in GPGPUs of an exact alignment tool for nucleotide sequences based on the Burrows-Wheeler Transform. We compare this algorithm with state-of-the-art implementations of the same algorithm over standard CPUs, and considering the same conditions in terms of I/O. Excluding disk transfers, the implementation of the algorithm in GPUs shows a speedup larger than 12, when compared to CPU execution. This implementation exploits the parallelism by concurrently searching different sequences on the same reference search tree, maximizing memory locality and ensuring a symmetric access to the data. The paper describes the behavior of the algorithm in GPU, showing a good scalability in the performance, only limited by the size of the GPU inner memory.

The use of data Grids for sharing relevant data has proven to be successful in many research disciplines. However, the use of these environments when personal data are involved (such as in health) is reduced due to its lack of trust. There are many approaches that provide encrypted storages and key shares to prevent the access from unauthorized users. However, these approaches are additional layers that should be managed along with the authorization policies. We present in this paper a privacy-enhancing technique that uses encryption and relates to the structure of the data and their organizations, providing a natural way to propagate authorization and also a framework that fits with many use cases. The paper describes the architecture and processes, and also shows results obtained in a medical imaging platform.

Abstract Contact: [email protected]

Most medical imaging departments of hospital and primary care services in developed countries have migrated in the last years to full digital storing, management and processing. The availability of clinical medical images is larger than ever, but the exploitation of its intrinsic knowledge is poor. Hospitals, clinics and primary care centres store the information locally with a poor degree of interoperability. Although current archiving, flow-control and processing means and protocols of PACS are successful for reporting and patient follow-up, the archiving systems are not designed to exchange interesting cases, to search by medical content or to aggregate epidemiological information. This article presents a Middleware based on Grid Technologies that addresses the problem of sharing, transferring and processing DICOM medical images in a distributed environment. It tackles the problem of incompatibilities by providing a virtual storage that builds-up virtual repositories integrating different individual and incompatible storages providing global searching, progressive transmission, automatic encryption and pseudo-anonimisation and a hook to remote processing services. Users from a virtual organisation can share the cases that arc relevant for their topic of research, epidemiological study or even deeper analysis of a complex individual case. The article describes the architecture, the technologies used, the problems faced and the solutions proposed, and the results in the form of a brief performance analysis and several snapshots.