Cloud computing provisions computer resources at a cost-effective way based on demand. Therefore it has become a viable solution for big data analytics and artificial intelligence which have been widely adopted in various domain science. Data security in certain fields such as biomedical research remains a major concern when moving their workflows to cloud, because cloud environments are generally outsourced which are more exposed to risks. We present a secure cloud architecture and describes how it enables workflow packaging and scheduling while keeping its data, logic and computation secure in transit, in use and at rest.

Containers improve the efficiency in application deployment and thus have been widely utilised on Cloud and lately in High Performance Computing (HPC) environments. Containers encapsulate complex programs with their dependencies in isolated environments making applications more compatible and portable. Often HPC systems have higher security levels compared to Cloud systems, which restrict users’ ability to customise environments. Therefore, containers on HPC need to include a heavy package of libraries making their size relatively large. These libraries usually are specifically optimised for the hardware, which compromises portability of containers.Per contra, a Cloud container has smaller volume and is more portable. Furthermore, containers would benefit from orchestrators that facilitate deployment and management of containers at a large scale. Cloud systems in practice usually incorporate sophisticated container orchestration mechanisms as opposed to HPC systems. Nevertheless, some solutions to enable container orchestration on HPC systems have been proposed in state of the art. This paper gives a survey and taxonomy of efforts in both containerisation and its orchestration strategies on HPC systems. It highlights differences thereof between Cloud and HPC. Lastly, challenges are discussed and the potentials for research and engineering are envisioned.

Computational resources are increasingly provisioned to users through cloud-like interfaces. Both academic and commercial cloud offerings exist, but no single standardized interface for common actions such as configuration, launching, and termination of virtual resources exists. This imposes huge technical burden on domain scientist that attempt to take advantage of these resources; even expert users spend considerable time to port their applications from one cloud platform to another.

This work presents NSDF-FUSE, a testbed for evaluating settings and performance of FUSE-based file systems on top of S3-compatible object storage; the testbed is part of a suite of services from the National Science Data Fabric (NSDF) project (an NSF-funded project that is delivering cyberinfrastructures for data scientists). We demonstrate how NSDF-FUSE can be deployed to evaluate eight different mapping packages that mount S3-compatible object storage to a file system, as well as six data patterns representing different I/O operations on two cloud platforms. NSDF-FUSE is open-source and can be easily extended to run with other software mapping packages and different cloud platforms.

Nowadays, machine learning (ML), especially deep learning(DL) methods, provide ever more real-life solutions. However, the lack of training data is often a crucial issue for these learning algorithms, the performance accuracy of which relies on the amount and the quality of the available data. This is particularly true when applying ML/DL based methods for specific areas e.g. material characteristics identification, as it requires huge cost of time and manual power getting observational data from real life. In the mean while, simulations on HPC have already been commonly used in computational science due to the fact that it has the ability of generating sufficient and noise free data, which can be used for training the ML/DL based models. However, in order to achieve accurate simulation results the input parameters usually have to be determined and validated by a large number of tests. Furthermore, the evaluation and validation of such input parameters for the simulation often require a deep understanding of the domain specific knowledge, software and programming skills, which can in turn be solved by ML/DL based methods. In this paper, a novel hybrid workflow combining a multi-task neural network and the simulation on high performance computers(HPC) is proposed, which can address the problem of data sparsity and reduce the demand for expertise, resources, and time in determining the validated parameters for simulation. This work is demonstrated through experiments on determination of material behaviors, and the results prove a promising performance (MSE = 0.0386) through this workflow.

Due to the increasing demand for fish consumption, sustainable fishery become more and more challenging. To prevent from overfishing, massive data in open sea fishing have been collected and analyzed to achieve efficient management of fishery. Still, it is extremely difficult for fishers and fishery managers to exploit available data for accurate prediction, because of their limited data processing capacities, and the overall lack of adequate database systems [1].The goal of this work is therefore to analyze the relationship between data collected from all sensors installed on-board fishing vessels and catch weight, to better support generating a map showing likely fishing effort allocation. To do so, we train neural networks to predict catch weight using all available data from sensors on fishing vessels. The raw data are pre-processed using random sampling techniques to be fed into a neural network for training. A multi-layer perceptron (MLP) neural network is proposed as the baseline. We propose a data augmentation method and a training strategy in order to optimize the prediction accuracy of the model. Our data augmentation method conducts random sampling of the original data multiple times, which reduces the root mean square error (RMSE) by 15.8%, as compared with the results obtained by the model trained without data augmentation. Our training strategy works well to further optimize the prediction accuracy of the model trained with an augmented dataset, which significantly decreased the RMSE by 11. 2%. To the best of our knowledge, this is the first study on the catch weight prediction using neural networks.

Containerisation demonstrates its efficiency in application deployment in cloud computing. Containers can encapsulate complex programs with their dependencies in isolated environments, hence are being adopted in HPC clusters. HPC workload managers lack micro-services support and deeply integrated container management, as opposed to container orchestrators (e.g. Kubernetes). We introduce Torque-Operator (a plugin) which serves as a bridge between HPC workload managers and container Orchestrators.

Summary A parallel program needs to manage the trade‐off between the time spent in synchronisation and computation. This trade‐off is significantly affected by its parallelism degree. A high parallelism degree may decrease computing time while increasing synchronisation cost. Furthermore, thread placement on processor cores may impact program performance, as the data access time can vary from one core to another due to intricacies of the underlying memory architecture. Alas, there is no universal rule to decide thread parallelism and its mapping to cores from an offline view, especially for a program with online behaviour variation. Moreover, offline tuning is less precise. We present our work on dynamic control of thread parallelism and mapping. We address concurrency issues via Software Transactional Memory (STM). STM bypasses locks to tackle synchronisation through transactions. Autonomic computing offers designers a framework of methods and techniques to build autonomic systems with well‐mastered behaviours. Its key idea is to implement feedback control loops to design safe, efficient, and predictable controllers, which enable monitoring and adjusting controlled systems dynamically while keeping overhead low. We implement feedback control loops to automate management of threads and diminish program execution time.