CyVerse, the largest publicly-funded open-source research cyberinfrastructure for life sciences, has played a crucial role in advancing data-driven research since the 2010s. As the technology landscape evolved with the emergence of cloud computing platforms, machine learning and artificial intelligence (AI) applications, CyVerse has enabled access by providing interfaces, Software as a Service (SaaS), and cloud-native Infrastructure as Code (IaC) to leverage new technologies. CyVerse services enable researchers to integrate institutional and private computational resources, custom software, perform analyses, and publish data in accordance with open science principles. Over the past 13 years, CyVerse has registered more than 124,000 verified accounts from 160 countries and was used for over 1,600 peer-reviewed publications. Since 2011, 45,000 students and researchers have been trained to use CyVerse. The platform has been replicated and deployed in three countries outside the US, with additional private deployments on commercial clouds for US government agencies and multinational corporations. In this manuscript, we present a strategic blueprint for creating and managing SaaS cyberinfrastructure and IaC as free and open-source software.

Computer Science experimental testbeds allow investigators to explore a broad range of different state-of-the-art hardware options, assess scalability of their systems, and provide conditions that allow deep reconfigurability and isolation so that one user does not impact the experiments of another. An experimental testbed is also in a unique position to support methods facilitating experiment analysis and improve repeatability and reproducibility of experiments. Providing these capabilities at least partially within a commodity framework improves the sustainability of systems experiments and thus makes them available to a broader range of experimenters.

The growth in the capacity and capability of NAND Flash based storage systems have changed the face of data oriented computational systems. These systems have become both more capable and flexible in how they are used. With these changes comes both increased potential and user complexity. While many systems attempt to hide this complexity through the addition of more layers of storage caches, the design of the Wrangler system went a different route, choosing instead to build a simple yet flexible web based interface to allow users to easily configure this complex data computing system based on their service and software needs. This allows users to work in the environments best suited to their workflows while optimally utilizing the systems high performance and high capacity storage systems. This interface also allows users to schedule long term periods of reserved capacity, "data campaigns", for projects. Finally, the system has been designed to support the data storage and sharing capacities of the system to enable these key aspects of data research. We discuss the capabilities with respect to three already existing workflows on the system to highlight the diversity and flexibility provided by this environment to data researchers.

With the growth of data in science and engineering fields and the I/O intense technologies used to carry out research with these massive datasets, it has become clear new solutions to support data research is required. In support of this, the Texas Advanced Computing Center presents Wrangler, the first open science research platform built from the ground up in support of data. Wrangler features a replicated 10 PB Lustre based parallel file system, compute capacity of 120 Intel Haswell nodes and 15 TB of RAM. In addition to the base system, Wrangler features a unique NAND flash-based storage system from DSSD, providing users with 0.5 PB of storage 1 TB/s bandwidth and 250 million IOP/s across the cluster. Supporting Hadoop, but not just Hadoop, Wrangler will provide current and future researchers with an environment supporting the most I/O intensive workflows in fields from astronomy to paleontology. With data at the forefront of Wrangler's mission, support for ETL workflows, data curation, and data publication will enable users as they both discover new results and publish their own research. Support for both SQL and noSQL databases and GIS based extensions will also be provided, allowing users to leverage these tools for both data cataloging and cross-study integration. Wrangler will allow users to focus more on what is most important to them, the data and knowledge gained from its analysis, and less on the details of curation and I/O optimization.

SUMMARY The partial correlation coefficient with information theory (PCIT) method is an important technique for detecting interactions between networks. The PCIT algorithm has been used in the biological context to infer complex regulatory mechanisms and interactions in genetic networks, in genome wide association studies, and in other similar problems. In this work, the PCIT algorithm is re‐implemented with exemplary parallel, vector, input/output (I/O), memory, and instruction optimizations for today's multi‐core and many‐core architectures. The evolution and performance of the new code targets the processor architectures of the Stampede supercomputer but will also benefit other architectures. The Stampede system consists of an Intel Xeon E5 processor base system with an innovative component consist of Intel Xeon Phi Coprocessors. Optimized results and an analysis are presented for both the Xeon and the Xeon Phi. Copyright © 2014 John Wiley & Sons, Ltd.

This paper provides and overview of the {\it GDBase} framework for offline parallel debuggers. The framework was designed to become the basis of debugging tools which scale successfully on systems with tens to hundreds of thousands of cores. With several systems coming online at more than 50,000 cores in the past year, debuggers which can run at these scales are now required. The proposed framework offers two features not found in current generation debugging tools: the ability to debug "offline'', and a central database to act as a repository of debugging information. These two features enable the GDBase debugger to offer several advantages. The debugger can be used in conjunction with modern batch systems with low overhead, with user interaction taking place after the parallel system resources are freed. The use of a database and a simple API allows for multiple interfaces and data mining tools to be implemented to provide novel ways of viewing and analyzing debugging data. The database also enables cross-run analysis, and the combination of debugging, performance, and system health information. Evidence is provided of the scalability of the framework, as well as output from several simple analysis tools that have been implemented.

Multiple clusters co-existing in a single research campus has become commonplace at many university and government labs, but effectively leveraging those resources is difficult. Intelligently forwarding and spanning jobs across clusters can increase throughput, decrease turnaround time, and improve overall utilization. Dynamic virtual clustering (DVC) is a system of virtual machines, deployed in a single or multi-cluster environment, to increase cluster utilization by enabling job forwarding and spanning, flexibly allow software environment changes, and effectively sandbox users and processes from each other and the system. This paper presents both the initial implementation of DVC and performance results from synthetic workloads executed under DVC.

This paper examines the suitability of different virtualization techniques in a high performance cluster environment. A survey of visualization techniques is presented. Two representative technologies (Xen and User Mode Linux) are selected for an in depth analysis of cluster readiness in terms of their performance, reliability, and their overall impact on complexity of cluster administration

Blade severs are being increasingly deployed in modern datacenters due to their high performance/cost ratio and compact size. In this study, we document our work on blade server based datacenter thermal management. Our goal is to minimize the total energy costs (usage) of datacenter operation while providing a reasonable thermal environment for their reliable operation. Due to special characteristics of blade servers, we argue that previously proposed power-oriented schemes are ineffective for blade server-based datacenters and that task-oriented scheduling is a more practicable approach since the contribution to the total energy cost from cooling and computing systems vary according to the utilization rates. CFD simulations are used to evaluate scheduling results of three different task scheduling algorithms: Uniform Outlet Profile (UOP), Minimal Computing Energy (MCE), and Uniform Task (UT), under four different blade-server energy consumption models: DiscreteNonOptimal (DNO), DiscreteOptimal (DO), AnalogNonOptimal (ANO), and AnalogOptimal (AO). Simulation results show that the MCE algorithm, in most cases, results in a minimal total energy cost - a conclusion that differs from the findings of previous research. UOP performs better than UT at low datacenter utilization rates, whereas UT outperforms UOP at high utilization rates.

InfiniBand has emerged as a new high bandwidth, low latency standard for high performance computing, but as a technology, is still focused on Layer 2 switching. Standards have not yet been defined for InfiniBand Layer 3 Routing, required for additional scalability, distance reach, security, and fault tolerance and isolation.The meeting will consist of:Product Leads from InfiniBand vendors discussing the unique requirements and advantages around InfiniBand routing.A Chance to Discuss potential direction and standards for InfiniBand routing, proactively soliciting input from real-world end users.An Opportunity for HPC researchers and end users to directly discuss how InfiniBand routing might be able to help optimize their infrastructure, and discuss 1:1 with leading InfiniBand technologists.

Summary form only given. The ability to easily span parallel and distributed jobs over multiple physical clusters is a potentially attractive proposal. Such an ability would allow researchers to pool all available cluster resources at a given site. Multiple clusters at a single research site has become the norm whether in an academic, government, or industrial environment. However, there are substantial barriers to transparently spanning clusters in this environment, not the least of which are authentication, authorization, sharing data, library mismatching, version skew, compiler access, environment variables, and networking. These problems arise from the heterogeneity of software environments, and in order to circumvent many of these issues, we look at visualization, and its potential use to create dynamic virtual clusters in order to provide a consistent, transparent environment for job spanning. The Grid computing community has addressed the problem of spanning jobs across distributed resources. However, grid solutions tend to focus on authorization, authentication, and data migration problems associated with far flung resources communicating over public network links. In a campus grid environment, where clusters can be directly connected via internal networks, many of these problems disappear. An alternative to the grid approach way to accomplish this spanning is to use common visualization techniques to create virtual clusters made up of one or more physical clusters. In this work, we examine the ontology of visualization techniques from the least amount of visualization to the most, and evaluate their suitability for building dynamic virtual clusters

The interaction of simultaneously co-allocated jobs can often create contention in the network infrastructure of a dedicated computational grid. This contention can lead to degraded job run-time performance. We present several bandwidth-aware co-allocating meta-schedulers. These schedulers take into account inter-cluster network utilization as a means by which to mitigate this impact. We make use of a bandwidth-centric parallel job communication model that captures the time-varying utilization of shared inter-cluster network resources. By doing so, we are able to evaluate the performance of grid scheduling algorithms that focus not only on node resource allocation, but also on shared inter-cluster network bandwidth.

Summary form only given. We present a bandwidth-centric parallel job communication model that takes into account inter-cluster network utilization as a means by which to capture the interaction and impact of simultaneously co-allocated jobs in a mini-grid. Our model captures the time-varying utilization of shared inter-cluster network resources in the grid. We compare our dynamic model with previous research that utilizes a fixed execution time penalty for co-allocated jobs. We have found that the fixed penalty model is more generous in its prediction of job turnaround time than our dynamic communication model. Additionally, we see that the penalty co-allocated jobs may experience without causing a severe performance degradation decreases as the number of clusters increases.

Presents the design of the Coven framework for construction of problem solving environments (PSEs) for parallel computers. PSEs are an integral part of modern high performance computing (HPC) and Coven attempts to simplify PSE construction. Coven targets Beowulf cluster parallel computers but independent of any particular domain for the PSE. Multithreaded parallel applications are created with Coven that are capable of supporting most of the constructs in a typical parallel programming language. Coven uses an agent-based front-end which allows multiple custom interfaces to be constructed. Examples of the use of Coven in the construction of prototype PSEs are shown, and the effectiveness of these PSEs is evaluated in terms of the performance of the applications they generate.

Integral equations solved by the method of moments (MOM) are an important and computationally intense class of problems in antenna design and other areas of electromagnetics. Particularly when structures become electrically large, MOM solutions become intractable as they lead to large, densely-filled, complex matrices, the solution of which is numerically and computationally intensive. Several numerical techniques have been applied to make these solutions more tractable, notably spatial decomposition and a multigrain method known as reduced current fidelity. This paper investigates the impact of these techniques on parallel computing approaches to solving this class of problem, using networks of workstations (NOWs). The resulting effects on the efficiency and communication patterns of the parallel programs are explored.