This paper introduces a novel visual analytics approach, DCPViz, to enable climate scientists to explore massive climate data interactively without requiring the upfront movement of massive data. Thus, climate scientists are afforded more effective approaches to support the identification of potential trends and patterns in climate projections and their subsequent impacts. We designed the DCPViz pipeline to fetch and extract NEX-DCP30 data with minimal data transfer from their public sources. We implemented DCPViz to demonstrate its scalability and scientific value and to evaluate its utility under three use cases based on different models and through domain expert feedback.

This paper proposes a visual analytics framework that addresses the complex user interactions required through a command-line interface to run analyses in distributed data analysis systems. The visual analytics framework facilitates the user to manage access to the distributed servers, incorporate data from the source, run data-driven analysis, monitor the progress, and explore the result using interactive visualizations. We provide a user interface embedded with generalized functionalities and access protocols and integrate it with a distributed analysis system. To demonstrate our proof of concept, we present two use cases from the earth science and Sustainable Human Building Ecosystem research domain.

Big data analytics traditionally involves download of massive amounts of datasets to common server/cluster for processing. Analytic process gets slower with increasing size of required data and network conditions. Data scientists also need explicit access to data locations to download required data. Explicit access to required data may not always be granted due to security reasons. To simplify and accelerate the analytics process on distributed big data with security considerations, we proposed the Virtual Information Fabric Infrastructure (VIFI) for data driven discoveries. Instead of moving large amounts of data to a common place of processing, VIFI allows automatic transfer of required analytics programs to the distributed data locations for in-place processing of relevant data. VIFI allows data scientists to conduct and coordinate complex analytics processes on distributed data repositories using containerization technology and open-source workflow design tools. VIFI alleviates users from having detailed knowledge of distributed data locations, as well as required dependencies, installation and configuration of analytical libraries. In this paper, we demonstrate our current and future work to improve the VIFI architecture using previous and additional uses cases, data management layer that simplifies search of relevant data sets through addition of metadata, integration with security policies at different institutions with the proposed VIFI security layer, and the use of a user-friendly web interface to carry different VIFI activities.

Big Data are revolutionizing nearly every aspect of the modern society. One area where this can have a profound positive societal impact is the field of Health Care Informatics (HCI), which faces many challenges. The key idea behind this study is: can we use some of the experience and technical and methodological solutions from the fields that have successfully adapted to the Big Data era, namely astronomy and space science, to help accelerate the progress of HCI? We illustrate this with examples from the Virtual Observatory framework, and the NCI EDRN project. An effective sharing and reuse of tools, methods, and experiences from different fields can save a lot of effort, time, and expense. HCI can thus benefit from the proven solutions to big data challenges from other domains.

In this paper, we describe an emergent tool called DAWN (short for "Distributed Analytics, Workflows and Numeric") which is a model for simulating, analyzing and optimizing system architectures for executing arbitrary data processing pipelines. As an example, we will apply DAWN to the investigation of a real-life Big Data use case in climate science: the evaluation of simulated rainfall characteristics using high-resolution observational data. We will show how DAWN can help in determining the optimal architecture, and science algorithms, to execute this case study analyzing distributed datasets, as a tradeoff between the overall time cost and the uncertainty of calculated metrics for model evaluation. We will also show how DAWN can guide architectural decisions for future research, specifically impacting how data should be generated and analyzed to cope with future projected data volumes.

The National Institutes of Health (NIH), National Cancer Institute's Early Detection Research Network (EDRN) is a cross-institutional collaborative initiative seeking to accelerate the clinical application of cancer biomarker research. Over the past decade, it has been our role, as EDRN's Informatics Center (IC), to develop a comprehensive information services infrastructure as well as a set of software tools and services to support this overall initiative. We have recently developed a novel application called the Laboratory Catalog and Archive Service (LabCAS) whose focus is to extend EDRN IC data management and processing capabilities to EDRN laboratories. By leveraging the same technologies used to manage and process NASA Earth and Planetary data sets, we offer EDRN researchers an effective way of managing their laboratory data. More specifically, LabCAS enables EDRN researchers to reliably archive their experimental data, to optionally share these data in a controlled manner with other researchers, and to gain insight into these data through highly configurable data analysis pipelines tailored to the broad range of biomarker related experiments. This particular collaboration leverages expertise from NASA's Jet Propulsion Laboratory, Vanderbilt University Medical Center, and Dartmouth Medical School, as well as builds upon existing cross-governmental collaboration between NASA and the NIH.

Research has shown that the amount of data now available often overwhelms key functions of an information system. This situation necessitates the design of information architectures that scale to meet the challenges. The Planetary Data System, a NASA funded project, has developed an information architecture for the planetary science community that addresses this and other big science data issues noted in a National Research Council report regarding architectures for big data management and analysis and end-to-end data lifecycle management across diverse disciplines. The report identified enabling technology trends including distributed systems, service-oriented architectures, ontologies, models and information representation, scalable database systems, federated data security mechanisms, and technologies for moving big data. This paper will present the PDS4 information architecture, its successful implementation in a multi-discipline big-data environment.

The Earth System Grid Federation (ESGF) is a multi-agency, international collaboration that aims at developing the software infrastructure needed to facilitate and empower the study of climate change on a global scale. The ESGF's architecture employs a system of geographically distributed peer nodes, which are independently administered yet united by the adoption of common federation protocols and application programming interfaces (APIs). The cornerstones of its interoperability are the peer-to-peer messaging that is continuously exchanged among all nodes in the federation; a shared architecture and API for search and discovery; and a security infrastructure based on industry standards (OpenID, SSL, GSI and SAML). The ESGF software is developed collaboratively across institutional boundaries and made available to the community as open source. It has now been adopted by multiple Earth science projects and allows access to petabytes of geophysical data, including the entire model output used for the next international assessment report on climate change (IPCC-AR5) and a suite of satellite observations (obs4MIPs) and reanalysis data sets (ANA4MIPs).

The Laura P. and Leland K. Whittier Virtual Pediatric Intensive Care Unit (VPICU) is an ambitious research network focused on building online databases for improving decision-making in pediatric intensive care units. Increasingly, there is a need to unify previously distributed and heterogeneous information captured in these databases to support both traditional retrospective support ad-hoc studies, and ad-hoc analyses. VPICU and NASA's Jet Propulsion Laboratory have constructed a reference architecture and implementation framework that addresses these needs. The architecture is unobtrusive, scalable, and secure, with a strong focus on rapid deployment and integration. This paper reports on the current status of our efforts and details the strength of the framework via our recent work in unsupervised discovery of patient similarity within the hospital.

The dramatic increase in data in the area of cancer research has elevated the importance of effectively managing the quality and consistency of research results from multiple providers. The U.S. National Cancer Institute's Early Detection Research Network (EDRN) is a prime example of a virtual organization, sponsoring distributed, collaborative work at dozens of institutions around the country. As part of a comprehensive informatics infrastructure, The NASA Jet Propulsion Laboratory, in collaboration with Dartmouth Medical School, has developed a web application for the curation of cancer biomarker research results. In this paper, we describe and evaluate the application in the context of the EDRN content management process, and detail our experience using the tool in an operational environment to capture and annotate biomarker research data generated by the EDRN.

Informatics in biomedicine is becoming increasingly interconnected via distributed information services, interdisciplinary correlation, and crossinstitutional collaboration. Partnering with NASA, the Early Detection Research Network (EDRN), a program managed by the National Cancer Institute, has been defining and building an informatics architecture to support the discovery of biomarkers in their earliest stages. The architecture established by EDRN serves as a blueprint for constructing a set of services focused on the capture, processing, management and distribution of information through the phases of biomarker discovery and validation.

Modern scientific research is increasingly conducted by virtual communities of scientists distributed around the world. The data volumes created by these communities are extremely large, and growing rapidly. The management of the resulting highly distributed, virtual data systems is a complex task, characterized by a number of formidable technical challenges, many of which are of a software engineering nature. In this paper we describe our experience over the past seven years in constructing and deploying OODT, a software framework that supports large, distributed, virtual scientific communities. We outline the key software engineering challenges that we faced, and addressed, along the way. We argue that a major contributor to the success of OODT was its explicit focus on software architecture. We describe several large-scale, real-world deployments of OODT, and the manner in which OODT helped us to address the domain-specific challenges induced by each deployment.

The sheer amount of data produced by modern science research has created a need for the construction and understanding of "data-intensive systems", large-scale, distributed systems which integrate information. The formal nature of constructing such software systems; however, is relatively unstudied, and has been a large focus of the super-computing and distributed computing communicates, rather than the software engineering communities. These data-intensive systems exhibit characteristics which appear fruitful for research from a software engineering, and software architectural focus. From our experience, the methodologies and notations for design and implementation of data-intensive systems look to be a good starting point for this important research area. This paper presents our experience with OODT (object-oriented data technology), a software architectural style, and middleware-based implementation for data-intensive systems developed and maintained at the Jet Propulsion Laboratory. To date, OODT has been successfully evaluated in several different science domains including Planetary Science with NASA's Planetary Data System (PDS) and Cancer Research with the National Cancer Institute (NCI).

Access to biospecimens is essential for enabling cancer biomarker discovery. The National Cancer Institute's (NCI) Early Detection Research Network (EDRN) comprises and integrates a large number of cancer research institutions into a network in order to establish a collaborative scientific environment to discover and validate disease markers. The diversity of both the institutions and the collaborative focus has created the need for establishing cross-disciplinary teams focused on integrating expertise in cancer research, computational and biostatistics, and computer science. Given the collaborative design of the network, the EDRN needed an informatics infrastructure. The Fred Hutchinson Cancer Research Center, the National Cancer Institute, and NASA's Jet Propulsion Laboratory (JPL) teamed up to build an informatics infrastructure creating a collaborative, science-driven research environment despite the geographic and morphologic differences of the information systems that existed within the diverse network. EDRN investigators identified the need to share biospecimen data captured across the country managed in disparate databases. As a result, the informatics team initiated an effort to create a virtual biospecimen database whereby scientists could search and retrieve details about specimens located at collaborating institutions. Each database, however, was locally implemented and integrated into collection processes and methods unique to each institution. This meant that efforts to integrate databases needed to be done in a manner that did not require redesign or re-implementation of existing systems.

Knowledge discovery and data correlation require a unified approach to basic data management. However, achieving such an approach is nearly impossible with hundreds of disparate data sources, legacy systems and data formats. This problem is pervasive in the biomedical research community, where data models, taxonomies and data management systems are locally implemented. These local implementations create an environment where interoperability and collaboration between researchers and research institutions are limited. The authors demonstrate how technology developed by NASA's Jet Propulsion Laboratory (JPL) for space science can be used to build an interoperable data architecture for bioinformatics. JPL has taken a novel approach towards solving this problem by exploiting Web technologies usually dedicated to e-commerce, combined with a rich, metadata-based environment. This paper discusses the approach taken to develop a prototype data architecture for the discovery and validation of disease biomarkers within a biomedical research network. Biomarkers are measured parameters of normal biological processes, pathogenic processes or pharmacological responses to a therapeutic intervention. Biomarkers are of growing importance in biomedical research for therapeutic discovery, disease prevention and detection. A bioinformatics infrastructure is crucial to support the integration and analysis of large, complex biological and epidemiological data sets.