Emergence of new sensor technologies and increased parallelism has increased the size of scientific datasets dramatically in recent years. At the same time, research communities are demanding support for reproducible research, in which datasets and results are made available to other researchers. Such datasets do not change frequently, but future experiments can be derived from them.The volume of these datasets requires storage systems that scale across many machines, and also support highly selective queries to minimize the costs associated with unwanted data access.Decentralized Content Addressable Storage (CAS) systems such as the Inter-Planetary File System (IPFS) are promising as a way of disseminating datasets resulting from scientific research since they are scalable, the stored data is immutable, and versioning is straightforward. DS2, our prototype system for decentralized storage of spatial scientific data, bridges the gap between a scientist’s spatial view of a dataset and IPFS, the underlying decentralized filesystem. DS2 is intended to allow selective retrieval of data from very large datasets spanning many hosts.In this paper we investigate the feasibility of DS2 running on top of IPFS to store and retrieve large spatial scientific datasets. We also describe the decentralized metadata structures developed to support storage and retrieval of spatial datasets.

Because of the importance of Delaunay Triangulation in science and engineering, researchers have devoted extensive attention to parallelizing this fundamental algorithm. However, generating unstructured meshes for extremely large point sets remains a barrier for scientists working with large scale or high resolution datasets.

Scientific datasets are steadily growing in size, due to increasing resolution and scale. Unstructured meshes are essential to certain fields of engineering and science, but they present special challenges for efficient access and processing. The work described in this paper accelerates range queries for very large unstructured meshes using the GPU. Prior work in the area introduced a preprocessing phase that partitions large unstructured meshes in order to improve locality in storage and memory. Here, we apply the computational power and bandwidth of GPUs to the partitioning problem, significantly reducing preprocessing time. In order to keep the GPU busy, we have to overcome the poor locality of the original unstructured mesh. Toward this end, we developed our own approach to unstructured mesh I/O, called Direct Load. We show that Direct Load significantly outperforms a typical LRU cache. Our ultimate goal is to accelerate range queries. Our preprocessing steps allow us to parallelize range query processing with relatively simple GPU code. Experimental results show that our implementation outperforms the serial implementations by 4x for preprocessing and over 100 χ for range queries.

Scientific datasets today are often far too large to fit into a single machine's memory or even a single disk. Partitioning multidimensional arrays across several machines or disks has become increasingly necessary. However, relatively little work has been done for unstructured grids composed of a collection of simplicial cells. Our previous work investigated partitioning unstructured grids at the disk level and its effect on overall system performance. In this paper, we build upon prior work by investigating the effect of an in-core partitioning performed on top of the existing disk level partitioning. The granularity of in-core partitioning has varying effect on the overall system performance. Based on our test results, we propose a formula for choosing an effective partitioning for large unstructured grids to facilitate fast data retrieval. We also examine the performance benefits of declustering unstructured grids across several disks. Given this declustered dataset, we describe and explore a parallel data retrieval method that takes advantage of prior knowledge of a user access pattern. Our test results demonstrate very significant performance gains.

Due to the increasing quality of instruments and availability of computational resources, the size of spatial scientific datasets has been steadily increasing. However, much of the research on efficient storage and access to spatial datasets has focused on large multidimensional arrays. In contrast, unstructured datasets consisting of collections of simplices (e.g. triangles or tetrahedra) present special challenges that have received less attention. Data values found at the vertices of the simplices may be dispersed throughout a datafile, producing especially poor disk locality. In this paper, we address this important problem of poor locality in two major ways. First, we reorganize the unstructured dataset to improve locality in both the dataset space and in the data file on disk using a specialized chunking approach that maintains the spatial neighborhood relationships inherent in the unstructured data. This reorganization produces significant gains in performance by reducing the number of accesses made to the data file. Second, we extend our previous work and describe a prefetching method that takes advantage of prior knowledge of the user's access pattern. Applying this prefetching method to unstructured data produces further performance gains over and above the gains seen from reorganization alone.

The implementation of partial or incomplete replicas, which represent only a subset of a larger dataset, has been an active topic of research. Partial Spatial Replicas extend this functionality to spatial data, allowing us to distribute a spatial dataset in pieces over several locations. Accessing only a subset of a spatial replica usually results in a large number of relatively small read requests made to the underlying storage device. For this reason, an accurate model of disk access is important when working with spatial subsets. We make two primary contributions in this paper. First, we describe a model for disk access performance that takes filesystem prefetching into account and is sufficiently accurate for spatial replica selection. Second, making a few simplifying assumptions, we propose a fast replica selection algorithm for partial spatial replicas. The algorithm uses a greedy approach that attempts to maximize performance by choosing a collection of replica subsets that allow fast data retrieval by a client machine. Experiments show that the performance of the solution found by our algorithm is on average always at least 91% and 93.4% of the performance of the optimal solution in 4-node and 8-node tests respectively.

We describe IDEA, an API designed specifically for the parallel processing of large spatial datasets on a cluster. Because such datasets present special challenges for efficient I/O and communication, it is especially valuable to provide an API that frees the user from the burden of partitioning the data among the processors. IDEA allows the user to address a communication to neighboring blocks of data, rather than processes or nodes. In addition to being very natural for the user, this data-centric view allows communication to a data block before it has been assigned a process. This is a key ability when handling data sets larger than the aggregate memory capacity of the cluster, since the dataset must be processed in a piecewise fashion.

High performance parallel computing infrastructures, such as computing clusters, have recently become freely available for scientific researchers to solve problems of unprecedented scale through data parallelization. However scientists are not necessarily skilled in writing efficient parallel code, especially when dealing with spatial datasets. Two important performance issues involved are the heavy I/O costs and the communication overhead. To address this issue, we are developing an scheme that helps scientists realize I/O friendly and scalable data parallelization for spatial computation.

Disk and network latency must be taken into account when applying parallel computing to large multidimensional datasets because they can hinder performance by reducing the rate at which data can be fed to the compute nodes. Existing methods aggregate some number of data requests from cluster nodes to improve overall performance by reducing the number of latency penalties. However, an even more significant reduction can be achieved by taking advantage of prior knowledge of the access pattern expressed as an iteration. Within the context of the granite scientific database system, we created a new iteration aware data distribution system that accelerates data transfer between a data server and the client cluster. This system reduces both disk and network latency by transforming a large number of small requests into a small number of large requests that fill an n-dimensional cache block on the cluster head node

Downloading an entire file is not practical for very large n-dimensional (n-d) datasets, especially if the region of interest (ROI) is small. It is therefore important to develop methods to allow researchers to remotely access n-d subsets of large datasets. Since researchers often wish to access a series of subsets, an awareness of the relationship between dataset storage organization and the application access pattern is important to guarantee effective performance. In our previous work, we reduced the impact of network latency costs even when efficient n-d access is available. This reduction in network latency costs exposes disk latency as an important performance factor. To address this new bottleneck and other problems, we propose the addition of n-d replicas to the data grid

Although processing speed, storage capacity and network bandwidth are steadily increasing, network latency remains a bottleneck for scientists accessing large remote data sets. This problem is most acute with n-dimensional data. Grid researchers have only recently begun to develop tools for efficient remote access to n-dimensional data sets. Within the context of the Granite Scientific Database system, we show that latency penalties can be dramatically reduced using explicit knowledge of a user's access pattern represented as an iterator. The iterator not only performs an n-dimensional iteration for the user, but also communicates the access pattern to Granite so that a prefetching cache can be constructed that is tuned to the user's access pattern. We experimentally evaluate a scenario for incorporating Granite's prefetching mechanism into the grid, demonstrating extraordinary performance gains. In light of these results, we describe planned additions to existing grid services to allow selection of datasets according to the user access pattern