MOTIVATION: Whole genome shotgun based next-generation transcriptomics and metagenomics studies often generate 100-1000 GB sequence data derived from tens of thousands of different genes or microbial species. Assembly of these data sets requires tradeoffs between scalability and accuracy. Current assembly methods optimized for scalability often sacrifice accuracy and vice versa. An ideal solution would both scale and produce optimal accuracy for individual genes or genomes. RESULTS: Here we describe an Apache Spark-based scalable sequence clustering application, SparkReadClust (SpaRC), that partitions reads based on their molecule of origin to enable downstream assembly optimization. SpaRC produces high clustering performance on transcriptomes and metagenomes from both short and long read sequencing technologies. It achieves near-linear scalability with input data size and number of compute nodes. SpaRC can run on both cloud computing and HPC environments without modification while delivering similar performance. Our results demonstrate that SpaRC provides a scalable solution for clustering billions of reads from next-generation sequencing experiments, and Apache Spark represents a cost-effective solution with rapid development/deployment cycles for similar large-scale sequence data analysis problems. AVAILABILITY AND IMPLEMENTATION: https://bitbucket.org/berkeleylab/jgi-sparc.

The frequency, nature, and scope of Soft Error Events (SEEs) is becoming an increasingly significant reliability concern as transistors continuously scale down into ever smaller nanometer memory chip geometries. As part of the on-going research efforts required to address this steadily developing situation, two significant research questions were addressed by the research work presented within this paper. The first research question addressed the possibility of crafting a novel methodology whose application would consistently lead to the creation of viable Monte Carlo SEE Simulators. Each of these Monte Carlo SEE Simulators would need to be capable of adequately profiling SEEs as well as simulating and modeling their impacts on the reliability of a variety of simulated 45-nm SRAM computer memories. The Standard 2-Parameter Weibull Soft Error Event (S2P-WSE) Simulator proposed within this paper as a direct application of this novel methodology successfully produced simulation results that were comparatively equivalent both directly and statistically to the results calculated by the empirically validated Weibull Row-Clustered Soft Error Reliability (WRCSER) model. This WRCSER Model was independently developed and empirically validated against proprietary reliability datasets obtained from testing a CISCO 45-nm SRAM computer memory device in a laboratory setting. The S2P-WSE Simulator itself was validated by means of a Kruskal-Wallis ANOVA test and by direct comparison of both the model and the simulator as a function of their respective failure probability and reliability results on 45-nm SRAM simulated computer memory devices subjected to the same SEE impacts over the course of an entire simulation run. The second research question sought to determine the significance of SEE topography versus a Row-Depth-Only bit error severity assumption on the reliability of simulated 45-nm SRAM computer memories shielded from the accumulating impacts of SEEs by variable error-correction codes, row-depth interleaving distances, and periodic scrubbing time intervals. The simulation runs for both the S2P-WSE Simulator and the T2P-WSE Simulator were carefully setup to ensure that only the potentially divergent factor between the results generated by the two simulators was the T2P-WSE Simulator's stochastically estimated topographical shape for each SEE impacting the simulated computer memory device. The work within this paper successfully demonstrated through direct plots of the two simulators' failure probability and reliability results and the results of the Kruskal-Wallis ANOVA tests that the topography of SEEs does represent a significant impact on the overall failure probability and reliability of 45-nm SRAM computer memory devices.

In this paper, a random walk based solver is presented which calculates the capacitances for verifying the touchscreen design. To suit the complicated conductor geometries in touchscreen structures, we extend the floating random walk (FRW) method for handling non-Manhattan conductors. A unified dielectric pre-characterization scheme is proposed to suit arbitrary dielectric profiles while keeping high accuracy. The algorithm is finally implemented on a computer cluster, which enables massively parallel computing. Numerical experiments validate the accuracy of proposed techniques and the up to 67X parallel speedup. While comparing with other schemes, the unified dielectric pre-characterization scheme achieves the highest accuracy while costing the least memory usage.

Summary form only given. In this talk we summarize some of our work in creating computational infrastructure to enable Monte Carlo computations in serial, parallel, distributed, and grid-based environments. We begin with a brief overview of the scalable parallel random number generators (SPRNG) library. This provides high quality pseudorandom numbers in all the above environments. We then discuss specific grid services for Monte Carlo that we recently developed. These services reduce wall clock time and improve the trustworthiness and integrity of grid-based computations. We then discuss quasirandom numbers based on scrambling in this context. Finally, we present results that differ for quasi-Monte Carlo methods on the grid from those presented for pseudorandom numbers.

High performance computing on a large-scale computational grid is complicated by the heterogeneous computational capabilities of each node, node unavailability, and unreliable network connectivity. Replicating computation on multiple nodes can significantly improve performance by reducing task completion time on a grid's dynamic environment. We develop an analytical model to determine the number of task replicas to meet the performance goals in different computational grid configurations. Furthermore, taking advantage of the statistical nature of grid-based Monte Carlo applications, we extend the computational replication technique to an N-out-of-M scheduling strategy for grid-based Monte Carlo applications, which can potentially form a large category of grid-computing applications. In addition, we establish a corresponding model for the N-out-of-M scheduling mechanism. Simulations are used to validate the computational replication models. Our preliminary results show that the models we use are effective in predicting the required number of replicas to achieve short task completion time with a given high probability.

In the hybrid Molecular Dynamics (MD)/Brownian Dynamics (BD) algorithm for simulating the long-time, nonequilibrium dynamics of receptor-ligand interactions, the evaluation of the force autocorrelation function can be computationally costly but fortunately is highly amenable to multimode processing methods. In this paper, taking advantage of the computational grid's large-scale computational resources and the nice characteristics of grid-based Monte Carlo applications, we developed a grid-based receptor-ligand interactions simulation application using the MD/BD algorithm. We expect to provide high-performance and trustworthy computing for analyzing long-time dynamics of proteins and protein-protein interaction to predict and understand cell signaling processes and small molecule drug efficacies. Our preliminary results showed that our grid-based application could provide a faster and more accurate computation for the force autocorrelation function in our MD/BD simulation than previous parallel implementations.

In this article we present background, rationale, and a description of the Scalable Parallel Random Number Generators (SPRNG) library. We begin by presenting some methods for parallel pseudorandom number generation. We will focus on methods based on parameterization, meaning that we will not consider splitting methods such as the leap-frog or blocking methods. We describe, in detail, parameterized versions of the following pseudorandom number generators: (i) linear congruential generators, (ii) shift-register generators, and (iii) lagged-Fibonacci generators. We briefly describe the methods, detail some advantages and disadvantages of each method, and recount results from number theory that impact our understanding of their quality in parallel applications. SPRNG was designed around the uniform implementation of different families of parameterized random number generators. We then present a short description of SPRNG. The description contained within this document is meant only to outline the rationale behind and the capabilities of SPRNG. Much more information, including examples and detailed documentation aimed at helping users with putting and using SPRNG on scalable systems is available at htt;//sprng.sc.fsu.edu. In this description of SPRNG we discuss the random-number generator library as well as the suite of tests of randomness that is an integral part of SPRNG. Random-number tools for parallel Monte Carlo applications must be subjected to classical as well as new types of empirical tests of randomness to eliminate generators that show defects when used in scalable envionments.

In this article we present background, rationale, and a description of the Scalable Parallel Random Number Generators (SPRNG) library. We begin by presenting some methods for parallel pseudorandom number generation. We will focus on methods based on parameterization, meaning that we will not consider splitting methods such as the leap-frog or blocking methods. We describe, in detail, parameterized versions of the following pseudorandom number generators: (1) linear congruential generators, (ii) shift-register generators, and (iii) lagged-Fibonacci generators. We briefly describe the methods, detail some advantages and disadvantages of each method, and recount results from number theory that impact our understanding of their quality of parallel applications. SPRNG was designed around the uniform implementation of different families of parameterized random number generators. We then present a short description of SPRNG. The description contained within this document is meant only to outline the rationale behind and the capabilities of SPRNG. Much more information, including examples and detailed documentation aimed at helping users with putting and using SPRNG on scalable systems is available at http://sprng.cs.fsu.edu. In this description of SPRNG we discuss the random-number generator library as well as the suite of tests of randomness that is an integral part of SPRNG. Random-number tools for parallel Monte Carlo applications must be subjected to classical as well as new types of empirical tests of randomness to eliminate generators that show defects when used in scalable environments.

We describe in detail the parallel implementation of a family of additive lagged-Fibonacci pseudorandom number generators. The theoretical structure of these generators is exploited to preserve their well-known randomness properties and to provide a parallel system of distinct cycles. The algorithm presented here solves the reproducibility problem for a far larger class of parallel Monte Carlo applications than has been previously possible. In particular, Monte Carlo applications that undergo "splitting" can be coded to be reproducible, independent both of the number of processors and the execution order of the parallel processes. A library of portable C routines that implements these ideas is also described.>