Measuring the overhead of memory management in software applications is a hard problem due to the lack of a baseline; it cannot be turned off, only replaced with a different strategy with different tradeoffs. We present a straightforward technique to approximate a baseline that relies on functionfirst simulation and demonstrate its application with a Python memory management overhead analysis.

BACKGROUND: elPrep is an established multi-threaded framework for preparing SAM and BAM files in sequencing pipelines. To achieve good performance, its software architecture makes only a single pass through a SAM/BAM file for multiple preparation steps, and keeps sequencing data as much as possible in main memory. Similar to other SAM/BAM tools, management of heap memory is a complex task in elPrep, and it became a serious productivity bottleneck in its original implementation language during recent further development of elPrep. We therefore investigated three alternative programming languages: Go and Java using a concurrent, parallel garbage collector on the one hand, and C++17 using reference counting on the other hand for handling large amounts of heap objects. We reimplemented elPrep in all three languages and benchmarked their runtime performance and memory use. RESULTS: The Go implementation performs best, yielding the best balance between runtime performance and memory use. While the Java benchmarks report a somewhat faster runtime than the Go benchmarks, the memory use of the Java runs is significantly higher. The C++17 benchmarks run significantly slower than both Go and Java, while using somewhat more memory than the Go runs. Our analysis shows that concurrent, parallel garbage collection is better at managing a large heap of objects than reference counting in our case. CONCLUSIONS: Based on our benchmark results, we selected Go as our new implementation language for elPrep, and recommend considering Go as a good candidate for developing other bioinformatics tools for processing SAM/BAM data as well.

Motivation: Pairwise sequence alignment is undoubtedly a central tool in many bioinformatics analyses. In this paper, we present a generically accelerated module for pairwise sequence alignments applicable for a broad range of applications. In our module, we unified the standard dynamic programming kernel used for pairwise sequence alignments and extended it with a generalized inter-sequence vectorization layout, such that many alignments can be computed simultaneously by exploiting SIMD (single instruction multiple data) instructions of modern processors. We then extended the module by adding two layers of thread-level parallelization, where we (a) distribute many independent alignments on multiple threads and (b) inherently parallelize a single alignment computation using a work stealing approach producing a dynamic wavefront progressing along the minor diagonal. Results: We evaluated our alignment vectorization and parallelization on different processors, including the newest Intel® Xeon® (Skylake) and Intel® Xeon PhiTM (KNL) processors, and use cases. The instruction set AVX512-BW (Byte and Word), available on Skylake processors, can genuinely improve the performance of vectorized alignments. We could run single alignments 1600 times faster on the Xeon PhiTM and 1400 times faster on the Xeon® than executing them with our previous sequential alignment module. Availability and implementation: The module is programmed in C++ using the SeqAn (Reinert et al., 2017) library and distributed with version 2.4 under the BSD license. We support SSE4, AVX2, AVX512 instructions and included UME: SIMD, a SIMD-instruction wrapper library, to extend our module for further instruction sets. We thoroughly test all alignment components with all major C++ compilers on various platforms. Supplementary information: Supplementary data are available at Bioinformatics online.

Summary This paper describes ExaShark, a hybrid n‐dimensional array toolkit offered as a high‐level library for scientists to compute large‐scale simulations. It offers a global‐array–like interface while its runtime can be configured to use shared memory threading techniques, inter‐node distribution techniques, or combinations of both. ExaShark takes advantage of the latest HPC technologies, helping to scale to future generation systems. It has been used to develop several scientific applications including stencil codes, solvers, and matrix factorization algorithms. These applications are used to demonstrate that it improves on the state of the art by providing a user‐friendly, generic API without sacrificing performance.

Abstract Motivation: Post-sequencing DNA analysis typically consists of read mapping followed by variant calling. Especially for whole genome sequencing, this computational step is very time-consuming, even when using multithreading on a multi-core machine. Results: We present Halvade, a framework that enables sequencing pipelines to be executed in parallel on a multi-node and/or multi-core compute infrastructure in a highly efficient manner. As an example, a DNA sequencing analysis pipeline for variant calling has been implemented according to the GATK Best Practices recommendations, supporting both whole genome and whole exome sequencing. Using a 15-node computer cluster with 360 CPU cores in total, Halvade processes the NA12878 dataset (human, 100 bp paired-end reads, 50× coverage) in <3 h with very high parallel efficiency. Even on a single, multi-core machine, Halvade attains a significant speedup compared with running the individual tools with multithreading. Availability and implementation: Halvade is written in Java and uses the Hadoop MapReduce 2.0 API. It supports a wide range of distributions of Hadoop, including Cloudera and Amazon EMR. Its source is available at http://bioinformatics.intec.ugent.be/halvade under GPL license. Contact: [email protected] Supplementary information: Supplementary data are available at Bioinformatics online.

SUMMARY The iPhone SDK provides a powerful platform for the development of applications that make use of iPhone capabilities, such as sensors, GPS, Wi‐Fi, or Bluetooth connectivity. We observe that so far the development of iPhone applications has mostly been restricted to using Objective‐C. However, developing applications in plain Objective‐C on the iPhone OS suffers from limitations, such as the need for explicit memory management and lack of syntactic extension mechanism. Moreover, when developing distributed applications in Objective‐C, programmers have to manually deal with distribution concerns, such as service discovery, remote communication, and failure handling. In this paper, we discuss our experience in porting the Scheme programming language to the iPhone OS and how it can be used together with Objective‐C to develop iPhone applications. To support the interaction between Scheme programs and the underlying iPhone APIs, we have implemented a language symbiosis layer that enables programmers to access the iPhone SDK libraries from Scheme. In addition, we have designed high‐level distribution constructs to ease the development of distributed iPhone applications in an event‐driven style. We validate and discuss these constructs with a series of examples, including an iPod controller, a maps application, and a distributed multiplayer Scrabble‐like game. We discuss the lessons learned from this experience for other programming language ports to mobile platforms. Copyright © 2011 John Wiley & Sons, Ltd.

While most approaches to automatic parallelization focus on compilation approaches for parallelizing loop iterations, we advocate the need for new virtual machines that can parallelize the execution of recursive programs. In this paper, we show that recursive programs can be effectively parallelized when arguments to procedures are evaluated concurrently and branches of conditional statements are speculatively executed in parallel. We introduce the continuator concept, a runtime structure that tracks and manages the control dependences between such concurrently spawned tasks, ensuring adherence to the sequential semantics of the parallelized program. As a proof of concept, we discuss the details of a parallel interpreter for Scheme (implemented in Common Lisp) based on these ideas, and show the results from executing the Clinger benchmark suite for Scheme.

Software transactional memory (STM) is a promising approach for coordinating concurrent threads, for which many implementation strategies are currently being researched. Although some first steps exist to ease experimenting with different strategies, this still remains a relatively complex and cumbersome task. The reason is that software transactions require STM-specific dynamic crosscutting adaptations, but this is not accounted for in current STM implementations. This paper presents CSTM, an STM framework based on Context-oriented Progamming, in which transactions are modelled as dynamically scoped layer activations. It enables expressing transactional variable accesses as user-defined crosscutting concerns, without requiring invasive changes in the rest of a program. This paper presents a proof-of-concept implementation based on ContextL for Common Lisp, along with example STM strategies and preliminary benchmarks, and introduces some of ContextL's unique features for context-dependent variable accesses.

Predicate dispatching is a generalized form of dynamic dispatch, which has strong limitations when arbitrary predicates of the underlying base language are used. Unlike classes, which enforce subset relationships between their sets of instances, arbitrary predicates generally do not designate subsets of each other, so methods whose applicability is based on predicates cannot be ordered according to their specificity in the general case. This paper introduces a decidable but expressive alternative mechanism called filtered dispatch that adds a simple preprocessing step before the actual method dispatch is performed and thus enables the use of arbitrary predicates for selecting and applying methods.

ContextL is an extension to the Common Lisp Object System that allows for Context-oriented Programming. It provides means to associate partial class and method definitions with layers and to activate and deactivate such layers in the control flow of a running program. When a layer is activated, the partial definitions become part of the program until this layer is deactivated. This has the effect that the behavior of a program can be modified according to the context of its use without the need to mention such context dependencies in the affected base program. We illustrate these ideas by providing different UI views on the same object while, at the same time, keeping the conceptual simplicity of object-oriented programming that objects know by themselves how to behave, in our case how to display themselves. These seemingly contradictory goals can be achieved by separating class definitions into distinct layers instead of factoring out the display code into different classes.