Over the years, interest in computational storage devices has been growing steadily. This is largely due to the rise of data-intensive applications, such as machine learning, online video distribution, astrophysics, and genomics. Moving compute operations closer to the data provides benefits in terms of scaling possibilities and energy efficiency. The development of computational storage devices has been limited by the need for specialized and complex hardware. In this work, we propose a portable Linux-based firmware framework for the development of NVMe computational storage devices. Our firmware runs on a variety of hardware platforms ranging from expensive FPGA solutions to inexpensive off-the-shelf single board computers. The firmware leverages the vast Linux software ecosystem to facilitate the development and prototyping of novel computational storage devices. We benchmark our firmware on multiple hardware platforms and demonstrate its versatility through several computational examples including a content-aware disk image search engine based on natural language processing and AI-driven image recognition.

MOTIVATION: Generation of genotype data has been growing exponentially over the last decade. With the large size of recent datasets comes a storage and computational burden with ever increasing costs. To reduce this burden, we propose XSI, a file format with reduced storage footprint that also allows computation on the compressed data and we show how this can improve future analyses. RESULTS: We show that xSqueezeIt (XSI) allows for a file size reduction of 4-20× compared with compressed BCF and demonstrate its potential for 'compressive genomics' on the UK Biobank whole-genome sequencing genotypes with 8× faster loading times, 5× faster run of homozygozity computation, 30× faster dot products computation and 280× faster allele counts. AVAILABILITY AND IMPLEMENTATION: The XSI file format specifications, API and command line tool are released under open-source (MIT) license and are available at https://github.com/rwk-unil/xSqueezeIt. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

The Pair-HMM forward algorithm is an essential algorithm found in many genomic related analyses. The high number of floating point operations in the algorithm makes it one of the main contributors to the compute time of analysis pipelines. To speed-up computations we propose an FPGA-based hardware accelerator for the Amazon AWS F1 Cloud platform. The accelerator is open source and has been tested within the popular Genomic Analysis Toolkit (GATK) pipeline. The accelerator achieved up to 15 × speed-up against the software implementation when used in-pipeline. The accelerator has also been tested in the experimental Spark (distributed) version of the GATK HaplotypeCaller tool. An in-depth analysis of the compute time contributions allowed to point out the main bottlenecks for accelerators in the GATK pipeline, resulting in a hybrid CPU-FPGA solution to best exploit both resources.