MOTIVATION: Rapid technological progress in DNA sequencing has stimulated interest in compressing the vast datasets that are now routinely produced. Relatively little attention has been paid to compressing the quality scores that are assigned to each sequence, even though these scores may be harder to compress than the sequences themselves. By aggregating a set of reads into a compressed index, we find that the majority of bases can be predicted from the sequence of bases that are adjacent to them and, hence, are likely to be less informative for variant calling or other applications. The quality scores for such bases are aggressively compressed, leaving a relatively small number at full resolution. As our approach relies directly on redundancy present in the reads, it does not need a reference sequence and is, therefore, applicable to data from metagenomics and de novo experiments as well as to re-sequencing data. RESULTS: We show that a conservative smoothing strategy affecting 75% of the quality scores above Q2 leads to an overall quality score compression of 1 bit per value with a negligible effect on variant calling. A compression of 0.68 bit per quality value is achieved using a more aggressive smoothing strategy, again with a very small effect on variant calling. AVAILABILITY: Code to construct the BWT and LCP-array on large genomic data sets is part of the BEETL library, available as a github repository at [email protected]:BEETL/BEETL.git.

MOTIVATION: FASTQ is a standard file format for DNA sequencing data, which stores both nucleotides and quality scores. A typical sequencing study can easily generate hundreds of gigabytes of FASTQ files, while public archives such as ENA and NCBI and large international collaborations such as the Cancer Genome Atlas can accumulate many terabytes of data in this format. Compression tools such as gzip are often used to reduce the storage burden but have the disadvantage that the data must be decompressed before they can be used. Here, we present BEETL-fastq, a tool that not only compresses FASTQ-formatted DNA reads more compactly than gzip but also permits rapid search for k-mer queries within the archived sequences. Importantly, the full FASTQ record of each matching read or read pair is returned, allowing the search results to be piped directly to any of the many standard tools that accept FASTQ data as input. RESULTS: We show that 6.6 terabytes of human reads in FASTQ format can be transformed into 1.7 terabytes of indexed files, from where we can search for 1, 10, 100, 1000 and a million of 30-mers in 3, 8, 14, 45 and 567 s, respectively, plus 20 ms per output read. Useful applications of the search capability are highlighted, including the genotyping of structural variant breakpoints and 'in silico pull-down' experiments in which only the reads that cover a region of interest are selectively extracted for the purposes of variant calling or visualization. AVAILABILITY AND IMPLEMENTATION: BEETL-fastq is part of the BEETL library, available as a github repository at github.com/BEETL/BEETL.

Asynchronous circuit design shows many similarities with software design. This is due to the modular construction style associated with asynchronous circuits, where each asynchronous module can be designed as a standalone object communicating with other modules. We propose a visualisation system for asynchronous circuit design tools, based on clustered graph visualisation and coordinated views. The novelty of our approach is to apply classical software visualisation techniques to a hardware environment and take advantage of properties specific to our asynchronous design flow. This system is based on a control and data flow graph representation of the asynchronous circuit compiled from a high-level description by a syntax-directed transparent process and transferable to the final hardware circuit by a direct synthesis process. Compared to software compilation flows, this transparent design flow offers opportunities for visualisation, with representations at different stages being easily mapped onto each other to combine their visualisation qualities. Each representation also exhibits properties based on some physical attributes of the final hardware circuit, allowing us to render some abstract properties on top of real-world-based ones. This paper shows how the handshake circuit graph is used as the underlying structure onto which properties extracted from other structures are mapped, and then how this structured graph is used as the central piece in a coordinated views environment. These visualisation techniques have been exercised by designers optimising the design of a fully asynchronous ARM processor.