BACKGROUND: Microbiome studies commonly use 16S rRNA gene amplicon sequencing to characterize microbial communities. Errors introduced at multiple steps in this process can affect the interpretation of the data. Here we evaluate the accuracy of operational taxonomic unit (OTU) generation, taxonomic classification, alpha- and beta-diversity measures for different settings in QIIME, MOTHUR and a pplacer-based classification pipeline, using a novel software package: DECARD. RESULTS: In-silico we generated 100 synthetic bacterial communities approximating human stool microbiomes to be used as a gold-standard for evaluating the colligative performance of microbiome analysis software. Our synthetic data closely matched the composition and complexity of actual healthy human stool microbiomes. Genus-level taxonomic classification was correctly done for only 50.4-74.8% of the source organisms. Miscall rates varied from 11.9 to 23.5%. Species-level classification was less successful, (6.9-18.9% correct); miscall rates were comparable to those of genus-level targets (12.5-26.2%). The degree of miscall varied by clade of organism, pipeline and specific settings used. OTU generation accuracy varied by strategy (closed, de novo or subsampling), reference database, algorithm and software implementation. Shannon diversity estimation accuracy correlated generally with OTU-generation accuracy. Beta-diversity estimates with Double Principle Coordinate Analysis (DPCoA) were more robust against errors introduced in processing than Weighted UniFrac. The settings suggested in the tutorials were among the worst performing in all outcomes tested. CONCLUSIONS: Even when using the same classification pipeline, the specific OTU-generation strategy, reference database and downstream analysis methods selection can have a dramatic effect on the accuracy of taxonomic classification, and alpha- and beta-diversity estimation. Even minor changes in settings adversely affected the accuracy of the results, bringing them far from the best-observed result. Thus, specific details of how a pipeline is used (including OTU generation strategy, reference sets, clustering algorithm and specific software implementation) should be specified in the methods section of all microbiome studies. Researchers should evaluate their chosen pipeline and settings to confirm it can adequately answer the research question rather than assuming the tutorial or standard-operating-procedure settings will be adequate or optimal.

OBJECTIVE: To reduce wasteful ordering of rare 1,25-OH vitamin D lab tests through use of a noninterruptive decision support tool. MATERIALS AND METHODS: We conducted a time series quality improvement study at 2 academic hospitals. The titles of vitamin D tests and the order in which they appeared in search results were changed to reflect the purpose and rarity of the tests. We used interruptive time series analyses to evaluate the changes we made. RESULTS: The estimated number of monthly tests ordered at the 2 hospitals increased, by 24.8 and 14.2, following the introduction of computerized provider order entry (CPOE) (both P < .001). When we changed the titles of the tests, the estimated number of monthly tests decreased at the 2 hospitals, by 22.1 and 11.3 (both P < .001). The search order did not affect test utilization. DISCUSSION: Changing catalog names in CPOE systems for infrequently used tests can reduce unintentional overuse. Users may prefer this to interruptive or restrictive interventions. CONCLUSION: CPOE vendors and users should refine interfaces by incorporating human factors engineering. Health care institutions should monitor test utilization for unintended changes after CPOE implementation.

UNLABELLED: The execution of a software application or pipeline using various combinations of parameters and inputs is a common task in bioinformatics. In the absence of a specialized tool to organize, streamline and formalize this process, scientists must write frequently complex scripts to perform these tasks. We present nestly, a Python package to facilitate running tools with nested combinations of parameters and inputs. nestly provides three components. First, a module to build nested directory structures corresponding to choices of parameters. Second, the nestrun script to run a given command using each set of parameter choices. Third, the nestagg script to aggregate results of the individual runs into a CSV file, as well as support for more complex aggregation. We also include a module for easily specifying nested dependencies for the SCons build tool, enabling incremental builds. AVAILABILITY: Source, documentation and tutorial examples are available at http://github.com/fhcrc/nestly. nestly can be installed from the Python Package Index via pip; it is open source (MIT license).