Research in computational biology has given rise to a vast number of methods developed to solve scientific problems.For areas in which many approaches exist, researchers have a hard time deciding which tool to select to address a scientific challenge, as essentially all publications introducing a new method will claim better performance than all others.Not all of these claims can be correct.Equally, for this same reason, developers struggle to demonstrate convincingly that they created a new and superior algorithm or implementation.Moreover, the developer community often has difficulty discerning which new approaches constitute true scientific advances for the field.The obvious answer to this conundrum is to develop benchmarks-meaning standard points of reference that facilitate evaluating the performance of different tools-allowing both users and developers to compare multiple tools in an unbiased fashion.Broadly speaking, benchmarks consist of input data that methods are meant to operate upon, expected output data against which tool output can be compared, a specification of metrics used to assess performance, and performance values of sets of tools that have been run through the benchmark.Developing good and comprehensive benchmarks, in which the performance metrics of each tool reflect its real-world utility, requires a significant effort.For highly competitive and established fields, such as protein structure predictions, community experiments evaluating the methods have been held periodically to provide blinded assessments of prediction performance.These blinded assessments are perhaps the gold standard on how benchmarks should be run.However, in most areas of computational biology, no such regular blinded contests are available.Instead, many tool developers end up generating their own benchmarks, which they publish alongside a newly developed tool to show its improved performance.The downside of this approach is that, if a new approach is developed in parallel to assembly of the benchmark on which it is evaluated, there is a strong selection bias encouraging the authors to report tool development approaches performing well against the benchmark compared to previous tools.This reporting bias makes most benchmarks that accompany newly developed tools questionable.Even if the authors are aware of this problem and take conscious steps to separate

Library of Science, has evolved its own gloss from the piquantly camelesque "PLoS" to the modern "

MOTIVATION: Rapid methods for protein structure search enable biological discoveries based on flexibly defined structural similarity, unleashing the power of the ever greater number of solved protein structures. Projection methods show promise for the development of fast structural database search solutions. Projection methods map a structure to a point in a high-dimensional space and compare two structures by measuring distance between their projected points. These methods offer a tremendous increase in speed over residue-level structural alignment methods. However, current projection methods are not practical, partly because they are unable to identify local similarities. RESULTS: We propose a new projection-based approach that can rapidly detect global as well as local structural similarities. Local structural search is enabled by a topology-inspired writhe decomposition protocol that produces a small number of fragments while ensuring that similar structures are cut in a similar manner. In benchmark tests, we show that our method, writher, improves accuracy over existing projection methods in terms of recognizing scop domains out of multi-domain proteins, while maintaining accuracy comparable with existing projection methods in a standard single-domain benchmark test. AVAILABILITY: The source code is available at the following website: http://compbio.berkeley.edu/proj/writher/.

Scientific research is an international endeavor, and computational biology is no exception. Last year we were fortunate enough to attend conferences and visit laboratories in a number of different countries and developing economies. Some of those countries had well-established programs in computational biology, while others had fledgling efforts in place and big plans. Because of its cost structure, computational biology is a particularly promising field for emerging economies. Each country we visited had unique features in areas such as the education programs they offered, the types of research being undertaken, and the ways that research is funded. Each nation also has unique challenges, and many nations are developing so quickly that people who leave them for study abroad may find programs entirely different when they return. Given these differences and rapid developments, we feel that there is much we can learn from each other. To this end we have begun a series of Perspective articles from computational biologists in a variety of countries, each of whom offers their personal perspectives on the history, current status, and future of computational biology in their region. We asked the authors to describe the specific challenges they have faced, their perceived strengths, as well as to discuss the institutions (government and private), opportunities, and difficulties of computational biology in their country as a whole. This month we begin the series with a Perspective on the development of computational and genomic biology in Mexico from two of the leaders in the field. Dr. Rafael Palacios de la Lama is a foreign member of the US National Academy of Sciences and one of the world's experts on the genetics of Rhizobium and genome dynamics. Dr. Julio Collado-Vides is currently a professor and the Director of the Center for Genomic Sciences at the Universidad Nacional Autonoma de Mexico, a leader in understanding regulation of the complete Escherichia coli genome. In their article, we learn about the pioneering effort of a Mexican research group participating in the E. coli genome project. E. coli was the first among still few bacterial genomes with quality predictions of operons and upstream regulatory elements. Various other ambitious projects are under way in Mexico, such as determining the DNA sequence of the genome of Phaseolus vulgaris. To these ends, the authors educate an elite set of undergraduates in genomic sciences. Beyond Mexico, in the coming months we will be touring a number of countries where computational biology is expanding. The Intelligent Systems in Molecular Biology (ISMB) conference brought close to a thousand computational biologists to Fortaleza, Brazil, in the summer of 2006. Memories of that experience will revive as we read about the character and nature of research efforts that are being undertaken in Brazil. We will learn also about the unique challenges of doing computational biology in Cuba, a country impacted by the US embargo, yet with a determined education system and a strong research emphasis. The passion of Cuban scientists is an inspiration, and their soup-to-nuts approach to research represents an interesting solution to their situation. We can also expect perspectives from South Africa, Thailand, Argentina, and China, shedding light on current approaches in these countries to a young, rapidly evolving, and changing discipline. We hope reading these accounts will inspire you to comment, or to write a Perspective on computational biology in your part of the world. If you are interested, please e-mail us at gro.solp@loibpmocsolp, and we will be happy to send you guidelines for a submission. The pursuit of scientific endeavors around the world allows individuals and nations to capitalize on their potential. It furthers progress and mutual understanding where political and other means have reached their limits. We hope you will learn from the differing approaches and enjoy reading these viewpoints as much as we did.

We present a simple statistical model of molecular function evolution to predict protein function. The model description encodes general knowledge of how molecular function evolves within a phylogenetic tree based on the proteins' sequence. Inputs are a phylogeny for a set of evolutionarily related protein sequences and any available function characterizations for those proteins. Posterior probabilities for each protein are used to predict the molecular function of that protein. We present results from applying our model to three protein families, and compare our prediction results on the extant proteins to other available protein function prediction methods. For the deaminase family, our method achieves 93.9% where related methods BLAST achieves 72.7%, GOtcha achieves 87.9%, and Orthostrapper achieves 72.7% in prediction accuracy.

Summary: ANDY (seArch coordination aND analYsis) is a set ofPerl programs and modules for distributing large biological databasesearches, and in general any sequence of commands, across the nodes of aLinux computer cluster. ANDY is compatible with several commonly usedDistributed Resource Management (DRM) systems, and it can be easilyextended to new DRMs. A distinctive feature of ANDY is the choice ofeither dedicated or fair-use operation: ANDY is almost as efficient assingle-purpose tools that require a dedicated cluster, but it runs on ageneral-purpose cluster along with any other jobs scheduled by a DRM.Other features include communication through named pipes for performance,flexible customizable routines for error-checking and summarizingresults, and multiple fault-tolerance mechanisms. Availability: ANDY isfreely available and may be obtained fromhttp://compbio.berkeley.edu/proj/andy; this site also containssupplemental data and figures and amore detailed overview of thesoftware.

MOTIVATION: The observed correlations between pairs of homologous protein sequences are typically explained in terms of a Markovian dynamic of amino acid substitution. This model assumes that every location on the protein sequence has the same background distribution of amino acids, an assumption that is incompatible with the observed heterogeneity of protein amino acid profiles and with the success of profile multiple sequence alignment. RESULTS: We propose an alternative model of amino acid replacement during protein evolution based upon the assumption that the variation of the amino acid background distribution from one residue to the next is sufficient to explain the observed sequence correlations of homologs. The resulting dynamical model of independent replacements drawn from heterogeneous backgrounds is simple and consistent, and provides a unified homology match score for sequence-sequence, sequence-profile and profile-profile alignment.

MOTIVATION: Standard algorithms for pairwise protein sequence alignment make the simplifying assumption that amino acid substitutions at neighboring sites are uncorrelated. This assumption allows implementation of fast algorithms for pairwise sequence alignment, but it ignores information that could conceivably increase the power of remote homolog detection. We examine the validity of this assumption by constructing extended substitution matrices that encapsulate the observed correlations between neighboring sites, by developing an efficient and rigorous algorithm for pairwise protein sequence alignment that incorporates these local substitution correlations and by assessing the ability of this algorithm to detect remote homologies. RESULTS: Our analysis indicates that local correlations between substitutions are not strong on the average. Furthermore, incorporating local substitution correlations into pairwise alignment did not lead to a statistically significant improvement in remote homology detection. Therefore, the standard assumption that individual residues within protein sequences evolve independently of neighboring positions appears to be an efficient and appropriate approximation.

MOTIVATION: Protein sequence comparison methods are routinely used to infer the intricate network of evolutionary relationships found within the rapidly growing library of protein sequences, and thereby to predict the structure and function of uncharacterized proteins. In the present study, we detail an improved statistical benchmark of pairwise protein sequence comparison algorithms. We use bootstrap resampling techniques to determine standard statistical errors and to estimate the confidence of our conclusions. We show that the underlying structure within benchmark databases causes Efron's standard, non-parametric bootstrap to be biased. Consequently, the standard bootstrap underpredicts average performance when used in the context of evaluating sequence comparison methods. We have developed, as an alternative, an unbiased statistical evaluation based on the Bayesian bootstrap, a resampling method operationally similar to the standard bootstrap. RESULTS: We apply our analysis to the comparative study of amino acid substitution matrix families and find that using modern matrices results in a small, but statistically significant improvement in remote homology detection compared with the classic PAM and BLOSUM matrices. AVAILABILITY: The sequence sets and code for performing these analyses are available from http://compbio.berkeley.edu/. CONTACT: [email protected].

DOAJ is a unique and extensive index of diverse open access journals from around the world, driven by a growing community, committed to ensuring quality content is freely available online for everyone.

We present a statistical graphical model to infer specific molecular function for unannotated protein sequences using homology. Based on phylogenomic principles, SIFTER (Statistical Inference of Function Through Evolutionary Relationships) accurately predicts molecular function for members of a protein family given a reconciled phylogeny and available function annotations, even when the data are sparse or noisy. Our method produced specific and consistent molecular function predictions across 100 Pfam families in comparison to the Gene Ontology annotation database, BLAST, GOtcha, and Orthostrapper. We performed a more detailed exploration of functional predictions on the adenosine-5'-monophosphate/adenosine deaminase family and the lactate/malate dehydrogenase family, in the former case comparing the predictions against a gold standard set of published functional characterizations. Given function annotations for 3% of the proteins in the deaminase family, SIFTER achieves 96% accuracy in predicting molecular function for experimentally characterized proteins as reported in the literature. The accuracy of SIFTER on this dataset is a significant improvement over other currently available methods such as BLAST (75%), GeneQuiz (64%), GOtcha (89%), and Orthostrapper (11%). We also experimentally characterized the adenosine deaminase from Plasmodium falciparum, confirming SIFTER's prediction. The results illustrate the predictive power of exploiting a statistical model of function evolution in phylogenomic problems. A software implementation of SIFTER is available from the authors.

The exponentially growing library of known protein sequences represents molecules connected by, an intricate network of evolutionary and functional relationships. To reveal these relationships, virtually every molecular biology experiment incorporates computational sequence analysis. The workhorse methods for this task make alignments between two sequences to measure their similarity. Informed use of these methods, such as NCBI BLAST, WU-BLAST, FASTA and SSEARCH, requires understanding of their effectiveness. To permit informed sequence analysis, we. have assessed the effectiveness of modern versions of these algorithms using the trusted relationships among ASTRAL sequences in the Structural Classification of Proteins database classification of protein structures. We have reduced database representation artifacts through the use of a normalization method that addresses the uneven distribution of superfamily sizes. To allow for more meaningful and interpretable comparisons of results, we have implemented a bootstrapping procedure. We find that the most difficult pairwise relations to detect are those between members of larger superfamilies, and our test set is biased toward these. However even when results are normalized, most distant evolutionary relationships elude detection.