Vishnepolsky and Pirtskhalava offer two primary critiques of our previous work. We begin by responding to the second of the two critiques, because it involves a clear error on our part. Indeed, as Vishnepolsky and Pirtskhalava point out, the script that we used to calculate results for their tool, DBAASP, included a bug such that the number of true positives should be 306 instead of 121 in Table 1 of our paper. The corrected version of Table 1 is included in this response. We apologize for this error and thank Vishnepolsky and Pirtskhalava for pointing it out. Summary of threshold-based results for the DAMPD benchmark dataset Summary of threshold-based results for the DAMPD benchmark dataset The second critique concerns the inclusion in our benchmark of peptides that are explicitly disallowed as input by some of the tools. Specifically, several of the tools do not make predictions for peptides with lengths >100 amino acid and also do not handle peptides that contain non-canonical amino acids (e.g. ‘X’ as an ambiguity code). On this point, we disagree with Vishnepolsky and Pirtskhalava. Restricting the benchmark dataset to the task of predicting only with respect to short peptides makes the benchmark unrealistically easy. We did acknowledge in the original manuscript that several of the tested tools were not designed to handle longer sequences, and we even explicitly included results from a version of the benchmark in which longer peptides are included (Fig. 4 in the original paper). But we believe that a proper comparison should include realistic peptides, and as we discussed in the manuscript, some actual antimicrobial peptides (AMPs) are longer than 100 amino acids. Similarly, though we did not point this out explicitly, some AMPs in UniProt contain non-canonical amino acids. A tool that can handle these cases is, in our opinion, more generally useful than a tool that cannot handle them. We do agree with Vishnepolsky and Pirtskhalava that explicitly pointing out, in the original manuscript, the impact of the non-canonical amino acids on our results would have been a good idea. Vishnepolsky and Pirtskhalava also suggest that evaluation of their DBAASP tool should be done by taking into account C-terminal amidation. However, the DAMPD database from which we generated our benchmark does not include indications of C-terminal amidation. We therefore made the assumption that the peptides were not modified in this way. Given that the C-terminal amidation information is not provided, we are not sure how Vishnepolsky and Pirtskhalava were able to perform this analysis and achieve the stated improvement in sensitivity of 5% because the details of their analysis were omitted. Finally, Vishnepolsky and Pirtskhalava carry out an additional comparison using a list of AMPs from the DAMPD dataset (L-DAMPD). The results in their Table 4 clearly suggest that CAMPR3(RF) outperforms DBAASP in terms of sensitivity and balanced accuracy. A further analysis using the Delong test, which compares the difference in receiver operating characteristic area for paired tools, would be required to show which method is better in a rank-based setting. These results from the DAMPD dataset, coupled with the fact that CAMPR3(RF) not only predicts whether a peptide is AMP or non-AMP but also provides an associated probability, support our initial claim that CAMPR3(RF) is a generally useful AMP prediction tool. This result is not inconsistent with the possibility, as claimed by Vishnepolsky and Pirtskhalava, that DBAASP works better for some subclasses of AMPs than others. This work was supported by King Abdullah International Medical Research Center [grant #RC16/089]; and by National Institutes of Health award [P41 GM103533]. Conflict of Interest: none declared.

MOTIVATION: Antimicrobial peptides (AMPs) are innate immune molecules that exhibit activities against a range of microbes, including bacteria, fungi, viruses and protozoa. Recent increases in microbial resistance against current drugs has led to a concomitant increase in the need for novel antimicrobial agents. Over the last decade, a number of AMP prediction tools have been designed and made freely available online. These AMP prediction tools show potential to discriminate AMPs from non-AMPs, but the relative quality of the predictions produced by the various tools is difficult to quantify. RESULTS: We compiled two sets of AMP and non-AMP peptides, separated into three categories-antimicrobial, antibacterial and bacteriocins. Using these benchmark data sets, we carried out a systematic evaluation of ten publicly available AMP prediction methods. Among the six general AMP prediction tools-ADAM, CAMPR3(RF), CAMPR3(SVM), MLAMP, DBAASP and MLAMP-we find that CAMPR3(RF) provides a statistically significant improvement in performance, as measured by the area under the receiver operating characteristic (ROC) curve, relative to the other five methods. Surprisingly, for antibacterial prediction, the original AntiBP method significantly outperforms its successor, AntiBP2 based on one benchmark dataset. The two bacteriocin prediction tools, BAGEL3 and BACTIBASE, both provide very good performance and BAGEL3 outperforms its predecessor, BACTIBASE, on the larger of the two benchmarks. CONTACT: [email protected] or [email protected]. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.