The JMLR study Do we need hundreds of classifiers to solve real world classification problems? benchmarks 179 classifiers in 17 families on 121 data sets from the UCI repository and claims that “the random forest is clearly the best family of classifier”. In this response, we show that the study's results are biased by the lack of a held-out test set and the exclusion of trials with errors. Further, the study's own statistical tests indicate that random forests do not have significantly higher percent accuracy than support vector machines and neural networks, calling into question the conclusion that random forests are the best classifiers.

Modern field-programmable gate array (FPGA) circuit designs often contain multiple clocks and complex timing constraints, and achieving these constraints requires timing optimization at all stages of the computer-aided design (CAD) flow. To our knowledge, no prior published work has either described or quantitatively evaluated how to compute connection timing criticalities for circuits with multiple timing constraints in order to best guide CAD optimization algorithms. While single-clock techniques have a simple extension to multi-clock circuits, this formulation is not robust for circuits with multiple constraints of different magnitudes, or impossible constraints. We describe a robust method of timing optimization for circuits with multiple timing constraints, implemented in the open-source versatile place and route FPGA CAD tool. Our formulation can optimize multiple constraints well, even in the case where some constraints are impossible, and achieves over 20% greater clock speed with aggressive constraints than a straight-forward extension of single-clock work.

Exploring architectures for large, modern FPGAs requires sophisticated software that can model and target hypothetical devices. Furthermore, research into new CAD algorithms often requires a complete and open source baseline CAD flow. This article describes recent advances in the open source Verilog-to-Routing (VTR) CAD flow that enable further research in these areas. VTR now supports designs with multiple clocks in both timing analysis and optimization. Hard adder/carry logic can be included in an architecture in various ways and significantly improves the performance of arithmetic circuits. The flow now models energy consumption, an increasingly important concern. The speed and quality of the packing algorithms have been significantly improved. VTR can now generate a netlist of the final post-routed circuit which enables detailed simulation of a design for a variety of purposes. We also release new FPGA architecture files and models that are much closer to modern commercial architectures, enabling more realistic experiments. Finally, we show that while this version of VTR supports new and complex features, it has a 1.5× compile time speed-up for simple architectures and a 6× speed-up for complex architectures compared to the previous release, with no degradation to timing or wire-length quality.