Rice leaf disease (RLD) is one of the major factors that cause the decline in production, and the automatic recognition of such diseases under natural field conditions is of great significance for timely targeted rice management. Although many machine learning approaches have been proposed for RLD recognition, scale variation is still a challenging problem that affects prediction accuracy, especially in uncontrolled environments, such as natural fields. Also, the existing RLD data sets are collected in laboratory environments or with a constant scale, which cannot be used to develop the RLD classification algorithms under natural field conditions. To tackle these particular challenges, we propose a multiscale voting mechanism for RLD recognition under natural field conditions. First, data from 26 rice fields were collected to build a data set containing 6046 images of RLD. Afterwards, a feature pyramid was embedded into a mainstream classification architecture (EfficientNet) with a bottom-up and top-down pathway for feature fusion at different scales. To further reduce the inconsistency among multiscaled features, a multiscale voting strategy with regard to probability distribution was proposed to integrate the decisions from various scales. Each proposed module was carefully validated through an ablation study to demonstrate its effectiveness, and the proposed method was compared with a few state-of-the-art algorithms, including the Single Shot MultiBox Detector, Feature Pyramid Networks, Path Aggregation Network, and Bidirectional Feature Pyramid Network. Experimental results have shown that the classification accuracy of our model can reach 90.24%, which is 4.48% higher than that of the original EfficientNet-b0 model and 1.08% higher than that of existing multiscale networks. Finally, we exploit and demonstrate a visualized explanation for the boosted performance from the proposed model. As an extra outcome, our data set and codes are available at http://github.com/huanghsheng/multiscale-voting-mechanism to benefit the whole research community.