Discriminative correlation filters (DCFs) have shown promising tracking performance in recent years thanks to the powerful representation ability of deep features. However, a large number of target-irrelevant channels in deep features limits the tracking performance and increases the computational cost. To eliminate the negative impact of noisy channels and improve the utilization efficiency of deep features in DCF-based trackers, we present an adaptive feature channel selection method for robust visual tracking. Our method adaptively chooses the most discriminative channels to learn a more robust target appearance model, which is achieved by evaluating the energy relationship between background and foreground in each feature channel. Moreover, according to the feedback of channel selection, an adaptive model update strategy is proposed to alleviate the model degradation problem caused by incorrect model updating. Extensive experimental results obtained on five popular tracking benchmarks demonstrate the effectiveness of the proposed algorithm and its superiority over the state-of-the-art trackers.