First break picking is crucial for estimating and simulating surface and shallow medium velocities, particularly in complex mountainous regions. Here, the significant variations in near-surface elevation, the thickness of low-velocity zones, and lateral changes in near-surface velocity result in noticeable differences in trace-to-trace arrival times. The manual picking method is inefficient and unrealistic on massive seismic data. Accordingly, various automatic picking methods have been developed over time, including approaches utilizing seismic record attributes and deep learning techniques, among others. In this paper, we proposed a feature template based nonlinear trace time difference (FT-NltD) correction method to correct nonlinear time difference, turning the nonlinear time difference to linear. Then, we proposed geometric structure constraint multi-attribute Markov decision process (GCMDP) for robust and high-precision automatic first break picking. In the GCMDP, we use linear geometry structure as constraint, dynamically optimize in the picking process, and apply multi-step prediction to the first break position and multi-attribute constraint at the same time, so as to realize the first break picking stably. Finally, we use field data demonstrate the effectiveness of the proposed first break picking method.