Focus measurement, one of the key tasks in multifocus image fusion (MFIF) frameworks, identifies the clearer parts of multifocus images pairs. Most of the existing methods aim to achieve disposable pixel‐level focus measurement. However, the lack of sufficient accuracy often gives rise to misjudgments in the results. To this end, a novel two‐stage focus measurement with joint boundary refinement network is proposed for MFIF. In this work, we adopt a coarse‐to‐fine strategy to gradually achieve block‐level and pixel‐level focus measurement for producing more fine‐grained focus probability maps, instead of directly predicting at the pixel level. In addition, the joint boundary refinement optimizes the performance on the focused/defocused boundary component (FDB) during the focus measurement. To improve feature extraction capability, both CNN and transformer are employed to, respectively, encode local patterns and capture long‐range dependencies. Then, the features from two input branches are legitimately aggregated by modeling the spatial complementary relationship in each pair of multifocus images. Extensive experiments demonstrate that the proposed model achieves state‐of‐the‐art performance in both subjective perception and objective assessment.