This paper focuses on fisheye image rectification. Existing learning-based solutions learn image representations that mix distortion features and content features. Since the distortion feature dominates the rectification process, we propose a novel distortion-aware representation learning framework, which decouples the distortion feature from the content feature, for fisheye image rectification. Specifically, we first pre-train a Vision Transformer with a supervised pre-text task, which regresses the distortion distribution map of a distorted image. The pre-training equips the Vision Transformer with the ability to capture distortion-related patterns. After that, the pre-trained model is fine-tuned to predict the pixel-wise flow map to rectify the fisheye images. Extensive experiments are conducted to evaluate our approach and verify our idea of feature decoupling. The experiment results demonstrate the state-of-the-art performance of our approach compared to existing algorithms, as well as its generality on real-world images. Our source code is publicly available at https://github.com/lzk9508/DaFIR.

Camera lenses often suffer from optical aberrations, causing radial distortion in the captured images. In those images, there exists a clear and general physical distortion model. However, in existing solutions, such rich geometric prior is under-utilized, and the formulation of an effective prediction target is under-explored. To this end, we introduce Radial Distortion TRansformer (RDTR), a new framework for radial distortion rectification. Our RDTR includes a model-aware pre-training stage for distortion feature extraction and a deformation estimation stage for distortion rectification. Technically, on the one hand, we formulate the general radial distortion (i.e., barrel distortion and pincushion distortion) in camera-captured images with a shared geometric distortion model and perform a unified model-aware pre-training for its learning. With the pre-training, the network is capable of encoding the specific distortion pattern of a radially distorted image. After that, we transfer the learned representations to the learning of distortion rectification. On the other hand, we introduce a new prediction target called backward warping flow for rectifying images with any resolution while avoiding image defects. Extensive experiments are conducted on our synthetic dataset, and the results demonstrate that our method achieves state-of-the-art performance while operating in real-time. Besides, we also validate the generalization of RDTR on real-world images. Our source code and the proposed dataset are publicly available at https://github.com/wwd-ustc/RDTR.