Recently, many deep stereo matching methods have begun to focus on cross-domain performance, achieving impressive achievements. However, these methods did not deal with the significant volatility of generalization performance among different training epochs. Inspired by masked representation learning and multi-task learning, this paper designs a simple and effective masked representation for domain generalized stereo matching. First, we feed the masked left and complete right images as input into the models. Then, we add a lightweight and simple decoder following the feature extraction module to recover the original left image. Finally, we train the models with two tasks (stereo matching and image reconstruction) as a pseudo-multi-task learning framework, promoting models to learn structure information and to improve generalization performance. We implement our method on two well-known architectures (CFNet and LacGwcNet) to demonstrate its effectiveness. Experimental results on multi-datasets show that: (1) our method can be easily plugged into the current various stereo matching models to improve generalization performance; (2) our method can reduce the significant volatility of generalization performance among different training epochs; (3) we find that the current methods prefer to choose the best results among different training epochs as generalization performance, but it is impossible to select the best performance by ground truth in practice.

Stereo matching CNNs suffer from performance deteriorate when evaluated under different distributions from training data. Previous domain adaptation/generalization methods are hard to maintain a robust performance in different baselines and usually require difficult adversarial optimization or intricate network structure. To solve this problem, we propose Fourier-based amplitude transform (FAT), mapping the source image to the target style without altering semantic content, which requires no training to perform the domain alignment. Specifically, we leverage the Fourier transform and its inverse to swap the low-frequency amplitude component of the source data with the target data. To effectively map style and relieve the artifacts, we introduce two factors to control the replacing area: the distance of HSV distribution between source and target images; and the difference between the source left image and its warped left image. Experiments testify FAT can significantly bridge domain gaps, making source data distribution closer to target data. Furthermore, when only training on synthetic datasets, FAT can also help different baselines achieve competitive cross-domain generalization capabilities on real datasets.

Most deep learning-based state-of-the-art stereo matching methods significantly depend on large-scale datasets. However, it is implausible to collect sufficient real-world samples with dense and clear ground-truth disparity maps in practice. Although synthetic datasets’ appearance has alleviated the demand for extensive real data, there is a domain shift between synthetic and real sets. To tackle this problem, we propose an individually trained synthetic-to-real domain adaptation (SDA) network that maps synthetic images into the real domain. Specifically, our approach translates the data style from synthetic domain to real domain while maintaining the content and the spatial information. First, edge cues are leveraged to guide domain adaptation in preserving the spatial consistency between input and the generated image. Second, we combine the spatial feature transform (SFT) layer to effectively fuse features from the edge map and the source image. Extensive experiments demonstrate that: 1) when only trained on synthetic data and generalized to real data, our model evidently outperforms many state-of-the-art domain adaptation methods; 2) our translated synthetic datasets (TSD) help to improve the generalization capability of any stereo matching CNNs. Codes and data will be available athttps://github.com/Archaic-Atom/SDA_network.