In recent years, stereo image super-resolution based on convolutional neural network has been extensively researched and achieved impressive performance by introducing complementary information from another view. However, most existing methods still cannot fully capture both intra- and cross-view information due to the neglect of multi-scale information perception, multi-scale binocular alignment and the excitation of large scale to small scale in human vision system. And they generated blurry results due to the consideration of irrelevant information in search for cross-view information. To address these issues, we propose a multi-scale visual perception based progressive feature interaction network (MS-PFINet) for stereo image super-resolution. Specifically, to exploit comprehensive intra- and cross-view information for image reconstruction, we design a two-stream network with multi-branch structure to extract multi-scale features and progressively use cross-view interaction at larger scales to guide that at smaller scales. Moreover, to explore more proper and accurate cross-view information, we propose a feature transformer module (FTM) to search and transfer the most relevant features from another view by hard attention maps and soft attention maps, which are calculated by patch-wise similarity rather than pixel-wise. In addition, in order to encourage a more effective way to transfer texture features for the target view, we propose a perceptual texture matching loss to supervise the accuracy of feature transformer modules. Experimental results show that our proposed method is superior to the state-of-the-art methods in most cases.

Recently, parallax attention based stereo image super-resolution (SR) methods, which can better explore cross-view information, have been widely studied. Despite the impressive performance of these methods, almost all of them calculate parallax attention maps at a single low resolution, which will lead to ambiguous stereo correspondence. Besides, the widely used parallax attention module (PAM) cannot handle the illuminance variations in stereo image pairs, and cannot distinguish the contribution of the captured cross-view features to the reconstruction of the target view. To this end, in this paper, we propose a coarse-to-fine cross-view interaction based network (C2FNet) to achieve more accurate cross-view information capturing. Firstly, in C2FNet, a coarse-to-fine cascaded parallax attention structure (C2F-CPAS), which conforms with the human visual mechanism, is constructed to gradually perform parallax attention from the low-resolution to high-resolution level. Thus, richer textures can be used to learn more reliable stereo correspondence. Meanwhile, a multi-level attention transfer loss is designed to further calibrate the accuracy of stereo correspondence at each level. Secondly, we propose a modified PAM (MPAM) to alleviate the limitations of common PAM so that illuminance-robust stereo correspondence can be learned and more important cross-view information can be selected. Extensive experimental results show that our proposed C2FNet outperforms the state-of-the-art methods on various datasets.

Considering that the human brain always follows a coarse-to-fine (low-to-high spatial frequency) visual processing and fusion mechanism, we propose a coarse-to-fine feedback guidance based stereo image quality assessment (SIQA) network which considers a coarse-to-fine feedback guidance and adaptive dominant eye mechanism. The proposed network consists of two main sub-network streams, each of which has three branches to extract low, middle and high spatial frequency information in parallel. To better realize the guidance of the high-level features in the low spatial frequency branch to the low-level features in the high spatial frequency branch, an information feedback guidance module (IFGM) is proposed, which realizes a top-down guidance mechanism in each sub-network stream. Simultaneously, according to the theory of ocular dominance in human visual system (HVS), we design an adaptive bi-directional parallax-based binocular fusion module (BPBFM), which synthesizes two types of fusion feature by taking the left and right view features as dominant eye input. Furthermore, in order to obtain the better perceptual quality of stereo images, we design a weighted fusion strategy to weigh the quality scores from the two types of fusion features obtained by using an ensemble model with two multi-layer perceptrons (MLPs). The experimental results on four public stereo image datasets show that the proposed method is superior to the mainstream metrics and achieves an excellent performance.