The Markov random field (MRF) for stereo matching can be solved using belief propagation (BP). However, the solution space grows significantly with the introduction of high-resolution stereo images and 3D plane labels, making the traditional BP algorithms impractical in inference time and convergence. We present an accurate and efficient hierarchical BP framework using the representation of the image segmentation pyramid (ISP). The pixel-level MRF can be solved by a top-down inference on the ISP. We design a hierarchy of MRF networks using the graph of superpixels at each ISP level. From the highest/image to the lowest/pixel level, the MRF models can be efficiently inferred with constant global guidance using the optimal labels of the previous level. The large texture-less regions can be handled effectively by the MRF model on a high level. The advanced 3D continuous labels and a novel support-points regularization are integrated into our framework for stereo matching. We provide a data-level parallelism implementation which is orders of magnitude faster than the best graph cuts (GC) algorithm. The proposed framework, HBP-ISP, outperforms the best GC algorithm on the Middlebury stereo matching benchmark.

This paper proposes a hierarchical superpixel segmentation by representing an image as a hierarchy of 1-nearest neighbor (1-NN) graphs with pixels/superpixels denoting the graph vertices. The 1-NN graphs are built from the pixel/superpixel adjacent matrices to ensure connectivity. To determine the next-level superpixel hierarchy, inspired by FINCH clustering, the weakly connected components (WCCs) of the 1-NN graph are labeled as superpixels. We reveal that the WCCs of a 1-NN graph consist of a forest of cycle-root-trees (CRTrees). The forest-like structure inspires us to propose a two-stage parallel CRTrees labeling which first links the child vertices to the cycle-roots and then labels all the vertices by the cycle-roots. We also propose an inter-inner superpixel distance penalization and a Lab color lightness penalization base on the property that the distance of a CRTree decreases monotonically from the child to root vertices. Experiments show the parallel CRTrees labeling is several times faster than recent advanced sequential and parallel connected components labeling algorithms. The proposed hierarchical superpixel segmentation has comparable performance to the best performer ETPS (state-of-the-arts) on the BSDS500, NYUV2, and Fash datasets. At the same time, it can achieve 200FPS for 480P video streams.

In this paper, we propose a disparity refinement method that directly refines the winner-take-all (WTA) disparity map by exploring its statistical significance. According to the primary steps of the segment-based stereo matching, the reference image is over-segmented into superpixels and a disparity plane is fitted for each superpixel by an improved random sample consensus (RANSAC). We design a two-layer optimization to refine the disparity plane. In the global optimization, mean disparities of superpixels are estimated by Markov random field (MRF) inference, and then, a 3D neighborhood system is derived from the mean disparities for occlusion handling. In the local optimization, a probability model exploiting Bayesian inference and Bayesian prediction is adopted and achieves second-order smoothness implicitly among 3D neighbors. The two-layer optimization is a pure disparity refinement method because no correlation information between stereo image pairs is demanded during the refinement. Experimental results on the Middlebury and KITTI datasets demonstrate that the proposed method can perform accurate stereo matching with a faster speed and handle the occlusion effectively. It can be indicated that the "matching cost computation + disparity refinement" framework is a possible solution to produce accurate disparity map at low computational cost.