Efficient Inference for Continuous MRFs
Markov random fields are widely used to model many computer vision problems that can be cast in an energy minimization framework composed of unary and pairwise potentials. While computationally tractable discrete optimizers such as Graph Cuts and belief propagation (BP) exist for multi-label discrete problems, they still face prohibitively high computational challenges when the labels reside in a huge or very densely sampled space. Integrating key ideas from PatchMatch of effective particle propagation and resampling, PatchMatch belief propagation (PMBP) has been demonstrated to have good performance in addressing continuous labeling problems and runs orders of magnitude faster than Particle BP (PBP). However, the quality of the PMBP solution is tightly coupled with the local window size, over which the raw data cost is aggregated to mitigate ambiguity in the data constraint. This dependency heavily influences the overall complexity, increasing linearly with the window size.
We propose a novel algorithm called sped-up PMBP (SPM-BP) to tackle this critical computational bottleneck and speeds up PMBP by 50-100 times. The crux of SPM-BP is on unifying efficient filter-based cost aggregation and message passing with PatchMatch-based particle generation in a highly effective way. Though simple in its formulation, SPM-BP achieves superior performance for sub-pixel accurate stereo and optical-flow on benchmark datasets when compared with more complex and task-specific approaches.
- Y. Li, D. Min, M. S. Brown, M. N. Do, and J. Lu, “SPM-BP: Sped-up PatchMatch Belief Propagation for Continuous MRFs,” in Proc. IEEE Int. Conf. Computer Vision (ICCV), Santiago, Chile, Dec. 2015. (Oral) [pdf][slides (PDF)][slides (PPT)][code]
- Our related work:
- J. Lu, H. Yang, D. Min, and M. N. Do, “PatchMatch Filter: Efficient Edge-Aware Filtering Meets Randomized Search for Fast Correspondence Field Estimation,” IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), Jun. 2013. (Oral) [pdf][ppt]
- H. Yang, W.-Y. Lin, and J. Lu, “Daisy Filter Flow: A Generalized Discrete Approach to Dense Correspondences,” in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, Jun. 2014. [pdf][web]
- J. Lu, Y. Li, H. Yang, D. Min, W. Eng, and M. N. Do, “PatchMatch Filter: Edge-Aware Filtering Meets Randomized Search for Correspondence Field Estimation,” IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2017. [preprint]