Bayesian Formulation of Image Patch Matching Using Cross-correlation

  • Håkan Ardö
  • Kalle Ågström


A classical solution for matching two image patches is to use the cross-correlation coefficient. This works well if there is a lot of structure within the patches, but not so well if the patches are close to uniform. This means that some patches are matched with more confidence than others. By estimating this uncertainty, more weight can be put on the confident matches than those that are more uncertain. To enable this two distribution functions for two different cases are used: (i) the correlation between two patches showing the same object but with different lighting conditions and different noise realisations and (ii) the correlation between two unrelated patches.

Using these two distributions the patch matching problem is, in this paper, formulated as a binary classification problem. The probability of two patches matching is derived. The model depends on the signal to noise ratio. The noise level is reasonably invariant over time, while the signal level, represented by the amount of structure in the patch or its spatial variance, has to be measured for every frame.

A common application where this is useful is feature point matching between different images. Another application is background/foreground segmentation. This paper will concentrate on the latter application. It is shown how the theory can be used to implement a very fast background/foreground segmentation algorithm by transforming the calculations to the DCT-domain and processing a motion-JPEG stream without uncompressing it. This allows the algorithm to be embedded on a 150 MHz ARM based network camera. It is also suggested to use recursive quantile estimation to estimate the background model. This gives very accurate background models even if there is a lot of foreground present during the initialisation of the model.


Patch-matching Lighting variations Background/Foreground-segmentation Bayesian classification 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Abramowitz, M., Stegun, I.A.: Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York (1972), 8. pr. with corr. edn MATHGoogle Scholar
  2. 2.
    Ardö, H.: Multi-target Tracking Using On-line Viterbi Optimisation and Stochastic Modelling. Centre for Mathematical Sciences LTH, Lund University, Sweden (2009) Google Scholar
  3. 3.
    Ardö, H., Ăström, K.: Bayesian formulation of image patch matching using cross-correlation. In: Third ACM/IEEE International Conference on Distributed Smart Cameras (2009) Google Scholar
  4. 4.
    Brunelli, R., Poggio, T.: Face recognition: features versus templates. IEEE Trans. Pattern Anal. Mach. Intell. 15(10), 1042–1052 (1993) CrossRefGoogle Scholar
  5. 5.
    Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley–Interscience, New York (1973) MATHGoogle Scholar
  6. 6.
    Friedman, N., Russell, S.: Image segmentation in video sequences: a probabilistic approach. In: Thirteenth Conference on Uncertainty in Artificial Intelligence, pp. 175–181 (1997). doi: URL Google Scholar
  7. 7.
    Gordon, G., Darrell, T., Harville, M., Woodfill, J.: Background estimation and removal based on range and color. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1999, vol. 2, p. 464 (1999). doi: 10.1109/CVPR.1999.784721. URL Google Scholar
  8. 8.
    Hotelling, H.: New light on the correlation coefficient and its transforms. J. R. Stat. Soc., Ser. B 15(2), 193–232 (1953) MathSciNetGoogle Scholar
  9. 9.
    Hu, W., Gong, H., Zhu, S.C., Wang, Y.: An integrated background model for video surveillance based on primal sketch and 3d scene geometry. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008). doi: 10.1109/CVPR.2008.4587541. URL CrossRefGoogle Scholar
  10. 10.
    Johnson, N.L.: Continuous Univariate Distributions, vol. 2. Wiley, New York (1995) MATHGoogle Scholar
  11. 11.
    Kaneko, S., Murase, I., Igarashi, S.: Robust image registration by increment sign correlation. Pattern Recognit. 35, 2223–2234 (2002) MATHCrossRefGoogle Scholar
  12. 12.
    Kohli, P., Torr, P.: Efficiently solving dynamic Markov random fields using graph cuts. In: ICCV 2005, vol. 2, pp. 922–929 (2005) Google Scholar
  13. 13.
    Mittal, A., Paragios, N.: Motion-based background subtraction using adaptive kernel density estimation. In: CVPR, vol. 02, pp. 302–309 (2004). URL Google Scholar
  14. 14.
    Möller, E., Grieszbach, G., Schack, B., Witte, H., Maurizio, P.: Statistical properties and control algorithms of recursive quantile estimators. Biom. J. 42(6), 729–746 (2000) MathSciNetMATHCrossRefGoogle Scholar
  15. 15.
    Noriega, P., Bernier, O.: Real time illumination invariant background subtraction using local kernel histograms. In: Proc. British Machine Vision Conference, p. III:979 (2006) Google Scholar
  16. 16.
    Spanne, S.: Konkret Analys. KFS AB, Solvegatan 22, Lund, Sweden (1997) Google Scholar
  17. 17.
    Stauffer, C.: Adaptive background mixture models for real-time tracking. In: Proc. Conf. Computer Vision and Pattern Recognition, pp. 246–252 (1999) Google Scholar
  18. 18.
    Sullivan, J., Blake, M., Isard, M., MacCormick, J.: Bayesian object localisation in images. Int. J. Comput. Vis. 44(2), 111–135 (2001) MATHCrossRefGoogle Scholar
  19. 19.
    Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Int. Conf. on Computer Vision, pp. 255–261 (1999) Google Scholar
  20. 20.
    Magnus, W.F., Bateman, H., Erdaelyi, A.: Higher Transcendental Functions, vol. II. McGraw–Hill, New York (1953) Google Scholar
  21. 21.
    Wayne, P., Johann, P., Schoonees, A.: Understanding background mixture models for foreground segmentation. In: Proceedings Image and Vision Computing (2002). URL Google Scholar
  22. 22.
    Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.P.: Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 780–785 (1997) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Centre for Mathematical SciencesLund UniversityLundSweden

Personalised recommendations