Journal of Mathematical Imaging and Vision

, Volume 44, Issue 2, pp 150–167 | Cite as

Hessian-Based Affine Adaptation of Salient Local Image Features

  • Ruan Lakemond
  • Sridha Sridharan
  • Clinton Fookes


Affine covariant local image features are a powerful tool for many applications, including matching and calibrating wide baseline images. Local feature extractors that use a saliency map to locate features require adaptation processes in order to extract affine covariant features. The most effective extractors make use of the second moment matrix (SMM) to iteratively estimate the affine shape of local image regions. This paper shows that the Hessian matrix can be used to estimate local affine shape in a similar fashion to the SMM. The Hessian matrix requires significantly less computation effort than the SMM, allowing more efficient affine adaptation. Experimental results indicate that using the Hessian matrix in conjunction with a feature extractor that selects features in regions with high second order gradients delivers equivalent quality correspondences in less than 17% of the processing time, compared to the same extractor using the SMM.


Local image features Affine adaptation Wide baseline matching Shape estimation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, New York (2003) Google Scholar
  2. 2.
    Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004) CrossRefGoogle Scholar
  3. 3.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Google Scholar
  4. 4.
    Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007) CrossRefGoogle Scholar
  5. 5.
    Carbonetto, P., Dorkó, G., Schmid, C., Kück, H., de Freitas, N.: Learning to recognize objects with little supervision. Int. J. Comput. Vis. 77(1), 219–237 (2008) CrossRefGoogle Scholar
  6. 6.
    Snavely, N., Seitz, S., Szeliski, R.: Modeling the world from Internet photo collections. Int. J. Comput. Vis. 80(2), 189–210 (2008) CrossRefGoogle Scholar
  7. 7.
    Fookes, C., Denman, S., Lakemond, R., Ryan, D., Sridharan, S., Piccardi, M.: Semi-supervised intelligent surveillance system for secure environments. In: IEEE International Symposium on Industrial Electronics (ISIE), pp. 2815–2820 (2010) Google Scholar
  8. 8.
    Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends Comput. Graph. Vis. 3(3), 177–280 (2008) CrossRefGoogle Scholar
  9. 9.
    Kadir, T., Zisserman, A., Brady, J.M.: An affine invariant salient region detector. In: Proc. European Conference on Computer Vision, pp. 228–241 (2004) Google Scholar
  10. 10.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. Int. J. Comput. Vis. 65(1-2), 43–72 (2005) CrossRefGoogle Scholar
  11. 11.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proc. British Machine Vision Conference, vol. 1, pp. 384–393 (2002) Google Scholar
  12. 12.
    Moreels, P., Perona, P.: Evaluation of features detectors and descriptors based on 3d objects. Int. J. Comput. Vis. 73(3), 263–284 (2007) CrossRefGoogle Scholar
  13. 13.
    Bay, H., Tuytelaars, T., Van Gool, L.: Surf: Speeded up robust features. In: Proc. European Conference on Computer Vision, pp. 404–417 (2006) Google Scholar
  14. 14.
    Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Proc. European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 3951, pp. 430–443. Springer, Berlin (2006) Google Scholar
  15. 15.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Proc. Alvey Vision Conference, pp. 189–192 (1988) Google Scholar
  16. 16.
    Beaudet, P.R.: Rotationally invariant image operators. In: Proc. International Joint Conference on Pattern Recognition, Kyoto, Japan, pp. 579–583 (1978) Google Scholar
  17. 17.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004) CrossRefGoogle Scholar
  18. 18.
    Lindeberg, T.: Scale-Space Theory in Computer Vision, 1st edn. Kluwer Academic, Boston (1994) Google Scholar
  19. 19.
    Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 77–116 (1998) Google Scholar
  20. 20.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: Proc. International Conference on Computer Vision, Corfu, Greece, pp. 1150–1157 (1999) CrossRefGoogle Scholar
  21. 21.
    Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. Int. J. Comput. Vis. 60, 63–86 (2004) CrossRefGoogle Scholar
  22. 22.
    Lakemond, R., McKinnon, D.N.R., Fookes, C., Sridharan, S.: A feature clustering algorithm for scale-space analysis of image structures. In: Proc. International Conference on Signal Processing and Communication Systems, pp. 186–192 (2007) Google Scholar
  23. 23.
    Lakemond, R.: Multiple camera management using wide baseline matching. Thesis (2010) Google Scholar
  24. 24.
    Lindeberg, T., Gårding, J.: Shape-adapted smoothing in estimation of 3-d depth cues from affine distortions of local 2-d brightness structure. In: Proc. European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 800, pp. 389–400. Springer, Berlin (1994) Google Scholar
  25. 25.
    Baumberg, A.: Reliable feature matching across widely separated views. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 774–781 (2000) CrossRefGoogle Scholar
  26. 26.
    Tuytelaars, T., Van Gool, L., D’haene, L., Koch, R.: Matching of affinely invariant regions for visual servoing. In: Proc. IEEE International Conference on Robotics and Automation, vol. 2, pp. 1601–1606 (1999) Google Scholar
  27. 27.
    Tuytelaars, T., Van Gool, L.: Wide baseline stereo matching based on local, affinely invariant regions. In: Proc. British Machine Vision Conference, Bristol, pp. 412–425 (2000) Google Scholar
  28. 28.
    van Gool, L., Tuytelaars, T., Turina, A.: Local features for image retrieval. In: Veltkamp, R.C., Burkhardt, H., Kriegel, H.-P. (eds.) State-Of-The-Art in Content-Based Image and Video Retrieval, pp. 21–41. Kluwer Academic, Dordrecht (2001) Google Scholar
  29. 29.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005) CrossRefGoogle Scholar
  30. 30.
    Lakemond, R., Fookes, C., Sridharan, S.: Affine adaptation of local image features using the Hessian matrix. In: Proc. IEEE conference on Advanced Video and Signal Based Surveillance, Genoa, Italy, pp. 496–501 (2009) Google Scholar
  31. 31.
    Yuen, K.: Bayesian Methods for Structural Dynamics and Civil Engineering. Wiley, Singapore (2010) CrossRefGoogle Scholar
  32. 32.
    Deriche, R.: Recursively implementing the Gaussian and its derivatives. Technical report, INRIA (April 1993) Google Scholar
  33. 33.
    Young, I.T., van Vliet, L.J.: Recursive implementation of the Gaussian filter. Signal Process. 44(2), 139–151 (1995) CrossRefGoogle Scholar
  34. 34.
    Lakemond, R., Fookes, C., Sridharan, S.: Dense correspondence extraction in difficult uncalibrated scenarios. In: Proc. Digital Image Computing: Techniques and Applications, Melbourne, Australia, pp. 53–60. IEEE Computer Society Conference Publishing Services, New York (2009) CrossRefGoogle Scholar
  35. 35.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981) MathSciNetCrossRefGoogle Scholar
  36. 36.
    Longuet-Higgins, H.C.: A computer algorithm for reconstructing a scene from two projections. In: Readings in Computer Vision: Issues, Problems, Principles, and Paradigms, p. 61. Morgan Kaufmann, San Francisco (1987) Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Ruan Lakemond
    • 1
  • Sridha Sridharan
    • 1
  • Clinton Fookes
    • 1
  1. 1.Image and Video Research LaboratoryQueensland University of TechnologyBrisbaneAustralia

Personalised recommendations