Advertisement

Corner Detection Using Multi-directional Structure Tensor with Multiple Scales

  • Weichuan ZhangEmail author
  • Changming Sun
Article

Abstract

Corners are important features for image analysis and computer vision tasks. Local structure tensors with multiple scales are widely used in intensity-based corner detectors. In this paper, the properties of intensity variations of a step edge, L-type corner, Y- or T-type corner, X-type corner, and star-type corner are investigated. The properties that we obtained indicate that the image intensity variations of a corner are not always large in all directions. The properties also demonstrate that existing structure tensor-based corner detection methods cannot depict the differences of intensity variations well between edges and corners which result in wrong corner detections. We present a new technique to extract the intensity variations from input images using anisotropic Gaussian directional derivative filters with multiple scales. We prove that the new extraction technique on image intensity variation has the ability to accurately depict the characteristics of edges and corners in the continuous domain. Furthermore, the properties of the intensity variations of step edges and corners enable us to derive a new multi-directional structure tensor with multiple scales, which has the ability to depict the intensity variation differences well between edges and corners in the discrete domain. The eigenvalues of the multi-directional structure tensor with multiple scales are used to develop a new corner detection method. Finally, the criteria on average repeatability (under affine image transformation, JPEG compression, and noise degradation), region repeatability based on the Oxford dataset, repeatability metric based on the DTU dataset, detection accuracy, and localization accuracy are used to evaluate the proposed detector against ten state-of-the-art methods. The experimental results show that our proposed detector outperforms all the other tested detectors.

Keywords

Corner detection Image intensity variation extraction Anisotropic Gaussian directional derivative filters Multi-directional structure tensor with multiple scales 

Notes

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 61401347). We thank the anonymous reviewers for their detailed comments that substantially improved the paper.

References

  1. Aanæs, H., Dahl, A. L., & Pedersen, K. S. (2012). Interesting interest points. International Journal of Computer Vision, 97(1), 18–35.CrossRefGoogle Scholar
  2. Alcantarilla, P., Bartoli, A., Davison, A. (2012). KAZE features. In European conference on computer vision (pp. 214–227). Springer.Google Scholar
  3. Awrangjeb, M., & Lu, G. (2008). Robust image corner detection based on the chord-to-point distance accumulation technique. IEEE Transactions on Multimedia, 10(6), 1059–1072.CrossRefGoogle Scholar
  4. Bay, H., Tuytelaars, T., & Van Gool, L. (2006). SURF: Speeded up robust features. In European conference on computer vision (pp. 404–417). Springer.Google Scholar
  5. Bowyer, K., Kranenburg, C., & Dougherty, S. (1999). Edge detector evaluation using empirical ROC curves. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 354–359).Google Scholar
  6. Brox, T., Weickert, J., Burgeth, B., & Mrázek, P. (2006). Nonlinear structure tensors. Image and Vision Computing, 24(1), 41–55.CrossRefGoogle Scholar
  7. Cornelis, N., & Van Gool, L. (2008). Fast scale invariant feature detection and matching on programmable graphics hardware. In Computer vision and pattern recognition workshops (pp. 1–8).Google Scholar
  8. Deriche, R., & Giraudon, G. (1993). A computational approach for corner and vertex detection. International Journal of Computer Vision, 10(2), 101–124.CrossRefGoogle Scholar
  9. DeTone, D., Malisiewicz, T., Rabinovich, A. (2018). Superpoint: Self-supervised interest point detection and description. In IEEE conference on computer vision and pattern recognition (pp. 224–236).Google Scholar
  10. Duval-Poo, M. A., Odone, F., & De Vito, E. (2015). Edges and corners with shearlets. IEEE Transactions on Image Processing, 24(11), 3768–3780.MathSciNetzbMATHCrossRefGoogle Scholar
  11. Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America A-Optics Image Science and Vision, 4(12), 2379–2394.CrossRefGoogle Scholar
  12. Gao, X., Sattar, F., & Venkateswarlu, R. (2007). Multiscale corner detection of gray level images based on LoG-Gabor wavelet transform. IEEE Transactions on Circuits and Systems for Video Technology, 17(7), 868–875.CrossRefGoogle Scholar
  13. Gårding, J., & Lindeberg, T. (1996). Direct computation of shape cues using scale-adapted spatial derivative operators. International Journal of Computer Vision, 17(2), 163–191.CrossRefGoogle Scholar
  14. Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Alvey vision conference (pp. 147–151).Google Scholar
  15. Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.zbMATHCrossRefGoogle Scholar
  16. Huang, F. C., Huang, S. Y., Ker, J. W., & Chen, Y. C. (2012). High-performance SIFT hardware accelerator for real-time image feature extraction. IEEE Transactions on Circuits and Systems for Video Technology, 22(3), 340–351.CrossRefGoogle Scholar
  17. Kenney, C. S., Manjunath, B., Zuliani, M., Hewer, G. A., & Van Nevel, A. (2003). A condition number for point matching with application to registration and postregistration error estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(11), 1437–1454.CrossRefGoogle Scholar
  18. Koenderink, J. J. (1984). The structure of images. Biological Cybernetics, 50(5), 363–370.MathSciNetzbMATHCrossRefGoogle Scholar
  19. Laptev, I. (2005). On space–time interest points. International Journal of Computer Vision, 64(2–3), 107–123.CrossRefGoogle Scholar
  20. Lee, J. S., Sun, Y. N., & Chen, C. H. (1995). Multiscale corner detection by using wavelet transform. IEEE Transactions on Image Processing, 4(1), 100–104.CrossRefGoogle Scholar
  21. Lenc, K., & Vedaldi, A. (2016). Learning covariant feature detectors. In European conference on computer vision (pp. 100–117). Springer.Google Scholar
  22. Lepetit, V., & Fua, P. (2006). Keypoint recognition using randomized trees. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9), 1465–1479.CrossRefGoogle Scholar
  23. Lindeberg, T. (1998). Feature detection with automatic scale selection. International Journal of Computer Vision, 30(2), 79–116.CrossRefGoogle Scholar
  24. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.CrossRefGoogle Scholar
  25. Marimon, D., Bonnin, A., Adamek, T., & Gimeno, R. (2010). DARTs: Efficient scale-space extraction of DAISY keypoints. In IEEE conference on computer vision and pattern recognition (pp. 2416–2423).Google Scholar
  26. Maver, J. (2010). Self-similarity and points of interest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(7), 1211–1226.CrossRefGoogle Scholar
  27. Miao, Z., & Jiang, X. (2013). Interest point detection using rank order LoG filter. Pattern Recognition, 46(11), 2890–2901.CrossRefGoogle Scholar
  28. Mikolajczyk, K., & Schmid, C. (2004). Scale & affine invariant interest point detectors. International Journal of Computer Vision, 60(1), 63–86.CrossRefGoogle Scholar
  29. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., et al. (2005). A comparison of affine region detectors. International Journal of Computer Vision, 65(1–2), 43–72.CrossRefGoogle Scholar
  30. Mokhtarian, F., & Suomela, R. (1998). Robust image corner detection through curvature scale space. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12), 1376–1381.CrossRefGoogle Scholar
  31. Moravec, H. P. (1979). Visual mapping by a robot rover. In Proceedings of the 6th international joint conference on artificial intelligence (Vol. 1, pp. 598–600).Google Scholar
  32. Noble, J. A. (1988). Finding corners. Image and Vision Computing, 6(2), 121–128.CrossRefGoogle Scholar
  33. Olson, C. F. (2000). Adaptive-scale filtering and feature detection using range data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9), 983–991.CrossRefGoogle Scholar
  34. Perona, P., & Malik, J. (1990). Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7), 629–639.CrossRefGoogle Scholar
  35. Pham, T. A., Delalandre, M., Barrat, S., & Ramel, J. Y. (2014). Accurate junction detection and characterization in line-drawing images. Pattern Recognition, 47(1), 282–295.CrossRefGoogle Scholar
  36. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1(1), 81–106.Google Scholar
  37. Rattarangsi, A., & Chin, R. T. (1990). Scale-based detection of corners of planar curves. In 10th international conference on pattern recognition (Vol. 1, pp. 923–930).Google Scholar
  38. Rosten, E., Porter, R., & Drummond, T. (2010). Faster and better: A machine learning approach to corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 105–119.CrossRefGoogle Scholar
  39. Ruzon, M. A., & Tomasi, C. (2001). Edge, junction, and corner detection using color distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1281–1295.CrossRefGoogle Scholar
  40. Schmid, C., Mohr, R., & Bauckhage, C. (2000). Evaluation of interest point detectors. International Journal of Computer Vision, 37(2), 151–172.zbMATHCrossRefGoogle Scholar
  41. Shui, P., & Zhang, W. (2012). Noise-robust edge detector combining isotropic and anisotropic Gaussian kernels. Pattern Recognition, 45(2), 806–820.zbMATHCrossRefGoogle Scholar
  42. Shui, P., & Zhang, W. (2013). Corner detection and classification using anisotropic directional derivative representations. IEEE Transactions on Image Processing, 22(8), 3204–3218.CrossRefGoogle Scholar
  43. Smith, S. M., & Brady, J. M. (1997). SUSAN—A new approach to low level image processing. International Journal of Computer Vision, 23(1), 45–78.CrossRefGoogle Scholar
  44. Snavely, N., Seitz, S. M., & Szeliski, R. (2006). Photo tourism: Exploring photo collections in 3D. ACM Transactions on Graphics, 26(3), 835–846.CrossRefGoogle Scholar
  45. Su, R., Sun, C., & Pham, T. D. (2012). Junction detection for linear structures based on Hessian, correlation and shape information. Pattern Recognition, 45(10), 3695–3706.CrossRefGoogle Scholar
  46. Teh, C. H., & Chin, R. T. (1989). On the detection of dominant points on digital curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(8), 859–872.CrossRefGoogle Scholar
  47. Trujillo, L., Olague, G. (2006). Synthesis of interest point detectors through genetic programming. In Proceedings of the 8th annual conference on genetic and evolutionary computation (pp. 887–894).Google Scholar
  48. Verdie, Y., Yi, V., Fua, P., & Lepetit, V. (2015). TILDE: A temporally invariant learned detector. In IEEE conference on computer vision and pattern recognition (pp. 5279–5288).Google Scholar
  49. Wang, Y.-P. (1999). Image representations using multiscale differential operators. IEEE Transactions on Image Processing, 8(12), 1757–1771.MathSciNetzbMATHCrossRefGoogle Scholar
  50. Weickert, J., Romeny, B. T. H., & Viergever, M. A. (1998). Efficient and reliable schemes for nonlinear diffusion filtering. IEEE Transactions on Image Processing, 7(3), 398–410.CrossRefGoogle Scholar
  51. Widynski, N., & Mignotte, M. (2014). A multiscale particle filter framework for contour detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(10), 1922–1935.CrossRefGoogle Scholar
  52. Wilson, K., & Snavely, N. (2014). Robust global translations with 1DSfM. In European conference on computer vision (pp. 61–75). Springer.Google Scholar
  53. Witkin, A. (1984). Scale-space filtering: A new approach to multi-scale description. In IEEE international conference on acoustics, speech, and signal processing (Vol. 9, pp. 150–153).Google Scholar
  54. Xia, G., Delon, J., & Gousseau, Y. (2014). Accurate junction detection and characterization in natural images. International Journal of Computer Vision, 106(1), 31–56.MathSciNetzbMATHCrossRefGoogle Scholar
  55. Yi, K. M., Trulls, E., Lepetit, V., & Fua, P. (2016). LIFT: Learned invariant feature transform. In European conference on computer vision (pp. 467–483).Google Scholar
  56. Zhang, X., Qu, Y., Yang, D., Wang, H., & Kymer, J. (2015). Laplacian scale-space behavior of planar curve corners. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(11), 2207–2217.CrossRefGoogle Scholar
  57. Zhang, W., & Shui, P. (2015). Contour-based corner detection via angle difference of principal directions of anisotropic Gaussian directional derivatives. Pattern Recognition, 48(9), 2785–2797.CrossRefGoogle Scholar
  58. Zhang, W., Sun, C., Breckon, T., & Alshammari, N. (2019). Discrete curvature representations for noise robust image corner detection. IEEE Transactions on Image Processing, 28(9), 4444–4459.MathSciNetzbMATHCrossRefGoogle Scholar
  59. Zhang, W., Wang, F., Zhu, L., & Zhou, Z. (2014). Corner detection using Gabor filters. IET Image Processing, 8(11), 639–646.CrossRefGoogle Scholar
  60. Zhang, W.-C., Zhao, Y., Breckon, T. P., & Chen, L. (2017). Noise robust image edge detection based upon the automatic anisotropic Gaussian kernels. Pattern Recognition, 63(2), 193–205.CrossRefGoogle Scholar
  61. Zhang, X., Yu, F. X., Karaman, S., Chang, S.-F. (2017). Learning discriminative and transformation covariant local feature detectors. In IEEE conference on computer vision and pattern recognition (pp. 6818–6826).Google Scholar
  62. Zhong, B., & Liao, W. (2007). Direct curvature scale space: Theory and corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(3), 100–108.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.College of Electrical and InformationXi’an Polytechnic UniversityXi’anChina
  2. 2.CSIRO Data61EppingAustralia

Personalised recommendations