Comparative Evaluation of Binary Features

  • Jared Heinly
  • Enrique Dunn
  • Jan-Michael Frahm
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7573)


Performance evaluation of salient features has a long-standing tradition in computer vision. In this paper, we fill the gap of evaluation for the recent wave of binary feature descriptors, which aim to provide robustness while achieving high computational efficiency. We use established metrics to embed our assessment into the body of existing evaluations, allowing us to provide a novel taxonomy unifying both traditional and novel binary features. Moreover, we analyze the performance of different detector and descriptor pairings, which are often used in practice but have been infrequently analyzed. Additionally, we complement existing datasets with novel data testing for illumination change, pure camera rotation, pure scale change, and the variety present in photo-collections. Our performance analysis clearly demonstrates the power of the new class of features. To benefit the community, we also provide a website for the automatic testing of new description methods using our provided metrics and datasets ( ).


binary features comparison evaluation 


  1. 1.
    Chang, H.J., et al.: P-SLAM: Simultaneous Localization and Mapping With Environmental-Structure Prediction. IEEE Trans. Robot. 23(2), 281–293 (2007)CrossRefGoogle Scholar
  2. 2.
    Frahm, J.-M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y.-H., Dunn, E., Clipp, B., Lazebnik, S., Pollefeys, M.: Building Rome on a Cloudless Day. In: Daniilidis, K., Maragos, P., Paragios, N., et al. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 368–381. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo Tourism: Exploring Photo Collections in 3D. In: SIGGRAPH Conference Proceedings, pp. 835–846 (2006)Google Scholar
  4. 4.
    Nister, D., Stewenius, H.: Scalable Recognition with a Vocabulary Tree. In: CVPR, pp. 2161–2168 (2006)Google Scholar
  5. 5.
    Brown, M., Lowe, D.G.: Recognising Panoramas. In: ICCV, pp. 1218–1225 (2003)Google Scholar
  6. 6.
    Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary Robust Independent Elementary Features. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 778–792. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: An Efficient Alternative to SIFT or SURF. In: ICCV, pp. 2564–2571 (2011)Google Scholar
  8. 8.
    Leutenegger, S., Chli, M., Siegwart, R.: BRISK: Binary Robust Invariant Scalable Keypoints. In: ICCV, pp. 2548–2555 (2011)Google Scholar
  9. 9.
    Mikolajczyk, K., et al.: A Comparison of Affine Region Detectors. IJCV 65(1-2), 43–72 (2005)CrossRefGoogle Scholar
  10. 10.
    Mikolajczyk, K., Schmid, C.: A Performance Evaluation of Local Descriptors. IEEE Trans. PAMI 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  11. 11.
    Moreels, P., Perona, P.: Evaluation of Features Detectors and Descriptors based on 3D Objects. IJCV 73(3), 263–284 (2007)CrossRefGoogle Scholar
  12. 12.
    Strecha, C., et al.: On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery. In: CVPR, pp. 1–8 (2008)Google Scholar
  13. 13.
    Aanæs, H., et al.: Interesting Interest Points. IJCV 97(1), 18–35 (2012)CrossRefGoogle Scholar
  14. 14.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)CrossRefGoogle Scholar
  15. 15.
    Wu, C.: SiftGPU (2007),
  16. 16.
    Schulz, A., et al.: CUDA SURF - A real-time implementation for SURF (2010),
  17. 17.
    Hirschmüller, H., Scharstein, D.: Evaluation of Cost Functions for Stereo Matching. In: CVPR, pp. 1–8 (2007)Google Scholar
  18. 18.
    Agarwal, S., et al.: Building Rome in a Day. In: ICCV, pp. 72–79 (2009)Google Scholar
  19. 19.
    Bay, H., et al.: Speeded-Up Robust Features (SURF). Comp. Vis. and Image Understanding 110(3), 346–359 (2008)CrossRefGoogle Scholar
  20. 20.
    Tola, E., Lepetit, V., Fua, P.: DAISY: An Efficient Dense Descriptor Applied to Wide Baseline Stereo. IEEE Trans. PAMI 32(5), 815–830 (2010)CrossRefGoogle Scholar
  21. 21.
    Ke, Y., Sukthankar, R.: PCA-SIFT: a more distinctive representation for local image descriptors. In: CVPR, pp. 506–513 (2004)Google Scholar
  22. 22.
    Strecha, C., et al.: LDAHash: Improved Matching with Smaller Descriptors. IEEE Trans. PAMI 34(1), 66–78 (2012)CrossRefGoogle Scholar
  23. 23.
    Yeo, C., Ahammad, P., Ramchandran, K.: Coding of Image Feature Descriptors for Distributed Rate-efficient Visual Correspondences. IJCV 94(3), 267–281 (2011)zbMATHCrossRefGoogle Scholar
  24. 24.
    Raginsky, M., Lazebnik, S.: Locality-Sensitive Binary Codes from Shift-Invariant Kernels. In: Advances in Neural Info. Processing Systems, pp. 1509–1517 (2009)Google Scholar
  25. 25.
    Gong, Y., Lazebnik, S.: Iterative Quantization: A Procrustean Approach to Learning Binary Codes. In: CVPR, pp. 817–824 (2011)Google Scholar
  26. 26.
    Agrawal, M., Konolige, K., Blas, M.R.: CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 102–115. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  27. 27.
    Rosin, P.L.: Measuring Corner Properties. Comp. Vis. and Image Understanding, 291–307 (1999)Google Scholar
  28. 28.
    Gauglitz, S., et al.: Improving Keypoint Orientation Assignment. In: BMVC (2011)Google Scholar
  29. 29.
    Rosten, E., Porter, R., Drummond, T.: Faster and Better: A Machine Learning Approach to Corner Detection. IEEE Trans. PAMI 32(1), 105–119 (2010)CrossRefGoogle Scholar
  30. 30.
    Harris, C., Stephens, M.: A Combined Corner and Edge Detector. In: Proc. of The Fourth Alvey Vision Conference, pp. 147–151 (1988)Google Scholar
  31. 31.
    Mair, E., Hager, G.D., Burschka, D., Suppa, M., Hirzinger, G.: Adaptive and Generic Corner Detection Based on the Accelerated Segment Test. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 183–196. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  32. 32.
    Dahl, A.L., Aanæs, H., Pedersen, K.S.: Finding the Best Feature Detector-Descriptor Combination. In: 3DIMPVT, pp. 318–325 (2011)Google Scholar
  33. 33.
    Fischler, M.A., Bolles, R.C.: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Communications of the ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  34. 34.
    Zitnick, L., Krishnan, R.: Edge Foci Interest Points. In: ICCV, pp. 359–366 (2011)Google Scholar
  35. 35.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)Google Scholar
  36. 36.
    Kurz, D., BenHimane, S.: Inertial sensor-aligned visual feature descriptors. In: CVPR, pp. 161–166 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jared Heinly
    • 1
  • Enrique Dunn
    • 1
  • Jan-Michael Frahm
    • 1
  1. 1.The University of North Carolina at Chapel HillUSA

Personalised recommendations