Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Glasses detection on real images based on robust alignment

Abstract

Automatic glasses detection on real face images is a challenging problem due to different appearance variations. Nevertheless, glasses detection on face images has not been thoroughly investigated. In this paper, an innovative algorithm for automatic glasses detection based on Robust Local Binary Pattern and robust alignment is proposed. Firstly, images are preprocessed and normalized in order to deal with scale and rotation. Secondly, eye glasses region is detected considering that the nosepiece of the glasses is usually placed at the same level as the center of the eyes in both height and width. Thirdly, Robust Local Binary Pattern is built to describe the eyes region, and finally, support vector machine is used to classify the LBP features. This algorithm can be applied as the first step of a glasses removal algorithm due to its robustness and speed. The proposed algorithm has been tested over the Labeled Faces in the Wild database showing a 98.65 % recognition rate. Influences of the resolution, the alignment of the normalized images and the number of divisions in the LBP operator are also investigated.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  1. 1.

    Jiang, X., Binkert, M., Bernard, A., Bunke, H.: Towards detection of glasses in facial images. Pattern Anal. Appl. 3(1), 9–18 (2000)

  2. 2.

    Dantcheva, A., Velardo, C., Dangelo, A., Dugelay, J.-L.: Bag of soft biometrics for person identification. Multimed. Tool. Appl. 51(2), 739–777 (2011)

  3. 3.

    Jing, Z., Mariani, R.: Glasses detection and extraction by deformable contour. In: Proceedings 15th International Conference on Pattern Recognition, 2000, vol. 2, pp. 933–936. IEEE (2000)

  4. 4.

    Vaquero, D.A., Feris, R.S., Tran, D., Brown, L., Hampapur, A., Turk, M.: Attribute-based people search in surveillance environments. In: Workshop on Applications of Computer Vision (WACV), 2009, pp. 1–8. IEEE (2009)

  5. 5.

    Wu, B., Ai, H., Liu, R.: Glasses detection by boosting simple wavelet features. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004, vol. 1, pp. 292–295. IEEE (2004)

  6. 6.

    Wu, H., Yoshikawa, G., Shioyama, T., Lao, T., Kawade, T.: Glasses frame detection with 3d hough transform. In: Proceedings 16th International Conference on Pattern Recognition, 2002, vol. 2, pp. 346–349. IEEE (20020

  7. 7.

    Shan, S., Gao, W., Lu, Y., Cao, B., Chen, X., Zhao, D., Zeng, W.: Unified framework for classifying facial images based on facial attribute-specific subspaces and minimum reconstruction error. In: Proc. of ACCV, vol. 2. Citeseer (2002)

  8. 8.

    Chenyu, W., Liu, C., Shum, H.-Y., Xy, Y.-Q., Zhang, Z.: Automatic eyeglasses removal from face images. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 322–336 (2004)

  9. 9.

    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, CVPR 2001, vol. 1, pp. I-511. IEEE (2001)

  10. 10.

    Heo, J., Kong, S.G., Abidi, B.R., Abidi, M.A.: Fusion of visual and thermal signatures with eyeglass removal for robust face recognition. In: Conference on Computer Vision and Pattern Recognition Workshop, 2004, CVPRW’04, pp. 122–122. IEEE (2004)

  11. 11.

    Jing, Z., Mariani, R., Wang, J.: Glasses detection for face recognition using bayes rules. In: Advances in Multimodal InterfacesICMI 2000, pp. 127–134. Springer, Berlin (2000)

  12. 12.

    Schapire, R.E., Singer, Y.: Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37(3), 297–336 (1999)

  13. 13.

    Xiao, Y., Yan, H.: Extraction of glasses in human face images. In: Biometric Authentication, pp. 214–220. Springer, Berlin (2004)

  14. 14.

    Park, J.-S., Oh, Y.H., Ahn, S.C., Lee, S.-W.: Glasses removal from facial image using recursive error compensation. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 805–811 (2005)

  15. 15.

    Uricár, M., Franc, V., Hlavác, V.: Detector of facial landmarks learned by the structured output svm. In: VISAPP12: Proceedings of the 7th International Conference on Computer Vision Theory and Applications, vol. 1, pp. 547–556 (2012)

  16. 16.

    Wolf, L., Hassner, T., Taigman, Y.: Similarity scores based on background samples. In: Computer Vision-ACCV 2009, pp. 88–97. Springer, Berlin (2010)

  17. 17.

    Huang, G.B., Jain, V., Learned-Miller, E.: Unsupervised joint alignment of complex images. In: IEEE 11th International Conference on Computer Vision, 2007, ICCV 2007, pp. 1–8. IEEE (2007)

  18. 18.

    Huang, G.B., Mattar, M., Lee, H., Learned-Miller, E.G.: Learning to align from scratch. In: Neural Information Processing Systems. Neural Information Processing Systems (NIPS), Lake Tahoe (2012)

  19. 19.

    Makinen, E., Raisamo, R.: Evaluation of gender classification methods with automatically detected and aligned faces. IEEE Trans. Pattern Anal. Mach. Intell. 30(3), 541–547 (2008)

  20. 20.

    Li, G., Cai, X., Li, X., Liu, Y.: An efficient face normalization algorithm based on eyes detection. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, 2006, pp. 3843–3848. IEEE (2006)

  21. 21.

    Mäkinen, E., Raisamo, R.: An experimental comparison of gender classification methods. Pattern Recogn. Lett. 29(10), 1544–1556 (2008)

  22. 22.

    Wu, H., Yokoyama, T., Pramadihanto, D., Yachida, M.: Face and facial feature extraction from color image. In: Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, 1996, pp. 345–350. IEEE (1996)

  23. 23.

    Wu, H., Inada, J., Shioyama, T., Chen, Q., Simada, T.: Automatic facial feature points detection with susan operator. In: Proceedings of the Scandinavian Conference on Image Analysis, pp. 257–263 (2001)

  24. 24.

    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck\(+\)): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2010, pp. 94–101. IEEE (2010)

  25. 25.

    Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011)

  26. 26.

    Song, F., Tan, X., Chen, S., Zhou, Z.-H.: A literature survey on robust and efficient eye localization in real-life scenarios. Pattern Recogn. 46(12), 3157–3173 (2013)

  27. 27.

    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. In: Technical Report 07–49. University of Massachusetts, Amherst (2007)

  28. 28.

    Ojala, T., Pietikäinen, M., Harwood, D.: A comparative study of texture measures with classification based on featured distributions. Pattern Recogn. 29(1), 51–59 (1996)

  29. 29.

    Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

  30. 30.

    Guo, Z., Zhang, L., Zhang, D.: A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Proc. 19(6), 1657–1663 (2010)

  31. 31.

    Zhao, Y., Jia, W., Hu, R.-X., Min, H.: Completed robust local binary pattern for texture classification. Neurocomputing 106, 68–76 (2013)

  32. 32.

    Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)

  33. 33.

    Shan, C.: Learning local binary patterns for gender classification on real-world face images. Pattern Recogn. Lett. 33(4), 431–437 (2012)

  34. 34.

    Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: IEEE 12th International Conference on Computer Vision, 2009, pp. 365–372. IEEE (2009)

  35. 35.

    Pietikäinen, M., Hadid, A., Zhao, G., Ahonen, T.: Local binary patterns for still images. In: Computer Vision Using Local Binary Patterns, pp. 13–47. Springer, Berlin (2011)

  36. 36.

    Huang, D., Shan, C., Ardabilian, M., Wang, Y., Chen, L.: Facial image analysis based on local binary patterns: a survey. IEEE Trans. Sys. Man Cyber.-Part C 41(6), 765–781 (2011)

  37. 37.

    Ahonen, T., Hadid, A., Pietikäinen, M.: Face recognition with local binary patterns. In: Computer Vision-eccv 2004, pp. 469–481. Springer, Berlin (2004)

  38. 38.

    Zou, W.W., Yuen, P.C., Chellappa, R.: Low-resolution face tracker robust to illumination variations. IEEE Trans. Image Proc. 22(5), 1726–1739 (2013)

  39. 39.

    Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The feret evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)

  40. 40.

    Huang, D., Shan, C., Ardabilian, M., Wang, Y., Chen, L.: Local binary patterns and its application to facial image analysis: a survey. IEEE Trans. Syst. Man Cyber. Part C Appl. Rev. 41(6), 765–781 (2011)

  41. 41.

    Song, F., Tan, X., Liu, X., Chen, S.: Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients. Pattern Recogn. 47(9), 2825–2838 (2014)

  42. 42.

    Yuan, J.-H., Huang, D.-S., Zhu, H.-D., Gan, Y.: Completed hybrid local binary pattern for texture classification. In: International Joint Conference on Neural Networks (IJCNN), 2014, pp. 2050–2057. IEEE (2014)

  43. 43.

    Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, pp. 1521–1528. IEEE (2011)

Download references

Acknowledgments

Authors are grateful to anonymous reviewers for constructive feedback and insightful suggestions that greatly improved this article.

Author information

Correspondence to Alberto Fernández.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fernández, A., García, R., Usamentiaga, R. et al. Glasses detection on real images based on robust alignment. Machine Vision and Applications 26, 519–531 (2015). https://doi.org/10.1007/s00138-015-0674-1

Download citation

Keywords

  • Glasses detection
  • Face alignment
  • Robust Local Binary Pattern
  • Face image processing