Recognizing Character-Matching CAPTCHA Using Convolutional Neural Networks with Triple Loss

  • Junfeng Hu
  • Wenchao Ma
  • Aamir Khan
  • Li LiuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11062)


Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA) is a widely used type of challenge-response test to determine whether or not the user is human in many web applications. The traditional CAPTCHAs with English and Chinese characters can be automatically recognized with high accuracy. Yet current methods are limited in recognizing new CAPTCHAs such as character-matching CAPTCHA. We present an approach that combines convolution neural network with triple loss to solve character-matching CAPTCHA. We evaluate our approach on five types of CAPTCHAs including character-matching CAPTCHA. The experimental results show that our approach outperforms other four common recognition methods in the aspects of both accuracy and convergence speed.


CAPTCHA Convolutional neural network Triple loss Recognition 



This work was supported by grants from the Fundamental Research Funds for the Key Research Programm of Chongqing Science & Technology Commission (grant no. cstc2017rgzn-zdyf0064), the Chongqing Provincial Human Resource and Social Security Department (grant no. cx2017092), the Central Universities in China (grant nos. CQU0225001104447).


  1. 1.
    von Ahn, L., Blum, M., Hopper, N.J., Langford, J.: CAPTCHA: using hard AI problems for security. In: Biham, E. (ed.) EUROCRYPT 2003. LNCS, vol. 2656, pp. 294–311. Springer, Heidelberg (2003). Scholar
  2. 2.
    Bursztein, E., Beauxis, R., Paskov, H., Perito, D.: The failure of noise-based non-continuous audio captchas. In: Security and Privacy, pp. 19–31 (2011)Google Scholar
  3. 3.
    Bursztein, E., Martin, M., Mitchell, J.: Text-based CAPTCHA strengths and weaknesses. In: ACM Conference on Computer and Communications Security, pp. 125–138 (2011)Google Scholar
  4. 4.
    Chellapilla, K., Simard, P.Y.: Using machine learning to break visual human interaction proofs (HIPS). In: Advances in Neural Information Processing Systems, pp. 265–272 (2004)Google Scholar
  5. 5.
    Datta, R., Li, J., Wang, J.Z.: IMAGINATION: a robust image-based CAPTCHA generation system. In: ACM International Conference on Multimedia, November, Singapore, pp. 331–334 (2005)Google Scholar
  6. 6.
    Gao, H., Liu, H., Yao, D., Liu, X., Aickelin, U.: An audio CAPTCHA to distinguish humans from computers, pp. 265–269. Social Science Electronic Publishing (2010)Google Scholar
  7. 7.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. J. Mach. Learn. Res. 9, 249–256 (2010)Google Scholar
  8. 8.
    Goodfellow, I.J., Bulatov, Y., Ibarz, J., Arnoud, S., Shet, V.: Multi-digit number recognition from street view imagery using deep convolutional neural networks. Comput. Sci. (2013)Google Scholar
  9. 9.
    IEEE: IEEE xplore abstract - a threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. (1979)Google Scholar
  10. 10.
    Jaderberg, M., Vedaldi, A., Zisserman, A.: Deep features for text spotting. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 512–528. Springer, Cham (2014). Scholar
  11. 11.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. Comput. Sci. (2014)Google Scholar
  12. 12.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  13. 13.
    Lin, D., Lin, F., Lv, Y., Cai, F., Cao, D.: Chinese character CAPTCHA recognition and performance estimation via deep neural network. Neurocomputing 288, 11–19 (2018)CrossRefGoogle Scholar
  14. 14.
    Lu, Y.: Machine printed character segmentation; an overview. Pattern Recognit. 28(1), 67–80 (1995)CrossRefGoogle Scholar
  15. 15.
    Mori, G., Malik, J.: Recognizing objects in adversarial clutter: breaking a visual CAPTCHA. In: Proceedings of CVPR, vol. 1, p. 134 (2003)Google Scholar
  16. 16.
    Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823 (2015)Google Scholar
  17. 17.
    Simard, P.Y., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks (2003)Google Scholar
  18. 18.
    Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Stark, F., Hazirbas, C., Triebel, R., Cremers, D.: CAPTCHA recognition with active deep learning. In: German Conference on Pattern Recognition Workshop (2015)Google Scholar
  20. 20.
    Von, A.L., Maurer, B., Mcmillen, C., Abraham, D., Blum, M.: reCAPTCHA: human-based character recognition via web security measures. Science 321(5895), 1465–1468 (2008)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Wu, Y., Lin, J.J.: An improved adaptive noise estimation in Kalman filtering. J. East China Univ. Sci. Technol. (2004)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Big Data and Software EngineeringChongqing UniversityChongqingChina

Personalised recommendations