A Machine Vision Attack Model on Image Based CAPTCHAs Challenge: Large Scale Evaluation

  • Ajeet SinghEmail author
  • Vikas TiwariEmail author
  • Appala Naidu TentuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11348)


Over the past decade, several public web services made an attempt to prevent automated scripts and exploitation by bots by interrogating a user to solve a Turing-test challenge (commonly known as a CAPTCHA) before using the service. A CAPTCHA is a cryptographic protocol whose underlying hardness assumption is based on an artificial intelligence problem. CAPTCHAs challenges rely on the problem of distinguishing images of living or non-living objects (a task that is easy for humans). User studies proves, it can be solved by humans 99.7% of the time in under 30 s while this task is difficult for machines. The security of image based CAPTCHAs challenge is based on the presumed difficulty of classifying CAPTCHAs database images automatically.

In this paper, we proposed a classification model which is 95.2% accurate in telling apart the images used in the CAPTCHA database. Our method utilizes layered features optimal tuning with an improved VGG16 architecture of Convolutional Neural Networks. Experimental simulation is performed using Caffe deep learning framework. Later, we compared our experimental results with significant state-of-the-art approaches in this domain.


Computing and information systems CAPTCHA Botnets Security Machine learning Advanced neural networks Supervised learning 



We would like to thank P.V. Ananda Mohan, Ashutosh Saxena for their valuable suggestions, insights and observations. We would also like to thank the anonymous reviewers whose comments helped improve this paper.


  1. 1. On the web.
  2. 2.
    Chellapilla, K., Larson, K., Simard, P., Czerwinski, M.: Designing human friendly human interaction proofs (HIPs). In: Proceedings of ACM CHI 2005 Conference on Human Factors in Computing Systems. Email and Security, pp. 711–720 (2005)Google Scholar
  3. 3.
    Chew, M., Tygar, J.D.: Image recognition CAPTCHAs. In: Zhang, K., Zheng, Y. (eds.) ISC 2004. LNCS, vol. 3225, pp. 268–279. Springer, Heidelberg (2004). Scholar
  4. 4.
  5. 5.
    Naor, M.: Verification of a human in the loop or Identification via the Turing Test. Unpublished Manuscript (1997). Electronically:
  6. 6.
    Lillibridge, M.D., Adabi, M., Bharat, K., Broder, A.: Method for selectively restricting access to computer systems. Technical report, US Patent 6,195,698, Applied April 1998 and Approved February 2001Google Scholar
  7. 7.
    Coates, A.L., Baird, H.S., Fateman, R.J.: Pessimal print: a reverse turing test. In: Proceedings of the International Conference on Document Analysis and Recognition (ICDAR 2001), Seattle WA, pp. 1154–1159 (2001)Google Scholar
  8. 8.
    Xu, J., Lipton, R., Essa, I.: Hello, are you human. Technical Report GIT-CC-00-28, Georgia Institute of Technology, November 2000Google Scholar
  9. 9.
    von Ahn, L., Blum, M., Hopper, N.J., Langford, J.: The CAPTCHA (2000).
  10. 10.
    von Ahn, L., Blum, M., Langford, J.: Telling humans and computers apart (Automatically) or how lazy cryptographers do AI. Commun. ACM (2002, to appear)Google Scholar
  11. 11.
    Simard, P., Steinkraus, D., Platt, J.C.: Best practices for convolutional neural networks applied to visual document analysis. In: International Conference on Document Analysis and Recognition, pp. 958–962. IEEE Computer Society (2003)Google Scholar
  12. 12.
    Mori, G., Malik, J.: Recognizing objects in adversarial clutter: breaking a visual CAPTCHA. In: Conference on Computer Vision and Pattern Recognition (CVPR 2003), pp. 134–144. IEEE Computer Society (2003)Google Scholar
  13. 13.
    Kwon, S., Cha, S.: A paradigm shift for the CAPTCHA race: adding uncertainty to the process. IEEE Softw. 33(6), 80–85 (2016)CrossRefGoogle Scholar
  14. 14.
    Elson, J., Douceur, J., Howell, J., Saul, J.: Asirra: a CAPTCHA that exploits interest-aligned manual image categorization. In: Proceedings of ACM CCS 2007, pp. 366–374 (2007)Google Scholar
  15. 15.
    Azakami, T., Shibata, C., Uda, R.: Challenge to impede deep learning against CAPTCHA with ergonomic design. In: IEEE 41st Annual Computer Software and Applications Conference, Italy (2017)Google Scholar
  16. 16.
    Golle, P., Wagner, D.: Cryptanalysis of a cognitive authentication scheme. In: Proceedings of the 2007 IEEE Symposium on Security and Privacy, pp. 66–70. IEEE Computer Society (2007)Google Scholar
  17. 17.
    Mori, G., Malik, J.: Recognizing objects in adversarial clutter: breaking a visual CAPTCHA. In: Proceedings of the 2003 Conference on Computer Vision and Pattern Recognition, pp. 134–144. IEEE Computer Society (2003)Google Scholar
  18. 18.
    SlashDot. Yahoo CAPTCHA Hacked. Accessed 29 Jan 2008
  19. 19.
    Websense Blog: Google’s CAPTCHA busted in recent spammer tactics, 22 February 2008.
  20. 20.
    Yan, J., El Ahmad, A.: A low-cost attack on a Microsoft CAPTCHA. In: Proceedings of ACM CCS (2008, to appear)Google Scholar
  21. 21.
    Chow, R., Golle, P., Jakobsson, M., Wang, X., Wang, L.: Making CAPTCHAs clickable. In: Proceedings of HotMobile (2008)Google Scholar
  22. 22.
    Goodfellow, I.J., Bulatov, Y., Ibarz, J., Arnoud, S., Shet, V.: Multi-digit number recognition from street view imagery using deep convolutional neural networks. In: Proceedings of ICLR, April 2014Google Scholar
  23. 23.
    Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. arXiv:1411.4555 [cs.CV], 20 April 2015
  24. 24.
  25. 25.
    Golle, P.: Machine learning attacks against the Asirra CAPTCHA. In: CCS 2008, Virginia, USA, 27–31 October 2008Google Scholar
  26. 26.
    Althamary, I.A., El-Alfy, E.M.: A more secure scheme for CAPTCHA-based authentication in cloud environment. In: 8th International Conference on Information Technology (ICIT), Jordan, May 2017Google Scholar
  27. 27.
    Tang, M., Gao, H., Zhang, Y.: Research on deep learning techniques in breaking text-based CAPTCHAs and designing image-based CAPTCHA. IEEE Trans. Inf. Forensics Secur. 13(10), 2522–2537 (2018)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.C.R. Rao Advanced Institute of Mathematics, Statistics, and Computer ScienceUniversity of Hyderabad CampusHyderabadIndia

Personalised recommendations