Object Recognition in Hand Drawn Images Using Machine Ensembling Techniques and Smote Sampling

  • Mohit GuptaEmail author
  • Pulkit Mehndiratta
  • Akanksha Bhardwaj
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1025)


With the increase in the advancement of technology, the size of multimedia content generated is increasing every day. Handling and management of this data to extract the patterns result in a more optimized and efficient way is the need of the hour. In this proposed work, we have presented techniques to classify the stroke-based hand-drawn object. We have used the Quick Draw dataset which is a repository of approximately 50 million hand-drawn drawings of 345 different objects. Our research presents an approach to the classification of these drawings created using hand strokes. We are converting the given raw image data, to much more simplified and concise data and then performed oversampling on data belonging to classes with fewer instances using Synthetic Minority Over-sampling Technique (SMOTE) method to balance the distribution of each class in the dataset. Finally, we are classifying the drawings using K-Nearest Neighbor (K-NN), Random Forest Classifier (RFC), Support Vector Classifier (SVC) and Multi-Layer Perceptron model (MLP) by working on their hyperparameters for the best-achieved classification result. The proposed solution attains the accuracy of 82.34% using best hyperparameter selection.


SMOTE Machine learning Classification Hyper-parameter selection Ensembling 


  1. 1.
    Cao, Y., Wang, H., Wang, C., Li, Z., Zhang, L., Zhang, L.: Mindfinder: interactive sketch-based image search on millions of images. In: Proceedings of the 18th ACM International Conference on Multimedia, pp. 1605–1608. ACM, October 2010Google Scholar
  2. 2.
    Plamondon, R., Srihari, S.N.: Online and off-line handwriting recognition: a comprehensive survey. IEEE Trans. Pattern Anal. Mach. Intell. 22(1), 63–84 (2000)CrossRefGoogle Scholar
  3. 3.
    Paulson, B., Eoff, B., Wolin, A., Johnston, J., Hammond, T.: Sketch-based educational games: drawing kids away from traditional interfaces. In: Proceedings of the 7th International Conference on Interaction Design and Children, pp. 133–136. ACM, June 2008Google Scholar
  4. 4.
    Davis, R.: Magic paper: sketch-understanding research. Computer 40(9), 34–41 (2007)CrossRefGoogle Scholar
  5. 5.
    Hammond, T., Davis, R.: LADDER, a sketching language for user interface developers. In: ACM SIGGRAPH 2007 Courses, p. 35. ACM, August 2007Google Scholar
  6. 6.
    Sun, Z., Wang, C., Zhang, L., Zhang, L.: Sketch2Tag: automatic hand-drawn sketch recognition. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 1255–1256). ACM, October 2012Google Scholar
  7. 7.
    Kokkinos, I., Maragos, P., Yuille, A.: Bottom-up & top-down object detection using primal sketch features and graphical models. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1893–1900. IEEE (2006)Google Scholar
  8. 8.
    Qi, Y., Guo, J., Li, Y., Zhang, H., Xiang, T., Song, Y.Z.: Sketching by perceptual grouping. In: 2013 20th IEEE International Conference on Image Processing (ICIP), pp. 270–274. IEEE, September 2013Google Scholar
  9. 9.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  10. 10.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)Google Scholar
  11. 11.
    Donahue, J., et al.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655, January 2014Google Scholar
  12. 12.
    Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. 31(4), 44 (2012)Google Scholar
  13. 13.
    Schneider, R.G., Tuytelaars, T.: Sketch classification and classification-driven analysis using fisher vectors. ACM Trans. Graph. (TOG) 33(6), 174 (2014)CrossRefGoogle Scholar
  14. 14.
    Davis, R.: Position statement and overview: sketch recognition at MIT. In: AAAI Sketch Understanding Symposium (2002)Google Scholar
  15. 15.
    Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)CrossRefGoogle Scholar
  16. 16.
    Deepa, T., Punithavalli, M.: An E-SMOTE technique for feature selection in high-dimensional imbalanced dataset. In: 2011 3rd International Conference on Electronics Computer Technology (ICECT), vol. 2, pp. 322–324. IEEE, April 2011Google Scholar
  17. 17.
    Keller, J.M., Gray, M.R., Givens, J.A.: A fuzzy k-nearest neighbor algorithm. IEEE Trans. Syst. Man Cybern. 4, 580–585 (1985)CrossRefGoogle Scholar
  18. 18.
    Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U.: When is “Nearest Neighbor” meaningful? In: Beeri, C., Buneman, P. (eds.) ICDT 1999. LNCS, vol. 1540, pp. 217–235. Springer, Heidelberg (1999). Scholar
  19. 19.
    Dudani, S.A.: The distance-weighted k-nearest-neighbor rule. IEEE Trans. Syst. Man Cybern. 4, 325–327 (1976)CrossRefGoogle Scholar
  20. 20.
    Suykens, J.A., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)CrossRefGoogle Scholar
  21. 21.
    Zhang, M., Zhang, D.X.: Trained SVMs based rules extraction method for text classification. In: IEEE International Symposium on IT in Medicine and Education, ITME 2008, pp. 16–19. IEEE, December 2008Google Scholar
  22. 22.
    Lau, K.W., Wu, Q.H.: Online training of support vector classifier. Pattern Recogn. 36(8), 1913–1920 (2003)CrossRefGoogle Scholar
  23. 23.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  24. 24.
    Jain, A.K., Mao, J., Mohiuddin, K.M.: Artificial neural networks: a tutorial. Computer 29(3), 31–44 (1996)CrossRefGoogle Scholar
  25. 25.
  26. 26.

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceJaypee Institute of Information TechnologyNoidaIndia

Personalised recommendations