Multi-model LSTM-based convolutional neural networks for detection of apple diseases and pests

  • Muammer TurkogluEmail author
  • Davut Hanbay
  • Abdulkadir Sengur
Original Research


In this paper, we proposed Multi-model LSTM-based Pre-trained Convolutional Neural Networks (MLP-CNNs) as an ensemble majority voting classifier for the detection of plant diseases and pests. The proposed hybrid model is based on the combination of LSTM network with pre-trained CNN models. Specifically, in transfer learning, we adopted deep feature extraction from various fully connected layers of these pre-trained deep models. AlexNet, GoogleNet and DenseNet201 models are used in this work for feature extraction. The extracted deep features are then fed into the LSTM layer in order to construct a robust hybrid model for apple disease and pest detection. Later, the output predictions of three LSTM layers determined the class labels of the input images by majority voting classifier. In addition, we use an automatic scheme for determining the best choice of the network parameters of the LSTM layer. The experiments are carried out using data consisting of real-time apple disease and pest images from Turkey and the accuracy rates are calculated for performance evaluation. The experimental results show that by using the proposed ensemble combination structure, the results are comparable to, or better than, the pre-trained deep architectures.


Plant diseases and pests detection Convolutional neural networks Deep learning architectures Deep features LSTM 



  1. Akcayol MA (2018) Derin Öğrenme. Gazi Üniversitesi, Accessed 25 Mar 2018
  2. Amara J, Bouaziz B, Algergawy AA (2017) Deep learning-based approach for banana leaf diseases classification. In: Lecture notes in informatics (LNI). Gesellschaft für Informatik, Bonn, Germany, pp 79–88Google Scholar
  3. Arivazhagan S, Shebiah RN, Ananthi S, Varthini SV (2013) Detection of unhealthy region of plant leaves and classification of plant leaf diseases using texture features. Agric Eng Int CIGR J 15(1):211–217Google Scholar
  4. Athanikar G, Badar P (2016) Potato leaf diseases detection and classification system. Int J Comp Sci Mob Comput (IJCSMC) 5:76–78Google Scholar
  5. Badnakhe MR, Deshmukh PR (2011) An application of K-means clustering and artificial intelligence in pattern recognition for crop diseases. Int Conf Adv Inf Technol IPCSIT 20:134–138Google Scholar
  6. Barbedo JGA, Koenigkan LV, Santos TT (2016) Identifying multiple plant diseases using digital image processing. Biosys Eng 147:104–116Google Scholar
  7. Brahimi M, Boukhalfa K, Moussaoui A (2017) Deep learning for tomato diseases: classification and symptoms visualization. Appl Artif Intell 31(4):1–17Google Scholar
  8. Brahimi M, Arsenovic M, Laraba S, Sladojevic S, Boukhalfa K, Moussaoui A (2018) Deep learning for plant diseases: detection and saliency map visualisation. In: Human-computer interaction series, Springer, Cham, pp 93–117Google Scholar
  9. Brownlee J (2018) What is the difference between a batch and an epoch in a neural network. machine learning mastery, Accessed 26 Sep 2018
  10. Budak U, Bajaj V, Akbulut Y, Atilla O, Sengur A (2019) An effective hybrid model for EEG-based drowsiness detection. IEEE Sens J 19(17):7624–7631Google Scholar
  11. Carkaci N (2018) Derin Öğrenme Uygulamalarında En Sık kullanılan Hiper-parametreler. Deep Learning Turkey, Accessed 15 Sep 2018
  12. Chaudhary P, Chaudhari AK, Cheeran AN, Godara S (2012) Color transform based approach for disease spot detection on plant leaf. Int Comput Sci Telecommun 3(6):65–70Google Scholar
  13. Choi K, Fazekas G, Sandler M (2016) Text-based LSTM networks for automatic music composition. arXiv preprint arXiv:1604.05358
  14. Chuanlei Z, Shanwen Z, Jucheng Y, Yancui S, Jia C (2017) Apple leaf disease identification using genetic algorithm and correlation based feature selection method. Int J Agric Biol Eng 10(2):74–83Google Scholar
  15. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555
  16. DeChant C, Wiesner-Hanks T, Chen S, Stewart EL, Yosinski J, Gore MA, Nelson RJ, Lipson H (2017) Automated identification of northern leaf blight-infected maize plants from field imagery using deep learning. Phytopathology 107(11):1426–1432Google Scholar
  17. Dolek I (2018) LSTM. Deep Learning Turkey, Accessed 10 June 2018
  18. Donahue J, Anne Hendricks L, Guadarrama S, Rohrbach M, Venugopalan S, Saenko K, Darrell T (2015) Long-term recurrent convolutional networks for visual recognition and description. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2625–2634Google Scholar
  19. Dubey SR, Jalal AS (2012) Detection and classification of apple fruit diseases using complete local binary patterns. In: Computer and communication technology (ICCCT), 2012 Third international conference on IEEE, pp 346–351Google Scholar
  20. Duneja A, Puyalnithi T, Vankadara MV, Chilamkurti N (2018) Analysis of inter-concept dependencies in disease diagnostic cognitive maps using recurrent neural network and genetic algorithms in time series clinical data for targeted treatment. J Ambient Intell Humaniz Comput 10(10):1–9Google Scholar
  21. Erguder H (2018) Recurrent Neural Network Nedir. Deep Learning Turkey, Accessed 12 Sep 2018
  22. Fei M, Jiang W, Mao W (2018) Creating personalized video summaries via semantic event detection. J Ambient Intell Humaniz Comput.
  23. Ferentinos KP (2018) Deep learning models for plant disease detection and diagnosis. Comput Electron Agric 145:311–318Google Scholar
  24. Fuentes A, Yoon S, Kim SC, Park DS (2017) A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors 17(9):2022Google Scholar
  25. Guzel M (2012) The importance of good agricultural practices (gap) in the context of quality practices in agriculture and a sample application. PhD Thesis, DokuzEylul UniversityGoogle Scholar
  26. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780Google Scholar
  27. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. CVPR 1(2):3Google Scholar
  28. Im DJ, Tao M, Branson K (2016) An empirical analysis of deep network loss surfaces. arXiv preprint arXiv:1612.04010
  29. Johnson R, Zhang T (2016) Supervised and semi-supervised text categorization using LSTM for region embeddings. arXiv preprint arXiv:1602.02373
  30. Kathuria A (2018) Intro to optimization in deep learning: Momentum, RMSProp and Adam. Accessed 20 Oct 2018
  31. Kawasaki Y, Uga H, Kagiwada S, Iyatomi H (2015) Basic study of automated diagnosis of viral plant diseases using convolutional neural networks. In: International symposium on visual computing, Springer, Cham, pp 638–645Google Scholar
  32. Keskar NS, Socher R (2017) Improving generalization performance by switching from adam to sgd. arXiv preprint arXiv:1712.07628
  33. Kilimci ZH, Akyokus S (2018) Deep learning-and word embedding-based heterogeneous classifier ensembles for text classification. Complexity 2018:7130146. Google Scholar
  34. Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: International conference on learning representations, pp 1–13Google Scholar
  35. Koutnik J, Greff K, Gomez F, Schmidhuber J (2014) A clockwork rnn. arXiv preprint arXiv:1402.3511
  36. Krizhevsk A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105Google Scholar
  37. Liu B, Zhang Y, He D, Li Y (2017) Identification of apple leaf diseases based on deep convolutional neural networks. Symmetry 10(1):11Google Scholar
  38. Lu Y, Yi S, Zeng N, Liu Y, Zhang Y (2017) Identification of rice diseases using deep convolutional neural networks. Neurocomputing 267:378–384Google Scholar
  39. Moawad A (2018) The magic of LSTM neural networks. Deep Learning Turkey, Accessed 26 July 2007
  40. Mohanty SP, Hughes DP, Salathé M (2016) Using deep learning for image-based plant disease detection. Front Plant Sci 7:1419Google Scholar
  41. Prashar K, Talwar R, Kant C (2017) Robust automatic cotton crop disease recognition (ACDR) method using the hybrid feature descriptor with SVM. In: 4th International conference on computing on sustainable global development, vol INDIACom-2017, PaschimVihar, New DelhiGoogle Scholar
  42. Priya CA, Balasaravanan T, Thanamani AS (2012) An efficient leaf recognition algorithm for plant classification using support vector machine. Pattern recognition, informatics and medical engineering (PRIME), In: 2012 International conference on IEEE, pp 428–432Google Scholar
  43. Pydipati R, Burks TF, Lee WS (2006) Identification of citrus disease using color texture features and discriminant analysis. Comput electron Agric 52(1-2):49–59Google Scholar
  44. Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747
  45. Russakovsky O et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252MathSciNetGoogle Scholar
  46. Shabanzade M, Zahedi M, Aghvami SA (2011) Combination of local descriptors and global features for leaf recognition. Signal Image Process 2(3):23–31Google Scholar
  47. Singh V, Misra AK (2017) Detection of plant leaf diseases using image segmentation and soft computing techniques. Inf Process Agric 4(1):41–49Google Scholar
  48. Sladojevic S, Arsenovic M, Anderla A, Culibrk D, Stefanovic D (2016) Deep neural networks based recognition of plant diseases by leaf image classification. Comput Intell Neurosci 2016:3289801. Google Scholar
  49. Srivastava S, Mukherjee P, Lall B, Jaiswal K (2017) Object classification using ensemble of local and deep features. In: 2017 Ninth international conference on advances in pattern recognition (ICAPR), pp 1–6Google Scholar
  50. Szegedy C et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9Google Scholar
  51. Tan JH et al (2018) Application of stacked convolutional and long short-term memory network for accurate identification of CAD ECG signals. Comput Biol Med 94:19–26Google Scholar
  52. Tigadi B, Sharma B (2016) Banana plant disease detection and grading using image processing. Int J Eng Sci 6512Google Scholar
  53. TUIK (2018) Türkiye İstatistik Kurumu. Accessed 12 Sep 2018
  54. Tumen V, Yıldırım O, Ergen B (2018) Detection of driver drowsiness in driving environment using deep learning methods. In: 2018 electric electronics, computer science, biomedical engineerings’ meeting (EBBT), pp 1–5Google Scholar
  55. Turkoglu M, Hanbay D (2015) Classification of the grape varieties based on leaf recognition by using SVM classifier. In: Signal processing and communications applications conference (SIU), 2015 23th, pp 2674–2677Google Scholar
  56. Vedaldi A, Lenc K (2015) Matconvnet: Convolutional neural networks for matlab. In: Proceedings of the 23rd ACM international conference on multimedia, ACM, pp 689–692Google Scholar
  57. Vladimir VN, Vapnik V (1995) The nature of statistical learning theory. Springer, New York, USAGoogle Scholar
  58. Wang J, Yang Y, Mao J, Huang Z, Huang C, Xu W (2016) Cnn-rnn: a unified framework for multi-label image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2285–2294Google Scholar
  59. Wang G, Su Y, Wang J (2017) Automatic image-based plant disease severity estimation using deep learning. Comput Intell Neurosci 2017:2917536. Google Scholar
  60. Inik O, Ulker E (2017) Deep learning and deep learning models used in image analysis. Gaziosmanpaşa Bilimsel Araştırma Dergisi 6(3):85–104Google Scholar
  61. Yao K, Cohn T, Vylomova K, Duh K, Dyer C (2015) Depth-gated recurrent neural networks. arXiv preprint arXiv:1508.03790
  62. Yigit A (2017) İşsüreçlerindeinsangörüsünüderinöğrenmeiledestekleme. Master’s thesis, TrakyaÜniversitesiGoogle Scholar
  63. Yu J, Xie L, Xiao X, Chng ES (2017) A hybrid neural network hidden Markov model approach for automatic story segmentation. J Ambient Intell Humaniz Comput 8(6):925–936Google Scholar
  64. Zhou X, Xie L, Zhang P, Zhang Y (2017) Online object tracking based on BLSTM-RNN with contextual-sequential labeling. J Ambient Intell Humaniz Comput 8(6):861–870Google Scholar
  65. Zhu W, Lan C, Xing J, Zeng W, Li Y, Shen L, Xie X (2016) Co-occurrence feature learning for skeleton based action recognition using regularized deep LSTM networks. AAAI 2(5):6Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Computer Engineering DepartmentBingol UniversityBingölTurkey
  2. 2.Computer Engineering DepartmentInonu UniversityMalatyaTurkey
  3. 3.Electrical and Electronics Engineering DepartmentFirat UniversityElazigTurkey

Personalised recommendations