Failure Prediction and Health Status Assessment of Storage Systems with Decision Trees

  • Kamaljit KaurEmail author
  • Kuljit Kaur
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 955)


Prediction of imminent failures of large scale storage systems is critical to prevent loss of data. Various machine learning and statistical methods based on SMART attributes have been proposed by different researchers. Although they have achieved good prediction accuracy, but most of them focus on predicting the status of hard drives as “good” or “failed”. Moreover, the performance of hard drives deteriorates slowly than abruptly as indicated by continuous change in their corresponding SMART attributes. So, these models cannot predict this kind of continuous change. This paper gives decision tree based failure prediction model for hard drives which gives a better prediction accuracy. Experiments show that decision tree based model anticipates through 99.99% of failures, along with a false alarm rate under 0.001%. Also, we introduce prediction of lead time that proactively quantifies health status of hard drives to generate warnings in advance for triggering backups. We test the proposed model on a real-world dataset.


Fault tolerance Decision trees ID3 


  1. 1.
    Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol. Biol. 147, 195–197 (1981)CrossRefGoogle Scholar
  2. 2.
    Schroeder, B., Gibson, G.A.: Disk failures in the real world: what does an MTTF of 1, 000, 000 hours mean to you? In: FAST, vol. 7, pp. 1–16 (2007)Google Scholar
  3. 3.
    Xin, Q., Miller, E.L., Schwarz, T., Long, D.D., Brandt, S.A., Litwin, W.: Reliability mechanisms for very large storage systems. In: Proceedings of 20th IEEE/11th NASA Goddard Conference on IEEE Mass Storage Systems and Technologies, 2003, pp. 146–156 (2003)Google Scholar
  4. 4.
    Murray, J.F., Hughes, G.F., Kreutz-Delgado, K.: Machine learning methods for predicting failures in hard drives: a multiple-instance application. J. Mach. Learn. Res. 6, 783–816 (2005)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Hamerly, G., Elkan, C.: Bayesian approaches to failure prediction for disk drives. In: ICML, vol. 1, pp. 202–209 (2001)Google Scholar
  6. 6.
    Hughes, G.F., Murray, J.F., Kreutz-Delgado, K., Elkan, C.: Improved disk-drive failure warnings. IEEE Trans. Reliab. 51, 350–357 (2002)CrossRefGoogle Scholar
  7. 7.
    Murray, J.F., Hughes, G.F., Kreutz-Delgado, K.: Hard drive failure prediction using non-parametric statistical methods. In: Proceedings of ICANN/ICONIP (2003)Google Scholar
  8. 8.
    Zhao, Y., Liu, X., Gan, S., Zheng, W.: Predicting disk failures with HMM-and HSMM-based approaches. In: Perner, P. (ed.) ICDM 2010. LNCS (LNAI), vol. 6171, pp. 390–404. Springer, Heidelberg (2010). Scholar
  9. 9.
    Zhu, B., Wang, G., Liu, X., Hu, D., Lin, S., Ma, J.: Proactive drive failure prediction for large scale storage systems. In: 2013 IEEE 29th Symposium on Mass Storage Systems and Technologies (MSST), pp. 1–5. IEEE (2013)Google Scholar
  10. 10.
    Wang, Y., Miao, Q., Pecht, M.: Health monitoring of hard disk drive based on Mahalanobis distance. In: Prognostics and System Health Management Conference (PHM-Shenzhen), 2011, pp. 1–8. IEEE (2011)Google Scholar
  11. 11.
    Wang, Y., Miao, Q., Ma, E.W., Tsui, K.L., Pecht, M.G.: Online anomaly detection for hard disk drives based on Mahalanobis distance. IEEE Trans. Reliab. 62, 136–145 (2013)CrossRefGoogle Scholar
  12. 12.
    Li, J., et al.: Hard drive failure prediction using classification and regression trees. In: P2014 44th Annual IEEE/IFIP International Conference on IEEE Dependable Systems and Networks (DSN), pp. 383–394 (2014)Google Scholar
  13. 13.
    Xu, C., Wang, G., Liu, X., Guo, D., Liu, T.Y.: Health status assessment and failure prediction for hard drives with recurrent neural networks. IEEE Trans. Comput. 65, 3502–3508 (2016)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Allen, B.: Monitoring hard disks with smart. Linux J. 117, 74–77 (2004)Google Scholar
  15. 15.

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Computer Engineering and TechnologyGuru Nanak Dev UniversityAmritsarIndia

Personalised recommendations