Classifying Heart Sounds Using Images of Motifs, MFCC and Temporal Features

  • Diogo Marcelo NogueiraEmail author
  • Carlos Abreu Ferreira
  • Elsa Ferreira Gomes
  • Alípio M. Jorge
Image & Signal Processing
Part of the following topical collections:
  1. Artificial Intelligence in Medicine (AIM)


Cardiovascular disease is the leading cause of death in the world, and its early detection is a key to improving long-term health outcomes. The auscultation of the heart is still an important method in the medical process because it is very simple and cheap. To detect possible heart anomalies at an early stage, an automatic method enabling cardiac health low-cost screening for the general population would be highly valuable. By analyzing the phonocardiogram signals, it is possible to perform cardiac diagnosis and find possible anomalies at an early-term. Therefore, the development of intelligent and automated analysis tools of the phonocardiogram is very relevant. In this work, we use simultaneously collected electrocardiograms and phonocardiograms from the Physionet Challenge database with the main objective of determining whether a phonocardiogram corresponds to a “normal” or “abnormal” physiological state. Our main contribution is the methodological combination of time domain features and frequency domain features of phonocardiogram signals to improve cardiac disease automatic classification. This novel approach is developed using both features. First, the phonocardiogram signals are segmented with an algorithm based on a logistic regression hidden semi-Markov model, which uses electrocardiogram signals as a reference. Then, two groups of features from the time and frequency domain are extracted from the phonocardiogram segments. One group is based on motifs and the other on Mel-frequency cepstral coefficients. After that, we combine these features into a two-dimensional time-frequency heat map representation. Lastly, a binary classifier is applied to both groups of features to learn a model that discriminates between normal and abnormal phonocardiogram signals. In the experiments, three classification algorithms are used: Support Vector Machines, Convolutional Neural Network, and Random Forest. The best results are achieved when both time and Mel-frequency cepstral coefficients features are considered using a Support Vector Machines with a radial kernel.


Phonocardiogram Electrocardiogram Mel-frequency cepstral coefficients Motifs Time features 



This work is supported by the NanoSTIMA Project: Macro-to-Nano Human Sensing: Towards Integrated Multimodal Health Monitoring and Analytics/NORTE-01-0145-FEDER-000016 which is financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF).

Funding Information

This study was funded by the NanoSTIMA Project: Macro-to-Nano Human Sensing: Towards Integrated Multimodal Health Monitoring and Analytics/ NORTE-01-0145-FEDER-000016 which is financed by the North Portugal Regional Operational Programme (NORTE 2020), under the PORTUGAL 2020 Partnership Agreement, and through the European Regional Development Fund (ERDF).

Compliance with Ethical Standards

Conflict of interests


Informed Consent

Informed consent was obtained from all individual participants included in the study.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.


  1. 1.
  2. 2.
    Balili, C.C., Sobrepena, M.C.C., and Naval, P.C.: Classification of heart sounds using discrete and continuous wavelet transform and random forests. In: 2015 3Rd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 655–659, 2015Google Scholar
  3. 3.
    Barschdorff, D., Bothe, A., and Rengshausen, U.: Heart sound analysis using neural and statistical classifiers: a comparison. In: [1989] Proceedings. Computers in Cardiology, pp. 415–418, 1989Google Scholar
  4. 4.
    Boussaa, M., Atouf, I., Atibi, M., and Bennis, A.: Ecg signals classification using mfcc coefficients and ann classifier. In: 2016 International Conference on Electrical and Information Technologies, pp. 480–484, 2016Google Scholar
  5. 5.
    Castro, N., and Azevedo, P.: Multiresolution Motif Discovery in Time Series, pp. 665–676.
  6. 6.
    Chen, T.E., Yang, S.I., Ho, L.T., et al., S1 and s2 heart sound recognition using deep neural networks. IEEE Trans. Biomed. Eng. 64(2):372–380, 2017.CrossRefGoogle Scholar
  7. 7.
    Clifford, G.D., Liu, C., Moody, B., Springer, D., Silva, I., Li, Q., and Mark, R.G.: Classification of normal/abnormal heart sound recordings: The physionet/computing in cardiology challenge 2016. In: 2016 Computing in Cardiology Conference (cinc), pp. 609–612, 2016Google Scholar
  8. 8.
    Colonna, J., Peet, T., Ferreira, C.A., Jorge, A.M., Gomes, E.F., and Gama, J.A.: Automatic classification of anuran sounds using convolutional neural networks. In: Proceedings of the Ninth International C* Conference on Computer Science & Software Engineering, C3S2E ’16, pp. 73–78. ACM, 2016Google Scholar
  9. 9.
    Ergen, B., Tatar, Y., and Gulcur, H.O., Time–frequency analysis of phonocardiogram signals using wavelet transform: a comparative study. Comput. Methods Biomech. Biomed. Engin. 15(4):371–381, 2012.CrossRefGoogle Scholar
  10. 10.
    Gomes, E., Bentley, P., Coimbra, M., Pereira, E., and Deng, Y.: Classifying heart sounds: Approaches to the pascal challenge pp 337–340, 2013Google Scholar
  11. 11.
    Gomes, E.F., Jorge, A.M., and Azevedo, P.J.: Classifying heart sounds using multiresolution time series motifs: an exploratory study. In: Proceedings of the International C* Conference on Computer Science and Software Engineering, pp. 23–30. ACM, 2013Google Scholar
  12. 12.
    Gomes, E.F., Jorge, A.M., and Azevedo, P.J.: Classifying heart sounds using sax motifs, random forests and text mining techniques. In: Proceedings of the 18th International Database Engineering & Applications Symposium, pp. 334–337. ACM, 2014Google Scholar
  13. 13.
    Huiying, L., Sakari, L., and Iiro, H.: A heart sound segmentation algorithm using wavelet decomposition and reconstruction. In: Engineering in Medicine and Biology Society, 1997. Proceedings of the 19th Annual International Conference of the IEEE, vol. 4, pp. 1630–1633, 1997Google Scholar
  14. 14.
    Inc., P.T.: Collaborative data science., 2015
  15. 15.
    Kishore, K.V.K., and Satish, P.K.: Emotion recognition in speech using mfcc and wavelet features. In: 2013 3rd IEEE International Advance Computing Conference (IACC), pp. 842–847, 2013Google Scholar
  16. 16.
    Kumar, D., Carvalho, P., Antunes, M., Paiva, R.P., and Henriques, J.: Heart murmur classification with feature selection. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, pp. 4566–4569, 2010Google Scholar
  17. 17.
    Lalitha, S., Geyasruti, D., Narayanan, R., and Shravani, M., Emotion detection using mfcc and cepstrum features. Prog. Comput. Sci. 70:29–35, 2015.CrossRefGoogle Scholar
  18. 18.
    Lin, J., Keogh, E., Lonardi, S., and Patel, P.: Finding motifs in time series. In: Proceedings of the 2nd Workshop on Temporal Data Mining, pp. 53–68, 2002Google Scholar
  19. 19.
    Liu, C., Springer, D., and Li, Q., Et. al.: An open access database for the evaluation of heart sound algorithms. Physiol. Meas. 37(12):2181, 2016.CrossRefGoogle Scholar
  20. 20.
    Lubaib, P., and Muneer, K.A., The heart defect analysis based on pcg signals using pattern recognition techniques. Procedia Technol. 24:1024–1031, 2016.CrossRefGoogle Scholar
  21. 21.
    Malarvili, M.B., Kamarulafizam, I., Hussain, S., and Helmi, D.: Heart sound segmentation algorithm based on instantaneous energy of electrocardiogram. In: Computers in Cardiology, pp. 327–330, 2003Google Scholar
  22. 22.
    Mozaffarian, D., Benjamin, E.J., and Go Alan, S.: E.a.: Heart disease and stroke statistics–2016 update. Circulation, 2015Google Scholar
  23. 23.
    Nogueira, D.M., Ferreira, C.A., and Jorge, A.M.: Classifying heart sounds using images of MFCC and temporal features. In: Progress in Artificial Intelligence - 18th EPIA Conference on Artificial Intelligence, EPIA 2017, pp. 186–203, 2017CrossRefGoogle Scholar
  24. 24.
    Obaidat, M.S., Phonocardiogram signal analysis: techniques and performance comparison. J. Med. Eng. Technol. 17(6):221–7, 1993.CrossRefGoogle Scholar
  25. 25.
    Rangayyan, R., and Lehner, R., Phonocardiogram signal analysis: a review. Crit Rev Biomed Eng. 15(3): 211–236, 1987.PubMedGoogle Scholar
  26. 26.
    Rubin, J., Abreu, R., Ganguli, A., Nelaturi, S., Matei, I., and Sricharan, K.: Classifying heart sound recordings using deep convolutional neural networks and mel-frequency cepstral coefficients. In: Computing in Cardiology Conference (cinc), 2016, pp. 813–816. IEEE, 2016Google Scholar
  27. 27.
    Segal, B.L., Phonocardiology: Integrated study of heart sounds and murmurs. JAMA 224(11):1536–1536, 1973.CrossRefGoogle Scholar
  28. 28.
    Shi, W., Gong, Y., and Wang, J.: Improving cnn performance with min-max objective. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, pp. 2004–2010, 2016Google Scholar
  29. 29.
    Springer, D.B., Tarassenko, L., and Clifford, G.D., Logistic regression-hsmm-based heart sound segmentation. IEEE Trans. Biomed. Eng. 63(4):822–832, 2016.PubMedGoogle Scholar
  30. 30.
    White, P.R., Collis, W.B., and Salmon, A.P.: Time-frequency analysis of heart murmurs in children. In: IEE Colloquium on Time-Frequency Analysis of Biomedical Signals (Digest No. 1997/006), pp. 3/1–3/4Google Scholar
  31. 31.
    Wu, J., Zhou, S., Wu, Z.M., and Wu, X.: Research on the method of characteristic extraction and classification of phonocardiogram. In: 2012 International Conference on Systems and Informatics, pp. 1732–1735, 2012.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.INESC TECPortoPortugal
  2. 2.Instituto Superior de Engenharia do PortoPortoPortugal
  3. 3.Faculdade de Ciências da Universidade do PortoPortoPortugal

Personalised recommendations