Abstract
Heart sound analysis is a preliminary procedure performed by a physician and involves examining the heart beats to detect the symptoms of cardiovascular diseases (CVDs). With recent developments in clinical science and the availability of devices to capture heart beats, researchers are now exploring the possibility of a machine assisted heart sound analysis system that can augment the clinical expertise of the physician in early detection of CVD. In this paper, we study the application of machine learning algorithms in classifying abnormal/normal heart sounds based on the short (\(\le \)120 s) audio phonocardiogram (PCG) recordings. To this end, we use the largest public audio PCG dataset released as part of the 2016 PhysioNet/Cardiology in Computing Challenge. The data comes from different patients, most of who have had no previous history of cardiac disease and some with known cardiac diseases. In our study, we use these audio recordings to train three different classification algorithms and discuss the effects of class imbalance (normal vs. abnormal) on the precision-recall trade-off of the prediction task. Specifically, our goal is to find a suitable model that takes into account the inherent imbalance and optimize the precision-recall trade-off with a higher emphasis on increasing recall. Bagged random forest models with majority (normal) class under sampling gave us the best configuration resulting in average recall over 91% with nearly 64% average precision.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
Henceforth, we only discuss the results using the random forest approach. Comparisons with the other two classifiers are presented in Sect. 6.
- 2.
We emphasize that prediction of abnormality is made per recording, not per cycle, given a full recording’s multiple cycles together provide the signal for prediction.
- 3.
The notion of accuracy used here is the same as in the 2016 CinC challenge where it is set to (recall+specificity)/2.
- 4.
Even this may not be exact comparison because the numbers of folds were different.
References
American Heart Association: Heart disease and stroke statistics (2017). At-a-glance. https://www.heart.org/idc/groups/ahamah-public/@wcm/@sop/@smd/documents/downloadable/ucm_491265.pdf
Archer, K.J., Kimes, R.V.: Empirical characterization of random forest variable importance measures. Comput. Stat. Data Anal. 52(4), 2249–2260 (2008)
Olshausen, B.A.: Aliasing. http://redwood.berkeley.edu/bruno/npb261/aliasing.pdf
Cleveland Clinic: Heart and blood vessels: how does the heart beat. https://my.clevelandclinic.org/health/articles/heart-blood-vessels-heart-beat
Clifford, G.D., Liu, C., Moody, B., Springer, D., Silva, I., Li, Q., Mark, R.G.: Classification of normal/abnormal heart sound recordings: the physionet/computing in cardiology challenge 2016. In: Computing in Cardiology Conference (CinC), pp. 609–612. IEEE (2016)
Goldberger, A.L., Amaral, L.A., Glass, L., Hausdorff, J.M., Ivanov, P.C., Mark, R.G., Mietus, J.E., Moody, G.B., Peng, C.-K., Stanley, H.E.: Physiobank, physiotoolkit, and physionet. Circulation 101(23), e215–e220 (2000)
Hasan, M.R., Jamil, M., Rabbani, M.G., Rahman, M.S.: Speaker identification using mel frequency cepstral coefficients. In: 3rd International Conference on Electrical and Computer Engineering, pp. 565–568 (2004)
Liu, C., Springer, D., Li, Q., Moody, B., Juan, R.A., Chorro, F.J., Castells, F., Roig, J.M., Silva, I., Johnson, A.E., et al.: An open access database for the evaluation of heart sound algorithms. Physiol. Meas. 37(12), 2181 (2016)
Oppenheim, A.V., Verghese, G.C.: Signals, Systems and Inference. Pearson, Boston (2015)
Ozenne, B., Subtil, F., Maucort-Boulch, D.: The precision-recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases. J. Clin. Epidemiol. 68(8), 855–859 (2015)
Potes, C., Parvaneh, S., Rahman, A., Conroy, B.: Ensemble of feature-based and deep learning-based classifiers for detection of abnormal heart sounds. In: Computing in Cardiology Conference (CinC), pp. 621–624. IEEE (2016)
Rubin, J., Abreu, R., Ganguli, A., Nelaturi, S., Matei, I., Sricharan, K.: Classifying heart sound recordings using deep convolutional neural networks and mel-frequency cepstral coefficients. In: Computing in Cardiology Conference (CinC), pp. 813–816. IEEE (2016)
Saito, T., Rehmsmeier, M.: The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE 10(3), e0118432 (2015)
Springer, D.B., Tarassenko, L., Clifford, G.D.: Logistic regression-hsmm-based heart sound segmentation. IEEE Trans. Biomed. Eng. 63(4), 822–832 (2016)
Wallace, B.C., Small, K., Brodley, C.E., Trikalinos, T.A.: Class imbalance, redux. In: 2011 IEEE 11th International Conference on Data Mining (ICDM), pp. 754–763. IEEE (2011)
Whitaker, B.M., Suresha, P.B., Liu, C., Clifford, G., Anderson, D.: Combining sparse coding and time-domain features for heart sound classification. Physiol. Meas. 38, 1701–1713 (2017)
Zabihi, M., Rad, A.B., Kiranyaz, S., Gabbouj, M., Katsaggelos, A.K.: Heart sound anomaly and quality detection using ensemble of neural networks without segmentation. In: Computing in Cardiology Conference (CinC), pp. 613–616. IEEE (2016)
Acknowledgements
We thank anonymous reviewers for their honest and constructive criticism of our paper. Our work is primarily supported by the National Library of Medicine through grant R21LM012274. We are also supported by the National Center for Advancing Translational Sciences through grant UL1TR001998. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bopaiah, J., Kavuluru, R. (2017). Precision/Recall Trade-Off Analysis in Abnormal/Normal Heart Sound Classification. In: Reddy, P., Sureka, A., Chakravarthy, S., Bhalla, S. (eds) Big Data Analytics. BDA 2017. Lecture Notes in Computer Science(), vol 10721. Springer, Cham. https://doi.org/10.1007/978-3-319-72413-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-72413-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-72412-6
Online ISBN: 978-3-319-72413-3
eBook Packages: Computer ScienceComputer Science (R0)