Advertisement

Abstract

Deep neural networks (DNNs) have been expanded into medical fields and triggered the revolution of some medical applications by extracting complex features and achieving high accuracy and performance, etc. On the contrast, the large-scale network brings high requirements of both memory storage and computation resource, especially for portable medical devices and other embedded systems. In this work, we first train a DNN for pneumonia detection using the dataset provided by RSNA Pneumonia Detection Challenge [4]. To overcome hardware limitation for implementing large-scale networks, we develop a systematic structured weight pruning method with filter sparsity, column sparsity and combined sparsity. Experiments show that we can achieve up to 36x compression ratio compared to the original model with 106 layers, while maintaining no accuracy degradation. We evaluate the proposed methods on an embedded low-power device, Jetson TX2, and achieve low power usage and high energy efficiency.

Keywords

Pneumonia detection YOLO Structured weight pruning 

References

  1. 1.
    Deaths: Final data for 2015. supplemental tables. https://www.cdc.gov/nchs/data/nvsr/nvsr66/nvsr66_06_tables.pdf. Accessed 24 May 2019
  2. 2.
  3. 3.
    National ambulatory medical care survey: 2015 emergency department summary tables. https://www.cdc.gov/nchs/data/nhamcs/web_tables/2015_ed_web_tables.pdf. Accessed 24 May 2019
  4. 4.
  5. 5.
    Han, S., Mao, H., Dally, W.J.: Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 (2015)
  6. 6.
    He, Y., Zhang, X., Sun, J.: Channel pruning for accelerating very deep neural networks. In: International Conference on Computer Vision (ICCV), vol. 2 (2017)Google Scholar
  7. 7.
    Li, H., et al.: ADMM-based weight pruning for real-time deep learning acceleration on mobile devices. In: Proceedings of the 2019 on Great Lakes Symposium on VLSI, pp. 501–506. ACM (2019)Google Scholar
  8. 8.
    Lin, S., et al.: FFT-based deep learning deployment in embedded systems. In: 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1045–1050. IEEE (2018)Google Scholar
  9. 9.
    Lodha, R., Kabra, S.K., Pandey, R.M.: Antibiotics for community-acquired pneumonia in children. Cochrane Database Syst. Rev. (6) (2013) Google Scholar
  10. 10.
    Rajpurkar, P., et al.: Chexnet: radiologist-level pneumonia detection on Chest X-rays with deep learning. arXiv preprint arXiv:1711.05225 (2017)
  11. 11.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)Google Scholar
  12. 12.
    Redmon, J., Farhadi, A.: YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)Google Scholar
  13. 13.
    Redmon, J., Farhadi, A.: YOLOv3: an incremental improvement. arXiv preprint arXiv:1804.02767 (2018)
  14. 14.
    Ruuskanen, O., Lahti, E., Jennings, L.C., Murdoch, D.R.: Viral pneumonia. Lancet 377(9773), 1264–1275 (2011)CrossRefGoogle Scholar
  15. 15.
    Shin, H.C., Roberts, K., Lu, L., Demner-Fushman, D., Yao, J., Summers, R.M.: Learning to read chest x-rays: recurrent neural cascade model for automated image annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2497–2506 (2016)Google Scholar
  16. 16.
    Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017)Google Scholar
  17. 17.
    Wang, X., Peng, Y., Lu, L., Lu, Z., Summers, R.M.: TieNet: text-image embedding network for common thorax disease classification and reporting in chest x-rays. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9049–9058 (2018)Google Scholar
  18. 18.
    Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H.: Learning structured sparsity in deep neural networks. In: Advances in Neural Information Processing Systems, pp. 2074–2082 (2016)Google Scholar
  19. 19.
    Xie, J., He, T., Zhang, Z., Zhang, H., Zhang, Z., Li, M.: Bag of tricks for image classification with convolutional neural networks. arXiv preprint arXiv:1812.01187 (2018)
  20. 20.
    Zhang, T., et al.: A systematic DNN weight pruning framework using alternating direction method of multipliers. arXiv preprint arXiv:1804.03294 (2018)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Hongjia Li
    • 1
    Email author
  • Sheng Lin
    • 1
  • Ning Liu
    • 1
  • Caiwen Ding
    • 2
  • Yanzhi Wang
    • 1
  1. 1.Northeastern UniversityBostonUSA
  2. 2.University of ConnecticutStorrsUSA

Personalised recommendations