High-Performance Detection of Concealed Forbidden Objects on Human Body with Deep Neural Networks Based on Passive Millimeter Wave and Visible Imagery

  • Lin Guo
  • Shiyin QinEmail author


In this paper, a high-performance detection algorithm of concealed forbidden objects on human body is presented based on deep neural networks (DNN) and complementary advantages of passive millimeter wave imagery (PMMWI) and visible imagery (VI). With well capacity of penetrability, PMMWI can effectively reveal suspected forbidden objects concealed on human body without harm of ionizing radiation compared with conventional X-ray methods. However, due to its current limited imaging capability, the resolution of PMMWI is still unsatisfactory and easy to result in false alarms. Therefore, by complementarity of superiorities, VI is employed to overcome the deficiency of confusions. In this way, massive image samples of PMMWI and VI are simultaneously acquired and manually annotated as necessary training datasets to carry out deep learning on DNN models so as to achieve high-performance human body profile segmentation on both PMMWI and VI. Then, high-precision region registration of human body profiles is implemented between PMMWI and VI to localize and confirm high confident suspected targets and remove false alarm regions as well. According to the principle of synthetic integration and global optimization, a high performance detection algorithm system is constructed, analyzed, and assessed. A series of comprehensive experiment results demonstrate the outstanding performance of our proposed detection algorithm.


Passive millimeter wave imagery Concealed forbidden object detection Deep neural networks Deep learning Semantic segmentation Security check 


Funding Information

This work was supported in part by the National Nature Science Foundation of China (Grant Nos. 61731001 and U1435220).


  1. 1.
    Maqueda I G, De La Blanca N P, Molina R, et al. Fast millimeter wave threat detection algorithm[C]. European Signal Processing Conference (EUSIPCO), 2015, pp:599–603.Google Scholar
  2. 2.
    Wang S, Ye Z, Wang Y. Real–time dangerous objects detection in millimeter wave images[C]. Tenth International Conference on Digital Image Processing (ICDIP 2018). International Society for Optics and Photonics, 2018, 10806: 108060Z, pp:1–7.Google Scholar
  3. 3.
    Yeom S., Lee D S, Jang Y, et al. Real–time concealed–object detection and recognition with passive millimeter wave imaging[J], Optics Express, 2012, 20(9): pp. 9371—-9381,.Google Scholar
  4. 4.
    Meng Y, Qing A, Lin C, et al. Passive millimeter wave imaging system for public security check[C]. Applied Computational Electromagnetics Society Symposium (ACES), 2017: 1–2.Google Scholar
  5. 5.
    Appleby R, Anderton R N. Millimeter–wave and submillimeter–wave imaging for security and surveillance[J]. Proceedings of the IEEE, 2007, 95(8): 1683–1690.Google Scholar
  6. 6.
    Yeom S, Lee D S, Son J Y, et al. Concealed object detection using passive millimeter wave imaging[C]. Universal Communication Symposium. IEEE, 2010:383–386.Google Scholar
  7. 7.
    Tapia S L, Molina R, de la Blanca N P. Detection and localization of objects in Passive Millimeter Wave Images[C]. European Signal Processing Conference, 2016: 2101–2105.Google Scholar
  8. 8.
    Heinz E, May T, Born D, et al. Passive 350 ghz video imaging systems for security applications[J]. Journal of Infrared, Millimeter, and Terahertz Waves, 2015, 36(10): 879–895.Google Scholar
  9. 9.
    Accardo J, Chaudhry M A. Radiation exposure and privacy concerns surrounding full–body scanners in airports[J]. Journal of Radiation Research and Applied Sciences, 2014, 7(2): 198–200.Google Scholar
  10. 10.
    Du K, Zhang L, Chen W, et al. Concealed objects detection based on FWT in active millimeter–wave images[C]. Seventh International Conference on Electronics and Information Engineering, 2017, 10322: 103221O. pp:1–7.Google Scholar
  11. 11.
    Fang G, Gao X, Li C, et al. A novel active millimeter–wave imaging scheme suitable for fast personal screening[C]. International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW–THz), 2013, pp:1–2.Google Scholar
  12. 12.
    Dong S L, Chen X G, Yu W Y, et al. Indoor Passive Millimeter–Wave Imaging for Concealed Object Detection[J]. Advanced Materials Research, 2013, 760–762.Google Scholar
  13. 13.
    Li X, Li S, Zhao G, et al. Multi–polarized millimeter–wave imaging for concealed weapon detection[C]. IEEE International Conference on Microwave and Millimeter Wave Technology (ICMMT), 2016, 2: 892–894.Google Scholar
  14. 14.
    Babacan S D, Luessi M, Spinoulas L, et al. Compressive passive millimeter–wave imaging[C]. IEEE International Conference on Image Processing (ICIP), 2011: 2705–2708.Google Scholar
  15. 15.
    AlSaafin W, Villena S, Vega M, et al. Compressive sensing super resolution from multiple observations with application to passive millimeter wave images[J]. Digital Signal Processing, 2016, 50: 180–190.Google Scholar
  16. 16.
    Kowalski M, Kastek M. Comparative studies of passive imaging in terahertz and mid–wavelength infrared ranges for object detection[J]. IEEE Transactions on Information Forensics and Security, 2016, 11(9): 2028–2035.Google Scholar
  17. 17.
    Mosavi M R, Bisjerdi M H, Rezai–Rad G. Optimal Target–Oriented Fusion of Passive Millimeter Wave Images with Visible Images Based on Contourlet Transform[J]. Wireless Personal Communications, 2017, 95(4): 4643–4666.Google Scholar
  18. 18.
    Kowalski M, Palka N, Piszczek M, et al. Hidden Object Detection System Based on Fusion of THz and VIS Images[J]. Acta Physica Polonica, A., 2013, 124(3):1–4.Google Scholar
  19. 19.
    Gonzalez–Valdes B, Alvarez Y, Mantzavinos S, et al. Improving security screening: A comparison of multistatic radar configurations for human body imaging[J]. IEEE Antennas and Propagation Magazine, 2016, 58(4): 35–47.Google Scholar
  20. 20.
    Farsaei A A, Mokhtari–Koushyar F, Seyed–Talebi S M J, et al. Improved two–dimensional millimeter–wave imaging for concealed weapon detection through partial Fourier sampling[J]. Journal of Infrared, Millimeter, and Terahertz Waves, 2016, 37(3): 267–280.Google Scholar
  21. 21.
    Yu W, Chen X, Wu L. Segmentation of concealed objects in passive millimeter–wave images based on the Gaussian mixture model[J]. Journal of Infrared, Millimeter, and Terahertz Waves, 2015, 36(4): 400–421.Google Scholar
  22. 22.
    Xiao Z, Lu X, Yan J, et al. Automatic detection of concealed pistols using passive millimeter wave imaging[C]. Imaging Systems and Techniques (IST), 2015 IEEE International Conference on. IEEE, 2015: 1–4.Google Scholar
  23. 23.
    López–Tapia S, Molina R, de la Blanca N P. Deep CNNs for Object Detection using Passive Millimeter Sensors[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2017: 1–11.Google Scholar
  24. 24.
    Yang J, Wright J, Huang T S, et al. Image super–resolution via sparse representation[J]. IEEE transactions on image processing, 2010, 19(11): 2861–2873.Google Scholar
  25. 25.
    Carreira J, Sminchisescu C. Cpmc: Automatic object segmentation using constrained parametric min cuts[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2011 (7): 1312–1328.Google Scholar
  26. 26.
    Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]. Advances in neural information processing systems. 2012: 1097–1105.Google Scholar
  27. 27.
    Peng C, Zhang X, Yu G, et al. Large kernel matters——improve semantic segmentation by global convolutional network[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 1743–1751.Google Scholar
  28. 28.
    Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation[C]. Proceedings of the IEEE international conference on computer vision. 2015: 1520–1528.Google Scholar
  29. 29.
    Farabet C, Couprie C, Najman L, et al. Learning hierarchical features for scene labeling[J]. IEEE transactions on pattern analysis and machine intelligence, 2013, 35(8): 1915–1929.Google Scholar
  30. 30.
    Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 3431–3440.Google Scholar
  31. 31.
    Badrinarayanan V, Kendall A, Cipolla R. SegNet: A Deep Convolutional Encoder–Decoder Architecture for Image Segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017 (12): 2481–2495.Google Scholar
  32. 32.
    Zheng S, Jayasumana S, Romera–Paredes B, et al. Conditional random fields as recurrent neural networks[C]. Proceedings of the IEEE international conference on computer vision. 2015: 1529– 1537.Google Scholar
  33. 33.
    Chen L C, Papandreou G, Kokkinos I, et al. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs[J], Computer Science, 2014: pp. 357–361.Google Scholar
  34. 34.
    Ronneberger O, Fischer P, Brox T. U–net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical image computing and computer–assisted intervention. Springer, Cham, 2015: 234–241.Google Scholar
  35. 35.
    Ciresan D, Giusti A, Gambardella L M, et al. Deep neural networks segment neuronal membranes in electron microscopy images[C]. Advances in neural information processing systems. 2012: 2843–2851.Google Scholar
  36. 36.
    Zhang Y, Li Y, Chen J. Detection of concealed objects in passive millimeter wave imaging based on CS theory[C]. Microwave Conference Proceedings (APMC), 2013 Asia–Pacific. IEEE, 2013: 981–983.Google Scholar
  37. 37.
    Torralba A, Russell B C, Yuen J. Labelme: Online image annotation and applications[J]. Proceedings of the IEEE, 2010, 98(8): 1467–1484.Google Scholar
  38. 38.
    He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770–778.Google Scholar
  39. 39.
    Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017: 2261–2269.Google Scholar
  40. 40.
    Su W, Wang Z. Widening residual skipped network for semantic segmentation[J]. IET Image Processing, 2017, 11(10): 880–887.Google Scholar
  41. 41.
    Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980, 2014.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Automation Science and Electrical EngineeringBeihang UniversityBeijingChina

Personalised recommendations