Handgun Detection in Single-Spectrum Multiple X-ray Views Based on 3D Object Recognition

  • Vladimir RiffoEmail author
  • Ivan Godoy
  • Domingo Mery


In the last years, many computer vision algorithms have been proposed for baggage inspection using X-ray images. In these approaches, the idea is to detect automatically threat objects. In baggage inspection, however, a single view is insufficient because there could be occluded parts or intricate projections that cannot be observed with a single view. In order to avoid a misinterpretation based on a single view, we propose the use of single-spectrum multiple X-ray views. Our approach computes a 3D reconstruction using Space Carving, a method that reconstructs a 3D object from its 2D silhouettes (that have been segmented using Geodesic Active Contours). The detection is performed by analyzing 3D features (obtained from the 3D reconstruction). Instead of dual-energy, that is typically used in baggage inspection to analyze the material of the reconstructed objects, we propose simply to use a single-spectrum X-ray system for the detection of threat objects that can be recognized by analyzing the shape, such as handguns. The approach has been successfully tested on X-ray images of travel-bags that contain handguns. In the evaluation of our method, we have used sequences of X-ray images for the 3D reconstruction of objects inside travel-bags, where each sequence consists of 90 X-ray images, we obtained 0.97 in both recall and precision. We strongly believe that it is possible to design an automated aid for the human inspection task using these computer vision algorithms.


X-ray testing Threat objects detection X-ray baggage security 3D reconstruction applications 



This work was supported in part by DIUDA Grant No. 22277 and No. 22345 from Universidad de Atacama, and in part by Fondecyt Grant No. 1161314 from CONICYT, Chile.


  1. 1.
    Zentai, G.: X-ray imaging for homeland security. In: IEEE International Workshop on Imaging Systems and Techniques (IST 2008), pp. 1–6 (2008)Google Scholar
  2. 2.
    Parliament, E.: Aviation security with a special focus on security scanners. European Parliament Resolution (2010/2154(INI)), pp. 1–10 (2012)Google Scholar
  3. 3.
    Schwaninger, A., Bolfing, A., Halbherr, T., Helman, S., Belyavin, A., Hay, L.: The impact of image based factors and training on threat detection performance in X-ray screening. In: Proceedings of the 3rd International Conference on Research in Air Transportation, ICRAT 2008, pp. 317–324 (2008)Google Scholar
  4. 4.
    von Bastian, C., Schwaninger, A., Michel, S.: Do Multi-View X-ray Systems Improve X-ray Image Interpretation in Airport Security Screening?, vol. 52. GRIN Verlag, Munich (2010)Google Scholar
  5. 5.
    Hemayed, E.E., Farag, A.A.: Object modeling using space carving. In: 2000 International Conference on Image Processing, 2000. Proceedings. Volume 2, IEEE, pp. 760–763 (2000)Google Scholar
  6. 6.
    Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38(3), 199–218 (2000)zbMATHGoogle Scholar
  7. 7.
    Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis. 22(1), 61–79 (1997)zbMATHGoogle Scholar
  8. 8.
    Hättenschwiler, N., Sterchi, Y., Mendes, M., Schwaninger, A.: Automation in airport security X-ray screening of cabin baggage: examining benefits and possible implementations of automated explosives detection. Appl. Ergonom. 72, 58–68 (2018)Google Scholar
  9. 9.
    Blalock, G., Kadiyali, V., Simon, D.H.: The impact of post-9/11 airport security measures on the demand for air travel. J. Law Econ. 50(4), 731–755 (2007)Google Scholar
  10. 10.
    Michel, S., Koller, S., de Ruiter, J., Moerland, R., Hogervorst, M., Schwaninger, A.: Computer-based training increases efficiency in X-ray image interpretation by aviation security screeners. In: 2007 41st Annual IEEE International Carnahan Conference on Security Technology, pp. 201–206 (2007)Google Scholar
  11. 11.
    Mery, D., Svec, E., Arias, M., Riffo, V., Saavedra, J., Banerjee, S.: Modern computer vision techniques for X-ray testing in baggage inspection. IEEE Trans. Syst. Man Cybern. 47, 682 (2016)Google Scholar
  12. 12.
    Mery, D.: Computer Vision for X-ray Testing. Springer, New York (2015)zbMATHGoogle Scholar
  13. 13.
    Riffo, V., Mery, D.: Automated detection of threat objects using adapted implicit shape model. IEEE Trans. Syst. Man Cybern. 46(4), 472–482 (2016)Google Scholar
  14. 14.
    Mery, D., Svec, E., Arias, M.: Object recognition in X-ray testing using adaptive sparse representations. J. Nondestr. Eval. 35(3), 1–19 (2016)Google Scholar
  15. 15.
    Heitz, G., Chechik, G.: Object separation in X-ray image sets. 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2093–2100 (2010)Google Scholar
  16. 16.
    Mery, D., Katsaggelos, A.: A logarithmic X-ray imaging model for baggage inspection: simulation and object detection. In: 13th IEEE CVPR Workshop on Perception Beyond the Visible Spectrum (PBVS 2017) (2017)Google Scholar
  17. 17.
    Uroukov, I., Speller, R.: A preliminary approach to intelligent X-ray imaging for baggage inspection at airports. Signal Process. Res. 4, 1–11 (2015)Google Scholar
  18. 18.
    Turcsany, D., Mouton, A., Breckon, T.P.: Improving feature-based object recognition for X-ray baggage security screening using primed visualwords. In: IEEE International Conference on Industrial Technology (ICIT 2013), pp. 1140–1145 (2013)Google Scholar
  19. 19.
    Baştan, M., Yousefi, M.R., Breuel, T.M.: Visual words on baggage X-ray images. In: Computer Analysis of Images and Patterns, pp. 360–368. Springer (2011)Google Scholar
  20. 20.
    Kundegorski, M.E., Akçay, S., Devereux, M., Mouton, A., Breckon, T.P.: On using feature descriptors as visual words for object detection within X-ray baggage security screening. In: International Conference on Imaging for Crime Detection and Prevention (IET2016) (2016)Google Scholar
  21. 21.
    Zhang, N., Zhu, J.: A study of X-ray machine image local semantic features extraction model based on bag-of-words for airport security. Int. J. Smart Sens. Intell. Syst. 8(1), 45–64 (2015)Google Scholar
  22. 22.
    Abidi, B.R., Zheng, Y., Gribok, A.V., Abidi, M.A.: Improving weapon detection in single energy X-ray images through pseudocoloring. IEEE Trans. Syst. Man Cybern. C 36(6), 784–796 (2006)Google Scholar
  23. 23.
    Mery, D.: Inspection of complex objects using multiple-X-ray views. IEEE/ASME Trans. Mechatron. 20(1), 338–347 (2015)Google Scholar
  24. 24.
    Mery, D., Riffo, V., Zuccar, I., Pieringer, C.: Automated X-ray object recognition using an efficient search algorithm in multiple views. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE Computer Society, pp. 368–374 (2013)Google Scholar
  25. 25.
    Riffo, V., Mery, D.: Active X-ray testing of complex objects. Insight-Non-Destr. Test. Cond. Monit. 54(1), 28–35 (2012)Google Scholar
  26. 26.
    Riffo, V., Flores, S., Mery, D.: Threat objects detection in X-ray images using an active vision approach. J. Nondestr. Eval. 36(3), 44 (2017)Google Scholar
  27. 27.
    Baştan, M., Byeon, W., Breuel, T.M.: Object recognition in multi-view dual X-ray images. In: British Machine Vision Conference BMVC (2013)Google Scholar
  28. 28.
    Baştan, M.: Multi-view object detection in dual-energy X-ray images. Mach. Vis. Appl. 26(7–8), 1045–1060 (2015)Google Scholar
  29. 29.
    Franzel, T., Schmidt, U., Roth, S.: Object detection in multi-view X-ray images. Pattern Recognition (2012)Google Scholar
  30. 30.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)Google Scholar
  31. 31.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)Google Scholar
  32. 32.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR arXiv:1512.03385 (2015)
  33. 33.
    Akçay, S., Kundegorski, M.E., Devereux, M., Breckon, T.P.: Transfer learning using convolutional neural networks for object classification within X-ray baggage security imagery. In: 2016 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 1057–1061 (2016)Google Scholar
  34. 34.
    Akcay, S., Kundegorski, M.E., Willcocks, C.G., Breckon, T.P.: Using deep convolutional neural network architectures for object classification and detection within X-ray baggage security imagery. IEEE Trans. Inf. Forensics Secur. 13(9), 2203–2215 (2018)Google Scholar
  35. 35.
    Akcay, S., Breckon, T.P.: An evaluation of region based object detection strategies within X-ray baggage security imagery. In: 2017 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 1337–1341 (2017)Google Scholar
  36. 36.
    Poggio, T., Edelman, S.: A network that learns to recognize 3D objects. Nature (1990)Google Scholar
  37. 37.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)Google Scholar
  38. 38.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)Google Scholar
  39. 39.
    Bay, H.: SURF: speeded up robust features. In: ECCV’06: Proceedings of the 9th European conference on Computer Vision, Berlin, Heidelberg, Springer-Verlag, pp. 404–417 (2006)Google Scholar
  40. 40.
    Shen, L., Jiang, C.J., Liu, G.J.: Satellite objects extraction and classification based on similarity measure. IEEE Trans. Syst. Man Cybern. 46(8), 1148–1154 (2016)Google Scholar
  41. 41.
    Sivic, J., Zisserman, A.: Efficient visual search of videos cast as text retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 31(4), 591–606 (2009)Google Scholar
  42. 42.
    Leibe, B., Leonardis, A., Schiele, B.: Combined object categorization and segmentation with an implicit shape model. In: Workshop on Statistical Learning in Computer Vision, ECCV (2004)Google Scholar
  43. 43.
    Boureau, Y.L., Bach, F., LeCun, Y., Ponce, J.: Learning mid-level features for recognition. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 2559–2566 (2010)Google Scholar
  44. 44.
    Tošić, I., Frossard, P.: Dictionary learning. Signal Process. Mag. IEEE 28(2), 27–38 (2011)zbMATHGoogle Scholar
  45. 45.
    Bo, L., Lai, K., Ren, X., Fox, D.: Object recognition with hierarchical kernel descriptors. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 1729–1736 (2011)Google Scholar
  46. 46.
    Simonyan, K., Parkhi, O.M., Vedaldi, A., Zisserman, A.: Fisher vector faces in the wild. In: BMVC, p. 4 (2013)Google Scholar
  47. 47.
    Arandjelovic, R., Zisserman, A.: All about vlad. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1578–1585 (2013)Google Scholar
  48. 48.
    Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T., Yan, S.: Sparse representation for computer vision and pattern recognition. Proc. IEEE 98(6), 1031–1044 (2010)Google Scholar
  49. 49.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 1794–1801 (2009)Google Scholar
  50. 50.
    Thomas, A., Van Gool, L., Tuytelaars, T., Ferrari, V., Leibe, B., Schiele, B.: Towards multi-view object class detection. In: CVPR ’06: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society (2006)Google Scholar
  51. 51.
    Su, H., Savarese, S., Sun, M., Fei-Fei, L.: Learning a dense multi-view representation for detection, viewpoint classification and synthesis of object categories. In: 2009 IEEE 12th International Conference on Computer Vision, IEEE, pp. 213–220 (2009)Google Scholar
  52. 52.
    Ferrari, V., Tuytelaars, T., Van Gool, L.: Simultaneous object recognition and segmentation from single or multiple model views. Int. J. Comput. Vis. 67(2), 159–188 (2006)Google Scholar
  53. 53.
    Savarese, S., Fei-Fei, L.: 3D generic object categorization, localization and pose estimation. In: IEEE 11th International Conference on Computer Vision, 2007. ICCV 2007, IEEE, pp. 1–8 (2007)Google Scholar
  54. 54.
    Liebelt, J., Schmid, C.: Multi-view object class detection with a 3D geometric model. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1688–1695 (2010)Google Scholar
  55. 55.
    Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., Van Gool, L.: Robust tracking-by-detection using a detector confidence particle filter. In: 2009 IEEE 12th International Conference on Computer Vision, IEEE, pp. 1515–1522 (2009)Google Scholar
  56. 56.
    Mouton, A., Flitton, G.T., Bizot, S.: An evaluation of image denoising techniques applied to CT baggage screening imagery. In: IEEE International Conference on Industrial Technology (ICIT 2013), IEEE (2013)Google Scholar
  57. 57.
    Mouton, A., Breckon, T.P.: A review of automated image understanding within 3D baggage computed tomography security screening. J. X-ray Sci. Technol. 23(5), 531–555 (2015)Google Scholar
  58. 58.
    Flitton, G., Breckon, T.P., Megherbi, N.: A comparison of 3D interest point descriptors with application to airport baggage object detection in complex CT imagery. Pattern Recogn. 46(9), 2420–2436 (2013)Google Scholar
  59. 59.
    Megherbi, N., Han, J., Breckon, T.P., Flitton, G.T.: A comparison of classification approaches for threat detection in CT based baggage screening. In: 2012 19th IEEE International Conference on Image Processing (ICIP), IEEE, pp. 3109–3112 (2012)Google Scholar
  60. 60.
    Flitton, G., Mouton, A., Breckon, T.P.: Object classification in 3D baggage security computed tomography imagery using visual codebooks. Pattern Recogn. 48(8), 2489–2499 (2015)Google Scholar
  61. 61.
    Mouton, A., Breckon, T.P.: Materials-based 3D segmentation of unknown objects from dual-energy computed tomography imagery in baggage security screening. Pattern Recogn. 48(6), 1961–1978 (2015)Google Scholar
  62. 62.
    Pears, N., Liu, Y., Bunting, P.: 3D Imaging Analysis and Applications. Springer, London (2012)Google Scholar
  63. 63.
    Li, X., Iyengar, S.S.: On computing mapping of 3D objects. ACM Comput. Surv. 47(2), 1–45 (2015)Google Scholar
  64. 64.
    Yubin, Y., Hui, L., Yao, Z.: Content-based 3D model retrieval: a survey. IEEE Trans. Syst. Man Cybern. C 37(6), 1081–1098 (2007)Google Scholar
  65. 65.
    Tangelder, T., Veltkamp, R.: A survey of content based 3D shape retrieval methods. Multimed. Tools Appl. 39, 441–471 (2008)Google Scholar
  66. 66.
    SHREC, H.: Shape Retrieval Contest (SHREC) Home Page. (Visited in year 2018)
  67. 67.
    Sipiran, I., Bustos, B.: A robust 3d interest points detector based on harris operator. In: Proceedings of Eurographics Workshop on 3D Object Retrieval (2010), Eurographics Association, pp. 7–14 (2010)Google Scholar
  68. 68.
    Zaharescu, E. Boyer, K.V., Horaud, R.: Surface feature detection and description with applications to mesh matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009) (2009)Google Scholar
  69. 69.
    Johnson, J.: Spin-images: a representation for 3D surface matching. Ph.D. thesis, Carnegie Mellon University (1997)Google Scholar
  70. 70.
    Heikkilä, M., Pietikäinen, M., Schmid, C.: Description of interest regions with center-symmetric local binary patterns. In: Computer Vision, Graphics and Image Processing, pp. 58–69. Springer (2006)Google Scholar
  71. 71.
    Frome, A., Huber, D., Kolluri, R., Bülow, T., Malik, J.: Recognizing objects in range data using regional point descriptors. In: European Conference on Computer Vision, pp. 224–237. Springer (2004)Google Scholar
  72. 72.
    Tombari, F., Salti, S., Stefano, L.D.: Unique signatures of histograms for local surface description. Comput. Vis.—ECCV 2010(6313), 356–369 (2009)Google Scholar
  73. 73.
    Tombari, F., Salti, S., Di Stefano, L.: Unique shape context for 3d data description. In: Proceedings of the ACM workshop on 3D object retrieval, ACM, pp. 57–62 (2010)Google Scholar
  74. 74.
    Rusu, R.B., Blodow, N., Beetz, M.: Fast point feature histograms (fpfh) for 3d registration. IEEE International Conference on Robotics and Automation, 2009. ICRA ’09 (2009)Google Scholar
  75. 75.
    Kutulakos, K., Seitz, S.: A theory of shape by space carving. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999 (1999)Google Scholar
  76. 76.
    Potmesil, M.: Generating octree models of 3D objects from their silhouettes in a sequence of images. Comput. Vis. Graph. Image Process. 40(1), 1–29 (1987)Google Scholar
  77. 77.
    Mery, D.: Explicit geometric model of a radioscopic imaging system. NDT E Int. 36(8), 587–599 (2003)Google Scholar
  78. 78.
    Zhong, Y.: Intrinsic shape signatures: A shape descriptor for 3d object recognition. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 689–696 (2009)Google Scholar
  79. 79.
    Mian, A., Bennamoun, M., Owens, R.: On the repeatability and quality of keypoints for local feature-based 3D object retrieval from cluttered scenes. Int. J. Comput. Vis. 89(2), 348–361 (2010)Google Scholar
  80. 80.
    Aldoma, A., Marton, Z.C., Tombari, F., Wohlkinger, W., Potthast, C., Zeisl, B., Rusu, R.B., Gedikli, S., Vincze, M.: Tutorial: point cloud library: three-dimensional object recognition and 6 dof pose estimation. IEEE Robot. Autom. Mag. 19(3), 80–91 (2012)Google Scholar
  81. 81.
    Chen, H., Bhanu, B.: 3d free-form object recognition in range images using local surface patches. Pattern Recogn. Lett. 28(10), 1252–1262 (2007)Google Scholar
  82. 82.
    Mery, D., Riffo, V., Zscherpel, U., Mondragón, G., Lillo, I., Zuccar, I., Lobel, H., Carrasco, M.: GDXray: the database of X-ray images for nondestructive testing. J. Nondestr. Eval. 34(4), 1–12 (2015)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Departamento de Ingeniería Informática y Ciencias de la ComputaciónUniversidad de AtacamaCopiapóChile
  2. 2.Department of Computer SciencePontificia Universidad Católica de ChileSantiagoChile

Personalised recommendations