Advertisement

Hyperspectral Imaging for Real-Time Unmanned Aerial Vehicle Maritime Target Detection

  • Sara Freitas
  • Hugo Silva
  • José Almeida
  • Eduardo Silva
Open Access
Article

Abstract

This work address hyperspectral imaging systems use for maritime target detection using unmanned aerial vehicles. Specifically, by working in the creation of a hyperspectral real-time data processing system pipeline. We develop a boresight calibration method that allows to calibrate the position of the navigation sensor related to the camera imaging sensor, and improve substantially the accuracy of the target geo-reference. We also develop an unsupervised method for segmenting targets (boats) from their dominant background in real-time. We evaluated the performance of our proposed system for target detection in real-time with UAV flight data and present detection results comparing favorably our approach against other state-of- the-art method.

Keywords

Unmanned aerial vehicles Hyperspectral imaging Surveillance Target detection Computer vision 

Notes

Acknowledgments

The authors would like to thank the Portuguese Air Force and the Portuguese Navy for providing the UAV and the boats for the data collection. The authors would also like to thank SPECIM and XENICS for providing the Hyperspectral camera and the IR camera for the payload used in this dataset campaign. This work is financed by the ERDF - European Regional Development Fund through the Operational Program for Competitiveness and Internationalization - COMPETE 2020 Program within project POCI-01-0145-FEDER-006961, by the SUNNY Project funded by the European Commission under the FP7 program Grant Agreement number: 313243, and by National Funds through the FCT - Fundacao para a Ciencia e a Tecnologia (Portuguese Foundation for Science and Technology) as part of project UID/EEA/50014/2013.

References

  1. 1.
    ACORN: Atmospheric correction now. Analytical imaging and geophysics LLC version 3.12Google Scholar
  2. 2.
    Adler-Golder, S., Berk, A., Bernstein, L.S., Richtsmeier, S., Acharya, P., Matthew, M., Anderson, G., Allred, C.L., Jeong, L.S., Chetwynd, J.: Flaash, a modtran4 atmospheric correction package for hyper-spectral data retrievals and simulations. In: Summaries of the Seventh JPL Airborne Earth Science Workshop (1998)Google Scholar
  3. 3.
    Bachar, A., Polerecky, L., Fischer, J.P., Vamvakopoulos, K., Beer, D.D., Jonkers, H.M.: Two-dimensional mapping of photopigment distribution and activity of chloroflexus-like bacteria in a hypersaline microbial mat. FEMS Microbiol. Ecol. 65(3), 434–448 (2008).  https://doi.org/10.1111/j.1574-6941.2008.00534.x CrossRefGoogle Scholar
  4. 4.
    Barbieux, K., Constantin, D., Merminod, B.: Correction of airborne pushbroom images orientation using bundle adjustment of frame images. In: ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLI-B3, pp. 813–818 (2016).  https://doi.org/10.5194/isprs-archives-XLI-B3-813-2016
  5. 5.
    Barott, K., Smith, J., Dinsdale, E., Hatay, M., Sandin, S., Rohwer, F.: Hyperspectral and physiological analyses of coral-algal interactions. PLoS ONE 4(11), e8043 (2009).  https://doi.org/10.1371/journal.pone.0008043 CrossRefGoogle Scholar
  6. 6.
    Ben-Dor, E.: Quality assessment of several methods to recover surface reflectance using synthetic imaging spectroscopy data. Remote Sens. Environ. 90(3), 389–404 (2004).  https://doi.org/10.1016/j.rse.2004.01.014 CrossRefGoogle Scholar
  7. 7.
    Berk, A., Conforti, P., Kennett, R., Perkins, T., Hawes, F., van den Bosch, J.: MODTRAN6: a major upgrade of the MODTRAN radiative transfer code. In: Velez-Reyes, M., Kruse, F.A. (eds.) Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XX. SPIE.  https://doi.org/10.1117/12.2050433 (2014)
  8. 8.
    Bernstein, L.S.: Quick atmospheric correction code: algorithm description and recent upgrades. Opt. Eng. 51 (11), 111,719 (2012).  https://doi.org/10.1117/1.oe.51.11.111719 CrossRefGoogle Scholar
  9. 9.
    Bethel, J.S., Lee, C., Landgrebe, D.A.: Geometric registration and classification of hyperspectral airborne pushbroom data. Int Arch. Photogramm. Remote Sens. XXXIII(B7) (2000)Google Scholar
  10. 10.
    Chang, C.I., Chiang, S.S.: Anomaly detection and classification for hyperspectral imagery. IEEE Trans. Geosci. Remote Sens. 40(6), 1314–1325 (2002).  https://doi.org/10.1109/tgrs.2002.800280 CrossRefGoogle Scholar
  11. 11.
    Conel, J.E., Green, R.O., Vane, G., Bruegge, C.J., Alley, R.E., Curtiss, B.J.: Airborne imaging spectrometer-2: Radiometric spectral characteristics and comparison of ways to compensate for the atmosphere. In: Vane, G. (ed.) Imaging Spectroscopy II. SPIE (1987).  https://doi.org/10.1117/12.942294
  12. 12.
    Cramer, M., Stallmann, D., Haala, N.: Direct georeferencing using gps/inertial exterior orientations for photogrammetric applications. In: International Archives of Photogrammetry and Remote Sensing, vol. 33, pp. 198–205 (2000)Google Scholar
  13. 13.
    Doneus, M., Verhoeven, G., Atzberger, C., Wess, M., Ruš, M.: New ways to extract archaeological information from hyperspectral pixels. J. Archaeol. Sci. 52, 84–96 (2014).  https://doi.org/10.1016/j.jas.2014.08.023 CrossRefGoogle Scholar
  14. 14.
    Gao, B.C., Heidebrecht, K.B., Goetz, A.F.: Derivation of scaled surface reflectances from AVIRIS data. Remote Sens. Environ. 44(2-3), 165–178 (1993).  https://doi.org/10.1016/0034-4257(93)90014-o CrossRefGoogle Scholar
  15. 15.
    Goetz, A.F.H., Srivastava, V.: Mineralogical mapping in the cuprite mining district, nevada. In: Proceedings of the Airborne Imaging Spectrometer Data (1985)Google Scholar
  16. 16.
    Haala, N., Stallmann, D., Cramer, M.: Calibration of directly measured position and attitude by aerotriangulation of three-line airborne imagery (1998)Google Scholar
  17. 17.
    Haavardsholm, T.V., Arisholm, G., Kavara, A., Skauli, T.: Architecture of the real-time target detection processing in an airborne hyperspectral demonstrator system. In: 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing. IEEE (2010).  https://doi.org/10.1109/whispers.2010.5594940
  18. 18.
    Holzwarth, S., Müller, R., Simon, C.: Determination and monitoring of boresight misalignment angles during the hymap campaigns hyeurope 2003 and hyeurope (2004)Google Scholar
  19. 19.
    Keith, C.J., Repasky, K.S., Lawrence, R.L., Jay, S.C., Carlsten, J.L.: Monitoring effects of a controlled subsurface carbon dioxide release on vegetation using a hyperspectral imager. Int. J. Greenh. Gas Control 3(5), 626–632 (2009).  https://doi.org/10.1016/j.ijggc.2009.03.003 CrossRefGoogle Scholar
  20. 20.
    Knauer, U., Seiffert, U.: An approach for automated registration of hyperspectral images for boresight calibration. DGPF Tagungsband (2013)Google Scholar
  21. 21.
    Kruse, F., Lefkoff, A., Boardman, J., Heidebrecht, K., Shapiro, A., Barloon, P., Goetz, A.: The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer data. Remote Sens. Environ. 44(2-3), 145–163 (1993).  https://doi.org/10.1016/0034-4257(93)90013-n CrossRefGoogle Scholar
  22. 22.
    Kühl, M., Polerecky, L.: Functional and structural imaging of phototrophic microbial communities and symbioses. Aquat. Microb. Ecol. 53, 99–118 (2008).  https://doi.org/10.3354/ame01224 CrossRefGoogle Scholar
  23. 23.
    Lenz, A., Schilling, H., Perpeet, D., Wuttke, S., Gross, W., Middelmann, W.: Automatic in-flight boresight calibration considering topography for hyperspectral pushbroom sensors. In: 2014 IEEE Geoscience and Remote Sensing Symposium. IEEE (2014).  https://doi.org/10.1109/igarss.2014.6947103
  24. 24.
    Manolakis, D., Lockwood, R., Cooley, T., Jacobson, J.: Is there a best hyperspectral detection algorithm? In: Shen, S.S., Lewis, P.E. (eds.) Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XV. SPIE (2009).  https://doi.org/10.1117/12.816917
  25. 25.
    Manolakis, D.G., Ingle, V.K., Kogon, S.M.: Statistical and Adaptive Signal Processing: Spectral Estimation, Signal Modeling, Adaptive Filtering, and Array Processing Artech House (2005)Google Scholar
  26. 26.
    Mostafa, M.M., Schwarz, K.P.: A multi-sensor system for airborne image capture and georeferencing. Photogramm. Eng. Remote Sens. 66(12), 1417–1423 (2000)Google Scholar
  27. 27.
    Müller, R., Lehner, M., Müller, R., Reinartz, P., Schroeder, M., Vollmer, B.: A program for direct georeferencing of airborne and spaceborne line scanner images. In: Petri G, Kennie TJM (eds.) ISPRS, Commission I, WG I/5. Information available at: http://www.isprs.org/commission1/proceedings02/paper/00072.pdf. Accessed 23 Jun 2016, pp. 1–3 (2002)
  28. 28.
    Nansen, C., Zhao, G., Dakin, N., Zhao, C., Turner, S.R.: Using hyperspectral imaging to determine germination of native australian plant seeds. J. Photochem. Photobiol. B: Biol. 145, 19–24 (2015).  https://doi.org/10.1016/j.jphotobiol.2015.02.015 CrossRefGoogle Scholar
  29. 29.
    O’Neill, N.T., Royer, A., Nguyn, M.N.: Canadian advanced modified 5s (cam5s). Internal Report, CARTEL-1996-0202 Centre d’applications et de recherches en teledetection (CARTELl). Universitie de Sherbrooke, Sherbrooke (1996)Google Scholar
  30. 30.
    Plaza, A., Benediktsson, J.A., Boardman, J.W., Brazile, J., Bruzzone, L., Camps-Valls, G., Chanussot, J., Fauvel, M., Gamba, P., Gualtieri, A., Marconcini, M., Tilton, J.C., Trianni, G.: Recent advances in techniques for hyperspectral image processing. Remote Sens. Environ. 113, S110–S122 (2009).  https://doi.org/10.1016/j.rse.2007.07.028 CrossRefGoogle Scholar
  31. 31.
    Polerecky, L., Bissett, A., Al-Najjar, M., Faerber, P., Osmers, H., Suci, P.A., Stoodley, P., de Beer, D.: Modular spectral imaging system for discrimination of pigments in cells and microbial communities. Appl. Environ. Microbiol. 75(3), 758–771 (2008).  https://doi.org/10.1128/aem.00819-08 CrossRefGoogle Scholar
  32. 32.
    Puckrin, E., Turcotte, C.S., Gagnon, M.A., Bastedo, J., Farley, V., Chamberland, M.: Airborne infrared hyperspectral imager for intelligence, surveillance, and reconnaissance applications. In: Airborne Intelligence, Surveillance, Reconnaissance (ISR) Systems and Applications IX. SPIE (2012).  https://doi.org/10.1117/12.918251
  33. 33.
    Qu, Z., Kindel, B., Goetz, A.: The high accuracy atmospheric correction for hyperspectral data (hatch) model. IEEE Trans. Geosci. Remote Sens. 41(6), 1223–1231 (2003).  https://doi.org/10.1109/tgrs.2003.813125 CrossRefGoogle Scholar
  34. 34.
    Richter, R.: A spatially adaptive fast atmospheric correction algorithm. Int. J. Remote Sens. 17(6), 1201–1214 (1996).  https://doi.org/10.1080/01431169608949077 CrossRefGoogle Scholar
  35. 35.
    Roberts, D.A., Yamaguchi, Y., Lyon, R.J.P.: Comparison of various techniques for calibration of ais data. In: Proceedings of the 2nd AIS Workshop (1986)Google Scholar
  36. 36.
    Rzhanov, Y., Pe’eri, S.: Pushbroom-frame imagery co-registration. Mar. Geodesy 35(2), 141–157 (2012).  https://doi.org/10.1080/01490419.2011.634964 CrossRefGoogle Scholar
  37. 37.
    Schaum, A.: Hyperspectral detection algorithms: from old ideas to operational concepts to next generation. In: Shen, S.S., Lewis, P.E. (eds.) Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XII. SPIE (2006).  https://doi.org/10.1117/12.665754
  38. 38.
    Silva, H., Almeida, J.M., Lopes, F., Ribeiro, J., Freitas, S., Amaral, G., Almeida, C., Martins, A., Silva, E.: Uav trials for multi-spectral imaging target detection and recognition in maritime environment. In: OCEANS 2016 MTS/IEEE Monterey. IEEE (2016).  https://doi.org/10.1109/oceans.2016.7761259
  39. 39.
    Slabaugh, G.G.: Computing euler angles from a rotation matrixGoogle Scholar
  40. 40.
    Spangler, L.H., Dobeck, L.M., Repasky, K.S., Nehrir, A.R., Humphries, S.D., Barr, J.L., Keith, C.J., Shaw, J.A., Rouse, J.H., Cunningham, A.B., Benson, S.M., Oldenburg, C.M., Lewicki, J.L., Wells, A.W., Diehl, J.R., Strazisar, B.R., Fessenden, J.E., Rahn, T.A., Amonette, J.E., Barr, J.L., Pickles, W.L., Jacobson, J.D., Silver, E.A., Male, E.J., Rauch, H.W., Gullickson, K.S., Trautz, R., Kharaka, Y., Birkholzer, J., Wielopolski, L.: A shallow subsurface controlled release facility in bozeman, montana, USA, for testing near surface CO2 detection techniques and transport models. Environ. Earth Sci. 60(2), 227–239 (2009).  https://doi.org/10.1007/s12665-009-0400-2 CrossRefGoogle Scholar
  41. 41.
    Stein, D., Schoonmaker, J., Coolbaugh, E.: Hyperspectral imaging for intelligence, surveillance and reconnaissance (2001)Google Scholar
  42. 42.
    Taitano, Y., Geier, B., Bauer, K.: A locally adaptable iterative RX detector. EURASIP J. Adv. Signal Process. 2010(1), 341,908 (2010).  https://doi.org/10.1155/2010/341908 CrossRefGoogle Scholar
  43. 43.
    Tarabalka, Y., Haavardsholm, T.V., Kaasen, I., Skauli, T.: Real-time anomaly detection in hyperspectral images using multivariate normal mixture models and GPU processing. Journal of Real-Time Image Processing 4 (3), 287–300 (2008).  https://doi.org/10.1007/s11554-008-0105-x CrossRefGoogle Scholar
  44. 44.
    Toth, C.K., Oh, J., Grejner-Brzezinska, D.A.: Airborne hyperspectral image georeferencing aided by high-resolution satellite images. ISPRS TC VII Symposium (2010)Google Scholar
  45. 45.
    Tuo, H., Liu, Y.: A new coarse-to-fine rectification algorithm for airborne push-broom hyperspectral images. Pattern Recogn. Lett. 26(11), 1782–1791 (2005).  https://doi.org/10.1016/j.patrec.2005.02.005 CrossRefGoogle Scholar
  46. 46.
    Zhang, A., Hu, S., Meng, X., Yang, L., Li, H.: Toward high altitude airship ground-based boresight calibration of hyperspectral pushbroom imaging sensors. Remote Sens. 7(12), 17,297–17,311 (2015).  https://doi.org/10.3390/rs71215883 CrossRefGoogle Scholar
  47. 47.
    Zhang, X., Nansen, C., Aryamanesh, N., Yan, G., Boussaid, F.: Importance of spatial and spectral data reduction in the detection of internal defects in food products. Appl. Spectrosc. 69(4), 473–480 (2015).  https://doi.org/10.1366/14-07672 CrossRefGoogle Scholar

Copyright information

© The Author(s) 2017

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  • Sara Freitas
    • 1
  • Hugo Silva
    • 1
  • José Almeida
    • 2
  • Eduardo Silva
    • 2
  1. 1.INESC TECPortoPortugal
  2. 2.INESC TEC, Instituto Superior de Engenharia do PortoPortoPortugal

Personalised recommendations