Skip to main content
SpringerLink
Log in

Stabilization of Airborne Video Using Sensor Exterior Orientation with Analytical Homography Modeling

  • Chapter
  • First Online:
Machine Vision and Navigation

Abstract

Aerial video captured from an airborne platform has an expanding range of applications including scene understanding, photogrammetry, surveying and mapping, traffic monitoring, bridge and civil infrastructure inspection, architecture and construction, delivery, disaster and emergency response, news and film, precision agriculture, and environmental monitoring and conservation. Some of the challenges in analyzing aerial video to track pedestrians, vehicles, and objects include small object size, relative motion of the object and platform, sensor jitter, and quality of imaging optics. An analytic image stabilization approach is described in this chapter where pixel information from the focal plane of the camera is stabilized and georegistered in a global reference frame. The aerial video is stabilized to maintain a fixed relative displacement between the moving platform and the scene. The proposed algorithm can be used to stabilize aerial imagery even when the available GPS and IMU measurements from the platform and sensor are inaccurate and noisy. Camera 3D poses are optimized using a homography-based robust cost function, but unlike most existing methods, the homography transformations are estimated without using any image-to-image estimation techniques. We derive a direct closed-form analytic expression from 3D camera poses that is robust even in the presence of significant scene parallax (i.e. very tall 3D buildings and man-made or natural structures). A robust non-linear least squares cost function is used to deal with outliers and speeds up computation by avoiding the use of RANdom SAmple Consensus (RANSAC). The proposed method and its efficiency is validated using several datasets and scenarios including DARPA Video and Image Retrieval and Analysis Tool (VIRAT) and high resolution wide area motion imagery (WAMI). scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 299.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Palaniappan, K., Rao, R., & Seetharaman, G. (2011). Wide-area persistent airborne video: Architecture and challenges. In Distributed video sensor networks: Research challenges and future directions (pp. 349–371). London: Springer.

    Chapter  Google Scholar 

  2. Porter, R., Fraser, A. M., & Hush, D. (2010). Wide-area motion imagery. IEEE Signal Processing Magazine, 27(5), 56–65.

    Article  Google Scholar 

  3. Poostchi, M., Aliakbarpour, H., Viguier, R., Bunyak, F., Palaniappan, K., & Seetharaman, G. (2016). Semantic depth map fusion for moving vehicle detection in aerial video. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (pp. 1575–1583).

    Google Scholar 

  4. Palaniappan, K., Poostchi, M., Aliakbarpour, H., Viguier, R., Fraser, J., Bunyak, F., et al. (2016). Moving object detection for vehicle tracking in wide area motion imagery using 4D filtering. In IEEE International Conference on Pattern Recognition (ICPR).

    Google Scholar 

  5. Nilosek, D. R., Walvoord, D. J., & Salvaggio, C. (2014). Assessing geoaccuracy of structure from motion point clouds from long-range image collections. Optical Engineering, 53(11), 1–10.

    Article  Google Scholar 

  6. Linger, M. E., & Goshtasby, A. (2015). Aerial image registration for tracking. IEEE Transactions on Geoscience and Remote Sensing, 53(4), 2137–2145.

    Article  Google Scholar 

  7. Holtkamp, D. J., & Goshtasby, A. A. (2009). Precision registration and mosaicking of multicamera images. IEEE Transactions on Geoscience and Remote Sensing, 47, 3446–3455.

    Article  Google Scholar 

  8. Lee, J., Cai, X., Schonlieb, C.-B., & Coomes, D. A. (2015). Nonparametric image registration of airborne LiDAR, hyperspectral and photographic imagery of wooded landscapes. IEEE Transactions on Geoscience and Remote Sensing, PP(99), 1–12.

    Google Scholar 

  9. Crispell, D., Mundy, J. L., & Taubin, G. (2008). Parallax-free registration of aerial video. In Proceedings of the British Machine Vision Conference (pp. 1–4).

    Google Scholar 

  10. Aliakbarpour, H., Palaniappan, K., & Seetharaman, G. (2015). Fast structure from motion for sequential and wide area motion imagery. In The IEEE International Conference on Computer Vision (ICCV) Workshops.

    Google Scholar 

  11. Aliakbarpour, H., Palaniappan, K., & Seetharaman, G. (2015). Robust camera pose refinement and rapid SfM for multiview aerial imagery - without RANSAC. IEEE Geoscience and Remote Sensing Letters, 12(11), 2203–2207.

    Article  Google Scholar 

  12. Aliakbarpour, H., Palaniappan, K., & Seetharaman, G. (2017). Parallax-tolerant aerial image georegistration and efficient camera pose refinement- without piecewise homographies. IEEE Transactions on Geoscience and Remote Sensing, 55(8), 4618–4637.

    Article  Google Scholar 

  13. Aliakbarpour, H., Prasath, V. B. S., Palaniappan, K., Seetharaman, G., & Dias, J. (2016). Heterogeneous multi-view information fusion: Review of 3-D reconstruction methods and a new registration with uncertainty modeling. IEEE Access, 4, 8264–8285.

    Article  Google Scholar 

  14. Aliakbarpour, H., & Dias, J. (2012). Three-dimensional reconstruction based on multiple virtual planes by using fusion-based camera network. IET Computer Vision, 6(4), 355.

    Article  MathSciNet  Google Scholar 

  15. Aliakbarpour, H., Almeida, L., Menezes, P., & Dias, J. (2011). Multi-sensor 3D volumetric reconstruction using CUDA. 3D Research, 2(4), 1–14.

    Article  Google Scholar 

  16. Aliakbarpour, H., & Dias, J. (2011). PhD forum: Volumetric 3D reconstruction without planar ground assumption. In Fifth ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC) (pp. 1–2). Piscataway: IEEE.

    Google Scholar 

  17. Molina, E., & Zhu, Z. (2014). Persistent aerial video registration and fast multi-view mosaicing. IEEE Transactions on Image Processing, 23(5), 2184–2192.

    Article  MathSciNet  Google Scholar 

  18. Saleemi, I., & Shah, M. (2013). Multiframe many-many point correspondence for vehicle tracking in high density wide area aerial videos. International Journal of Computer Vision, 104(2), 198–219.

    Article  Google Scholar 

  19. Medioni, G., Cohen, I., Bremond, F., Hongeng, S. & Nevatia, R. (2001). Event detection and analysis from video streams. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(8), 873–889.

    Article  Google Scholar 

  20. Lin, Y., Yu, Q., & Medioni, G. (2010). Efficient detection and tracking of moving objects in geo-coordinates. Machine Vision and Applications, 22, 505–520.

    Google Scholar 

  21. Chellappa, V. M., & Govindu, R. (2011). Feature-based image to image registration. In Image Registration for Remote Sensing (pp. 215–239). Cambridge: Cambridge University Press.

    Google Scholar 

  22. Taylor, C. N. (2013). Improved evaluation of geo-registration algorithms for airborne EO/IR imagery. In SPIE, Geospatial Infofusion III (Vol. 8747, p. 874709).

    Google Scholar 

  23. Vasquez, J., Hytla, P., Asari, V., Jackovitz, K., & Balster, E. (2012). Registration of region of interest for object tracking applications in wide area motion imagery. In The IEEE Applied Imagery Pattern Recognition Workshop (AIPR) (pp. 1–8).

    Google Scholar 

  24. Stone, H. S., Orchard, M. T., Chang, E. C., & Martucci, S. A. (2001). A fast direct Fourier-based algorithm for subpixel registration of images. IEEE Transactions on Geoscience and Remote Sensing, 39(10), 2235–2243.

    Article  Google Scholar 

  25. Hafiane, A., Palaniappan, K., & Seetharaman, G. (2008). UAV-video registration using block-based features. In IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium (Vol. 2, no. 1, pp. II–1104–II–1107)

    Google Scholar 

  26. Seetharaman, G., Gasperas, G., & Palaniappan, K. (2000). A piecewise affine model for image registration in nonrigid motion analysis. In Proceedings of the International Conference on Image Processing, 2000 (pp. 561–564).

    Google Scholar 

  27. Zhu, Z., Hanson, A. R., & Riseman, E. M. (2004). Generalized parallel-perspective stereo mosaics from airborne video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2), 226–237.

    Article  Google Scholar 

  28. Jackson, B. P., & Goshtasby, A. (2014). Adaptive registration of very large images. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (pp. 351–356).

    Google Scholar 

  29. Castro-Toscano, M. J., Rodríguez-Quiñonez, J. C., Hernández-Balbuena, D., Lindner, L., Sergiyenko, O., Rivas-Lopez, M., & Flores-Fuentes, W. (June 2017). A methodological use of inertial navigation systems for strapdown navigation task. In 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE) (pp. 1589–1595).

    Google Scholar 

  30. Lindner, L., Sergiyenko, O., Rivas-López, O., Ivanov, M., Rodríguez-Quiñonez, J. C., Hernández-Balbuena, D., et al. (June 2017). Machine vision system errors for unmanned aerial vehicle navigation. In 2017 IEEE 26th International Symposium on Industrial Electronics (ISIE) (pp. 1615–1620).

    Google Scholar 

  31. Lindner, L., Sergiyenko, O., Rivas-Lõpez, M., Hernández-Balbuena, D., Flores-Fuentes, W., Rodríguez-Quiñonez, J. C., et al. (2017). Exact laser beam positioning for measurement of vegetation vitality. In Industrial Robot: the International Journal of Robotics Research and Application, 44(4), 532–541.

    Article  Google Scholar 

  32. Turner, D., Lucieer, A., & Wallace, L. (2014). Direct georeferencing of ultrahigh-resolution UAV imagery. IEEE Transactions on Geoscience and Remote Sensing, 52(5), 2738–2745.

    Article  Google Scholar 

  33. Agisoft, agisoft photoscan professional. http://www.agisoft.com

  34. Pix4D. http://pix4d.com

  35. Snavely, N. Bundler: Structure from motion (SfM) for unordered image collections. http://phototour.cs.washington.edu/bundler

  36. Snavely, N., Seitz, S. M., & Szeliski, R. (2008). Modeling the world from Internet photo collections. International Journal of Computer Vision, 80, 189–210.

    Article  Google Scholar 

  37. Pritt, M. D. (2014). Fast orthorectified mosaics of thousands of aerial photographs from small UAVs. In Applied Imagery Pattern Recognition Workshop (AIPR). Piscataway: IEEE.

    Google Scholar 

  38. Aliakbarpour, H., Nuez, H. P., Prado, J., Khoshhal, K., & Dias, J. (2009). An efficient algorithm for extrinsic calibration between a 3D laser range finder and a stereo camera for surveillance. In 2009 International Conference on Advanced Robotics.

    Google Scholar 

  39. Redmill, K. A., Martin, J. I., & Ozguner, U. (2009). Aerial image registration incorporating GPS/IMU data. Proceedings of SPIE, 7347(1), 73470H–73470H–15.

    Google Scholar 

  40. Ma, J., Zhou, H., Zhao, J., Gao, Y., Jiang, J., & Tian, J. (2015). Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Transactions on Geoscience and Remote Sensing, PP(99), 1–13.

    Google Scholar 

  41. Liu, Z., An, J., & Jing, Y. (2012). A simple and robust feature point matching algorithm based on restricted spatial order constraints for aerial image registration. IEEE Transactions on Geoscience and Remote Sensing, 50(2), 514–527.

    Article  Google Scholar 

  42. Aktar, R., Aliakbarpour, H., Bunyak, F., Kazic, T., Seetharaman, G., & Palaniappan, K. (2018). Geospatial content summarization of UAV aerial imagery using mosaicking. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 10645).

    Google Scholar 

  43. Viguier, R., Lin, C. C., AliAkbarpour, H., Bunyak, F., Pankanti, S., Seetharaman, G., et al. (2015). Automatic video content summarization using geospatial mosaics of aerial imagery. In 2015 IEEE International Symposium on Multimedia (ISM) (pp. 249–253).

    Google Scholar 

  44. Viguier, R., Lin, C. C., Swaminathan, K., Vega, A., Buyuktosunoglu, A., Pankanti, S., et al. (2015). Resilient mobile cognition: Algorithms, innovations, and architectures. In Proceedings of the 33rd IEEE International Conference on Computer Design, ICCD 2015 (pp. 728–731).

    Google Scholar 

  45. Triggs, B., Mclauchlan, P. F., Hartley, R. I., & Fitzgibbon, A. W. (2000). Bundle adjustment : A modern synthesis. Vision algorithms: theory and practice. S, 34099, 298–372.

    Google Scholar 

  46. Aliakbarpour, H., Palaniappan, K., & Seetharaman, G. (2015). Robust camera pose refinement and rapid SfM for multi-view aerial imagery without RANSAC. IEEE Journal of Geoscience and Remote Sensing Letters, 12(11), 2203–2207.

    Article  Google Scholar 

  47. Johan, A. A., Suykens, A. K., & Signoretto, M. (2014). Regularization, optimization, kernels, and support vector machines. London/Boca Raton: Chapman and Hall/CRC Press.

    MATH  Google Scholar 

  48. Barron, J. T. (2017). A more general robust loss function. Arxiv, 1(5), 2–5. https://arxiv.org/abs/1701.03077

    Google Scholar 

Download references

Acknowledgements

This research was partially sponsored by the Army Research Laboratory and Air Force Research Laboratory under Cooperative Agreements W911NF-18-2-0285 and FA8750-19-2-0001, respectively. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

Author information

Authors and Affiliations

  1. Computational Imaging and VisAnalysis (CIVA) Lab, EECS, University of Missouri, Columbia, MO, USA

    Hadi Aliakbarpour & Kannappan Palaniappan

  2. Advanced Computing Concepts, U.S. Naval Research Laboratory, Washington, DC, USA

    Guna Seetharaman

Authors
  1. Hadi Aliakbarpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kannappan Palaniappan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guna Seetharaman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hadi Aliakbarpour .

Editor information

Editors and Affiliations

  1. Universidad Autónoma de Baja California, Mexicali, Baja California, Mexico

    Oleg Sergiyenko

  2. Universidad Autónoma de Baja California, Mexicali, Baja California, Mexico

    Wendy Flores-Fuentes

  3. Leuphana University of Lueneburg, Lueneburg, Low Saxony, Niedersachsen, Germany

    Paolo Mercorelli

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Aliakbarpour, H., Palaniappan, K., Seetharaman, G. (2020). Stabilization of Airborne Video Using Sensor Exterior Orientation with Analytical Homography Modeling. In: Sergiyenko, O., Flores-Fuentes, W., Mercorelli, P. (eds) Machine Vision and Navigation. Springer, Cham. https://doi.org/10.1007/978-3-030-22587-2_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-22587-2_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-22586-5

  • Online ISBN: 978-3-030-22587-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation