Skip to main content
SpringerLink
Log in

Augmented Reality in the Control Tower: A Rendering Pipeline for Multiple Head-Tracked Head-up Displays

  • Conference paper
  • First Online:
Augmented Reality, Virtual Reality, and Computer Graphics (AVR 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9768))

Abstract

The purpose of the air traffic management system is to accomplish the safe and efficient flow of air traffic. However, the primary goals of safety and efficiency are to some extent conflicting. In fact, to deliver a greater level of safety, separation between aircrafts would have to be greater than it currently is, but this would negatively impact the efficiency. In an attempt to avoid the trade-off between these goals, the long-range vision for the Single European Sky includes objectives for operating as safely and efficiently in Visual Meteorological Conditions as in Instrument Meteorological Conditions. In this respect, a wide set of virtual/augmented reality tools has been developed and effectively used in both civil and military aviation for piloting and training purposes (e.g., Head-Up Displays, Enhanced Vision Systems, Synthetic Vision Systems, Combined Vision Systems, etc.). These concepts could be transferred to air traffic control with a relatively low effort and substantial benefits for controllers’ situation awareness. Therefore, this study focuses on the see-through, head-tracked, head-up display that may help controllers dealing with zero/low visibility conditions and increased traffic density at the airport. However, there are several open issues associated with the use of this technology. One is the difficulty of obtaining a constant overlap between the scene-linked symbols and the background view based on the user’s viewpoint, which is known as ‘registration’. Another one is the presence of multiple, arbitrary oriented Head-Up Displays (HUDs) in the control tower, which further complicates the generation of the Augmented Reality (AR) content. In this paper, we propose a modified rendering pipeline for a HUD system that can be made out of several, arbitrary oriented, head-tracked, AR displays. Our algorithm is capable of generating a constant and coherent overplay between the AR layer and the outside view from the control tower. However a 3D model of the airport and the airport’s surroundings is needed, which must be populated with all the necessary AR overlays (both static and dynamic). We plan to use this concept as a basis for further research in the field of see-through HUDs for the control tower.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    OpenGL (Open Graphics Library) is a cross-language, multi-platform Application Programming Interface (API) for rendering 2D and 3D vector graphics.

  2. 2.

    A frustum is a six-sided truncated pyramid that originates sectioning the shape the virtual camera FOV by means of two user-defined clipping planes. These are known as the ‘far clipping plane’ and the ‘near clipping plane’. The latter is the one on which the scene must be projected as a necessary step of the rendering pipeline.

  3. 3.

    For the sake of readability, we refer to the straight line being orthogonal to the screen and passing by the centre of it simply as the screen normal.

  4. 4.

    In linear algebra an orthonormal basis for an inner product space is a basis whose vectors are all unit vectors orthogonal to each other.

References

  1. Office for Official Publications of the European Communities: Single European Sky - Results from the transport research programme. http://www.transport-research.info/sites/default/files/brochure/20040617_111440_74437_singlesky.pdf

  2. SESAR JU: Enhanced Ground Controller Situational Awareness in all Weather Conditions. https://www.atmmasterplan.eu/oisteps/AO-0201

  3. Clean Sky JU: Clean Sky 2 - Joint Technical Programme. http://ec.europa.eu/research/participants/data/ref/h2020/other/guide-appl/jti/h2020-guide-techprog-cleansky-ju_en.pdf

  4. SESAR JU: Honeywell in SESAR: Advancing ATM in the heart of Europe (2012). http://www.sesarju.eu/sites/default/files/documents/reports/SESAR_magazine_issue_7.pdf

  5. Reisman, R., Brown, D.: Design of augmented reality tools for air traffic control towers. In: Proceedings of 6th AIAA Aviation Technology, Integration and Operations Conference (ATIO). American Institute of Aeronautics and Astronautics, Wichita, KS (2006)

    Google Scholar 

  6. Tavanti, M.: Control Tower Operations: A Literature Review of Task Analysis Studies. EEC Technical/Scientific Report No. 2006-013 (2006)

    Google Scholar 

  7. Pinska, E.: An investigation of the head-up time at tower and ground control positions. In: Proceeding of the 5th Eurocontrol Innovative Research Workshop, pp. 81–86 (2006)

    Google Scholar 

  8. Kunert, T., Krömker, H.: A pattern-based framework for the exploration of design alternatives. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4550, pp. 1119–1128. Springer, Heidelberg (2007)

    Google Scholar 

  9. Ellis, S.R.: Towards determination of visual requirements for augmented reality displays and virtual environments for the airport tower. In: Virtual Media for Military Applications RTO-MP-HFM-136, pp. 31-1–31-10. RTO, Neuilly-sur-Seine (2006)

    Google Scholar 

  10. Shackelford, R., Karpe, P.: Low/Zero Visibility Tower Tools (L/ZVTT) Benefits Assessment., NASA Ames Research Center Technical Reports, Moffett Field, CA 94035 (1998)

    Google Scholar 

  11. Neveu, C., Blackmon, T., Stark, L.: Evaluation of the effects of a head-mounted display on ocular accommodation. Presence 7, 278–289 (1998)

    Article  Google Scholar 

  12. Ellis, S.R., Adelstein, B.D., Reisman, R.J., Schmidt-Ott, J.R., Gips, J., Krozel, J.: Augmented Reality in a Simulated Tower Environment: Effect of Field of View on Aircraft Detection. NASA Ames Research Center Technical Reports (2002)

    Google Scholar 

  13. Weintraub, D.J., Ensing, M.J.: Human factors issues in head-up display and design: the book of HUD. Crew System Ergonomics Information Analysis Center, Wright-Patterson AFB, Ohio (1992)

    Google Scholar 

  14. Rediess, H.: An augmented reality pilot display for airport operations under low and zero visibility conditions. In: Guidance, Navigation, and Control Conference. American Institute of Aeronautics and Astronautics (1997)

    Google Scholar 

  15. Ruffner, J.W., Deaver, D.M., Henry, D.J.: Requirements analysis for an air traffic control tower surface surveillance enhanced vision system. Presented at the SPIE-The International Society for Optical Engineering (2003)

    Google Scholar 

  16. Fürstenau, N.: Virtual tower. In: 5th ATM R&D Symposium, Braunschweig (2005)

    Google Scholar 

  17. Azuma, R.: A survey of augmented reality. Presence: Teleoper. Virtual Env. 6, 355–385 (1997)

    Article  Google Scholar 

  18. Reisman, R.J., Feiner, S.K., Brown, D.M.: Augmented reality tower technology flight test. In: International Conference on Human-Computer Interaction in Aerospace (HCI-Aero), Santa Clara, CA (2014)

    Google Scholar 

  19. Reisman, R.J., Brown, D.M.: Augmented Reality Tower Technology Assessment. NASA Ames Research Center Technical Reports (2010)

    Google Scholar 

  20. De Crescenzio F., Bagassi S., Fantini M., Lucchi F.: Virtual Reality based HUD (Head Up Display) to simulate 3D conformal symbols in the design of future cockpits. Presented at the Council of European Aerospace Societies, Venice, Italy, 24–28 October 2011

    Google Scholar 

  21. Bagassi, S., De Crescenzio, F., Lucchi, F., Persiani, F.: Innovation in man machine interfaces: use of 3D conformal symbols in the design of future HUDs (Head-Up Displays). Presented at the 28th International Congress of the Aeronautical Sciences, Brisbane, Australia, 23 September 2012

    Google Scholar 

  22. SESAR JU: System Wide Information Management (SWIM). http://www.sesarju.eu/sesar-solutions/swim

  23. SESAR JU: SWIM Concept of Operations. https://www.eurocontrol.int/sites/default/files/publication/files/del08.01.01-d41-swim_conops.pdf

  24. SESAR JU: Time Based Separation. http://www.sesarju.eu/sesar-solutions/airport-integration-and-throughput/time-based-separation

  25. SESAR: 06.09.03 Remote & virtual TWR. https://www.atmmasterplan.eu/projects/16484

  26. Fürstenau, N., Schmidt, M., Rudolph, M., Möhlenbrink, C., Papenfuß, A., Kaltenhäuser, S.: Steps towards the virtual tower: Remote Airport traffic control Center (RAiCe). Presented at the ENRI International Workshop on ATM/CNS, Tokyo, Japan (2009)

    Google Scholar 

  27. Schulz-Rückert, D.: Future of aerodrome traffic control. Presented at the Aachen Aviation Convention, Aachen, Germany (2007)

    Google Scholar 

  28. Bimber, O., Raskar, R.: Spatial Augmented Reality: Merging Real and Virtual Worlds. A. K. Peters Ltd., Natick (2005)

    Book  Google Scholar 

  29. Peterson, S.: Very large format stereoscopic head-up display for the airport tower. In: Proceedings of the Virtual Images Seminar, number 16. CNRS/Renault (2007)

    Google Scholar 

  30. Peterson, S., Pinska, E.: Human performance with simulated collimation in transparent projection screens. In: Proceedings of the Second International Conference on Research in Air Transportation (ICRAT) (2006)

    Google Scholar 

  31. Rolland, J.P., Baillot, Y., Goon, A.A.: A survey of tracking technology for virtual environments. Fundam. Wearable Comput. Augmented Reality 1, 67–112 (2001)

    Google Scholar 

  32. Zhou, F., Duh, H.B.-L., Billinghurst, M.: Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, ISMAR 2008, pp. 193–202 (2008)

    Google Scholar 

  33. Masotti, N., Persiani, F.: Gaze-coupled perspective for enhanced human-machine interfaces in aeronautics. Presented at the Conferences in Air Transport & Operations, Delft University of Technology, Delft, The Netherlands, 20 July 2015

    Google Scholar 

  34. OpenGL: gluPerspective. https://www.opengl.org/sdk/docs/man2/xhtml/gluPerspective.xml

  35. Song Ho Ahn: OpenGL Projection Matrix. http://www.songho.ca/opengl/gl_projectionmatrix.html

  36. Woo, M., Neider, J., Davis, T., Shreiner, D.: OpenGL programming guide: the official guide to learning OpenGL, Version 1.2. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)

    Google Scholar 

  37. Microsoft: DirectX Graphics and Gaming. https://msdn.microsoft.com/en-us/library/windows/desktop/ee663274(v=vs.85).aspx

  38. glFrustum – OpenGL. https://www.opengl.org/sdk/docs/man2/xhtml/glFrustum.xml

  39. Kooima, R.: Generalized Perspective Projection. J. Sch. Electron. Eng. Comput. Sci. (2009)

    Google Scholar 

  40. Liverani, A., Persiani, F., De Crescenzio, F.: An Immersive Reconfigurable Room (I.R.R.) for virtual reality simulation. In: Electronic Proceedings of 12th International Conference on Design Tools and Methods in Industrial Engineering, Rimini, Italy, pp. E1-9–E1-16 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, University of Bologna, Bologna, Italy

    Nicola Masotti, Francesca De Crescenzio & Sara Bagassi

Authors
  1. Nicola Masotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesca De Crescenzio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sara Bagassi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicola Masotti .

Editor information

Editors and Affiliations

  1. University of Salento, Lecce, Italy

    Lucio Tommaso De Paolis

  2. University of Salento, Lecce, Italy

    Antonio Mongelli

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Masotti, N., De Crescenzio, F., Bagassi, S. (2016). Augmented Reality in the Control Tower: A Rendering Pipeline for Multiple Head-Tracked Head-up Displays. In: De Paolis, L., Mongelli, A. (eds) Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2016. Lecture Notes in Computer Science(), vol 9768. Springer, Cham. https://doi.org/10.1007/978-3-319-40621-3_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-40621-3_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-40620-6

  • Online ISBN: 978-3-319-40621-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation