Comparison of two different methodologies for correcting refraction in vertical angles


The term atmospheric or geodetic refraction is the main factor that needed to be known in order to correct the vertical angles in high-precision geodetic applications. This term is found in the relevant surveying literature as mean to describe the alteration in the direction of the light curve as it propagates through the different layers of the lower part of the Earth’s atmosphere in the frame of standard surveying applications. So, what are the new things that the new article deals with? First substantial add is the analysis of the adjustment results of three more 3D geodetic networks with the TG method (totally, four 3D geodetic networks) in the TT1 tunnel at CERN as well as the comparisons of these results with the HLS measurements. However, the main task of this research work is the processing of the same geodetic data with the trigonometrical levelling network adjustment method (TLNA method) in order to detect and eliminate the effect of the refraction in the zenith angle measurements. A significant part of this research work after the result analysis is the comparison of the effectiveness of these two methods. Finally, after the analysis of the results of the two methodologies is proven that the TG method is more adequate than the TLNA method for such accurate measurements. With the TG method, the maximum difference between the nominal height differences of HLSystems and the calculated height differences after the 3D network adjustments with the corrected zenith angles is very promising and approaches the value of 50 μm.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12


  1. Arabatzi O (2007) Investigation of the methodologies for the vertical position of a point in engineering projects - applications in Greece. Athens: Dissertation, National Technical University of Athens

  2. Bahnert G (1986) Refraction und Refruktiomkoeffient. (Vol. VT. Vol. 34)

  3. Baiocchi N, Barbarella M, D’Alessio MT, Lelo K, Troisi S (2016) The sundial of Augusts and its survey: unresolved issues and possible solutions. Acta Geod Geophys 51:527–540.

    Article  Google Scholar 

  4. Boerez J, Hinderer J, Rivera L, Jones M (2012) Analysis and modeling of the effect of tides on the hydrostatic levelling system at CERN. Surv Rev 44:256–264

    Article  Google Scholar 

  5. Bomford G (1952) Geodesy. London: Oxford University Press, Amen House

  6. Brunner F (1984) Modelling of atmospheric effects on terestrial geodetic measurements. In: Brunner FK (ed) Geodetik refraction, effects of electromagnetic wave propagation through the atmosphere. Springer Verlag, Berin, Heidelberg, New York, Tokio

    Google Scholar 

  7. Brunner F, Fraser C (1977) application of the atmospheric turbulent transfer model (TTM) for the reduction of microwave FDM (Vol. no. 27). Unisurv G. Austral J Geodesy Photogrammetry & Surveying

  8. Elmiger AW (1983) Dreidimensionale Berechnung von geodätischen Netzen: Methoden und Erfahrungen (Vol. Bericht Nr. 73). ETH Zurich

  9. Flach P (2000) Analysis of refraction influences in geodesy using image processing and turbulence models. Zurich

  10. Gottawald R (1985) Zur Genauigkeitssteigerung und Erstellung eines automatisierten Datenflusses beim trigonometrischen Nivellement mit kurzen Zielweiten (Vol. Nr. 37.). Aachen

  11. Hennes M, Dönicke R, Christ H (1999) Zur Bestimmung der temperaturgradienteninduzierten Richtungsverschwenkuni beim Tunnelvortrieb (Vol. VPK. 8/99)

  12. Herty A, Mainaud-Durand H, Marin A (2004) Test and calibration facility for HLS and WPS sensors. IWAA2004. Geneva

  13. Hertzsprung E (1912) Photographische Messung der atmosphärischen Dispersion (Vol. Vol. 192). Astron Nachr, 309, 320

  14. Ingensand H, Bockem B (1997) Automatic location and pointing techniques in local positioning systems. In: Grun A., Kahmen H., 4th conference on optical 3-D measurements techniques. Zurich

  15. Jenkins FA, White HE (2001) Fundamentals of optics, 4th edn. McGraw-Hill Companies, Inc.

  16. Jordan W, Eggert O, Kneissl M (1956) Handbuch der Vernes - sungskunde (Vol. Vol III)

  17. Kahmen H, Faig W (1988) Surveying. Walter De Gruyter Inc.

  18. Kharagani G (1987) Propagation of refraction errors in trigonometric height traversing and geodetic levelling. Canada: Technical report No.132

  19. Korittke N (1996) Influence of horizontal refraction on the traverse measurements in tunnels with small diameters, Institute for Deposits and Surveying, DMT, Bochum, W-Germany

  20. Lambrou E, Pantazis G (2010) Applied geodesy. Thessaloniki: Ziti

  21. Law J, Rennie R (2015) A Dictionary of physics (7 ed.). Publisher: Oxford University Press

  22. Linkwitz K, Eisele V, Hans-Joachim Mönicke (1991) Applications of geodesy to engineering. International Union of Geodesy and Geophysics International Association of Geodesy. Symposium No. 108, Stuttgard, Germany, 284–285

  23. Nikolitsas K, Lambrou E (2019) A methodology for correcting refraction in vertical angles for precise monitoring in tunnels. 4th Joint International Symposium on Deformation Monitoring (JISDM), 15–17 May 2019, Athens

  24. Ogundare OJ (2016) Precision surveying: the principles and geomatics practice. Published by Wiley, Hoboken, 503–504

  25. Tsoulis D, Petrovic S, Kilian N (2008) Theoretical and numerical aspects of the geodetic method for determining the atmospheric refraction coefficient using simultaneous and mutual zenith observations (Vol. Vol. 134). J Surv Eng, 3, 12

  26. Wilhelm W (1993) Empirische Bestimmunii der Seitenrefration an 10 Objekten in der Praxis. Zurich: TGP Bericht Nr. 219

Online Documents

  1. Gyromat 5000. Accessed 03 Mar 2020

  2. Herty A (2008) HLS - Market Survey Cern EDMs Document No 449104.!master/navigator/document?D:1991624687:1991624687:subDocs Accessed 27 Aug 2019

  3. Normal distribution table. ElectricEngineer/tablez.pdf Accessed 21 Jun 2017

  4. Product Brochure - Leica Absolute Tracker AT401. Accessed 09 Aug 2018

Download references


I am especially indebted and grateful to Jean – Christophe Gayde, EN/SMM/ESA section leader at European Organization for Nuclear Research (CERN), and Dirk Mergelkuhl European Organization for Nuclear Research (CERN) for their assistance during this research work.

This paper is dedicated to the memory of my wonderful colleague and mentor Evangelia Lambrou, professor at National Technical University of Athens (NTUA), who recently passed away.

Author information



Corresponding author

Correspondence to K. Nikolitsas.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nikolitsas, K., Lambrou, E. Comparison of two different methodologies for correcting refraction in vertical angles. Appl Geomat 13, 119–129 (2021).

Download citation


  • Refraction
  • Laser tracker measurements
  • Temperature gradient
  • Hydrostatic levelling systems
  • CERN