Parameter Estimation, Variance Components and Statistical Analysis in Errors-in-Variables Models

  • Peiliang XuEmail author
Living reference work entry
Part of the Springer Reference Naturwissenschaften book series (SRN)


Der Beitrag diskutiert statistische und numerische Aspekte von (,,errors-in-variable“) EIV-Modellen mit oder ohne Nebenbedingungen. Die Parameter in einem EIV-Modell lassen sich oftmals unter Benutzung dreier methodischer Kategorien bewerten: konventionelle gewichtete LS-Methode, normierte orthogonale Regression und gewichtete TLS-Methode. Die vorliegende Arbeit beschftigt sich mit der Reformulierung eines EIV-Models als nichtlineares Ausgleichungsmodell ohne Nebenbedingungen.


Errors-in-variables Estimability of variance components Nonlinear adjustment Orthogonal regression Total least squares Variance components Weighted least squares 


This chapter discusses statistical and numerical aspects of constrained and unconstrained errors-in-variables (EIV) models. The parameters in an EIV model can often be estimated by using three categories of methods: the conventional weighted least squares (LS) method, normed orthogonal regression, and the weighted total least squares (TLS) method. The conventional weighted LS method is of significantly computational advantage but not rigorous statistically. We systematically investigate the effects of random errors in the design matrix on the weighted LS estimates of parameters and variance components, construct the N-calibrated almost unbiased weighted LS estimator of parameters and derive almost unbiased estimates for the variance of unit weight. Although orthogonal regression can be used to estimate the parameters in an EIV model, it is not statistically optimal either. The weighted TLS method is most rigorous and optimal to statistically estimate the parameters in an EIV model at the cost of substantially increasing computation. We reformulate an EIV model as a nonlinear adjustment model without constraints and investigate the statistical effects of nonlinearity on the nonlinear TLS estimate, including the first order approximation of accuracy, nonlinear confidence region and bias of the nonlinear TLS estimate. Closed form solutions to coordinate transformation have been presented as well. Finally, we prove that variance components in an EIV model with the simplest stochastic structure are not estimable.


  1. 1.
    Adcock, R.J.: Note on the method of least squares. Analyst 4, 183–184 (1877)CrossRefGoogle Scholar
  2. 2.
    Adcock, R.J.: A problem in least squares. Analyst 5, 53–54 (1878)CrossRefGoogle Scholar
  3. 3.
    Akyilmaz, O.: Total least squares solution of coordinate transformation. Surv. Rev. 39, 68–80 (2007)CrossRefGoogle Scholar
  4. 4.
    Altmann, S.L.: Rotations, Quaternions, and Double Groups. Clarendon Press, Oxford (1982)Google Scholar
  5. 5.
    Amiri-Simkooei, A., Jazaeri, S.: Data-snooping procedure applied to errors-in-variables models. Stud. Geophys. Geod. 57, 426–441 (2013)CrossRefGoogle Scholar
  6. 6.
    Amiri-Simkooei, A.R., Zangeneh-Nejad, F., Asgari, J.: On the covariance matrix of weighted total least-squares estimates. J. Surv. Eng. 142, 04015014 (2016)CrossRefGoogle Scholar
  7. 7.
    Aoki, M., Yue, P.C.: On a priori error estimates of some identification methods. IEEE Trans. Auto. Contr. AC-15, 541–548 (1970)CrossRefGoogle Scholar
  8. 8.
    Arun, K.S., Huang, T.S., BLOSTEIN, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9, 698–700 (1987)CrossRefGoogle Scholar
  9. 9.
    Bab-Hadiashar, A., Suter, D.: Robust total least squares based optic flow computation. Int. J. Comput. Vis. 29, 566–573 (1998)CrossRefGoogle Scholar
  10. 10.
    Beale, E.M.L.: Confidence regions in non-linear estimation (with discussions). J. R. Stat. Soc. B22, 41–88 (1960)Google Scholar
  11. 11.
    Ben-Israel, A., Greville, T.: Generalized Inverses: Theory and Applications. Springer, Berlin (2000)Google Scholar
  12. 12.
    Boor, B.D.: Structured total least squares and L2 approximation problems. Lin. Algebra. Appl. 188, 163–205 (1993)Google Scholar
  13. 13.
    Bormann, P., Wylegalla, K.: Untersuchung der Korrelationsbeziehungen zwischen verschiedenen Arten der Magnitudenbestimmung der Station Moxa in Abhangigkeit vom Gerraetetyp und vom Herdgebiet. Publications of Institute of Geophysics, Pol. Acad. Sci. 93, 159–175 (1975)Google Scholar
  14. 14.
    Box, M.J.: Bias in nonlinear estimation (with discussions). J. R. Stat. Soc. B33, 171–201 (1971)Google Scholar
  15. 15.
    Branham R.L. Jr.: Astronomical data reduction with total least squares. New Astron. Rev. 45, 649–661 (2001)CrossRefGoogle Scholar
  16. 16.
    Cai, J.Q., Grafarend, E.: Systematical analysis of the transformation between Gauss-Krueger-coordinate/DHDN and UTM-coordinate/ETRS89 in Baden-Württemberg with different estimation methods. In: Drewes, H. (ed.) Geodetic Reference Frames, International Association of Geodesy Symposia 134, pp. 205–211. Springer, Berlin (2009)Google Scholar
  17. 17.
    Carroll, R.J., Ruppert, D., Stefanski, L.A., Crainiceanu, C.M.: Measurement Error in Nonlinear Models – a Modern Perspective, 2nd edn. Chapman and Hall, London (2006)CrossRefGoogle Scholar
  18. 18.
    Castellaro, S., Mulargia, F., Kagan, Y.Y.: Seismology regression problems for magnitudes. Geophys. J. Int. 165, 913–930 (2006)CrossRefGoogle Scholar
  19. 19.
    Chang, G.B., Xu, T.H., Wang, Q.X.: M-estimator for the 3D symmetric Helmert coordinate transformation. J. Geod. 92, 47–58 (2018)CrossRefGoogle Scholar
  20. 20.
    Cheng, C., van Ness, J.W.: Generalized M-estimators for errors-in-variables regression. Ann. Stat. 20, 385–397 (1992)CrossRefGoogle Scholar
  21. 21.
    Coolidge, J.L.: Two geometrical applications of the method of least squares. Am. Math. Mon. 20, 187–190 (1913)CrossRefGoogle Scholar
  22. 22.
    Davies, R.B., Hutton, B.: The effect of errors in the independent variables in linear regression. Biometrika 62, 383–391 (1975)CrossRefGoogle Scholar
  23. 23.
    Deming, W.E.: The application of least squares. Philos. Mag. 11, 146–158 (1931)CrossRefGoogle Scholar
  24. 24.
    Deming, W.E.: On the application of least squares — II. Philos. Mag. 17, 804–829 (1934)CrossRefGoogle Scholar
  25. 25.
    Deming, W.E.: Statistical Adjustment of Data. Dover Publications, New York (1964)Google Scholar
  26. 26.
    Dennis, Jr., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. SIAM Classics in Applied Mathematics. SIAM, Philadelphia (1996)Google Scholar
  27. 27.
    Dermanis, A.: The photogrammetric inner constraints. ISPRS J. Photogramm. Remote Sens. 49, 25–39 (1994)CrossRefGoogle Scholar
  28. 28.
    Dermanis, A.: Generalized inverses of nonlinear mappings and the nonlinear geodetic datum problem. J. Geod. 72, 71–100 (1998)CrossRefGoogle Scholar
  29. 29.
    Fang, X.: Weighted total least squares solutions for applications in geodesy. Wissenschaftliche Arbeiten der Fachrichtung Geodäsie und Geoinformatik, PhD Dissertation Nr.294, Leibniz University Hannover (2011)Google Scholar
  30. 30.
    Felus, Y.A.: Application of total least squares for spatial point process analysis. J. Surv. Eng. 130, 126–133 (2004)CrossRefGoogle Scholar
  31. 31.
    Felus, Y.A., Burtch, R.C.: On symmetrical three-dimensional datum conversion. GPS Solut. 13, 65–74 (2009)CrossRefGoogle Scholar
  32. 32.
    Fuller, W.A.: Measurement Error Models. Wiley Interscience, New York (1987)CrossRefGoogle Scholar
  33. 33.
    Fuller, W.A.: Introduction to Statistical Time Series, 2nd edn. Wiley, New York (1996)Google Scholar
  34. 34.
    Gerhold, G.A.: Least-squares adjustment of weighted data to a general linear equation. Am. J. Phys. 37, 156–161 (1969)CrossRefGoogle Scholar
  35. 35.
    Gleser, L.: Estimation in a multivariate “errors in variables” regression model: large sample results. Ann. Stat. 9, 24–44 (1981)CrossRefGoogle Scholar
  36. 36.
    Golub, G.H., van Loan, C.F.: An analysis of the total least squares problem. SIAM J. Numer. Anal. 17, 883–893 (1980)CrossRefGoogle Scholar
  37. 37.
    Grafarend, E., Schaffrin, B.: Unbiased free net adjustment. Surv. Rev. 22, 200–218 (1974)CrossRefGoogle Scholar
  38. 38.
    Grafarend, E., Schaffrin, B.: Equivalence of estimable quantities and invariants in geodetic networks. ZfV 101, 485–491 (1976)Google Scholar
  39. 39.
    Hodges, S.D., Moore, P.G.: Data uncertainties and least squares regression. Appl. Stat. 21, 185–195 (1972)CrossRefGoogle Scholar
  40. 40.
    Horn, B.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. 4, 629–642 (1987)CrossRefGoogle Scholar
  41. 41.
    Horn, B., Hilden, H.M., Negahdripour, S.: Closed-form solution of absolute orientation using orthonormal matrices. J. Opt. Soc. Am. 5, 1127–1135 (1988)CrossRefGoogle Scholar
  42. 42.
    Horn, S.D., Horn, R.A., Duncan, D.B.: Estimating heteroscedastic variances in linear models. J. Am. Stat. Ass. 70, 380–385 (1975)CrossRefGoogle Scholar
  43. 43.
    Kanatani, K., Niitsuma, H.: Optimal computation of 3-D similarity: Gauss-Newton vs. Gauss-Helmert. Comput. Stat. Data Anal. 56, 4470–4483 (2012)CrossRefGoogle Scholar
  44. 44.
    Keat, J.E.: Analysis of least-squares attitude determination routine DOAOP. Technical Report CSC/TM-77/6034, Computer Sciences Corp. (1977)Google Scholar
  45. 45.
    Koch, K.R.: S-transformations and projections for obtaining estimable parameters. In: 40 Years of Thought, Anniversary Volume on the Occasion of Prof. Baarda’s 65th Birthday, Delft, Vol. 1, pp. 136–144 (1982)Google Scholar
  46. 46.
    Koch, K.R.: Parameter estimation and hypothesis testing in linear models, 2nd edn. Springer, Berlin (1999)CrossRefGoogle Scholar
  47. 47.
    Kotsakis, C.: Reference frame stability and nonlinear distortion in minimum-constrained network adjustment. J. Geod. 86, 755–774 (2012)CrossRefGoogle Scholar
  48. 48.
    Kotsakis, C.: Generalized inner constraints for geodetic network densification problems. J. Geod. 87, 661–673 (2013)CrossRefGoogle Scholar
  49. 49.
    Krakiwsky, E.D., Thomson, D.B.: Mathematical models for the combination of terrestrial and satellite networks. Can. Surv. 28, 606–615 (1974)Google Scholar
  50. 50.
    Kummell, C.H.: Reduction of observation equations which contain more than one observed quantity. Analyst 6, 97–105 (1879)CrossRefGoogle Scholar
  51. 51.
    Lemmerling, P., van Huffel, S.: Structured total least squares. In: van Huffel, S., Lemmerling, P. (eds.) Total Least Squares and Errors-in-Variables Modelling, pp. 79–91. Kluwer Academic Publishers, Dordrecht (2002)CrossRefGoogle Scholar
  52. 52.
    Li, Y., Tang, H., Lin, X.: Spatial linear mixed models with covariate measurement errors. Stat. Sin. 19, 1077–1093 (2009)Google Scholar
  53. 53.
    Liu, J.N.: The equivalence of coordinate transformation models for the combination of satellite and terrestrial networks. J. Wuhan Tech. Univ. Surv. Mapp. 8, 37–50 (1983). (in Chinese with English abstract)Google Scholar
  54. 54.
    Liu, J.N., Liu, D.J.: The influence of the accuracy in geodetic and geocentric coordinates on combined adjustment. Acta Geod. Cartogr. Sinica. 14, 133–144 (1985). (in Chinese with English abstract)Google Scholar
  55. 55.
    Liu, J.N., Liu, D.J., Cui, X.Z.: Theory and applications of combined adjustment of satellite and terrestrial networks. J. Wuhan Tech. Univ. Surv. Mapp. 12(4), 1–9 (1987). (in Chinese with English abstract)Google Scholar
  56. 56.
    Lu, G.: Development of a GPS multi-antenna system for attitude determination. PhD Thesis, UCGE Reports No. 20073. Department of Geomatics Engineering, The University of Calgary (1995)Google Scholar
  57. 57.
    Magnus, J.R., Neudecker, H.: Matrix differential calculus with applications in statistics and econometrics. Wiley, New York (1988)CrossRefGoogle Scholar
  58. 58.
    Markovsky, I., van Huffel, S.: High-performance numerical algorithms and software for structured total least squares. J. Comput. Appl. Math. 180, 311–331 (2005)CrossRefGoogle Scholar
  59. 59.
    Markovsky, I., van Huffel, S.: On weighted structured total least squares. In: Lirkov, I., Margenov, S., Waśniewski, J. (eds.) LSSC 2005, LNCS 3743, pp. 695–702 (2006)Google Scholar
  60. 60.
    Markovsky, I., van Huffel, S.: Overview of total least squares methods. Sig. Proc. 87, 2283–2302 (2007)CrossRefGoogle Scholar
  61. 61.
    Mastronardi, N., Lemmerling, M., van Huffel, S.: Fast structured total least squares algorithm for solving the basic deconvolution problem. SIAM J. Matrix Anal. Appl. 22, 533–553 (2000)CrossRefGoogle Scholar
  62. 62.
    Meissl, P.: Die innere Genauigkeit eines Punkthaufens. Öster. ZfV 50, 159–165, 186–194 (1962)Google Scholar
  63. 63.
    Meissl, P.: Über die innere Genauigkeit dreidimensionaler Punkthaufen. ZfV 90, 109–118 (1965)Google Scholar
  64. 64.
    Neitzel, F.: Generalization of total least-squares on example of unweighted and weighted 2D similarity transformation. J. Geod. 84, 751–762 (2010)CrossRefGoogle Scholar
  65. 65.
    Nievergelt, Y.: Total least squares: state-of-the-art regression in numerical analysis. SIAM Rev. 36, 258–264 (1994)CrossRefGoogle Scholar
  66. 66.
    Papo, H., Perelmuter, A.: Datum definition by free net adjustment. Bull. Geod. 55, 218–226 (1981)CrossRefGoogle Scholar
  67. 67.
    Pearson, K.: On lines and planes of closest fit to systems of points in space. Philos. Mag. 2, 559–572 (1901)CrossRefGoogle Scholar
  68. 68.
    Rao, C.R., Kleffe, J.: Estimation of Variance Components and Applications. North-Holland, Amsterdam (1988)Google Scholar
  69. 69.
    Rao, C.R., Mitra, S.K.: Generalized Inverse of Matrices and Its Applications. Wiley, New York (1971)Google Scholar
  70. 70.
    Ratkowsky, D.A.: Nonlinear Regression Modeling – A Practical Unified Approach. Marcel Dekker, New York (1983)Google Scholar
  71. 71.
    Rinner, K.: Systematic investigations of geodetic networks in space. European Res Office, Contract No. 91–591-EUC 3584 (1966)Google Scholar
  72. 72.
    Rosen, J.B., Park, H., Glick, J.: Total least squares formulation and solution for structured problems. SIAM J. Matrix Anal. Appl. 17, 110–126 (1996)CrossRefGoogle Scholar
  73. 73.
    Ross, G.J.S.: Nonlinear Estimation. Springer, New York (1990)CrossRefGoogle Scholar
  74. 74.
    Sanso, F.: An exact solution of the roto-translation problem. Photogrammetria 29, 203–216 (1973)CrossRefGoogle Scholar
  75. 75.
    Schaffrin, B.: Aspects of network design. In: Grafarend, E., Sanso, F. (eds.) Optimization and Design of Geodetic Networks, pp. 548–597. Springer, Berlin (1985)CrossRefGoogle Scholar
  76. 76.
    Schaffrin, B.: A note on constrained total least-squares estimation. Linear Algebra Appl. 417, 245–258 (2006)CrossRefGoogle Scholar
  77. 77.
    Schaffrin, B., Wieser, A.: On weighted total least-squares adjustment for linear regression. J. Geod. 82, 415–421 (2008)CrossRefGoogle Scholar
  78. 78.
    Schaffrin, B., Felus, Y.A.: An algorithmic approach to the total least-squares problem with linear and quadratic constraints. Stud. Geophys. Geod. 53, 1–16 (2009)CrossRefGoogle Scholar
  79. 79.
    Schaffrin, B., Lee, I., Choi, Y., Felus, Y.A.: Total least-squares (TLS) for geodetic straight-line and plane adjustment. Boll. Geod. Sci. Aff. LXV, 141–168 (2006)Google Scholar
  80. 80.
    Schönemann, P.H.: A generalized solution of the orthogonal procrustes problem. Psychometrika 31, 1–10 (1966)CrossRefGoogle Scholar
  81. 81.
    Searle, S.R.: Linear Models. Wiley, New York (1971)Google Scholar
  82. 82.
    Seber, G., Wild, C.: Nonlinear Regression. Wiley, New York (1989)CrossRefGoogle Scholar
  83. 83.
    Shen, Y.Z., Li, B.F., Chen, Y.: An iterative solution of weighted total least squares adjustment. J. Geod. 85, 229–238 (2011)CrossRefGoogle Scholar
  84. 84.
    Shi, Y., Xu, P.L.: Unidentifiability of errors-in-variables models with rank deficiency from measurements (2018, submitted)Google Scholar
  85. 85.
    Shi, Y., Xu, P.L., Liu, J.N., Shi, C.: Alternative formulae for parameter estimation in partial errors-in-variables models. J. Geod. 89, 13–16 (2015)CrossRefGoogle Scholar
  86. 86.
    Teunissen, P.: Zero order design: generalized inverse, adjustment, the datum problem and S-transformations. In: Grafarend, E., Sanso, F. (eds.) Optimization and Design of Geodetic Networks, pp. 11–55. Springer, Berlin (1985)CrossRefGoogle Scholar
  87. 87.
    Thompson, E.H.: An exact linear solution of the problem of absolute orientation. Photogrammetria 15, 163–179 (1959)CrossRefGoogle Scholar
  88. 88.
    Tong, X.H., Jin, Y.M., Zhang, S.L., Li, L.Y., Liu, S.J.: Bias-corrected weighted total least-squares adjustment of condition equations. J. Surv. Eng. 141, 04014013 (2015)CrossRefGoogle Scholar
  89. 89.
    Umeyama, S.: Least squares estimation of transformation parameters between two point patterns. IEEE Trans. Pattern Anal. Mach. Intell. 13, 376–380 (1991)CrossRefGoogle Scholar
  90. 90.
    Ursin, B.: Methods for estimating the seismic reflection response. Geophysics 62, 1990–1995 (1997)CrossRefGoogle Scholar
  91. 91.
    van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. SIAM, Philadelphia (1991)CrossRefGoogle Scholar
  92. 92.
    Wang, N., Davidian, M.: A note on covariate measurement error in nonlinear mixed effects models. Biometrika 83, 801–812 (1996)CrossRefGoogle Scholar
  93. 93.
    Wang, N., Lin, X., Gutierrez, R., Carroll, R.J.: Bias analysis and SIMEX approach in generalized linear mixed measurement error models. J. Am. Stat. Ass. 93, 249–261 (1998)CrossRefGoogle Scholar
  94. 94.
    Wolf, H.: Helmerts Lösung zum Problem der freien Netze mit singulärer Normalgleichungsmatrix. ZfV 97, 189–192 (1972)Google Scholar
  95. 95.
    Wolf, H.: Die Helmert-Inverse bei freien geodätischen Netzen. ZfV 98, 396–398 (1973)Google Scholar
  96. 96.
    Wolf, H.: Scale and orientation in combined doppler and triangulation nets. Bull Géod 54, 45–53 (1980)CrossRefGoogle Scholar
  97. 97.
    Xu, P.L.: Testing the hypotheses of non-estimable functions in free net adjustment models. Manuscr. Geodaet. 20, 73–81 (1995)Google Scholar
  98. 98.
    Xu, P.L.: A general solution in nonlinear rank-defect geodetic models. Boll. Geod. Sci. Aff. 56, 1–25 (1997)Google Scholar
  99. 99.
    Xu, P.L.: Biases and accuracy of, and an alternative to, discrete nonlinear filters. J. Geod. 73, 35–46 (1999)CrossRefGoogle Scholar
  100. 100.
    Xu, P.L.: Nonlinear filtering of continuous systems: foundational problems and new results. J. Geod. 77, 247–256 (2003)CrossRefGoogle Scholar
  101. 101.
    Xu, P.L.: Determination of regional stress tensors from fault-slip data. Geophys. J. Int. 157, 1316–1330 (2004)CrossRefGoogle Scholar
  102. 102.
    Xu, P.L.: The effect of errors-in-variables on variance component estimation. J. Geod. 90, 681–701 (2016)CrossRefGoogle Scholar
  103. 103.
    Xu, P.L., Grafarend, E.: Statistics and geometry of the eigenspectra of 3-D second-rank symmetric random tensors. Geophys. J. Int. 127, 744-756 (1996)CrossRefGoogle Scholar
  104. 104.
    Xu, P.L., Liu, J.N.: Variance components in errors-in-variables models: estimability, stability and bias analysis. Invited talk, VIII Hotine-Marussi Symposium on Mathematical Geodesy, Rome, 17–21 June (2013)Google Scholar
  105. 105.
    Xu, P.L., Liu, J.N.: Variance components in errors-in-variables models: estimability, stability and bias analysis. J. Geod. 88, 719–734 (2014)CrossRefGoogle Scholar
  106. 106.
    Xu, P.L., Shimada, S.: Least squares parameter estimation in multiplicative noise models. Commun. Stat. B29, 83–96 (2000)CrossRefGoogle Scholar
  107. 107.
    Xu, P.L., Liu, J.N., Shi, C.: Total least squares adjustment in partial errors-in-variables models: algorithm and statistical analysis. J. Geod. 86, 661–675 (2012)CrossRefGoogle Scholar
  108. 108.
    Xu, P.L., Liu, J.N., Zeng, W., Shen, Y.Z.: Effects of errors-in-variables on weighted least squares estimation. J. Geod. 88, 705–716 (2014)CrossRefGoogle Scholar
  109. 109.
    Zeng, H.: Analytical algorithm of weighted 3D datum transformation using the constraint of orthonormal matrix. EPS 67, art.105 (2015)Google Scholar
  110. 110.
    Zhou, J.W.: Quasi-stable adjustment of monitoring networks. Wuhan Inst. Geod. Geophys. Spec. Publ. No. 2 (1980). (in Chinese)Google Scholar
  111. 111.
    Zumar, R.H.: Robust estimation in the errors-in-variables model. Biometrika 76, 149–160 (1989)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Deutschland, ein Teil von Springer Nature 2018

Authors and Affiliations

  1. 1.Disaster Prevention Research InstituteKyoto UniversityUji, KyotoJapan

Personalised recommendations