# Risking to underestimate the integrity risk

- 206 Downloads

## Abstract

As parameter estimation and statistical testing are often intimately linked in the processing of observational data, the uncertainties involved in both estimation and testing need to be properly propagated into the final results produced. This necessitates the use of conditional distributions when evaluating the quality of the resulting estimator. As the conditioning should be on the identified hypothesis as well as on the corresponding testing outcome, omission of the latter will result in an incorrect description of the estimator’s distribution. In this contribution, we analyse the impact this omission or approximation has on the considered distribution of the estimator and its integrity risk. For a relatively simple observational model it is mathematically proven that the rigorous integrity risk exceeds the approximation for the contributions under the null hypothesis, which typically has a much larger probability of occurrence than an alternative. Actual GNSS-based positioning examples confirm this finding. Overall we observe a tendency of the approximate integrity risk being smaller than the rigorous one. The approximate approach may, therefore, provide a too optimistic description of the integrity risk and thereby not sufficiently safeguard against possibly hazardous situations. We, therefore, strongly recommend the use of the rigorous approach to evaluate the integrity risk, as underestimating the integrity risk in practice, and also the risk to do so, cannot be acceptable particularly in critical and safety-of-life applications.

## Keywords

Detection, identification and adaptation (DIA) DIA estimator Integrity risk Statistical testing Conditional distribution## Introduction

The DIA method for the detection, identification and adaptation of model misspecifications combines estimation with testing. Parameter estimation is conducted to determine estimates for the parameters of interest, and statistical testing is conducted to validate the results with the aim of removing any unwanted biases that may be present. To rigorously capture the estimation–testing combination, the DIA estimator has recently been introduced by Teunissen (2017) together with a unifying probabilistic framework. This allows one to take into account the intricacies of the combination when evaluating the contributions of the decisions and estimators involved. Procedures followed in practice are usually conditional ones implying that the quality and the performance of the resulting estimator must be described based on its subsequent conditional distribution. Hence, employing the distribution of the estimator under an identified hypothesis without regard to the conditioning process that led to the decision of accepting this hypothesis may impact the quality description of the resulting estimator in terms of precision, unbiasedness, confidence region and integrity risk.

In this contribution, we turn our attention specifically to the integrity risk, which—in short—is the probability that, whatever hypothesis is true in reality, the estimator of unknown parameters, directed by the testing outcome, is outside an acceptable area or volume around its targeted value. The estimator’s integrity risk is thus one minus its confidence level. As integrity plays a crucial role in critical and safety-of-life applications, for instance in aviation, when GNSS positioning is used to fly an approach to an airport, stringent requirements on integrity obviously apply. We compare, using a number of GNSS positioning examples, the integrity risk using unconditional distributions with the one obtained by rigorous evaluation of the correct conditional ones. We demonstrate that with the approximate approach of using unconditional distributions to evaluate the integrity risk, for instance when accounting for the event of removing from the solution an observation identified as faulty, one may obtain too optimistic figures, and thereby compromise the whole concept of integrity. The actual integrity risk, evaluated using the correct conditional distributions, may, in fact, be significantly larger.

This contribution is organized as follows. We start with a brief review of the detection, identification, and adaptation procedure, including the DIA estimator and its statistical distribution. Next, the integrity risk is defined, rigorous as well as approximate, with the latter following from neglecting the conditioning on the testing outcome. We then demonstrate in graphical form, using a simple observational model with just a single unknown parameter, both the unconditional and conditional distributions, so that the different contributions to the integrity risk, as well as the differences between the two approaches, are understood. We also prove in this section that, under the null hypothesis, the rigorous integrity risk always exceeds the approximate one. The integrity risk comparison is then continued, but now for a number of actual satellite-based single-point positioning examples. These findings show that one indeed runs a serious risk of underestimating the actual integrity risk (or overestimating the confidence level) when using the unconditional distributions instead of the conditional ones. We hereby note that our findings, although demonstrated by means of an application of the DIA procedure, are equally valid for any other method of fault detection and exclusion, and, therefore, hold true for a wide variety of different applications, such as geodetic quality control (Kösters and van der Marel 1990; Amiri Simkooei 2001; Perfetti 2006), navigational integrity (Teunissen 1990b; Gillissen and Elema 1996; Yang et al. 2014), structural health integrity (Verhoef and de Heus 1995; Yavaşoğlu et al. 2017; Durdag et al. 2018), and integrity monitoring of GNSS (Jonkman and de Jong 2000; Kuusniemi et al. 2004; Hewitson and Wang 2006). Finally, a summary with conclusions are presented.

## Detection, identification and adaptation (DIA)

A brief recap of the DIA-datasnooping procedure is provided, and the DIA estimator introduced in Teunissen (2017) is presented. Then an inventory of all possible testing decisions is compiled, and the distribution of the DIA estimator is decomposed into contributions, conditioned on the testing outcome and the hypothesis.

### Statistical hypotheses

### DIA-datasnooping procedure

*misclosure*vector \(t \in {{\mathbb{R}}^r}\) given as

As \(t\) is zero-mean under \({\mathcal{H}_0}\) and also independent of \({\hat {x}_0},\) it provides all the available information useful for validation of \({\mathcal{H}_0}\) (Teunissen 2017). Thus, an unambiguous testing procedure can be established through assigning the outcomes of \(t\) to the statistical hypotheses \({\mathcal{H}_i}\) for \(i=0,1, \ldots ,m.\)

The DIA-datasnooping procedure is specified as follows.

*Detection*: Accept \({\mathcal{H}_0}\)if \(t \in {\mathcal{P}_0}\) with

*Identification*: Compute Baarda’s test statistic for all alternative hypotheses as (Baarda 1967; Teunissen 2000)

*i*th column of matrix \({B^T}\) since \({c_i}\) is a canonical unit vector. Select \({\mathcal{H}_{i \ne 0}}\) if \(t \in {\mathcal{P}_{i \ne 0}}\) with

*Adaptation*: When \({\mathcal{H}_i}\) is selected, then \({\hat {x}_i}\) is provided as the estimate of \(x\).

The partitioning \({\mathcal{P}_i}\) in terms of the (original) misclosure vector is introduced in Teunissen (2017), and an example is shown in Fig. 3 in [Ibid]. Note that the above datasnooping partitioning would need modification in case of ‘iterated datasnooping’ (Kok et al. 1984) for multiple outlier testing which involves consecutive rounds of detection and identification until no further outlier is detected.

### DIA estimator

*combined*in DIA-datasnooping. Teunissen (2017) presents a unifying framework to rigorously capture the probabilistic properties of this combination, see also Teunissen et al. (2017). As such, the

*DIA estimator*\(\bar {x}\) was introduced, which captures the whole estimation–testing scheme and it is given as

### Testing decisions

### Decomposition of \({f_{\bar {x}}}(\theta |{\mathcal{H}_i})\)

*t*, we have

Comparing the structure of (20) with that of (18), we achieve (16). For \(j=0,\) we have \({f_{{{\hat {x}}_0},t}}\left( {\theta , \tau |{\mathcal{H}_i}} \right)= {f_{{{\hat {x}}_0}}}(\theta |{\mathcal{H}_i}) {f_t}\left( {\tau |{\mathcal{H}_i}} \right)\) of which the substitution into (20) gives (17). Note that the conditional PDFs \({f_{{{\hat {x}}_j}|t \in {\mathcal{P}_j}}}\left( {\theta |t \in {\mathcal{P}_j},{\mathcal{H}_i}} \right)\) for \(j \ne 0\) are* non-normal*, which is further discussed in the following sections.

## Integrity risk

*x*-centered region \({\mathcal{B}_x}\)’, and ‘sufficiently large’ as ‘\(1 - \varepsilon\)’ for a very small \(\varepsilon\), then \(\bar {x}\) is an acceptable estimator of \(x\) if \( \text{P}(\bar {x} \in {\mathcal{B}_x}) \geqslant 1 - \varepsilon\) or equivalently

*j*th measurement, cf. (2), the integrity risk can be decomposed, using the total probability rule, as

*t*, and use the unconditional PDFs \({f_{{{\hat {x}}_j}}}\left( {\theta |{\mathcal{H}_i}} \right)\) instead, an approximation of the rigorous integrity risk \(\overline {{\text{IR}}} |{\mathcal{H}_i}\) is obtained as

*not*on the testing outcome.

*t*, driven by \({Q_{{{\hat {x}}_0}{{\hat {x}}_0}}}\), \({Q_{tt}}\) and \({L_j}\). In addition, the conditional PDF \({f_{{{\hat {x}}_j}|t \in {\mathcal{P}_j}}}\left( {\theta |t \in {\mathcal{P}_j},{\mathcal{H}_i}} \right),\) using the total probability rule, can be written as

Finally, we note that the issue of correlation-neglect between \({\hat {x}_j}\) and \(t\) also comes up if one would use the outcomes of testing in an a posteriori evaluation. In that case one would have to work with the PDF of \(\left( {\bar {x}|t \in {\mathcal{P}_j}} \right)\) which is different from that of \({\hat {x}_j},\) despite the fact that both random vectors, \(\left( {\bar {x}|t \in {\mathcal{P}_j}} \right)\) and \({\hat {x}_j},\) have the same sample outcome (Teunissen 2017). For instance, if \({\mathcal{H}_j}\) is the identified hypothesis, confidence levels are typically evaluated in practice as \(\text{P}\left( {{{\hat {x}}_j} \in {\mathcal{B}_x}|{\mathcal{H}_i}} \right)\), see e.g. (Wieser 2004; Devoti et al. 2011; Dheenathayalan et al. 2016), while they should be evaluated as \(\text{P}\left( {{{\hat {x}}_j} \in {\mathcal{B}_x}|t \in {\mathcal{P}_j},{\mathcal{H}_i}} \right).\) The difference between their hypothesis averaged versions will then provide differences as those between (23) and (24).

## Numerical analysis: single alternative hypothesis

*n*= 1) and also the redundancy of the model is one (

*r*= 1), i.e., \(x \in {\mathbb{R}}\) and \(t \in {\mathbb{R}}.\) The canonical form of such a model, applying the Tienstra-transformation \(\mathcal{T}\) to the (assumed) normally distributed vector of observables \(y\) (Teunissen 2017), reads

The corresponding DIA-datasnooping procedure is defined as

*Detection*: Accept \({\mathcal{H}_0}\) if \(t \in {\mathcal{P}_0}\) with

Provide \({\hat {x}_0}\) as the estimate of \(x.\)

*Identification*: Select \({\mathcal{H}_a}\) if \(t \in \mathcal{P}_{0}^{c}\) with \(\mathcal{P}_{0}^{c}={\mathbb{R}}/{\mathcal{P}_0}\).

*Adaptation*: When \({\mathcal{H}_a}\) is selected, \({\hat {x}_a}\) is provided as the estimate of *x*.

### Decomposition of \({f_{\bar {x}}}\left( {\theta |{\mathcal{H}_0}} \right)\) and \({f_{\bar {x}}}\left( {\theta |{\mathcal{H}_a}} \right)\)

*black*curve shows \({f_{{{\hat {x}}_a}}}\left( {\theta |{\mathcal{H}_0}} \right),\) which is also equal to\({f_{{{\hat {x}}_a}}}\left( {\theta |{\mathcal{H}_\text{a}}} \right).\) The probability of false alarm P

_{FA}is usually user defined by setting the appropriate size of \(\mathcal{P}_{0}\), hence an input to the DIA procedure both under null and alternative hypotheses. To assess the PDF of \(\hat{x}_{0}\) and \(\bar{x}\) under the alternative

*H*

_{a}, one additionally needs to set the size of the bias

*b*

_{a}, or alternatively, one may choose to set the correct detection probability P

_{CD}as we did here.

*non-normal*, which for the case of one single alternative can be expressed as

### Non-normality of \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA})\) and \({f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD})\)

To appreciate the non-normality of the two PDFs \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA})\) and \({f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD}),\) we show them for different values of the contributing factors, namely \({\text{P}_{\text{FA}}},\) \({\text{P}_{\text{MD}}}\left( {=1 - {\text{P}_{\text{CD}}}} \right),\) \({\sigma _{{{\hat {x}}_0}}},\) \({\sigma _t}\) and \({L_a},\) in Fig. 2. To highlight the non-normality of \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA})\) and \({f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD}),\) we have also plotted (for reference) their normal counterparts having the same mean and variance in black. These normal PDFs, respectively, correspond with the random variables

Shown to the right of Fig. 2 are the graphs of \({f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD})\) together with those of \({f_{{{\hat {x}}_{\text{cd}}}}}(\theta ).\) The response of \({f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD})\) to the changes in the parameters \({\sigma _{{{\hat {x}}_0}}},\) \({\sigma _{t}}\) and \({L_{a}}\) is similar to that of \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA}).\) \({f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD})\) in addition depends on \({\text{P}_{\text{CD}}}\) according to (37). Since \({\text{P}_{\text{CD}}}=1 - {\text{P}_{\text{MD}}}\), increasing \({\text{P}_{\text{CD}}}\) (thus decreasing \({\text{P}_{\text{MD}}}\)) leads to smaller differences between \({f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD})\) and \({f_{{{\hat {x}}_a}}}\left( {\theta |{\mathcal{H}_a}} \right).\) This can also be seen by comparing the dashed blue curves with the solid black ones in the second and third rows of Fig. 1.

### Rigorous vs approximate integrity risk

Integrity risk based on the DIA estimator (\(\overline {{\text{IR}}}\)) and its approximation by ignoring the correlation between \({\hat {x}_a}\) and \(t\) (\(\text{IR}^{o}\)) for the case of a null- and a single alternative hypotheses. \(\mathcal{B}_{x}^{c}={{\mathbb{R}}^n}/{\mathcal{B}_x}\)

\(\begin{aligned} \overline {{\text{IR}}} & =\text{P}({\mathcal{H}_0}) \times \int\limits_{{\mathcal{B}_{x}^{c}}} {\left[ {{f_{{{\hat {x}}_0}}}(\theta |{\mathcal{H}_0}) {\text{P}_{\text{CA}}} +{f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA}) {\text{P}_{\text{FA}}}} \right]\text{d}\theta } \\ & \quad + \text{P}({\mathcal{H}_a}) \times \int\limits_{{\mathcal{B}_{x}^{c}}} {\left[ {{f_{{{\hat {x}}_0}}}(\theta |{\mathcal{H}_a}) {\text{P}_{\text{MD}}} +{f_{{{\hat {x}}_a}|\text{CD}}}(\theta |\text{CD}) {\text{P}_{\text{CD}}}} \right]\text{d}\theta } \\ \text{IR}^{o} & =\text{P}({\mathcal{H}_0}) \times \int\limits_{{\mathcal{B}_{x}^{c}}} {\left[ {{f_{{{\hat {x}}_0}}}(\theta |{\mathcal{H}_0}) {\text{P}_{\text{CA}}}+{f_{{{\hat {x}}_a}}}(\theta |{\mathcal{H}_0}) {\text{P}_{\text{FA}}}} \right]\text{d}\theta } \\ & \quad + \text{P}({\mathcal{H}_a}) \times \int\limits_{{\mathcal{B}_{x}^{c}}} {\left[ {{f_{{{\hat {x}}_0}}}(\theta |{\mathcal{H}_a}) {\text{P}_{\text{MA}}}+{f_{{{\hat {x}}_a}}}(\theta |{\mathcal{H}_a}) {\text{P}_{\text{CD}}}} \right]\text{d}\theta } \\ \end{aligned}\) |

### Evaluation of \(\left( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} \right)\)

It can be shown that the difference \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) in (42) is always positive. For a proof, see the Appendix. Assuming \(\sigma _{{{{\hat {x}}_0}}}^{2}=0.5{\text{m}^2}\) and \(\sigma _{t}^{2}=2\,{\text{m}^2},\) Fig. 3 illustrates graphs of \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}})\) (solid lines) and those of \(( {\overline {{\text{IR}}} |{\mathcal{H}_0}})\) (dashed lines) as a function of AL, for \({\text{P}_{\text{FA}}}={10^{ - 1}}\) (top) and \({\text{P}_{\text{FA}}}={10^{ - 3}}\) (bottom), and \({L_a}=0.5, 1.5\) (in blue and red, resp.). Comparing the solid lines with their corresponding dashed lines, we note, depending on the values of \({L_a}\) and \({\text{P}_{\text{FA}}}\), that after a certain alert limit, the values of \(( {\overline {{\text{IR}}} |{\mathcal{H}_0}} )\) and \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) approach each other, implying that the approximate integrity risk \(\left( {\text{IR}^{o}|{\mathcal{H}_0}} \right)\) gets very small indeed. We explain this behavior for the *blue* curves at the bottom when AL = 4 m. The probability mass of the PDFs \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA})\) and \({f_{{{\hat {x}}_a}}}\left( {\theta |{\mathcal{H}_0}} \right),\) the dashed blue curve and the black curve in the upper left panel in Fig. 1, are at the level of \(2 \times {10^{ - 2}}\) and \(6 \times {10^{ - 5}},\) respectively, outside \({\mathcal{B}_x}=[x - 4 , x+4].\) In addition, the probability mass of the PDF \({f_{{{\hat {x}}_0}}}\left( {\theta |{\mathcal{H}_0}} \right)\) outside \({\mathcal{B}_x}=[x - 4 , x+4]\) is at the level of \({10^{ - 8}}.\) These values, given \({P_{\text{FA}}}={10^{ - 3}},\) will then result in a difference at the level of \(8 \times {10^{ - 8}}\) between \(( {\overline {{\text{IR}}} |{\mathcal{H}_0}} )\) and \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} ).\) As a consequence of this case, the approximate integrity risk under the null hypothesis is too optimistic by a factor of 300.

As shown in the Appendix, \({\text{P}}(|\hat{x}_{a}-x|\le {\text{AL}}|\mathcal{H}_{0})\) is always greater than \({\text{P}}(|\hat{x}_{a}-x|\le {\text{AL}}|{\text{FA}}).\) Therefore, it can be concluded that the PDF \({f_{{{\hat {x}}_a}}}\left( {\theta |{\mathcal{H}_0}} \right)\) is more peaked around \(x\) than \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA}).\) Therefore, when AL increases, \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|{\mathcal{H}_0}} \right)\) decreases more rapidly than \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|\text{FA}} \right).\) This, together with (44) and the fact that the probabilities \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|{\mathcal{H}_0}} \right)\) and \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|\text{FA}} \right)\) are continuous functions of AL, results in an increasing and then decreasing behavior for \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\).

In Fig. 3, when \({L_a}\) increases, the curve of \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) stretches over a larger range of values of AL. This is due to the fact that increasing \({L_a}\) reduces the peakedness of the PDFs \({f_{{{\hat {x}}_a}}}\left( {\theta |{\mathcal{H}_0}} \right)\) and \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA})\) around \(x\). The impact of changing \({L_a}\) on \({f_{{{\hat {x}}_a}}}\left( {\theta |{\mathcal{H}_0}} \right)\) and \({f_{{{\hat {x}}_a}|\text{FA}}}(\theta |\text{FA})\) is demonstrated in (31) and Fig. 2 (top-left), respectively. Therefore, both the probabilities \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|{\mathcal{H}_0}} \right)\) and \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|\text{FA}} \right)\) behave more smoothly as function of AL, and so does their difference.

Decreasing \(\alpha\) by a factor of \({10^2},\) we note that the values of \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) in Fig. 3 also decrease by almost a factor of \({10^2}.\) Reducing \(\alpha\) by a factor of \({10^2}\) will reduce \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) by the same amount if \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|\text{FA}} \right)\) remains invariant. However, as Fig. 2 (bottom-left) shows, reducing \(\alpha\) by a factor of \({10^2}\) increases \(\text{P}\left( {|{{\hat {x}}_a} - x|>\text{AL}|\text{FA}} \right),\) which results in \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) decreasing by a factor slightly smaller than \({10^2}.\)

### Evaluation of \(\left( {\overline {{\text{IR}}} - \text{IR}^{o}} \right)\)

Here, we investigate the behavior of the integrity risks when both the null- and alternative hypotheses are taken into account. The difference \(( {\overline {{\text{IR}}} - \text{IR}^{o}} )\) in (41), due to the contribution of \({\mathcal{H}_a},\) depends on the bias value \({b_a}\). Considering \(\sigma _{{{{\hat {x}}_0}}}^{2}=0.5\,{\text{m}^2},\) \(\sigma _{t}^{2}=2\,{\text{m}^2}\) and \(\text{P}\left( {{\mathcal{H}_0}} \right)=0.9\) (thus \(\text{P}\left( {{\mathcal{H}_a}} \right)=0.1;\) the latter probability is usually much smaller in practice), Fig. 4 shows the curves of \(( {\overline {{\text{IR}}} - \text{IR}^{o}})\) as a function of \({b_a}\) for two different values of AL in \({\mathcal{B}_x}=[x - \text{AL}, x+\text{AL}],\) \({L_a}\) and \({\text{P}_{\text{FA}}}.\) Decreasing \({L_a},\) although the correlation \({\rho _{{{\hat {x}}_a},t}}\) decreases, the difference \(( {\overline {{\text{IR}}} - \text{IR}^{o}} )\) may increase or decrease depending on AL and \({\text{P}_{\text{FA}}}.\) It can be seen that \(( {\overline {{\text{IR}}} - \text{IR}^{o}} )\) does not change too much as a function of \({b_a}.\) This is due to the fact that any change in \({b_a}\) will change \(( {\overline {{\text{IR}}} |{\mathcal{H}_a} - \text{IR}^{o}|{\mathcal{H}_a}} ),\) of which the impact is downweighted by \(\text{P}\left( {{\mathcal{H}_a}} \right)=0.1.\) Note that \(\overline {{\text{IR}}}\) is larger than \(\text{IR}^{o}\) in most cases, revealing that using the approximate integrity risk instead of the rigorous one generally will be hazardous. The \(\text{IR}^{o}\) does not provide a safe bound to the actual \(\overline {{\text{IR}}}\).

## Numerical analysis: multiple alternative hypotheses

So far, for simplicity, we have been working with an observational model with one unknown parameter and one redundancy. In this section, we work with a satellite-based single-point positioning (SPP) model based on the observations of \(m\) satellites with four unknown parameters (\(n=4\)) and \(r=m - 4\) redundancy. As alternative hypotheses, we consider those given in (2). In that case there are as many alternative hypotheses as there are observations.

We first present the observational model, and then we analyze, by means of three practical examples, the difference between the rigorous and approximate integrity risk for the contributions under the null hypothesis (\({\mathcal{H}_0}\)).

### SPP observational model

*m*-vector of ones, and again \({c_0}{b_0}=0.\) The unknown receiver coordinate components and clock error are, respectively, denoted by the 3-vector \(x\) and scalar \({d}t.\) The dispersion of the observables is characterized through the standard deviation \({\sigma _p}\) and the identity matrix \({I_m}\). At this stage, to simplify our analysis, we do not consider a satellite-elevation-dependent variance matrix. In the following, we only concentrate on the \({\mathcal{H}_0}\)-driven difference between \(\overline {{\text{IR}}}\) and \(\text{IR}^{o},\) as the probability of the occurrence of \({\mathcal{H}_0}\) is by far larger than that of the alternative hypotheses.

### Evaluation of \(\left( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} \right)\)

*Example 1*: Fig. 5

The skyplot in Fig. 5 (top) shows a geometry of six satellites. The graphs of the difference \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) as a function of AL are shown for the vertical component [second row] and the horizontal components (third and fourth rows) for different values of \({\sigma _p}\) (cf. 45) and \({\text{P}_{\text{FA}}}=\alpha .\) The third row corresponds with \({\mathcal{B}_{{x_H}}}\), cf. (48), while the fourth row corresponds with \({\tilde {\mathcal{B}}_{{x_H}}}\), cf. (49). It is important to note that \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) has always ‘positive’ values for all components, meaning that employing the approximate integrity risk instead of the rigorous one could be dangerous, depending on the application at hand. To get a better appreciation of such danger, the values of \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) together with \(( {\overline {{\text{IR}}} |{\mathcal{H}_0}} )\) are tabulated in Table 2 for some values of AL. For example, for the vertical component with the settings of \({\text{P}_{\text{FA}}}={10^{ - 1}},\) \({\sigma _p}=1\text{m}\) and AL = 15 m, we have \(( {\overline {{\text{IR}}} |{\mathcal{H}_0}} )=0.0168\) and \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )=0.0148,\) implying that the approximate integrity risk \(( {\overline {{\text{IR}}} |{\mathcal{H}_0}} )\) is too optimistic by a factor of 8.

Values of the rigorous integrity risk \((\overline {{\text{IR}}} |{\mathcal{H}_0})\) and its difference with its approximation \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} ),\) taken from the graphs in Fig. 5

AL (m) | \(\overline {{\text{IR}}} |{\mathcal{H}_0}\) | \(\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}\) | AL (m) | \(\overline {{\text{IR}}} |{\mathcal{H}_0}\) | \(\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}\) | |
---|---|---|---|---|---|---|

P
| ||||||

\({\mathcal{B}_{{x_V}}}\) | 15 | 0.0168 | 0.0148 | 30 | 0.0015 | 0.0015 |

\({\mathcal{B}_{{x_H}}}\) | 10 | 0.0112 | 0.0107 | 18 | 0.0001 | 0.0001 |

\({\tilde {\mathcal{B}}_{{x_H}}}\) | 5 | 0.0211 | 0.0180 | 10 | 0.0055 | 0.0054 |

P
| ||||||

\({\mathcal{B}_{{x_V}}}\) | 15 | 0.0260 | 0.0179 | 30 | 0.0168 | 0.0148 |

\({\mathcal{B}_{{x_H}}}\) | 10 | 0.0213 | 0.0171 | 18 | 0.0137 | 0.0129 |

\({\tilde {\mathcal{B}}_{{x_H}}}\) | 5 | 0.1011 | 0.0440 | 10 | 0.0215 | 0.0184 |

P
| ||||||

\({\mathcal{B}_{{x_V}}}\) | 15 | 0.0017 | 0.0015 | 30 | 0.0011 | 0.0011 |

\({\mathcal{B}_{{x_H}}}\) | 10 | 0.0015 | 0.0015 | 18 | 0.0001 | 0.0001 |

\({\tilde {\mathcal{B}}_{{x_H}}}\) | 5 | 0.0027 | 0.0024 | 10 | 0.0014 | 0.0014 |

P
| ||||||

\({\mathcal{B}_{{x_V}}}\) | 15 | 0.0041 | 0.0021 | 30 | 0.0017 | 0.0015 |

\({\mathcal{B}_{{x_H}}}\) | 10 | 0.0025 | 0.0021 | 18 | 0.0015 | 0.0014 |

\({\tilde {\mathcal{B}}_{{x_H}}}\) | 5 | 0.0506 | 0.0055 | 10 | 0.0026 | 0.0023 |

*blue*curves) and \(\alpha ={10^{ - 2}}\) (solid red curves), for all six alternative hypotheses \(j=1, \ldots ,6.\) It can be seen that the PDF \({f_{{{\hat {x}}_{{V_j}}}}}\left( {{\theta _V}|{\mathcal{H}_0}} \right)\) (dashed) is more peaked around zero than \({f_{{{\hat {x}}_{{V_j}}}|t \in {\mathcal{P}_j}}}\left( {{\theta _V}|t \in {\mathcal{P}_j}, {\mathcal{H}_0}} \right).\) Therefore, when the AL increases, then \(\text{P}( {{{\hat {x}}_{{V_j}}} \in \mathcal{B}_{{{x_V}}}^{c}|{\mathcal{H}_0}} )\) decreases more rapidly than \(\text{P}( {{{\hat {x}}_{{V_j}}} \in \mathcal{B}_{{{x_V}}}^{c}|t \in {\mathcal{P}_j}, {\mathcal{H}_0}} ).\) This, together with (50) and the fact that the probabilities \(\text{P}( {{{\hat {x}}_{{V_j}}} \in \mathcal{B}_{{{x_V}}}^{c}|{\mathcal{H}_0}} )\) and \(\text{P}( {{{\hat {x}}_{{V_j}}} \in \mathcal{B}_{{{x_V}}}^{c}|t \in {\mathcal{P}_j}, {\mathcal{H}_0}} )\) are continuous functions of AL, results in increasing and then decreasing behavior for \(\text{P}( {{{\hat {x}}_{{V_j}}} \in \mathcal{B}_{{{x_V}}}^{c}|t \in {\mathcal{P}_j}, {\mathcal{H}_0}} ) - \text{P}( {{{\hat {x}}_{{V_j}}} \in \mathcal{B}_{{{x_V}}}^{c}|{\mathcal{H}_0}})\) as function of AL.

In Fig. 5, when \({\sigma _p}\) increases, the graph of \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) stretches over a larger range of AL. This is due to the fact that increasing \({\sigma _p}\) reduces the peakedness of the PDFs \({f_{{{\hat {x}}_{{*_j}}}}}\left( {{\theta _*}|{\mathcal{H}_0}} \right)\) and \({f_{{{\hat {x}}_{{*_j}}}|t \in {\mathcal{P}_j}}}\left( {{\theta _*}|t \in {\mathcal{P}_j}, {\mathcal{H}_0}} \right)\) around zero. Therefore, both the probabilities \(\text{P}( {{{\hat {x}}_{{*_j}}} \in \mathcal{B}_{{{x_*}}}^{c}|t \in {\mathcal{P}_j}, {\mathcal{H}_0}} )\) and \(\text{P}( {{{\hat {x}}_{{*_j}}} \in \mathcal{B}_{{{x_*}}}^{c}|{\mathcal{H}_0}} )\) behave more smoothly as function of AL, and so does their difference.

\(j\) | 6 | 3 | 5 | 2 | 1 | 4 |
---|---|---|---|---|---|---|

\({{|q_{{{{\hat {x}}_0}{{\hat {x}}_{{V_0}}}}}^{T}\left( {{u_j} - \bar {u}} \right)|} \mathord{\left/ {\vphantom {{|q_{{{{\hat {x}}_0}{{\hat {x}}_{{V_0}}}}}^{T}\left( {{u_j} - \bar {u}} \right)|} {||{c_{{t_j}}}|{|_{ {Q_{tt}}}}}}} \right. \kern-0pt} {||{c_{{t_j}}}|{|_{ {Q_{tt}}}}}}\) | 0.47 | 0.73 | 0.80 | 0.89 | 1.58 | 9.79 |

*Example 2*: Fig. 7

Figure 7 presents the same type of information as Fig. 5 but for a different geometry of six satellites. For this example, again we note the ‘positive’ values for \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} )\) in case of both the vertical and horizontal components. The difference in integrity risk (rigorous minus approximate) in general behaves similar to the earlier example.

*Example 3*: Fig. 8

In Fig. 8, we present the same type of information as in Figs. 5 and 7, but now for a multi-constellation example given in Blanch et al. (2012), Appendix J. This constellation is made of five GPS (G) and five Galileo (E) satellites. In this case, we have three position parameters and two receiver clock parameters (one for GPS and one for Galileo). The misclosure vector is of the dimension of five. Note that the results in Fig. 8 use an elevation-dependent variance (\({C_{\text{int}}}\) in the mentioned paper) for the observations. Also here, like in the previous examples, we note the ‘positive’ values for \(( {\overline {{\text{IR}}} |{\mathcal{H}_0} - \text{IR}^{o}|{\mathcal{H}_0}} ).\)

## Summary and conclusion

The message of this contribution finds its origin in the combination of parameter estimation and statistical testing. These two activities are typically disconnected in practice when it comes to describing the quality of the eventual estimator. That is, the distribution of the estimator under an identified statistical hypothesis is used without regard to the conditioning process that led to the decision to accept this hypothesis as the working model. We analyzed what the contribution of this simplification is to the actual integrity risk.

Considering a null hypothesis and a single alternative, the different distributions were first shown graphically for a simple observational model with a one-dimensional unknown parameter and a one-dimensional misclosure for statistical testing. It was demonstrated that, with normally distributed observables and linear models, the distributions of the estimators conditioned on false alarm and correct detection turn out to be no longer normal. To compute the integrity risk rigorously, one needs to condition on both the hypothesis and the testing outcome. An approximate risk is obtained, however, when one omits the connection between testing and estimation and thus only conditions on the hypothesis and not on the testing outcome.

For the simple observational model, it was mathematically proven that the rigorous integrity risk exceeds the approximate one. This comparison of the rigorous and approximate integrity risk was then continued by means of a number of satellite-based single-point positioning examples, focusing again on the contributions under the null hypothesis. Although a mathematical proof for the multi-dimensional case does not yet exist, these examples support the previously obtained conclusion that the approximate integrity risk has a tendency of being smaller than its rigorous counterpart. Thus, by including the uncertainty of the decision process driven by statistical testing and using conditional distributions instead of unconditional ones, the actual integrity risk may end up being larger than the computed approximate one. In other words, the approximate approach may provide a too optimistic description of the integrity risk and thereby not sufficiently safeguard against possibly hazardous situations.

We, therefore, clearly advocate the use of the rigorous approach to evaluate the integrity risk, as underestimating the risk, or knowingly allowing this possibility to exist, cannot be acceptable particularly in critical and safety-of-life applications.

## Notes

### Acknowledgements

The second author is the recipient of an Australian Research Council (ARC) Federation Fellowship (project number FF0883188). This support is greatly acknowledged.

## References

- Amiri Simkooei A (2001) Comparison of reliability and geometrical strength criteria in geodetic networks. J Geodesy 75(4):227–233CrossRefGoogle Scholar
- Baarda W (1967) Statistical concepts in geodesy. Netherlands Geodetic Commission, Publications on Geodesy, New Series, Vol. 2, No. 4, pp 74Google Scholar
- Blanch J, Walter T, Enge P, Lee Y, Pervan B, RipplM, Spletter A (2012) Advanced RAIM user algorithm description: integrity support message processing, fault detection, exclusion, and protection level calculation. In: Proc. of ION ITM-2012, ION, pp 2828–2849Google Scholar
- Devoti R, Esposito A, Pietrantonio G, Pisani AR, Riguzzi F (2011) Evidence of large scale deformation patterns from GPS data in the Italian subduction boundary. Earth Planet Sci Lett 311:230–241CrossRefGoogle Scholar
- Dheenathayalan P, Small D, Schubert A, Schubert A, Hanssen RF (2016) High-precision positioning of radar scatterers. J Geodesy 90(5):403–422CrossRefGoogle Scholar
- Durdag UM, Hekimoglu S, Erdoğan B (2018) Reliability of models in kinematic deformation analysis. ASCE J Surv Eng 144(3):040118004CrossRefGoogle Scholar
- Gillissen I, Elema IA (1996) Test results of DIA: a real-time adaptive integrity monitoring procedure, used in an integrated navigation system. Int Hydrogr Rev 73(1):75–100, March 1996Google Scholar
- Hewitson S, Wang J (2006) GNSS receiver autonomous integrity monitoring (RAIM) performance analysis. GPS Solut 10:155–170CrossRefGoogle Scholar
- Jonkman N, de Jong K (2000) Integrity monitoring of IGEX-98 data, part II: cycle slip and outlier detection. GPS Solut 3(4):24–34CrossRefGoogle Scholar
- Kok JJ, NOAA, National Oceanic, Atmospheric Administration (1984) On data snooping and multiple outlier testing. NOAA technical report NOS NGS:30;1984, U.S. Department of Commerce, National Oceanic and Atmospheric Administration (NOAA). https://books.google.com.au/books?id=tJqqMQEACAAJ
- Kösters AJM, van der Marel H (1990) Statistical testing and quality analysis of 3-D networks (parts I and II). In: Bock Y, Leppard N (eds) Global positioning system: an overview, vol 102. International Association of Geodesy Symposia, Springer, New YorkGoogle Scholar
- Kuusniemi H, Lachapelle G, Takala JH (2004) Position and velocity reliability testing in degraded GPS signal environments. GPS Solut 8(4):226–237CrossRefGoogle Scholar
- Perfetti N (2006) Detection of station coordinate discontinuities within the Italian GPS Fiducial Network. J Geodesy 80(7):381–396CrossRefGoogle Scholar
- Salós D, Macabiau C, Martineau A, Bonhoure B, Kubrak D (2010) Analysis of GNSS integrity requirements for road user charging applications. In: Satellite navigation technologies and European workshop on GNSS signals and signal processing (NAVITEC), 2010 5th ESA Workshop on. IEEE, 2010, IEEE, pp 1–8Google Scholar
- Schuster W, Bai J, Feng S, Ochieng W (2007) Airport surface movement–performance requirements, architecture considerations and integrity algorithms. In: Proc. of ION GNSS 2007, ION, pp 2950–2961Google Scholar
- Teunissen PJG (1990a) An integrity and quality control procedure for use in multi-sensor integration. In: Proc. of ION GPS 1990, ION, pp 513–522Google Scholar
- Teunissen PJG (1990b) Quality control in integrated navigation systems. IEEE Aerosp Electron Syst Mag 5(7):35–41CrossRefGoogle Scholar
- Teunissen PJG (2000) Testing theory: an introduction. Series on Mathematical Geodesy. Delft University Press, DelftGoogle Scholar
- Teunissen PJG (2017) Distributional theory for the DIA method. J Geodesy. https://doi.org/10.1007/s00190-017-1045-7 CrossRefGoogle Scholar
- Teunissen PJG, Imparato D, Tiberius CCJM (2017) Does RAIM with correct exclusion produce unbiased positions? Sensors 17(7):1508CrossRefGoogle Scholar
- Verhoef HME, de Heus HM (1995) On the estimation of polynomial breakpoints in the subsidence of the Groningen gasfield. Surv Rev 33(255):17–30CrossRefGoogle Scholar
- Wieser A (2004) Reliability checking for GNSS baseline and network processing. GPS Solut 8(3):55–66CrossRefGoogle Scholar
- Wu Y, Wang J, Jiang Y (2013) Advanced receiver autonomous integrity monitoring (ARAIM) schemes with GNSS time offsets. Adv Space Res 52:52–61CrossRefGoogle Scholar
- Yang L, Li Y, Wu Y, Rizos C (2014) An enhanced MEMS-INS/GNSS integrated system with fault detection and exclusion capability for land vehicle navigation in urban areas. GPS Solut 18:593–603CrossRefGoogle Scholar
- Yavaşoğlu HH, Kalkan Y, Tiryakioğlu İ, Yigit CO, Özbey V, Alkan MN, Bilgi S, Alkan RM (2017) Monitoring the deformation and strain analysis on the Ataturk Dam, Turkey. Geomat Nat Hazards Risk 9(1):94–107CrossRefGoogle Scholar

## Copyright information

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.