Sometimes Size Does Not Matter

Abstract

Recently Díaz, Hössjer and Marks (DHM) presented a Bayesian framework to measure cosmological tuning (either fine or coarse) that uses maximum entropy (maxent) distributions on unbounded sample spaces as priors for the parameters of the physical models (https://doi.org/10.1088/1475-7516/2021/07/020). The DHM framework stands in contrast to previous attempts to measure tuning that rely on a uniform prior assumption. However, since the parameters of the models often take values in spaces of infinite size, the uniformity assumption is unwarranted. This is known as the normalization problem. In this paper we explain why and how the DHM framework not only evades the normalization problem but also circumvents other objections to the tuning measurement like the so called weak anthropic principle, the selection of a single maxent distribution and, importantly, the lack of invariance of maxent distributions with respect to data transformations. We also propose to treat fine-tuning as an emergence problem to avoid infinite loops in the prior distribution of hyperparameters (common to all Bayesian analysis), and explain that previous attempts to measure tuning using uniform priors are particular cases of the DHM framework. Finally, we prove a theorem, explaining when tuning is fine or coarse for different families of distributions. The theorem is summarized in a table for ease of reference, and the tuning of three physical parameters is analyzed using the conclusions of the theorem.

Appendices

Appendix 1: Entropy and Maximum Entropy Distributions

Suppose X is a discrete random variable defined on a countable sample space $\Omega = \{x_1,x_2,\ldots \}$ with distribution F. Then the entropy of F is

$$\begin{aligned} H(F) = -\sum _i \ln (\pi _i)\pi _i, \end{aligned}$$

(14)

with $\pi _i=F(\{x_i\})$.

For a continuous random variable X, defined on a subset $\Omega$ of ${\mathbb {R}}^+$, with density function $f=F^\prime$, the entropy H(F) is not defined. However, for each $\delta >0$, the distribution F of X can be approximated by a discrete distribution $F_\delta$ such that $F_\delta (\{x_i\}) = F(I_i)$ for $i=1,2,\ldots$, where $I_i=((i-1)\delta ,i\delta ]$ and $x_i$ is the mid point of $I_i$. If $\delta > 0$ is small, we have approximately

$$\begin{aligned} H(F_\delta ) \approx -\int _\Omega \ln [f(x)]f(x)dx + \ln [\delta ^{-1}] = H_c(F) + \ln [\delta ^{-1}]. \end{aligned}$$

Although $H(F_\delta ) \rightarrow \infty$ as $\delta \rightarrow 0$, we may use

$$\begin{aligned} H_c(F) = \lim _{\delta \rightarrow 0} \left[ H(F_\delta ) - \ln \left( \delta ^{-1}\right) \right] = -\int _\Omega \ln [f(x)]f(x)dx \end{aligned}$$

(15)

as a continuous analogue of the entropy. The motivation of $H_c$ as a continuous analogue of the entropy is similar for other unbounded sample spaces $\Omega$, such as ${\mathbb {R}}$, ${\mathbb {R}}^+\times {\mathbb {R}}$, and ${\mathbb {R}}^2$.

The goal is to find a distribution F that maximizes $H_c(F)$ subject to a constraint $\int _\Omega f(x)dx = 1$ and the d additional moment constraints $E[M_i(X)]=\int _\Omega M_i(x) f(x)dx=\theta _i$ for $i=1,\ldots ,d$. In particular, ordinary moment constraints correspond to choosing $M_i(x)=x^i$ as polynomials of x of various orders i. The distribution F with maximal entropy $H_c$, subject to these n constraints, has a density function

$$\begin{aligned} f(x) = \frac{\exp (\lambda _1 M_1(x)+\ldots + \lambda _d M_d(x))}{Z(\lambda _1,\ldots ,\lambda _d)}, \end{aligned}$$

(16)

where $\lambda _1,\ldots ,\lambda _d$ are Lagrange multipliers chosen to satisfy the moment constraints, whereas Z is a normalizing partition function, chosen so that f integrates to 1.

Though this is the most common way to find maximum entropy distributions, a more general approach is possible that finds maximum entropy distributions, even in some cases for which Lagrange multipliers do not work [58]. The details go beyond the scope of this paper.

Appendix 2: Conditional Probability

This section shows how the formal construction of conditional expectation avoids problem with infinity.

In the context of tuning, suppose we have two constants of nature, $X,Y\in {\mathbb {R}}^+$. We want to find the conditional distribution $x\rightarrow F_{X \mid Y}(x) = P(X\le x \mid Y=y)$ of X given an observed value y of Y. This can be formulated as a conditional probability $P(A \mid G) = P(A \cap G)/P(G)$, where A and G are the sets of outcomes for which $X\le x$ and $Y=y$ respectively. However, this formula does not work when Y has a continuous distribution and $P(G)=0$. Given this limitation of the classical notion of conditional probability, a more general definition of conditional probability is needed in order to find the conditional distribution of X given Y. Given this, “the whole point of [the measure-theoretic treatment of conditional probability] is the systematic development of a notion of conditional probability that covers conditioning with respect to events of probability 0. This is accomplished by conditioning with respect to collections of events—that is, with respect to $\sigma$-fields ${\mathcal {G}}$” [59].

The formal definition of conditional probability is as follows: Given a probability space $(\Sigma ,{{\mathcal {H}}},P)$, a set $A\in {{\mathcal {H}}}$, and and a $\sigma$-field ${\mathcal {G}} \subset {\mathcal {H}}$, there exists a function f (whose existence is guaranteed by the Radon-Nikodym theorem), ${\mathcal {G}}$-measurable and integrable with respect to P, such that $P(A \cap G) = \nu (G) = \int _G fdP$ for all $G \in {\mathcal {G}}$. This function f can be conveniently noted as $P(A \mid {\mathcal {G}})$. The function $P(A \mid {\mathcal {G}})$ thus has two properties that define it:

• (i) $P(A \mid {\mathcal {G}})$ is ${\mathcal {G}}$-measurable and integrable.

• (ii) $P(A \mid {\mathcal {G}})$ satisfies the functional equation

  $$\begin{aligned} \int _G P(A \mid {\mathcal {G}}) dP = P(A \cap G), \ \ \ \ \ \ \ G \in {\mathcal {G}}. \end{aligned}$$

The important point here is that writing f as $P(A \mid {\mathcal {G}})$ is just a notational convenience that can be interpreted as follows: when ${\mathcal {G}}$ is generated by a partition ${\mathcal {P}} = \{G_1, G_2 \ldots \}$ of $\Sigma$, conditioning on ${\mathcal {G}}$ can thus be seen as performing an experiment whose outcome

$$\begin{aligned} f(x) = P(A \mid {\mathcal {G}})(x) = \sum _{i=1}^\infty P(A \mid G_i) 1(x\in G_i) \end{aligned}$$

will determine which event $G_i$ of the partition occurred.¹⁴ In general, there is a whole family of random variables satisfying properties (i) and (ii). Such random variables are equal with probability 1 (i.e., they can only be different in a set of zero probability); for this reason, each member of the family is called a version of the others. Thus, $P(A \mid {\mathcal {G}})$ stands for any member of this family. Therefore, if $P(G_i) = 0$ for some nonempty $G_i \in {\mathcal {P}}$, a constant value c must be selected to make $P(A \mid {\mathcal {G}}) = c$ on $G_i$. Regardless of the choice of $c\in [0,1]$, $P(A \mid {\mathcal {G}})$ can still be considered a probability measure on $(\Sigma ,{{\mathcal {H}}})$ that assigns probabilities to all $A\in {{\mathcal {H}}}$. That is, a version of the conditional probability is selected for any value of c.

However, this interpretation for the notation $P(A \mid {\mathcal {G}})$ does not hold when ${\mathcal {G}}$ is not generated by a partition of $\Sigma$, which in our motivating example corresponds to Y having a continuous distribution. Nonetheless, even though the intuition of the conditional probability as the realization of an experiment, with an outcome in ${\mathcal {P}} = \{G_1,G_2,\ldots \}$, is gone, the mathematical formalism stands, independently of the conditioning $\sigma$-field. That is, there still exists a family of functions $\{f\}$ satisfying properties (i) and (ii). As in any area of mathematics, problems would arise when dividing by zero, but the formalism permits to circumvent this situation by selecting a well-defined version of the conditional probability.

Appendix 3: Upper Bounds for Fine-Tuning Probabilities

Let $x_{\text {obs}} \ne 0$ be the observed value of a constant of nature $X\in \Omega$ (or a ratio X of two constants of nature), where $\Omega$ is the sample space. The prior distribution of X has a density $f=F^\prime$ that belongs a parametric family

$$\begin{aligned} {\mathcal {F}} = \{f(x;\theta ); \,\, \theta = (\theta _1,\ldots ,\theta _d)\in \Theta \}, \end{aligned}$$

(17)

with $\Theta \subset {\mathbb {R}}^d$ the parameter space to which the hyperparameter $\theta$ belongs. This framework is more general than (7), since (17) does not assume that the hyperparameters $\theta _i$ represent moment constraints $\theta _i=E[M_i(X)]$ of Appendix 1. Let

$$\begin{aligned} I = I_X = x_{\text {obs}} [1-\epsilon ,1-\epsilon ] \end{aligned}$$

(18)

be the life-permitting interval of X, with $\epsilon >0$ a small number that quantifies half the relative size of I. For a fixed $\theta$ the tuning probability is

$$\begin{aligned} \text {TP}(\theta ) = F(I;\theta ) = \int _I f(x;\theta ) dx. \end{aligned}$$

(19)

Our quantity of interest is a useful approximation of the upper bound

$$\begin{aligned} \text {TP}_{\text {max}} (I) = \max _{\theta \in \Theta } F(I;\theta ) \end{aligned}$$

(20)

of the tuning probability, assuming that the maximum in (20) is taken over all hyperparameter vectors that appear in (17).

Appendices 3.1–3.5 will give explicit approximations of the maximal tuning probability (20) for give different families ${{\mathcal {F}}}$ of prior distributions. Then in Table 1 of Appendix 3.6 the results are summarized.

1.1 Appendix 3.1: Scale Parameter for $\Omega = {\mathbb {R}}^+$

A scale-parameter corresponds to $d=1$ and $\theta =\sigma >0$, so that $\Theta = {\mathbb {R}}^+$. The prior density is

$$\begin{aligned} f(x;\sigma ) = \frac{1}{\sigma } g\left( \frac{x}{\sigma }\right) . \end{aligned}$$

(21)

A typical example is the family of exponential distributions ($g(x)=e^{-x}$). Since

$$\begin{aligned} F(I;\sigma ) \approx 2\epsilon \frac{x_\text {obs}}{\sigma } g\left( \frac{x_\text {obs}}{\sigma } \right) , \end{aligned}$$

(22)

it follows that

$$\begin{aligned} \begin{aligned} \text {TP}_{\text {max}}(I)&\le 2\epsilon \max _{x>0}\{xf(x)\}\\&= 2\epsilon C_1. \end{aligned} \end{aligned}$$

(23)

It is easily seen that $C_1=e^{-1}$ for the family of exponential distributions.

1.2 Appendix 3.2: Form and Scale Parameter for $\Omega = {\mathbb {R}}^+$

This scenario corresponds to $d=2$, $\theta =(\psi ,\sigma )$, where $\psi >0$ is the form parameter and $\sigma >0$ the scale parameter. Consequently

$$\begin{aligned} f(x;\psi ,\sigma ) = \frac{1}{\sigma } g\left( \frac{x}{\sigma };\psi \right) , \end{aligned}$$

(24)

where $g(\cdot ;\psi )$ is the density of the distribution with shape parameter $\psi$ and scale parameter 1. Typical examples are the family of gamma distributions ($g(x;\psi )=x^{\psi -1}e^{-x}/\Gamma (\psi )$) or the family of Weibull distributions ($g(x;\psi )=\psi x^{\psi -1}e^{-x^\psi }$). The expected value and variance of X are

$$\begin{aligned} E(X)&= \mu = h_1(\psi )\sigma ,\\ \text {Var}(X)&= h_2(\psi )\sigma ^2, \end{aligned}$$

respectively, for some functions $h_1$ and $h_2$ (for instance $h_1(\psi )=h_2(\psi )=\psi$ for the family of gamma distributions, whereas $h_1(\psi )=\Gamma (1+1/\psi )$ and $h_2(\psi )=\Gamma (1+2/\psi )-\Gamma ^2(1+1/\psi )$ for the family of Weibull distributions). Let $h(\psi )=h_1^2(\psi )/h_2(\psi )$. We consider a parameter space

$$\begin{aligned} \Theta = \left\{ (\psi ,\sigma ): \frac{E^2(X)}{\text {Var}(X)} = h(\psi ) \le S\right\} \end{aligned}$$

(25)

of hyperparameters such that the signal-to-noise ratio

$$\begin{aligned} \text {SNR} = \frac{E^2(X)}{\text {Var}(X)} \le S \end{aligned}$$

(26)

of the prior distribution is at most S. We will prove that

$$\begin{aligned} \text {TP}_{\text {max}} (I) \le 2\epsilon C_2\sqrt{S} \end{aligned}$$

(27)

for some constant $C_2$ (defined below), whenever S is large and

$$\begin{aligned} \epsilon \sqrt{S} \ll 1 \Longleftrightarrow S \ll \frac{1}{\epsilon ^2}. \end{aligned}$$

(28)

It follows that essentially, (28) is the condition for the upper bound $\text {TP}_{\text {max}} (I)$ of the tuning probability to be small.

Proof of (27). Equation (24) implies

$$\begin{aligned} F(I;\psi ,\sigma ) = F\left( \frac{I}{x_\text {obs}};\psi ,\frac{\sigma }{x_\text {obs}} \right) , \end{aligned}$$

from which

$$\begin{aligned} \text {TP}_{\text {max}} (I) = \text {TP}_{\text {max}} \left( \frac{I}{x_\text {obs}}\right) \end{aligned}$$

(29)

follows. Since $I/x_\text {obs} = [1-\epsilon ,1+\epsilon ]$, we assume, without loss of generality, that $x_\text {obs}=1$. With this choice of $x_\text {obs}$,

$$\begin{aligned} F(I;\psi ,\sigma ) \approx \frac{2\epsilon }{\sigma } g\left( \frac{1}{\sigma };\psi \right) . \end{aligned}$$

(30)

In view of (25) and (30),

$$\begin{aligned} \text {TP}_{\text {max}}(I)&= \max _{\begin{array}{c} \psi:h(\psi )\le S;\\ \sigma>0 \end{array}} F(I;\psi ,\sigma )\\&\approx 2\epsilon \left( \max _{\begin{array}{c} \psi:h(\psi )\le S;\\ \sigma>0 \end{array}} \left\{ \frac{1}{\sigma } g\left( \frac{1}{\sigma };\psi \right) \right\} \right) \\&= 2\epsilon \left( \max _{\begin{array}{c} \psi:h(\psi )\le S;\\ x >0 \end{array}} \left\{ x g(x;\psi )\right\} \right) \\&\le 2\epsilon C_2 \sqrt{S}. \end{aligned}$$

Moreover, it can be seen that

$$\begin{aligned} \begin{aligned} C_2&= \frac{1}{\sqrt{S}}\left( \max _{\begin{array}{c} \psi:h(\psi )\le S;\\ x >0 \end{array}} \{x g(x;\psi )\}\right) \\&\approx \frac{1}{\sqrt{S}} \cdot h_1\left( h^{-1}(S)\right) g\left( h_1\left( h^{-1}(S)\right) ;h^{-1}(S)\right) \end{aligned} \end{aligned}$$

(31)

when S is large, since the maximum of $x g(x;\psi )$ with respect to x approximately equals $E(X)g(E(X);\psi )$. For the gamma and Weibull families, we have that $g(x;\psi )$ is approximately a Gaussian density for large signal-to-noise ratios $h(\psi )$. This implies

$$\begin{aligned} g(x;\psi ) \approx \frac{1}{\sqrt{h_2(\psi )}}\cdot \phi \left( \frac{x-h_1(\psi )}{\sqrt{h_2(\psi )}}\right) \end{aligned}$$

(32)

when $h(\psi )$ is large, where $\phi (x)=e^{-x^2/2}/\sqrt{2\pi }$ is the density of a standard normal distribution. Inserting (32) into (31) we find (with $\psi _1 = h^{-1}(S)$) that

$$\begin{aligned} C_2 \approx \frac{1}{\sqrt{S}}\cdot h_1(\psi _1) \cdot \frac{1}{\sqrt{h_2(\psi _1)}} \phi \left( \frac{h_1(\psi _1)-h_1(\psi _1)}{\sqrt{h_2(\psi _1)}}\right) = \frac{1}{\sqrt{2\pi }}, \end{aligned}$$

where in the last step we used that $h_1(\psi _1)/\sqrt{h_2(\psi _1)}=\sqrt{S}$ and $\phi (0)=1/\sqrt{2\pi }$.

1.3 Appendix 3.3: Scale Parameter for $\Omega = {\mathbb {R}}$

This is the same kind of density as in (21), with $d=1$ and $\theta =\sigma >0$, so that $\Theta = {\mathbb {R}}^+$. But g is now a symmetric density defined on the whole real line. A typical example is the class of double exponential or Laplace distributions ($g(x)=e^{-\vert x\vert }/2$), the class of symmetric normal distributions ($g(x)=\phi (x)$) or the class of symmetric Cauchy distributions ($g(x)=1/\left[ \pi \left( 1+x^2\right) \right]$). It can be seen that

$$\begin{aligned} F(I;\sigma ) \approx 2\epsilon \frac{\vert x_\text {obs}\vert }{\sigma } g\left( \frac{x_\text {obs}}{\sigma } \right) , \end{aligned}$$

(33)

whereas (23) still holds.

1.4 Appendix 3.4: Location Parameter for $\Omega = {\mathbb {R}}$

A location parameter corresponds to $d=1$, $\theta =\mu$, $\Theta ={\mathbb {R}}$, and

$$\begin{aligned} f(x;\mu ) = g(x-\mu ). \end{aligned}$$

(34)

Typical examples are the family of normal distributions with fixed variance $\sigma ^2$ ($g(x)=\phi (x/\sigma )/\sigma$), and the family of shifted Cauchy distributions with a fixed scale $\sigma$ ($g(x)=1/\left[ \sigma \pi \left( 1+(x/\sigma )^2\right) \right]$). Since

$$\begin{aligned} F(I;\mu ) \approx 2\epsilon g(x_\text {obs} - \mu ), \end{aligned}$$

(35)

it follows that

$$\begin{aligned} \text {TP}_{\text {max}}(I)&\le 2\epsilon \max _{x\in {\mathbb {R}}} g(x) \end{aligned}$$

(36)

$$\begin{aligned}&= 2\epsilon C_3. \end{aligned}$$

(37)

1.5 Appendix 3.5: Location and Scale Parameters for $\Omega = {\mathbb {R}}$

The two-parameter location-scale family corresponds to $d=2$, $\theta =(\mu ,\sigma )$,

$$\begin{aligned} f(x;\mu ,\sigma ) = \frac{1}{\sigma } g\left( \frac{x-\mu }{\sigma }\right) , \end{aligned}$$

and

$$\begin{aligned} \Theta = \left\{ \theta =(\mu ,\sigma ): \mu \in {\mathbb {R}}, \sigma >0, \frac{\mu ^2}{\sigma ^2}\le S\right\} . \end{aligned}$$

(38)

If the first two moments of the prior exist and $X\sim g$ is standardized to have $E(X)=0$ and $\text {Var}(X)=1$, then (38) consists of all densities with a signal-to-noise ratio

$$\begin{aligned} \frac{\mu ^2}{\sigma ^2} \le S \end{aligned}$$

(39)

that is upper bounded by a pre-chosen number S, as in (26). A typical example is the family of normal distributions ($g(x)=\phi (x)$). On the other hand, if g corresponds to a Cauchy distribution ($g(x)=1/\left[ \pi \left( 1+x^2\right) \right]$) the first two moments of $X\sim g$ do not exist. Then $\mu ^2/\sigma ^2$ represents a generalized signal-to-noise ratio, which according to (39) is upper bounded by S. Below we prove that

$$\begin{aligned} \text {TP}_{\text {max}} (I)&\le 2\epsilon \left[ \sqrt{S} \left( \max _{x\in {\mathbb {R}}}g(x)\right) + \max _{x\in {\mathbb {R}}} \{\vert x\vert g(x)\}\right] \end{aligned}$$

(40)

$$\begin{aligned}&= 2\epsilon \left( C_3\sqrt{S} + C_1\right) , \end{aligned}$$

(41)

which holds whenever (28) is satisfied, which is also a condition for the upper bound of the tuning probability to be small. In the transition from (40) to (41) we assumed that g is symmetric, so that $C_1=\max _{x>0} \{x g(x)\}$, as in Appendix 3.1.

Proof of (40). By a change of variables, it is easily seen that

$$\begin{aligned} F(I;\mu ,\sigma ) = F\left( \frac{I}{x_\text {obs}};\frac{\mu }{x_\text {obs}},\frac{\sigma }{x_\text {obs}}\right) , \end{aligned}$$

from which it follows that (29) holds, and we may, without loss of generality, assume $x_\text {obs}=1$. This gives

$$\begin{aligned} \begin{aligned} F(I;\mu ,\sigma )&= \frac{1}{\sigma } \int _{1-\epsilon }^{1+\epsilon } g\left( \frac{x-\mu }{\sigma }\right) dx\\&= \int _{(1-\epsilon )/\sigma }^{(1+\epsilon )/\sigma } g\left( z-\frac{\mu }{\sigma }\right) dz\\&\approx \frac{2\epsilon }{\sigma } g\left( \frac{1-\mu }{\sigma }\right) , \end{aligned} \end{aligned}$$

(42)

where in the second step we substituted $z=x/\sigma$. We have that

$$\begin{aligned} \text {TP}_{\text {max}} (I) = \max _{\mu \in {\mathbb {R}}} \text {TP}_{\text {max}}(I;\mu ), \end{aligned}$$

(43)

where

$$\begin{aligned} \text {TP}_{\text {max}}(I;\mu )&= \max _{\sigma \ge \vert \mu \vert /\sqrt{S}} F(I;\mu ,\sigma )\nonumber \\&\approx \max _{\sigma \ge \vert \mu \vert /\sqrt{S}} \left\{ \frac{2\epsilon }{\sigma } g\left( \frac{1-\mu }{\sigma }\right) \right\} \nonumber \\&= 2\epsilon \cdot h\left( 1-\mu ,\frac{\vert \mu \vert }{\sqrt{S}}\right) \end{aligned}$$

(44)

whenever $\mu \ne 0$, and

$$\begin{aligned} \begin{aligned} h(x,\sigma _0)&= \max _{\sigma \ge \sigma _0} \left\{ \frac{1}{\sigma } g\left( \frac{x}{\sigma }\right) \right\} \\&= \max _{0<y\le \vert x\vert /\sigma _0} \left\{ \frac{y}{\vert x\vert } g(y\, \text{ sgn }(x))\right\} \\&= \frac{1}{\vert x\vert } \left( \max _{0<y\le \vert x\vert /\sigma _0} \{y g(y\, \text{ sgn }(x))\}\right) \\&\le \min \left\{ \frac{C_3}{\sigma _0},\frac{C_1}{\vert x\vert }\right\} . \end{aligned} \end{aligned}$$

(45)

Insertion of (45) into (44) yields

$$\begin{aligned} \text {TP}_{\text {max}} (I;\mu ) \le 2\epsilon \min \left\{ \frac{C_3 \sqrt{S}}{\vert \mu \vert },\frac{C_1}{\vert 1-\mu \vert }\right\} . \end{aligned}$$

(46)

Finally, inserting (46) into (43), we find that

$$\begin{aligned} \text {TP}_{\text {max}}(I)&\le 2\epsilon \max _{\mu \in {\mathbb {R}}} \left\{ \min \left\{ \frac{C_3 \sqrt{S}}{\vert \mu \vert },\frac{C_1}{\vert 1-\mu \vert }\right\} \right\} \end{aligned}$$

(47)

$$\begin{aligned}&= 2\epsilon (C_3 \sqrt{S} + C_1), \end{aligned}$$

(48)

which proves the desired result. In the last step of (47) we used that the maximum of (47) is obtained when $\mu$ is such that the two functions within the minimum operator have the same value.

1.6 Appendix 3.6: Summary of Results

The upper bounds of the tuning probability are summarized in the following table:

Table 1 Maximal tuning probabilities for diverse parametric families ${\mathcal {F}}$ of prior distributions, given certain constraints

The constants that appear in the rightmost column of the table are

$$\begin{aligned} \begin{aligned} C_1&= \max _{x>0}\{xg(x)\},\\ C_2&= 1/\sqrt{2\pi },\\ C_3&= \max _{x\in {\mathbb {R}}}g(x). \end{aligned} \end{aligned}$$

(49)

Remark 1

Each proof in this section is based on the supposition $\epsilon \ll 1$. This can be seen in (22), (30), (33), (35), and (42), since in all these equations the assumption was that $\epsilon$ was small enough to warrant that the prior density of X was constant over the life-permitting interval $I_X$.

Remark 2

The cases where $\text {TP}_{\text {max}} = 1$ in Table 1 are produced because it is possible that the distribution is highly concentrated inside the life-permitting interval $I_X$. For instance, this happens for the location-scale parameter of Appendix 3.5 when the scale parameter $\sigma$ converges to zero (or $S\rightarrow \infty$).

Remark 3

Since fine-tuning requires a non-degenerate random variable X, its variance must be positive. Therefore, suppose there exists $\sigma _0>0$ such that $\text {Var}(X) \ge \sigma _0^2$. This assumption implies that SNR$=E^2(X)/\text {Var}(X) \ne 0$ when $E(X)\ne 0$ in Table 1. For this scenario a sufficient condition for fine-tuning is $\epsilon /\sigma _0 \ll 1$, regardless of $\text{ SNR }$. However, the requirement $\text {Var}(X)\ge \sigma _0^2$ is not invariant with respect to scaling of X, and therefore less general than the dimensionless constraint $\text{ SNR }\le S<\infty$ (which also excludes degenerate priors). Even when the variance does not exist (as in the Cauchy distribution), the fact that a non-degenerate random variable is under scrutiny warrants that the scale parameter $\sigma$ must be positive ($\sigma \ge \sigma _0$), and then the analogous sufficient condition for fine-tuning applies.