Effect of a Single Nonzero Open Covariance on the Simple Closure Test

Abstract

In small samples the simple closure test is markedly, and rather disappointingly, insensitive to a nonzero open covariance if the number of variables, m, is small. Only trivial results will be obtained if m < 4. If m = 4 but one of the open variances is small relative to the others, the situation is little changed. Even if open variances do not differ greatly in size, or are homogeneous, there will be a complementary relationship between s_ij and s_kℓ if m = 4, so that if r_ij is incompatible with ρ_ij, r_kℓ may be incompatible with ρ_kℓ. Interactions weaken rapidly with increase in m. Experimental evidence reviewed suggests that if m ≥ 8 the test detects a single fairly strong nonzero open covariance with remarkable reliability, provided rather large samples (N > 30) are used. The probability that the null hypothesis will be rejected for the wrong reason, i.e., because the test points to nonrandom association between open variables known to be uncorrelated, is large if m is small but decreases rapidly with increase in m. Except when the open variances are homogeneous, however, it seems to be materially greater than expected even if m = 10, the largest value used in the work reported here.