# Resampling methods to reduce the selection bias in genetic effect estimation in genome-wide scans

## Abstract

Using the simulated data of Problem 2 for Genetic Analysis Workshop 14 (GAW14), we investigated the ability of three bootstrap-based resampling estimators (a shrinkage, an out-of-sample, and a weighted estimator) to reduce the selection bias for genetic effect estimation in genome-wide linkage scans. For the given marker density in the preliminary genome scans (7 cM for microsatellite and 3 cM for SNP), we found that the two sets of markers produce comparable results in terms of power to detect linkage, localization accuracy, and magnitude of test statistic at the peak location. At the locations detected in the scan, application of the three bootstrap-based estimators substantially reduced the upward selection bias in genetic effect estimation for both true and false positives. The relative effectiveness of the estimators depended on the true genetic effect size and the inherent power to detect it. The shrinkage estimator is recommended when the power to detect the disease locus is low. Otherwise, the weighted estimator is recommended.

## Keywords

Genetic Effect Bootstrap Replication Weighted Estimator Genetic Analysis Workshop Shrinkage Estimator## Abbreviations

- GAW
Genetic Analysis Workshop

- IBD
Identity by descent

- SNP
Single-nucleotide polymorphism

## Background

After a genetic marker or candidate gene has been identified from a genome-wide scan as a putative disease susceptibility locus, it is of interest to estimate the associated genetic effect on the related phenotype. However, locus-specific effect estimates are subject to upward selection bias because of stringent test criteria adopted in genome-wide scans. Göring et al. [1] formally raised this issue and argued that reliable locus-specific parameter estimates can only be obtained in an independent sample. Sun and Bull [2] proposed three resampling-based estimators that can be applied to the original sample at the location where the maximum test statistic exceeds a genome-wide significance criterion. They demonstrated effective bias reduction in analytic and simulation studies of a homogenous population with a single disease gene. In their simulation studies, they compared a catalog of resampling methods, including cross-validation and bootstrapping, and their results suggested that bootstrap methods perform best in terms of smaller mean squared error. Therefore, we focused on the bootstrap method in the current study.

The simulated data of Problem 2 for Genetic Analysis Workshop 14 (GAW14) provided a microsatellite marker map of 416 markers with a resolution of 7 cM and a denser single-nucleotide polymorphism (SNP) marker map of 917 markers with 3-cM density. The disease expression was under the influence of multiple genes in a complex manner. We compared performance of the two maps in multipoint linkage analysis in terms of power and localization accuracy. The main objective of this study was to further investigate the effectiveness of bootstrap resampling methods in reducing the bias of genetic effect estimates in genome-wide linkage scans. The new methods, were applied to both the microsatellite and SNP data for selected replicates. With the knowledge of the answers to the simulated data, we were able to investigate the performance of the new methods under stratification of true and false positives.

## Methods

To evaluate the power to detect linkage, we conducted multipoint analyses in all the 100 replicates using ALLEGRO [3]. There were four populations Aipotu (AI), Danacaa (DA), Karangar (KA), and New York (NY) in each replicate. AI, DA, and KA included only nuclear families, while NY had multigeneration extended pedigrees. Because some of the large NY families (size > 25 bits) required too much execution time to complete the analysis in a reasonable time, the NY population was excluded.

We adopted the exponential allele-sharing model of Kong and Cox [4] and used *Spair* as the scoring function for affected relatives. The genetic effect was measured by *δ* the excess identity-by-descent (IBD) allele-sharing parameter in this model. The genome-wide significance criterion was set to 2.2 × 10^{-5} [5], corresponding to a *Zlr* value of 4.09, where *Zlr* is the test statistic for linkage in the exponential model. In each replicate of the three populations, we identified all loci that met the significance criterion.

We implemented a simple bootstrapping method in this study. Suppose that the original dataset has *n* families; we repeatedly drew random samples of size *n* with replacement from it. In each bootstrap replication *b* (*b* = 1, ..., *B*), the selected families constitute the detection sample, and the remaining families (out-of-sample families) comprise the estimation sample, thus providing independence within each of the *B* resampling replications. To reduce the upward selection bias in the genetic effect estimates of δ we implemented three bootstrap-based estimators [2]: a shrinkage estimator defined by Open image in new window , an out-of-sample estimator Open image in new window , and a weighted estimator Open image in new window , where *ω* = 0.632 was analogous to Efron's 0.632 estimator [6].

*m*

_{ D }, where the maximum test statistic exceeded the significance criterion in the original data. Note that the location

*m*

_{ D }was the overall gene localization and the three bootstrap-based estimators were then applied only to genetic effect estimation at this location. The shrinkage estimator was constructed by reducing the naïve estimate Open image in new window by a shrinkage factor of Open image in new window , which was constructed by taking the average of the difference between Open image in new window and Open image in new window over

*B** bootstrap replications, with

*B**≤

*B*, where

*B**is the number of replications with significant results. In bootstrap replication

*b*, Open image in new window is the genetic effect estimate at location Open image in new window with the maximum significant genome-wide test statistic in the detection sample; Open image in new window is the genetic effect estimate at the same location Open image in new window in the estimation sample. Note that Open image in new window could be different from

*m*

_{ D }. The out-of-sample estimator was the average of Open image in new window at location Open image in new window in the estimation sample over

*B** bootstrap replications. It resembles the estimate that would have been obtained in an independent sample. The weighted estimator combined Open image in new window and Open image in new window with the weight of

*ω*. The weight was chosen to be 0.632, which was derived from a distance argument based on the fact that bootstrap samples are supported by about 0.632

*n*of the original families [6, 7]. Note that the weighted estimator can also be written as Open image in new window . Therefore, it can be considered as a variant of shrinkage estimator, with the amount of shrinkage depending on

*ω*and Open image in new window . Although an adaptive choice of the weight is attractive, as in the 0.632+ method [7], time constraints precluded its inclusion in this study.

Bias reduction of the three estimators was compared according to whether the localization was a true or false positive. We classified significant findings in the 100 replicates into true or false positives, according to the answers (disease loci D1 and D2 on chromosomes 1 and 3 for the AI, KA, and DA populations, and disease loci D3 and D4 on chromosomes 5 and 9 for AI and KA). A true positive was defined if the detection was within 10 cM of the true disease gene location. The true genetic effects were estimated by averaging corresponding estimates from all 100 replicates.

## Results and discussion

Comparison of linkage analysis results between microsatellite and SNP based genome scans

Location estimates (mode) | Power to detect linkage | Mean test statistic | ||||||
---|---|---|---|---|---|---|---|---|

Pop | Chr. | True location (cM) | MS (cM) | SNP (cM) | MS | SNP | MS | SNP |

AI | 1 | 168.98 | 169.85 | 167.4 | 4/100 | 7/100 | 4.33 | 4.56 |

3 | 299.32 | 293.61 | 295.6 | | | 4.79 | 4.77 | |

5 | 5.45 | 7.34 | 5.94 | 11/100 | 11/100 | 4.53 | 4.63 | |

9 | 5.88 | 4.78 | 5.54 | 10/100 | 9/100 | 4.39 | 4.38 | |

KA | 1 | 168.98 | 169.97 | 168.0 | 2/100 | 2/100 | 4.07 | 4.24 |

3 | 299.32 | 293.75 | 295.9 | | | 4.41 | 4.77 | |

5 | 5.45 | 7.01 | 5.69 | 20/100 | 40/100 | 4.67 | 4.71 | |

9 | 5.88 | 6.80 | 5.96 | | | 4.95 | 4.92 | |

DA | 1 | 168.98 | 169.95 | 168.8 | | | 5.20 | 5.43 |

3 | 299.32 | 293.62 | 295.7 | 48/100 | 56/100 | 4.89 | 4.77 |

Comparison of linkage analysis results and genetic effect estimates for the naïve and three bootstrap estimates using microsatellite markers and SNPs.

Bootstrap estimate (bias) | |||||||||
---|---|---|---|---|---|---|---|---|---|

Replicate | Population | Chromosome | Highest peak (cM) | True genetic effect | T/F positive | ||||

Microsatellite Markers | |||||||||

1 | DA | 1 | 169.97 | 0.49 ± 0.10 | T | 0.55 (0.06) | 0.50 (0.01) | 0.48 (-0.01) | 0.47 (-0.03) |

1 | AI | 3 | 294.68 | 0.33 ± 0.16 | T | 0.43 (0.10) | 0.31 (-0.01) | 0.25 (-0.08) | 0.19 (-0.13) |

1 | KA | 9 | 2.76 | 0.43 ± 0.11 | T | 0.49 (0.06) | 0.40 (-0.04) | 0.34 (-0.09) | 0.24 (-0.19) |

35 | DA | 6 | 192.09 | -0.01 ± 0.11 | F | 0.41 (0.42) | 0.26 (0.27) | 0.17 (0.18) | 0.10 (0.11) |

SNP Markers | |||||||||

1 | DA | 1 | 168.94 | 0.53 ± 0.10 | T | 0.53 (0.00) | 0.49 (-0.04) | 0.46 (-0.07) | 0.43 (-0.10) |

1 | AI | 3 | 304.58 | 0.33 ± 0.11 | T | 0.41 (0.08) | 0.25 (-0.08) | 0.16 (-0.17) | 0.07 (-0.26) |

1 | KA | 3 | 305.81 | 0.29 ± 0.11 | T | 0.57 (0.28) | 0.48 (0.19) | 0.43 (0.14) | 0.33 (0.04) |

27 | DA | 9 | 200.12 | 0.02 ± 0.13 | F | 0.45 (0.43) | 0.32 (0.30) | 0.24 (0.22) | 0.16 (0.14) |

67 | DA | 5 | 214.33 | -0.01 ± 0.11 | F | 0.42 (0.43) | 0.27 (0.28) | 0.18 (0.19) | 0.11 (0.12) |

The bootstrap-based estimators reduced the upward selection bias in genetic effect estimation for both microsatellite and SNP based linkage analysis. The performance of the three estimators differed according to true or false positive status. The shrinkage estimator had the smallest bias for false positives but over-corrected for the true positives. On the other hand, the weighted estimator had the smallest bias for true positives but under-corrected for the false positives. It has been shown that the bias depends on the power to detect linkage [2]. In these examples from the simulated data, we found that the shrinkage estimator had lower bias when the power was less than 20%. Otherwise, the weighted estimator provided lower bias.

In this study, our bootstrap estimators focused on genetic effect estimation for the most significant locus in a genome scan, without considering other loci that also exceeded genome-wide significance criteria. However, the underlying genetic model has multiple loci. Further research is warranted to construct a joint estimator that would simultaneously handle multiple significant loci and thereby extend bias-reduction methods to more general settings.

## Conclusion

The reliability of gene detection, the accuracy of locus-specific effect estimates, and the failure to replicate initial claims of linkage or association have emerged as major concerns in genome-wide studies. Estimation of the genetic effect for a specific locus in a genome-wide scan is subject to upward bias because of selection by strict significance criteria. This bias is most severe for locations with small genetic effect and low power. Our results indicate that, in a complex disease setting, the three bootstrap-based estimators appear to be effective in reducing the selection bias of the naïve estimator. The shrinkage estimator is recommended when the power to detect the disease loci is low. Otherwise, the weighted estimator is recommended.

## Notes

### Acknowledgements

This research was supported by research grants from the Canadian Institutes of Health Research (CIHR) and the Network of Centres of Excellence in Mathematics (MITACS). LS and SBB also received support from the Natural Sciences and Engineering Research Council (Canada). SBB holds a CIHR Senior Investigator Award.

## References

- 1.Göring H, Terwilliger JD, Blangero J: Large upward bias in estimation of locus-specific effects from genomewide scans. Am J Hum Genet. 2001, 69: 1357-1369. 10.1086/324471.PubMedCentralCrossRefPubMedGoogle Scholar
- 2.Sun L, Bull SB: Reduction of selection bias in genomewide studies by resampling. Genet Epidemiol. 2005, 28: 352-367. 10.1002/gepi.20068.CrossRefPubMedGoogle Scholar
- 3.Gudbjartsson DF, Jonasson K, Frigge M, Kong A: Allegro, a new computer program for multipoint linkage analysis. Nat Genet. 2000, 25: 12-13. 10.1038/75514.CrossRefPubMedGoogle Scholar
- 4.Kong A, Cox NJ: Allele-sharing models: LOD scores and accurate linkage tests. Am J Hum Genet. 1997, 61: 1179-1188. 10.1086/301592.PubMedCentralCrossRefPubMedGoogle Scholar
- 5.Lander E, Kruglyak L: Genetic dissection of complex traits guidelines for interpreting and reporting linkage results. Nat Genet. 1995, 11: 241-247. 10.1086/301592.CrossRefPubMedGoogle Scholar
- 6.Efron B: Estimating the error rate of a prediction rule: some improvements on cross-validation. J Am Statist Assoc. 1983, 78: 316-331. 10.2307/2288636.CrossRefGoogle Scholar
- 7.Efron B, Tibshirani R: Improvements on cross-validation: the .632+ bootstrap method. J Am Statist Assoc. 1997, 92: 548-560. 10.2307/2965703.Google Scholar

## Copyright information

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.