Familial Cancer

, Volume 17, Issue 2, pp 295–302 | Cite as

A comparison of cosegregation analysis methods for the clinical setting

  • John Michael O. Rañola
  • Quanhui Liu
  • Elisabeth A. Rosenthal
  • Brian H. Shirts
Original Article


Quantitative cosegregation analysis can help evaluate the pathogenicity of genetic variants. However, genetics professionals without statistical training often use simple methods, reporting only qualitative findings. We evaluate the potential utility of quantitative cosegregation in the clinical setting by comparing three methods. One thousand pedigrees each were simulated for benign and pathogenic variants in BRCA1 and MLH1 using United States historical demographic data to produce pedigrees similar to those seen in the clinic. These pedigrees were analyzed using two robust methods, full likelihood Bayes factors (FLB) and cosegregation likelihood ratios (CSLR), and a simpler method, counting meioses. Both FLB and CSLR outperform counting meioses when dealing with pathogenic variants, though counting meioses is not far behind. For benign variants, FLB and CSLR greatly outperform as counting meioses is unable to generate evidence for benign variants. Comparing FLB and CSLR, we find that the two methods perform similarly, indicating that quantitative results from either of these methods could be combined in multifactorial calculations. Combining quantitative information will be important as isolated use of cosegregation in single families will yield classification for less than 1% of variants. To encourage wider use of robust cosegregation analysis, we present a website ( which implements the CSLR, FLB, and Counting Meioses methods for ATM, BRCA1, BRCA2, CHEK2, MEN1, MLH1, MSH2, MSH6, and PMS2. We also present an R package, CoSeg, which performs the CSLR analysis on any gene with user supplied parameters. Future variant classification guidelines should allow nuanced inclusion of cosegregation evidence against pathogenicity.


Bayes factor Counting meioses Likelihood ratio Linkage analysis Variants of uncertain significance 



This study was supported by the Damon Runyon Cancer Research Foundation (DRR-33-15), the National Human Genome Research Institute (R21HG008513), and the Fred Hutch/University of Washington Cancer Consortium (NCI 5P30 CA015704-39).

Supplementary material

10689_2017_17_MOESM1_ESM.docx (1.2 mb)
Supplementary material 1 (DOCX 1219 KB)


  1. 1.
    Rosenthal EA, Ranola JO, Shirts BH (2017) Power of pedigree likelihood analysis in extended pedigrees to classify rare variants of uncertain significance in cancer risk genes. Fam Cancer. doi:  10.1007/s10689-017-9989-6 Google Scholar
  2. 2.
    Plon SE et al (2008) Sequence variant classification and reporting: recommendations for improving the interpretation of cancer susceptibility genetic test results. Hum Mutat 29(11):1282–1291CrossRefPubMedPubMedCentralGoogle Scholar
  3. 3.
    Thompson D, Easton DF, Goldgar DE (2003) A full-likelihood method for the evaluation of causality of sequence variants from family data. Am J Hum Genet 73(3):652–655CrossRefPubMedPubMedCentralGoogle Scholar
  4. 4.
    Felix GE et al (2014) Germline mutations in BRCA1, BRCA2, CHEK2 and TP53 in patients at high-risk for HBOC: characterizing a Northeast Brazilian Population. Hum Genome Var 1:14012CrossRefPubMedPubMedCentralGoogle Scholar
  5. 5.
    De Nicolo A et al (2009) Multimodal assessment of protein functional deficiency supports pathogenicity of BRCA1 p.V1688del. Cancer Res 69(17):7030–7037CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Tischkowitz M et al (2008) Pathogenicity of the BRCA1 missense variant M1775K is determined by the disruption of the BRCT phosphopeptide-binding pocket: a multi-modal approach. Eur J Hum Genet 16(7):820–832CrossRefPubMedPubMedCentralGoogle Scholar
  7. 7.
    Johnston JJ et al (2012) Secondary variants in individuals undergoing exome sequencing: screening of 572 individuals identifies high-penetrance mutations in cancer-susceptibility genes. Am J Hum Genet 91(1):97–108CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Arnold S et al (2009) Classifying MLH1 and MSH2 variants using bioinformatic prediction, splicing assays, segregation, and tumor characteristics. Hum Mutat 30(5):757–770CrossRefPubMedPubMedCentralGoogle Scholar
  9. 9.
    Jarvik GP, Browning BL (2016) Consideration of cosegregation in the pathogenicity classification of genomic variants. Am J Hum Genet 98(6):1077–1081CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Richards S et al (2015) Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med 17(5):405–424CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Mohammadi L et al (2009) A simple method for co-segregation analysis to evaluate the pathogenicity of unclassified variants; BRCA1 and BRCA2 as an example. BMC Cancer 9:211CrossRefPubMedPubMedCentralGoogle Scholar
  12. 12.
    Ranola JO, Shirts BH (2016) CoSeg R package. Available from: Accessed 6 July 2017
  13. 13.
    Ranola JO, Shirts BH (2017) Analyze my variant website. Available from: Accessed 6 July 2017
  14. 14.
    United Nations, Department of Economic and Social Affairs. Population Division (1994) World population prospects: the 1998 revision. Volume I: comprehensive. United Nations,New York, p 627Google Scholar
  15. 15.
    United Nations, Department of Economic and Social Affairs. Population Division (1999) World population prospects: the 1998 revision. Volume II: sex and age. United Nations, New York, p.883Google Scholar
  16. 16.
    Bell FC, Wade AH, Goss SC (1992) Life tables for the United States, Social Security area: 1900–2080. Actuarial study. U.S. Social Security Administration, Office of the Actuary, Baltimore, p 106Google Scholar
  17. 17.
    Shirts BH et al (2016) Improving performance of multigene panels for genomic analysis of cancer predisposition. Genet Med 18(10):974–981CrossRefPubMedGoogle Scholar
  18. 18.
    van der Kolk DM et al (2010) Penetrance of breast cancer, ovarian cancer and contralateral breast cancer in BRCA1 and BRCA2 families: high cancer incidence at older age. Breast Cancer Res Treat 124(3):643–651CrossRefPubMedGoogle Scholar
  19. 19.
    Quehenberger F, Vasen HF, van Houwelingen HC (2005) Risk of colorectal and endometrial cancer for carriers of mutations of the hMLH1 and hMSH2 gene: correction for ascertainment. J Med Genet 42(6):491–496CrossRefPubMedPubMedCentralGoogle Scholar
  20. 20.
    Wei EK et al (2009) Cumulative risk of colon cancer up to age 70 years by risk factor status using data from the Nurses’ Health Study. Am J Epidemiol 170(7):863–872CrossRefPubMedPubMedCentralGoogle Scholar
  21. 21.
    Cottingham RW, Idury RM, Schäffer AA (1993) Faster sequential genetic linkage computations. Am J Hum Genet 53(1):252–263PubMedPubMedCentralGoogle Scholar
  22. 22.
    Schäffer AA et al (1994) Avoiding recomputation in linkage analysis. Hum Hered 44(4):225–237CrossRefPubMedGoogle Scholar
  23. 23.
    Lathrop GM, Lalouel JM (1984) Easy calculations of lod scores and genetic risks on small computers. Am J Hum Genet 36(2):460–465PubMedPubMedCentralGoogle Scholar
  24. 24.
    Lathrop GM et al (1984) Strategies for multilocus linkage analysis in humans. Proc Natl Acad Sci USA 81(11):3443–3446CrossRefPubMedPubMedCentralGoogle Scholar
  25. 25.
    Mohammadi L et al (2016) CSLR Website. Available from: Accessed 6 July 2017
  26. 26.
    Eggington JM et al (2014) A comprehensive laboratory-based program for classification of variants of uncertain significance in hereditary cancer genes. Clin Genet 86(3):229–237CrossRefPubMedGoogle Scholar
  27. 27.
    Murray ML et al (2011) Follow-up of carriers of BRCA1 and BRCA2 variants of unknown significance: variant reclassification and surgical decisions. Genet Med 13(12):998–1005CrossRefPubMedGoogle Scholar
  28. 28.
    de la Hoya M et al (2016) Combined genetic and splicing analysis of BRCA1 c.[594-2A > C; 641A > G] highlights the relevance of naturally occurring in-frame transcripts for developing disease gene variant classification algorithms. Hum Mol Genet 25(11):2256–2268CrossRefPubMedPubMedCentralGoogle Scholar
  29. 29.
    Tricarico R et al (2017) Assessment of the InSiGHT interpretation criteria for the clinical classification of 24 MLH1 and MSH2 gene variants. Hum Mutat 38(1):64–77CrossRefPubMedGoogle Scholar
  30. 30.
    Thompson BA et al (2013) A multifactorial likelihood model for MMR gene variant classification incorporating probabilities based on sequence bioinformatics and tumor characteristics: a report from the Colon Cancer Family Registry. Hum Mutat 34(1):200–209CrossRefPubMedGoogle Scholar
  31. 31.
    Whiley PJ et al (2011) Splicing and multifactorial analysis of intronic BRCA1 and BRCA2 sequence variants identifies clinically significant splicing aberrations up to 12 nucleotides from the intron/exon boundary. Hum Mutat 32(6):678–687CrossRefPubMedPubMedCentralGoogle Scholar
  32. 32.
    Bayrak-Toydemir P et al (2008) Likelihood ratios to assess genetic evidence for clinical significance of uncertain variants: hereditary hemorrhagic telangiectasia as a model. Exp Mol Pathol 85(1):45–49CrossRefPubMedGoogle Scholar
  33. 33.
    Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160CrossRefPubMedGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  • John Michael O. Rañola
    • 1
  • Quanhui Liu
    • 2
  • Elisabeth A. Rosenthal
    • 3
  • Brian H. Shirts
    • 1
  1. 1.Department of Laboratory MedicineUniversity of WashingtonSeattleUSA
  2. 2.Department of BioengineeringUniversity of WashingtonSeattleUSA
  3. 3.Department of Medicine, Division of Medical GeneticsUniversity of WashingtonSeattleUSA

Personalised recommendations