Phylogeny, Evolution, and Ecology of Sexual Systems Across the Land Plants

  • Andrea L. CaseEmail author
  • Linley K. Jesson


Land plants dominate nearly every terrestrial habitat and include some of the largest and longest-lived organisms on earth. They are well known for their sexual diversity, reflecting tremendous variation in sex expression, and elaborate reproductive structures and behaviors. Much of what we understand about plant sexual diversity comes from studies of a single group—the flowering plants. Here, we discuss our current state of knowledge about sexual systems across the land plants and how principles and concepts derived from studies of angiosperms can (or cannot) be applied to mosses, hornworts, liverworts, ferns, fern allies, and gymnosperms. First, we show how variation in the expression and lability of sexual systems across the land-plant phylogeny raises fundamental questions about sexual-system evolution. Second, we discuss selective mechanisms, focusing specifically on polyploidy as a mechanism that may either constrain or facilitate evolutionary changes in sexual systems. Finally, we compare ecological traits that are commonly associated with alternate sexual systems in angiosperms and their (not so obvious) cognates in other land-plant groups.



The authors are grateful to Janet L. Leonard for inviting our contribution and to John R. Pannell, Sean Graham, and an anonymous reviewer for providing comments on earlier versions of this manuscript.


  1. Ashman TL, Kwok A, Husband BC (2013) Revisiting the dioecy-polyploidy association: alternate pathways and research opportunities. Cytogenet Genome Res 140(2–4):241–255CrossRefGoogle Scholar
  2. Bachtrog D, Kirkpatrick M, Mank JE, McDaniel SF, Chris Pires J, Rice W, Valenzuela N (2011) Are all sex chromosomes created equal? Trends Genet 27(9):350–357CrossRefGoogle Scholar
  3. Barker MS (2013) Karyotype and genome evolution in pteridophytes. In: Greilhuber J, Dolezel J, Wendel J (eds) Plant genome diversity, vol 2. Springer, Berlin, pp 245–253CrossRefGoogle Scholar
  4. Barrett SCH (2002) The evolution of plant sexual diversity. Nat Rev Genet 3:274–284CrossRefGoogle Scholar
  5. Bateman RM, Dimichele WA (1994) Heterospory: the most iterative key innovation in the evolutionary history of the plant kingdom. Biol Rev 69(3):345–417CrossRefGoogle Scholar
  6. Bateman RM, Dimichele WA (2002) Generating and filtering major phenotypic novelties: neoGoldschmidtian saltation revisited. In: Cronk QCB, Bateman RM, Hawkins JA (eds) Developmental genetics and plant evolution. London, Taylor and Francis, pp 109–159CrossRefGoogle Scholar
  7. Bateman RM, Hilton J, Rudall P (2011) Spatial separation and developmental divergence of male and female reproductive units in gymnosperms, and their relevance to the origin of the angiosperm flower. In: Wanntorp L, DeCraene LPR (eds) Flowers on the tree of life. Cambridge University Press, Cambridge, pp 8–48CrossRefGoogle Scholar
  8. Bisang I, Ehrlén J, Persson C, Hedenäs L (2014) Family affiliation, sex ratio and sporophyte frequency in unisexual mosses. Bot J Linn Soc 174:163–172CrossRefGoogle Scholar
  9. Blank CM, Levin RA, Miller JS (2014) Intraspecific variation in gender strategies in Lycium (Solanaceae): associations with ploidy and changes in floral form following the evolution of gender dimorphism. Am J Bot 101(12):2160–2168CrossRefGoogle Scholar
  10. Burleigh JG, Brad Barbazuk W, Davis JM, Morse AM, Soltis PS (2012) Exploring diversification and genome size evolution in extant gymnosperms through phylogenetic synthesis. J Bot 2012(3):1–6Google Scholar
  11. Cargill DC, Vella NGF, Sharma I, Miller JT (2013) Cryptic speciation and species diversity among Australian and New Zealand Hornwort Taxa of Megaceros (Dendrocerotaceae). Aust Syst Bot 26(5):356–377CrossRefGoogle Scholar
  12. Crawford M, Jesson LK, Garnock-Jones PJ (2009) Correlated evolution of sexual system and life-history traits in mosses. Evolution 63(5):1129–1142CrossRefGoogle Scholar
  13. Cronberg N, Natcheva R, Hedlund K (2006) Microarthropods mediate sperm transfer in Mosses. Science 313(5791):1255CrossRefGoogle Scholar
  14. Cronk Q (2009) Sporangium to seed. In: The molecular organography of plants. Oxford University Press, Oxford, pp 122–196CrossRefGoogle Scholar
  15. Delph L, Wolf DE (2005) Evolutionary consequences of gender plasticity in genetically dimorphic breeding systems. New Phytol 166:119–128CrossRefGoogle Scholar
  16. DeSoto L, Quintanilla LG, Méndez M (2008) Environmental sex determination in ferns: effects of nutrient availability and individual density in Woodwardia radicans. J Ecol 96(6):1319–1327CrossRefGoogle Scholar
  17. Devos N, Renner MAM, Gradstein R, Jonathan Shaw A, Laenen B, Vanderpoorten A (2011) Evolution of sexual systems, dispersal strategies and habitat selection in the Liverwort Genus Radula. New Phytol 192(1):225–236CrossRefGoogle Scholar
  18. Donoghue MJ (1989) Phylogenies and the analysis of evolutionary sequences, with examples from seed plants. Evolution 43(6):1137–1165CrossRefGoogle Scholar
  19. Duminil J, Hardy OJ, Petit RJ (2009) Plant traits correlated with generation time directly affect inbreeding depression and mating system and indirectly genetic structure. BMC Evol Biol 9(1):177CrossRefGoogle Scholar
  20. Durand R, Durand B (1992) Dioecy, monoecy, polyploidy and speciation in annual mercuries. Bull Soc Bot Fr Lett Bot 139:377–399Google Scholar
  21. Endress PK, Doyle JA (2015) Ancestral traits and specializations in the flowers of the basal grade of living angiosperms. Taxon 64:1093–1116CrossRefGoogle Scholar
  22. Engel JJ, Glenny D (2008) A flora of the liverworts and Hornworts of New Zealand, vol 1. Missouri Botanical Garden Press, St. LouisGoogle Scholar
  23. Eppley SM, Jesson LK (2008) Moving to mate: the evolution of separate and combined sexes in multicellular organisms. J Evol Biol 21(3):727–736CrossRefGoogle Scholar
  24. Friedman J, Barrett SCH (2009) Wind of change: new insights on the ecology and evolution of pollination and mating in wind-pollinated plants. Ann Bot 103(9):1515–1527CrossRefGoogle Scholar
  25. Haig D (2016) Living together, living apart: the sexual lives of bryophytes. Philos Trans B 371:20150535CrossRefGoogle Scholar
  26. Haufler CH (2014) Ever Since Klekowski: testing a set of radical hypotheses revives the genetics of ferns and lycophytes. Am J Bot 101(12):2036–2042CrossRefGoogle Scholar
  27. Heilbuth JC (2000) Lower species richness in dioecious clades. Am Nat 156(3):221–241CrossRefGoogle Scholar
  28. Heilbuth JC, Ilves KL, Otto SP (2001) The consequences of dioecy for seed dispersal: modeling the seed-shadow handicap. Evolution 55(5):880–888CrossRefGoogle Scholar
  29. Igic B, Lande R, Kohn JR (2008) Loss of self-incompatibility and its evolutionary consequences. Int J Plant Sci 169(1):93–104CrossRefGoogle Scholar
  30. Jesson LK, Garnock-Jones PJ (2012) Can classifications of functional gender be extended to all land plants? Perspect Plant Ecol Evol Syst 14(2):153–160CrossRefGoogle Scholar
  31. Jiao Y, Wickett NJ, Ayyampalayam S, Chanderbali AS, Landherr L, Ralph PE, Tomsho LP et al (2011) Ancestral polyploidy in seed plants and angiosperms. Nature 473(7345):97–100CrossRefGoogle Scholar
  32. Käfer J, de Boer HJ, Mousset S, Kool A, Dufäy M, Marais GAB (2014) Dioecy is associated with higher diversification rates in flowering plants. J Evol Biol 27(7):1478–1490CrossRefGoogle Scholar
  33. Klekowski EJ, Baker HG (1966) Evolutionary significance of polyploidy in the pteridophyta. Science 153(3733):305–307CrossRefGoogle Scholar
  34. Korpelainen H (1998) Labile sex expression in plants. Biol Rev 73:157–180CrossRefGoogle Scholar
  35. Laenen B, Antonin M, S Robbert Gradstein, Shaw B, Patiño J, Désamoré A, Goffinet B, Cox CJ, Jonathan Shaw A, Vanderpoorten A (2016) Increased diversification rates follow shifts to bisexuality in liverworts. New Phytol. CrossRefGoogle Scholar
  36. Leslie AB, Beaulieu JM, Crane PR, Donoghue MJ (2013) Explaining the distribution of breeding and dispersal syndromes in conifers. Proc R Soc Lond B Biol Sci 280(1770):20131812. CrossRefGoogle Scholar
  37. Lloyd DG (1982) Selection of combined versus separate sexes in seed plants. Am Nat 120(5):571–585CrossRefGoogle Scholar
  38. Longton RE, Schuster RM (1983) Reproductive biology. In: Schuster RM (ed) New manual of bryology. Hattori Botanical Laboratory, Nichinan, pp 386–463Google Scholar
  39. McDaniel SF, Willis JH, Jonathan Shaw A (2007) A linkage map reveals a complex basis for segregation distortion in an interpopulation cross in the moss Ceratodon purpureus. Genetics 176(4):2489–2500CrossRefGoogle Scholar
  40. McDaniel SF, Atwood J, Gordon Burleigh J (2012) Recurrent evolution of dioecy in bryophytes. Evolution 67(2):567–572CrossRefGoogle Scholar
  41. Miller JS, Venable DL (2000) Polyploidy and the evolution of gender dimorphism in plants. Science 289(5488):2335–2338CrossRefGoogle Scholar
  42. Miller JS, Venable DL (2002) The transition to gender dimorphism on an evolutionary background of self-incompatibility: an example from Lycium (Solanaceae). Am J Bot 89(12):1907–1915CrossRefGoogle Scholar
  43. Norrell TE, Jones KS, Payton AC, McDaniel SF (2014) Meiotic sex ratio variation in natural populations of Ceratodon purpureus (Ditrichaceae). Am J Bot 101(9):1572–1576CrossRefGoogle Scholar
  44. Obbard DJ, Harris SA, Buggs RJA, Pannell JR (2006) Hybridization, polyploidy, and the evolution of sexual systems in Mercurialis (Euphorbiaceae). Evolution 60(9):1801–1815CrossRefGoogle Scholar
  45. Obeso JR (2002) The costs of reproduction in plants. New Phytol 155(3):321–348CrossRefGoogle Scholar
  46. Otto SP, Whitton J (2000) Polyploid incidence and evolution. Annu Rev Genet 34(1):401–437CrossRefGoogle Scholar
  47. Pannell J (1997) Mixed genetic and environmental sex determination in an androdioecious population of Mercurialis annua. Heredity 78:50–56CrossRefGoogle Scholar
  48. Pannell JR, Obbard DJ, Buggs RJA (2004) Polyploidy and the sexual system: what can we learn from Mercurialis annua? Biol J Linn Soc 82(4):547–560CrossRefGoogle Scholar
  49. Perley DS, Jesson LK (2015) Hybridization is associated with changes in sexual system in the bryophyte genus Atrichum. Am J Bot 102(4):555–565CrossRefGoogle Scholar
  50. Ranker TA, Geiger JO (2008) Population genetics. In: Ranker TA, Haufler CH (eds) Biology and evolution of ferns and lycophytes. Cambridge University Press, Cambridge, pp 107–133CrossRefGoogle Scholar
  51. Renner SS (2014) The relative and absolute frequencies of angiosperm sexual systems: dioecy, monoecy, gynodioecy, and an updated online database. Am J Bot 101(10):1588–1596CrossRefGoogle Scholar
  52. Renner SS (2016) Pathways for making unisexual flowers and unisexual plants: moving beyond the ‘two mutations linked on one chromosome’ model. Am J Bot 1–3. CrossRefGoogle Scholar
  53. Ricca M, Shaw AJ (2010) Allopolyploidy and homoploid hybridization in the Sphagnum subsecundum Complex (Sphagnaceae: Bryophyta). Biol J Linn Soc 99(1):135–151CrossRefGoogle Scholar
  54. Rosenstiel TN, Shortlidge EE, Melnychenko AN, Pankow JF, Eppley SM (2012) Sex-specific volatile compounds influence microarthropod-mediated fertilization of moss. Nature 489(7416):431–433CrossRefGoogle Scholar
  55. Russell JRW, Pannell JR (2014) Sex determination in dioecious Mercurialis annua and its close diploid and polyploid relatives. Heredity 114(3):262–271CrossRefGoogle Scholar
  56. Sabath N, Goldberg EE, Click L, Einhorn M, Ashman T-L, Ming R, Otto SP, Vamosi JC, Mayrose I (2016) Dioecy does not consistently accelerate or slow lineage diversification across multiple genera of angiosperms. New Phytol 209:1290–1300CrossRefGoogle Scholar
  57. Sakai AK, Weller SG (1999) Gender and sexual dimorphism in flowering plants: a review of terminology, biogeographic patterns, ecological correlates, and phylogenetic approaches. In: Geber MA, Dawson TD, Delph LF (eds) Gender and sexual dimorphism in flowering plants. Springer, Berlin, pp 1–31Google Scholar
  58. Sauquet H, von Balthazar M, Magallón S, Doyle JA, Endress PK, Bailes EJ, de Morais EB, Bull-Hereñu K, Carrive L, Chartier M, Chomicki G, Coiro M, Cornette R, El Ottra JHL, Epicoco C, Foster CSP, Jabbour F, Haevermans A, Haevermans T, Hernández R, Little SA, Löfstrand S, Luna JA, Massoni J, Nadot S, Pamperl S, Prieu C, Reyes E, dos Santos P, Schoonderwoerd KM, Sontag S, Soulebeau A, Staedler Y, Tschan GF, Leung AW-S, Schönenberger J (2017) The ancestral flower of angiosperms and its early diversification. Nat Commun 8:16047. CrossRefPubMedPubMedCentralGoogle Scholar
  59. Scofield DG, Schultz ST (2006) Mitosis, stature and evolution of plant mating systems: low-phi and high-phi plants. Proc R Soc Lond B: Biol Sci 273(1584):275–282CrossRefGoogle Scholar
  60. Stark LR, Nicholas McLetchie D, Eppley SM (2010) Sex ratios and the shy male hypothesis in the moss Bryum argenteum (Bryaceae). Bryologist 113(4):788–797CrossRefGoogle Scholar
  61. Szovenyi P, Devos N, Weston DJ, Yang X, Hock Z, Shaw JA, Shimizu KK, McDaniel SF, Wagner A (2014) Efficient purging of deleterious mutations in plants with haploid selfing. Genome Biol Evol 6(5):1238–1252CrossRefGoogle Scholar
  62. Taylor PJ, Eppley SM, Jesson LK (2007) Sporophytic inbreeding depression in mosses occurs in a species with separate sexes but not in a species with combined sexes. Am J Bot 94(11):1853–1859CrossRefGoogle Scholar
  63. The Plant List (2013) Version 1.1. Published on the internet.
  64. Vamosi JC, Vamosi SM (2004) The role of diversification in causing the correlates of dioecy. Evolution 58(4):723–731CrossRefGoogle Scholar
  65. Vamosi JC, Otto SP, Barrett SCH (2003) Phylogenetic analysis of the ecological correlates of dioecy in angiosperms. J Evol Biol 16(5):1006–1018CrossRefGoogle Scholar
  66. Vega-Frutis R, Macías-Ordóñez R, Guevara R, Fromhage L (2014) Sex change in plants and animals: a unified perspective. J Evol Biol 27(4):667–675CrossRefGoogle Scholar
  67. Vilas JS, Pannell JR (2012) Do plants adjust their sex allocation and secondary sexual morphology in response to their neighbours? Ann Bot 110(7):1471–1478CrossRefGoogle Scholar
  68. Villarreal JC, Renner SS (2013) Correlates of monoicy and dioicy in hornworts, the apparent sister group to vascular plants. BMC Evol Biol 13:239CrossRefGoogle Scholar
  69. Wickett NJ, Mirarab S, Nguyen N, Warnow T, Carpenter E, Matasci N, Ayyampalayam S et al (2014) Phylotranscriptomic analysis of the origin and early diversification of land plants. Proc Natl Acad Sci USA 111(45):E4859–E4868CrossRefGoogle Scholar
  70. Yeung K, Miller JS, Savage AE, Husband BC, Igic B, Kohn JR (2005) Association of ploidy and sexual system in Lycium californicum (Solanaceae). Evolution 59(9):2048–2055CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Biological SciencesKent State UniversityKentUSA
  2. 2.Department of BiologyUniversity of New BrunswickFrederictonCanada

Personalised recommendations