Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A representation of exchangeable hierarchies by sampling from random real trees

  • 205 Accesses

  • 1 Citations

Abstract

A hierarchy on a set S, also called a total partition of S, is a collection \(\mathcal {H}\) of subsets of S such that \(S \in \mathcal {H}\), each singleton subset of S belongs to \(\mathcal {H}\), and if \(A, B \in \mathcal {H}\) then \(A \cap B\) equals either A or B or \(\varnothing \). Every exchangeable random hierarchy of positive integers has the same distribution as a random hierarchy \(\mathcal {H}\) associated as follows with a random real tree \(\mathcal {T}\) equipped with root element 0 and a random probability distribution p on the Borel subsets of \(\mathcal {T}\): given \((\mathcal {T},p)\), let \(t_1,t_2, \ldots \) be independent and identically distributed according to p, and let \(\mathcal {H}\) comprise all singleton subsets of \({\mathbb {N}}\), and every subset of the form \(\{j:t_j \in F(x)\}\) as x ranges over \(\mathcal {T}\), where F(x) is the fringe subtree of \(\mathcal {T}\) rooted at x. There is also the alternative characterization: every exchangeable random hierarchy of positive integers has the same distribution as a random hierarchy \(\mathcal {H}\) derived as follows from a random hierarchy \({\mathscr {H}}\) on [0, 1] and a family \((U_j)\) of i.i.d. Uniform [0,1] random variables independent of \({\mathscr {H}}\): let \(\mathcal {H}\) comprise all sets of the form \(\{j:U_j \in B\}\) as B ranges over the members of \({\mathscr {H}}\).

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  1. 1.

    Aldous, D.: The continuum random tree. I. Ann. Probab. 19(1), 1–28 (1991)

  2. 2.

    Aldous, D.: The continuum random tree. II. An overview. In Stochastic analysis (Durham, 1990), volume 167 of London Math. Soc. Lecture Note Ser., pp. 23–70. Cambridge University Press, Cambridge (1991)

  3. 3.

    Aldous, D.: The continuum random tree. III. Ann. Probab. 21(1), 248–289 (1993)

  4. 4.

    Aldous, D.: Brownian excursions, critical random graphs and the multiplicative coalescent. Ann. Probab. 25(2), 812–854 (1997)

  5. 5.

    Aldous, D., Krikun, M., Popovic, L.: Stochastic models for phylogenetic trees on higher-order taxa. J. Math. Biol. 56(4), 525–557 (2008)

  6. 6.

    Aldous, D., Pitman, J.: A family of random trees with random edge lengths. Random Struct. Algorithms 15(2), 176–195 (1999)

  7. 7.

    Aldous, D.J.: Representations for partially exchangeable arrays of random variables. J. Multivar. Anal. 11(4), 581–598 (1981)

  8. 8.

    Aldous, D.J.: Exchangeability and related topics. In: École d’été de probabilités de Saint-Flour, XIII—1983, vol. 1117 of Lecture Notes in Math., pp. 1–198. Springer, Berlin (1985)

  9. 9.

    Aldous, D.J.: Stochastic models and descriptive statistics for phylogenetic trees, from Yule to today. Stat. Sci. 16(1), 23–34 (2001)

  10. 10.

    Austin, T.: On exchangeable random variables and the statistics of large graphs and hypergraphs. Probab. Surv. 5, 80–145 (2008)

  11. 11.

    Austin, T., Panchenko, D.: A hierarchical version of the de Finetti and Aldous–Hoover representations. Probab. Theory Relat. Fields 159(3–4), 809–823 (2014)

  12. 12.

    Berestycki, J.: Ranked fragmentations. ESAIM Probab. Stat. 6, 157–175 (2002). (electronic)

  13. 13.

    Berestycki, J., Berestycki, N., Schweinsberg, J.: Small-time behavior of beta coalescents. Ann. Inst. Henri Poincaré Probab. Stat. 44(2), 214–238 (2008)

  14. 14.

    Bertoin, J.: A fragmentation process connected to Brownian motion. Probab. Theory Relat. Fields 117(2), 289–301 (2000)

  15. 15.

    Bertoin, J.: Eternal additive coalescents and certain bridges with exchangeable increments. Ann. Probab. 29(1), 344–360 (2001)

  16. 16.

    Bertoin, J.: Self-similar fragmentations. Ann. Inst. H. Poincaré Probab. Stat. 38(3), 319–340 (2002)

  17. 17.

    Bertoin, J.: Random covering of an interval and a variation of Kingman’s coalescent. Random Struct. Algorithms 25(3), 277–292 (2004)

  18. 18.

    Bertoin, J.: Random Fragmentation and Coagulation Processes, Cambridge Studies in Advanced Mathematics, vol. 102. Cambridge University Press, Cambridge (2006)

  19. 19.

    Bertoin, J.: Homogeneous Multitype Fragmentations. In: In and out of equilibrium. 2, vol. 60 of Progr. Probab., pp. 161–183. Birkhäuser, Basel (2008)

  20. 20.

    Bertoin, J., Le Gall, J.-F.: Stochastic flows associated to coalescent processes. Probab. Theory Relat. Fields 126(2), 261–288 (2003)

  21. 21.

    Bertoin, J., Rouault, A.: Discretization methods for homogeneous fragmentations. J. Lond. Math. Soc. (2) 72(1), 91–109 (2005)

  22. 22.

    Blei, D.M., Griffiths, T.L., Jordan, M.I.: The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. J. ACM 57(2), 7 (2010)

  23. 23.

    Blundell, C., Teh, Y.W., Heller, K.A.: Bayesian rose trees. In: Proceedings of the International Conference on Uncertainty in Artificial Intelligence (2010)

  24. 24.

    Chen, B., Winkel, M.: Restricted exchangeable partitions and embedding of associated hierarchies in continuum random trees. Ann. Inst. Henri Poincaré Probab. Stat. 49(3), 839–872 (2013)

  25. 25.

    Diaconis, P., Freedman, D.: de Finetti’s theorem for Markov chains. Ann. Probab. 8(1), 115–130 (1980)

  26. 26.

    Diaconis, P., Freedman, D.: Finite exchangeable sequences. Ann. Probab. 8(4), 745–764 (1980)

  27. 27.

    Diaconis, P., Janson, S.: Graph limits and exchangeable random graphs. Rend. Mat. Appl. (7) 28(1), 33–61 (2008)

  28. 28.

    Donnelly, P., Evans, S.N., Fleischmann, K., Kurtz, T.G., Zhou, X.: Continuum-sites stepping-stone models, coalescing exchangeable partitions and random trees. Ann. Probab. 28(3), 1063–1110 (2000)

  29. 29.

    Donnelly, P., Joyce, P.: Consistent ordered sampling distributions: characterization and convergence. Adv. Appl. Probab. 23(2), 229–258 (1991)

  30. 30.

    Donnelly, P., Kurtz, T.G.: Particle representations for measure-valued population models. Ann. Probab. 27(1), 166–205 (1999)

  31. 31.

    Duquesne, T.: A limit theorem for the contour process of conditioned Galton–Watson trees. Ann. Probab. 31(2), 996–1027 (2003)

  32. 32.

    Duquesne, T., Le Gall, J-F.: Random trees, Lévy processes and spatial branching processes. Astérisque, (281):vi+147 (2002)

  33. 33.

    Durrett, R.: Probability: Theory and Examples, third edn. Thomson, Belmont (2005)

  34. 34.

    Evans, S.N.: Kingman’s coalescent as a random metric space. In: Stochastic models (Ottawa, ON, 1998), vol. 26 of CMS Conference Proceedings, pp. 105–114. American Mathematical Society, Providence, RI (2000)

  35. 35.

    Evans, S.N.: Probability and real trees, vol. 1920 of Lecture Notes in Mathematics. Springer, Berlin, 2008. Lectures from the 35th Summer School on Probability Theory held in Saint-Flour, July 6–23 (2005)

  36. 36.

    Evans, S.N., Grübel, R., Wakolbinger, A.: Doob–Martin boundary of Rémy’s tree growth chain. Ann. Probab. 45(1), 225–277 (2017)

  37. 37.

    Freedman, D., Diaconis, P.: de Finetti’s theorem for symmetric location families. Ann. Stat. 10(1), 184–189 (1982)

  38. 38.

    Gelbaum, B.R., Olmsted, J.M.H.: Counterexamples in Analysis. Dover Publications, Inc., Mineola, NY, (2003). Corrected reprint of the second (1965) edition

  39. 39.

    Ghahramani, Z., Jordan, M.I., Adams, R.P.: Tree-structured stick breaking for hierarchical data. In: Advances in neural information processing systems, pp. 19–27 (2010)

  40. 40.

    Gnedin, A., Iksanov, A., Möhle, M.: On asymptotics of exchangeable coalescents with multiple collisions. J. Appl. Probab. 45(4), 1186–1195 (2008)

  41. 41.

    Gnedin, A.., Olshanski, G.: A \(q\)-analogue of de Finetti’s theorem. Electron. J. Combin. 16(1):Research Paper 78, 16 (2009)

  42. 42.

    Gnedin, A.V.: On a class of exchangeable sequences. Stat. Probab. Lett. 25(4), 351–355 (1995)

  43. 43.

    Gnedin, A.V.: The representation of composition structures. Ann. Probab. 25(3), 1437–1450 (1997)

  44. 44.

    Goldschmidt, C., Martin, J.B.: Random recursive trees and the Bolthausen–Sznitman coalescent. Electron. J. Probab 10(21), 718–745 (2005). (electronic)

  45. 45.

    Gufler, S.: A representation for exchangeable coalescent trees and generalized tree-valued fleming-viot processes. arXiv:1608.08074v2 [math.PR], October (2016)

  46. 46.

    Haas, B., Miermont, G.: The genealogy of self-similar fragmentations with negative index as a continuum random tree. Electron. J. Probab. 9(4), 57–97 (2004). (electronic)

  47. 47.

    Haas, B., Miermont, G., Pitman, J., Winkel, M.: Continuum tree asymptotics of discrete fragmentations and applications to phylogenetic models. Ann. Probab. 36(5), 1790–1837 (2008)

  48. 48.

    Haas, B., Pitman, J., Winkel, M.: Spinal partitions and invariance under re-rooting of continuum random trees. Ann. Probab. 37(4), 1381–1411 (2009)

  49. 49.

    Heller, K.A., Ghahramani, Z.: Bayesian hierarchical clustering. In: Proceedings of the 22nd International Conference on Machine learning, pp. 297–304. ACM (2005)

  50. 50.

    Hewitt, E., Savage, L.J.: Symmetric measures on Cartesian products. Trans. Am. Math. Soc. 80, 470–501 (1955)

  51. 51.

    Hirth, U.: Exchangeable random ordered trees by positive definite functions. J. Theor. Probab. 16(2), 339–344 (2003)

  52. 52.

    Hirth, U., Ressel, P.: Random partitions by semigroup methods. Semigroup Forum 59(1), 126–140 (1999)

  53. 53.

    Hirth, U., Ressel, P.: Exchangeable random orders and almost uniform distributions. J. Theor. Probab. 13(3), 609–634 (2000)

  54. 54.

    Jacka, S., Warren, J.: Random orderings of the integers and card shuffling. Stoch. Process. Appl. 117(6), 708–719 (2007)

  55. 55.

    Janson, S.: Poset limits and exchangeable random posets. Institut Mittag-Leffler preprint 02 (2009)

  56. 56.

    Kallenberg, O.: Exchangeable random measures in the plane. J. Theor. Probab. 3(1), 81–136 (1990)

  57. 57.

    Kallenberg, O.: Probabilistic Symmetries and Invariance Principles. Probability and its Applications New York. Springer, New York (2005)

  58. 58.

    Kerov, S.V.: Asymptotic representation theory of the symmetric group and its applications in analysis, vol. 219 of Translations of Mathematical Monographs. American Mathematical Society, Providence, RI: Translated from the Russian manuscript by N, vol. Tsilevich. With a foreword by A. Vershik and comments by G, Olshanski (2003)

  59. 59.

    Kingman, J.F.C.: The representation of partition structures. J. Lond. Math. Soc. (2) 18(2), 374–380 (1978)

  60. 60.

    Kingman, J.F.C.: The coalescent. Stoch. Process. Appl. 13(3), 235–248 (1982)

  61. 61.

    Kingman, J.F.C.: Exchangeability and the evolution of large populations. In: Exchangeability in probability and statistics (Rome, 1981), pp. 97–112. North-Holland, Amsterdam (1982)

  62. 62.

    Le Gall, J.-F.: Random trees and applications. Probab. Surv. 2, 245–311 (2005)

  63. 63.

    Meeds, E.W., Ross, D., Zemel, R.S., Roweis, S.T., et al.: Learning stick-figure models using nonparametric bayesian priors over trees. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8. IEEE (2008)

  64. 64.

    Möhle, M., Sagitov, S.: A classification of coalescent processes for haploid exchangeable population models. Ann. Probab. 29(4), 1547–1562 (2001)

  65. 65.

    Pemantle, R.: Automorphism invariant measures on trees. Ann. Probab. 20(3), 1549–1566 (1992)

  66. 66.

    Pitman, J.: Combinatorial stochastic processes, vol. 1875 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, (2006). Lectures from the 32nd Summer School on Probability Theory held in Saint-Flour, July 7–24, 2002, With a foreword by Jean Picard

  67. 67.

    Pitman, J.: Exchangeable and partially exchangeable random partitions. Probab. Theory Relat. Fields 102(2), 145–158 (1995)

  68. 68.

    Pitman, J., Winkel, M.: Regenerative tree growth: binary self-similar continuum random trees and Poisson–Dirichlet compositions. Ann. Probab. 37(5), 1999–2041 (2009)

  69. 69.

    Ressel, P.: Subdiagonal and almost uniform distributions. Electron. Commun. Probab. 7, 97–101 (2002). (electronic)

  70. 70.

    Ressel, P.: Exchangeability and semigroups. Rend. Mat. Appl. (7) 28(1), 63–81 (2008)

  71. 71.

    Sagitov, S.: The general coalescent with asynchronous mergers of ancestral lines. J. Appl. Probab. 36(4), 1116–1125 (1999)

  72. 72.

    Stephenson, R.: General fragmentation trees. Electron. J. Probab. 18(101), 45 (2013)

Download references

Author information

Correspondence to Noah Forman.

Additional information

Research supported in part by NSF Grants DMS-0806118 and DMS-1444084 and EPSRC Grant EP/K029797/1.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Forman, N., Haulk, C. & Pitman, J. A representation of exchangeable hierarchies by sampling from random real trees. Probab. Theory Relat. Fields 172, 1–29 (2018). https://doi.org/10.1007/s00440-017-0799-4

Download citation

Keywords

  • Exchangeable
  • Hierarchy
  • Total partition
  • Random composition
  • Random partition
  • Continuum random tree
  • Weighted real tree

Mathematics Subject Classification

  • Primary 60G09
  • Secondary 60C05
  • 62B05