Combining permutation tests to rank systemically important banks


In this work we propose the use of a nonparametric procedure to investigate the relationship between the Regulator’s Global Systemically Important Banks (G-SIBs) classification and the equity-based systemic risk measures. The proposed procedure combines several permutation tests to investigate the equality of the multivariate distribution of two groups and assumes only the hypothesis of exchangeability of variables. In our novel approach, the weights used in the combination of tests are obtained using the Particle Swarm Optimization heuristic and quantify the informativeness about the selection. Finally, the p value of the combined test measures the reliability of the result. Empirical results about the selection of G-SIBs show how considering the systematic (\(\beta \)), stress (\(\varDelta \)CoVaR) and connectedness components (in–out connection) of systemic risk cover more than \(70\%\) of weight in all the considered years.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    The analyzed dataset is available on request.

  2. 2.

    The complete list of included institutions is available upon request.


  1. Acharya VV, Pedersen LH, Philippon T, Richardson MP (2010) Measuring systemic risk. Technical report, Department of Finance, NYU

  2. Acharya V, Engle R, Richardson M (2012) Capital shortfall: a new approach to ranking and regulating systemic risks. Am Econ Rev 102(3):59–64

    Article  Google Scholar 

  3. Adrian T, Brunnermeier MK (2016) Covar. Am Econ Rev 106(7):1705–41

    Article  Google Scholar 

  4. Benoit S, Colliard JE, Hurlin C, Pérignon C (2017) Where the risks lie: a survey on systemic risk. Rev Finance 21(1):109–152

    Article  Google Scholar 

  5. Billio M, Getmanski M, Lo A, Pelizzon L (2012) Econometric measures of connectedness and systemic risk in the finance and insurance sectors. J Financ Econ 104:535–559

    Article  Google Scholar 

  6. BIS (2013) Global systemically important banks: updated assessment methodology and the higher loss absorbency requirement. Accessed 3 July 2013

  7. BIS (2014) The G-SIB assessment methodology—score calculation. Accessed 6 Nov 2014

  8. Brownlees CT, Engle R (2017) SRISK: a conditional capital shortfall measure of systemic risk. Rev Financ Stud 30:48–79

    Article  Google Scholar 

  9. Campbell G (1994) Advances in statistical methodology for the evaluation of diagnostic and laboratory tests. Stat Med 13(5–7):499–508

    Article  Google Scholar 

  10. Demirer M, Diebold FX, Liu L, Yilmaz K (2015) Estimating global bank network connectedness. Working paper 1512, KoU̧niversity–TUSIAD Economic Research Forum, SSRN 2631479

  11. Diebold FX, Yılmaz K (2014) On the network topology of variance decompositions: measuring the connectedness of financial firms. J Econom 182(1):119–134

    MathSciNet  Article  Google Scholar 

  12. Frattarolo L, Parpinel F, Pizzi C (2016) Systemically important banks: a permutation test approach. Rivista Italiana di Economia Demografia e Statistica LXX:41–52

  13. FSB (2010) Reducing the moral hazard posed by systemically important financial institutions. Accessed 11 Nov 2010

  14. Giglio S, Kelly B, Pruitt S (2016) Systemic risk and the macroeconomy: an empirical evaluation. J Financ Econ 119(3):457–471

    Article  Google Scholar 

  15. Girardi G, Ergün AT (2013) Systemic risk measurement: multivariate GARCH estimation of CoVaR. J Bank Finance 37(8):3169–3180

    Article  Google Scholar 

  16. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks, vol 4, pp 1942–1948.

  17. Kinlaw W, Kritzman M, Turkington D (2012) Toward determining systemic importance. J Portf Manag 38(4):100–111

    Article  Google Scholar 

  18. Kritzman M, Li Y, Page S, Rigobon R (2011) Principal components as a measure of systemic risk. J Portf Manag 37(4):112–126

    Article  Google Scholar 

  19. Li J, Tseng GC (2011) An adaptively weighted statistic for detecting differential gene expression when combining multiple transcriptomic studies. Ann Appl Stat 5(2A):994–1019

    MathSciNet  Article  Google Scholar 

  20. Moenninghoff SC, Ongena S, Wieandt A (2015) The perennial challenge to counter too-big-to-fail in banking: empirical evidence from the new international regulation dealing with global systemically important banks. J Bank Finance 61(Supplement C):221–236

    Article  Google Scholar 

  21. Pesarin F, Salmaso L (2010) Permutation tests for complex data: theory. Applications and Software. Wiley, West Sussex

    Book  Google Scholar 

  22. Silva W, Kimura H, Sobreiro VA (2017) An analysis of the literature on systemic financial risk: a survey. J Financ Stab 28:91–114

    Article  Google Scholar 

  23. Winkler AM, Webster MA, Brooks JC, Tracey I, Smith SM, Nichols TE (2016) Non-parametric combination and related permutation tests for neuroimaging. Hum Brain Mapp 37(4):1486–1511

    Article  Google Scholar 

Download references


We wish to thank the Editor and two anonymous referees for very useful comments and suggestions which have helped to improve and develop the paper further. The authors are also grateful to The System for Scientific Computing of Ca’ Foscari (SCSCF) for computations.

Author information



Corresponding author

Correspondence to Claudio Pizzi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Frattarolo, L., Parpinel, F. & Pizzi, C. Combining permutation tests to rank systemically important banks. Stat Methods Appl 29, 581–596 (2020).

Download citation


  • Systemic risk
  • Global Systemically Important Bank
  • Particle Swarm Optimization
  • Permutation tests