Abstract
In the field of sensitivity analysis, Sobol’ indices are sensitivity measures widely used to assess the importance of inputs of a model to its output. The estimation of these indices is often performed through Monte Carlo or quasi-Monte Carlo methods. A notable method is the replication procedure that estimates first-order indices at a reduced cost in terms of number of model evaluations. An inherent practical problem of this estimation is how to quantify the number of model evaluations needed to ensure that estimates satisfy a desired error tolerance. This article addresses this challenge by proposing a reliable error bound for first-order and total effect Sobol’ indices. Starting from the integral formula of the indices, the error bound is defined in terms of the discrete Walsh coefficients of the different integrands. We propose a sequential estimation procedure of Sobol’ indices using the error bound as a stopping criterion. The sequential procedure combines Sobol’ sequences with either Saltelli’s strategy to estimate both first-order and total effect indices, or the replication procedure to estimate only first-order indices.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bratley, P., Fox, B.L., Niederreiter, H.: Implementation and tests of low-discrepancy sequences. ACM Trans. Model. Comput. Simul. 2(3), 195–213 (1992)
Forrester, A., Sóbester, A., Keane, A.: Engineering Design Via Surrogate Modelling. Wiley, Chichester (2008)
Gilquin, L., Arnaud, E., Monod, H., Prieur, C.: Recursive estimation procedure of Sobol’ indices based on replicated designs. Preprint available at https://hal.inria.fr/hal-01291769 (2016)
Gilquin, L., Jiménez Rugama, L.A., Arnaud, E., Hickernell, F.J., Monod, H., Prieur, C.: Iterative construction of replicated designs based on Sobol’ sequences. C. R. Math. 355, 10–14 (2017)
Hickernell, F.J., Jiménez Rugama, L.A.: Reliable adaptive cubature using digital sequences: Monte Carlo and Quasi-Monte Carlo. Methods 2014(163), 367–383 (2016)
Hickernell, F.J., Jiménez Rugama, L.A., Li, D.: Adaptive quasi-Monte Carlo methods for cubature. Preprint available at https://arxiv.org/pdf/1702.01491.pdf (2017)
Hoeffding, W.F.: A class of statistics with asymptotically normal distribution. Ann. Math. Stat. 19(3), 293–325 (1948)
Hong, H.S., Hickernell, F.J.: Algorithm 823: Implementing scrambled digital nets. ACM Trans. Math. Softw. 29, 95–109 (2003)
Janon, A., Klein, T., Lagnoux, A., Nodet, M., Prieur, C.: Asymptotic normality and efficiency of two Sobol’ index estimators. ESAIM Probab. Stat. 18, 342–364 (2014)
Jiménez Rugama, L.A., Hickernell, F.J.: Adaptive multidimensional integration based on rank-1 lattices: Monte Carlo and Quasi-Monte Carlo. Methods 2014(163), 407–422 (2016)
Joe, S., Kuo, F.Y.: Constructing Sobol sequences with better two-dimensional projections. SIAM J. Sci. Comput. 30(8), 2635–2654 (2008)
Lemieux, C.: Monte Carlo and quasi-Monte Carlo Sampling. Springer, New York (2009)
Mara, T.A., Rakoto Joseph, O.: Comparison of some efficient methods to evaluate the main effect of computer model factors. J. Stat. Comput. Simul. 78(2), 167–178 (2008)
McKay, M.D.: Evaluating prediction uncertainty, Los Alamos National Laboratory Report NUREG/CR- 6311, LA-12915-MS. (1995)
Niederreiter, H.: Random Number Generation and Quasi-Monte Carlo Methods: CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 63. SIAM, Philadelphia (1992)
Owen, A.B.: Randomly permuted \((t, m, s)\)-nets and \((t, s)\)-sequences: Monte Carlo and Quasi-Monte Carlo Methods in Scientific. Computing 106, 299–317 (1995)
Owen, A.B.: Better estimation of small Sobol’ sensitivity indices. ACM Trans. Model. Comput. Simul. 23(2), 11 (2013)
Saltelli, A.: Making best use of models evaluations to compute sensitivity indices. Comput. Phys. Commun. 145(2), 280–297 (2002)
Sobol’, I.M.: On the distribution of points in a cube and the approximate evaluation of integrals. USSR Comput. Math. Math. Phys. 7(4), 86–112 (1967)
Sobol’, I.M.: Sensitivity indices for nonlinear mathematical models. Math. Model. Comput. Exp. 1, 407–414 (1993)
Tissot, J.Y., Prieur, C.: A randomized orthogonal array-based procedure for the estimation of first- and second-order Sobol’ indices. J. Statist. Comput. Simul. 85(7), 1358–1381 (2015)
Tong, C.: Self-validated variance-based methods for sensitivity analysis of model outputs. Reliab. Eng. Syst. Saf. 95(3), 301–309 (2010)
Acknowledgements
The authors thank Fred J. Hickernell and Clémentine Prieur for initiating this collaborative work, and Elise Arnaud for her proofreading. The authors are grateful to Stephen Joe, Frances Y. Kuo and Art B. Owen for their helpful answers and suggestions. The authors also thank the associate editor and the two anonymous reviewers for their helpful suggestions and comments which substantially improved the quality of this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work is supported by the CITiES project funded by the Agence Nationale de la Recherche (Grant ANR-12-MONU-0020) and by the United States National Science Foundation (Grant DMS-1522687).
Rights and permissions
About this article
Cite this article
Jiménez Rugama, L.A., Gilquin, L. Reliable error estimation for Sobol’ indices. Stat Comput 28, 725–738 (2018). https://doi.org/10.1007/s11222-017-9759-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-017-9759-1