Abstract
Estimating statistical significance of detected differences between two groups of medical scans is a challenging problem due to the high dimensionality of the data and the relatively small number of training examples. In this paper, we demonstrate a non-parametric technique for estimation of statistical significance in the context of discriminative analysis (i.e., training a classifier function to label new examples into one of two groups). Our approach adopts permutation tests, first developed in classical statistics for hypothesis testing, to estimate how likely we are to obtain the observed classification performance, as measured by testing on a hold-out set or cross-validation, by chance. We demonstrate the method on examples of both structural and functional neuroimaging studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bullmore, E.T., et al.: Global, Voxel, and Cluster Tests, by Theory and Permutation, for a Difference Between Two Groups of Structural MR Images of the Brain. IEEE Transactions on Medical Imaging 18(1), 32–42 (1999)
Csernansky, J.G., et al.: Hippocampal Morphometry in Schizophrenia by High Dimensional Brain Mapping. Proceedings of National Academy of Science 95(19), 11406–11411 (1998)
Dale, A.M., et al.: Cortical Surface-Based Analysis I: Segmentation and Surface Reconstruction. NeuroImage 9, 179–194 (1999)
Efron, B.: The Jacknife, The Bootstrap, and Other Resampling Plans. SIAM, Philadelphia, PA (1982)
Fischl, B., et al.: Cortical Surface-Based Analysis II: Inflation, Flattening, a Surface- Based Coordinate System. NeuroImage 9, 195–207 (1999)
Fischl, B., et al.: High-resolution intersubject averaging and a coordinate system for the cortical surface. Human Brain Mapping 8, 272–284 (1999)
Fischl, B., et al.: Measuring the thickness of the human cerebral cortex from magnetic resonance images. PNAS 26, 11050–11055 (2000)
Fischl, B., Liu, A., Dale, A.M.: Automated Manifold Surgery: Constructing Geometrically Accurate and Topologically Correct Models of the Human Cerebral Cortex. IEEE Transactions on Medical Imaging 20(1), 70–80 (2001)
Gerig, G., et al.: Shape versus Size: Improved Understanding of the Morphology of Brain Structures. In: Niessen, W.J., Viergever, M.A. (eds.) MICCAI 2001. LNCS, vol. 2208, pp. 24–32. Springer, Heidelberg (2001)
Golland, P., et al.: Discriminative Analysis for Image-Based Studies. In: Dohi, T., Kikinis, R. (eds.) MICCAI 2002. LNCS, vol. 2488, pp. 508–515. Springer, Heidelberg (2002)
Good, P.: Permutation Tests: A Practical guide to Resampling Methods for Testing Hypothesis. Springer, Heidelberg (1994)
Guyon, I., et al.: What Size Test Set Gives Good Error Rate Estimates? IEEE Trans. Pattern Analysis and Machine Intelligence 20(1), 52–64 (1998)
Haxby, J.V., et al.: Distributed and Overlapping Representations of Faces and Objects In Ventral Temporal Cortex. Science 293, 2425–2430 (2001)
Martin, J., Pentland, A., Kikinis, R.: Shape Analysis of Brain Structures Using Physical and Experimental Models. In: Proceedings of CVPR 1994, pp. 752–755 (1994)
Nichols, T.E., Holmes, A.P.: Nonparametric Permutation Tests For Functional Neuroimaging: A Primer with Examples. Human Brain Mapping 15, 1–25 (2001)
Thomson, P.M., et al.: Dynamics of Gray Matter Loss in Alzheimer’s Disease. Journal of Neuroscience 23(3) (2003)
Sachs, L.: Applied Statistics: A Handbook of Techniques. Springer, Heidelberg (1984)
Spiridon, M., Kanwisher, N.: How distributed is visual category information in human occipito-temporal cortex? An fMRI study. Neuron 35(6), 1157–1165 (2002)
Vapnik, V.N.: Statistical Learning Theory. John Wiley & Sons, Chichester (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Golland, P., Fischl, B. (2003). Permutation Tests for Classification: Towards Statistical Significance in Image-Based Studies. In: Taylor, C., Noble, J.A. (eds) Information Processing in Medical Imaging. IPMI 2003. Lecture Notes in Computer Science, vol 2732. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45087-0_28
Download citation
DOI: https://doi.org/10.1007/978-3-540-45087-0_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40560-3
Online ISBN: 978-3-540-45087-0
eBook Packages: Springer Book Archive