Abstract
Longitudinal designs must deal with the confound between increasing age and increasing task experience (i.e., retest effects). Most existing methods for disentangling these factors rely on large sample sizes and are impractical for smaller scale projects. Here, we show that a measurement burst design combined with a model of retest effects can be used to study age-related change with modest sample sizes. A combined model of age-related change and retest-related effects was developed. In a simulation experiment, we show that with sample sizes as small as n = 8, the model can reliably detect age effects of the size reported in the longitudinal literature while avoiding false positives when there is no age effect. We applied the model to data from a measurement burst study in which eight subjects completed a burst of seven sessions of free recall every year for 5 years. Six additional subjects completed a burst only in years 1 and 5. They should, therefore, have smaller retest effects but equal age effects. The raw data suggested slight improvement in memory over 5 years. However, applying the model to the yearly-testing group revealed that a substantial positive retest effect was obscuring stability in memory performance. Supporting this finding, the control group showed a smaller retest effect but an equal age effect. Measurement burst designs combined with models of retest effects allow researchers to employ longitudinal designs in areas where previously only cross-sectional designs were feasible.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Accessibility
The data reported in this study as well as code for fitting the model can be freely accessed at https://cbcc.psy.msu.edu/data/BroiEtal19_data.zip.
Notes
Rather than fitting each subject separately, as we have done, one could instead fit all subjects simultaneously within a hierarchical model in which hyper-parameters specify the distributions and covariance structure of the individual-level parameters. For applications where the the nature of the distributions (e.g., Gaussian vs. exGaussian, unimodal vs. bimodal, etc.) can be reasonably hypothesised a priori, such a hierarchical approach would be ideal. In situations where the nature of the distributions is unknown, fitting individual subjects and examining the resulting empirical distributions would be more appropriate.
We explored how the use of different fitting algorithms influenced power and false alarms. Fast heuristic algorithms (e.g., multistart, Ugray et al. 2007) provided slightly lower power and type I error rates whereas a slower but more exhaustive grid search provided higher power. We encourage researchers to consider this tradeoff when determining how to fit their own data.
References
Anderson, J., Fincham, J., Douglass, S. (1999). Practice and retention: a unifying analysis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25(5), 1120–1136.
Baltes, P.B. (1968). Longitudinal and cross-sectional sequences in the study of age and generation effects. Human Development, 11(3), 145–171.
Healey, M.K., Crutchley, P., Kahana, M.J. (2014). Individual differences in memory search and their relation to intelligence. Journal of Experimental Psychology: General, 143(4), 1553–1569. https://doi.org/10.1037/a0036306.
Healey, M.K., & Kahana, M.J. (2014). Is memory search governed by universal principles or idiosyncratic strategies? Journal of Experimental Psychology: General, 143, 575–596. https://doi.org/10.1037/a0033715.
Healey, M.K., & Kahana, M.J. (2016). A four–component model of age–related memory change. Psychological Review, 123(1), 23–69. https://doi.org/10.1037/rev0000015.
Hoffman, L., Hofer, S.M., Sliwinski, M.J. (2011). On the confounds among retest gains and age-cohort differences in the estimation of within-person change in longitudinal studies: a simulation study. Psychology and Aging, 26(4), 778.
Kahana, M.J., Aggarwal, E., Phan, T.D. (2018). The variability puzzle in human memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(12), 1857–1863.
Lohnas, L.J., & Kahana, M.J. (2013). Parametric effects of word frequency effect in memory for mixed frequency lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(6), 1943–1946. https://doi.org/10.1037/a0033669.
Lohnas, L.J., & Kahana, M.J. (2014). Compound cuing in free recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(1), 12–24. https://doi.org/10.1037/a0033698.
Miller, J.F., Kahana, M.J., Weidemann, C.T. (2012). Recall termination in free recall. Memory & Cognition, 40(4), 540–550. https://doi.org/10.3758/s13421-011-0178-9.
Miller, M.A., & Rahe, R.H. (1997). Life changes scaling for the 1990s. Journal of Psychosomatic Research, 43(3), 279–292.
Munoz, E., Sliwinski, M.J., Scott, S.B., Hofer, S. (2015). Global perceived stress predicts cognitive change among older adults. Psychology and Aging, 30(3), 487.
Nesselroade, J.R. (1991) In R. Downs, & L. Liben (Eds.), The warp and the woof of the developmental fabric, (pp. 213–240). Hillsdale: Erlbaum.
Nilsson, L.G. (2003). Memory function in normal aging. Acta Neurologica Scandinavica, 107, 7–13.
Nilsson, L.G., BÄCkman, L., Erngrund, K., Nyberg, L., Adolfsson, R., Bucht, G., Winblad, B. (1997). The betula prospective cohort study: Memory, health, and aging. Aging, Neuropsychology, and Cognition, 4(1), 1–32.
Salthouse, T.A. (2015). Test experience effects in longitudinal comparisons of adult cognitive functioning. Developmental Psychology, 51(9), 1262.
Salthouse, T.A. (2016). Aging cognition unconfounded by prior test experience. Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 71(1), 49. https://doi.org/10.1093/geronb/gbu063.
Sliwinski, M.J. (2008). Measurement-burst designs for social health research. Social and Personality Psychology Compass, 2(1), 245–261.
Sliwinski, M.J., Hoffman, L., Hofer, S. (2010). Modeling retest and aging effects in a measurement burst design. In P. Molenaar, & K.M. Newel (Eds.) Individual pathways of change in learning and development (pp. 37–50). Washington: American Psychological Association.
Ugray, Z., Lasdon, L., Plummer, J., Glover, F., Kelly, J., Martí, R. (2007). Scatter search and local nlp solvers: a multistart framework for global optimization. INFORMS Journal on Computing, 19(3), 328–340.
Acknowledgments
We thank Ada Aka, Elizabeth Crutchley, Patrick Crutchley, Kylie Hower, Joel Kuhn, Jonathan Miller, Logan O’Sullivan, and Isaac Pedisich for assistance conducting the study.
Funding
This work was supported by the National Institute on Aging at the National Institutes of Health (grant number AG048233) and the National Institute of Mental Health at the National Institutes of Health (grant number MH55687).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Broitman, A.W., Kahana, M.J. & Healey, M.K. Modeling Retest Effects in a Longitudinal Measurement Burst Study of Memory. Comput Brain Behav 3, 200–207 (2020). https://doi.org/10.1007/s42113-019-00047-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42113-019-00047-w