Modeling Retest Effects in a Longitudinal Measurement Burst Study of Memory

Abstract

Longitudinal designs must deal with the confound between increasing age and increasing task experience (i.e., retest effects). Most existing methods for disentangling these factors rely on large sample sizes and are impractical for smaller scale projects. Here, we show that a measurement burst design combined with a model of retest effects can be used to study age-related change with modest sample sizes. A combined model of age-related change and retest-related effects was developed. In a simulation experiment, we show that with sample sizes as small as n = 8, the model can reliably detect age effects of the size reported in the longitudinal literature while avoiding false positives when there is no age effect. We applied the model to data from a measurement burst study in which eight subjects completed a burst of seven sessions of free recall every year for 5 years. Six additional subjects completed a burst only in years 1 and 5. They should, therefore, have smaller retest effects but equal age effects. The raw data suggested slight improvement in memory over 5 years. However, applying the model to the yearly-testing group revealed that a substantial positive retest effect was obscuring stability in memory performance. Supporting this finding, the control group showed a smaller retest effect but an equal age effect. Measurement burst designs combined with models of retest effects allow researchers to employ longitudinal designs in areas where previously only cross-sectional designs were feasible.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

Data Accessibility

The data reported in this study as well as code for fitting the model can be freely accessed at https://cbcc.psy.msu.edu/data/BroiEtal19_data.zip.

Notes

  1. 1.

    Rather than fitting each subject separately, as we have done, one could instead fit all subjects simultaneously within a hierarchical model in which hyper-parameters specify the distributions and covariance structure of the individual-level parameters. For applications where the the nature of the distributions (e.g., Gaussian vs. exGaussian, unimodal vs. bimodal, etc.) can be reasonably hypothesised a priori, such a hierarchical approach would be ideal. In situations where the nature of the distributions is unknown, fitting individual subjects and examining the resulting empirical distributions would be more appropriate.

  2. 2.

    We explored how the use of different fitting algorithms influenced power and false alarms. Fast heuristic algorithms (e.g., multistart, Ugray et al. 2007) provided slightly lower power and type I error rates whereas a slower but more exhaustive grid search provided higher power. We encourage researchers to consider this tradeoff when determining how to fit their own data.

References

  1. Anderson, J., Fincham, J., Douglass, S. (1999). Practice and retention: a unifying analysis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25(5), 1120–1136.

    PubMed  Google Scholar 

  2. Baltes, P.B. (1968). Longitudinal and cross-sectional sequences in the study of age and generation effects. Human Development, 11(3), 145–171.

    PubMed  Google Scholar 

  3. Healey, M.K., Crutchley, P., Kahana, M.J. (2014). Individual differences in memory search and their relation to intelligence. Journal of Experimental Psychology: General, 143(4), 1553–1569. https://doi.org/10.1037/a0036306.

    Google Scholar 

  4. Healey, M.K., & Kahana, M.J. (2014). Is memory search governed by universal principles or idiosyncratic strategies? Journal of Experimental Psychology: General, 143, 575–596. https://doi.org/10.1037/a0033715.

    Google Scholar 

  5. Healey, M.K., & Kahana, M.J. (2016). A four–component model of age–related memory change. Psychological Review, 123(1), 23–69. https://doi.org/10.1037/rev0000015.

    PubMed  Google Scholar 

  6. Hoffman, L., Hofer, S.M., Sliwinski, M.J. (2011). On the confounds among retest gains and age-cohort differences in the estimation of within-person change in longitudinal studies: a simulation study. Psychology and Aging, 26(4), 778.

    PubMed  PubMed Central  Google Scholar 

  7. Kahana, M.J., Aggarwal, E., Phan, T.D. (2018). The variability puzzle in human memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(12), 1857–1863.

    PubMed  Google Scholar 

  8. Lohnas, L.J., & Kahana, M.J. (2013). Parametric effects of word frequency effect in memory for mixed frequency lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(6), 1943–1946. https://doi.org/10.1037/a0033669.

    PubMed  Google Scholar 

  9. Lohnas, L.J., & Kahana, M.J. (2014). Compound cuing in free recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(1), 12–24. https://doi.org/10.1037/a0033698.

    PubMed  Google Scholar 

  10. Miller, J.F., Kahana, M.J., Weidemann, C.T. (2012). Recall termination in free recall. Memory & Cognition, 40(4), 540–550. https://doi.org/10.3758/s13421-011-0178-9.

    Google Scholar 

  11. Miller, M.A., & Rahe, R.H. (1997). Life changes scaling for the 1990s. Journal of Psychosomatic Research, 43(3), 279–292.

    PubMed  Google Scholar 

  12. Munoz, E., Sliwinski, M.J., Scott, S.B., Hofer, S. (2015). Global perceived stress predicts cognitive change among older adults. Psychology and Aging, 30(3), 487.

    PubMed  PubMed Central  Google Scholar 

  13. Nesselroade, J.R. (1991) In R. Downs, & L. Liben (Eds.), The warp and the woof of the developmental fabric, (pp. 213–240). Hillsdale: Erlbaum.

  14. Nilsson, L.G. (2003). Memory function in normal aging. Acta Neurologica Scandinavica, 107, 7–13.

    Google Scholar 

  15. Nilsson, L.G., BÄCkman, L., Erngrund, K., Nyberg, L., Adolfsson, R., Bucht, G., Winblad, B. (1997). The betula prospective cohort study: Memory, health, and aging. Aging, Neuropsychology, and Cognition, 4(1), 1–32.

    Google Scholar 

  16. Salthouse, T.A. (2015). Test experience effects in longitudinal comparisons of adult cognitive functioning. Developmental Psychology, 51(9), 1262.

    PubMed  Google Scholar 

  17. Salthouse, T.A. (2016). Aging cognition unconfounded by prior test experience. Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 71(1), 49. https://doi.org/10.1093/geronb/gbu063.

    Google Scholar 

  18. Sliwinski, M.J. (2008). Measurement-burst designs for social health research. Social and Personality Psychology Compass, 2(1), 245–261.

    Google Scholar 

  19. Sliwinski, M.J., Hoffman, L., Hofer, S. (2010). Modeling retest and aging effects in a measurement burst design. In P. Molenaar, & K.M. Newel (Eds.) Individual pathways of change in learning and development (pp. 37–50). Washington: American Psychological Association.

  20. Ugray, Z., Lasdon, L., Plummer, J., Glover, F., Kelly, J., Martí, R. (2007). Scatter search and local nlp solvers: a multistart framework for global optimization. INFORMS Journal on Computing, 19(3), 328–340.

    Google Scholar 

Download references

Acknowledgments

We thank Ada Aka, Elizabeth Crutchley, Patrick Crutchley, Kylie Hower, Joel Kuhn, Jonathan Miller, Logan O’Sullivan, and Isaac Pedisich for assistance conducting the study.

Funding

This work was supported by the National Institute on Aging at the National Institutes of Health (grant number AG048233) and the National Institute of Mental Health at the National Institutes of Health (grant number MH55687).

Author information

Affiliations

Authors

Corresponding author

Correspondence to M. Karl Healey.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 87.9 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Broitman, A.W., Kahana, M.J. & Healey, M.K. Modeling Retest Effects in a Longitudinal Measurement Burst Study of Memory. Comput Brain Behav 3, 200–207 (2020). https://doi.org/10.1007/s42113-019-00047-w

Download citation

Keywords

  • Free recall
  • Memory models
  • Stability
  • Aging
  • Practice effects