Are divergence point analyses suitable for response time data?

Abstract

Estimating the time course of the influence of different factors in human performance is one of the major topics of research in cognitive psychology/neuroscience. Over the past decades, researchers have proposed several methods to tackle this question using latency data. Here we examine a recently proposed procedure that employs survival analyses on latency data to provide precise estimates of the timing of the first discernible influence of a given factor (e.g., word frequency on lexical access) on performance (e.g., fixation durations or response times). A number of articles have used this method in recent years, and hence an exploration of its strengths and its potential weaknesses is in order. Unfortunately, our analysis revealed that the technique has conceptual flaws, and it might lead researchers into believing that they are obtaining a measurement of processing components when, in fact, they are obtaining an uninterpretable measurement.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. 1.

    Reingold, Sheridan, and colleagues define divergence on survival functions rather than on cumulative distribution functions. The survival function S is 1 − F, where F is the cumulative distribution function. Hence, divergence may be defined equivalently on survival or cumulative distribution functions (CDFs). We choose CDFs because we expect more readers are familiar with cumulative distribution functions than with survival functions.

  2. 2.

    We, furthermore, used the R-packages dplyr (Version 0.8.3, Wickham et al., 2019), forcats (Version 0.4.0, Wickham, 2019a, 2019b), gamlss.dist (Version 5.1.4, Stasinopoulos & Rigby 2019), ggplot2 (Version 3.2.1, Wickham 2016), ggpmisc (Version 0.3.1, Aphalo, 2016), MASS (Version 7.3.51.4, Venables & Ripley 2002), papaja (Version 0.1.0.9842, Aust, (Aust, 2018)), purrr (Version 0.3.2, Henry & Wickham, 2019), readr (Version 1.3.1, Wickham et al., 2018), retimes (Version 0.1.2, Massidda, 2013), scales (Version 1.0.0, Wickham, 2018), stringr (Version 1.4.0, Wickham, 2019a, 2019b), tibble (Version 2.1.3, Müller & Wickham 2019), tidyr (Version 0.8.3, Wickham & Henry, 2019), tidyverse (Version 1.2.1, Wickham, 2017), and truncnorm (Version 1.0.8; Mersmann, Trautmann, Steuer, and Bornkamp, 2018).

References

  1. Aphalo, PJ (2016). Learn R...as you learnt your mother tongue. Leanpub. Retrieved from https://leanpub.com/learnr.

  2. Aust, F (2018). papaja: Create APA manuscripts with R Markdown. Retrieved from https://github.com/crsh/papaja.

  3. Balota, DA, & Yap, MJ (2011). Moving beyond the mean in studies of mental chronometry: The power of response time distributional analyses. Current Directions in Psychological Science, 20(3), 160–166.

    Article  Google Scholar 

  4. Brown, SD, & Heathcote, A (2008). The simplest complete model of choice reaction time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178.

    Article  Google Scholar 

  5. De Jong, R, Liang, CC, & Lauber, E (1994). Conditional and unconditional automaticity: A dual-process model of effects of spatial stimulus-response concordance. Journal of Experimental Psychology: Human Perception and Performance, 20, 731–750.

    PubMed  Google Scholar 

  6. Dzhafarov, EN (1992). The structure of simple reaction time to step-function signals. Journal of Mathematical Psychology, 36, 235–268.

    Article  Google Scholar 

  7. Ellinghaus, R, & Miller, J (2018). Delta plots with negative-going slopes as a potential marker of decreasing response activation in masked semantic priming. Psychological Research Psychologische Forschung, 82(3), 590–599.

    Article  Google Scholar 

  8. Estes, WK (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53(2), 134–140.

    Article  Google Scholar 

  9. Everitt, BS, & Hand, DJ. (1981) Finite mixture distributions. London: Chapman; Hall.

    Google Scholar 

  10. Falmagne, J-C (1968). Note on a simple fixed-point property of binary mixtures. British Journal of Mathematical and Statistical Psychology, 21, 131–132.

    Article  Google Scholar 

  11. Gould, SJ. (1996) The mismeasure of man. New York: WW Norton & Company.

    Google Scholar 

  12. Heathcote, A, Popiel, SJ, & Mewhort, D (1991). Analysis of response time distributions: An example using the Stroop task. Psychological Bulletin, 109(2), 340–347.

    Article  Google Scholar 

  13. Henry, L, & Wickham, H (2019). Purrr: Functional programming tools. Retrieved from https://CRAN.R-project.org/package=purrr.

  14. Leinenger, M (2018). Survival analyses reveal how early phonological processing affects eye movements during reading. Journal of Experimental Psychology, Learning, Memory, and Cognition.

  15. Luce, RD. (1986) Response times. New York: Oxford University Press.

    Google Scholar 

  16. Massidda, D (2013). Retimes: Reaction time analysis. Retrieved from https://CRAN.R-project.org/package=retimes.

  17. Mersmann, O, Trautmann, H, Steuer, D, & Bornkamp, B (2018). Truncnorm: Truncated normal distribution. Retrieved from https://CRAN.R-project.org/package=truncnorm.

  18. Müller, K, & Wickham, H (2019). Tibble: Simple data frames. Retrieved from https://CRAN.R-project.org/package=tibble.

  19. Ratcliff, R (1978). A theory of memory retrieval. Psychological Review, 85(2), 59–108.

    Article  Google Scholar 

  20. Ratcliff, R (1979). Group reaction time distributions and an analysis of distribution statistics. Psychological Bulletin, 86(3), 446–461.

    Article  Google Scholar 

  21. R Core Team (2019). R: A language and environment for statistical computing, Vienna, Austria. R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/.

  22. Reichle, ED, Pollatsek, A, Fisher, DL, & Rayner, K (1998). Toward a model of eye movement control in reading. Psychological Review, 105(1), 125–157.

    Article  Google Scholar 

  23. Reingold, EM, & Sheridan, H (2014). Estimating the divergence point: A novel distributional analysis procedure for determining the onset of the influence of experimental variables. Frontiers in Psychology, 5, 1432.

    Article  Google Scholar 

  24. Reingold, EM, & Sheridan, H (2018). On using distributional analysis techniques for determining the onset of the influence of experimental variables. Quarterly Journal of Experimental Psychology, 71(1), 260–271.

    Article  Google Scholar 

  25. Reingold, EM, Reichle, ED, Glaholt, MG, & Sheridan, H (2012). Direct lexical control of eye movements in reading: Evidence from a survival analysis of fixation durations. Cognitive Psychology, 65(2), 177–206.

    Article  Google Scholar 

  26. Rotello, CM, & Heit, E (1999). Two-process models of recognition memory: Evidence for recall-to-reject? Journal of Memory and Language, 40(3), 432–453.

    Article  Google Scholar 

  27. Rouder, JN (2000). Assessing the roles of change discrimination and luminance integration: Evidence for a hybrid race model of perceptual decision making in luminance discrimination. Journal of Experimental Psychology: Human Perception and Performance, 26, 359–378.

    PubMed  Google Scholar 

  28. Rouder, JN, Lu, J, Speckman, P, Sun, D, & Jiang, Y (2005). A hierarchical model for estimating response time distributions. Psychonomic Bulletin & Review, 12(2), 195–223.

    Article  Google Scholar 

  29. Schmidtke, D, & Kuperman, V (2019). A paradox of apparent brainless behavior: The time-course of compound word recognition. Cortex, 116, 250–267.

    Article  Google Scholar 

  30. Schmidtke, D, Matsuki, K, & Kuperman, V (2017). Surviving blind decomposition: A distributional analysis of the time-course of complex word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(11), 1793.

    PubMed  Google Scholar 

  31. Sheridan, H (2013). The time-course of lexical influences on fixation durations during reading. Evidence from distributional analyses (PhD thesis).

  32. Sheridan, H, Rayner, K, & Reingold, EM (2013). Unsegmented text delays word identification: Evidence from a survival analysis of fixation durations. Visual Cognition, 21(1), 38–60.

    Article  Google Scholar 

  33. Stasinopoulos, M, & Rigby, R (2019). Gamlss.dist: Distributions for generalized additive models for location scale and shape. Retrieved from https://CRAN.R-project.org/package=gamlss.dist.

  34. Staub, A (2011). The effect of lexical predictability on distributions of eye fixation durations. Psychonomic Bulletin & Review, 18(2), 371–376.

    Article  Google Scholar 

  35. Venables, WN, & Ripley, BD (2002). Modern applied statistics with S (Fourth.), Springer, New York. Retrieved from http://www.stats.ox.ac.uk/pub/MASS4.

  36. Wickham, H. (2016) Ggplot2: Elegant graphics for data analysis. New York: Springer. Retrieved from https://ggplot2.tidyverse.org.

    Google Scholar 

  37. Wickham, H (2017). Tidyverse: Easily install and load the ‘tidyverse’. Retrieved from https://CRAN.R-project.org/package=tidyverse.

  38. Wickham, H (2018). Scales: Scale functions for visualization. Retrieved from https://CRAN.R-project.org/package=scales.

  39. Wickham, H (2019a). Forcats: Tools for working with categorical variables (factors). Retrieved from https://CRAN.R-project.org/package=forcats.

  40. Wickham, H (2019b). Stringr: Simple, consistent wrappers for common string operations. Retrieved from https://CRAN.R-project.org/package=stringr.

  41. Wickham, H, & Henry, L (2019). Tidyr: Easily tidy data with ‘spread()’ and ‘gather()’ functions. Retrieved from https://CRAN.R-project.org/package=tidyr.

  42. Wickham, H, Hester, J, & Francois, R (2018). Readr: Read rectangular text data. Retrieved from https://CRAN.R-project.org/package=readr.

  43. Wickham, H, François, R, Henry, L, & Müller, K (2019). Dplyr: A grammar of data manipulation. Retrieved from https://CRAN.R-project.org/package=dplyr.

  44. Yantis, S, Meyer, DE, & Smith, JEK (1991). Analysis of multinomial mixture distributions: New tests for stochastic models of cognitive action. Psychological Bulletin, 110, 350–374.

    Article  Google Scholar 

Download references

Acknowledgements

Funded by a grant PSI2017-86210-P from the Spanish Ministry of Science, Innovation, and Universities, and by the grant 0115/2018 (Estades d’investigadors convidats) from the Universitat de València.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Pablo Gómez.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gómez, P., Breithaupt, J., Perea, M. et al. Are divergence point analyses suitable for response time data?. Behav Res 53, 49–58 (2021). https://doi.org/10.3758/s13428-020-01424-1

Download citation

Keywords

  • Latencies
  • Divergence
  • Mental chronometry