Skip to main content
SpringerLink
Log in
Book cover

Information Systems and Neuroscience pp 129–135Cite as

What Can NeuroIS Learn from the Replication Crisis in Psychological Science?

  • Conference paper
  • First Online:

  • 1229 Accesses

Part of the book series: Lecture Notes in Information Systems and Organisation ((LNISO,volume 32))

Abstract

The Reproducibility Crisis is a phenomenon that has gained considerable attention in the psychological sciences. Scholars in these fields have found that many high profile findings are either difficult to reproduce or could not be replicated. These findings have ultimately encouraged researchers to adopt pre-registered results, replication in study design and open data. As an emerging field, NeuroIS has an opportunity to learn from this crisis and adopt new practices based on the lessons learned in the psychological sciences. We explored the current state of NeuroIS research from the perspective of reproducibility by conducting a survey of the extant NeuroIS literature. We conclude by suggesting two practices that the NeuroIS community can undertake to help address the replication problem.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Discrepancies between the findings of this paper and Riedl et al. [16] can be attributed to differences in how authors interpreted studies as complete or empirical, and the subjective judgement employed by us when identifying a primary research method for each study.

References

  1. Hempel, C. G. (1968). Maximal specificity and lawlikeness in probabilistic explanation. Philosophy of Science, 35(2), 116–133.

    Article  Google Scholar 

  2. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.

    Google Scholar 

  3. Baker, M. (2016). Is There a Reproducibility Crisis? Nature, 533.

    Google Scholar 

  4. Francis, G. (2012). Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review, 19(2), 151–156.

    Article  Google Scholar 

  5. Francis, G. (2012). Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin & Review, 19(6), 975–991.

    Article  Google Scholar 

  6. Gonzales, J. E., & Cunningham, C. A. (2015). The promise of pre-registration in psychological research. https://www.apa.org/science/about/psa/2015/08/pre-registration.

  7. Foster, E. D., & Deardorff, A. (2017). Open science framework (OSF). Journal of the Medical Library Association, 105(2), 203.

    Google Scholar 

  8. Luck, S. J., & Gaspelin, N. (2017). How to get statistically significant effects in any ERP experiment (and why you shouldn’t). Psychophysiology, 54, 146–157.

    Article  Google Scholar 

  9. Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munafo, M. R., et al. (2017). Scanning the horizon: Towards transparent and reproducible neuroimaging research. Nature Reviews, 18.

    Google Scholar 

  10. Van Essen, D. C., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T. E. J., Bucholz, R., et al. (2012). The Human Connectome Project: A data acquisition perspective. Neuroimage, 62(4), 2222–2231.

    Article  Google Scholar 

  11. Taylor, J. R., Williams, N., Cusack, R., Auer, T., Shafto, M. A., Dixon, M., et al. (2017). The Cambridge Centre for Ageing and Neuroscience (CamCAN) data repository: Structural and functional MRI, MEG, and cognitive data from a cross-sectional adult lifespan sample. NeuroImage, 144, 262–269.

    Article  Google Scholar 

  12. Mueller, S. G., Weiner, M. W., Thal, L. J., Petersen, R. C., Jack, C. R., Jagust, W., et al. (2005). Ways toward an early diagnosis in Alzheimer’s disease: The Alzheimer’s Disease Neuroimaging Initiative (ADNI). Alzheimer’s and Dementia, 1(1), 55–66.

    Article  Google Scholar 

  13. Dimoka, A., Davis, F. D., Gupta, A., Pavlou, P. A., Banker, R. D., Dennis, A. R., et al. (2012). On the use of neurophysiological tools in IS research: Developing a research agenda for NeuroIS. MIS Quarterly, 36(3), 679–702.

    Article  Google Scholar 

  14. vom Brocke, J., & Liang, T.-P. (2014). Guidelines for neuroscience studies in information systems research. Journal of Management Information Systems.

    Google Scholar 

  15. Dennis, A. R., & Valacich, J. S. (2014). A replication manifesto. AIS Transactions on Replication Research, 1, 1–4.

    Article  Google Scholar 

  16. Riedl, R., Fischer, T., & Léger, P.-M. (2017). A decade of NeuroIS research: Status quo, challenges, and future directions. In Proceedings of the 38th International Conference on Information Systems, Seoul.

    Google Scholar 

  17. Dimoka, A. (2010). What does the brain tell us about trust and distrust? Evidence from a functional neuroimaging study. MIS Quarterly, 34(2), 373–396.

    Article  Google Scholar 

  18. De Guinea, A. O., & Webster, J. (2013). An investigation of information systems use patterns: Technological events as triggers, the effect of time, and consequences for performance. MIS Quarterly, 38(4), 1165–1188.

    Article  Google Scholar 

  19. Goggins, S. P., Schmidt, M., Guajardo, J., & Moore, J. (2010). Assessing multiple perspectives in three dimensional virtual worlds: Eye tracking and all views qualitative analysis (AVQA). In 2010 43rd Hawaii International Conference on System Sciences (pp. 1–10). IEEE.

    Google Scholar 

  20. Gregor, S., Lin, A. C., Gedeon, T., Riaz, A., & Zhu, D. (2014). Neuroscience and a nomological network for the understanding and assessment of emotions in information systems research. Journal of Management Information Systems, 30(4), 13–48.

    Article  Google Scholar 

  21. Perrin, J. L., Paillé, D., & Baccino, T. (2014). Reading tilted: Does the use of tablets impact performance? An oculometric study. Computers in Human Behavior, 39, 339–345.

    Article  Google Scholar 

  22. Clayton, R. B., Leshner, G., & Almond, A. (2015). The extended iSelf: The impact of iPhone separation on cognition, emotion, and physiology. Journal of Computer-Mediated Communication, 20(2), 119–135.

    Article  Google Scholar 

  23. Cole, M. J., Hendahewa, C., Belkin, N. J., & Shah, C. (2015). User activity patterns during information search. ACM Transactions on Information Systems (TOIS), 33(1), 1–39.

    Article  Google Scholar 

  24. Jenkins, J. L., Anderson, B. B., Vance, A., Kirwan, C. B., & Eargle, D. (2016). More harm than good? How messages that interrupt can make us vulnerable. Information Systems Research, 27(4), 880–896.

    Article  Google Scholar 

  25. Luan, J., Yao, Z., Zhao, F., & Liu, H. (2016). Search product and experience product online reviews: An eye-tracking study on consumers’ review search behavior. Computers in Human Behavior, 65, 420–430.

    Article  Google Scholar 

  26. Bahr, G. S., & Ford, R. A. (2011). How and why pop-ups don’t work: Pop-up prompted eye movements, user affect and decision making. Computers in Human Behavior, 27(2), 776–783.

    Article  Google Scholar 

  27. Nunamaker, J. F., Derrick, D. C., Elkins, A. C., Burgoon, J. K., & Patton, M. W. (2011). Embodied conversational agent-based kiosk for automated interviewing. Journal of Management Information Systems, 28(1), 17–48.

    Article  Google Scholar 

  28. Astor, P. J., Adam, M. T., Jerčić, P., Schaaff, K., & Weinhardt, C. (2013). Integrating biosignals into information systems: A NeuroIS tool for improving emotion regulation. Journal of Management Information Systems, 30(3), 247–278.

    Article  Google Scholar 

  29. De Guinea, A. O., Titah, R., & Léger, P. M. (2013). Measure for measure: A two study multi-trait multi-method investigation of construct validity in IS research. Computers in Human Behavior, 29(3), 833–844.

    Article  Google Scholar 

  30. Jay, C., Brown, A., & Harper, S. (2013). Predicting whether users view dynamic content on the world wide web. ACM Transactions on Computer-Human Interaction (TOCHI), 20(2), 1–33.

    Article  Google Scholar 

  31. Dogusoy-Taylan, B., & Cagiltay, K. (2014). Cognitive analysis of experts’ and novices’ concept mapping processes: An eye tracking study. Computers in Human Behavior, 36, 82–93.

    Article  Google Scholar 

  32. Molina, A. I., Redondo, M. A., Lacave, C., & Ortega, M. (2014). Assessing the effectiveness of new devices for accessing learning materials: An empirical analysis based on eye tracking and learner subjective perception. Computers in Human Behavior, 31, 475–490.

    Article  Google Scholar 

  33. Wang, C. C., & Hsu, M. C. (2014). An exploratory study using inexpensive electroencephalography (EEG) to understand flow experience in computer-based instruction. Information & Management, 51(7), 912–923.

    Article  Google Scholar 

  34. Galluch, P. S., Grover, V., & Thatcher, J. B. (2015). Interrupting the workplace: Examining stressors in an information technology context. Journal of the Association for Information Systems, 16(1), 1–47.

    Article  Google Scholar 

  35. Huang, Y. F., Kuo, F. Y., Luu, P., Tucker, D., & Hsieh, P. J. (2015). Hedonic evaluation can be automatically performed: An electroencephalography study of website impression across two cultures. Computers in Human Behavior, 49, 138–146.

    Article  Google Scholar 

  36. Hu, Q., West, R., & Smarandescu, L. (2015). The role of self-control in information security violations: Insights from a cognitive neuroscience perspective. Journal of Management Information Systems, 31(4), 6–48.

    Article  Google Scholar 

  37. Adam, M. T., Krämer, J., & Weinhardt, C. (2012). Excitement up! Price down! Measuring emotions in Dutch auctions. International Journal of Electronic Commerce, 17(2), 7–40.

    Article  Google Scholar 

  38. Riedl, R., Mohr, P. N., Kenning, P. H., Davis, F. D., & Heekeren, H. R. (2014). Trusting humans and avatars: A brain imaging study based on evolution theory. Journal of Management Information Systems, 30(4), 83–114.

    Article  Google Scholar 

  39. Labonté-LeMoyne, É., Santhanam, R., Léger, P. M., Courtemanche, F., Fredette, M., & Sénécal, S. (2015). The delayed effect of treadmill desk usage on recall and attention. Computers in Human Behavior, 46, 1–5.

    Article  Google Scholar 

  40. Association for Information Systems: Senior Scholars’ Basket of Journals. https://aisnet.org/page/SeniorScholarBasket.

  41. van der Aalst, W., Bichler, M., & Heinzl, A. (2016). Open research in business and information systems engineering. Business & Information Systems Engineering, 58(6), 375–379.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dalhousie University, Halifax, Canada

    Colin Conrad & Lyam Bailey

Authors
  1. Colin Conrad
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lyam Bailey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Colin Conrad .

Editor information

Editors and Affiliations

  1. Information Systems and Quantitative Sciences (ISQS), Texas Tech University, Lubbock, TX, USA

    Fred D. Davis

  2. University of Applied Sciences Upper Austria, Steyr, Oberösterreich, Austria

    René Riedl

  3. Department of Information Systems, University of Liechtenstein, Vaduz, Liechtenstein

    Jan vom Brocke

  4. Department of Information Technology, HEC Montréal, Montreal, QC, Canada

    Pierre-Majorique Léger

  5. Department of Information Systems, Kennesaw State University, Kennesaw, GA, USA

    Adriane Randolph

  6. Department for Digital Business, University of Applied Sciences Upper Austria, Steyr, Oberösterreich, Austria

    Thomas Fischer

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Conrad, C., Bailey, L. (2020). What Can NeuroIS Learn from the Replication Crisis in Psychological Science?. In: Davis, F., Riedl, R., vom Brocke, J., Léger, PM., Randolph, A., Fischer, T. (eds) Information Systems and Neuroscience. Lecture Notes in Information Systems and Organisation, vol 32. Springer, Cham. https://doi.org/10.1007/978-3-030-28144-1_14

Download citation

Publish with us

Policies and ethics

Search

Navigation