Advertisement

Do Not Cross Me: Optimizing the Use of Cross-Sectional Designs

  • Paul E. SpectorEmail author
Original Research

Abstract

The cross-sectional research design, especially when used with self-report surveys, is held in low esteem despite its widespread use. It is generally accepted that the longitudinal design offers considerable advantages and should be preferred due to its ability to shed light on causal connections. In this paper, I will argue that the ability of the longitudinal design to reflect causality has been overstated and that it offers limited advantages over the cross-sectional design in most cases in which it is used. The nature of causal inference from a philosophy of science perspective is used to illustrate how cross-sectional designs can provide evidence for relationships among variables and can be used to rule out many potential alternative explanations for those relationships. Strategies for optimizing the use of cross-sectional designs are noted, including the inclusion of control variables to rule out spurious relationships, the addition of alternative sources of data, and the incorporation of experimental methods. Best practice advice is offered for the use of both cross-sectional and longitudinal designs, as well as for authors writing and for reviewers evaluating papers that report results of cross-sectional studies.

Keywords

Causal inference Causality Cross-sectional design Longitudinal design Method variance Philosophy of science Research design Research methodology 

Notes

References

  1. Antonakis, J., Bendahan, S., Jacquart, P., & Lalive, R. (2010). On making causal claims: A review and recommendations. The Leadership Quarterly, 21(6), 1086–1120.  https://doi.org/10.1016/j.leaqua.2010.10.010.Google Scholar
  2. Berofsky, B. (1966). Causality and general laws. The Journal of Philosophy, 63(6), 148–157.  https://doi.org/10.2307/2024170.Google Scholar
  3. Bolino, M. C. (1999). Citizenship and impression management: Good soldiers or good actors? The Academy of Management Review, 24(1), 82–98.  https://doi.org/10.2307/259038.Google Scholar
  4. Brief, A. P., Burke, M. J., George, J. M., Robinson, B. S., & Webster, J. (1988). Should negative affectivity remain an unmeasured variable in the study of job stress? Journal of Applied Psychology, 73(2), 193–198.Google Scholar
  5. Chen, P. Y., & Spector, P. E. (1991). Negative affectivity as the underlying cause of correlations between stressors and strains. Journal of Applied Psychology, 76(3), 398–407.  https://doi.org/10.1037/0021-9010.76.3.398.Google Scholar
  6. Cole, D. A., Martin, N. C., & Steiger, J. H. (2005). Empirical and conceptual problems with longitudinal trait-state models: Introducing a trait-state-occasion model. Psychological Methods, 10(1), 3–20.  https://doi.org/10.1037/1082-989X.10.1.3.Google Scholar
  7. Dalal, R. S. (2005). A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology, 90(6), 1241–1255.  https://doi.org/10.1037/0021-9010.90.6.1241.Google Scholar
  8. Frese, M., & Zapf, D. (1988). Methodological issues in the study of work stress: Objective vs subjective measurement of work stress and the question of longitudinal studies. In C. L. Cooper & R. Payne (Eds.), Causes, coping and consequences of stress at work (pp. 375–411). Oxford, England: John Wiley & Sons.Google Scholar
  9. Glick, W. H., Huber, G. P., Miller, C. C., Doty, D. H., & Sutcliffe, K. M. (1990). Studying changes in organizational design and effectiveness: Retrospective event histories and periodic assessments. Organization Science, 1(3), 293–312.  https://doi.org/10.2307/2635007.Google Scholar
  10. Glick, W. H., Jenkins, G., & Gupta, N. (1986). Method versus substance: How strong are underlying relationships between job characteristics and attitudinal outcomes? Academy of Management Journal, 29(3), 441–464.Google Scholar
  11. Griffin, R. W. (1991). Effects of work redesign on employee perceptions, attitudes, and behaviors: A long-term investigation. Academy of Management Journal, 34(2), 425–435.  https://doi.org/10.2307/256449.Google Scholar
  12. Hausman, D. M., & Woodward, J. (1999). Independence, invariance and the causal Markov condition. The British Journal for the Philosophy of Science, 50(4), 521–583.  https://doi.org/10.1093/bjps/50.4.521.Google Scholar
  13. Illari, P., & Russo, F. (2014). Causality: Philosophical theory meets scientific practice. Oxford, UK: Oxford University Press.Google Scholar
  14. Imai, K., Keele, L., & Tingley, D. (2010). A general approach to causal mediation analysis. Psychological Methods, 15(4), 309–334.  https://doi.org/10.1037/a0020761.Google Scholar
  15. Kraemer, H. C., Stice, E., Kazdin, A., Offord, D., & Kupfer, D. (2001). How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. The American Journal of Psychiatry, 158(6), 848–856.  https://doi.org/10.1176/appi.ajp.158.6.848.Google Scholar
  16. Meehl, P. E. (1971). High school yearbooks: A reply to Schwarz. Journal of Abnormal Psychology, 77(2), 143–148.Google Scholar
  17. Mitchell, T. R., & James, L. R. (2001). Building better theory: Time and the specification of when things happen. The Academy of Management Review, 26(4), 530–547.  https://doi.org/10.2307/3560240.Google Scholar
  18. Nixon, A. E., Mazzola, J. J., Bauer, J., Krueger, J. R., & Spector, P. E. (2011). Can work make you sick? A meta-analysis of the relationships between job stressors and physical symptoms. Work & Stress, 25(1), 1–22.  https://doi.org/10.1080/02678373.2011.569175.Google Scholar
  19. Pearl, J. (2014). Interpretation and identification of causal mediation. Psychological Methods, 19(4), 459–481.  https://doi.org/10.1037/a0036434.Google Scholar
  20. Pindek, S., & Spector, P. E. (2016). Organizational constraints: A meta-analysis of a major stressor. Work & Stress, 30(1), 7–25.  https://doi.org/10.1080/02678373.2015.1137376.Google Scholar
  21. Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569.  https://doi.org/10.1146/annurev-psych-120710-100452.Google Scholar
  22. Schwarzmüller, T., Brosi, P., & Welpe, I. M. (2018). Sparking anger and anxiety: Why intense leader anger displays trigger both more deviance and higher work effort in followers. Journal of Business and Psychology, 33(6), 761–777.  https://doi.org/10.1007/s10869-017-9523-8.Google Scholar
  23. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.Google Scholar
  24. Spector, P. E. (1992). A consideration of the validity and meaning of self-report measures of job conditions. In C. L. Cooper & I. T. Robertson (Eds.), International Review of Industrial and Organizational Psychology (pp. 123–151). West Sussex, UK: John Wiley.Google Scholar
  25. Spector, P. E. (2006). Method variance in organizational research: Truth or urban legend? Organizational Research Methods, 9(2), 221–232.  https://doi.org/10.1177/1094428105284955.Google Scholar
  26. Spector, P. E., Bauer, J. A., & Fox, S. (2010). Measurement artifacts in the assessment of counterproductive work behavior and organizational citizenship behavior: Do we know what we think we know? Journal of Applied Psychology, 95(4), 781–790.  https://doi.org/10.1037/a0019477.Google Scholar
  27. Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2), 287–305.  https://doi.org/10.1177/1094428110369842.Google Scholar
  28. Spector, P. E., Fox, S., & Van Katwyk, P. T. (1999). The role of negative affectivity in employee reactions to job characteristics: Bias effect or substantive effect? Journal of Occupational and Organizational Psychology, 72(2), 205–218.  https://doi.org/10.1348/096317999166608.Google Scholar
  29. Spector, P. E., & Meier, L. L. (2014). Methodologies for the study of organizational behavior processes: How to find your keys in the dark. Journal of Organizational Behavior, 35(8), 1109–1119.  https://doi.org/10.1002/job.1966.Google Scholar
  30. Spector, P. E., & Pindek, S. (2016). The future of research methods in work and occupational health psychology. Applied Psychology: An International Review, 65(2), 412–431.  https://doi.org/10.1111/apps.12056.Google Scholar
  31. Spector, P. E., Rosen, C. C., Richardson, H. A., Williams, L. J., & Johnson, R. E. (2017). A new perspective on method variance: A measure-centric approach. Journal of Management, 0(0), 0149206316687295.  https://doi.org/10.1177/0149206316687295.Google Scholar
  32. Spector, P. E., Yang, L.-Q., & Zhou, Z. E. (2015). A longitudinal investigation of the role of violence prevention climate in exposure to workplace physical violence and verbal abuse. Work & Stress, 29(4), 325–340.  https://doi.org/10.1080/02678373.2015.1076537.Google Scholar
  33. Stone-Romero, E. F., & Rosopa, P. J. (2008). The relative validity of inferences about mediation as a function of research design characteristics. Organizational Research Methods, 11(2), 326–352.  https://doi.org/10.1177/1094428107300342.Google Scholar
  34. Tuma, N. B., & Hannan, M. T. (1984). Social dynamics models and methods. Saint Louis, US: Elsevier.Google Scholar
  35. Watson, D., Pennebaker, J. W., & Folger, R. (1986). Beyond negative affectivity: Measuring stress and satisfaction in the workplace. Journal of Organizational Behavior Management, 8(2), 141–157.  https://doi.org/10.1300/J075v08n02_09.Google Scholar
  36. Woodward, J. (2003). Making things happen: A theory of causal explanation. Oxford: New York City.Google Scholar
  37. Woodward, J. (2017). Scientific explanation. In E. N. Zalta (Ed.), Stanford encyclopedia of philosophy. Retrieved from https://plato.stanford.edu/archives/fall2017/entries/scientific-explanation. Accessed 28 Dec 2018.
  38. Zapf, D., Dormann, C., & Frese, M. (1996). Longitudinal studies in organizational stress research: A review of the literature with reference to methodological issues. Journal of Occupational Health Psychology, 1(2), 145–169.  https://doi.org/10.1037/1076-8998.1.2.145.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of PsychologyUniversity of South FloridaTampaUSA

Personalised recommendations