Abstract
Response processes have been recognized as a primary source of validity evidence but few validation studies have included such evidence. This chapter provides a critical evaluation of the current state of response processes research. We begin by discussing the main factors that have contributed to current shortcomings in the field, and the implications of this situation on validation practices. We next propose several future directions of exploration in response processes research, namely: (a) expanding the theoretical and methodological grounding of this research, (b) expanding the scope of research from studying individual cognitive processes to investigating interpersonal, situated, and sociocultural response processes, and (c) expanding the role of response processes in validation and test development work. The theoretical and methodological directions proposed by the authors invite some new perspectives and renewed energy toward conducting future research studies investigating response processes as they pertain to validating test inferences.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education [AERA, APA, & NCME]. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
Bakhtin, M. (1990). Art and answerability: Early philosophical essays. Austin, TX: University of Texas Press.
Bazerman, C. (1995). Constructing experience. Carbondale, IL: Southern Illinois University Press.
Berkovich-Ohana, A., & Glicksohn, J. (2014). The consciousness state space (CSS) – A model for a unified self and consciousness. Frontiers in Psychology, 5, 1–19.
Bornstein, R. F. (2011). Toward a process-focused model of test score validity: Improving psychological assessment in science and practice. Psychological Assessment, 23, 535–544.
Borsboom, D., Mellenbergh, G. J., & Van Heerden, J. (2004). The concept of validity. Psychological Review, 111, 1061–1071.
Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.
Caracelli, V. J., & Greene, J. C. (1997). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15, 195–207.
Derrida, J. (1982). Eyes of the university: The right to philosophy. Stanford, CA: Stanford University Press.
Edwards, D., & Potter, J. (1992). Discursive psychology. London, UK: SAGE.
Embretson, S. E. (Ed.). (2010). Measuring psychological constructs: Advances in model-based approaches. Washington, DC: American Psychological Association Books.
Embretson, S. E., & Gorin, J. (2001). Improving construct validity with cognitive psychology principles. Journal of Educational Measurement, 38, 343–368.
Erickson, F. (2012). Comments on causality in qualitative inquiry. Qualitative Inquiry, 18, 686–688.
Foucault, M. (1991). Questions of methods. In G. Burchell & P. Miller (Eds.), The Foucault effect: Studies in governmetality (pp. 73–86). Chicago, IL: University of Chicago Press.
Freedman, D. A. (2010). Statistical models and shoe leather. In D. A. Freedman (Ed.), Statistical models and causal inference: A dialogue with the social sciences (pp. 45–62). Cambridge, UK: Cambridge University Press.
Geertz, C. (1973). The interpretation of cultures. New York, NY: Basic Books.
Gergen, K. (2009). Relational being: Beyond self and community. New York, NY: Oxford University Press.
Guba, E. G., & Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 191–215). Thousand Oaks, CA: SAGE.
Halkier, B. (2011). Methodological practicalities in analytical generalization. Qualitative Inquiry, 17, 787–797.
Heidegger, M. (1975). Poetry, thought, language. New York, NY: Harper & Collins Perennial Library.
Hesse-Biber, S. N., & Burke Johnson, R. (2015). The Oxford handbook of multimethods and mixed methods research inquiy. Oxford, UK: Oxford Library of Psychology.
Hoffmeyer, J. (2013). Why do we need a semiotic understanding of life? Beyond mechanism: Putting life back into biology. Plymouth, UK: Lexington Books.
Holtgraves, T. (2004). Social desirability and self-reports: Testing models of socially desirable responding. Personality and Social Psychology Bulletin, 30, 161–172. doi:10.1177/0146167203259930.
Lamiell, J. T. (1987). The psychology of personality: An epistemological inquiry. New York, NY: Columbia University Press.
Leighton, J. P. (2015). Accounting for affective states in response processing data: Impact for validation. Paper presented at the annual meeting of the National Council on Measurement in Education (NCME), Chicago, IL, USA.
Leontiev, D. (2014). Extending the contexts of existence: Benefits of meaning-guided living. In A. Batthyany (Ed.), Meaning in existential and positive psychology (pp. 97–114). Dodrecht, The Netherlands: Springer.
Linell, P. (2009). Rethinking language, mind, and world dialogically. Charlotte, NC: Information Age Publishing.
Long, D. M. (2013). Pragmatism, realism, and psychology: Understanding theory selection criteria. Journal of Contextual Behavioral Science, 2, 61–67.
Markus, K. A., & Borsboom, D. (2013). Frontiers of test validity theory: Measurement, causation, and meaning. New York, NY: Routledge.
Maturana, H. (1990). Biology of cognition and epistemology. Temuco, Chile: Ed Universidad de la Frontera.
Maxcy, S. J. (2003). Pragmatic threads in mixed methods research for multiple modes: The search for multiple modes of inquiry and the end of the philosophy of formalism. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioal research (pp. 51–89). Thousand Oaks, CA: SAGE.
Maxwell, J. A. (2012). The importance of qualitative research for causal explanation in education. Qualitative Inquiry, 18, 655–661. doi:10.1080/14733140112331385100.
Mehl, M. R., & Connor, T. S. (2012). Handbook for research methods for studying daily life. New York, NY: Guildford Press.
Merten, D. M. (2013). Mixed methods and the politics of human research: The transformative-emancipatory perspective. In A. Tasakkori & C. Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp. 135–164). Thousand Oaks, CA: SAGE.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from person’s responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 747–749.
Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., et al. (2001). Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128–165.
Mischel, W., & Shoda, Y. (1995). A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102, 246–268. doi:10.1037/0033-295X.102.2.246.
Nalini, A., & Rosenthal, R. (1992). Thin slices of expressive behavior as predictors of interpersonal consequences. Psychological Bulletin, 111, 256–274.
O’Donnell, C. R., Tharp, R. G., & Wilson, K. (1993). Activity settings as the unit of analysis: A theoretical basis for community intervention and development. American Journal of Community Psychology, 21, 501–520. doi:10.1007/BF00942157.
Pearce, L. D. (2015). Thinking outside the Q boxes: Further motivating a mixed research perspective. In S. N. Hesse-Biber & R. Burke Johnson (Eds.), The Oxford handbook of mixed and multimethod research. New York, NY: Oxford University Press.
Piaget, J. (1972). The principles of genetic epistomology. New York, NY: Basic Books.
Rauthmann, J. F., Gallardo-Pujol, D., Guillaume, E. M., Todd, E., Nave, C. S., Sherman, R. A., & Funder, D. C. (2014). The Situational Eight DIAMONDS: A taxonomy of major dimensions of situation characteristics. Journal of Personality and Soical Psychology, 107, 677–718. doi:10.1037/a0037250.
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105.
Shotter, J. (1993). Conversational realities: Constructing life through language. Thousand Oaks, CA: SAGE.
Tashakkori, A., & Teddlie, C. (Eds.). (2010). Handbook of mixed methods in social and behavioral research (2nd ed.). Thousand Oaks, CA: SAGE.
Toomela, A. (2009). How methodology became a toolbox – And how it escapes from that box. In J. Valsiner, P. Molenaar, M. Lyra, & N. Chaudhary (Eds.), Dynamic process methodology in the social and delevolpmental sicences (pp. 45–66). New York, NY: Springer.
Tourangeau, R., Rips, L. J., & Rasinski, K. (2004). The psychology of survey response. Cambridge, UK: Cambridge University Press.
Tukey, J. W. (1993). Issues relevant to an honest account of data-based inference, partially in the light of Laurie Davies’ paper. Princeton, NJ: Princeton University.
van Fraassen, B. C. (1980). The scientific image. Oxford, UK: Oxford University Press.
Varela, F. J., Thompson, E., & Rosch, E. (1991). The embodied mind: Cognitive science and human experience. Cambridge, MA: MIT Press.
Vygotsky, L. S. (1994). The problem of the environment. In R. van der Veer & J. Valsiner (Eds.), The Vygotsky reader (pp. 338–354). Oxford, UK: Blackwell.
Wagoner, B. (2009). The experimental methodology of constructive microgenesis. In J. Valsiner, P. Molenaar, N. Chaudhary, & M. Lyra (Eds.), Handbook of dynamic process methodology in the social and developmental sciences (pp. 99–121). New York, NY: Springer.
Westerman, M. A. (2003). Quantitative research as an interpretive enterprise: The mostly unacknowledged role of interpretation in research efforts and suggestions for explicitly interpretive quantitative investigations. New Ideas in Psychology, 24, 189–211.
Wills, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: SAGE.
Wong, V. C., Wing, C., Steiner, P. M., Wong, M., & Cook, T. D. (2012). Research designs for program evalutaion. In I. B. Weiner, J. A. Schinka, & W. F. Velicer (Eds.), Handbook of psychology, Reseach methods in psychology (Vol. 2, 2nd ed., pp. 316–341). Hoboken, NJ: Wiley.
Zumbo, B. D. (2009). Validity as contextualized and pragmatic explanation, and its implications for validation practice. In R. W. Lissitz (Ed.), The concept of validity: Revisions, new directions and applications (pp. 65–82). Charlotte, NC: Information Age Publishing.
Zumbo, B. D., Liu, Y., Wu, A. D., Shear, B. R., Olvera Astivia, O. L., & Ark, T. (2015). A methodology for Zumbo’s third generation DIF analyses and the ecology of item responding. Language Assessment Quarterly, 12, 136–151. doi:10.1080/15434303.2014.972559.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Launeanu, M., Hubley, A.M. (2017). Some Observations on Response Processes Research and Its Future Theoretical and Methodological Directions. In: Zumbo, B., Hubley, A. (eds) Understanding and Investigating Response Processes in Validation Research. Social Indicators Research Series, vol 69. Springer, Cham. https://doi.org/10.1007/978-3-319-56129-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-56129-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56128-8
Online ISBN: 978-3-319-56129-5
eBook Packages: Social SciencesSocial Sciences (R0)