Advertisement

Using cognitive interviews to pilot an international survey of principal preparation: A Western Australian perspective

Article

Abstract

This paper provides an example of the application of the cognitive interview, a qualitative tool for pre-testing a survey instrument to check its cognitive validity, that is, whether the items mean to respondents what they mean to the item designers. The instrument is the survey used in the final phase of the International Study of Principal Preparation (ISPP http://www.ucalgary.ca/~cwebber/ISPP/index.htm). This study involving researchers in 13 countries investigates those aspects of principals’ work perceived by them to be most challenging in their early years in the position and the extent to which principals believe they were prepared for these challenges. Both probing and think aloud approaches to the cognitive interview were used and revealed a small number of items requiring amendment.

Keywords

Cognitive interviews Qualitative research Survey development Principal preparation 

References

  1. Ackermann, A.C., & Blair, J. (2006). Efficient respondent selection for cognitive interviewing. In Proceedings of the Survey Research Methods Section, American Statistical Association. http://www.amstat.org/sections/srms/Proceedings/y2006f.html. Accessed 21 August 2008.
  2. Andrich, D. (1988). Rasch models for measurement. Sage University Series on Quantitative Approaches to the Social Sciences, pp. 07–068. Beverley Hills & London: Sage.Google Scholar
  3. Andrich, D., Sheridan, B. S., & Luo, G. (2005). RUMM2020: Rasch Unidimensional Models for Measurement. Perth, Western Australia: RUMM Laboratory.Google Scholar
  4. Blair, J., Conrad, F., Ackermann, A.C., & Claxton, G. (2006). The effect of sample size on cognitive interview findings. In Proceedings of the Survey Research Methods Section, American Statistical Association. http://www.amstat.org/sections/srms/Proceedings/y2006f.html. Accessed 21 August 2008.
  5. Clarke, S., Wildy, H., & Pepper, C. (2007). Connecting preparation with reality: Primary principals’ experiences of their first year out in Western Australia. Leading & Managing, 13(1), 81–90.Google Scholar
  6. Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238. doi: 10.1023/A:1023254226592.CrossRefGoogle Scholar
  7. Cowie, M., & Crawford, M. (2007). Principal preparation—still an act of faith? School Leadership & Management, 27(2), 129–146. doi: 10.1080/13632430701237198.CrossRefGoogle Scholar
  8. Desimone, L. M., & Le Floch, K. C. (2004). Are we asking the right questions: Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis, 26(1), 1–22. doi: 10.3102/01623737026001001.CrossRefGoogle Scholar
  9. Drennan, J. (2003). Cognitive interviewing: Verbal data in the design and pretesting of questionnaires. Journal of Advanced Nursing, 42(1), 52–63. doi: 10.1046/j.1365-2648.2003.02579.x.CrossRefGoogle Scholar
  10. Ericsson, K. A., & Simon, H. (1984). Protocol analysis, verbal reports as data. Cambridge, MA: MIT.Google Scholar
  11. Karabenick, S., Woolley, M., Friedel, J., Ammon, B., Blazevski, J., Bonney, C., et al. (2007). Cognitive processing of self-report items in educational research: Do they think what we mean? Educational Psychologist, 42(3), 139–151.Google Scholar
  12. Knafl, K., Deatrick, J., Gallo, A., Holcombe, G., Bakitas, M., Dixon, J., et al. (2007). The analysis and interpretation of cognitive interviews for instrument development. Research in Nursing & Health, 30(2), 224–234. doi: 10.1002/nur.20195.CrossRefGoogle Scholar
  13. Rasch, G. (1960). Probablistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research)). Expanded edition (1980) with foreword and afterword by B.D. Wright. Chicago: The University of Chicago Press.Google Scholar
  14. Rasch, G. (1980). Probablistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research)). Expanded edition (1980) with foreword and afterword by B.D. Wright. Chicago: The University of Chicago Press.Google Scholar
  15. Slater, C. L., Boone, M., Nelson, S., De La Colina, M., Garcia, E., Grimaldo, L., et al. (2007). El Escalafón y el Doble Turno: An international perspective on school director preparation. Journal for Educational Research and Policy Studies, 6(2), 60–90.Google Scholar
  16. Wildy, H., & Clarke, S. (2008a). Charting an arid landscape: The preparation of novice primary principals in Western Australia. School Leadership & Management, 28(5), 469–487. doi: 10.1080/13632430802500106.CrossRefGoogle Scholar
  17. Wildy, H., & Clarke, S. (2008b). Principals on L-plates: Rear view mirror reflections. Journal of Educational Administration, 46(6), 727–738. doi: 10.1108/09578230810908316.CrossRefGoogle Scholar
  18. Wildy, H., Clarke, S., & Slater, C. (2007). International perspectives of principal preparation: How does Australia fare? Leading & Managing Special Edition, 13(2), 1–14.Google Scholar
  19. Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage.Google Scholar
  20. Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. C. Herschberger (Eds.), The new rules of measurement, pp. 65–104. New York: Lawrence Erlbaum.Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2009

Authors and Affiliations

  1. 1.The University of Western AustraliaNedlandsAustralia

Personalised recommendations