Skip to main content
SpringerLink
Log in

Constructing and Evaluating Self-Report Measures

  • Chapter
  • First Online:
Principles of Research Methodology

Abstract

A self-report (a.k.a. survey) is a measure where the respondent supplies information about him or herself. Self-reports are important in medical research because some variables (e.g., attitudes, beliefs, self-judged ability) only can be assessed from information directly furnished by the patient or other subject. A self-report is obtained by questionnaire, interview, or related methods. Questionnaires are written documents that can be self-completed without interviewer involvement or read aloud as part of an interview; interviews usually (but not always) are administered orally; both can be structured (comprise closed-ended questions), unstructured (comprise open-ended questions), or semistructured (comprise a mix of both question types). If answers to a research question can be obtained only via self-report, the investigator should first determine whether an instrument already exists that is reliable, valid, and otherwise suitable for the population of interest. In situations where a new instrument must be developed, the investigator must clearly define the question(s) of interest; identify the population to be surveyed; select the preferred type of self-report/format of measurement; consider inclusion of validation questions, pretest, and pilot test and edit the measure; and test the final battery of questions for reliability and validity. When developing or implementing a survey, the investigator must be certain to observe all ethical and legal aspects of survey methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Polit DF, Beck CT. Nursing research: principles and methods. 7th ed. Philadelphia: Lippincott, Williams and Wilkins; 2004.

    Google Scholar 

  2. Kish L. Survey sampling. New York: Wiley; 1995.

    Google Scholar 

  3. Groves RM, Fowler FJ, Couper MP, Lepkowski JM, Singer E. Survey methodology. New York: Wiley; 2004.

    Google Scholar 

  4. Cochran WG. Sampling techniques. 3rd ed. New York: Wiley; 1977.

    Google Scholar 

  5. Derogatis LR. BSI: Brief Symptom Inventory: administration, scoring and procedures manual. Minneapolis: National Computer Systems; 1993.

    Google Scholar 

  6. Ware JE, Snow KK, Kosinski M, Gandek B. SF-36 health survey: manual and interpretation guide. Lincoln: RI, QualityMetric, Inc.; 2000.

    Google Scholar 

  7. Skevington SM, Bradshaw J, Saxena S. Selecting national items for the WHOQOL: conceptual and psychometric considerations. Soc Sci Med. 1999;48:473–487.

    Article  PubMed  CAS  Google Scholar 

  8. Rector TS, Cohn JN. Assessment of patient outcome with the Minnesota Living with Heart Failure Questionnaire: reliability and validity during a randomized, double-blind, placebo-controlled trial of pimobendan. Pimobendan Multicenter Research Group. Am Heart J. 1992;124:1017–25.

    Article  PubMed  CAS  Google Scholar 

  9. Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of medicine adherence. Med Care. 1986;24:67–72.

    Article  PubMed  CAS  Google Scholar 

  10. Strauss AL, Corbin JM. Basics of qualitative research: techniques and procedures for developing grounded theory. 2nd ed. Thousand Oaks: Sage; 1998.

    Google Scholar 

  11. Cohen MZ, Ley C, Tarzian AJ. Isolation in blood and marrow transplantation. West J Nurs Res. 2001;25:37–48.

    Google Scholar 

  12. Fadiman A. The spirit catches you and you fall down: a Hmong child, her American doctors, and the collision of two cultures. New York: Farrar, Straus and Giroux; 1998.

    Google Scholar 

  13. Drever E. Using semi-structured interviews in small-scale research, a teacher’s guide. ERIC. Edinburgh: SCRE; 1995.

    Google Scholar 

  14. Benson J, Britten N. Respecting the autonomy of cancer patients when talking with their families: qualitative analysis of semistructured interviews with patients. BMJ. 1996;313:729–731.

    Article  PubMed  CAS  Google Scholar 

  15. O’Dea JA. Consumption of nutritional supplements among adolescents: usage and perceived benefits. Health Educ Res. 2003;18:98–107.

    Article  PubMed  Google Scholar 

  16. Allan G. A note on interviewing spouses together. J Marriage Fam. 1980;42:205–210.

    Article  Google Scholar 

  17. Kalischuk RG, Davies B. A theory of healing in the aftermath of youth suicide. J Holist Nurs. 2001;19:163–186.

    Article  PubMed  CAS  Google Scholar 

  18. Dolezal C, Mellins C, Brackis-Cott E, Abrams EJ. The reliability of reports of medical adherence from children with HIV and their adult care givers. J Pediatr Psychol. 2003;28:355–361.

    Article  PubMed  Google Scholar 

  19. Dym B, Berman S. The primary health care team: family physician and family therapist in joint practice. Fam Syst Med. 1986;4:9–21.

    Article  Google Scholar 

  20. Morrison-Beedy D, Côté-Arsenault D, Feinstein NF. Maximizing results with focus groups: moderator and analysis issues. Appl Nurs Res. 2001;14:48–53.

    Article  PubMed  CAS  Google Scholar 

  21. Quatromoni PA, Milbauer M, Posner BM, Carballeira NP, Brunt M, Chipkin SR. Use of focus groups to explore nutrition practices and health beliefs of urban Caribbean Latinos with diabetes. Diabetes Care. 1994;17:869–873.

    Article  PubMed  CAS  Google Scholar 

  22. Hicks LK, Lin Y, Robertson DW, Robinson DL, Woodrow SI. Understanding the clinical dilemmas that shape medical students’ ethical development: questionnaire survey and focus group study. BMJ. 2001;322:709–710.

    Article  PubMed  CAS  Google Scholar 

  23. Flanagan JC. The critical incident technique. Psychol Bull. 1954;51:327–358.

    Article  PubMed  CAS  Google Scholar 

  24. Coté CJ, Notterman DA, Karl HW, Weinberg JA, McClosky C. Adverse sedation events in pediatrics: a critical incident analysis of contributing factors. Pediatrics. 2000;105:805–14.

    Article  PubMed  Google Scholar 

  25. Branch W, Pels RJ, Arky R. Becoming a doctor. Critical-incident reports from third-year medical students. N Engl J Med. 1993;329:1130–1132.

    Article  PubMed  CAS  Google Scholar 

  26. Allery LA, Owen PA, Robling MR. Why general practitioners and consultants change their clinical practice: a critical incident study. BMJ. 1997;314:870–874.

    Article  PubMed  CAS  Google Scholar 

  27. Faithfull S. The diary method for nursing research. Eur J Cancer Care. 2007;1:13–18.

    Article  Google Scholar 

  28. Bruijnzeels NA, Foets M, van der Wooden JC, Prins A, van den Houvel WJ. Measuring morbidity of children in the community: a comparison of interview and diary data. Int J Epidemiol. 1998;27:96–100.

    Article  PubMed  CAS  Google Scholar 

  29. White MM, Howie-Esquivel J, Caldwell MA. Improving heart failure symptom recognition: a diary analysis. Cardiovasc Nurs. 2010;25:7–12.

    Google Scholar 

  30. Woodfield R, Goodyear-Smith F, Arroll B. N-of-1 trials of quinine efficacy in skeletal muscle cramps of the leg. Br J Gen Pract. 2005;55(512):181–185.

    PubMed  Google Scholar 

  31. Aitken L, Mardegan KJ. Thinking aloud: data collection in the natural setting. Western J Nurs Res. 2000;22:841–853.

    Article  CAS  Google Scholar 

  32. Fonetyn M, Fisher A. Use of think aloud method to study nurse’s reasoning and decision making in clinical practice settings. J Neurosci Nurs. 1995;27:124–128.

    Article  Google Scholar 

  33. Ericsson K, Simon H. Protocol analysis: verbal reports as data. London: MIT Press; 1993.

    Google Scholar 

  34. Murphy LL, Spies RA, Plake BS, editors. Tests in print VII. Lincoln: Buros Institute of Mental Measurements; 2006.

    Google Scholar 

  35. Geisinger KF, Spies RA, Carlson JF, Plake BS, editors. The seventeenth mental measurements yearbook. Lincoln: Buros Institute of Mental Measurements; 2007.

    Google Scholar 

  36. Goldman BA, Mitchell DF, Egelson PE, editors. Directory of unpublished experimental mental measures. Washington, DC: American Psychological Association; 2007.

    Google Scholar 

  37. Bieri D, Reeve R, Champion GD, Addicoat L, Ziegler JB. The Faces Pain Scale for the self-­assessment of the severity of pain experienced by children: development, initial validation and preliminary investigation for ratio scale properties. Pain. 1990;41:139–150.

    Article  PubMed  CAS  Google Scholar 

  38. Choi BCK, Pak AWP. A catalog of biases in questionnaires. Prev Chronic Dis. 2005;2:1–13.

    Article  Google Scholar 

  39. Fowler FJ. Improving survey questions. Thousand Oaks: Sage; 1995.

    Google Scholar 

  40. DeVellis RF. Scale development: theory and applications. Newbury Park: Sage; 1991.

    Google Scholar 

  41. Chang AM, Chau JPC, Holroyd E. Translation of questionnaires and issues of equivalence. J Adv Nurs. 2010;29:316–322.

    Article  Google Scholar 

  42. Healey B, Gendall P. Asking the age question in mail and online surveys. Austral and New Zeal Marketing Acad (ANZMAC) Conference 2007. Dunedin; 2007.

    Google Scholar 

  43. Heise DR. The semantic differential and attitude research. In: Summers GF, editor. Attitude measurement. Chicago: Rand McNally; 1970.

    Google Scholar 

  44. Aiken LR. Rating scales and checklists. New York: Wiley; 1996.

    Google Scholar 

  45. DeLoach LJ, Higgins MS, Caplan AB, Stiff JL. The visual analog scale in the immediate postoperative period: intrasubject variability and correlation with a numeric scale. Anesth Analg. 1998;86:102–106.

    PubMed  CAS  Google Scholar 

  46. Brealey SD, Atwell C, Bryan S, Coulton S, Cox H, Cross B, Fylan F, Garratt A, Gilbert FG, Gillan MGC, Hendry M, Hood K, Houston H, King D, Morton V, Orchard J, Robling M, Russell IT, Torgerson D, Wadsworth V, Wilkinson C. Improving response rates using a monetary incentive for patient completion of questionnaires: an observational study. BMC Med Res Methodol. 2007;7:12–16.

    Article  PubMed  Google Scholar 

  47. Asch D, Jedrziewski MK, Christakis N. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50:1129–1136.

    Article  PubMed  CAS  Google Scholar 

  48. Diment K, Garrett-Jones S. How demographic characteristics affect mode preference in a postal/web mixed survey of Australian researchers. Soc Sci Comput Rev. 2007;25:410–417.

    Article  Google Scholar 

  49. Shih TH. Comparing response rates from web and mail surveys: a meta-analysis. Field Methods. 2008;20:249–271.

    Article  Google Scholar 

  50. O’Toole J, Sinclair M, Leder K. Maximising response rates in household telephone surveys. BMC Med Res Methodol. 2008;8:71.

    Article  PubMed  Google Scholar 

  51. Tourangeau R, Smith TW. Asking sensitive questions: the impact of data collection mode, question format and question context. Public Opin Q. 1996;60:275–304.

    Article  Google Scholar 

  52. Turner CF, Al-Tayyib AA, Rogers SM, Eggleston MA, Villarroel MA, Roman AM, Chromy JR, Cooley PC. Improving epidemiological surveys of sexual behavior conducted by telephone. Int J Epidemiol. 2009;38:1118–1127.

    Article  PubMed  Google Scholar 

  53. Couper MP, Nicholls II WL. The history and development of computer assisted survey information collection methods. In: Couper MP et al., editors. Computer assisted survey information collection. New York: Wiley; 1998.

    Google Scholar 

  54. Vataja R, Pohjasvaara T, Leppävuori A, Mäntylä R, Aronen HJ, Salonen O, Kaste M, Erkinjuntti T. Magnetic resonance imaging correlates of depression after ischemic stroke. Arch Gen Psychiatry. 2001;58:925–31.

    Article  PubMed  CAS  Google Scholar 

  55. Schackman BR, Dastur Z, Rubin DS, Berger J, Camhi E, Netherland J, Ni Q, Finkelstein R. Feasibility of using audio computer-assisted self-interview (ACASI) screening in routine HIV care. AIDS Care. 2009;21:992–999.

    Article  PubMed  Google Scholar 

  56. Oetting ER, Beauvais F. Adolescent drug use. J Consult Clin Psychol. 1990;58:385–394.

    Article  PubMed  CAS  Google Scholar 

  57. Fidler DS, Kleinknec RE. Randomized response versus direct questioning: two data-collection methods for sensitive information. Psychol Bull. 1977;84:1045–1049.

    Article  Google Scholar 

  58. Lensvelt-Mulders GJLM, Hox JJ, van der Heijden PGM, Maas CJM. Meta-analysis of randomized response research. Sociol Method Res. 2005;33:319–348.

    Article  Google Scholar 

  59. Edwards P, Roberts I, Clarke M, DiGuisseppi C, Pratap S, Wentz R, Kwan I. Increasing response rates to postal questionnaires. BMJ. 2002;324:1183–91.

    Article  PubMed  Google Scholar 

  60. Brennan M, Charbonnau J. Improving mail survey response rate using chocolate and replacement questionnaires. Public Opin Q. 2009;73:368–378.

    Article  Google Scholar 

  61. Merriam-Webster Online. Available at http://www.m-w.com/. Accessed 27 July 2010.

  62. Waltz CF, Strickland OL, Lenz ER. Measurement in nursing and research. New York: Springer Publishing Inc; 2005.

    Google Scholar 

  63. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166.e7–166.e16.

    Article  Google Scholar 

  64. Litwin MS. How to measure survey reliability and validity. Thousand Oaks: Sage; 1995.

    Google Scholar 

  65. Beck AT, Steer R, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio: Psychological Corporation; 1996.

    Google Scholar 

  66. Williams RA. Women’s health content validity of the family medicine in-training exam. Fam Med. 2007;39:572–577.

    PubMed  Google Scholar 

  67. Ware JE, Sherbourne CD. The MOS 36 item short form health survey. Med Care. 1992;30:473–483.

    Article  PubMed  Google Scholar 

  68. Feldman AB, Haley SM, Coryell J. Concurrent and construct validity of the pediatric evaluation of disability inventory. Phys Ther. 1990;70:602–610.

    PubMed  CAS  Google Scholar 

  69. Lin JM, Brimmer DJ, Maloney EM, Nyarko E, BeLue R, Reeves WC. Further validation of the Multidimensional Fatigue Inventory in a US adult population sample. Popul Health Metr. 2009; 7:18 doi:10.1186/1478-7954-7-18.

  70. McHorney CA, Ware Jr JE, Raczek AE. The MOS 36-item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 1993;31:247–263.

    Article  PubMed  CAS  Google Scholar 

  71. Management Sciences for Health. Creating a climate that motivates staff and improves performance. The Manager. 2003;11:1–22.

    Google Scholar 

  72. Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Parkinson J, Secker J, Stewart-Brown S. The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS): development and UK validation. Health and Quality of Life Outcomes 2007; 5:63doi:10.1186/1477-7525-5-63.

  73. Cooper SM, Baker JS, Tong RJ, Roberts E, Hanford M. The repeatability and criterion related validity of the 20 m Multistage Fitness Test as a predictor of maximal oxygen uptake in active young men. Br J Sports Med. 2005;39:e19.

    Article  PubMed  Google Scholar 

  74. Reuben DB, Siu AL, Kimpau S. The predictive validity of self-report and performance-based measures of function and health. J Gerontol. 1991;47:M106–M110.

    Google Scholar 

  75. Heather N, Rollnick S, Bell A. Predictive validity of the readiness to change questionnaire. Addiction. 1993;88:1667–1677.

    Article  PubMed  CAS  Google Scholar 

  76. Portney LG, Watkins MP. Foundations of clinical research. Applications to practice. Upper Saddle River: Prentice Hall Health; 2000.

    Google Scholar 

  77. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis. 1987;40:171–178.

    Article  PubMed  CAS  Google Scholar 

  78. Hays RD, Hadorn D. Responsiveness to change: an aspect of validity, not a separate dimension. Qual Life Res. 1992;1:73–75.

    Article  PubMed  CAS  Google Scholar 

  79. Beaton DE, Bombadier C, Katz JN, Wright JG. A taxonomy for responsiveness. J Clin Epidemiol. 2001;54:1204–1217.

    Article  PubMed  CAS  Google Scholar 

  80. Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol. 2000;53:459–468.

    Article  PubMed  CAS  Google Scholar 

  81. Roach KE. Measurement of health outcomes: reliability, validity and responsiveness. JPO. 2006;18:8–12.

    Google Scholar 

  82. Liang MH, Fossel AH, Larson MG. Comparison of five health status instruments for orthopedic evaluation. Med Care. 1990;28:632–642.

    Article  PubMed  CAS  Google Scholar 

  83. Angst F, Verra ML, Lehmann S, Aeschlimann A. Responsiveness of five condition-specific and generic outcome assessment instruments for chronic pain. BMC Med Res Methodol 2008;8:26 (published online 2008 April 25 doi:10.1186/1471-2288-8-26).

  84. Tavris C, Aronson E. Mistakes were made, but not by me. Orlando: Harcourt Books; 2008.

    Google Scholar 

  85. Crowne DP, Marlowe D. A new scale of social desirability independent of psychopathology. J Consult Psychol. 1960;24:349–354.

    Article  PubMed  CAS  Google Scholar 

  86. Strahan R, Kerbasi K. Short homogenous version of the Marlowe-Crowne Social Desirability Scale. J Cin Psychol. 1972;28:191–193.

    Article  Google Scholar 

  87. Furnham A, Henderson M. The good, the bad and the mad: Response bias in self-report measures. Pers Indiv Differ. 1982;3:311–320.

    Article  Google Scholar 

  88. Leary MR, Kowalski RM. Impression management: a literature review and two-component model. Psychol Bull. 1990;107:34–47.

    Article  Google Scholar 

  89. Lenski GE, Leggett JC. Caste, class, and deference in the research interview. Am J Sociol. 1960;65:463–467.

    Article  Google Scholar 

  90. Krosnick JA, Alwin DF. An evaluation of cognitive theory of response order effects in survey measurement. Public Opin Q. 1987;51:201–219.

    Article  Google Scholar 

  91. Toner B. Impact of agreement bias on the rating of questionnaire response. J Soc Psychol. 1987;127:221–222.

    Article  Google Scholar 

  92. Nelson NW, Parsons TD, Grote CL, Smith CA, Sisung II JR. The MMPI-2 Fake Bad Scale: concordance and specificity of true and estimate scores. J Clin Exp Neuropsychol. 2006;28:1–12.

    Article  PubMed  Google Scholar 

  93. Thorndike EL. A constant error in psychological rating. J Appl Psychol. 1920;4:25–29.

    Article  Google Scholar 

  94. Roeckelein J. Elsevier’s dictionary of psychological theories. Amsterdam: Elsevier BV; 2006.

    Google Scholar 

  95. Feldt LS, Brennan RL. Reliability. In: Linn RL, editor. Educational measurement. 3rd ed. New York: American Council on Education and Macmillan; 1989.

    Google Scholar 

  96. Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37:830–837.

    Article  PubMed  Google Scholar 

  97. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174.

    Article  PubMed  CAS  Google Scholar 

  98. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428.

    Article  PubMed  CAS  Google Scholar 

  99. McDowell I, Newell C. Measuring health. A guide to rating scales and questionnaires. 2nd ed. New York: Oxford University Press; 1996.

    Google Scholar 

  100. Kuder GF, Richardson MW. The theory of the estimation of test reliability. Psychometrika. 1937;2:151–60.

    Article  Google Scholar 

  101. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.

    Article  Google Scholar 

  102. George D, Mallery P. SPSS for Windows step by step. Boston: Allyn & Bacon; 2003.

    Google Scholar 

  103. Faugier J, Sargeant M. Sampling hard to reach populations. J Adv Nurs. 1997;26:790–797.

    Article  PubMed  CAS  Google Scholar 

  104. Code of Federal Regulations, Title 45 Public welfare, department of Health and Human Services, Revised 15 Jan 2009, (Effective 14 July 2009).

    Google Scholar 

  105. Nunnally JC. Psychometric theory. New York: McGraw-Hill; 1978.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Peter Flom Consulting, LLC, 515 West End Ave, New York, NY, 10024, USA

    Peter L. Flom PhD

  2. Department of Medicine, College of Medicine, SUNY Downstate Medical Center, 450 Clarkson Avenue, 1199, Brooklyn, NY, 11203, USA

    Phyllis G. Supino EdD

  3. SUNY Downstate Medical Center, 9006 Kirkdale Road, Bethesda, MD, 29817, USA

    N. Philip Ross BS, MS, PhD Statistics

Authors
  1. Peter L. Flom PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Phyllis G. Supino EdD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. N. Philip Ross BS, MS, PhD Statistics
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter L. Flom PhD .

Editor information

Editors and Affiliations

  1. , Cardiovascular Medicine, SUNY Downstate Medical Center, Clarkson Avenue, box 1199 450, Brooklyn, 11203, USA

    Phyllis G. Supino

  2. , Cardiovascualr Medicine, SUNY Downstate Medical Center, Clarkson Avenue 450, Brooklyn, 11203, USA

    Jeffrey S. Borer

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Flom, P.L., Supino, P.G., Ross, N.P. (2012). Constructing and Evaluating Self-Report Measures. In: Supino, P., Borer, J. (eds) Principles of Research Methodology. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-3360-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-3360-6_8

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-3359-0

  • Online ISBN: 978-1-4614-3360-6

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation