, Volume 127, Issue 1–2, pp 291–301 | Cite as

Computer-administered clinical rating scales

A review
  • Kenneth A. Kobak
  • John H. Greist
  • James W. Jefferson
  • David J. Katzelnick


While clinician-administered symptom rating scales are the most commonly used outcome measures in pharmaceutical research, error variance due to poor inter-rater reliability increases the risk of type II errors in multi-center clinical trials. Such error variance could obscure true differences between active drug and placebo, or between two comparator compounds. Computer-administered versions of symptom rating scales originally designed to be administered by trained clinicians offer a solution to this problem. This paper reviews the empirical data on the reliability, validity and equivalence of computer-administered rating scales. Computer-administered versions of clinician-administered scales are now available for the assessment of depression, anxiety, obsessive-compulsive disorder, and social phobia. Validation studies support the reliability, validity and equivalence of these scales. Patient reaction has been positive, with patients generally more honest with and often preferring the computer for assessing sensitive areas such as suicide, alcohol or drug abuse, sexual behavior, or HIV related symptoms. Applications using Interactive Voice Response (IVR) technology facilitate longitudinal monitoring of patients without requiring office visits to collect data, increase the accessibility of information to the clinician, and the quality of patient care through more informed decision making. When used in accordance with established ethical guidelines, computers offer a reliable, inexpensive, accessible, and time-efficient means of assessing psychiatric symptoms.

Key words

Computer Assessment Rating scales 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. American Educational Research Association (1985) Standards for educational and psychological testing. American Psychological Association. Washington, DCGoogle Scholar
  2. Ancill RJ, Rogers D, Carr AC (1985) Comparison of computerised self-rating scales for depression with conventional observer ratings. Acta Psychiatr Scand 71:315–317PubMedCrossRefGoogle Scholar
  3. Ancil R, Hilton S, Carr T, Tooley M, McKenzie A (1986) Screening for antenatal and postnatal depressive symptoms in general practice using a microcomputer-delivered questionnaire. J R Coll Gen Pract 36:276–279Google Scholar
  4. Baer L, Minichiello WE, Jenike MA (1987) Use of a portable-computer program in behavioral treatment of obsessive-compulsive disorder. Am J Psychiatry 144:1101PubMedGoogle Scholar
  5. Baer L, Brown-Beasley MW, Sorce J, Henriques AI, (1993) Computer-assisted telephone administration of a structured interview for obsessive-compulsive disorder. Am J Psychiatry 150: 1737–1738PubMedGoogle Scholar
  6. Baer L, Jacobs DG, Cukor P, O’Laughlen J, Coyle JT, Magruder KM (1995) Automated telephone screening survey for depression. JAMA 273:1943–1944PubMedCrossRefGoogle Scholar
  7. Bartko JJ (1966) The intraclass correlation coefficient as a measure of reliability. Psychol Rep 19:3–11PubMedGoogle Scholar
  8. Bech P, Bolwig P, Kramp P, Rafaelsen OJ (1979) The Bech-Rafaelsen Mania Scale and the Hamilton Depression Scale. Acta Psychiatr Scand 59:420–430PubMedCrossRefGoogle Scholar
  9. Bech P, Kastrup M, Rafaelsen OJ (1986) Minicompedium of rating scales for anxiety, depression, mania, schizophrenia with corresponding DSM-III syndromes. Acta Psychiatr Scand 73 [suppl. 326]:23–38Google Scholar
  10. Beneke M (1987)_Methodological investigations of the Hamilton Anxiety Scale. Pharmacopsychiatry 20:249–255PubMedGoogle Scholar
  11. Bresolin MJ (1984) A comparative study of computer administration of the Minnesota Multiphasic Personality Inventory in an inpatient psychiatric setting. Unpublished doctoral dissertation, Loyola University, ChicagoGoogle Scholar
  12. Brown EJ, Heimberg RG, Juster HR (1996) Social phobia subtype and avoidant personality disorder: effect on severity of social phobia, impairment, and outcome of cognitive-behavioral treatment. Behav Ther (in press)Google Scholar
  13. Butcher JN (1994) Psychological assessment by computer: potential gains and problems to avoid. Psychiatric Ann 24:20–24Google Scholar
  14. Carr AC, Ancill RJ, Ghosh A, Margo A (1981) Direct assessment of depression by microcomputer. Acta Psychiatr Scand 64: 415–422PubMedCrossRefGoogle Scholar
  15. Carr AC, Ghosh A, Ancill RJ (1983) Can a computer take a psychiatric history? Psychol Med 13:151–158PubMedGoogle Scholar
  16. Davidson JRT, Potts NLS, Richichi EA, Krishnan RR, Ford SM, Smith RD, Wilson W (1993) The treatment of social phobia with clonazepam and placebo. J Clin Psychopharmacol 13: 423–428PubMedCrossRefGoogle Scholar
  17. Erdman H, Klein MH, Greist JH (1983) The reliability of a computer interview for drug use/abuse information. Behav Res Methods Instrum 15:66–68Google Scholar
  18. Erdman H, Klein MH, Greist JH (1985) Direct patient computer interviewing. J Consult Clin Psychol 53:760–763PubMedCrossRefGoogle Scholar
  19. Erdman HP, Klein MH, Greist JH, Skare SS, Husted J, Robins L.N., Helzer JE, Goldring E, Hamburger M, Miller JP (1992) A comparison of two computer-adminstered versions of the NIMH Diagnostic Interview Schedule. J Psychiatr Res 26: 85–95PubMedCrossRefGoogle Scholar
  20. Fairbairn AS, Wood CH, Fletcher CM (1959) Variability in answers to a questionnaire on respiratory symptoms. Br J Prev Soc Med 13:175–193PubMedGoogle Scholar
  21. Ghosh A, Marks IM, Carr AC (1988) Therapist contact and outcome of self-exposure treatment of phobias. A controlled study. Br J Psychiatry 152:234–238PubMedGoogle Scholar
  22. Gjerris A, Bech P, Bojholm S, Bolwing TG, Kramp P, Clemmesen L, Andersen J, Jensen E, Rafaelson OJ (1983) The Hamilton Anxiety Scale: Evaluation of homogeneity and inter-observer reliability in patients with depressive disorders. J Affect Disord 5:163–170PubMedCrossRefGoogle Scholar
  23. Goldberg DP, Cooper B, Eastwood MR, Keddward HB, Shepherd M (1970) A standardised psychiatric interview for use in community surveys. Br J Prevent Soc Med 24:18–23Google Scholar
  24. Goodman WK, Price LH, Rasmussen SA et al. (1989) The Yale-Brown Obsessive Compulsive Scale, I: development, use, and reliability. II. Validity. Arch Gen Psychiatry 46:1006–1011, 1012–1016PubMedGoogle Scholar
  25. Greist JH, Gustafson DH, Stauss FF, Rowse GL, Laughren TP, Chiles JA (1973) A computer interview for suicide-risk prediction. Am J Psychiatry 130:1327–1332PubMedGoogle Scholar
  26. Greist JH, Klein MH (1980) Computer programs for patients, clinicians, and researcher in psychiatry. In: Sidowski JB, Johnson JH, Williams TA (eds) Technology in mental health care delivery systems. Norwood, NJ, Ablex, pp 161–182Google Scholar
  27. Greist JH, Jefferson JW, Kobak KA, Katzelnick DJ, Serlin RC (1995) Efficacy and tolerability of serotonin transport inhibitors in obsessive compulsive disorder: a meta-analysis. Arch Gen Psychiatry 52:53–60PubMedGoogle Scholar
  28. Greist JH, Jefferson JW, Wenzel K, Kobak KA, Ross TM, Katzelnick (1996) Telephone Assessment Program: Efficient Patient Monitoring and Clinician Feedback. Paper presented at the 36th Annual Meeting of the NIMH New Clinical Drug Evaluation Unit, Boca Raton, Fla.Google Scholar
  29. Hamilton M (1959) The assessment of anxiety states by rating. Br J Med Psychol 32:50–55PubMedGoogle Scholar
  30. Hamilton M (1960) A rating scale for depression. J Neurol Neurosurg Psychiatry 23:56–62PubMedGoogle Scholar
  31. Hamilton M (1967) Diagnosis and rating of anxiety. Br J Psychiatry 3:76–79Google Scholar
  32. Hamilton M (1974) General problems of psychiatric rating scales (especially for depression). In: Pichot P (ed) Psychological measurements in psychopharmacology (Modern Problems in Pharmacopsychiatry series, vol. 7). Karger, Basel, Paris, pp 125–138Google Scholar
  33. Helzer JE, Robins LN, Croughan JL, Ratcliff KS (1981) National Institute of Mental Health Diagnostic Interview Schedule: its history, characteristics and validity. Arch Gen Psychiatry 38: 381–389PubMedGoogle Scholar
  34. Hofer PJ, Green BF (1985) The challenge of competence and creativity in computerized psychological testing. J Consult Clin Psychol 53:826–838PubMedCrossRefGoogle Scholar
  35. Holt CS, Heimberg RG, Hope DA (1992a) Avoidant personality disorder and the generalized subtype of social phobia. J Abnorm Psychol 101:318–325PubMedCrossRefGoogle Scholar
  36. Holt CS, Heimberg RG, Hope DA, Liebowitz MR (1992b) Situational domains of social phobia. J Anx Disord 6:63–77CrossRefGoogle Scholar
  37. Honaker LM (1988) The equivalency of computerized and conventional MMPI administration: a critical review. Clin Psychol Rev 8:561–577CrossRefGoogle Scholar
  38. Hooijer C, Zitman EG, Griez E, van Trilburg W, Willemse A, Dinkgreve MAHM (1991) The Hamilton Depression Rating Scale (HDRS): changes in scores as a function of training and version used. J Affect Disord 22:21–29PubMedCrossRefGoogle Scholar
  39. Katzelnick DJ, Kobak KA, Greist JH, Jefferson JW (1995a) Depression among high utilization primary care patients: effect of identification and treatment on service utilization, disability, and quality of life. Paper presented at National Institute of Mental Health International Conference on Mental Health Services Research, Bethesda, MdGoogle Scholar
  40. Katzelnick DJ, Kobak KA, Greist JH, Jefferson JW (1995b) Sertraline in social phobia: a double-blind, placebo-controlled crossover study. Am J Psychiatry 152:1368–1371PubMedGoogle Scholar
  41. Kobak KA, Reynolds WM, Rosenfeld R, Greist JH (1990) Development and validation of a computer-administered version of the Hamilton Depression Rating Scale. Psychol Assess. J Consult Clin Psychol 2:56–63Google Scholar
  42. Kobak KA, Reynolds WM, Greist JH (1994) Devepment and validation of a computer-administered version of the Hamilton Anxiety Scale. Psychol Assess 5:487–492CrossRefGoogle Scholar
  43. Kobak KA, Reynolds WM, Greist JH (1994) Computerized and clinician assessment of depression and anxiety: respondent evaluation and satisfaction. J Person Assess 63: 173–180CrossRefGoogle Scholar
  44. Kobak KA, Schaettle S, Katzelnick DJ, Simon G (1995) Guidelines for the Hamilton Depression Rating Scale: modified for the Depression in Primary Care Study. Dean Foundation, Madison, Wise.Google Scholar
  45. Kobak KA, Greist JH, Jefferson JW, Katzelnick DJ, Schaettle SC (1996) Computerized Assessment in Clinical Drug Trials. National Institute of Mental Health, New Clinical Drug Evaluation Unit, 36th Annual Meeting, Boca Raton, Fla.Google Scholar
  46. Levine S, Ancil RJ, Roberts AP (1989) Assessment of suicide risk by computer-delivered self-rating questionnaire: preliminary findings. Acta Psychiatr Scand 80:216–220PubMedCrossRefGoogle Scholar
  47. Lewis G, Pelosi AJ, Glove E, Wilkinson G, Stansfeld SA, Williams P, Shepherd M (1988) The development of a computerized assessment for minor psychiatric disorder. Psychol Med 18: 737–745PubMedGoogle Scholar
  48. Liebowitz MR (1987) Social phobia. Mod Probl Pharmacopsychiatry 22:141–173PubMedGoogle Scholar
  49. Liebowitz MR, Schneier F, Campeas R, Hollander E, Hatterer J, Fyer A, Gorman JM, Papp L, Davies S, Gully R, Klein DF (1992) Phenelzine vs atenolol in social phobia: A placebo controlled comparison. Arch Gen Psychiatry 49:290–300PubMedGoogle Scholar
  50. Locke SE, Kowaloff HB, Hoff RG, Safran C, Popovsky MA, Cotton DJ, Finkelstein DM, Page PL, Slack WV (1992) Computer-based interview for screening blood donors for risk of HIV transmission. JAMA 268:1301–1305PubMedCrossRefGoogle Scholar
  51. Lucas RW, Mullins PJ, Luna CB, McInroy DC (1977) Psychiatrists and a computer as interrogators of patients with alcohol-related illnesses. a comparison. Br J Psychiatry 131:160–167PubMedGoogle Scholar
  52. Maier W, Buller R, Philipp M, Heuser I (1988) The Hamilton Anxiety Scale: reliability, validity, and sensitivity to change in anxiety and depressive disorders. J Affect Disord 14:61–68PubMedCrossRefGoogle Scholar
  53. Margo A, Johnson C, Ancill R, Carr T (1983) Assessment of depression by microcomputer. Acta Psychiatr Scand 67:434–435PubMedCrossRefGoogle Scholar
  54. Meehl PE (1954) Clinical versus statistical prediction: a theoretical analysis and a review of the evidence. University of Minnesota Press, MinneapolisGoogle Scholar
  55. Miller IW, Bishop S, Norman WH, Maddever H (1985) The modified Hamilton Depression Rating Scale for Depression: reliability and validity. Psychiatry Res 14:131–142PubMedCrossRefGoogle Scholar
  56. Montgomery SA, Asberg M (1979) A new depression scale designed to be sensitive to change. Br J Psychiatry 134:382–389PubMedGoogle Scholar
  57. Moore NC, Summer KR, Bloor RN (1984) Do patients like psychometric testing by computer? J Clin Psychol 40[3]: 875–877PubMedCrossRefGoogle Scholar
  58. Munjack DJ, Bruns J, Baltazar PL, Brown R, Leonard M, Nagy R, Kock R, Crocker B, Schafer S (1991) A pilot study of buspirone in the treatment of social phobia. J Anx Disord 5:87–98CrossRefGoogle Scholar
  59. Petrie K, Abell W (1994) Responses of parasuicides to a computerized interview. Comput Hum Behav 10:415–418CrossRefGoogle Scholar
  60. Reich J, Yates W (1988) A pilot study of the treatment of social phobia with alprazolam. Am J Psychiatry 145:590–594PubMedGoogle Scholar
  61. Riskind JH, Beck AT, Brown G, Steer RA (1987) Taking the measure of anxiety and depression: validity of the reconstructed Hamilton scales. J Nerv Ment Dis 175:474–479PubMedCrossRefGoogle Scholar
  62. Rosenfeld R, Dar R, Anderson D, Kobak KA, Greist JH (1992) A computer administered version of the Yale-Brown Obessive Compulsive Scale. Psychol Assess 4:329–332CrossRefGoogle Scholar
  63. Rozensky RH, Honor LF, Rasinski K, Tovian SM, Herz GI (1986) Paper-and-pencil versus, computer-administered MMPIs: a comparison of patients attitudes. Comput Hum Behav 2: 111–116CrossRefGoogle Scholar
  64. Russell CKC, Peace KA, Mellsop GW (1986) The reliability of a micro-computer administration of the MMPI. J Clin Psychol, 42:120–122PubMedCrossRefGoogle Scholar
  65. Selmi PM, Klein MH, Greist JH, Sorrell SP, Erdman HP (1990). Computer-administrered cognitive-behavioral therapy for depression. Am J Psychiatry 147:51–56PubMedGoogle Scholar
  66. Shepherd M, Clare AC (1981) Addendum to Shepherd M, Cooper B, Brown AC, Kalton G. Psychiatric Illness in General Practice (2nd edn). Oxford University Press, Oxford.Google Scholar
  67. Simmons EM, Miller OW (1971) Automated patient history-taking. Hospitals 45:56–59PubMedGoogle Scholar
  68. Skinner HA, Allen BA (1983) Does the computer make a difference? Computerized versus face to face versus self-report assessment of alcohol, drug and tobacco use. J Consult Clin Psychol 51: 267–275PubMedCrossRefGoogle Scholar
  69. Slack WV (1971) Computer based interviewing system dealing with nonverbal behavior as well as keyboard responses. Science 171: 84–87PubMedCrossRefGoogle Scholar
  70. Slack WV, Hicks GP, Reed CE, Van Cura LJ (1966) A computer-based medical history system. N Engl J Med 274:194–198PubMedCrossRefGoogle Scholar
  71. Slack WV, Safran C, Kowaloff HB, Pearce J, Delbanco TI (1995) A computer-administered health screening interview for hospital personnel. MD Computing 12:25–30PubMedGoogle Scholar
  72. Snaith RP, Ahmed SN, Mehta S, Hamilton M (1971) Assessment of the severity of primary depressive illness: Wakefield self-assessment depression inventory. Psychol Med 1:143–149PubMedCrossRefGoogle Scholar
  73. Spitzer RL, Endicott J, Robins E (1978) Research diagnostic criteria: rationale and reliability. Arch Gen Psychiatry 35:773–782PubMedGoogle Scholar
  74. Weizenbaum J (1966) ELIZA-A computer program for the study of natural language communication between man and machine. Commun Assoc Comp Machinery 9:36–45Google Scholar
  75. Wenzel K, Greist JH, Jeffereson JW, Kobak KA (1994) Parameter driven interview driver. Computer software program. Dean Foundation. Madison, Wise.Google Scholar
  76. White DM, Clements CB, Fowler RD (1985) A comparison of computer administration with standard administration of the MMPI. Comput Hum Behav 1:143–162CrossRefGoogle Scholar
  77. Wiggins JS (1973) Personality and prediction: principles of personality assessment. Addison-Wesley, Reading, Mass.Google Scholar
  78. Williams JBW (1988) A structured interview guide for the Hamilton Depression Rating Scale. Arch Gen Psychiatry 45:742–747PubMedGoogle Scholar

Copyright information

© Springer-Verlag 1996

Authors and Affiliations

  • Kenneth A. Kobak
    • 1
  • John H. Greist
    • 1
  • James W. Jefferson
    • 1
  • David J. Katzelnick
    • 1
  1. 1.Dean Foundation for HealthResearch and EducationMadisonUSA

Personalised recommendations