Measuring Quality of Care in Community Mental Health: Validation of Concordant Clinician and Client Quality-of-Care Scales

  • Lauren LutherEmail author
  • Sadaaki Fukui
  • Jennifer M. Garabrant
  • Angela L. Rollins
  • Gary Morse
  • Nancy Henry
  • Dawn Shimp
  • Timothy Gearhart
  • Michelle P. Salyers


Measuring quality of care can transform care, but few tools exist to measure quality from the client’s perspective. The aim of this study was to create concordant clinician and client self-report quality-of-care scales in a sample of community mental health clinicians (n = 189) and clients (n = 469). The client scale had three distinct factors (Person-Centered Care, Negative Staff Interactions, and Inattentive Care), while the clinician scale had two: Person-Centered Care and Discordant Care. Both versions demonstrated adequate internal consistency and validity with measures related to satisfaction and the therapeutic relationship. These measures are promising, brief quality assessment tools.


Quality of care Community mental health Scale development Person-centered care 



The statements in this publication are solely the responsibility of the authors and do not necessarily represent the views of the Indiana Clinical and Translational Sciences Institute, the National Institutes of Health, the Patient-Centered Outcomes Research Institute (PCORI), PCORI’s Board of Governors or Methodology Committee, or the Department of Veterans Affairs or the United States government.


Research reported in this publication was partially funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (No. IH-1304-6597) and from an Award (No. UL1TR001108) from the Indiana Clinical and Translational Sciences Institute, which is funded in part by a National Institutes of Health, National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award.

Compliance with Ethical Standards

Conflict of Interest Statement

The authors declare that they have no conflict of interest.


  1. 1.
    Demyttenaere K, Bruffaerts R, Posada-Villa J, et al. Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization world mental health surveys. JAMA. 2004;291(21):2581–2590.Google Scholar
  2. 2.
    Whiteford HA, Degenhardt L, Rehm J, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. The Lancet. 2013;382(9904):1575–1586.Google Scholar
  3. 3.
    Kessler RC, Heeringa S, Lakoma MD, et al. Individual and societal effects of mental disorders on earnings in the United States: results from the national comorbidity survey replication. The American Journal of Psychiatry. 2008;165(6):703–711.Google Scholar
  4. 4.
    Dixon LB, Dickerson F, Bellack AS, et al. The 2009 Schizophrenia PORT psychosocial treatment recommendations and summary statements. Schizophrenia Bulletin. 2010;36(1):48–70.Google Scholar
  5. 5.
    Institute of Medicine: Improving the quality of health care for mental and substance-use conditions: quality chasm series. Washington DC: National Academy Press, 2006.Google Scholar
  6. 6.
    Lora A. Call for information, call for quality in mental health care. Epidemiology and Psychiatric Sciences. 2013;22(1):9–13.Google Scholar
  7. 7.
    Rebello TJ, Marques A, Gureje O, et al. Innovative strategies for closing the mental health treatment gap globally. Current Opinion in Psychiatry. 2014;27(4):308–314.Google Scholar
  8. 8.
    Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001.Google Scholar
  9. 9.
    Chassin MR, Loeb JM, Schmaltz SP, et al. Accountability measures: using measurement to promote quality improvement. New England Journal of Medicine. 2010;363(7):683–688.Google Scholar
  10. 10.
    Pincus HA, Spaeth-Rublee B, Watkins, KE. The case for measuring quality in mental health and substance abuse care. Health Affairs. 2011;30(4):730–736.Google Scholar
  11. 11.
    Fisher ES, Staiger DO, Bynum JP, et al. Creating accountable care organizations: the extended hospital medical staff. Health Affairs. 2007;26(1):w44-w57.Google Scholar
  12. 12.
    Kilbourne AM, Keyser D, Pincus HA. Challenges and opportunities in measuring the quality of mental health care. The Canadian Journal of Psychiatry. 2010;55(9):549–557.Google Scholar
  13. 13.
    Campbell SM, Roland MO, Buetow SA. Defining quality of care. Social Science & Medicine. 2000;51(11):1611–1625.Google Scholar
  14. 14.
    Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly. 1966;44(3 suppl):166–206.Google Scholar
  15. 15.
    Donabedian A, Wheeler JR, Wyszewianski L. Quality, cost, and health: an integrative model. Medical Care. 1982;975–992.Google Scholar
  16. 16.
    Hermann RC, Leff HS, Palmer RH, et al. Quality Measures for mental health care: results from a national inventory. Medical Care Research and Review. 2000;57(3 suppl):136–154.Google Scholar
  17. 17.
    Moran V, O'Connor S, Borowitz, M. International approaches to measuring the quality of mental health care. Epidemiology and Psychiatric Sciences. 2013;22(1):3–7.Google Scholar
  18. 18.
    Parameswaran SG, Spaeth-Rublee B, Pincus HA. Measuring the quality of mental health care: consensus perspectives from selected industrialized countries. Administration and Policy in Mental Health and Mental Health Services Research. 2015;42(3):288–295.Google Scholar
  19. 19.
    Patel MM, Brown JD, Croake S, et al. The current state of behavioral health quality measures: where are the gaps? Psychiatric Services. 2015;66(8):865–871.Google Scholar
  20. 20.
    Lyu H, Wick EC, Housman M, et al. Patient satisfaction as a possible indicator of quality surgical care. Journal of American Medical Association Surgery. 2013;148(4):362–367.Google Scholar
  21. 21.
    Chang JT, Hays RD, Shekelle PG, et al. Patients’ global ratings of their health care are not associated with the technical quality of their care. Annals of Internal Medicine. 2006;144(9):665–672.Google Scholar
  22. 22.
    Borrelli, B. The assessment, monitoring, and enhancement of treatment fidelity in public heath trials. Journal of Public Health Dentistry. 2011;71(s1):S52–S63.Google Scholar
  23. 23.
    McHugh MD, Stimpfel AW. Nurse reported quality of care: a measure of hospital quality. Research in Nursing & Health. 2012;35(6):566–575.Google Scholar
  24. 24.
    You L-M, Aiken LH, Sloane DM, et al. Hospital nursing, care quality, and patient satisfaction: cross-sectional surveys of nurses and patients in hospitals in China and Europe. International Journal of Nursing Studies. 2013;50(2):154–161.Google Scholar
  25. 25.
    Lucero RJ, Lake ET, Aiken LH. Nursing care quality and adverse events in U.S. hospitals. Journal of Clinical Nursing. 2010;19(15–16):2185–2195.Google Scholar
  26. 26.
    Ball JE, Murrells T, Rafferty AM, et al. ‘Care left undone’ during nursing shifts: associations with workload and perceived quality of care. British Medical Journal Quality & Safety. 2013;23(2):116–125.Google Scholar
  27. 27.
    Shanafelt TD, Bradley KA, Wipf JE, et al. Burnout and self-reported patient care in an internal medicine residency program. Annals of Internal Medicine. 2002;136(5):358–367.Google Scholar
  28. 28.
    Salyers MP, Fukui S, Rollins AL, et al. Burnout and self-reported quality of care in community mental health. Administration and Policy in Mental Health and Mental Health Services Research. 2015;42(1):61–69.Google Scholar
  29. 29.
    Lee Y-Y, Lin JL. Do patient autonomy preferences matter? Linking patient-centered care to patient–physician relationships and health outcomes. Social Science & Medicine. 2010;71(10):1811–1818.Google Scholar
  30. 30.
    Howell SJ. Empathy and its role in quality of care. Osteopathic Family Physician, 2016; 8(4):20–23.Google Scholar
  31. 31.
    Rollins AL, Kukla M, Morse G, et al. Comparative effectiveness of a burnout reduction intervention for behavioral health providers. Psychiatric Services. 2016;67(8):920–923.Google Scholar
  32. 32.
    Salyers MP, Hudson C, Morse G, et al. BREATHE: a pilot study of a one-day retreat to reduce burnout among mental health professionals. Psychiatric Services. 2011;62(2):214–217.Google Scholar
  33. 33.
    Miller WR, Rollnick S. Motivational interviewing: preparing people for change, second edition. New York, NY: Guilford Press, 2002.Google Scholar
  34. 34.
    Luther L, Miller AP, Hedrick HM, et al. Client and provider perspectives on the impact of burnout on quality of care and client outcomes in community mental health. Journal of Behavioral and Social Sciences. 2016;3(2):73–85.Google Scholar
  35. 35.
    Horvath AO, Greenberg LS. Development and validation of the working alliance inventory. Journal of Counseling Psychology. 1989;36(2):223–233.Google Scholar
  36. 36.
    Tracey TJ, Kokotovic AM. Factor structure of the working alliance inventory. Psychological Assessment: a Journal Of Consulting and Clinical Psychology. 1989;1(3):207–210.Google Scholar
  37. 37.
    Hanson WE, Curry KT, Bandalos DL. Reliability generalization of working alliance inventory scale scores. Educational and Psychological Measurement. 2002;62(4):659–673.Google Scholar
  38. 38.
    Larsen DL, Attkisson CC, Hargreaves WA, et al. Assessment of client/patient satisfaction: development of a general scale. Evaluation and Program Planning. 1979;2(3):197–207.Google Scholar
  39. 39.
    Williams GC, Grow VM, Freedman ZR, et al. Motivational predictors of weight loss and weight-loss maintenance. Psychological Assessment: a Journal Of Consulting and Clinical Psychology. 1996;70(1):115–126.Google Scholar
  40. 40.
    Williams GC, McGregor HA, King D. Variation in perceived competence, glycemic control, and patient satisfaction: relationship to autonomy support from physicians. Patient Education Counseling. 2005;57(1):39–45.Google Scholar
  41. 41.
    Nagy MS. Using a Single-Item Approach to Measure Facet Job Satisfaction. Journal of Occupational and Organizational Psychology. 2002;75(1):77–87.Google Scholar
  42. 42.
    Wanous JP, Reichers AE, Hudy MJ. Overall job satisfaction: how good are single-item measures? Journal of Applied Psychology. 1997;82(2):247–252.Google Scholar
  43. 43.
    Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling: a Multidisciplinary Journal. 1999;6(1):1–55.Google Scholar
  44. 44.
    Epstein RM, Fiscella K, Lesser CS, et al. Why the nation needs a policy push on patient-centered health care. Health Affairs. 2010;29(8):1489–1495.Google Scholar
  45. 45.
    Clark PA. Medical practices’ sensitivity to patients’ needs: opportunities and practices for improvement. Journal of Ambulatory Care Management. 2003;26(2):110–123.Google Scholar
  46. 46.
    Wanzer MB, Booth-Butterfield M, Gruber K. Perceptions of health care providers’ communication: relationships between patient-centered communication and satisfaction. Health Communication. 2004;16(3):363–384.Google Scholar
  47. 47.
    Pincus HA, Scholle SH, Spaeth-Rublee B, et al. Quality measures for mental health and substance use: gaps, opportunities, and challenges. Health Affairs. 2016;35(6):1000–1008.Google Scholar
  48. 48.
    Boulding W, Glickman SW, Manary MP, et al. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. The American Journal of Managed Care. 2011;17(1):41–48.Google Scholar
  49. 49.
    Glickman SW, Boulding W, Manary M, et al. Patient satisfaction and its relationship with clinical quality and inpatient mortality in acute myocardial infarction. Circulation: Cardiovascular Quality and Outcomes. 2010;3(2):188–195.Google Scholar
  50. 50.
    Isaac T, Zaslavsky AM, Cleary PD, et al. The relationship between patients' perception of care and measures of hospital quality and safety. Health Services Research. 2010;45(4):1024–1040.Google Scholar
  51. 51.
    Fenton JJ, Jerant AF, Bertakis KD, et al. The cost of satisfaction: a national study of patient satisfaction, health care utilization, expenditures, and mortality. Archives of Internal Medicine. 2012;172(5):405–411.Google Scholar
  52. 52.
    Tsai TC, Orav EJ, Jha AK. Patient satisfaction and quality of surgical care in U.S. hospitals. Annals of Surgery. 2015;261(1):2–8.Google Scholar
  53. 53.
    Hojat M, Louis DZ, Maio V, et al. Editorial: empathy and health care quality. American Journal of Medical Quality. 2013;28(1):6–7.Google Scholar
  54. 54.
    Rathert C, Wyrwich MD, Boren SA. Patient-centered care and outcomes: a systematic review of the literature. Medical Care Research and Review. 2012;70(4):351–379.Google Scholar
  55. 55.
    Wolf DM, Lehman L, Quinlin R, et al. Effect of patient-centered care on patient satisfaction and quality of care. Journal of Nursing Care Quality. 2008;23(4):316–321.Google Scholar
  56. 56.
    Epstein RM, Street RL. The values and value of patient-centered care. The Annals of Family Medicine. 2011;9(2):100–103.Google Scholar
  57. 57.
    Chang WY, Ma JC, Chiu HT, et al. Job satisfaction and perceptions of quality of patient care, collaboration, and teamwork in acute care hospitals. Journal of Advanced Nursing. 2009;65(9):1946–1955.Google Scholar
  58. 58.
    Grembowski D, Paschane D, Diehr P, et al. Managed care, physician job satisfaction, and the quality of primary care. Journal of General Internal Medicine. 2005;20(3):271–277.Google Scholar
  59. 59.
    Spence Laschinger HK, Fida R. Linking nurses’ perceptions of patient care quality to job satisfaction. Journal of Nursing Administration. 2015;45(5):276–283.Google Scholar
  60. 60.
    Fortney L, Luchterhand C, Zakletskaia L, et al. Abbreviated mindfulness intervention for job satisfaction, quality of life, and compassion in primary care clinicians: a pilot study. The Annals of Family Medicine. 2013;11(5):412–420.Google Scholar
  61. 61.
    Spector, PE, Van Katwyk, PT, Brannick, MT, et al. When two factors don't reflect two constructs: how item characteristics can produce artifactual factors. Journal of Management. 1997; 23(5):659–677.Google Scholar
  62. 62.
    Drake RE, Latimer E. Lessons learned in developing community mental health care in North America. World Psychiatry. 2012;11(1):47–51.Google Scholar
  63. 63.
    Koch JR, Breland AB, Nash M, et al. Assessing the utility of consumer surveys for improving the quality of behavioral health care services. Journal of Behavioral Health Services & Research. 2011;38(2):234–248.Google Scholar

Copyright information

© National Council for Behavioral Health 2018

Authors and Affiliations

  • Lauren Luther
    • 1
    • 2
    Email author
  • Sadaaki Fukui
    • 3
  • Jennifer M. Garabrant
    • 1
    • 2
  • Angela L. Rollins
    • 2
    • 4
  • Gary Morse
    • 5
    • 6
  • Nancy Henry
    • 1
    • 2
  • Dawn Shimp
    • 1
    • 2
  • Timothy Gearhart
    • 7
  • Michelle P. Salyers
    • 1
    • 2
  1. 1.Department of PsychologyIndiana University-Purdue University Indianapolis (IUPUI)IndianapolisUSA
  2. 2.ACT Center of IndianaIndianapolisUSA
  3. 3.Center for Mental Health Research and InnovationUniversity of Kansas School of Social WelfareLawrenceUSA
  4. 4.Center for Health Information and CommunicationRichard L. Roudebush Veterans Affairs Medical CenterIndianapolisUSA
  5. 5.Places for People, Inc.St. LouisUSA
  6. 6.Department of Psychological SciencesUniversity of Missouri-St. LouisSt. LouisUSA
  7. 7.Four County Counseling CenterLogansportUSA

Personalised recommendations