Improving Patients’ Choice of Clinician by Including Roll-up Measures in Public Healthcare Quality Reports: an Online Experiment
Public reports on healthcare quality typically include complex data. To lower the cognitive burden of interpreting these data, some report designers create summary, or roll-up, measures combining multiple indicators of quality into one score. Little is known about how the availability of roll-ups affects clinician choice.
To determine how presenting quality scores at different levels of aggregation affects patients’ clinician choices.
We conducted a simulated clinician-choice experiment, randomizing participants to three versions of a public reporting website and comparing their clinician choices. One version aggregated all clinician-level quality measures into roll-ups, the second provided disaggregated (drill-down) scores only, and the third offered both roll-ups and drill-downs.
Five hundred fifty panelists drawn from a probability-based Internet panel.
We assessed the amount of effort participants exerted by tracking the length of time spent on the website and the number of concrete actions taken on the website (e.g., clicking items). We evaluated decision quality by measuring whether participants selected a clinician who performed more poorly than others and incongruence between participants’ stated preferences for dimensions of quality and their chosen clinician’s performance on those dimensions.
Participants seeing drill-downs alone (mean = 14.9) or with roll-ups (mean = 19.2) took more actions than those who saw roll-ups alone (mean = 10.5) (ps < 0.05). However, participants seeing only drill-downs made poorer choices than those who saw roll-ups alone or with drill-downs. More participants seeing drill-downs chose a clinician who was outperformed (36.3% versus 23.4% [roll-up] and 25.6% [drill-down + roll-up], ps < 0.05) and made choices incongruent with stated preferences (51.2% versus 45.6% [roll-up] and 47.5% [drill-down + roll-up], ps < 0.05). The distinction between roll-up and drill-down was somewhat stronger for sicker participants.
Our results suggest that roll-ups in healthcare quality reports, alone or as a complement to drill-downs, can help patients make better decisions for themselves.
KEY WORDShealthcare quality patient experience patient satisfaction composite measure summary score
This paper was supported by two cooperative agreements (2U18HS016980 and 1U18HS016978) from AHRQ to RAND Corporation and Yale University, respectively.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
This research has not been presented previously.
- 10.Centers for Medicare and Medicaid Services. Hospital Compare star ratings fact sheet. Available at: https://www.cms.gov/newsroom/fact-sheets/hospital-compare-star-ratings-fact-sheet. Accessed 10 September 2018.
- 11.Cerully JL, Martino SC, Rybowski L, Finucane ML, Grob R, Parker AM, Schlesinger M, Shaller D, Martsolf GR. Using “roll-up” measures in health care quality reports: perspectives of report sponsors and national alliances. American Journal of Managed Care. 2017;23(6):e202–207.Google Scholar
- 12.Zaslavsky AM, Shaul JA, Zaborski LB, Cioffi MJ, Cleary PD. Combining health plan performance indicators into simpler composite measures. Health Care Financing Review. 2002;23(4):101–15.Google Scholar
- 13.Schwartz B. The Paradox of Choice: Why More is Less. New York, NY, US: Harper Collins Publishers; 2004.Google Scholar
- 15.Association of American Medical Colleges. Guiding principles for public reporting of provider performance. Available at: https://www.aamc.org/download/370236/data/guidingprinciplesforpublicreporting.pdf. Accessed 10 September 2018.
- 16.Thompson A. RE: Enhancements of the Overall Hospital Quality Star Rating, August 2017. Available at: https://www.aha.org/system/files/advocacy-issues/letter/2017/170925-let-thompson-cms-star-ratings.pdf. Accessed 10 September 2018.
- 17.Romano P, Hussey P, Ritley D. Selecting quality and resource use measures: a decision guide for community quality collaboratives. Available at: https://www.ahrq.gov/professionals/quality-patient-safety/quality-resources/tools/perfmeasguide/index.html. Accessed 10 September 2018.
- 18.Boyce T, Dixon A, Fasolo B, Reutskaja E. Choosing a high-quality hospital: the role of nudges, scorecard design and information. Available at: https://www.kingsfund.org.uk/sites/default/files/field/field_publication_file/Choosing-high-quality-hospital-role-report-Tammy-Boyce-Anna-Dixon-November2010.pdf. Accessed 10 September 2018.
- 19.Orlowski JM. Re: AAMC comments on the measure selection for Hospital Compare star ratings TEP report. Available at: https://www.aamc.org/download/425936/data/aamccommentletteroncmsstarratingstep.pdf. Accessed 10 September 2018.
- 20.Nickels TP. RE: CMS-1677-P, Medicare Program; Hospital Inpatient Prospective Payment Systems (PPS) for Acute Care Hospitals and the Long-Term Care Hospital PPS and Proposed Policy Changes and FY 2018 Rates; Quality Reporting Requirements for Specific Providers; Medicare and Medicaid Electronic Health Record (EHR) Incentive Program Requirements for Eligible Hospitals, Critical Access Hospitals, and Eligible Professionals; Provider-Based Status of Indian Health Service and Tribal Facilities and Organizations; Costs Reporting and Provider Requirements; Agreement Termination Notices; Proposed Rule (Vol. 82, No. 81) April 28, 2017. Available at: https://www.aha.org/system/files/advocacy-issues/letter/2017/170613-let-nickels-verma-cms-flexibility-efficiencies.pdf. Accessed 10 September 2018.
- 21.Cerully JL, Parker AM, Schlesinger M, Martino SC, Shaller D, Rybowski L, Grob R, Finucane ML. The Select MD 2.0 Provider Choice Experiment: Methodological Overview. Santa Monica, CA: RAND Corporation; 2016. Available at: https://www.rand.org/pubs/working_papers/WR1152.html. Accessed 10 September 2018.
- 22.GfK. Knowledge Panel® design summary. Available at: http://www.knowledgenetworks.com/knpanel/docs/KnowledgePanel(R)-Design-Summary-Description.pdf. Accessed 10 September 2018.
- 26.United States Census Bureau. Statistical abstract of the United States: 2012. Available at: https://www.census.gov/library/publications/2011/compendia/statab/131ed.html. Accessed 10 September 2018.
- 27.Ward BW, Schiller JS, Goodman RA. Multiple chronic conditions among US adults: a 2012 update. Preventing Chronic Disease. 2014;11.Google Scholar
- 28.Blackwell DL, Lucas JW, Clarke TC. Summary health statistics for U.S. adults: national health interview survey, 2012. Vital and Health Statistics, Series 10, Number 260. 2014.Google Scholar
- 29.Associated Press – NORC Center for Public Affairs. Finding quality doctors: how Americans evaluate provider quality in the United States. Available at: http://www.apnorc.org/projects/Pages/finding-quality-doctors-how-americans-evaluate-provider-quality-in-the-united-states.aspx. Accessed 10 September 2018.
- 31.Agency for Healthcare Research and Quality. Research on improving patient experience. Available at: https://www.ahrq.gov/cahps/quality-improvement/reports-and-case-studies/index.html. Accessed 10 September 2018.
- 32.Afendulis CC, Sinaiko AD, Frank RG. Dominated choices and Medicare Advantage enrollment. J Health Econ. 2015;119:72–83.Google Scholar