Journal of General Internal Medicine

, Volume 34, Issue 2, pp 243–249 | Cite as

Improving Patients’ Choice of Clinician by Including Roll-up Measures in Public Healthcare Quality Reports: an Online Experiment

  • Jennifer L. CerullyEmail author
  • Andrew M. Parker
  • Lise Rybowski
  • Mark Schlesinger
  • Dale Shaller
  • Rachel Grob
  • Melissa L. Finucane
  • Steven C. Martino
Original Research



Public reports on healthcare quality typically include complex data. To lower the cognitive burden of interpreting these data, some report designers create summary, or roll-up, measures combining multiple indicators of quality into one score. Little is known about how the availability of roll-ups affects clinician choice.


To determine how presenting quality scores at different levels of aggregation affects patients’ clinician choices.


We conducted a simulated clinician-choice experiment, randomizing participants to three versions of a public reporting website and comparing their clinician choices. One version aggregated all clinician-level quality measures into roll-ups, the second provided disaggregated (drill-down) scores only, and the third offered both roll-ups and drill-downs.


Five hundred fifty panelists drawn from a probability-based Internet panel.

Main Measures

We assessed the amount of effort participants exerted by tracking the length of time spent on the website and the number of concrete actions taken on the website (e.g., clicking items). We evaluated decision quality by measuring whether participants selected a clinician who performed more poorly than others and incongruence between participants’ stated preferences for dimensions of quality and their chosen clinician’s performance on those dimensions.

Key Results

Participants seeing drill-downs alone (mean = 14.9) or with roll-ups (mean = 19.2) took more actions than those who saw roll-ups alone (mean = 10.5) (ps < 0.05). However, participants seeing only drill-downs made poorer choices than those who saw roll-ups alone or with drill-downs. More participants seeing drill-downs chose a clinician who was outperformed (36.3% versus 23.4% [roll-up] and 25.6% [drill-down + roll-up], ps < 0.05) and made choices incongruent with stated preferences (51.2% versus 45.6% [roll-up] and 47.5% [drill-down + roll-up], ps < 0.05). The distinction between roll-up and drill-down was somewhat stronger for sicker participants.


Our results suggest that roll-ups in healthcare quality reports, alone or as a complement to drill-downs, can help patients make better decisions for themselves.


healthcare quality patient experience patient satisfaction composite measure summary score 



This paper was supported by two cooperative agreements (2U18HS016980 and 1U18HS016978) from AHRQ to RAND Corporation and Yale University, respectively.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

Prior Presentations

This research has not been presented previously.

Supplementary material

11606_2018_4725_MOESM1_ESM.docx (567 kb)
ESM 1 (DOCX 567 kb)


  1. 1.
    Findlay SD. Consumers’ interest in provider ratings grows, and improved report cards and other steps could accelerate their use. Health Aff. 2016;35(4):688–96.CrossRefGoogle Scholar
  2. 2.
    Martino SC, Kanouse DE, Miranda DJ, Elliott MN. Can a more user-friendly Medicare Plan Finder improve consumers’ selection of Medicare plans?. Health Serv Res. 2017;52(5):1749–71.CrossRefGoogle Scholar
  3. 3.
    Sinaiko AD, Eastman D, Rosenthal MB. How report cards on physicians, physician groups, and hospitals can have greater impact on consumer choices. Health Aff. 2012;31(3):602–11.CrossRefGoogle Scholar
  4. 4.
    Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu Rev Public Health. 2003;24:413–33.CrossRefGoogle Scholar
  5. 5.
    Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Educ Couns. 2016;99(1):36–43.CrossRefGoogle Scholar
  6. 6.
    Schlesinger M, Kanouse DE, Martino SC, Shaller D, Rybowski L. Complexity, public reporting, and choice of doctors: a look inside the blackest box of consumer behavior. Med Care Res Rev. 2014;71(5 Suppl):38s–64s.CrossRefGoogle Scholar
  7. 7.
    Schlesinger M, Kanouse DE, Rybowski L, Martino SC, Shaller D. Consumer response to patient experience measures in complex information environments. Med Care. 2012;50(Suppl):s56–64.CrossRefGoogle Scholar
  8. 8.
    Gebele C, Tscheulin DK, Lindenmeier J, Drevs F, Seemann AK. Applying the concept of consumer confusion to healthcare: development and validation of a patient confusion model. Health Services Management Research. 2014;27(1–2):10–21.CrossRefGoogle Scholar
  9. 9.
    Shi Y, Scanlon DP, Bhandari N, Christianson JB. Is anyone paying attention to physician report cards? The impact of increased availability on consumers’ awareness and use of physician quality information. Health Serv Res. 2017;52(4):1570–89.CrossRefGoogle Scholar
  10. 10.
    Centers for Medicare and Medicaid Services. Hospital Compare star ratings fact sheet. Available at: Accessed 10 September 2018.
  11. 11.
    Cerully JL, Martino SC, Rybowski L, Finucane ML, Grob R, Parker AM, Schlesinger M, Shaller D, Martsolf GR. Using “roll-up” measures in health care quality reports: perspectives of report sponsors and national alliances. American Journal of Managed Care. 2017;23(6):e202–207.Google Scholar
  12. 12.
    Zaslavsky AM, Shaul JA, Zaborski LB, Cioffi MJ, Cleary PD. Combining health plan performance indicators into simpler composite measures. Health Care Financing Review. 2002;23(4):101–15.Google Scholar
  13. 13.
    Schwartz B. The Paradox of Choice: Why More is Less. New York, NY, US: Harper Collins Publishers; 2004.Google Scholar
  14. 14.
    Martsolf GR, Scanlon DP, Christianson JB. Multistakeholder perspectives on composite measures of ambulatory care quality: a qualitative descriptive study. Med Care Res Rev. 2013;70(4):434–48.CrossRefGoogle Scholar
  15. 15.
    Association of American Medical Colleges. Guiding principles for public reporting of provider performance. Available at: Accessed 10 September 2018.
  16. 16.
    Thompson A. RE: Enhancements of the Overall Hospital Quality Star Rating, August 2017. Available at: Accessed 10 September 2018.
  17. 17.
    Romano P, Hussey P, Ritley D. Selecting quality and resource use measures: a decision guide for community quality collaboratives. Available at: Accessed 10 September 2018.
  18. 18.
    Boyce T, Dixon A, Fasolo B, Reutskaja E. Choosing a high-quality hospital: the role of nudges, scorecard design and information. Available at: Accessed 10 September 2018.
  19. 19.
    Orlowski JM. Re: AAMC comments on the measure selection for Hospital Compare star ratings TEP report. Available at: Accessed 10 September 2018.
  20. 20.
    Nickels TP. RE: CMS-1677-P, Medicare Program; Hospital Inpatient Prospective Payment Systems (PPS) for Acute Care Hospitals and the Long-Term Care Hospital PPS and Proposed Policy Changes and FY 2018 Rates; Quality Reporting Requirements for Specific Providers; Medicare and Medicaid Electronic Health Record (EHR) Incentive Program Requirements for Eligible Hospitals, Critical Access Hospitals, and Eligible Professionals; Provider-Based Status of Indian Health Service and Tribal Facilities and Organizations; Costs Reporting and Provider Requirements; Agreement Termination Notices; Proposed Rule (Vol. 82, No. 81) April 28, 2017. Available at: Accessed 10 September 2018.
  21. 21.
    Cerully JL, Parker AM, Schlesinger M, Martino SC, Shaller D, Rybowski L, Grob R, Finucane ML. The Select MD 2.0 Provider Choice Experiment: Methodological Overview. Santa Monica, CA: RAND Corporation; 2016. Available at: Accessed 10 September 2018.
  22. 22.
    GfK. Knowledge Panel® design summary. Available at: Accessed 10 September 2018.
  23. 23.
    Grob R, Schlesinger M, Parker AM, Shaller D, Barre LR, Martino SC, Finucane ML, Rybowski L, Cerully JL. Breaking narrative ground: innovative methods for rigorously eliciting and assessing patient narratives. Health Serv Res. 2016;51(Suppl 2):1248–72.CrossRefGoogle Scholar
  24. 24.
    Montgomery H. Decision rules and the search for a dominance structure: Towards a process model of decision making. Advances in Psychology. 1983;14:343–69.CrossRefGoogle Scholar
  25. 25.
    Fowler FJ Jr., Levin CA, Sepucha KR. Informing and involving patients to improve the quality of medical decisions. Health Aff. 2011;30(4):699–706.CrossRefGoogle Scholar
  26. 26.
    United States Census Bureau. Statistical abstract of the United States: 2012. Available at: Accessed 10 September 2018.
  27. 27.
    Ward BW, Schiller JS, Goodman RA. Multiple chronic conditions among US adults: a 2012 update. Preventing Chronic Disease. 2014;11.Google Scholar
  28. 28.
    Blackwell DL, Lucas JW, Clarke TC. Summary health statistics for U.S. adults: national health interview survey, 2012. Vital and Health Statistics, Series 10, Number 260. 2014.Google Scholar
  29. 29.
    Associated Press – NORC Center for Public Affairs. Finding quality doctors: how Americans evaluate provider quality in the United States. Available at: Accessed 10 September 2018.
  30. 30.
    Darby C, Crofton C, Clancy CM. Consumer Assessment of Health Providers and Systems (CAHPS): evolving to meet stakeholder needs. Am J Med Qual. 2006;21(2):144–7.CrossRefGoogle Scholar
  31. 31.
    Agency for Healthcare Research and Quality. Research on improving patient experience. Available at: Accessed 10 September 2018.
  32. 32.
    Afendulis CC, Sinaiko AD, Frank RG. Dominated choices and Medicare Advantage enrollment. J Health Econ. 2015;119:72–83.Google Scholar
  33. 33.
    Sinaiko AD, Hirth RA. Consumers, health insurance and dominated choices. J Health Econ. 2011;30:450–457.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2018

Authors and Affiliations

  • Jennifer L. Cerully
    • 1
    Email author
  • Andrew M. Parker
    • 1
  • Lise Rybowski
    • 2
  • Mark Schlesinger
    • 3
  • Dale Shaller
    • 4
  • Rachel Grob
    • 5
  • Melissa L. Finucane
    • 1
  • Steven C. Martino
    • 1
  1. 1.RAND CorporationPittsburghUSA
  2. 2.Severyn GroupAshburnUSA
  3. 3.Yale UniversityNew HavenUSA
  4. 4.Shaller Consulting GroupChesterfieldUSA
  5. 5.University of Wisconsin–MadisonMadisonUSA

Personalised recommendations