Does experience matter? A meta-analysis of physician rating websites of Orthopaedic Surgeons
To perform a systematic review evaluating online ratings of Orthopaedic Surgeons to determine: (1) the number of reviews per surgeon by website, (2) whether the number of reviews and rate of review acquisition correlated with years in practice, and (3) whether the use of ratings websites varied based on the surgeons’ geographic region of practice.
The USA was divided into nine geographic regions, and the most populous city in each region was selected. HealthGrades and the American Board of Orthopaedic Surgery (ABOS) database were used to identify and screen (respectively) all Orthopaedic Surgeons within each of these nine cities. These surgeons were divided into three “age” groups by years since board certification (0–10, 10–20, and 20–30 years were assigned as Groups 1, 2, and 3, respectively). An equal number of surgeons were randomly selected from each region for final analysis. The online profiles for each surgeon were reviewed on four online physician rating websites (PRW, i.e. HealthGrades, Vitals, RateMDs, Yelp) for the number of available patient reviews. Descriptive statistics, analysis of variance (ANOVA), and Pearson correlations were used.
Using HealthGrades, 2802 “Orthopaedic Surgeons” were identified in nine cities. However, 1271 (45%) of these were not found in the ABOS board certification database. After randomization, a total of 351 surgeons were included in the final analysis. For these 351 surgeons, the mean number of reviews per surgeon found on all four websites was 9.0 ± 14.8 (range 0–184). The mean number of reviews did not differ between the three age groups (p > 0.05) with 8.7 ± 14.4, (2) 10.3 ± 18.3, and (3) 8.0 ± 10.8 for Groups 1, 2, and 3, respectively. However, the rate that reviews were obtained (i.e. reviews per surgeon per year) was significantly higher (p < 0.001) in Group 1 (2.6 ± 7.7 reviews per year) compared to Group 2 (1.4 ± 2.4) and Group 3 (1.1 ± 1.4). There was no correlation between the number of reviews and years in practice (R < 0.001), and there was a poor correlation between number of reviews and regional population (R = 0.199).
The number of reviews per surgeon did not differ significantly between the three defined age groups based on years in practice. However, surgeons with less than 10 years in practice were accumulating reviews at a significantly higher rate. Interestingly nearly half of “Orthopaedic Surgeons” listed were not found to be ABOS-certified Orthopaedic Surgeons.
KeywordsOrthopaedic Surgeon Online Physician rating website Years in practice Geographic region
Compliance with ethical standards
Conflict of interest
RAJ, MBB, and SRL have no relevant conflicts of interest. KEV declares IP royalties with Solana, is a paid consultant with Solana, and has stock/stock options with Wright Medical. JDH declares association with Editorial Board for Arthroscopy: The Journal of Arthroscopic and Related Surgery, and Frontiers in Surgery, publication royalties from SLACK, Inc., research support from Depuy Synthes and Smith & Nephew, is a paid consultant from Smith & Nephew and NIA Magellan, and is on committees with AANA Research, AOSSM Self-Assessment, AAOS Osteoarthritis Pain and Function Workgroup. PCM is on the speakers' bureau for Genzyme, receives research support from Depuy, A Johnson & Johnson Company, Arthrex, and Zimmer, and is on the medical publication editorial board for the Journal of Knee Surgery and Orthobullets.com.
- 2.Mehta SJ (2015) Patient satisfaction reporting and its implications for patient care. http://journalofethics.ama-assn.org/2015/07/ecas3-1507.html. Accessed July 2016
- 3.CMS.gov. (2015) Hospital value-based purchasing. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/hospital-value-based-purchasing/index.html?redirect=/Hospital-Value-Based-Purchasing/. Accessed July 2016
- 8.Keckley P (2011) 2011 Survey of health care consumers in the United States: key findings, strategic implications. Deloitte Center for Health Solutions, Washington, DCGoogle Scholar
- 11.Ellimoottil C, Leichtle S, Wright C et al (2013) Online physician reviews: the good, the bad, and the ugly. http://bulletin.facs.org/2013/09/online-physician-reviews/. Accessed Jan 2016
- 16.Shute N (2013) Online grades for doctors get an incomplete. http://www.npr.org/sections/health-shots/2013/01/04/168626218/grades-for-doctors-get-an-incomplete. Accessed 1 Jan 2016
- 17.Falkenberg K (2013) Why rating your doctor Is bad for your health. http://www.forbes.com/sites/kaifalkenberg/2013/01/02/why-rating-your-doctor-is-bad-for-your-health/. Accessed 1 Jan 2016
- 19.Peckham C (2015) Medscape orthopedist compensation report 2015, p 7. http://www.medscape.com/features/slideshow/compensation/2015/orthopedics. Accessed Mar 2016
- 20.(2015) Annual estimates of the resident population for incorporated places of 50,000 or more, ranked by July 1, 2015 population: April 1, 2010 to July 1, 2015. http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=bkmk. Accessed Mar 2016
- 21.The American Board of Orthopaedic Surgery. Verify certification. https://www.abos.org/verify-certification.aspx. Accessed Mar 2016
- 22.Understanding the history of online reviews. https://reviewconcierge.com/lessons/online_review_survival_course/5/understanding-the-history-of-online-reviews. Accessed Mar 2016
- 23.C.S. Mott Children’s Hospital. National poll on children’s health: Many parents wary of online ratings for doctors 2016; http://mottnpch.org/sites/default/files/documents/032116_doctorratings_1.pdf. Accessed 1 Apr 2016
- 24.Friedberg M, Pronovost P, Shahian D et al (2015) A methodological critique of the ProPublica Surgeon Scorecard. http://www.rand.org/pubs/perspectives/PE170.html. Accessed Jan 2016