INTRODUCTION

In July 2016, the Centers for Medicare & Medicaid Services (CMS) released “Hospital Compare” ratings for 3576 U.S. hospitals to help consumers compare hospitals’ quality of care, which was distilled into a single overall rating between 1 and 5 stars based on 57 process- and outcomes-based clinical measures.1,2 A recent publication found differences in overall ratings by characteristics including a hospital’s teaching status.3 Professional groups (e.g., the American Hospital Association) have expressed concern that the current ratings scheme unfairly penalizes teaching hospitals, especially those serving low-income and otherwise disadvantaged patient populations.4

Our objective was to build a statistical model that adjusted  for a more complete set of covariates than previous studies to analyze discrepancies in Hospital Compare ratings between teaching and non-teaching hospitals. Informed by previous findings, we attempted to more fully capture key aspects of an academic mission-driven hospital, including trainee development, research, and uncompensated care.3,5 We based our model on U.S. geographic healthcare markets to assess the potential impact of adjusting overall quality ratings for hospital characteristics, including their locations and the sociodemographic characteristics of their surrounding patient populations.

METHODS

Publicly available data sets were merged from CMS Hospital Compare, CMS Inpatient Prospective Payment System, and the U.S. Census. Children’s hospitals were excluded due to exemption from data reporting. Sociodemographic indicators were mapped to hospital location by health service area (HSA). Our study was granted exempt status by the Institutional Review Board of the Hospital of the University of Pennsylvania.

The relationships between Hospital Compare ratings and various predictors were analyzed by univariate and multivariate linear regression models using R, version 3.4.0. We performed sensitivity analysis using two propensity score matching methods (multivariate logistic regression and random forests) to account for non-linear and non-additive associations.

RESULTS

In a multivariate linear regression model adjusting for hospital characteristics, geography, and sociodemographic factors, each increase of 1 resident (trainee) per 10 beds, a continuous measure of teaching intensity, was associated with a 0.09 Hospital Compare star rating decrease (p < 0.01, Table 1). Figure 1 also shows the correlation between increasing teaching intensity, as captured by resident-to-bed ratio, and overall mean Hospital Compare rating.

Table 1 Association Between Hospital, Location, and Sociodemographic Characteristics and Hospital Compare Ratings
Full size table
Fig. 1
figure 1

Hospital compare ratings based on resident-to-bed ratio. Increased teaching intensity, represented by higher resident-to-bed ratios, was associated with a decreasing mean overall Hospital Compare rating (between 1 and 5 stars) at U.S. teaching hospitals.

Full size image

Two propensity score sensitivity analyses, which matched teaching and non-teaching hospitals on all geographic and sociodemographic variables in Table 1 (except resident-to-bed ratio) produced similar results. With multivariate logistic regression and random forest propensity matching, teaching hospitals had 0.32 (95% CI, 0.24, 0.39) and 0.23 (95% CI, 0.16, 0.31) lower Hospital Compare ratings, respectively (out of a possible 5 stars).

Other non-teaching-related measures were also independently associated with significant differences in Hospital Compare ratings when all hospital and sociodemographic factors were included in a single, multivariate linear regression (Table 1). Hospitals offering emergency services were associated with lower ratings by 0.24 stars (p < 0.01). Location by U.S. region was highly associated with differences in hospital ratings. Compared to the largest cohort of hospitals (South Atlantic, n = 568), cohorts in the East North Central or Mid-Atlantic received up to 0.40 higher or 0.20 lower ratings, respectively (p < 0.01 for both).

In addition, HSA-level sociodemographic factors were independently associated with differences in hospital ratings (Table 1). Age, race, unemployment, and education (bachelor’s degree) of populations within a hospital’s HSA all contributed to variations in quality rating (p < 0.05 for race, others p < 0.01).

DISCUSSION

Our analysis suggests that an institution’s Hospital Compare score is strongly influenced by differences in hospital characteristics and sociodemographic factors. Even after adjusting for their location and surrounding patient populations, Hospital Compare ratings are lower for academic hospitals, and this rating penalty increases with higher teaching intensity. In addition, hospitals providing emergency services and those surrounded by a higher density of patients with sociodemographic risk factors also receive lower overall ratings.

While our study does not fully explain how higher teaching intensity, as measured by resident-to-bed ratio, is associated with poorer Hospital Compare ratings, it may be that academic hospitals are called upon to provide a larger spectrum of services or are receiving incomplete adjustment for clinical factors (e.g., disease severity or comorbidities). Policymakers must ensure that quality measurement, as embodied in Hospital Compare ratings, does not penalize academic hospitals for treating vulnerable and challenging patients, or for providing postgraduate medical education.