Our value of care and civic leadership metrics have never been applied to hospitals before.
We also measure patient outcomes, including satisfaction, safety, and mortality rates.
These three categories are combined for a measure of overall hospital performance.
The civic leadership grade combines assessments of pay equity, community benefit spending, and inclusivity to evaluate how strongly hospitals are engaged with improving community health and well-being.
The pay equity component measures the difference in compensation of a hospital executives compared to healthcare workers without advanced degrees.
Data on CEO compensation was obtained from three sources: For nonprofit hospitals, we used the IRS 990 forms; information on for-profit, publicly-traded hospital systems was obtained from Securities and Exchange Commission Filings; and information about public hospital CEO pay was gathered from publicly available records. When CEO pay was unavailable, this information was imputed (estimated) using known values in regression models.
We obtained average worker wages from two sources: CMS' Healthcare Cost Report Information System (HCRIS) and the Bureau of Labor Statistics (BLS). HCRIS wage index information contained hourly wages for all employees. We included lower-wage staff, such as janitorial and medical records personnel and excluded professional staff such as physicians, and nurse practitioners, whose jobs require specialized degrees. For 704 hospitals that had incomplete wage index information in HCRIS, we used BLS estimates of wages for healthcare employees within those metropolitan and non-metropolitan statistical areas. We estimated hourly wages for CEOs based on a 60-hour work week and then calculated a ratio of CEO pay to average worker pay. For hospital systems, we distributed the system CEO salary among the constituent hospitals using the percentage of total revenue each hospital generated.
The community benefit component measures the extent to which hospitals are investing in community health. The score is calculated based on hospital spending on charity care and other community health spending as a share of total expenses, and the proportion of hospital patient revenue from Medicaid.
The metric "Charity care and other community benefit spending" was calculated using data from two different sources. For private nonprofit hospitals, we used the Community Benefit Insights dataset generated from Internal Revenue Service 990 forms from 2016, the most recent year all data were available. For public and for-profit hospitals, data on charity care spending was obtained from the CMS Healthcare Cost Report Information System (HCRIS) for the year 2016. For hospitals with data available in both IRS and HCRIS data sets, calculations from each data set were given equal weight in the overall "Charity care and other community benefit spending" score.
The metric "Medicaid revenue as a share of patient revenue" was calculated for all hospitals using HCRIS data, based on the percentage of gross patient revenue from Medicaid. The overall Community Benefit component score was calculated by adding the two metric scores together, at a weight of 2:1 favoring "Charity care and other community benefit spending." For hospitals without Medicaid revenue available in HCRIS, the entire Community Benefit component score was based on "Charity care and other community benefit spending."
Within the IRS data, we included the subset of community benefit spending deemed to be the most meaningful: charity care (free or discounted care provided on the basis of the patient’s financial situation); subsidized health services, such as free clinics; community health improvement activities such as free immunizations; contributions to community organizations; and community building activities, such as building farmers markets and providing housing for homeless patients. We did not use several categories of community benefit reported on 990 forms including: shortfall from Medicaid and other government means-tested insurance programs (shortfall is the difference between the amount Medicaid or other programs pay and the costs to hospitals for caring for such patients); health professional training (which is already largely subsidized by the federal government); and research.
The inclusivity component evaluates the extent to which a hospital’s patient population reflects the demographics of the community in which it is located, based on race, income, and education levels.
The component was calculated using census data on income and education as proxies for social class, and self-reported race/ethnicity for race. For each variable, inclusivity is the ratio of patients coming to the hospital compared to that measure’s prevalence in the population of people who could have come to the hospital from within its catchment area. We defined catchment area by using the zip codes of the hospital’s patient population, sorted by the number of patients each zip code supplied. We then defined the radius of the catchment area as the distance to zip codes whose contribution to the total patient population became insignificant. The median was 26.6 miles, with urban settings having far smaller radii than rural hospitals.
To calculate the denominator, we applied the U.S. Census Bureau’s American Community Survey data for people over the age of 65 on race, income, and education levels within all zip codes that fell within the defined hospital catchment area. We calculated each rate using the total population counts. We exponentially reduced the contribution from zip codes beyond the point at which 50% of a hospital’s patients had come. We created the numerator of the ratio by using the actual beneficiary counts, weighted by contribution to the total, and without a distance attenuation. We then compared the catchment area score to the hospital score to obtain an inclusivity ratio.
The value of care category reflects how well a hospital avoids the use of low-value services, medical services that offer no clinical benefit to patients and may harm them. The value of care category is based on a single component measure, avoiding overuse.
Overuse (also known in the clinical community as low-value care) is the delivery of a health care service that is more likely to harm than benefit the patient. The component comprises rates of overuse of 13 commonly overused medical services, chosen based on their validation in previous research on measuring overuse. Some of these services have been shown in high-quality clinical trials to be ineffective and are always considered overuse. Others are only considered low value when prescribed to patients with certain conditions.
The 100 percent Medicare claims datasets (MEDPAR and outpatient) from the years 2015-2107 were used to search for instances when these 13 services were used. Hospitals without the capacity to perform a service were excluded from the rating for that service. Hospitals without capacity to perform any of the 13 services were excluded entirely from the overuse ratings.
For the low-value services deemed "always overuse" (vertebroplasty, arthroscopy, renal stenting, inferior vena cava filter, and pulmonary artery catheterization), we counted the number of instances of overuse. For the services that were inappropriate depending on the condition, we used additional diagnosis and procedure codes to identify appropriateness of use. We used two different methods to calculate a denominator for services that were not always overuse. For EEG for fainting, EEG for headache, carotid artery screening for fainting, and head imaging for fainting, we measured the proportion of patients with the diagnosis (fainting or headache) who received the low-value procedure. For hysterectomy, spinal fusion, coronary artery stenting, and carotid endarterectomy, we measured the proportion of these procedures that were done inappropriately.
We adjusted observed overuse rates to account for volume differences. We then used a statistical method called principal components analysis to reduce the data down to one variable to create an overuse score.
The patient outcomes category reflects a hospital's performance as it relates to their patients' health and experience of care. This category is calculated from three components: clinical outcomes, patient safety, and patient satisfaction, which were weighted in a ratio of 5:2:1 respectively.
The clinical outcomes component measures how well the hospital keeps patients alive and prevents return trips to the hospital, over various periods of time.
Using Medicare inpatient data from 35 million patient stays from 2015-2017, we evaluated hospital clinical outcomes based on mortality and readmissions. The clinical outcomes component was composed of risk-standardized rates of mortality and readmission weighted 4:1.
Mortality included rates of in-hospital, 30-day, 90-day, and 1-year mortality. These were weighted in a ratio of 4:4:2:1 respectively in an effort to balance the effects of hospital-based care with post-discharge care and coordination in the community. We used 30-day readmission rates and added 7-day readmission based on published data suggesting the hospital-attributable component of readmissions rates wanes by 7 days. The two rates were weighted equally.
Hospitalizations and readmissions rates were risk adjusted using the Risk Stratification Index (RSI), a Lown Institute-specific version of a machine-learning algorithm in the public domain that has been validated on multiple national, state-based and hospital-based datasets using billions of insurance claims. RSI has been shown to predict outcomes with greater discriminatory accuracy compared with other publicly available risk adjustment tools.
The patient safety component measures how well hospitals avoid preventable patient safety errors. We used 2017 data provided by CMS on its Hospital Compare website for five measures of preventable hospital infections and the CMS composite measure (PSI-90), which comprises 11 different measures of patient safety measures and adverse events, including pressure ulcers, falls, and accidental punctures.
Individual metrics were weighted equally to formulate the overall Patient Safety component score. Like CMS, we excluded critical access hospitals, only six of which met our criterion for having a value for more than three measures out of the PSI-90 and hospital-acquired infection values.
The patient satisfaction component measures aspects of the hospital experience as reported by patients. For this component we used CMS's annual Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey from 2017 to give a rating of patient experience across 11 variables, including hospital quietness, cleanliness, healthcare staff responsiveness, and communication. We took the average of the 11 linear mean scores published on Hospital Compare.
Hospital Compare did not report scores for 469 hospitals that had less than 100 responses. We chose to include 314 hospitals with between 50 and 100 responses after data analysis indicated that imputation of these scores would be reasonable, to account for CMS’s mean calculations and adjustment. We calculated scores for these hospitals by extrapolating to the nearest median score of hospitals with similar survey responses.
People's lives depend on hospitals are making the right decisions. Let’s make sure they are guided by the most relevant tools possible, tools that measure what matters.
The best hospital in your city? Hospitals with best pay equity? A collection of our most interesting findings.