Nursing home quality, COVID-19 deaths, and excess mortality. This is Democide

Abstract

The COVID-19 pandemic in the US has been particularly devastating for nursing home residents. A key question is how have some nursing homes been able to effectively protect their residents, while others have not? Using data on the universe of US nursing homes, we examine whether establishment quality is predictive of COVID-19 mortality. Higher-quality nursing homes, as measured by CMS overall five-star rating, have substantially lower COVID-19 mortality through September of 2020. Quality does not predict the ability to prevent any COVID-19 resident or staff cases, but higher-quality establishments prevent the spread of resident infections conditional on having one. Preventing COVID-19 cases and deaths may come at some cost, as high-quality homes have substantially higher non-COVID deaths. The positive correlation between establishment quality and non-COVID mortality is strong enough that high-quality homes also have more total deaths than their low-quality counterparts and this relationship has grown with time. As of late April 2021, five-star homes have experienced 8.4 percent more total deaths than one-star homes.

I. Introduction

The COVID-19 pandemic in the US has been particularly devastating for residents of nursing homes. As of August 15th, 2021, there were a total of 634,179 COVID-19-related deaths in the US.¹ We estimate that 21 percent of these deaths are among nursing home residents.² Through the end of 2020, the death rate for non-nursing home residents was about 87 per 100,000. The death rate for nursing home residents is more than 108 times that number at roughly 9200 per 100,000. The death rate in nursing homes is more 23 times larger than the death rate for those 65 and over outside of nursing homes, which we calculate to be about 390 per 100,000. In at least five states, Rhode Island, Indiana, New Jersey, South Dakota, and Connecticut, more than 12 percent of the nursing home population at the beginning of 2020 had died by the end of the year with COVID-19.

COVID-19 deaths among nursing home patients is heavily influenced by infection and transmission rates in the surrounding community, which varies substantially by region. For example, cumulative non-nursing home COVID-19 deaths rates at the end of 2020 varied from a low of 8 per 100,000 in Vermont to a high of 159 per 100,000 in New Jersey. Yet, community risk alone cannot explain the vast variation in nursing home death rates. Even in the five states mentioned above with the highest nursing home death rates, 17 percent of homes had not experienced a single COVID-19 mortality by the end of 2020. A key research question is then how have some nursing homes been able to effectively protect their residents, while others have not?

In this work, we focus on one particular dimension of the problem: nursing home quality. Starting in 2008, the Centers for Medicare and Medicaid (CMS) began providing a “five-star” rating of nursing home quality based on three elements: health inspections, staff-to-resident ratios, and quality metrics such as rates of falls and bedsores, with the first element having the greatest weight in an “overall” rating. We merge this quality data with data on COVID-19 cases and mortality that is reported weekly by nursing homes to CMS starting the week ending May 24th, 2020. In count-data models that control for local risk factors and nursing home characteristics, we find that the overall rating is highly predictive of mortality, with five-star homes having 15 percent fewer resident COVID-19 deaths by September 13th, 2020 than one-star homes. We find no such relationship between home quality and COVID-19 deaths after September of 2020.

We identify several mechanisms through which quality nursing homes lowered mortality. First, we show that the chance of having any COVID-19 cases among residents or staff is not explained by nursing home quality, meaning even high-quality homes were unable to prevent COVID-19 from entering their facility. That said, in models that condition on having at least one confirmed case, we show that higher-quality nursing homes more successfully prevented the spread of the disease among residents. Unfortunately, we do not have detailed data on the policies and procedures that enabled these establishments to prevent the spread of the disease. We do show that higher-quality homes experienced fewer staff shortages, which may have helped with patient monitoring and isolation, and somewhat weaker evidence of fewer personal protective equipment (PPE) outages. Furthermore, we show that high-quality homes tested residents and staff more frequently and received test results faster than low-quality homes. Consistent with high-quality homes simply following CMS guidelines more closely, we also show that once vaccines were made available, high-quality homes vaccinated residents and staff at a higher rate.

A likely scenario is that higher-quality establishments also did a better job of isolating residents from risks associated with both outside visitors and other residents. On March 13th, 2020, CMS recommended that nursing homes (i) restrict visitors and non-essential personnel from entering the home and (ii) cancel in-person dining and other group activities (CMS, 2020a).³ These precautions very well may have prevented deaths from COVID-19, but many have questioned whether those gains came at a cost. An abundance of qualitative evidence from nursing home staff, administrators, and resident family members suggests that the lack of in-person contact with loved ones and other residents not only generated feelings of loneliness, isolation, and despair, but may have also expedited death (Aronson, 2020; Paulin, 2020; Graham, 2020). A nationwide survey of nursing home residents by Altarum, a non-profit healthcare company, documents similar concerns (Montgomery et al., 2020). Existing research shows that social isolation increases both dementia severity and the likelihood of adverse outcomes among those with Alzheimer’s (Dyer et al., 2020) and mortality among the elderly (Steptoe et al., 2013). According to CDC provisional data, there were 56,464 excess deaths (i.e., realized deaths above that which is predicted using historical averages) among Alzheimer’s patients through early October of 2021;⁴ however, only 22,709 or 40 percent had a corresponding positive COVID-19 diagnosis,⁵ leaving 60 percent of these excess deaths not directly related to the disease but to other aspects of the pandemic.⁶

To investigate this claim, we return to our original model, but change the dependent variable from COVID-19 deaths to non-COVID deaths. We find that higher-quality nursing homes have much higher non-COVID mortality. In particular, as of September 13th, 2020, five-star homes had experienced 11.4 percent more non-COVID deaths than one-star homes, all else equal; by April 15, 2021, this figure had grown to nearly 15 percent. Research by Levere et al. (2020) suggests that these excess deaths likely resulted from isolation and loneliness. Using resident-level assessment data from Connecticut nursing homes, the authors document substantial weight loss and increases in severe pressure ulcers among residents who did not contract COVID-19. The resident survey mentioned above also documents severe isolation, finding that only 5 percent of respondents had visitors three or more times per week, compared to 56 percent before the pandemic, and just 13 percent reported dining in a communal setting, compared to 69 percent before the pandemic. Another possibly is that resident contact restrictions may coincide with, or even cause, a reduction in interactions with healthcare providers, both inside and outside the home, which would be consistent with widely documented reductions in healthcare receipt overall during the early stages of the pandemic (Bosworth et al., 2020; Ziedan et al., 2020; Cantor et al., 2020; Clemens et al., 2021).

An alternative explanation of our findings is that there is incomplete reporting of COVID-19 or strategic use of defining COVID-19 deaths by nursing homes to mitigate the damage to their reputation from the CMS reports. To investigate this, in Fig. 1 we report weekly deaths in nursing homes as reported to CMS from the end of May 2020 through June of 2021. Were deaths misreported, we would expect spikes in non-COVID deaths during the summer and winter waves of 2020. Such spikes are not present in the data. Moreover, when we eliminate deaths prior to June of 2020, when misreporting is most likely, our results are unaffected. Another plausible explanation of our findings is “harvesting”; i.e., low-quality homes experience fewer non-COVID deaths because the most fragile residents die from COVID-19. To test this theory, we estimate the impact of nursing home quality on total deaths. We find that between January 1st, 2020 and April 25th, 2021, five-star homes experienced 8.4 percent more total deaths than one-star homes, an average difference of 2.7 lives.

Given the number of COVID-19-related deaths and an early understanding that the elderly die at higher rates, there is surprisingly little research on deaths in nursing homes. As a result, this paper contributes to the literature on the health effects of COVID-19 along several dimensions. A number of papers have examined the general relationship between CMS five-star ratings and COVID-19 cases and/or deaths in nursing homes but much of this work was either in a single state (Bui et al., 2020; Harrington et al., 2020; He et al., 2020; Li et al., 2020a) or occurred very early in the pandemic (Abrams et al., 2020). Some studies have used various versions of the CMS data used in this study. Gorges and Konetzka (2020) show that county incidence rates are the strongest predictor of resident mortality and staff levels have modest impacts on the spread of the disease. Li et al. (2020b) document much higher COVID-19 death rates in homes with a larger share of minority patients. Chen et al. (2021) show that contractors that service multiple nursing homes in a local area helped spread the disease. The data from the CDC showing large excess death rates for the elderly and especially Alzheimer’s patients has helped put this issue into the public discussion, yet there is little research other than efforts to document the extent of the problem and almost no research on the factors that lead to excess mortality.

While much of the previous work studies determinants of outbreaks, only a few studies such as (Li, Cen, Cai and Temkin-Greener, 2020b) and Gorges and Konetzka (2020) consider mortality in a nationwide sample as we do. Moreover, we identify that the home quality/COVID-19 death relationship is most likely explained by high-quality homes reducing the spread of the virus once it enters the home and it is not due to a home’s ability to prevent the entry of the disease. Importantly, our paper is the first to establish a negative link between nursing home quality and non-COVID mortality. This relationship is so pronounced that high-quality nursing homes are found to have significantly higher total mortality than low-quality homes.

The COVID-19 pandemic presented an almost unprecedented challenge for nursing homes. In response to CMS directives and various state regulations, it is without question that the pandemic made nursing homes much lonelier places to live. In this paper, we show that the highest quality nursing homes have witnessed the greatest amount of death during the pandemic. The means by which quality is measured may be important to understanding this finding. Inputs to the CMS five-star rating measure a nursing home’s ability to abide by pre-determined guidelines (e.g., staffing ratios) and minimize objectively bad health outcomes (e.g., bed sores). In light of this, it is not clear that homes of high quality, as it is traditionally measured, should fare well in a highly uncertain environment where guidelines from the centralized authority (i.e., CMS), as well as public opinion, attach extraordinary weight to preventing a single bad outcome; namely, COVID-19 cases and deaths. This paper provides a clear example of high-quality homes underperforming in such an environment.⁷ We offer two takeaways.

First, an organization’s capacity to perform well in the face of adversity may be an important dimension of its quality. Such capacity is difficult to measure across organizations because adversity may not be observable and is rarely evenly distributed. The COVID-19 pandemic offers CMS an observable shock that affected all nursing homes. CMS should consider capitalizing on this opportunity by evaluating nursing home responses to the COVID-19 shock and integrating this evaluation into their quality metrics.

Second, through September of 2020, CMS guidelines clearly prioritized reducing nursing home resident exposure to COVID-19. Guidelines focused on proper PPE use, testing protocols, and isolating residents from the outside world and one another. COVID-19 cases and deaths were publicized and widely reported in the news media. Our results suggest that in response, the best nursing homes “hit their marks,” i.e., they followed guidelines and, for a time, avoided the bad health outcome of focus. That said, we show that these homes witnessed the most total death in the long-run. In the future, we hope that these findings serve as a reminder that even early guidelines and objective health metrics should be cognizant of resident wellbeing at large.

II. Data

II.A. Data sources and reporting accuracy

Our primary data source is a weekly data set released by CMS that has COVID-19 surveillance information by nursing home. On May 8th, 2020, CMS released a final ruling that required nursing homes to report confirmed and suspected COVID-19 cases of residents and staff to both the residents and their representatives. The ruling also required that nursing homes report weekly totals of surveillance items to the Centers for Disease Control (CDC) National Healthcare Safety Network, starting with the week ending May 24th, 2020. Reporting was mandatory with a $1000 fine issued after four-weeks of non-reporting. Thereafter, fines increased by $500 for each week of non-reporting (CMS, 2020c).

The first weekly file was released by CMS in early June and updates are released on Thursdays, 11 days after the end of the Monday to Sunday reporting period. The initial release of this data set was in a word, messy. There were obvious key-punch errors and variables were frequently reported in the wrong columns. Subsequent releases of the data corrected many of these recording errors and the most recent releases of the data are relatively free from these obvious errors, although some naturally exist. Reporting has always been high. In the first release of the data, 13,162 nursing homes were included and 97 percent passed a data quality assurance check. In the following few weeks, many homes missing the original deadline submitted both current and historic data. The number of homes reporting in a single week peaks at 15,330 for the week ending September 20th, 2020, and 98 percent passed a data quality assurance check. The number never dips below 15,167 after that. Based on the size of the CMS five-star data set, this represents 97 to 98 percent of all nursing homes.

The CMS nursing home data reports weekly and cumulative (since January 1st, 2020) values for confirmed COVID-19 cases for residents and staff, suspected cases for these two groups, and COVID-19 deaths (which includes suspected and confirmed), plus all deaths, among residents. Deaths are reported regardless of the location, so if a nursing home resident is moved to a hospital and dies there, the death is counted as a nursing home resident death. The survey asks nursing homes if they have shortages of key staff including nursing staff, clinical staff, and aids, and whether they have adequate supplies of specific PPE such as N95 masks, surgical masks, gowns, etc. From August 16th through November 22nd (2020), the survey asks a series of questions about COVID-19 testing procedures. Not until late May of 2021 did the survey ask about resident and staff vaccinations.

The CMS nursing home data has an important limitation. At the time of the first report (May 24th, 2020), CMS allowed nursing homes the choice to report cases and deaths from the prior week or cumulative cases and deaths since January 1st. Thereafter, homes report weekly counts and a cumulative count is calculated by CMS; thus, if a home fails to report the cumulative count since January 1st on May 24th, the cumulative count that CMS calculates in future weeks is too small. In Appendix Section B, we compare CMS nursing home death counts to those from the COVID-19 Tracking Project, which measures COVID-19 deaths in nursing homes at the state level in 37 states. We calculate that the initial undercount is about 15%. In contrast, the change in COVID-19 deaths between May 24th, 2020 and other dates as reported by these two sources differs by less than 4 percent. In our robustness analysis, we re-estimate our main model using cases and deaths since May 24th, which appears to be more accurately reported across the two samples.

Using these two data sources, we estimate total nursing home COVID-19 deaths in the nation as of August 15, 2021 to be 137,318, which is 21 percent of aggregate mortality (see Table A1 and associated discussion in the Appendix Section B).

The staggering consequences of the pandemic for nursing home residents can be seen when we calculate COVID-19 death rates for residents and compare to the general population through the end of 2020.⁸ The death rate (per 100,000 people) for non-nursing home residents is roughly 87. Dividing imputed COVID-19 deaths by the number of nursing home residents alive at the beginning of 2020,⁹ the death rate for nursing home residents is about 9200, or about 108 times the rate for the general population. We calculate that the COVID-19 death rate for people 65 and over living outside of nursing homes is 390,¹⁰ meaning the nursing home death rate is about 24 times this number.

II.B. Heterogeneity across nursing homes in COVID-19 mortality

There is tremendous variation across states in the severity with which the pandemic struck nursing homes. Using the CMS data as of the last week of 2020, deaths per 1000 nursing home residents varied from a low of 12.8 in Alaska to a high of 134 in Rhode Island (see Appendix Fig. A3). In the 14 states with the highest death rates, more than 10 percent of the nursing home population died from COVID-19 in 2020.

The risk nursing home residents face from the disease is strongly correlated with the underlying risk in their state. The correlation coefficient between the non-nursing home COVID-19 death rate at the state level and the same value for nursing home residents (from CMS data) is 0.54.

Despite the strong correlation between underlying risk and nursing home deaths, many nursing homes successfully avoid high death rates, even in the hardest hit areas. In counties from the top 10th percentile of total COVID-19 death rates by May 24th, 2020, just over half of nursing homes with more than 100 beds had two or fewer COVID-19 deaths per 100 beds. By April 24th, 2021 this figure was still in excess of 20 percent. Moreover, a disproportionately small share of nursing homes account for the majority of COVID-19 deaths. As of September 2020, the top 5 (1) percent of nursing homes, which is just 754 (151) homes or 8.3 (2.4) percent of the nation’s beds, accounted for 39 (14) percent of all COVID-19 deaths in nursing homes. By the end of April 2021, the top 5 (1) percent of nursing homes still accounted for 23 (8) percent of all COVID-19 deaths.

II.C. Analysis sample

The goal of this project is to explore whether observed nursing home characteristics can explain the variation in COVID-19 death rates. In particular, we test whether high-quality nursing homes, as measured by the CMS five-star ratings, did a better job of preventing deaths from COVID-19. Our initial sample contains 15,421 nursing homes reporting data to CMS at any point between May 25th, 2020 and July 18th, 2021. We lose 311 homes to inconsistent reporting and 205 homes because no five-star rating data was available; thus, our main analysis sample consists of 14,905 nursing homes. Sample construction is discussed in Appendix A and summary statistics for these data are shown in Appendix Table A3.

Several variables used in our analysis require explanation. Data on nursing home characteristics comes from “Long-term Care: Facts on Care in the US”, which is provided by a research center at Brown University.¹¹ Sample statistics are reported for observations with non-missing data. The acuity index, which ranges from zero to 23 in the data, measures of the amount of care needed by the average nursing home residents (higher values suggest more care). For the three variables – the share of residents using Medicaid, the acuity index, and for-profit status – the data contains values for all or none of them.

Nursing home star ratings come from data.medicare.gov. There are three separate ratings – inspection, quality measures (QM), and staffing – which are aggregated by CMS into an overall rating.¹² All three ratings, as well as the overall rating, measure quality in integer “star” values, where five-star is the best possible rating and one-star is the worst. We discuss the construction of these ratings and briefly outline the literature that evaluates the ratings in Appendix Section C. In our empirical analysis, we measure quality using the overall five-star rating, reported by CMS in June of 2020. All data informing these ratings was collected prior to March, when it was first recognized that COVID-19 was present and spreading in the United States.

In Table 1, we present cumulative case and death rates over time, starting January 1st, 2020. Throughout our analysis, we focus on cumulative deaths at and between four dates: (i) May 24th, 2020, which precedes the summer 2020 COVID-19 wave and is the first date we observe in the CMS data; (ii) September 13th, 2020, which follows the summer wave, but precedes a September 17th CMS memo that altered visitation protocols in nursing homes (discussed below); (iii) December 6th, 2020, which is the last date available prior to the start of vaccine distribution; and (iv) April 25th, 2021, a point in which nearly all US nursing home residents had a full four months to become vaccinated and COVID-19 death rates in nursing homes had flatlined (see Fig. 1). Resident and staff case variables measure confirmed cases reported to CMS. Nursing homes report both total deaths from any cause and COVID-19 deaths each week, which allows us to calculate deaths not from COVID-19.¹³ Note that case (and death) rates are calculated as (cases/total beds)*100. County-level COVID-19 case and death totals come from the same source used by the CDC (USAFacts, 2020).

Table 1. COVID-19 Cases and Death Rates by Cause over Time in Nursing Homes Means (standard deviations).

Variable	As of 5/24/20	As of 9/13/20	As of 12/06/20	As of 4/25/21
Cumulative case and death rates (per 100 beds)
Resident COVID-19 case rate	4.93 (12.79)	12.94 (19.31)	26.17 (24.80)	40.39 (27.39)
Staff COVID-19 case rate	2.86 (7.21)	11.11 (16.59)	24.27 (23.94)	38.11 (22.14)
Resident COVID-19 death rate	1.30 (3.69)	3.00 (5.77)	5.00 (6.90)	8.10 (8.57)
Resident non-COVID-19 death rate	3.12 (7.69)	9.69 (23.88)	14.60 (41.85)	22.25 (51.87)
County-level cumulative case and death rates (per 1000 residents)
COVID-19 cases rate	2.86 (4.76)	15.35 (9.97)	33.75 (15.85)	94.50 (24.77)
COVID-19 deaths rate	0.25 (0.42)	0.54 (0.54)	0.89 (0/61)	1.86 (0.86)

This table reports mean, cumulative case and death rates at four points in time. The nursing home case and death rates reported in the top panel come from the CMS COVID-19 nursing home surveillance survey described in Section II.A. Note that the number of resident non-COVID-19 deaths is calculated by subtracting the reported number of COVID-19 deaths from the reported number of total deaths in the home. The county-level case and death rates are calculated using counts from USAFacts (2020).

In Table 2, we present nursing home characteristics and relevant COVID-19 summary statistics for nursing homes with different overall star ratings as of September 13th, 2020.¹⁴ Many nursing home characteristics change monotonically with the overall star rating including: total beds (decreasing); the share of female residents (increasing); the share of residents under 65, black, Hispanic, and on Medicaid (decreasing), and the acuity index (decreasing). The persistent relationship between these observed characteristics and the overall star rating might mean there is little informational content in the ratings. However, a simple regression of the overall star rating on observed characteristics only produces and R² of 0.21. Regarding COVID-19, higher-quality homes have lower resident case rates, but slightly higher staff rates. Moreover, these homes have notably smaller COVID-19 death rates, but higher non-COVID death rates. On average, higher-quality homes have experienced fewer staff shortages and PPE outages than lower-quality homes since May 24th, 2020. COVID-19 testing is not strongly correlated with overall quality.

Table 2. Descriptive Statistics in Nursing Homes, Overall 5-Star Rating.

	1-star	2-star	3-star	4-star	5-star
NH characteristics
total beds	122.8	116.6	110.4	102.7	88.5
share of female residents	0.621	0.645	0.659	0.681	0.692
share of residents under 65 years old	0.277	0.252	0.233	0.195	0.159
share of black residents	0.271	0.206	0.177	0.127	0.084
share of Hispanic residents	0.072	0.063	0.053	0.038	0.032
share of residents on Medicaid	0.693	0.650	0.630	0.584	0.487
for profit	0.867	0.775	0.735	0.665	0.543
acuity index	12.25	12.22	12.20	12.16	12.14
Cumulative Cases and Deaths (9/13)
any resident COVID-19 cases?	0.683	0.680	0.650	0.613	0.560
resident COVID-19 cases/100 beds	15.06	15.15	13.35	12.39	10.37
any staff COVID-19 cases?	0.877	0.855	0.854	0.844	0.832
staff COVID-19 case/100 beds	10.40	11.30	11.16	11.02	11.51
any resident COVID-19 deaths?	0.509	0.492	0.455	0.415	0.359
resident COVID-19 death/100 beds	3.324	3.539	2.999	2.922	2.531
any resident deaths not from COVID-19?	0.846	0.857	0.874	0.863	0.831
resident death not from COVID-19/100 beds	8.15	9.35	9.74	9.85	10.87
NH Staff and Supplies (since week ending 5/24)
any nursing staff shortage?	0.477	0.410	0.383	0.359	0.311
any aide staff shortage?	0.522	0.452	0.428	0.399	0.346
any clinical staff shortage?	0.211	0.182	0.168	0.166	0.151
n95 mask outage?	0.208	0.185	0.177	0.157	0.155
surgical mask outage?	0.119	0.113	0.107	0.101	0.103
eye protection outage?	0.123	0.124	0.118	0.104	0.097
gown outage?	0.126	0.130	0.128	0.112	0.116
glove outage?	0.085	0.070	0.073	0.060	0.065
hand sanitizer?	0.093	0.075	0.068	0.061	0.065
COVID-19 Testing (week ending 9/13)
average time to test results
<1 day	0.104	0.086	0.079	0.074	0.076
1–2 days	0.397	0.431	0.460	0.480	0.503
3–7 days	0.465	0.459	0.436	0.430	0.408
>7 days	0.034	0.025	0.025	0.016	0.013
any resident test in past week?	0.556	0.612	0.617	0.619	0.607
any staff test in past week?	0.799	0.822	0.816	0.818	0.808
testing machine on site?	0.695	0.688	0.685	0.685	0.618
Observations	2279	2918	2684	3221	3803

This table reports average nursing home characteristics and COVID-19 metrics by the nursing home’s CMS overall star rating. We measure overall star rating for each nursing home as reported in June of 2020, which is available for download on the CMS website. All nursing home characteristics, except total number of beds, are taken from the LTC Focus database at Brown University. Total number of beds and all COVID-19 metrics come from the CMS COVID-19 nursing home surveillance data described in Section II.A. Case/Death information and Staff/Supply information are measured through September 13th, 2020; later dates are available upon request. Testing variables are reported for a single week (the week ending September 13th, 2020), as most testing-related questions were only asked from August 16th through November 22nd of 2020.

III. Statistical model and results

We estimate the effect of nursing home quality on the total number of nursing home deaths due to COVID-19 using a negative binomial model. That is, we write the probability of nursing home $i$ having COVID-19 deaths $Y_{i}$ as(1) $\Pr (Y_{i}) = \frac{Γ (Y_{i} + γ_{i})}{Γ (Y_{i} + 1) Γ (γ_{i})} {(\frac{θ}{1 + θ})}^{Y_{i}} {(\frac{1}{1 + θ})}^{γ_{i}}$ where $Γ (\cdot)$ is a gamma function and $γ_{i}$ and $θ$ are the shape and scale parameters, respectively, of a gamma distribution. We allow $γ_{i}$ to vary with nursing home and county characteristics, $X_{i}$ , such that $\ln (γ_{i}) = X_{i} β$ . The parameters $(θ, β)$ are estimated via maximum likelihood. All models include state fixed effects and standard errors are clustered at the state level.

We are primarily interested in the impact that nursing home quality has on COVID-19 deaths; however, we also control for the following: (logged) total number of beds; the percent female, under 65 years old, black, Hispanic, and on Medicaid; whether the home is for profit; acuity index;¹⁵ (logged) county population; and county COVID-19 cases per 1000 residents.¹⁶

III.A. COVID-19 mortality results

We estimate Eq. (1) across several different time periods. We report parameter estimates for the impact of nursing home quality, measured using the overall star rating, on COVID-19 deaths in the first seven rows of Table 3.¹⁷ The first four rows use cumulative counts of COVID-19 deaths, starting January 1st, 2020 through May 24th, September 13th, December 6th (of 2020), and finally April 25th (of 2021).

Table 3. Negative Binomial Estimates, Impact of Overall Star Ranking on Deaths in Nursing Homes.

		Coefficients (standard errors) on Overall star ranking
Period of analysis	Sample mean	2-star	3-star	4-star	5-star
		COVID-19 deaths
(1) As of 5/24/20	1.702	0.000 (0.048)	−0.164 (0.068)	−0.142 (0.073)	−0.173 (0.065)
(2) As of 9/13/20	3.670	0.033 (0.034)	−0.073 (0.052)	−0.092 (0.047)	−0.154 (0.056)
(3) As of 12/06/20	5.570	0.019 (0.023)	−0.045 (0.033)	−0.040 (0.035)	−0.094 (0.036)
(4) As of 4/25/21	8.704	0.026 (0.024)	−0.019 (0.034)	0.003 (0.030)	−0.072 (0.032)
(5) Δ 9/13/20 – 5/24/20	1.969	0.001 (0.042)	−0.066 (0.057)	−0.096 (0.053)	−0.211 (0.057)
(6) Δ 12/06/20 – 9/13/20	1.899	−0.005 (0.039)	0.001 (0.039)	0.008 (0.046)	−0.004 (0.048)
(7) Δ 4/25/21 – 12/06/20	3.135	0.045 (0.034)	0.056 (0.046)	0.058 (0.044)	0.023 (0.051)
		Non-COVID-19 deaths
(8) As of 5/24/20	3.381	0.013 (0.050)	0.015 (0.050)	0.030 (0.050)	0.038 (0.059)
(9) As of 9/13/20	10.146	0.034 (0.029)	0.070 (0.031)	0.088 (0.034)	0.114 (0.041)
(10) As of 12/06/20	15.383	0.032 (0.025)	0.074 (0.025)	0.105 (0.030)	0.123 (0.032)
(11) As of 4/25/21	23.170	0.055 (0.019)	0.092 (0.021)	0.127 (0.025)	0.148 (0.030)
(12) Δ 9/13/20 – 5/24/20	6.765	0.039 (0.026)	0.079 (0.028)	0.103 (0.030)	0.137 (0.033)
(13) Δ 12/06/20 – 9/13/20	5.236	0.059 (0.023)	0.135 (0.024)	0.175 (0.029)	0.200 (0.031)
(14) Δ 4/25/21 – 12/06/20	7.787	0.098 (0.023)	0.150 (0.028)	0.202 (0.033)	0.239 (0.036)
		All deaths
(15) As of 5/24/20	5.082	0.010 (0.043)	−0.007 (0.041)	0.013 (0.046)	0.010 (0.056)
(16) As of 9/13/20	13.816	0.025 (0.028)	0.027 (0.030)	0.045 (0.033)	0.050 (0.039)
(17) As of 12/06/20	20.952	0.027 (0.022)	0.031 (0.021)	0.064 (0.027)	0.061 (0.030)
(18) As of 4/25/21	31.874	0.047 (0.017)	0.055 (0.018)	0.084 (0.023)	0.084 (0.026)
(19) Δ 9/13/20 – 5/24/20	8.734	0.029 (0.028)	0.035 (0.030)	0.053 (0.029)	0.066 (0.036)
(20) Δ 12/06/20 – 9/13/20	7.136	0.049 (0.023)	0.102 (0.020)	0.134 (0.029)	0.153 (0.031)
(21) Δ 4/25/21 – 12/06/20	10.922	0.085 (0.018)	0.121 (0.028)	0.159 (0.035)	0.175 (0.035)

Standard errors allow for arbitrary correlation across observations within a state. Other controls in the models include logged total beds; percent of residents that are female, under 65, black, Hispanic, on Medicaid (along with corresponding indicators for missing variables); for-profit status; acuity index; county-level COVID-19 cases per 1000 residents (measured 23 days prior to death); logged county population; and a full set of state fixed effects.

Our results show that through September 13th, 2020, higher-quality nursing homes experienced fewer deaths from COVID-19. For example, nursing homes with a five-star rating had a COVID-19 death rate that was about 15 percent lower than those with a one-star rating. Rows 3 and 4 show that five-star homes experienced statistically fewer cumulative COVID-19 deaths than lower-quality homes as of December 6th, 2020 and April 25th, 2021; however, these effects are driven entirely by deaths leading up to September, as is shown in rows 6 and 7.¹⁸

We discuss above that the CMS data likely undercounts the true number of COVID-19 deaths in nursing homes, as homes were given the choice on May 24th to report deaths from the prior week or cumulative deaths since January 1st. To show that our results are not somehow driven by this measurement error, in row 5 we change the dependent variable to deaths between September 13th and May 24th (of 2020), which should be measured accurately in the CMS data. These results are similar to our cumulative September 13th findings. Estimated quality effects as of May 24th are also very similar to the results for mid-September (row 1).

We will return to the other two sets of results in Table 3 later in Section III.C

In Table 4 we show that our main findings – the impact of overall star quality on cumulative COVID-19 deaths as of September 13th, 2020 – are robust to a number of alternative empirical specifications, including (row 2) using a Poisson model with clustered standard errors, as suggested by Cameron and Trivedi (2005); using linear models with (row 3) ln(deaths+1), (row 5) the inverse hyperbolic sine of deaths, or (row 7) the inverse hyperbolic sine of death rates as the dependent variable; (rows 4, 6, 8) adding county fixed effects to any of the prior three models; (row 9) adding controls for staff hours per resident day, which may be correlated with home quality conditional on number of beds; and (row 10) adding controls for the county’s republican vote share in the 2016 presidential election, which others have found is negatively correlated with COVID-19 precaution taking, such as compliance with stay-at-home orders (Charoenwong et al., 2020) and vaccination rates (Agarwal et al., 2021).¹⁹ We will discuss the results in row (11) and (12) below.

Table 4. Robustness Analysis, deaths measured 9/13 unless stated otherwise.

			Overall Star Rating
Model	Sample mean	Obs	2-star	3-star	4-star	5-star
(1) Baseline (Table 3, row 2)	3.67	14,905	0.033	−0.073	−0.092	−0.154
			(0.034)	(0.052)	(0.047)	(0.056)
(2) Model (1), but Poisson	3.67	14,905	0.052	−0.126	−0.100	−0.169
			(0.048)	(0.064)	(0.052)	(0.054)
(3) Model (1) but OLS with ln(deaths+1)	0.82	14,905	0.030	−0.036	−0.042	−0.074
			(0.022)	(0.031)	(0.024)	(0.031)
(4) Model (3) but add county FE	0.82	14,905	0.024	−0.068	−0.061	−0.096
			(0.028)	(0.040)	(0.030)	(0.036)
(5) Model (1) but OLS with inverse hyperbolic sine of deaths	1.02	14,905	0.034	−0.043	−0.053	−0.094
			(0.028)	(0.039)	(0.030)	(0.038)
(6) Model (5) but add county FE	1.02	14,905	0.026	−0.082	−0.076	−0.121
			(0.034)	(0.049)	(0.037)	(0.044)
(7) Model (1) but OLS with inverse hyperbolic sine of death rate (per 100 beds)	1.02	14,905	0.034	−0.043	−0.053	−0.094
			(0.028)	(0.039)	(0.030)	(0.038)
(8) Model (7) but add county FE	1.02	14,905	0.026	−0.082	−0.076	−0.121
			(0.034)	(0.049)	(0.037)	(0.044)
(9) Model (1) but add controls for staff hours per resident day	3.67	14,558	0.038	−0.071	−0.079	−0.124
			(0.034)	(0.051)	(0.047)	(0.056)
(10) Model (1) but add controls for republican share of county	3.67	14,888	0.033	−0.073	−0.091	−0.154
			(0.034)	(0.052)	(0.047)	(0.055)
(11) Model (1) but add controls for any staff and resident cases, as well as counts	3.67	14,905	−0.045	−0.082	−0.169	−0.229
			(0.038)	(0.031)	(0.046)	(0.058)
(12) Model (1) but add controls for shortage counts	3.67	14,905	0.043	−0.060	−0.076	−0.136
			(0.033)	(0.051)	(0.045)	(0.054)

III.B. Mechanisms

How did high-quality nursing homes manage to prevent COVID-19 deaths? CMS first offered nursing homes and assisted living facilities guidelines for preventing and managing COVID-19 cases on March 13th, 2020. Since then, these guidelines have been updated and expanded repeatedly as the public health community has learned more about the virus. The CDC’s advice is expansive but is linked by several common themes (CDC, 2020d): First, keep COVID-19 out by limiting visitors and encouraging staff to stay home when ill. Second, clean hands, surfaces, and equipment thoroughly and repeatedly. Third, staff should closely monitor residents for signs of the virus, test symptomatic individuals and close contacts, and isolate those who are symptomatic. Fourth, staff should use PPE at all times. Fifth, after vaccines became available in December of 2020, all residents and staff should be vaccinated.

These recommendations guide our exploration of the potential mechanisms that enabled higher-quality nursing homes to prevent COVID-19 deaths. First, we examine whether higher-quality nursing homes were better able to prevent COVID-19 from entering the home at all. In the first block of results in Table 5 we report results from linear probability models that regress an indicator of whether a home has a single COVID-19 case among its staff (row 1) or among its residents (row 3) as of September 13th, 2020 on the home’s overall star rating and the same set of covariates from our earlier analysis. These results show that higher-quality nursing homes, despite lowering the death rate, were not able to prevent COVID-19 from entering the home. Second, we test whether higher-quality nursing homes were able to prevent the spread of the virus, conditional on having at least one case. In rows 2 (4) of Table 5, we report estimates from a linear regression of log staff (resident) cases on the overall rating and covariates, only for homes with at least one staff (resident) case. The results show that while higher-quality nursing homes were not more effective that lower-quality homes in preventing the spread of COVID-19 among their staff, these homes were more effective at preventing the virus’ spread among their residents. Conditional on having at least one case, all else equal, five-star homes saw roughly 15 percent fewer cases than one-star homes. As of September 13th, among homes with at least one case, the average home has 24.5 cases per 100 beds, meaning this difference amounts to about 3.7 cases per 100 beds.²⁰

Table 5. Negative Binomial Estimates, Impact of Overall Star Ranking on Non-COVID Deaths in Nursing Homes.

			Coefficients (standard errors) on Overall star rating
Period of analysis	Obs.	Sample mean	2-star	3-star	4-star	5-star
			As of 9/13/2020
(1) Any staff cases?	14,905	0.850	−0.010 (0.008)	−0.005 (0.010)	0.000 (0.009)	0.000 (0.013)
(2) ln(staff cases)	12,669	2.044	0.088 (0.027)	0.011 (0.021)	0.046 (0.029)	0.072 (0.030)
(3) Any resident cases?	14,905	0.629	0.024 (0.010)	0.014 (0.010)	0.009 (0.010)	−0.005 (0.010)
(4) ln(resident cases)	9392	2.385	0.027 (0.040)	−0.095 (0.047)	−0.087 (0.049)	−0.147 (0.040)
			From 9/13/2020 to 4/25/2021
(5) Any staff cases?	14,891	0.993	−0.001 (0.002)	0.001 (0.002)	0.000 (0.002)	0.002 (0.001)
(6) ln(staff cases)	14,782	9.959	0.031 (0.022)	0.084 (0.025)	0.116 (0.023)	0.146 (0.030)
(7) Any resident cases?	14,891	0.910	0.009 (0.008)	0.010 (0.007)	0.018 (0.009)	0.010 (0.009)
(8) ln(resident cases)	13,516	2.849	−0.010 (0.035)	0.018 (0.041)	0.036 (0.035)	−0.015 (0.044)

In the second block of results in Table 5, we re-estimate the models from rows (1) through (4) but use cases occurring between September 13th, 2020 and April 25th, 2021. The fraction of homes with any staff and resident cases is nearly one, as the winter wave is in the middle of this time period, so it is not a surprise that the quality measure explains little for these outcomes. Consistent with our COVID-19 deaths results earlier, past September there is no difference between low- and high-quality homes in terms of resident COVID spread.²¹

If high-quality nursing homes prevented deaths not by keeping COVID-19 out of the nursing home entirely, but by preventing spread among residents within the home, the next obvious question is: how? We test several plausible theories. First, both identifying and isolating residents with COVID-19 symptoms requires a capable staff that is of an adequate size; thus, we first test whether high-quality nursing homes have had fewer staffing shortages during the pandemic, which may explain their ability to prevent COVID-19 cases among their residents.

In Panel A of Table 6, we present results from three linear probability models that regress indicators for self-reported staffing shortages (nurses, aides, and clinical staff) at any point between May 24th and September 13th of 2020 on the overall star rating and other controls. Row 1 shows that, all else equal, five-star facilities were 10.7 percentage points less likely to have a nursing shortage over this time period than a one-star facility. The gap is 11.8 percentage points for aides (row 2) and 3.9 percentage points for clinical staff (row 3). All effects are statistically significant at the one-percent level. The estimates are 28, 28, and 23 percent, respectively, of the sample mean for the outcomes.

Table 6. OLS/Maximum Likelihood where Outcomes are Different Measures of Nursing Home Quality.

			Coefficient (Standard error) on Overall starrating
Dependent variable		Sample mean	2-star	3-star	4-star	5-star
A: Staff shortage (any 5/24/20 – 9/13/20)
	Any nursing shortage?	0.378	−0.045 (0.011)	−0.068 (0.014)	−0.085 (0.016)	−0.107 (0.014)
	Any aide shortage?	0.419	−0.050 (0.012)	−0.071 (0.017)	−0.093 (0.018)	−0.118 (0.016)
	Any clinical staff shortage?	0.172	−0.021 (0.010)	−0.033 (0.014)	−0.032 (0.012)	−0.039 (0.012)
B: PPE shortage (any 5/24/20 – 9/13/20)
	N95 outage?	0.173	−0.009 (0.009)	−0.012 (0.011)	−0.027 (0.011)	−0.020 (0.013)
	Surgical mask outage?	0.107	−0.003 (0.009)	−0.004 (0.011)	−0.009 (0.011)	−0.001 (0.014)
	Eye protection outage?	0.111	0.009 (0.008)	0.008 (0.010)	−0.002 (0.011)	−0.001 (0.012)
	Gown outage?	0.121	0.006 (0.008)	0.008 (0.010)	−0.007 (0.009)	0.000 (0.013)
	Glove outage?	0.069	−0.012 (0.007)	−0.006 (0.008)	−0.020 (0.008)	−0.014 (0.010)
	Hand sanitizer outage?	0.071	−0.016 (0.009)	−0.020 (0.008)	−0.028 (0.009)	−0.023 (0.010)
C: Testing procedures (week ending 9/13/20)
	Time to test	2.396	0.003 (0.064)	−0.061 (0.089)	−0.052 (0.062)	−0.103 (0.072)
	Test asymptomatic residents after a new resident case?	0.509	0.067 (0.026)	0.018 (0.029)	0.087 (0.033)	0.045 (0.029)
	Test asymptomatic staff after a new resident case?	0.445	0.070 (0.024)	0.033 (0.028)	0.068 (0.028)	0.060 (0.026)
	Ever test non-exposed resident?	0.374	0.017 (0.016)	0.034 (0.018)	0.048 (0.017)	0.050 (0.022)
	Ever test non-exposed staff?	0.642	0.001 (0.013)	−0.002 (0.018)	0.018 (0.014)	0.016 (0.016)
	Have in-home testing machine?	0.669	0.006 (0.015)	0.010 (0.015)	0.018 (0.018)	−0.007 (0.019)
D: Vaccinations, as of 6/20/21
	Share of residents vaccinated	0.800	0.013 (0.006)	0.024 (0.006)	0.042 (0.006)	0.056 (0.008)
	Share of staff vaccinated	0.567	0.022 (0.007)	0.039 (0.008)	0.066 (0.009)	0.083 (0.009)

Nursing homes may also prevent the spread of COVID-19 by following the CDC recommendation that all nursing home staff use PPE and wash their hands frequently. While these behaviors cannot be observed in our data, we are able to measure shortages in PPE (n95 masks, surgical masks, eye protection, gowns, and gloves) and hand sanitizer; thus, we test whether higher-quality facilities were less likely to have experienced such shortages between May 24th and September 13th of 2020.²² We again use linear probability models and control for the same set of potential confounders as above. Results are presented in Panel B of Table 6. For all five forms of PPE and hand sanitizer, the impact of quality is modest. There is suggestive evidence that higher-quality facilities have fewer shortages of all equipment except gowns, but effects are only statistically significant for a subset of coefficients in the N95 mask, glove, and hand sanitizer regressions.²³

Higher-quality homes may also do a better job of testing residents for the virus. Fortunately, the CMS data allows us to generate a number of measures of testing intensity and speed.²⁴ First, nursing homes report for the week ending September 13th that receiving test results takes “less than a day”, “between one and two days”, “three-to-seven days”, or “more than seven days.” Second, nursing homes report whether they tested any asymptomatic residents during the week ending September 13th in response to a new positive case.²⁵ Third, homes report whether they have tested asymptomatic staff or residents when there is no knowledge of exposure. Finally, homes report whether they have their own testing machine.

In Panel C of Table 6, we present estimates of the effect of nursing home quality on these testing measures. We find that higher-quality homes receive test results faster than lower-quality homes (row 1) but these effects are not precisely estimated. Higher-quality homes are statistically more likely to test asymptomatic residents (row 2) and staff (row 3) following a new case; they are also statistically more likely to have ever tested non-exposed residents (row 4), but are no more likely to test non-exposed staff (row 5). There is no relationship between home quality and the likelihood of having an in-home testing machine (row 6).

Finally, we examine whether higher-quality homes were more likely to vaccinate residents and staff. Starting with the May 30th, 2021 survey, CMS began to ask nursing homes about the number of residents and staff that were fully vaccinated. In Panel D of Table 6, we report results from regressions that have the share of current residents and staff vaccinated as of June 20th, 2021 as the outcomes of interest. Here, there is a monotonic relationship between facility quality and vaccination rates. All else equal, resident and staff vaccination rates are 5.6 and 8.3 percentage points higher in five-star homes than in one-star homes. Both results are statistically significant at conventional levels and represent 7 and 14.6 percent increases, respectively, over the sample means of these outcomes.

Overall, higher-quality nursing homes prevent COVID-19 deaths, not by preventing the disease from entering the home, but by preventing its spread among residents. The methods by which homes prevent the spread are consistent with CDC guidelines. Higher-quality homes had more testing, got tests results faster, and had fewer PPE and staff outages. Given that the negative association between home quality and COVID-19 deaths does not persist past September of 2020, higher vaccination rates at high-quality homes clearly cannot explain the relationship. That said, we take the vaccination results as further evidence that high-quality homes followed CDC guidance along observable dimensions and, thus, likely followed along unobservable dimensions as well.

III.C. Non-COVID and total mortality

In addition to PPE use, adequate staffing, and robust testing, an early prevention method used by virtually all nursing homes was to refuse all outside visitors. On March 13th, 2020, CMS issued memorandum QSO-20–14-NH recommending that all facilities nationwide “should restrict visitation of all visitors and non-essential health care personnel, except for certain compassionate care situations, such as an end-of-life situation” (CMS 2020a). The memorandum also advised cancelling “communal dining and all group activities, such as internal and external group activities.” On May 18th, CMS issued memorandum QSO-20–30-NH, that provided a three-phase reopening plan for nursing homes (CMS, 2020d). The plan did not allow for outside visitors until a nursing home entered phase three, which (loosely) required (i) that COVID-19 cases in the outside community have declined for 14 consecutive days, (ii) no new cases within the nursing home for 28 days, (iii) no staff or PPE shortages, and (iv) homes have the capacity and supplies to test residents and staff weekly. In light of the continued spread of the virus and the strict reopening criteria, many nursing homes were still closed to visitors months later. Finally, on September 17th, 2020, CMS issued memorandum QSO-20–39-NH, that relaxed visitation guidelines, citing resident distress (CMS, 2020b). This memorandum was updated again on March 10th, 2021 in light of vaccine distribution.

In Fig. 2, we use data from SafeGraph to document aggregate foot traffic to US nursing homes from the start of 2019 through mid-2021.²⁶ The figure shows (i) a large drop in nursing home foot-traffic in late January of 2020, approximately when the first COVID-19 case was discovered in the US and the World Health Organization (WHO) declared COVID-19 a global health emergency, followed by (ii) another drop in early March, when WHO declared COVID-19 a pandemic and President Trump declared a national emergency. Foot traffic rose slowly through August of 2020, but surprisingly, has remained mostly unchanged since then, despite several changes in the CMS visitation guidelines outlined above.

There may be some unfortunate downsides to these early policies that could have negatively impacted nursing home resident health. First, these policies may have generated extreme isolation among facility residents. A survey of nursing home residents from early fall of 2020 documents massive declines in resident interactions with outside visitors and fellow residents, trips off site, and even trips outside for fresh air (Montgomery et al., 2020). In the introduction, we noted that some observers caution that among individuals with Alzheimer’s, isolation may be deadly. Second, without group meals or meals supervised by staff, residents may not have been eating as well. In their analysis of Connecticut nursing homes during the early stages of COVID, Levere et al. (2021) found a large decline in patient weight among residents, a change they attribute to the isolation caused by facility safety protocols. Third, the lack of communal activities may have reduced exercise and increased the time patients spent in bed. Levere et al. (2021) also found increases in bed sores among nursing home residents during the early stages of COVID. Fourth, a number of authors have documented large declines in medical care use, especially in the Medicare population, as a result of the COVID-19 pandemic (Bosworth et al., 2020; Ziedan et al., 2020; Cantor et al., 2020; Clemens et al., 2021). We suspect declines in medical care use were mostly generated by precautionary behavior on the part of residents and their families; even the most stringent stay-at-home orders provided exemptions for medical care. There could be some supply response if providers limited the types of visits or reduced hours. Within nursing homes, staff or residents themselves may have been less likely to travel off-site to visit a medical care provider.

Given the potential downsides of efforts to control the spread of the virus, we next examine whether high-quality nursing homes also managed the non-COVID risks created by isolation. In particular, we estimate Eq. (1) above, but with non-COVID deaths as our dependent variable. Like our analysis of COVID-19 deaths, we estimate effects across seven different time periods. The second panel of Table 3(rows 8 through 14) contains parameter estimates. As of May 24th, 2020 (row 8), nursing home quality had no statistical effect on non-COVID deaths in nursing homes; however, by September 13th, 2020 (row 9), we document a large, positive, statistically significant impact of nursing home quality on non-COVID deaths. Five-star homes have about 11 percent more non-COVID deaths than one-star homes and deaths are increasing monotonically with quality. Rows 10 and 11 shows that the relationship between quality and cumulative non-COVID deaths has strengthened with time, as do rows 12 through 14, which look at non-COVID deaths occurring in early, middle, and more recent time intervals. That the relationship has strengthened with time is consistent with the narrative above. The longer patients are exposed to isolation from others and/or regular medical care is not received, the greater risk they face. Moreover, the patterns in Fig. 2 show that despite growing vaccination rates through early 2021, traffic in and out of nursing homes remained well below pre-pandemic levels. We repeat all robustness tests performed for COVID-19 deaths (reported in Table 4) for non-COVID deaths in Appendix Table A6, where non-COVID deaths are measured on September 13th, 2020. Results are robust to all tested specifications.

There is some concern that our findings can be explained by higher-quality nursing homes intentionally misreporting COVID-19 deaths as non-COVID deaths, potentially in an effort to protect their reputation. This concern is somewhat mitigated by Fig. 1. Were COVID-19 deaths consistently reported as non-COVID, we would expect stark rises in non-COVID deaths during the summer and winter waves of 2020; we see neither. Moreover, misreporting is most likely early in the pandemic and results using cumulative deaths as of September are nearly identical to those using deaths between May and September. Finally, the result is present in nursing homes even after COVID-19 vaccines were introduced and disease mortality fell to the lowest level in months.

Yet another alternative explanation of our findings is that COVID-19 “harvests” the most fragile residents from low-quality homes, meaning they are not around to die of non-COVID causes.²⁷ We address this concern, and further address the misreporting concern, by re-estimating Eq. (1) with total nursing home deaths as our dependent variable. We report parameter estimates in the third panel of Table 3(rows 15 through 21). Our results show that there is not any period of time over which high-quality homes experienced fewer total deaths than low-quality ones, even very early in the pandemic (row 15) when statistically, high-quality homes were experiencing far fewer COVID-19 deaths (row 1).²⁸ By December 6th, 2020, high-quality homes had experienced statistically more total deaths than low-quality homes (row 17) and the gap between the two has grown over time (row 18). The last three rows of Table 3 show the urgency of the problem. From summer 2020 (row 19), to fall 2020 (row 20), and finally winter/spring 2021 (row 21), the positive relationship between nursing home quality and total deaths has increased in magnitude. In the latter of these three periods (row 21), five-star homes experienced 17.5 percent more total deaths than one-star homes. On a base of 11 deaths per home, this amounts to roughly two additional deaths in five-star homes over just a four-and-a-half-month period. By our estimates, all of these excess deaths are due to non-COVID causes.

A related concern to the one above is that due to patient churn in and out of nursing homes over time, the composition, and therefore underlying health, of residents may change. For example, it is plausible that fear of COVID-19 led the wealthiest, healthiest potential residents to avoid nursing homes. For this behavior to drive our findings – specifically, our finding that the relationship between quality and total deaths increases over time – it would need to be the case that the average resident at higher-quality homes became sicker, without a similar change occurring at lower-quality homes. Unfortunately, as we do not observe changes in average patient acuity within the home over time, we cannot test this theory. Alternatively, selection could lead higher-quality homes to have higher occupancy rates than lower-quality homes over time. Indeed, over the life of the surveillance data, occupancy rates at high 5-star homes go from three percentage points higher than 1-star homes to just under five percentage points higher. That said, when we control for the home’s occupancy rate in our total death models, our results are very similar.²⁹

In a final attempt validate the important role that visitation restrictions play in our findings, we look for empirical evidence that high-quality nursing homes allowed fewer visitors during the COVID-19 pandemic. To this end, we matched the individual nursing homes in our main data file to the foot traffic data in the SafeGraph point of interest series. Using zip-code and longitude/latitude coordinates in both data files, we were able to identify 12,300 homes with both foot traffic and nursing home quality data. Among these homes, we calculated average daily foot traffic (i) in January 2020 (i.e., pre-pandemic) and (ii) between January 1st, 2021 and April 25th, 2021. We then calculated the percent change from (i) to (ii). On average, foot traffic in nursing homes declined by 34 percent (s.d. 18). We then regress the percent decline in foot traffic on home quality, using a variety of specifications (e.g., including additional controls, state fixed effects, and county fixed effects, and with sample limitations for nursing home size and measurement error).³⁰

We find that while quality is typically negatively associated with the change in foot traffic, the effects are never statistically different from zero; thus, we do not report results here, but they are available upon request. In addition to measurement error, the fact that SafeGraph does not distinguish between the cell phones of visitors, staff, and residents makes the foot traffic data an imperfect proxy for “number of visitors.” This feature of the data poses a challenge to our validation exercise if, for example, high-quality homes experience a decline in visitors, while low-quality homes experience staff shortages (a reality that we document above.) Another concern is that the residents of high-quality homes are both younger and wealthier, meaning they and their visitors are more likely to possess cell phones. As SafeGraph expanded their network over the course of the pandemic, higher cell phone densities in higher-quality nursing homes could boost foot traffic counts, even as the number of visitors declined.

IV. Conclusion

The COVID-19 pandemic has ravaged residents of nursing homes with roughly one fifth of COVID-19 deaths coming from this group. Not surprisingly, the impact of the pandemic varied in some systematic ways across homes. Initially, higher-quality homes were much more successful at limiting the impact of the pandemic, primarily by preventing the spread of the disease once it entered the nursing home, but these differences declined over time. Between January of 2020 and April of 2021, cumulative COVID-19 mortality was lower in higher-quality nursing homes, all else equal; however, starting sometime in the Fall of 2020, the marginal death counts from COVID-19 were no different across nursing home of different quality levels. This finding suggests that higher-quality homes adapted quickly at the start of the pandemic, while lower-quality homes took more time to understand how to effectively contain the virus among residents.

Our results raise a new concern about higher-quality facilities, in that they have higher non-COVID mortality than lower-quality places. This finding is not due to a misclassification of deaths, as the relationship persists even after COVID-19 vaccines were introduced and COVID-19 mortality fell to a fraction of the levels seen at the height of the pandemic. It is also not due to harvesting – the notion that lower COVID-19 deaths in higher-quality homes might mean more residents are available to die from other causes – as higher-quality homes have higher aggregate mortality. A more troubling aspect of our findings is that as the home quality/COVID-19 mortality gradient was eliminated over time, the quality/non-COVID-19 mortality gradient has steadily increased as the pandemic has aged.

Our paper is less successful at identifying the reason for the quality/non-COVID mortality gradient. Our results indicate that higher-quality homes were better at following CMS guidelines designed to control the spread of the virus, such as having PPE equipment on hand, not having staff shortages, more frequent testing of both residents and staff, and having higher staff and resident vaccination rates. It is logical to assume then that higher-quality homes were also better at generating more distance between residents and the outside world by preventing building entry and isolating residents from one another. Anecdotal reports from doctors, nurses, and resident family members (Aronson, 2020; Paulin, 2020; Graham, 2020), as well survey data from residents themselves (Montgomery et al., 2020), document frightening levels of depression, loneliness, and hopelessness. Consistent with these reports, CMS updated their visitation guidelines in mid-September of 2020 to combat the mental and physical distress of isolation.³¹ Analysis of the Minimum Data Set by Levere et al. (2021) during the early stages of the pandemic suggests that nursing home residents declined in health along dimensions consistent with increased isolation, such as unexplained weight loss, declines in cognitive function, and increases in depressive symptoms. This is, however, not the only pathway by which the pandemic could have altered non-COVID mortality. Isolation policies may coincide with, or even cause, reductions in routine medical care, residents’ physical activity, or food consumption. The rise in bed sores and drop in weight found in Levere et al. (2021) suggest these other mechanisms may play a role.

The good news from Fig. 1 is that once vaccines became available, mortality declined considerably. Weekly deaths of nursing home residents with COVID-19 peak at 6082 the week ending December 20th, 2020. By the week ending May 21st, 2021, this number was 179, a 97 percent drop. Fig. 1 provides some hope that things might be returning to normal within these group quarters. Despite this positive trend, not all has returned to normal. The decline in visits to nursing homes in the early stages of the pandemic as measured by cell phone movements was dramatic. By April 1st of 2020, visits to nursing homes were down 51 percent compared to January of 2020. As vaccines became available and COVID-19 deaths in nursing homes fell considerably in early 2021, visits to nursing homes increased but never returned to anywhere near pre-pandemic levels. By the end of June 2021, visits were still down by 35 percent compared to the January 2020 levels. These numbers are imperfect measures of visits because they include counts of family and friends visiting residents plus, entrances by workers, traveling nurses and aids such as physical therapists, etc. It is also not clear whether the persistent decline in visits to nursing homes is supply driven (continue restrictions on the part of nursing homes) or demand driven (friends and family do not want to visit the nursing homes). Despite these caveats, the fact that foot traffic in nursing homes never returns to anywhere near normal levels could explain another feature of our findings – that the impact of higher-quality homes on non-COVID mortality continues to grow with time.

The COVID-19 pandemic presented a unique challenge for nursing homes. Early CMS directives and various state regulations for nursing homes prioritized reducing resident and staff exposure to COVID-19. There was little discussion about the downside risks associated with reducing visitors, communal activities, and resident travel out of the home. Our results suggest that more balanced policies and guidelines that emphasize maximizing the health of residents, rather than just minimizing risk to one disease, may have improved outcomes. For a period of time, CMS and the news media at large measured nursing home COVID-19 performance using cases and deaths only, meaning the logical response on the part of the nursing home was to minimize these counts regardless of the cost. In retrospect, the tone of the discussion and the measurement of outcomes may have led to some deadly consequences. As economists continually stresses, there are benefits and costs to all regulations.

Author statement

This is the author statement for the paper referenced above. The two authors of this paper were engaged in all aspects of the production of this manuscript including: Conceptualization; methodology; software; validation; formal analysis; investigation; resources; data curation; writing;, visualization; supervision; project administration; and funding acquisition.

Acknowledgement

The authors gratefully acknowledge financial support for this work from Notre Dame Research as part of the COVID-19 Economic and Social Science Research Initiative. This research was also supported by NIH grant #R21AI164391. We thank Zachary Yamada, Mary Grace Donnelly, and Colin Davidson for excellent research assistance. We also received helpful comments from participants of the 2020 NBER Fall Health Economics Program Meeting, as well as the NBER Conference on COVID-19 and Health Outcomes. A version of this paper was first circulated as NBER working paper 28012 in October of 2020.

Appendix A. Preparation of the estimation sample

We downloaded the CMS Covid-19 Nursing Home data file on August 2nd, 2021, which contains information reported through the week of July 18th, 2021; 61 weeks total. Our analysis is limited to the weeks ending May 24th, 2020 through May 9th, 2021; 51 weeks. Over this time-frame, 15,421 homes report data to CMS in at least one week. Squaring the data produces 786,471 home-week observations. Roughly 10 percent of these observations are missing or are flagged by CMS for poor quality (henceforth, “problem” observations); however, a small minority of homes account for most of the errors. We drop all homes with more than ten problem observations; 15,110 homes and 770,610 observations remain, which includes just 4574 problem observations (0.59 percent of all observations), 28 percent of which are in the first two weeks of reporting.

Cumulative death counts are calculated by CMS from weekly death reports; thus, any problem observations create measurement error in all cumulative counts moving forward. As such, we impute weekly death counts for all problem observations and recalculate the cumulative counts using the procedure that follows. Step 1: For the weeks ending May 2nd and May 9th of 2021 (not used in any regression analysis in the paper), we regress weekly COVID-19 deaths and weekly non-COVID deaths on new county COVID-19 cases and the number of beds, as well as quadratics of these variables, for all non-problem observations. We then use the regression coefficients to predict weekly COVID and non-COVID deaths for problem observations in these weeks. Step 2: Starting with the week ending April 25th, 2021 (the last week used in regression analysis in the paper), we predict weekly COVID (non-COVID) deaths for problem observations using new county COVID cases and the number of beds, as well as quadratics of these variables, and COVID (non-COVID) deaths over the following two weeks at the nursing home. Step 3: repeats step 2 recursively for the weeks ending April 18th, 2021 through May 24th, 2020.³²

Finally, note that among the 15,110 homes that comprise our final CMS sample (e.g., see summary statistics in Appendix Table A3), 205 homes cannot be matched to a home in the star-quality data, which explains the sample size of 14,905 in our main regression analysis.

B. Data quality check

To validate aggregate death counts in the CMS nursing home data, we compared it to data compiled by the COVID-19 Tracking Project (CTP), a web page maintained by the Atlantic.³³ The authors of the web page aggregate weekly data on COVID-19 deaths among nursing home residents using state COVID-19 dashboards, private correspondences with states, and state press conferences. For many states, the CTP has data by individual facility. In these cases, we aggregated data by week up to the state level. Some states report weekly totals separated by sector (e.g., nursing homes, assisted living, etc.) while other states aggregate these sectors together making the data not comparable to the CMS statistics. Dropping states that either do not report nursing home deaths, do not distinguish between nursing homes and other senior living facilities like assisted living, or do not report until after the first CMS weekly report on March 24th, 2020, we can generate consistent data from the two sources for 37 states. The CTP stopped collecting this data by March 7th, 2021.

In Table A1, we report in the first column aggregate deaths in the first CMS nursing home report for the week ending May 24th, 2020, plus data from the CTP for the 37-state sample. The CMS data under-reports death counts by 14.5% for this first report. This is due to two potential limitations of the CMS data. First, some nursing homes did not report that first week. We believe this is a small component of the problem as mortality data is reported for 98% of nursing homes that week. Second, at the time of the first CMS report (May 24th), CMS allowed nursing homes the choice to report cases and deaths from the prior week or cumulative cases and deaths since January 1st. Thereafter, homes report weekly counts and a cumulative count is calculated by CMS; thus, if a home fails to report the cumulative count since January 1st on May 24th, the cumulative count that CMS calculates in future weeks is incorrect. In the second column of Table A1, we report cumulative COVID-19 deaths through the last week of data in the CTP and in the final column, we compare the difference between the last and first dates. The fraction undercount in the CMS data is only 5.6 percent in early March 2021, and the difference in counts between these two dates is only 3.3 percent. These results suggest that the major under-reporting in the CMS data is occurring in the first week, but that cumulative counts after the first week are more comparable to what nursing homes are reporting to states.

Table A1. Comparing State and CMS Reports of COVID-19 Deaths in Nursing Homes for 37 States.

Source	Deaths as of 5/24/2020 (a)	Deaths as of 3/7/2021 (b)	Δ deaths (b) – (a)
CMS (1)	21,189	104,374	83,185
State facility reports (2)	24,286	110,254	85,968
Difference (2) – (1)	3079	5880	2783
% difference (2) – (1)	14.5%	5.6%	3.3%

Calculations made for the 37 US states reporting COVID-19 deaths among nursing home residents on state dashboards. State facility reports refer to the death counts from these dashboards, while CMS refers to the deaths counts in the CMS COVID-19 surveillance data discussed in Section II.A.

This is visually verified in Fig. A1 where we plot on the horizontal axis the nursing home death counts as of March 24th, 2020 in the CTP data, while the vertical axis has the comparable data from the CMS data for our 37-state sample. There are a noticeable number of points that fall below the 45-degree line, indicating CMS undercounts relative to CTP (i.e., state reports) at that time.

In Fig. A2, we re-do Fig. A1 but use the difference in counts between March 2021 and May 2020 as the outcome of interest. Here there is a much more even spread of points around the 45- degree line.

These numbers suggest that to accurately assess the cumulative impact of COVID-19 mortality in nursing homes, we need to inflate the first week’s numbers then recalculate cumulative deaths after that point. In Table A2, we calculate cumulative deaths as of August 15th, 2021 by inflating the first CMS report by 14.5%, then adding to this the cumulative deaths reported to CMS between the first report and August 15th, 2021. Using this method, we estimate that there were 137,318 COVID-19 deaths among nursing home residents as of August 15th, 2021. At that point in time, there were 634,179 COVID-19 deaths in total in the US, meaning that 21.7% of COVID-19 deaths were to nursing home residents.

Table A2. National Estimates of COVID-19 Deaths in Nursing Homes from Inflated CMS Data,As of 8/15/2021.

Number	Death Count
(1) National estimates as of 5/24/2020	25,354
(2) Inflate by 14.5%	29,030
(3) Change between 8/15/2021 and 5/24/2020	108,288
(4) total (3) + (2)	137,318

Row 1 measures death counts among nursing home residents in all 50 states as reported in the CMS COVID-19 surveillance data discussed in Section II.A. Row 2 inflates this figure by the estimated 5/24/2020 undercount calculated in Table A1. Row 4 adds to this figure deaths reported to CMS after 5/24/2020 (Row 3) which we’ve show are more accurately reported.

Table A3. Sample Characteristics, CMS Data on Nursing Homes, as of September 13th, 2020.

Variable	Mean	S.D.
Total beds	106.222	58.858
Share of female residents	0.663	0.119
Share of female residents, missing	0.086	0.280
Share of residents under 65 years old	0.226	0.177
Share of residents under 65 years old, missing	0.547	0.498
Share of black residents	0.165	0.220
Share of black residents, missing	0.446	0.497
Share of hispanic residents	0.049	0.134
Share of black residents, missing	0.421	0.494
Share of residents on Medicaid	0.599	0.230
For profit	0.702	0.457
Acuity index	12.189	1.479
Medicaid, profit, and acuity missing	0.062	0.242
Observations	15,110

The construction of this sample is discussed in Appendix Section A. Total beds is measured using the CMS COVID-19 surveillance data discussed in Section II.A. All other variables come from the LTC Focus database at Brown University.

C. CMS star quality ratings

The nursing home star ratings come from data.medicare.gov. There are three separate ratings – inspection, quality measures (QM), and staffing – which are aggregated by CMS into an overall rating. All three ratings, as well as the overall rating, measure quality in integer “star” values, where five-star is the best possible rating and one-star is the worst.

The inspection rating is based on results from the home’s three most recent state health inspections in a three-year period, with more weight given to the most recent inspections, as well as investigations stemming from formal complaints. The ratings used in our analysis were first reported by CMS in June of 2020 and we can verify in the data that the latest inspections informing the rating took place in February of 2020, before the start of the pandemic. The QM rating is based on a home’s self-reported ability to manage and prevent certain negative health outcomes (e.g., bedsores, ED visits, chronic pain, major injuries resulting from falls, urinary tract infections, etc.). The staff rating is a function of the reported number of registered nurses and total staffing hours relative to the number of residents.

As the inspection rating is the only measure calculated from data that is not self-reported, it is viewed as the most objective and, thus, is given greater weight in the calculation of the overall star rating (Williams et al., 2010). The rating system has been criticized by many, especially the QM and staff rankings portions. Exposés by the New York Times at various points in time (e.g., Thomas, 2014; Silvger-Greenberg and Gebeloff, 2021) demonstrate that for many nursing homes, the self-reported data is at best incomplete and at worst fraudulent. Looking at the QM data, Sanghavi et al. (2019) document that only 57% of nursing home falls are reported to CMS. Comparing Medicare claims for inpatient services with data reported to CMS for the five-star ranking, Integra Med Analytics (2021) found little correlation for hospital claims based measured of quality and what is reported in the five-star data for urinary tract infections, falls, and bed sores. Numerous authors have shown that the inspection rating is predictive of better health outcomes among residents (Fuller et al., 2019; Perraillon et al., 2017), but the strength of the relationship is in question for some scales. In one of the largest studies to date, Neuman et al. (2014)found the inspection rating predicted hospital readmissions for people discharged to a nursing home but the staff rating did not.

Surveys suggest that the rating system is correlated with family and resident satisfaction with care (Çalikoglu et al., 2011), but that the inspection rating seems to be most correlated with these measures of satisfaction (Williams et al., 2016).

The rating distributions across homes in our sample can be found in Appendix Table A4 below. In Appendix Table A5 we report the correlation coefficients across nursing homes for the four measures. It is not surprising that the overall quality and the inspection scale are the most correlated since the latter is weighted most heavily when calculating the former. The level of correlation between the inspection rating and QM and staff ratings is very low; the latter two ratings are also not highly correlated with one another.

Table A4. Distribution of Inspection Ratings in CMS Nursing Home Data.

Star rating	Overall rating	Inspection rating	QM rating	Staff rating
1	0.151	0.193	0.049	0.075
2	0.193	0.236	0.126	0.247
3	0.178	0.224	0.196	0.276
4	0.213	0.233	0.251	0.215
5	0.252	0.102	0.363	0.114
missing	0.014	0.014	0.014	0.074
Observations	15,110

Sample construction is discussed in Appendix Section A. Star ratings are taken from the CMS website.

Table A5. Correlation if the Inspection Ratings in CMS Nursing Home Data.

Star rating	Overall rating	Inspection rating	QM rating	Staff rating
Overall rating	1.000
Inspection rating	0.860	1.000
QM Rating	0.504	0.233	1.000
Staff rating	0.478	0.221	0.212	1.000

Sample construction is discussed in Appendix Section A. Star ratings are taken from the CMS website.

Table A6. Robustness Analysis, non-COVID deaths measured 9/13/2020 unless stated otherwise.

			Overall star rating
Model	Sample Mean	Obs	2-star	3-star	4-star	5-star
(1) Baseline (Table 3, row 9)	10.15	14,905	0.034	0.070	0.088	0.114
			(0.029)	(0.031)	(0.034)	(0.041)
(2) Model (1), but Poisson	10.15	14,905	0.045	0.051	0.043	0.059
			(0.052)	(0.085)	(0.072)	(0.057)
(3) Model (1) but OLS with ln(deaths+1)	8.84	11,532	0.033	0.091	0.118	0.123
			(0.038)	(0.038)	(0.042)	(0.053)
(4) Model (3) but add county FE	1.84	14,905	0.041	0.073	0.099	0.127
			(0.034)	(0.034)	(0.036)	(0.047)
(5) Model (1) but OLS with inverse hyperbolic sine of deaths	1.84	14,905	0.069	0.071	0.127	0.159
			(0.046)	(0.046)	(0.049)	(0.064)
(6) Model (5) but add county FE	2.29	14,905	0.047	0.089	0.117	0.148
			(0.041)	(0.040)	(0.044)	(0.056)
(7) Model (1) but OLS with inverse hyperbolic sine of death rate (per 100 beds)	2.29	14,905	0.080	0.088	0.151	0.186
			(0.055)	(0.055)	(0.059)	(0.077)
(8) Model (7) but add county FE	2.29	14,905	0.047	0.089	0.117	0.148
			(0.041)	(0.040)	(0.044)	(0.056)
(9) Model (1) but add controls for staff hours per resident day	2.29	14,905	0.080	0.088	0.151	0.186
			(0.055)	(0.055)	(0.059)	(0.077)
(10) Model (1) but add controls for republican share of county	10.15	14,558	0.031	0.064	0.081	0.112
			(0.030)	(0.033)	(0.035)	(0.042)
(11) Model (1) but add controls for any staff and resident cases, as well as counts	10.15	14,888	0.033	0.071	0.090	0.118
			(0.029)	(0.031)	(0.034)	(0.041)
(12) Model (1) but add controls for shortage counts	10.15	14,905	0.032	0.072	0.087	0.112
			(0.029)	(0.031)	(0.034)	(0.039)

References

Abrams, Loomer, Gandhi and Grabowski, 2020: H.R. Abrams, L. Loomer, A. Gandhi, D.CGrabowski

Characteristics of us nursing homes with covid-19 cases

J. Am. Geriatr. Soc., 68 (8) (2020), pp. 1653-1656

View PDF CrossRef View Record in Scopus Google Scholar
Agarwal et al., 2021: R. Agarwal, M. Dugas, J. Ramaprasad, J. Luo, G. Li, G.G. Gao

Socioeconomic privilege and political ideology are associated with racial disparity in COVID-19 vaccination

Proceedings of the National Academy of Sciences, 118 (2021), Article e2107873118

View PDF CrossRef View Record in Scopus Google Scholar
Aronson, 2020: L. Aronson

For Older people, despair, As Well As COVID-19, is Costing Lives

The New York Times (2020)

June 8Retrieved October 10, 2021

https://www.nytimes.com/2020/06/08/opinion/coronavirus-elderly-suicide.html

Google Scholar
Bosworth, Ruhter, Sheingold and Zuckerman, 2020: A. Bosworth, J. Ruhter, S. Sheingold, R.Zuckerman

The Impact of the COVID-19 Pandemic on Medicare Beneficiary Use of Health Care Services and Payments to Providers: Early Data for the First 6 Months of 2020 (Issue Brief No. HP-2020-01)

Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, Washington, DC (2020)

Google Scholar
Bui et al., 2020: D.P. Bui, I. See, E.M. Hesse, K. Varela, R.R. Harvey, E.M. August, A. Winquist, S.Mullins, S. McBee, E. Thomasson, A. Atkins

Association Between CMS Quality Rating and COVID-19 Outbreaks in Nursing Homes – West Virginia, March 17-June 11, 2020

Morbidity Mortal. Weekly Rep., 69 (37) (2020), pp. 1300-1304

View PDF CrossRef View Record in Scopus Google Scholar
Cameron and Trivedi, 2005: A.C. Cameron, P.K. Trivedi

Microeconometrics: Methods and Applications

Cambridge University Press, New York (2005)

Google Scholar
Cantor et al., 2020: J.H. Cantor, N. Sood, D. Bravasta, M. Pera, C.M Whaley

The impact of the COVID-19 pandemic and policy response on health care utilization: Evidence from county-level medical claims and cellphone data

(2020)

Google Scholar
Centers for Disease Control and Prevention 2020a: Centers for Disease Control and Prevention

FastStats – Alzheimer’s disease

(2020)

Google Scholar
Centers for Disease Control and Prevention 2020b: Centers for Disease Control and Prevention. 2020b. Instructions for completion of the COVID-19 long-term care facility (LTCF): resident impact and facility capacity form. CDC 57.144. https://www.cdc.gov/nhsn/pdfs/covid19/ltcf/57.144-toi-508.pdf

Google Scholar
Centers for Disease Control and Prevention 2020c: Centers for Disease Control and Prevention. 2020c. Interim clinical guidance for management of patients with confirmed coronavirus disease (COVID-19). Retrieved September 16, 2020.https://www.cdc.gov/coronavirus/2019-ncov/hcp/clinical-guidance-management-patients.html

Google Scholar
Centers for Disease Control and Prevention 2020d: Centers for Disease Control and Prevention. 2020d Nursing homes and long-term care facilities. Retrieved August 24, 2020. https://www.cdc.gov/coronavirus/2019-ncov/hcp/nursing-home-long-term-care.html

Google Scholar
Centers for Medicare and Medicaid Services 2020a: Centers for Medicare and Medicaid Services. 2020a. Guidance for infection control and prevention of coronavirus disease 2019 (COVID-19) in nursing homes. Reference Number QSO-20-14-NH. https://www.cms.gov/files/document/qso-20-14-nh-revised.pdf

Google Scholar
Centers for Medicare and Medicaid Services 2020b: Centers for Medicare and Medicaid Services. 2020b. Nursing home visitation – COVID-19. Reference Number QSO-20-39-NH. https://www.cms.gov/files/document/qso-20-39-nh.pdf

Google Scholar
Centers for Medicare and Medicaid Services 2020c: Centers for Medicare and Medicaid Services. 2020c. Interim final rule updating requirements for notification of confirmed and suspected COVID-19 cases among residents and staff in nursing homes. Reference Number QSO-20-29-NH. https://www.cms.gov/files/document/qso-20-29-nh.pdf

Google Scholar
Centers for Medicare and Medicaid Services 2020d: Centers for Medicare and Medicaid Services. 2020d. Nursing home reopening recommendations for state and local officials. Reference Number QSO-20-30-NH. https://www.cms.gov/files/document/qso-20-30-nh.pdf

Google Scholar
Charoenwong, Kwan and Pursiainen, 2020: B. Charoenwong, A. Kwan, V. Pursiainen

Social Connections with COVID-19-affected ares increase compliance with mobility restrictions

Sci Adv, 6 (47) (2020), p. eabc3054

View PDF View Record in Scopus Google Scholar
Chen, Chevalier and Long, 2021: N.K. Chen, J.A. Chevalier, E.F Long

Nursing home staff networks and COVID-19

Proceedings of the National Academy of Sciences, 118 (2021), Article e2015455118

View PDF CrossRef Google Scholar
Clemens, Jacobi and Greenwood, 2021: J. Clemens, M. Jacobi, B.N Greenwood

Patient access to chronic medications during the COVID-19 pandemic: evidence from a comprehensive dataset if US insurance claims

PLoS ONE, 16 (4) (2021), Article e0249453

Google Scholar
Cornell, Grabowski, Norton and Rahman, 2019: P.Y. Cornell, D.C. Grabowski, E.C. Norton, M.Rahman

Do report cards predict future quality? The case of skilled nursing facilities

J. Health Econ., 66 (July) (2019), pp. 208-221

Article Download PDF View Record in Scopus Google Scholar
Dyer, Murphy, Lawlor and Kennelly, 2020: A.H. Dyer, C. Murphy, B. Lawlor, S.P. Kennelly, Study Group, for the NILVAD

Social networks in mild-to-moderate Alzheimer disease: longitudinal relationships with dementia severity, cognitive function, and adverse events

Aging Ment Health, 25 (10) (2020), pp. 1923-1929

Google Scholar
Fuller, Goldfield, Hughes and McCullough, 2019: R.L. Fuller, N.I. Goldfield, J.S. Hughes, E.CMcCullough

Nursing home compare star rankings and the variation in potentially preventable emergency department visits and hospital admissions

Popul. Health Manag., 22 (2) (2019), pp. 144-152

View PDF View Record in Scopus Google Scholar
Gorges and Konetzka, 2020: R.J. Gorges, R.T Konetzka

Staffing levels and COVID-19 cases and outbreaks in US nursing homes

J. Am. Geriatr. Soc., 68 (11) (2020), pp. 2462-2466

View PDF CrossRef View Record in Scopus Google Scholar
Graham, 2020: C. Graham

For seniors, COVID-19 Sets Off a Pandemic of Despair

Kaiser Health Network Web Publication (2020)

Retrieved May 28, 2020

https://khn.org/news/for-seniors-covid-19-sets-off-a-pandemic-of-despair/

Google Scholar
Harrington et al., 2020: C. Harrington, L. Ross, S. Chapman, E. Halifax, B. Spurlock, D.Bakerjian

Nurse staffing and coronavirus infections in California nursing homes

Policy Politic. Nurs. Practic., 21 (3) (2020), pp. 174-186

View PDF CrossRef View Record in Scopus Google Scholar
He, Li and Fang, 2020: M. He, Y. Li, F. Fang

Is there a link between nursing home reported quality and COVID-19 Cases? Evidence from California skilled nursing facilities

J. Am. Med. Dir. Assoc., 21 (7) (2020), pp. 905-908

Article Download PDF View Record in Scopus Google Scholar
Howley, 2019: E.K. Howley

Nursing Home Facts and Statistics

US News and World Report (2019)

Retrieved October 18, 2020

https://health.usnews.com/health-news/best-nursing-homes/articles/nursing-home-facts-and-statistics#:∼:text=This%20means%2015.5%25%20of%20the,a%20surgical%20procedure%20or%20injury

Google Scholar
Integra Med Analytics 2020: Integra Med Analytics. 2020. Underreporting in nursing home quality measures. Retrieved October 10, 2021. https://www.nursinghomereporting.com/post/underreporting-in-nursing-home-quality-measures.

Google Scholar
Konetzka and Gray, 2017: R.T. Konetzka, Z.S. Gray

The role of socioeconomic status in nursing home quality ratings

Seniors Hous. Care J., 25 (1) (2017), pp. 3-14

View Record in Scopus Google Scholar
Konetzka, Yan and Werner, 2021: R.T. Konetzka, K. Yan, R.M. Werner

Two decades of nursing home compare: what have we learned?

Med. Care Res. Rev., 78 (4) (2021), pp. 295-310

Google Scholar
Lepore, Ferrell and Weiner, 2017: M. Lepore, A. Ferrell, J.M Weiner

Living arrangements of people with Alzheimer’s disease and related dementias: implications for services and support

Issue Brief, RTI Int. (2017)

https://aspe.hhs.gov/system/files/pdf/257966/LivingArran.pdf

Google Scholar
Levere, Rowan and Wysocki, 2021: M. Levere, P. Rowan, A. Wysocki

The adverse effects of the COVID-19 pandemic on nursing home resident well-being

J. Am. Med. Dir. Assoc., 22 (5) (2021), pp. 948-954

View PDF View Record in Scopus Google Scholar
Li, Temkin-Greener, Shan and Cai, 2020a: Y Li, H. Temkin-Greener, G. Shan, X. Cai

COVID-19 infections and deaths among connecticut nursing home residents: facility correlates

J. Am. Geriatr. Soc., 68 (9) (2020), pp. 1899-1906

View PDF CrossRef View Record in Scopus Google Scholar
Li, Cen, Cai and Temkin-Greener, 2020b: Y. Li, X. Cen, X. Cai, H. Temkin-Greener

Racial and ethnic disparities in COVID-19 infections and deaths across US nursing homes

J. Am. Geriatr. Soc., 68 (11) (2020), pp. 2454-2461

View PDF CrossRef View Record in Scopus Google Scholar
Montgomery, Slocum and Stanik, 2020: A. Montgomery, S. Slocum, C. Stanik

Experiences of nursing home residents during the pandemic

Altarum Spec. Rep. (2020)

https://altarum.org/sites/default/files/uploaded-publication-files/Nursing-Home-Resident-Survey_Altarum-Special-Report_FINAL.pdf

Google Scholar
Neuman, Wirtalla and Werner, 2014: M.D. Neuman, C. Wirtalla, R.M. Werner

Association between skilled nursing facility quality indicators and hospital readmissions

JAMA, 312 (15) (2014), pp. 1542-1551

View PDF CrossRef View Record in Scopus Google Scholar
Paulin, 2020: E. Paulin

Is Extended Isolation Killing Older Adults in Long-Term Care?

AARP Web Publication (2020)

Retrieved September 3, 2020

https://www.aarp.org/caregiving/health/info-2020/covid-isolation-killing-nursing-home-residents.html

Google Scholar
Perraillon, Brauner and Konetzka, 2019: M.C. Perraillon, D.J. Brauner, R.T Konetzka

Nursing home response to nursing home compare: the provider perspective

Med. Care Res. Rev., 76 (4) (2019), pp. 425-443

View PDF CrossRef View Record in Scopus Google Scholar
Sanghavi, Ban and Caudry, 2019: P. Sanghavi, S. Ban, D. Caudry

Assessment of nursing home reporting of major falls for quality measurement on nursing home compare

Health Serv Res, 55 (2) (2019), pp. 201-210

Google Scholar
Silver-Greenberg and Gebeloff, 2021: J. Silver-Greenberg, R. Gebeloff

Maggots, Rape and Yet Five Stars: How U.S. Ratings of Nursing Homes Misled the Public

New York Times (2021)

March 13Retrieved October 10, 2021

https://www.nytimes.com/2021/03/13/business/nursing-homes-ratings-medicare-covid.html

Google Scholar
Steptoe, Shankar, Demakakos and Wardle, 2013: A. Steptoe, A. Shankar, P. Demakakos, J.Wardle

Social isolation, loneliness, and all-cause mortality in older men and women

Proceedings of the National Academy of Sciences, 110 (2013), pp. 5797-5801

View PDF CrossRef View Record in Scopus Google Scholar
Thomas, 2014: K. Thomas

Medicare Star Ratings System Allows Nursing Homes to Game the System

New York Times (2014)

August 24. Retrieved October 10, 2021

https://www.nytimes.com/2014/08/25/business/medicare-star-ratings-allow-nursing-homes-to-game-the-system.html

Google Scholar
Tolbert et al., 2020: J. Tolbert, C. Hall, K. Orgera, N. Singer, A. Mengistu, M. Tian, A.Ramaswamy

State Data and Policy Actions to Address Coronavirus

Kaiser Family Foundation (2020)

Interactive Tool

https://www.kff.org/report-section/state-data-and-policy-actions-to-address-coronavirus-maps-and-data/

Google Scholar
Unroe et al., 2012: K.T. Unroe, M.A. Greiner, C. Colon-Emeric, E.D. Peterson, L.H Curtis

Associations between published quality ratings of skilled nursing facilities and outcomes of Medicare beneficiaries with heart failure

J. Post-Acute Long-Term Care Med., 13 (2) (2012), pp. 188.e1-188.e6

Article Download PDF Google Scholar
U.S. Census Bureau 2020: U.S. Census Bureau. 2020. ACS demographic and housing estimates. Retrieved October 18, 2020. https://data.census.gov/cedsci/table?q=United%20States&tid=ACSDP1Y2019.DP05&hidePreview=false

Google Scholar
USAFacts 2020: USAFacts. 2020. US coronavirus cases and deaths. Retrieved October 5, 2020. https://usafacts.org/visualizations/coronavirus-covid-19-spread-map/

Google Scholar
Williams, Straker and Applebaum, 2016: A. Williams, J.K. Straker, R. Applebaum

The nursing home five star rating: how does it compare to resident and family views on care?

Gerontologist, 56 (2) (2016), pp. 234-242

View PDF CrossRef View Record in Scopus Google Scholar
Williams et al., 2010: C. Williams, L. Hadden, E. Mortimore, F. Nagy, M. Plotzke, A. White

Measuring nursing home quality – the five-star rating system

SAS Glob Forum (2010)

Google Scholar
Zhou et al., 2020: F. Zhou, T. Yu, R. Du, G. Fan, Y. Liu, Z. Liu, J. Xiang, Y. Wang, B. Song, X. Gu, L. Guan

Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study

The Lancet, 395 (10229) (2020), pp. 1054-1062

Article Download PDF View Record in Scopus Google Scholar
Ziedan, Simon and Wing, 2020: Ziedan, E., Simon, K.I., & Wing, C. 2020. Effects of state COVIV-19 closure policy on non-COVID-19 health care utilization. NBER Working Paper #27621.

Google Scholar

¹: https://coronavirus.1point3acres.com/.

²: We outline the data used in these calculations in the next section.

³: The Kaiser Family Foundation notes that early in the pandemic, 27 states banned visitors and 22 states recommended that nursing homes ban visitors (Tolbert et al., 2020). Two states provided no guidance.

⁴: https://www.cdc.gov/nchs/nvss/vsrr/covid19/excess_deaths.htm#data-tables

⁵: https://data.cdc.gov/NCHS/Conditions-Contributing-to-COVID-19-Deaths-by-Stat/hk9y-quqm.

⁶: While only 12.5 percent of Alzheimer patients live in nursing homes (Lepore et al., 2017), just under half of nursing home residents have Alzheimer’s (CDC, 2020a). In 2018, 50.6 percent of all deaths listing Alzheimer’s as an underlying cause occurred in nursing homes. (Authors’ calculations from CDC Wonder Multiple Cause of Death data.)

⁷: There is a public health literature that evaluates the extent to which CMS star ratings measure true quality. Konetzka et al. (2021) review this literature and point to two types of evidence that the overall star rating captures relevant information on quality. The first type of evidence is that the overall star rating is highly predictive of characteristics typically associated with quality, such as the share of Medicaid pay residents and resident education (Konetzka & Gray, 2017; Perraillon et al., 2019), something we document in this paper as well. The second type of evidence shows that homes with higher overall star ratings have fewer hospital admissions and readmissions and lower mortality (Cornell et al., 2019; Unroe et al., 2012). There is a more nuanced debate regarding the reliability of the component (i.e., inspection, quality metrics, and staff) star ratings. We review this literature in Appendix Section C.

⁸: We calculate these numbers through then end of 2020 as it is easiest to obtain a denominator for January of 2020.

⁹: The CMS data indicates there are 1.1 million nursing home residents as of the first CMS weekly report and there were about 72,000 deaths from all causes in nursing homes up to that point. Adding these two numbers together gives us roughly 1.2 million nursing home residents at the beginning of the year. This number is an approximation as residents could have moved into a nursing home and died. The CMS data report 107,342 deaths through the week ending January 3^rd, 2021. Applying the correction factor from the appendix to take into consideration the underreporting of deaths in the first week, this generates roughly 111,000 COVID-19 deaths in nursing homes in 2020, for a death rate of 9,252.

¹⁰: Census estimates a US population of 329 million in January of 2020, including 54 million people 65 and older (US Census Bureau, 2020). There were 392,356 COVID-19 deaths by the week ending January 2^nd, 2021, meaning 281,338 were outside of nursing homes. One estimate suggests 15.5 percent of the nursing home population is under 65 (Howley, 2019), leaving 1 million people aged 65 and over living in nursing homes and 55 million people aged 65 or over living outside of nursing homes. The CDC reports that 96 percent of deaths in nursing homes were to people 65 and older. Applying this ratio to the CMS numbers suggests that roughly 106,577 COVID-19 deaths in nursing homes were to people aged 65 and over. Subtracting this from the 317,020 COVID-19 deaths to people aged 65 and over in the US, there were 210,443 COVID-19 deaths for people aged 65 and over outside of nursing homes, for a death rate of 390. We acknowledge that this figure is likely overstated as the CDC reports place of death (e.g., hospital, at home, nursing home), so the counts for people outside nursing homes in places like hospitals, emergency rooms and hospice facilities would include some nursing home residents as well.

¹¹: More information can be found at http://ltcfocus.org/.

¹²: More details, as well as the methods for calculating the overall rating can be found at: https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/CertificationandComplianc/downloads/brieffivestartug.pdf

¹³: Homes are to report all deaths regardless of location, e.g., in the home or in a hospital (CMS, 2020c). The module instructions (CDC, 2020b) define a COVID-19 death as “a resident with suspected or a positive COVID-19 test result who died in the facility or another location as a result of COVID-19 related complications.” The instructions state the following regarding the reporting of marginal COVID-19 deaths: (i) suspected deaths are those that are being managed for COVID-19 symptoms, but do not have a positive test, and these symptoms play a role in their death; (ii) someone without a positive test or symptoms who dies from complications associated with COVID-19 and later has COVID-19 diagnosed in autopsy, should be coded (retrospectively) as a COVID-19 death; (iii) someone previously diagnosed or suspected to be COVID-19 positive, who dies after recovery should not be counted as a COVID-19 death.

¹⁴: Other time periods look very similar and are available upon request.

¹⁵: Nursing home characteristics are missing for some homes; thus, we include missing variable indicators as well.

¹⁶: County-level COVID-19 cases are included as a measure of the intensity of the virus locally. As death occurs on average 18.5 days after symptom onset (Zhou et al. 2020) and the incubation period is 4-5 days on average (CDC, 2020c) we measure cases 23 days prior to death. All models are robust to controlling for the county’s non-nursing home death rate rather than the case rate.

¹⁷: Results are very similar if the inspection rating is used in place of the overall rating. Results for all quality measures are available upon request.

¹⁸: Across the seven specifications, we find that homes with more beds, larger black populations, and located in larger cities with higher case rates nearly always have more COVID-19 deaths, while homes with younger populations have fewer deaths. We also find that for-profit homes have significantly more deaths. These results are available upon request.

¹⁹: We obtained 2016 presidential vote share data from https://electionlab.mit.edu/data.

²⁰: We explored an additional, related mechanism – that higher-quality nursing homes provided better care conditional on infections, leading to a lower death rate. To do so, we returned to Equation 1, but controlled (separately) for whether any staff or residents had tested positive for COVID-19, as well as the total number of staff and resident cases. If high-quality facilities only prevent death by reducing cases, then we would expect quality to have no impact on death counts in this model. Estimates can be found in Table 4, row 11. Conditional on cases, higher-quality facilities still have far fewer deaths. This could mean that higher-quality homes reduce mortality by doing a better job of managing COVID-19 cases, that is, they monitor patients closer, are more aggressive at seeking treatment, etc.

²¹: As of September 13^th, 2020, one-star homes reported significantly fewer staff cases, conditional on having a single staff case. By April 25^th, 2021, the staff cases were increasing with the star rating throughout the rating distribution. This somewhat unintuitive result is likely explained by higher-quality homes simply testing their staff more frequently and, thus, measuring more cases. We will provide evidence of this below.

²²: We note the possibility that facilities could have shortages of PPE because the staff is more aggressive at using this equipment. Moreover, as COVID-19 is primary spread through the air, gown, glove, and sanitizer shortages likely have little impact on disease transmission. At the same time, shortages of basic PPE may signal something about the quality of the home’s management and the home’s general adherence to COVID-19 protocols; hence, we present the results for these types of PPE as well.

²³: An interesting question in light of these findings is, “How much of the inverse relationship between COVID-19 deaths and nursing home quality is explained by higher-quality nursing homes avoiding staff and PPE shortages?” To answer this, we return to our baseline model, but add controls for shortages. In particular, among the nine staff and PPE measures, we calculate for each nursing home the number of shortages experienced between May 24^thand September 13^th of 2020 (e.g., if a home experienced a nursing shortage and an n95 mask shortage over this period, we would measure their total as 2). The results from this are in row 12 of Table 4. The results suggest that staff and PPE shortages can explain some of the quality gradient we report in Table 3. For example, in the basic model we estimate that five-star homes have 15.4 percent lower COVID-19 mortality than one-star homes. This number moves to 13.6 once we control for these shortages, meaning they can explain about 10 percent of the quality gradient.

²⁴: Testing data is only reported in the CMS data from August 16^th through November 22^nd of 2020.

²⁵: As nursing homes can only report such testing if a new positive case arises, we condition our analysis on facilities with a new positive case because higher-quality facilities have already been shown to have fewer positive cases.

²⁶: The data are from the Weekly Places Patterns data series, which SafeGraph makes available free of charge to researchers. The files contain hourly counts of foot traffic to about 4 million points of interest in the US. Traffic is monitored using cell phone location services. Locations are organized by NAICS code; nursing homes are code 623110. The vertical axis contains aggregate counts, adjusted for devices per person in the state, as is recommended by SafeGraph.

²⁷: To fix ideas, consider two nursing homes that differ only in overall quality. Each has ten fragile residents that, independent of COVID-19, would have died in 2020. Now, assume that five of the residents in the low-quality home die in early 2020 of COVID-19. In the data, we then observe ten non-COVID deaths in the high-quality home and just five such deaths in the low-quality home.

²⁸: In May and September of 2020 (rows 15 and 16), the relationship between nursing home quality and total deaths is positive, despite the fact that the impact of quality on COVID-19 deaths (rows 1 and 2) is larger in percentage terms than the impact of quality on non-COVID deaths (rows 8 and 9) in the same times periods. This peculiarity is explained by there simply being more non-COVID deaths than COVID-19 deaths. For example, consider September 13^th, 2020. Row 2 suggests that five-star homes had (0.154*3.672) 0.565 fewer COVID-19 deaths than one-star homes. Row 9 suggests that five-star homes had (0.114*10.151) 1.157 more non-COVID deaths than one-star homes.

²⁹: For this analysis, we use the “occupied beds” variable the CMS data to calculate the average weekly occupancy rate in each nursing home between May 24^th, 2020 and April 25^th, 2021. We then control for the occupancy rate in our April 25^th, 2021 total death regression (row 18 of Table 3). While the occupancy rate is positively associated with total deaths, the estimated quality effects are statistically indistinguishable from our main findings. (Results available upon request.) Note that we do not include the occupancy rate in all models because it is an imperfect (i.e., likely endogenous) control, as deaths in period t influence the occupancy rate in period t + 1.

³⁰: SafeGraph does not track the universe of cell phones; thus, the point of interest data can be sparse when facilities are small or located in rural environment. Among the 12,300 matched homes, over a thousand have zero visitors on more than a third of the days over our time-horizon. Thus, some of our specifications remove these 1000 homes with a high frequency of zero visitors, while others look only at homes with 100+ total beds. Both specifications remove measurement error, thereby improving the precision of our estimates.

³¹: In a memo released September 17^th outlining revised procedures for nursing homes during the pandemic, CMS notes that “…we recognize that physical separation from family and other loved ones has taken a physical and emotional toll on residents. Residents may feel socially isolated, leading to increased risk for depression, anxiety, and other expressions of distress. Residents living with cognitive impairment or other disabilities may find visitor restrictions and other ongoing changes related to COVID-19 confusing or upsetting” (CMS, 2020b). In this memo, CMS outlines policies for outdoor visitation and relaxed policies for indoor visitation in lower-risk settings such as counties with low positivity rates in the general population.

³²: Note that step 1 guarantees that COVID and non-COVID deaths are non-missing regressors in step 2. Furthermore, by imputing backwards, we guarantee that these regressors are non-missing in every week. Finally, note that the imputation takes place week-by-week to account for (i) various COVID-19 waves that have occurred over time and (ii) the fact that some homes report cumulative cases, while others report weekly cases in the first week of reporting, May 24^th, 2020.