|Year : 2022 | Volume
| Issue : 1 | Page : 24-31
Evaluating the impact of social distancing on COVID-19 hospitalizations using interrupted time series regression
Alecia James, Rikki Malagón-Morris, Shari Gurusinghe, Patricia Roblin, Christina Bloem, Tyler Wise, Michael A Joseph, Bonnie Arquilla, Pia Daniel
SUNY Downstate Health Sciences University, Brooklyn, NY, USA
|Date of Submission||20-Sep-2020|
|Date of Acceptance||07-Nov-2021|
|Date of Web Publication||30-Mar-2022|
Ms. Alecia James
School of Public Health, 450 Clarkson Ave, Brooklyn NY, 11203
Source of Support: None, Conflict of Interest: None
Introduction: The quasi-experimental approach of interrupted time series analysis has been used to assess public health interventions by statistically comparing preintervention and postintervention rates. In this study, we apply interrupted time series to assess the effectiveness of social distancing on COVID-19 hospitalizations in a patient population in New York City.
Materials and Methods: An interrupted time series design was used to evaluate the impact of the New York State on PAUSE executive order (social distancing measure), on admitted COVID-19 patients, and patients on ventilators, at a single center hospital in Brooklyn, NY. Time series data were collected from March 10, 2020 to April 28, 2020 and were modeled using segmented regression analysis, assuming a 2-week delay in the intervention's effect. ARIMA forecasting was also performed to determine the projected COVID-19 hospitalizations and ventilator use in the absence of social distancing.
Results: There was a significant change (decrease) in the upward daily trend in the mean number of COVID-19 admissions and patients on ventilators after the assumed effective date of the New York State on PAUSE mandate. For admitted patients, the coefficient of the variable “time after intervention,” or change in slope, was − 9.30 (P = 0.0009), and the corresponding value was − 2.27 (P < 0.0001) for patients on ventilators.
Conclusion: The assumed effective period of the implementation of the New York State on PAUSE executive order was shown to be significantly correlated with decreased COVID-19 hospitalizations and ventilator use in the population measured. Similar social distancing measures should be adopted in other cities and locales that are currently seeing a surge in COVID-19 transmissions with an assumption of a 2-week delay in impact.
The following core competencies are addressed in this article: Medical knowledge, Systems-based practice.
Keywords: ARIMA modeling, interrupted time series, interrupted time series and COVID-19
|How to cite this article:|
James A, Malagón-Morris R, Gurusinghe S, Roblin P, Bloem C, Wise T, Joseph MA, Arquilla B, Daniel P. Evaluating the impact of social distancing on COVID-19 hospitalizations using interrupted time series regression. Int J Acad Med 2022;8:24-31
|How to cite this URL:|
James A, Malagón-Morris R, Gurusinghe S, Roblin P, Bloem C, Wise T, Joseph MA, Arquilla B, Daniel P. Evaluating the impact of social distancing on COVID-19 hospitalizations using interrupted time series regression. Int J Acad Med [serial online] 2022 [cited 2022 Jul 3];8:24-31. Available from: https://www.ijam-web.org/text.asp?2022/8/1/24/341182
| Introduction|| |
When New York City became the epicenter of the severe acute respiratory syndrome coronavirus-2 (SARS CoV-2), policymakers were compelled to devise strategies to control its rapid transmission. One statewide policy intervention was the New York State on PAUSE executive order, a social distancing measure, effective as of March 22, 2020.
Measuring the impact of public health interventions, such as social distancing, is difficult. Accurate assessment of social distancing is compounded by a temporal lag between the intervention and its impact, the interplay of other confounding events, and the impracticality of randomized controlled trials. Some researchers have held that an interrupted time series analysis is the “strongest quasi-experimental research design,” for large-scale interventions and serves as a great alternative when randomized control trials are not feasible or ethical., In interrupted time series, the intervention “interrupts” the time series trend, creating a preintervention and a postintervention period. Segmented regression is a robust statistical method that can be used to analyze interrupted time series data., It utilizes a piecewise regression approach to produce parameter estimates that define the preintervention and postintervention period.
In this study, to statistically assess how well the intervention worked, interrupted time series analysis utilizing segmented regression is used to produce mathematical estimates. We also use ARIMA time series modeling to create counterfactuals of patient admissions and patients on ventilators; this allows for a visual comparison with observed values. For this research, the New York State on PAUSE intervention was estimated to have an impact 2 weeks after it became effective (as of April 5th), with no substantive changes prior, to account for the maximum disease incubation period. We hypothesize that the New York State on PAUSE intervention will be associated with a reversal of the upward trend of patient hospitalizations and ventilator use at the center studied.
| Methods|| |
Data for this study were obtained from the electronic medical records of COVID-19-positive patients admitted at the University Hospital of Brooklyn from March 10, 2020 to April 28, 2020. The University Hospital of Brooklyn is a 249-bed independent hospital and the only academic medical center serving Brooklyn, New York. Approval was obtained from the SUNY Downstate Health Sciences University IRB; the requirement for informed consent was waived as this is a minimal risk research.
This study assesses the impact of the New York State on PAUSE executive order, a social distancing measure implemented by Governor Cuomo, as of March 22, 2020, to control the spread of (SARS CoV-2). This intervention was estimated to have an assumed effective date of April 5th to account for the maximum disease incubation period. Its regulations include the closure of all nonessential businesses and temporary restrictions of nonessential gatherings. A subpolicy implemented was Matilda's Law that placed additional restrictions on vulnerable populations such as older individuals, the immunocompromised, and those with underlying health conditions. This order was directed to everyone residing in the state of New York (the intervention was in effect throughout the end of the study period).
The objective of the study is to apply interrupted time series to assess the effectiveness of social distancing on COVID-19 hospitalizations in a patient population in New York City.
The outcomes measured were patient admissions and patient-ventilator use following the assumed effective date of the intervention.
There was no determination made for sample size in this quasi-experimental study. Time series data – number of patients corresponding to a particular date – were obtained from patient electronic medical records.
All hospitalized COVID patients and patients on ventilators at the University Hospital of Brooklyn from March 10, 2020 to April 28, 2020 were eligible for participation in the study. The unit of assignment is individual.
Blinding is not applicable to this study since the intervention is a policy intervention made on a statewide (population) level; the data used for analysis were retrieved from patient electronic medical records after the study period.
Unit of analysis
Analysis was done using individual-level data by considering the total number of patient admissions and patients on ventilators for each date that time series data were used.
Interrupted time series
This study used the method of segmented regression analysis to model the time series data and assess the impact of the intervention. The time series data were dated from March 10th to April 28th, corresponding with the study duration. The New York State on PAUSE executive order can be seen as an intervention that essentially interrupts or “splits the time series,” creating a preintervention and a postintervention segment/period as shown on [Figure 1]a (for patients on ventilators) and 1b (for confirmed hospitalized patients). In this study, there is a predicted 2-week time frame before the intervention became effective. The effective intervention date is indicated on [Figure 1]a and [Figure 1]b as the April 5th mark. Each segment of the time series is estimated by a linear regression line that is defined by a level and a trend/slope., The level is the value of the time series at the beginning of a segment or period (intercept), while the trend or slope refers to the rate of change of the dependent variable (confirmed hospitalized patients and patients on ventilators). To evaluate the effect of an intervention, the changes in the level or trend of the number of hospitalized patients and patients on ventilators after the intervention are studied. The segmented regression analysis procedure statistically modeled these changes and determined their significance.
|Figure 1: (a) Segmented regression analysis showing an upwardly trending preintervention slope and a negative postintervention slope for patients on ventilators. The postintervention period is shaded in gray. (b) Segmented Regression analysis showing an upwardly trending preintervention slope and a negative postintervention slope for the confirmed hospitalized patients. The postintervention period is shaded in gray|
Click here to view
Statistical analysis of interrupted time series data
The time series data were analyzed using a stepwise autoregression method. The stepwise autoregression was done by the process of backward elimination which removed nonsignificant autoregressive parameters leaving only significant autoregressive terms. For this study, we report the parameter estimates of the final model from the “Parameter Estimates with Autoregressive Parameters Assumed Given” table generated [Table 1]a and [Table 1]b. A P < 0.05 (two-sided) was considered as statistically significant. SAS® 9.4 software (SAS Institute Inc., Cary, NC, USA) was used to perform the statistical analyses for the regression.
ARIMA time series modeling
Another method used to estimate the effect of an intervention is to compare the values of a time series data after an intervention, to a counterfactual– forecasted values of the dependent variable assuming no intervention. ARIMA time series modeling is a widely used method to produce forecasted values of a variable based on past values of that variable. Visually assessing a counterfactual against actual values of a time series trend helps to situate the impact of the intervention that “interrupted” the time series.
Thus, to further assess the impact of the New York State on PAUSE executive order, we used ARIMA (Autoregressive Integrated Moving Average) forecasting to create projections of the number of patients on ventilators, and the number of hospitalized patients (assuming there was no intervention implemented). An ARIMA model is defined by ARIMA (p, d, q) where P represents the autoregressive order, d, the order of differencing, and where q specifies the order of the moving average. This procedure uses past values of the dependent variable (in this study: From March 10th– April 4th) to forecast future values (April 5th– April 28th), representing a “lead” option of 24 days. We used these dates as our study assumed no substantive changes before April 5th, based on the predicted 2-week delay in the intervention's effect. An ARIMA (0, 1, 0) model was specified for both analyses, with a “back” option of 0, and an alpha of 0.05 indicated. To achieve stationarity, we used a differencing order of 1, as indicated in the model. The default number of lags used in this procedure was 24, and the maximum likelihood method was used to estimate the parameters. The final models had no significant autocorrelations in the residuals and had the best fit based on smaller AIC and SBC scores. The ARIMA projections are shown in [Table 2]a and [Table 2]b and are graphically displayed in [Figure 2]a and [Figure 2]b. SAS University Edition software, Version 9.4 M6, was used for the forecasting.
|Figure 2: (a) ARIMA forecasting demonstrating the effect of the New York State on Pause intervention on ventilator utilization. Patients on ventilators were measured before (red) and after (blue) the expected date of effect of the intervention. (b) ARIMA forecasting showing a correlation between the intervention of social distancing and a change in the rate of COVID inpatient census at the University Hospital of Brooklyn. Admitted patients were measured before (red) and after (blue) the expected date of effect of the intervention|
Click here to view
| Results|| |
This is not applicable to the current study. Daily counts for admitted patients and patients on ventilators were obtained through retrospective chart review.
This is not applicable to the current study since the method of choosing participants is based on retrospective review of chart records of hospitalized patients and patients on ventilators during the study period.
Demographic and clinical characteristics were not noted for the specific patients involved in the daily time series data. However, of the overall COVID-19 patients hospitalized at the University Hospital of Brooklyn for the study period, roughly 98% were from Brooklyn, NY; 91% were Black; 80% had two or more comorbidities, and the median age was 70.
No comparison groups were used.
Outcomes and estimation
The assumed effective date of the New York State on PAUSE executive order was April 5th, which accounted for the maximum disease incubation period. On [Figure 1]a and [Figure 1]b, the April 5th mark separates the preintervention and the postintervention period. Based on the study's assumptions, to assess the impact of the New York State on PAUSE intervention, we will mainly examine the postintervention slopes on [Figure 1]a and [Figure 1]b, as well as the coefficients of the “time after intervention” variable on [Table 1]a and [Table 1]b. The estimate or coefficient of the “time after intervention” variable represents the change in the trend/slope of the mean number of patients on ventilators and confirmed hospitalized patients, after the intervention, compared to the change in the trend of those numbers before the intervention.
The negative/downward trend of the postintervention slopes in both [Figure 1]a and [Figure 1b is evidence that we are no longer observing an increase in the number of admissions and ventilator patients; rather, we can observe that this trend has shifted in the opposite direction. To determine statistical significance, we will closely analyze the P values of the parameter estimates. The coefficient of the variable “time after intervention” is − 2.27 (P < 0.0001) for patients on ventilators and − 9.30 (P = 0.0009) for patient admissions. These P values and estimates indicate that there was a significant change in the slopes of the number of patients on ventilators, and hospitalized patients, after the assumed date that the social distancing measure became effective. These results suggest that the intervention was correlated with a change in the course of events.
The results of the ARIMA forecasting are displayed in [Table 2]a and [Table 2]b and are graphically represented in [Figure 2]a and [Figure 2]b. The actual number of patients will be compared with the forecasted to estimate the effect of the New York State on PAUSE intervention. [Figure 2]a depicts [Figure 2]a continued upward trend in the projected number of patients on ventilators, assuming that there was no intervention. The forecasted values for patients on ventilators ranged from 44.32 (95 confidence interval [CI]: [35.43–53.21]) to 74.68 (95 CI [31.14–118.22]) from April 5th to April 28th, while the actual number of patients during this timeframe (corresponding to the effective period of the intervention) decreased from 46 to 33 [Table 2]a and [Figure 2]a.
The data for admitted patients follows a similar pattern. On April 28th, the last date of the time series, the forecasted number of admitted patients was 313.6 (95 CI: [161.63–465.57]) [Table 2]b. However, with the intervention in place, the actual number of patients for this date was 91. [Figure 2]b highlights this sharp contrast in the actual number of admitted patients, versus the forecasted number. The green line indicates the expected trend in the number of admitted patients if there was no policy intervention.
This is not applicable to the current study.
This is not applicable to the current study.
| Discussion|| |
During a pandemic, assessing the impact of strategies to alleviate the strain placed on health-care systems is necessary to guide an effective response. Evidence-based practices that reduce community-based transmission will ultimately prevent the overwhelming of hospital resources.
For this research, the New York State on PAUSE intervention was estimated to have an impact 2 weeks after it became effective (as of April 5th), with no substantive changes prior, to account for the maximum disease incubation period. Based on publicly reported confirmed cases, the median incubation period to fever onset is said to be “5.7 days (CI, 4.9–6.8 days), with 2.5% of persons experiencing fever within 2.6 days (CI, 2.1–3.7 days) and 97.5% having fever within 12.5 days (CI, 8.2–17.7 days) of exposure.”
Our segmented regression analysis of the interrupted time series data demonstrated that the implementation of Governor Cuomo's New York State on PAUSE social distancing measure was correlated with a significant reduction in invasive ventilator use and the number of COVID-19 patient admissions at the University Hospital of Brooklyn. Further, the counterfactuals created by the time series modeling helped to paint the picture of the estimated burden that would have been encountered if there was no intervention.
Social distancing is an intervention strategy that has been implemented both locally and abroad, with discussion regarding the timing of impact. Our study assumes a 2-week delay in the impact of social distancing on community transmission of COVID-19. The choice to use a 2-week time interval is based upon the incubation period typical of COVID-19. In Courtemanche et al.'s recent study, they found that adherence to social distancing policies in U.S. counties resulted in a reduction of the COVID-19 daily case growth rate by 5.4 percentage points after 1–5 days and up to 9.1 after 16–20 days. Further, after controlling for voluntary social distancing, their study unfolded an eventual 10 times greater spread by April 27th in the absence of shelter-in-place orders. Our findings mirror this pattern, in that we experienced a decrease in the COVID-19 case burden in the postintervention period. In contrast, our counterfactuals depicted an upward trend in the COVID-19 case burden at the University Hospital of Brooklyn.
In Ngonghala et al.'s research, they used mathematical assessments to examine the impact of nonpharmacological interventions, including social distancing, to control the spread of the novel coronavirus in New York and the overall U.S. They projected a dramatic decrease in COVID-19 morbidity and mortality if strict social distancing measures are implemented and observed. Based on their models at the time of their study, they explained that a “highly effective social distancing strategy” that can reduce the baseline contact rate by at least 40% will decrease mortalities by 80% and 64%, in New York and the general U.S., respectively. While the current study utilized a segmented regression analysis model to estimate the effect of the social distancing intervention, Ngonghala et al. designed a Kermack-McKendrick-type mathematical model to analyze the transmission dynamics and control of COVID-19. However, the results of both models still suggest a correlation between social distancing and a decrease in COVID-19 case burden.
Internationally, early studies out of South Korea have examined the transmission potential and severity of COVID-19. Shim et al.'s findings suggested a “sustained disease transmission” in the region, which underscored the importance of instituting strategic social distancing policies to contain the disease burden in South Korea.
In Zhang et al.'s study of contact patterns and the dynamics of COVID-19 in China, they built a transmission model to investigate the impact of social distancing and school closures on the transmission of the virus in Wuhan and Shanghai. They found that the implementation of social distancing, by itself, was enough to control transmission in these cities. Since the earliest COVID-19 cases were reported in Wuhan, China, it follows that their successful containment strategies would serve as a model for other areas experiencing this pandemic.
Vokó and Pitter used an interrupted time series design to study the effects of social distancing measures on the COVID-19 pandemic in Europe. They reported a decline in the disease following national policy interventions such as stay-at-home orders. In their analyses of data from 28 European countries, these researchers found that before their detected change points, there was a mean growth in the incidence of new COVID-19 cases by 24% per day. In contrast, after the change point, there was a decline in the growth rate to “0.9%, 0.3% increase, and to 0.7% and 1.7% decrease by increasing social distancing quartiles.” They summarized this finding as a “dose-response association of the observed flattening of the epidemic curve with increasing social distancing” This suggests that more aggressive social distance measures may be required to contain the spread of COVID-19 in areas that are currently experiencing a surge in cases such as some southern states in the U.S.
The importance of curtailing the burden of disease cannot be overstated. When New York City was the epicenter of the COVID-19 pandemic, there was an overwhelming outcry to increase resources such as hospital beds, the health-care workforce, and significantly, the supply of ventilators to handle the exponential growth in COVID-19 cases. Ranney et al.'s article captured the intensity of this period and also highlighted the dire need for critical supplies including ventilators and PPE during this time. Significantly, Rocklöv has found that with scenarios like the transmission of COVID-19, the contact rate is proportional to population density. Thus, it follows naturally that proven strategies – such as social distancing – to curtail the proximity of individuals would be effective in controlling the spread of the virus and subsequently address the burden of COVID-19.
The effects of COVID-19 touch every aspect of our society, and the burden on the health-care system is severe. In the absence of vaccinations and large-scale pharmacological strategies, it is critical to adhere to density reduction policies during a respiratory pandemic., As New York City enters the recovery phase, it is essential for communities to utilize evidence-based practices, such as continued social distancing, proper hand hygiene, and the use of masks, as necessary, to curtail the progression of COVID-19.
This study is subject to several limitations. Data for this study were obtained by the process of manual chart review which is subject to retrieval errors. Another limitation is that interrupted time series analysis can show correlation, but not causation of an effect. Further studies are needed to more accurately define the impact and optimal timing of social distancing practices and to support causal inferences. This study is further limited in that its design uses population-level rates – as it relates to the social distancing intervention– so the results may be prone to ecological fallacy. In addition, there is a requirement for data collection at equally spaced intervals to use segmented regression analysis, and a minimum number (at least eight to twelve) of preintervention and postintervention points are required to use this procedure. Thus, this type of analysis may not work if these requirements are not met. Despite these limitations, our use of interrupted time series – a robust quasi-experimental approach for evaluating the impact of program/health interventions – was successful in demonstrating the effectiveness of the New York State on PAUSE social distancing measure. Further, to our knowledge, this is the first study that has utilized ARIMA time series modeling to create forecasts of COVID-19 patient admissions, and patients on ventilators in the New York region. This methodology provides a useful approach for visualizing and grasping the intensity of the disease burden and the effect of the social distancing intervention.
These findings may not be generalizable to all settings given other methods apart from social distancing (such as mask-wearing) that have now become widely used as a containment strategy.
| Conclusion|| |
Interrupted time series design is an effective approach for analyzing policy interventions; our use of segmented regression allowed the statistical assessment of the impact of the New York State on PAUSE social distancing measure by comparing the preintervention to the postintervention time series. The implementation of the New York State on PAUSE mandate was associated with a statistically significant change in the positive/upward trend of admitted patients and patients on ventilators at the University Hospital of Brooklyn after a delay of 2 weeks. Similar public health intervention strategies are recommended in other areas that are currently experiencing an increase in the number of COVID-19 cases. Moreover, as one of the hardest-hit epicenters of COVID-19 to date, adopting measures that have been proven effective in New York City is strongly recommended to curtail the burden of COVID-19 in other areas.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
Ethical conduct of research
The authors of this manuscript declare that this scientific work complies with reporting quality, formatting, and reproducibility guidelines set forth by the EQUATOR Network. The authors also attest that this clinical investigation was determined to require Institutional Review Board/Ethics Committee review, and the corresponding protocol/approval number is 11521.
| References|| |
Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr 2013;13:S38-44.
Taljaard M, McKenzie JE, Ramsay CR, Grimshaw JM. The use of segmented regression in analysing interrupted time series studies: An example in pre-hospital ambulance care. Implement Sci 2014;9:77.
Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: A tutorial. Int J Epidemiol 2017;46:348-55.
Devkaran S, O'Farrell PN. The impact of hospital accreditation on quality measures: An interrupted time series analysis. BMC Health Serv Res 2015;15:137.
Slavova S, Costich JF, Luu H, Fields J, Gabella BA, Tarima S, et al
. Interrupted time series design to evaluate the effect of the ICD-9-CM to ICD-10-CM coding transition on injury hospitalization trends. Inj Epidemiol 2018;5:36.
Singh RK, Rani M, Bhagavathula AS, Sah R, Rodriguez-Morales AJ, Kalita H, et al
. Prediction of the COVID-19 pandemic for the top 15 affected countries: Advanced Autoregressive Integrated Moving Average (ARIMA) model. JMIR Public Health Surveill 2020;6:e19115.
Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR, et al
. The incubation period of coronavirus disease 2019 (COVID-19) from publicly reported confirmed cases: Estimation and application. Ann Intern Med 2020;172:577-82.
Courtemanche C, Garuccio J, Le A, Pinkston J, Yelowitz A. Strong social distancing measures in the United States reduced the COVID-19 growth rate. Health Aff (Millwood) 2020;39:1237-46.
Ngonghala CN, Iboi E, Eikenberry S, Scotch M, MacIntyre CR, Bonds MH, et al
. Mathematical assessment of the impact of non-pharmaceutical interventions on curtailing the 2019 novel coronavirus. Math Biosci 2020;325:108364.
Shim E, Tariq A, Choi W, Lee Y, Chowell G. Transmission potential and severity of COVID-19 in South Korea. Int J Infect Dis 2020;93:339-44.
Zhang J, Litvinova M, Liang Y, Wang Y, Wang W, Zhao S, et al
. Changes in contact patterns shape the dynamics of the COVID-19 outbreak in China. Science 2020;368:1481-6.
Vokó Z, Pitter JG. The effect of social distance measures on COVID-19 epidemics in Europe: An interrupted time series analysis. Geroscience 2020;42:1075-82.
Ranney ML, Griffeth V, Jha AK. Critical supply shortages – The need for ventilators and personal protective equipment during the COVID-19 pandemic. N Engl J Med 2020;382:e41.
Rocklöv J, Sjödin H. High population densities catalyse the spread of COVID-19. J Travel Med 2020;27:taaa038.
Giordano G, Blanchini F, Bruno R, Colaneri P, Di Filippo A, Di Matteo A, et al
. Modelling the COVID-19 epidemic and implementation of population-wide interventions in Italy. Nat Med 2020;26:855-60.
[Figure 1], [Figure 2]
[Table 1], [Table 2]