The Effect of COVID-19 on the Estimands of Clinical Trials

In our latest blog of the series, we examine the topic of estimands with respect to clinical trials affected by the COVID-19 pandemic. When evaluating whether the primary objective of the trial is affected by the pandemic, it is necessary to consider the treatment effect in the absence of the COVID-19 pandemic and not confounded by pandemic-related events. This should be done via the estimands framework, looking at each of the five attributes of the estimand. It is critical to ensure that it is still possible to estimate the treatment effects aligned to the objectives and that the planned methods of analyses are still fully aligned to the estimands.

The Estimands Framework

Treatment Condition of Interest: This should remain the same as originally intended, although the pandemic may cause operational and logistical issues with study drug supply and delivery. Any pandemic-related issues with adherence to study drug and concomitant medications should be considered as intercurrent events.

Population: This should remain as originally intended to avoid deviating from the primary objective of the trial. However, consideration should be given to address the potential effect of COVID-19 vaccines, particularly in multi-region trials as COVID-19 vaccines became available at different times.

Variable: In most cases, the endpoint that is used to address the clinical question should remain as originally intended. If alternative data collection methods are required for the endpoint, it must be ensured that the endpoint is not compromised. All alternative data collection methods that are employed should be assessed for their potential effects on endpoint variability. Any different methods of collection that are used need to be comparable.

Population-Level Summary: this provides the basis for the comparison between treatments and should remain as originally intended. There may be rare cases where a population-level summary may need to be changed. For example, looking at COVID-19 exposed participants versus non-exposed participants or participants who have received a COVID-19 vaccine versus those who have not.

Intercurrent Events: Only pandemic-related intercurrent events should be considered. Non-pandemic-related intercurrent events should continue to be handled as originally planned. Pandemic-related intercurrent events should be categorised according to their impact on study treatment adherence (such as the event of discontinuation of study treatment) or the ability to assess the target outcomes (such as the event of death if death is not an expected outcome in the trial). The pandemic-related intercurrent events should then be categorised further according to pandemic-related factors (if there are problems with study drug supply, if a participant becomes infected with COVID-19 and if a participant receives concomitant medications for COVID-19 infection). There are different strategies for handling intercurrent events that may need to be considered:

  • Treatment policy strategy: under this strategy, whether an intercurrent event has occurred or not is considered irrelevant and the data is still analysed regardless, hence intercurrent events are considered irrelevant in defining the treatment effect. Since, under this strategy, the trial conclusions would not be generalisable post-pandemic, this strategy would not be an appropriate method for most pandemic-related intercurrent events for the majority of trials.
  • Composite variable strategy: under this strategy, the occurrence of an intercurrent event is incorporated into the definition of the endpoint. This is also unlikely to be an appropriate method for dealing with most pandemic-related intercurrent events.
  • Principal stratification strategy: for this strategy, the aim is to estimate the treatment effect within the stratum of participants for whom the intercurrent event did not occur. Stratifying on a pandemic-related event is unlikely to be appropriate since this limits the trial results and conclusions to a specific sub-population of participants applicable during the COVID-19 pandemic only, rather than a future post-pandemic perspective. Furthermore, it is not possible to define the stratum of participants in advance since it is impossible to know exactly which participants will have the intercurrent event.
  • While-on-treatment strategy: this strategy considers measurements of the endpoint up until the time of the intercurrent event. If the endpoint is measured at multiple time points during the trial, this strategy may still be relevant if it was originally planned for non-pandemic-related intercurrent events.
  • Hypothetical strategy: Under this strategy, the interest is in estimating the treatment effect in the hypothetical situation where the intercurrent event did not occur. This strategy is the most logical and the natural choice of method for handling most pandemic-related intercurrent events.

The different strategies highlight two key points. Firstly, that it is critical to collect additional data associated with pandemic-related factors in order to assess intercurrent events as pandemic- or non-pandemic related and to select appropriate strategies for handling them. Secondly, trials will handle an intercurrent event of death due to COVID-19 in different ways depending on the type of disease and the endpoint of the trial.