View from the Summit: Prof Thomas Jaki and the challenges of designing clinical trials during a pandemic

PHASTAR was delighted to welcome Prof. Thomas Jaki to the opening plenary session of PHASTAR’s Life Science Summit. Conducting clinical trials in a novel disease during a pandemic presents unique challenges and this presentation highlighted Prof. Jaki’s personal experiences from being involved in the current COVID-19 outbreakHere we present a summary of Prof. Jaki’s presentation and put your questions to him that he was unable to answer during the session. 

His talk kicked off by summarizing RCT experience during the 2014 Ebola epidemic in West Africa, which set the stage for fast RCT design and implementation during the current pandemic.

The first trial he described, the LOTUS trial, was a remarkable achievement in that it started so quickly after the outbreak began in Wuhan, China. This was an open-label trial without a formal design to evaluate a regimen of ritonavir/lopinavir using a 7-point ordinal scale endpoint. Without looking at the data, the trial endpoint was modified during the trial to time clinical improvement, defined as the time from randomization to either an improvement of 2 points or discharge from hospital. 

Prof. Jaki next discussed an early trial evaluating remdesivir in mild/moderate and severe cases using a group sequential design. The trial was designed around a primary endpoint of a 2-point improvement on 6-point ordinal scale; however, due to the waning of COVID-19 cases in Wuhan, China the trial was prematurely stopped resulting in too few data to draw conclusions.  

Clinical trials during a pandemic should be speedy, adaptive, efficiently discard useless treatments, study multiple treatments, and minimize the burden on frontline clinical staff. With this in mind, Prof. Jaki also spoke about the ongoing RECOVERY trial, which is a multi-platform trial to evaluate 4 active treatments on day 28 mortality with streamline data collection. Thus far, the trial has enrolled approximately 11,500 patients in 3 months’, with 175 clinical sites enrolling all age groups. Recently three treatment arms were dropped: dexamethasone due to early findings of efficacy in high risk subgroups and treatment arms lopinavir/ritonavir and hydroxychloroquine due to absence of effect.  

Prof. Jaki then spent time discussing the challenges in choosing an appropriate endpoint and the trade-off between using an ordinal endpoint and time to event outcomes. Difficulties with ordinal outcomes include choosing the appropriate time point over which to measure the response and difficulties with interpretation. Challenges when using a time to clinical improvement include deciding how to handle deaths and less power (over ordinal outcomes).  

Finally, Prof. Jaki introduced the design of the AGILE trial, which is a platform trial designed around a time to improvement endpoint in hospitalized patients and time to negative swab in patients not hospitalized. This early phase platform will look closely at dose, safety, and evidence of an effect. The first cohort is to evaluate starting treatment dose followed by dose escalation based on safety assessment.  

Prof. Jaki concluded by stating how RCT remain the gold standard even during a pandemic but that such trials should be adaptive with a focus on re-purposing drugs, dropping ineffective treatments early and trade off in choosing the appropriate endpoint.

Questions submitted to Prof. Jaki 

Q: I would be interested to hear about regulatory interactions in getting the trial set up so quickly? 

A: Our experiences where very different in different countries. In the early trials in China, there was more hesitance to embrace more novel methods, while the interactions with MHRA have been incredibly useful and effective. In either case, decisions and interactions were very quick - in 2 days or less we had feedback/decisions in most cases.  

Q: Including too may treatments in multi-arm trials would mean that treatments could have been dropped (either for efficacy or futility) as quickly as they have in the RECOVERY trial. What was the process behind deciding which treatments to include?  

A: In RECOVERY some very pragmatic rules have been applied. There needed to be scientific rationale plus sufficient supply for a large trials (i.e. thousands) and it needed to be scalable. 

Q: There have been reports that some people have reported long term damage after recovery from COVID-19. Once a person is classed as recovered, has there been any follow-up to see if there are any differences in more longer-term damage (such as lung function)?  

A: This is not planned in the trials in Wuhan and am not sure if it is with the NIH trials. For RECOVERY, there are (loose) plans to do so using data linkage.  

Q: Did you consider an adaptive approach in which you evaluated the ordinal endpoint at an early interim stage at various time points to refine the timing for the final analyses?  

A: We have not actively looked at such an approach. The main reason is that it would be difficult to do without unblinding and that would open up criticisms relating to data dredging. 

Q: In a pandemic like COVID-19 timing of the trial is of the essence. Too late, and the trial will not be useful and recruit enough. Yet, too early and we run a risk of mis-designing it and not finding a useful answer either. Should planning to be adaptive be the middle ground we need to design trials in this context?  

A: I think the risks are not symmetric in this case. Starting too early is less risky in my opinion than starting too late. If you start early, then your time to make adaptations that are not informed by the data in the study are longer and so unless you unblind the data too early, many things can be rectified. 

Q: Even starting as early as it did, it did not manage to recruit enough. Would a more adaptive version of that trial have done better?  

A: The first trial was stopped for reasons other than recruitment (namely prioritizing other treatments). Had this change not been made, it would have recruited sufficiently. Having said that, an adaptive design (in particular early stopping) would have been preferred.  

Q: Do you think it will be possible to "rigorously" compare Dexamethasone with Tocilizumab and plasma, given Dexamethasone has been stopped?  

A: The trial has not been powered for such a comparison and hence it will always be challenging to do so in this trial. There are, however, several hundred patients in each arm so that some learning can occur.  

Q: Does the severity of the disease in the population or the clinical practice change over time?  

A: I have not seen any evidence that the severity is changing, but the practice is definitely evolving