Double or (Single!) shot of good news - Key Points from the Novavax & Janssen COVID-19 Vaccine Announcements
Last week certainly was an interesting one for COVID-19 vaccine developments, with the results from two more vaccine studies being announced. Here PHASTAR’s Vice President for Statistical Research and Consultancy, Jennifer Rogers, walks you through the key points.
Let’s start with the Novavax announcement which happened on January 28th. Results from the Phase 3 UK study reported a vaccine efficacy (VE) of 89.3%. This was based on 62 symptomatic COVID-19 cases at the interim analysis, where 56 cases of COVID-19 were observed in the placebo group versus 6 cases in the NVX-CoV2373 group. This study enrolled more than 15,000 participants between 18-84 years of age and included 27% over the age of 65. Those over the age of 65 are at a much higher risk of hospitalisation and death if they catch COVID-19 and so understanding the efficacy of potential vaccines in this age group remains of particular interest. According to the study protocol an effort was made to enrol a target of at least 25% of participants who were at least 65 years of age.
The protocol also states that the target number of COVID-19 cases of 152 was chosen to provide 90% power for a 60% vaccine efficacy. Two study interim analysis were planned to formally asses efficacy and futility based on the accumulation of approximately 43% (66 events) and 72% (110 events) of the total anticipated primary endpoints. The corresponding hypothesis tests for the primary endpoint were carried out against H0: VE ≤ 30%. This means that if the appropriately alpha-adjusted confidence interval (CI) lower bound of the estimated VE is greater than 30%, a statistically significant vaccine effect is demonstrated. For the first interim analysis, a VE efficacy boundary that would give a confidence interval lower bound of greater than 30% was calculated to be 75%, meaning that if the estimated VE was greater than 75%, this would correspond to the prespecified study success criterion. The point estimate of 89.3% therefore meets this criterion and we are told that the 95% confidence interval for VE was 75.2 – 95.4%. For the first interim analysis, a one-sided alpha of 0.00063 was considered, meaning that we should consider a 99.874% confidence interval for VE and note if the lower bound of this confidence interval is greater than 30%. We don’t know the exact number of people in each treatment arm, but we can assume 7,500 per group to get an idea of what a reasonable 99.874% for VE could be. If we compute an exact 95% confidence interval for VE, with 7,500 in each treatment arm, we obtain 77.0 – 95.5%, so in fact slightly more optimistic than those presented by Novavax, but not too far away. Considering a 99.874% confidence interval gives 63.1– 97.2%. We see that the lower bound of this confidence interval is very far away from the required 30%, which explains why Novavax has so much confidence in their vaccine after just the first interim analysis.
One of the most common questions that I am asked about the vaccines, is how effective they are against the new strains of COVID-19 that we are seeing. This is something that Novavax looked at in more detail, albeit as a post hoc analysis (who knew when these trials were being designed that we would have multiple new strains to contend with when the data was actually being analysed!). Novavax report that preliminary analysis of their data indicated that of the 62 COVID-19 cases detected, 32 were the UK variant, 24 were non-variant cases, and 6 were unknown. Further analysis suggested that VE was found to be 95.6% against the original COVID-19 strain and 85.6% against the UK variant strain. This is in line with what medical experts are telling us. It is unlikely that the vaccines that have been developed will be completely useless against the new strains of COVID-19. Where the new strains are found to have an effect on VE, it is likely that this will result in a dilution of VE, but that the vaccines will still offer some level of protection against them .
Just a day after the Novavax announcement, Janssen (the Pharmaceutical Companies of Johnson and Johnson) released the results from an interim analysis of their Phase 3 ENSEMBLE Trial, reporting 66% VE from 468 symptomatic cases of COVID-19. This press release doesn’t tell us exactly how many COVID-19 cases were observed in each of the two treatment arms, but using the formula for VE:
VE = 1 – risk ratio
We calculate that 119 cases in the investigational vaccine group (and 349 in the placebo group) would give a VE of 65.9%. Similarly, 118 cases in the investigational vaccine group (and 350 in the placebo group) would give a VE of 66.3%. So, it is likely that the true COVID-19 case numbers in each treatment arm is close to these values.
As in the Novavax study, the Janssen trial was powered to detect a VE of 60% with 90% power and the corresponding hypothesis tests for the primary endpoint were carried out against H0: VE ≤ 30%. There are very few statistical details in the protocol, but if we use our most conservative estimate of 119 cases in the investigational vaccine group and 349 in the placebo group, giving a VE of 65.9%. A 95% exact confidence interval on this is 58.1 – 72.3% and a 99% exact confidence interval is 55.3% – 74.0%. The lower bounds for both confidence intervals are again way above 30, indicating that the success criterion of this trial was also easily met.
Further results in this announcement stated that the investigational vaccine was found to be 85% effecting in preventing severe COVID-19 disease and that it also demonstrated complete protection against COVID-19 related hospitalisation and death. We are also told that protection was generally consistent across age and race subgroups, and across multiple COVID-19 variants, including the South Africa variant.
As we have seen with other studies (for example the Oxford AstraZeneca vaccine trial), enrolment to the ENSEMBLE Trial was done in stages, with younger individuals being enrolled first. Older participants were then enrolled, once the safety profile in the younger participants has been confirmed. This is why we sometimes see fewer older participants in these vaccine studies when we get to interim analyses and typically have to wait for more data to be collected before VE in older age groups can be properly examined. Janssen aimed to have those over 60 years of age be a minimum of approximately 30% of all participants enrolled and we are told that their study population of 43,783 participants included 13,610 (31%) who were over the age of 60.
One of the major advantages of the Janssen vaccine is that it is single dose, meaning that full protection is offered after the first vaccine and a booster vaccine is not required. There has been much discussion about follow-up vaccines in the UK, after our government changed their vaccination strategy at the end of 2020. Rather than focussing on administering both vaccine doses to individuals, the vaccine strategy has changed to ensure that as many people as possible are given their first dose of vaccine, with the booster shot coming up to 12 weeks later. The idea behind this shift in strategy is that it will allow the vaccine to be given to more of the general public much quicker. But there are doubts as to whether this strategy is grounded on scientific evidence.
The Pfizer BioNTech vaccination study did not look at different intervals between the two vaccine doses, but the Oxford AstraZeneca study did. This publication in The Lancet presenting the results from the Phase 3 study states that the COV002 (UK) study originally had single dose cohorts. But the protocol was modified in July 2020 to offer a second dose to the participants in the single dose cohorts “as a result of robust booster responses identified in the evaluation of early immunogenicity cohorts”. This meant that participants in the original single dose cohorts had longer gaps between prime and booster vaccines than those enrolled into two-dose groups from the start. Furthermore, the COV003 (Brazil) study offered all participants two doses of the vaccine with administration up to 12 weeks apart but a target of 4 weeks. This means that the Oxford AstraZeneca group were able to look at whether interval duration had an effect on VE. Their results showed that VE in the <6 weeks interval group was 53.4% (CI: −2.5 – 78.8%) and in the ≥6 weeks interval group it was 65.4% (CI: 41.1 – 79.6%), with a p-value for interaction of 0.557 indicating no evidence of an interaction and so no difference in VE for the two subgroups. This means that it is reasonable to administer first doses of the Oxford AstraZeneca vaccine to as many people as possible and follow-up with the booster doses up to 12 weeks later. And even yesterday the BBC published this article suggesting that there will be more results to come from the Oxford AstraZeneca group that support a longer interval between first and second vaccine doses. But there is no evidence to back-up the use of this strategy for the Pfizer BioNTech vaccine. There is no evidence that a longer gap between these vaccines is damaging either. It just hasn’t been examined yet. This fact has not been missed by my fellow statisticians. Professor Sheila Bird is a colleague of mine on the Royal Statistical Society COVID-19 Task Force and has written to Matt Hancock calling for a study to investigate the effect of extending the gap between the first and second doses of the Pfizer BioNTech vaccine. At the time of writing this, I believe she is still yet to receive a response.