Rumination on Vaccination - A Statistician's View of the Pfizer COVID-19 Vaccine Data

A “great day for science and humanity”

Pfizer and BioNTech have today announced results from the first interim analysis from their Phase III study of a vaccine candidate against COVID-19 and have concluded over 90% vaccine effectiveness. Do these results mean that there is finally some light at the end of the very long COVID-19 tunnel? Here’s what we know so far…

The analysis evaluated 94 confirmed cases of COVID-19 in trial participants. According to the trial protocol, Pfizer and BioNTech planned to take 5 looks at the data, with planned interim analyses at 32, 64, 92, 120 COVID-19 cases, and the final analysis at 164. There had recently been a decision made to drop the 32-case interim analysis and conduct the first interim analysis at a minimum of 62 cases. Upon conclusion of these discussions, the case count had reached 94 and the first interim analysis was carried out.

It has been reported that the case split between vaccinated individuals and placebo controls indicated a vaccine efficacy rate above 90%. What can we conclude from this about the numbers in the case split? Vaccine efficacy/effectiveness (VE) is measured by calculating the risk of disease among vaccinated persons and placebo controls and determining the percentage reduction in risk of disease among vaccinated persons relative to unvaccinated persons. It is calculated as:


or

 

If we consider the scenario of 86 COVID-19 cases in the 8 cases in the vaccinated group, this gives a VE of 90.7%. It turns out that with 94 people, 8 COVID-19 cases occurring in the vaccinated group is the worst-case scenario that would give a VE greater than 90%. According to the trial protocol, the stopping boundary for efficacy at 92 cases is 62.7%, so we see that a VE of over 90% suggests overwhelming evidence of efficacy.

We can consider these numbers further and calculate the COVID-19 case rates in both the vaccine group and placebo controls. We are told that  43,538 participants have been enrolled to date (as of 8th November) and the protocol states that randomisation is 1:1, meaning that we would expect approximately 21,769 in each treatment group. Assuming this means that the estimated COVID-19 case rate is 0.037% in the vaccine group and 0.395% among the placebo controls. A two-tailed test of equality of these proportions yields a p-value of less than 0.00001, which is overwhelming evidence of efficacy.

A VE of 90% indicates a 90% reduction in disease occurrence among the vaccine group, and the statistics seem to suggest that this result is highly statistically significant. But we’re not quite there yet. Pfizer and BioNTech are continuing to accumulate safety data, but no serious safety concerns have been observed to date. It is expected that there is a submission for Emergency Use Authorization to the U.S. Food and Drug Administration planned for soon after the required safety milestone is achieved, which is currently expected to occur in the third week of November.

The results that we have seen today seem to encompass all COVID-19 cases, from mild to severe, but it would be interesting to see VE broken down by virus severity. It would also be interesting to see if there is an interaction between VE and age given the variety in COVID-19 risk for different age groups. Finally, it will be interesting to see completion rates considering that the vaccination schedule is two doses, given 21 days apart.

We are extremely excited to see more of this data as it becomes available, but for now maybe, just maybe, the world has taken a monumental leap forward in the fight against COVID-19.

* Note that protection is achieved 28 days after the initiation of the vaccination, which consists of a 2-dose schedule