95% or 29% – the Pfizer vaccine – how effective is it actually?

Here I outline new concerns about the trustworthiness and meaningfulness of the reported efficacy results.

“Suspected covid-19”

All attention has focused on the dramatic efficacy results: Pfizer reported 170 PCR confirmed covid-19 cases, split 8 to 162 between vaccine and placebo groups. But these numbers were dwarfed by a category of disease called “suspected covid-19”—those with symptomatic covid-19 that were not PCR confirmed. According to FDA’s report on Pfizer’s vaccine, there were “3410 total cases of suspected, but unconfirmed covid-19 in the overall study population, 1594 occurred in the vaccine group vs. 1816 in the placebo group.”

With 20 times more suspected than confirmed cases, this category of disease cannot be ignored simply because there was no positive PCR test result. Indeed this makes it all the more urgent to understand. A rough estimate of vaccine efficacy against developing covid-19 symptoms, with or without a positive PCR test result, would be a relative risk reduction of 19% —far below the 50% effectiveness threshold for vaccine authorization set by regulators.

Even after removing cases occurring within 7 days of vaccination (409 on Pfizer’s vaccine vs. 287 on placebo), which should include the majority of symptoms due to short-term vaccine reactogenicity, vaccine efficacy remains low: 29% and clearly NOT the 95% claimed.

If many or most of these suspected cases were in people who had a false negative PCR test result, this would dramatically decrease vaccine efficacy. But considering that influenza-like illnesses have always had myriad causes—

• rhinoviruses,
• influenza viruses,
• other coronaviruses,
• adenoviruses,
• respiratory syncytial virus, etc.

—some or many of the suspected covid-19 cases may be due to a different causative agent.

But why should the actual cause of the ‘suspected covid-19 cases’ matter? If those experiencing “suspected covid-19” had essentially the same clinical course as confirmed covid-19, then “suspected plus confirmed covid-19” may be a more clinically meaningful endpoint than just confirmed covid-19.

However, if confirmed covid-19 is on average more severe than suspected covid-19, we must still keep in mind that at the end of the day, it is not average clinical severity that matters, it’s the incidence of severe disease that affects hospital admissions. With 20 times more suspected covid-19 than confirmed covid-19, and trials not designed to assess whether the vaccines can interrupt viral transmission, an analysis of severe disease irrespective of etiologic agent—namely, rates of hospitalizations, ICU cases, and deaths amongst trial participants—seems warranted, and is the only way to assess the vaccines’ real ability to take the edge off the pandemic.

There is a clear need for data to answer these questions, but Pfizer’s 92-page report didn’t mention the 3410 “suspected covid-19” cases. Nor did its publication in the New England Journal of Medicine. Nor did any of the reports on Moderna’s vaccine. The only source that appears to have reported it is FDA’s review of Pfizer’s vaccine.

The 371 individuals excluded from Pfizer vaccine efficacy analysis

Another reason we need more data is to analyse an unexplained detail found in a table of FDA’s review of Pfizer’s vaccine: 371 individuals excluded from the efficacy analysis for “important protocol deviations on or prior to 7 days after Dose 2.”  What is concerning is the imbalance between randomized groups in the number of these excluded individuals:

311 from the vaccine group vs 60 on placebo.

(In contrast, in Moderna’s trial, there were just 36 participants excluded from the efficacy analysis for “major protocol deviation”—12 vaccine group vs 24 placebo group.)

What were these protocol deviations in Pfizer’s study, and why were there five times more participants excluded in the vaccine group?  The FDA report doesn’t say, and these exclusions are difficult to even spot in Pfizer’s report and journal publication.

We need the raw data

Addressing the many open questions about these trials requires access to the raw trial data. But no company seems to have shared data with any third party at this point.

Pfizer says it is making data available “upon request, and subject to review.” This stops far short of making data publicly available, but at least leaves the door open. How open is unclear, since the study protocol says Pfizer will only start making data available 24 months after study completion.

Moderna’s data sharing statement states data “may be available upon request once the trial is complete.” This translates to sometime in mid-to-late 2022, as follow-up is planned for 2 years.

Things may be no different for the Oxford/AstraZeneca vaccine which has pledged patient-level data “when the trial is complete.” And the ClinicalTrials.gov entry for the Russian Sputnik V vaccine says there are no plans to share individual participant data.

The European Medicines Agency and Health Canada, however, may share data for any authorized vaccines much earlier.  EMA has already pledged to publish the data submitted by Pfizer on its website “in due course,” as has Health Canada.