Covid19 vaccines: Adverse effects & Uncertainty

December 5, 2021. Updated December 15, 2021.
Zürich, Switzerland

A statistical proof why the Covid19 vaccine could cause myocarditis in 1 out of 524 vaccinated children, and this still not be detected in clinical trials.

“We're never going to learn about how safe this vaccine is unless we start giving it”, said Dr. Eric Rubin in a recent FDA hearing, “That’s just the way it goes. That's how we found out about rare complications of other vaccines like the rotavirus vaccine. And I do think we should vote to approve it”.

The hearing discussed whether the Pfizer-BioNTech Covid19-vaccine should be approved for children 5–11 years old. Dr. Rubin is an adjunct professor of immunology and a voting-member of the USA Food and Drug Administration (FDA). The members voted to approve the vaccine for emergency use.

Dr. Rubin is right; there are complications and side effects that almost certainly won't be discovered in clinical trials.

This is not a reference to reports of improperly conducted clinical trials by Pfizer, such as the recent whistleblower warning about falsified data, or the Project Veritas exposé indicating a company culture that may not hold scientific integrity as its top priority. Those claims should be taken seriously and investigated, but they're not the focus of this article.

The topic of this article is instead the inherent limitation of how certain we can be about the rates of adverse effects, given how a particular study is designed. An uncertainty is always present, even with perfectly conducted clinical trials. It's an inherent mathematical limitation — the statistical improbability of clinical trials discovering infrequent adverse effects.

In this article we will:

Intuitively describe why the inherent limitation exists.
Show a formula to quantify it mathematically.
Compute the potential risks of the vaccine.
Assess the risks of myocarditis and death from Covid19.
Compare the potential risks of the vaccine to Covid19.

We will restrict ourselves to children aged 5–11. At the time of writing, it is the most interesting age group, as their worldwide mass-vaccination is just about to start.

Conceptually, the analysis can be done for any age group. However, vaccination for other age groups is already underway, and we have real-world data of adverse effects in databases such as VAERS and EudraVigilance. Consequently, it's not as relevant to look solely at the clinical trials.

Intuition: Tossing a coin

To get an intuitive understanding of the problem, let's consider a coin. The coin has two sides, and when tossed it's guaranteed to land either heads or tails up. We don't know whether the coin is fair, a little bit unfair, or very unfair.

Let's suppose a gambler offers you a bet. If you flip the coin, and it lands heads up, you have to pay the gambler €1000. If it lands tails up, the gambler has to pay you €1000. Is it a good bet?

Since we don't know how unfair the coin is, it's hard to tell. The fact that a gambler has suggested a bet makes us suspicious. We ask the gambler if we can test the coin before accepting or rejecting the bet. The gambler agrees.

We toss the coin once; it lands tails up. Can we now conclude that the probability of tails is 100%, and the bet is fantastic? Of course not, we need more tosses. After 10 tosses, the score is 4 heads–6 tails. If we divide tails by the total number of tosses, tails appears about 60% probable. The bet looks good. After 10'000 coin-tosses the score is 5101 heads–4899 tails; 51% probability of heads up. The bet is not in our favor after all, so we reject it.

Naturally, the more coin tosses the more confident we can be in our probability-estimate of heads. With only one toss we can't say much, after 10 we get a rough idea, and after 10'000 we can be confident. The estimate of the heads-probability has a variable certainty that increases with the number of tosses. If we'd stopped after only 10 tosses, we would've had the incorrect perception that the coin would be in our favor.

Statistically, clinical trials have the same limitation, especially when it comes to safety. In them, coin tosses are volunteers, and heads can represent potential adverse effects. If nobody in a clinical trial of 1500 volunteers shows side effects, can we then conclude that there are none? Unfortunately no, as shown in our previous article, Swine flu vaccine induced narcolepsy.

But why not? 1500 is a lot of participants. However, the less frequent the side effect, the more participants are needed to be reasonably certain of discovering it. Let's consider a hypothetical side effect that has a baseline-rate of 1 per 1 million people, but the vaccine causes it to rise to a rate of 1 per 5000 people. Since it's far less frequent than the 1500 participants in the clinical trial, it's unlikely to occur at all — even though it's 200 times more likely with the vaccine.

Quantifying uncertainty

With Bayesian statistics we can quantify the certainty of our probability-estimate. It's a mathematical method to precisely describe the intuition of the coin-toss. This section is technical, and we will not elaborate the foundations of Bayesian statistics. To see the implications written in “plain English”, skip to the next section.

Each potential adverse effect can be expressed as a Boolean-valued outcome, with an unknown probability, $P (a)$ . A Boolean-valued outcome is suitable for death, as a person is either dead or not. For other adverse effects, the outcome is sometimes quantized into severity buckets. For analysis, they can still be modeled as Boolean-valued outcomes. Buckets are summed to, for example, “mild or high fever”. However, severity buckets are not relevant for our analysis. There are no severity buckets for myocarditis in the clinical trial, as it wasn't observed.

Assuming the clinical trial is appropriately sampled, the number of adverse effects, $n_{a}$ , in the clinical trial of $n$ participants will follow the Binomial distribution: $n_{a} \sim B (n, P (a))$

However, $P (a)$ , the posterior probability of an adverse effect, is unknown. By using the conjugate prior probability distribution of the Binomial, the Beta distribution, we can parameterize the unknown $P (a)$ as a function of $n_{a}$ and $n$ . $P (a) \sim B e t a (1 + n_{a}, 1 + n - n_{a})$

Here, we're assuming a uniform prior belief, $B e t a (1, 1)$ . The prior represents our belief before considering the relevant evidence. Since we have no particular belief before considering the evidence, we assume it is uniform.

Using the cumulative beta distribution and the results from the clinical trials, we can quantify how certain we can be that there are no adverse effects above some incidence-probability, $i$ .

Formula 1:

P (P (a) \leq i) = C D F [B e t a (1 + n_{a}, 1 + n - n_{a}), i]

We can also estimate the upper bound, $b$ , of $0 \leq P (a) < b$ within some credible interval, $c i$ , by using the percent point function.

Formula 2:

b_{c i} = PPF [B e t a (1 + n_{a}, 1 + n - n_{a}), c i]

The upper bound $b$ answers the question “With a $c i$ credible interval, what's the lowest $P (a)$ we can be certain of?”.

Statistically similar concepts to the credible interval are the confidence interval and p-value.

Pfizer-BioNTech vaccine risk

Let us compute the potential risk of a severe vaccine adverse effect, when the adverse effect wasn't observed in the clinical trials. Both myocarditis and death fit the criteria, but the argument is valid for any adverse effect.

Pfizer clinical trials (phase 1+2+3), children 5–11 years old

Vaccine group	1566
Placebo group	750
Median follow-up time	71 days from first injection
Deaths	0 (both groups)
Myocarditis	0 (both groups)

— Pfizer clinical trial: Evaluation of the BNT162b2 Covid-19 Vaccine in Children 5 to 11 Years of Age

Using Formula 2, we can compute an upper bound of any serious adverse effect, given the data from the three clinical trials. With a 95% credible interval, the common scientific standard for statistical testing, the upper bound is: $b_{c i = 0.95} = 1 in 524 children$

Expressed differently:

Given the clinical trials data, there is a 5% probability:

That more than 1 in 524 vaccinated will suffer a serious adverse effect, such as myocarditis or death, because of the vaccine.

Given the clinical trials data, there is a 50% probability:

That more than 1 in 2261 vaccinated will suffer a serious adverse effect, such as myocarditis or death, because of the vaccine.

Given the clinical trials data, there is a 95% probability:

That more than 1 in 30550 vaccinated will suffer a serious adverse effect, such as myocarditis or death, because of the vaccine.

The certainty of the rates of vaccine-induced myocarditis, death, or other serious adverse effects, given the data from the Pfizer clinical trials. Using Formula 1,

y = P (P (a) \leq x)

is plotted.

The median follow-up time of the clinical trial in the clinical trials is 71 days. The estimation will thus only capture the risk of short term adverse effects. There could still be long term adverse effects, as in our aforementioned example of swine flu induced narcolepsy. We may have to wait years before we can confidently reason about long term adverse effects. For now, statistically, we have no idea how serious it could be.

Covid19 Risks

Covid19 mortality

It's difficult to estimate the mortality of Covid19:

We don't know how many were infected. Using only verified infections will overestimate the mortality, as many children have mild to no symptoms. If they don't feel sick, they don't test, and consequently don't show up in the statistics.
We don't know whether each case is “death with Covid19” or “death from Covid19”. Indeed, it is not meaningful to attribute a death to Covid19 if the person had died regardless.
There's no detailed breakdown of co-factors. For example, obesity worsens the outcome of Covid19. That additional risk is not relevant for children of normal weight.
We don't know the risk of contracting Covid19 or the protection offered by cross-immunity. If you're already immune, you're not at risk.

We'll estimate coarse lower- and upper bounds, where we assume each counted death is because of Covid19. This will overestimate the risk. However, there's no better data available. As we will see, it won't alter our conclusion.

The estimation is imprecise, but it's not the same kind of imprecision as with the clinical trials. The clinical trials' uncertainty primarily originates from sampling a tiny part of the entire population of children. It selected and vaccinated about 1500 children (0.005%) out of a pool of at least 29 million children. It then draws conclusions for all 29 million children using only the 0.005% children in the study. Since sampling is well understood mathematically, we can reason about how well 0.005% represents the entire population precisely.

The uncertainty of the mortality-estimate is different. There's no sampling involved; we have the death count, verified infected count and population size. Unlike with sampling, we can't model the range between the upper- and lower bounds statistically; we have no way of quantifying how likely each possible mortality is.

Mortality: Children aged 5–11

Lower bound: Covid19-deaths divided by the population count; ≈ 2 per 1 million per year
Upper bound: Covid19-deaths divided by the number of verified infections; ≈ 49 per 1 million per year

Sources

Myocarditis risk

Myocarditis is an inflammation of the heart muscle (myocardium). The inflammation can reduce your heart's ability to pump and cause rapid or abnormal heart rhythms (arrhythmias).
Infection with a virus usually causes myocarditis. Sometimes myocarditis can result from a reaction to a drug or be part of a more general inflammatory condition. Signs and symptoms include chest pain, fatigue, shortness of breath, and rapid or irregular heartbeats.
Severe myocarditis weakens your heart so that the rest of your body doesn't get enough blood. Clots can form in your heart, leading to a stroke or heart attack.

— Mayo Clinic: Myocarditis

Myocarditis is a potential side effect of great interest. At least Sweden, Norway, and Finland have suspended one or more of the vaccines for young people as a precaution due to concerns of myocarditis. There's anecdotal evidence from newspapers that there's been an increase in blood clots, athletes collapsing, heart attacks, and other symptoms that could be explained by myocarditis.

To meaningfully compare the potential risk of the vaccine, we need to know the baseline incidence of myocarditis. As Covid19 itself may increase the risk of myocarditis, it's hard to estimate the true risk for the same reasons as for the mortality. Consequently, we'll estimate an upper- and lower bound for it too:

Myocarditis: Children aged 5–11

Lower bound: Myocarditis cases divided by the population count; ≈ 4 per 1 million per year
Upper bound: Myocarditis cases with verified Covid19 divided by the number of verified infections; ≈ 38 per 1 million per year

Note: The myocarditis-risk estimation suffers from yet another uncertainty: the smallest age group is 0–16. For our estimation, we linearly interpolate the data to the 5–11 age group. We assume that each age has an equal probability of myocarditis. This overestimates the risk, as the vast majority of myocarditis cases in the age group 0–16 occurs in children older than 11.

Sources

Vaccine/Covid19 relative risk assessment

At last, we have the necessary statistics, data, and theory to compare and quantify the potential risks of the vaccine to the risks of death and myocarditis. Our comparison is for children aged 5–11.

There are many potential risks from the vaccine and Covid19: stroke, hospitalization, long covid, antibody dependent enhancement, or something else we have yet to discover. The developed approach could be used for any risk. However, let's restrict the analysis to the most talked about: death and myocarditis.

Note that this makes no assumptions about how much the vaccine reduces the risks from Covid19. It is just comparing the risk of death or myocarditis posed by the virus, with the risk of death or myocarditis from taking the vaccine. This represents the best case scenario of the vaccine eliminating the risk from Covid19 for life from the dose administered in the trial — with no follow-on boosters required.

The risk of remaining unvaccinated is, in an ideal world, captured by the control group. However, the control group is even smaller than the vaccine group, with only half the number of participants. Thus, for the same reason that the vaccine-group can't uncover relatively infrequent adverse effects, the control-group is even less likely to discover the risks of remaining unvaccinated. Intuitively, a control group of only 750 children is incapable of capturing a 2 in 1 million risk of dying of Covid19.

However, Covid19-infections have been occurring for at least 2 years, and verified for most of its duration. We can thus compare the potential risks of the vaccine to the overall risk-estimations for death and myocarditis since the pandemic started.

Since we have estimated lower and upper probability-bounds for death and myocarditis, the relative potential risk-estimation will also be a range. The relative risk is computed by dividing the results of Formula 2 with the estimated risks of death and myocarditis.

As the children in the clinical trial were observed for a median of 71 days, we have converted the previously calculated per-year risks to per-71-days-risk.

Let us compare the risk for a few risk-tolerance levels:

Given the clinical trials data, there is a 5% probability:

That the vaccine increases the likelihood of myocarditis more than 258–2434 times.
That the vaccine is more than 199–4745 times deadlier than Covid19.

Given the clinical trials data, there is a 50% probability:

That the vaccine increases the likelihood of myocarditis more than 60–564 times.
That the vaccine is more than 46–1099 times deadlier than Covid19.

Given the clinical trials data, there is a 95% probability:

That the vaccine increases the likelihood of myocarditis more than 4–42 times.
That the vaccine is more than 3–81 times deadlier than Covid19.

The certainty of the vaccine not causing a large increase of myocarditis, given the data from the Pfizer clinical trials. Using Formula 1,

y = P (P (a) \leq x \times P (myocarditis))

is plotted for the upper and lower

P (myocarditis)

estimates.

The certainty of the vaccine not causing a large increase of deaths, given the data from the Pfizer clinical trials. Using Formula 1,

y = P (P (a) \leq x \times P (death))

is plotted for the upper and lower

P (death)

estimates.

Interpretation

The risk that severe adverse effects, if they exist, went unnoticed in the clinical trials is huge. It is, however, important to emphasize that the analysis is not asserting that the vaccine has risks that large. It instead shows that the clinical trials cannot demonstrate the Pfizer-BioNTech vaccine's safety compared to remaining unvaccinated, nor rule out the possibility that it drastically increases the risk of death, myocarditis, or another less common adverse effect.

The proof is statistical. It does not depend on any claim of misconduct or fraud. It uses officially reported numbers to derive its conclusion.

Vaccine benefits

For the same reason as why the clinical trial cannot demonstrate that the vaccine is safe, it fails to prove that the vaccine reduces severe risks from Covid19. Quoting the Pfizer clinical trial:

PHASE 2–3 EFFICACY
Among participants without evidence of previous SARS-CoV-2 infection, there were three cases of Covid-19 (with onset 7 days or more after the second dose) among BNT162b2 recipients and 16 among placebo recipients; the observed vaccine efficacy was 90.7% (95% CI, 67.7 to 98.3). Among all participants with data that could be evaluated, regardless of evidence of previous SARS-CoV-2 infection, no additional cases were reported; the observed vaccine efficacy was 90.7% (95% CI, 67.4 to 98.3) (Figure 3). No cases of severe Covid-19 or MIS-C were reported. [emphasis added]

No child in the vaccine group or the placebo group were reported to have had any difficulty clearing Covid19 infections.

The claim “the observed vaccine efficacy was 90.7%” does not assert any tangible benefits for the children in the study. It is reasonable to hypothesize that fewer infections are connected to fewer bad outcomes from Covid19. We can also hypothesize that, perhaps, it's better for children to go through a mild Covid19 while young, and consequently possibly have a stronger lasting immunity for the rest of their lives.

The key insight is that the clinical trials make no claim either way. Concretely, they indicate that the vaccine reduces the probability of testing positive for Covid19 from 2.1% to 0.2% — a decrease of 1.9 percentage points.

“The benefits outweigh the known and potential risks”

Concerned citizens and parents have questioned and expressed concern whether the vaccine is proven to be safe and beneficial. Government agencies, medical professionals, and Pfizer assert variations of “the benefits of Covid19 vaccination outweigh the known and potential risks. Reports of adverse events, like allergic reactions or myocarditis or pericarditis, are rare”.

These statements are misleading, especially for children aged 5–11. There are no proven tangible benefits, and the potential risks of the vaccine are considerably higher than the risks of Covid19 itself.

As we have shown, the clinical trials — using the common scientific 95% certainty standard — fail to demonstrate that the vaccine is safe from causing a serious adverse effect in more than 1 in 524 children. Furthermore, they fail to demonstrate that the vaccine doesn't multiply the risk of myocarditis or death between hundreds to thousands times, compared to remaining unvaccinated.

Pfizer recognizes the inherent limitation. Indeed, in the Overall Risk-Benefit Conclusions section at a recent advisory committee briefing, they conclude:

The number of participants in the current clinical development program is too small to detect any potential risks of myocarditis associated with vaccination. Long-term safety of COVID-19 vaccine in participants 5 to <12 years of age will be studied in 5 post-authorization safety studies, including a 5-year follow-up study to evaluate long term sequelae of post-vaccination myocarditis/pericarditis.

— Vaccines and Related Biological Products Advisory Committee October 26, 2021 Meeting Briefing Document - Sponsor (Pfizer)

Conclusion

At the time of writing, governments and medical agencies are starting or contemplating to mass-vaccinate children aged 5–11. A child's risk of Covid19 is minuscule. Neither safety nor significant benefits for the children, compared to remaining unvaccinated, are proven.

We have presented a statistical proof. It demonstrates that the clinical trials are fundamentally incapable of uncovering even a large increase of myocarditis, death, or other serious adverse effects, caused by the vaccine.

Dr. Rubin from the FDA recognizes this inherent limitation. He and the FDA want to proceed with the vaccination regardless. It's a large experiment, and every parent must ask themselves whether or not they want their children to participate.

“We're never going to learn about how safe this vaccine is unless we start giving it. That’s just the way it goes. That's how we found out about rare complications of other vaccines like the rotavirus vaccine. And I do think we should vote to approve it”.

Revisions

December 6, 2021

Clarified why the baseline myocarditis risk is overestimated.

December 15, 2021

“No evidence of side effects” section changed to “The benefits outweigh the known and potential risks”, which more accurately reflects the communication from government agencies.