Serious adverse event frequency: about like getting struck by lightning?

Explaining to COVID vaccine clinical trial participants that the probability of a certain serious adverse event is 1/50,000 or less doesn’t work. Most people instantly stop listening at the slightest whiff of mathematics, including ordinary numbers. But if you compare it to being struck by lightning, that makes sense to most people. Can you guess what happened next?

What’s the sitch?

If you’re going to run a randomized clinical trial, the gold standard of evidence in medicine, you need to understand 2 things: how serious adverse events (SAEs) are reported, and informed consent from the people participating.

Serious Adverse Event reporting

The reporting of Serious Adverse Events (SAEs) is maybe a lot more careful & thorough than you probably think!

Basically, you report everything, whether plausibly related to the treatment or not. You even report what happens to people in the placebo arm of the trial, for comparison. If somebody gets run over by a truck, you report it. (Could the treatment have made them absent-minded, and they walked out into traffic? ^[1]) If somebody dies in a plane crash, you report it. If somebody… well, you report it. Statisticians will sort it out later.

Informed consent of participants

The other thing is that the participants have to understand what the risks are, how likely they are, and similarly for the benefits. This involves getting a tiny bit of probability into the heads of the general public, which is no easy task.

Generally speaking, if you try to use math, you will fail. Even if you avoid math, and just use numbers – not even arithmetic, just quote a few numbers – most people will resist. Numbers slide off them like water off the back of a duck. They think it’s “some nerd thing” that is of no concern to “real people” such as themselves. It’s just plain mathematical illiteracy, which is an educational scandal, but… nothing we can fix at this point.

So people use analogies to explain the risks. The chance of, say, a heart attack in the trial is estimated (very crudely) as being about like getting struck by lightning (which the Centers for Disease Control estimate at about 1/500,000 per person per year). Most people can sort of wrap their brains around that.

Now can you predict where this is going?

3 SAEs in the treatment arm of the Moderna trial

Ok, you’re gonna make me spell it out, aren’t you?

If you dig a bit deeper into the FDA analysis of the Moderna data submitted for the EUA ^[2] (as we did previously, but apparently not deeply enough!), there is of course a report of exactly the 3 SAEs in the treatment arm of the trial, on page 54:

As of December 6, 2020, there were 3 SAEs reported in the vaccine group: a 65-year-old participant with community acquired pneumonia 25 days after vaccination, a 72-year-old participant with arrhythmia after being struck by lightning 28 days after vaccination, and an 87- year-old participant with worsening of chronic bradycardia 45 days after vaccination. On FDA review of the narratives, none of these SAEs are assessed as related. There were no cases of severe COVID-19 reported in the study.

We first note approvingly the final 2 sentences, that the FDA statisticians decided none of these were related to the treatment, which is the good news. Also, all the SAEs were in elderly participants, who are at higher risk for SAEs anyway, treatment or no. This is why you report everything, so nobody can accuse you of covering anything up, and the statisticians can absolve you of blame.

But… one of the participants was struck by lightning!

Now, you know what happens next here.

The more mathematically illiterate media will of course report this endlessly, sometimes with a wink & a nod to indicate they know better, but nonetheless baiting the less mathematically literate readers to mistake it for actual news.

This particular example ^[3] is from the Metro, a British tabloid for which I didn’t have much respect at the outset. And now I have even less respect for them, since the headline “Covid vaccine volunteer, 72, struck by lightning after getting Moderna injection” invites people to draw the most stupid conclusion possible, that the Moderna vaccine attracts lightning strikes.

Must be the ground strap for that microchip Bill Gates is putting in all of us via the vaccines, right? ^[4]

---

Notes & References

1: Back before the Affordable Care Act/Obamacare, I saw this used to deny coverage based on pre-existing conditions. A hypothetical example: suppose you were diabetic, and because you didn’t work for a large company, you had to accept insurance that excluded your pre-existing diabetes.

Now you get hit by a truck and your leg is broken. Are you covered?

Maybe not: insurers could argue your diabetes made you hypoglycemic, so you spaced out & walked into traffic, and thus they don’t have to pay.

This is the sort of evil behavior that the ACA tried to stamp out, with mixed results and hysterical resistance from Republicans. ↩

2: FDA staff, “FDA Briefing Document: Moderna COVID-19 Vaccine”, FDA.gov, downloaded 2020-Dec-15. ↩

3: J McCloskey, “Covid vaccine volunteer, 72, struck by lightning after getting Moderna injection”, Metro, 2020-Dec-18. ↩

4: J Goodman and F Carmichael, “Coronavirus: Bill Gates ‘microchip’ conspiracy theory and other vaccine claims fact-checked”, BBC News, 2020-May-29. ↩