Vaccine Efficacy, Delta Variant, and the CDC ChangesTagged:
Last week several bombs were dropped: COVID vaccine efficacy with the Delta variant, breakthrough infections, asymptomatic carriers, and changes in CDC guidelines. Was the panicky media coverage at all useful?
Vaccine efficacy vs Delta in the real world
First, let’s consider the news about “waning efficacy” of the mRNA vaccines. STAT News, as usual, had a very even-handed account.  They noted that a Pfizer preprint showed after 6 months:
- The Pfizer vaccine had overall efficacy of 91% averaged over the period following 44,000 volunteers. This is still outstandingly good.
- Efficacy was 96% in the first 2 months, and 84% by the end. So, a decline of about 3%/month, which is actually pretty good.
- It is unknown if that is due to fading immune response or the evolution of the virus into more virulent forms, like Delta.
- Against severe disease requiring hospitalization, it was 97% efficacy. Very good, indeed.
- At this rate of decline, efficacy would reach the benchmark threshold of 50% efficacy at 18 months, which would be the time for a booster. (Preferably a Delta-specific booster, but the original vaccine would also do.)
Other outlets were less measured in their reporting, along the lines of: “Glaah! The vaccines are fading, we’re all gonna die!” So let’s ignore them and look directly at the primary literature, namely the preprint itself.  (It’s a preprint, so it’s undergoing peer review right now. But take it with a small grain of salt.)
- They had 44,165 subjects studied in a placebo-controlled, observer-blinded, multinational trial. This isn’t some tiny little study!
- Overall efficacy was 91% (95% CI: 89.0% - 93.2%). That’s a nice, narrow confidence interval, so this looks statistically solid.
- Efficacy vs severe disease was 97% (95% CI: 80.3% - 99.9%).
Overall, it’s a good, solid result: the vaccines still work even with the variants present. At the rate of decline of efficacy, it should be about a year before boosters become de rigueur.
Conclusion: Don’t panic. At least not about this.
The CDC on Delta: ‘the war has changed’
The WaPo, the internal CDC slide deck, and media reaction
The WaPo “obtained” a CDC internal slide presentation and reported on the state of thinking at CDC.  They describe it as “striking an urgent note”, which no doubt led to some of the media reaction. They summarize the content in basically 2 arms:
- There is significant frustration at CDC with public communications. Given the amount of superstitious fear and general interference by nearly every Republican politician, this is entirely understandable. People are frustratingly becoming convinced that the vaccines no longer work. But as you saw above, the efficacy is still 85% - 90%, which is the very definition of “works”!
- We’ve been underestimating the the ease of transmission of Delta; it’s worse than we generally thought, and we already thought it was pretty bad. Vaccinated people can get asymptomatic Delta and spread it as easily as the much sicker unvaccinated. So universal masking is a good idea, until herd immunity (late 2021/early 2022?).
That provoked a lot of somewhat inflammatory reaction in other media. One of the slightly less inflammatory is shown here, from MSNBC. Keep in mind if you watch it that almost everything else I saw was more inflammatory than this. The things that went wrong here are:
- He’s not interviewing anybody with domain expertise or COVID knowledge; he’s interviewing one of the WaPo reporters. Basically reporters interviewing each other. This is how small errors get amplified.
- There is, at best, minimal discussion of the content of the CDC document, just what “people are saying” about it. The little content that is discussed is the communication problems, not the actual data about the Delta variant (which is the important part).
- His tone of voice is rather alarming, as if urging viewers to panic and outrage.
Predictably, this did not escape reaction from senior political folk, as reported when CNN talked to White House aides.  They used words like “hyperbolic”, “irresponsible”, and “alarmist”. Of course, they requested anonymity, which sort of lowers their credibility at least with me. Still…
“The media’s coverage doesn’t match the moment,” one of the Biden officials told me. “It has been hyperbolic and frankly irresponsible in a way that hardens vaccine hesitancy. The biggest problem we have is unvaccinated people getting and spreading the virus.”
At least some media outlets didn’t “bury the lede”. A prime example is at Axios, where Gonzalez & Alberti make the point sharply :
- The rate of breakthrough infections among the vaccinated is very, very low: possibly around 0.1%.
- The hospitalization rate is better, at around 0.004%.
- The death rate is even lower, around 0.001%.
Now that’s the right message: yes, there are breakthrough infections; but they are very rare, and almost none result in hospitalization or death.
As the CDC reported, breakthrough infections can be spreaders, so it’s a good idea to wear a mask even if you’re vaccinated. If you’re unvaccinated, then just realize you should also mask, but being voluntarily unvaccinated at this point is kind of suicidal.
Ok, so what exactly is in the internal CDC slide deck?
Here’s the (frustratingly anonymized!) CDC slide deck , archived here in case it gets removed from the WaPo web site.
There’s some entirely understandable venting about the problems of communicating with the public, especially at the same time as disinformation actors like Republican officials. Still, they manage to show that vaccines are _dramatically reducing risks as shown on slide 3:
- Infection risk is reduced 8 fold, corresponding to an efficacy of 100% * (1 - 1/8) = 87.5%.
- Both hospitalization and death risks are reduced 25 fold, for an efficacy of 100% * (1 - 1/25) = 96%.
That’s fantastically good evidence that vaccines work, and work well. Why wasn’t this one of the headlines in the news?
One item that got some rather nervous side-eye is shown on slide 15. Here the plot shows $R_0$ on the horizontal axis (basically how many new cases of infection does each case generate; it measures the ease of transmission for a disease). The vertical axis is the mortality on a log scale (what fraction of the infected people die). Obviously, the upper right corner (high transmission, high mortality) is where we don’t want to be.
The inital strain of SARS-CoV2 is sown in the pink box at the lower left. The box size apparently shows the uncertainty band, though they don’t quite come out and say so. It’s about as transmissible as the common cold, but it kills somewhere between 0.1% – 1.0% of patients. That’s pretty bad, but…
The blue box is the Delta variant. It’s not remarkably higher in mortality (maybe just a little). But it’s way more transmissible! It looks like they’re estimating $R_0 \sim 5-9$, vs $R_0 \sim 1.5 - 2.5$ for the original strain. So maybe 4 times more transmissible, on average? That’s a huge deal. It also puts Delta on a par with the highly transmissible chicken pox, though thankfully not on a par with the horror that would be measles ($R_0 \sim 15$).
Another bit of bad news is in slide 17: although breakthrough infections in vaccinated people are rare, once they happen those people can be asymptomatic spreaders to the unvaccinated. So the conclusion is: everybody needs to mask up, so the vaccinated won’t be spreaders of Delta to the unvaccinated.
Cue: infinite whining about masks, mask mandates, changing policy, etc. This is why the CDC said “the war has changed”: Delta is worse enough that we have to go back to masking, until the remaining unvaccinated can be made safe with vaccination.
The Weekend Conclusion
The bad news:
- Delta is more contagious than we thought, maybe on a par with chicken pox.
- Vaccinated people can (rarely!) get asymptomatic Delta, and be spreaders.
The good news:
- The rate of breakthrough infections in the vaccinated is very, very low.
- Vaccine efficacy remains high, both as measured in the Pfizer study and by the CDC study of cases, hospitalizations, and death rates.
So to prevent any (rare!) asymptomatic Delta cases in the vaccinated from being spread, everybody should mask up again.
Notes & References
1: D Garde & M Herper, “Efficacy of Pfizer/BioNTech Covid vaccine slips to 84% after six months, data show”, STAT News, 2021-Jul-28. ↩
2: SJ Thomas, et al.,, “Six Month Safety and Efficacy of the BNT162b2 mRNA COVID-19 Vaccine”, medRχiv 2021-Jul-28. DOI: 10.1101/2021.07.28.21261159.↩
3: Y Abutaleb, CY Johnson, & J Achenbach, “‘The war has changed’: Internal CDC document urges new messaging, warns delta infections likely more severe”, Washington Post, 2021-Jul-29. ↩
4: O Darcy, “White House frustrated with ‘hyperbolic’ and ‘irresponsible’ Delta variant coverage, sources say”, CNN Businesss, 2021-Aug-02. ↩
5: Authors redacted, “Improving comunications around vaccine breakthrough and vaccine effectiveness”, 2021-Jul-29. Originally obtained by the Washington Post. ↩
6: O Gonzalez & D Alberti, “Chart: Less than 0.1% of vaccinated Americans tested positive for COVID-19”, Axios, 2021-Jul-31. ↩
Hey, Kto –
I think I now understand the reason for our confusion: Simpson’s paradox.
See this new post. Basically, age is a correlate of both being vaccinated, and being hospitalized if there’s an infection. So you have to stratify by age.
There’s more, including calculating the right Bayesian posterior, but that’s the main thing.
All of this was uncovered by Morris, writing at COVID-19 Data Science. I just did a little “explainer” about it.
Thanks for planting a seed of criticism that made me go read some cool stuff.
I’m aware of Simpson’s paradox, but I don’t think it resolves our original issue. Morris’ data shows that when you stratify the data by age, you find ~80-90% efficacy against severe cases among the most vulnerable groups. Given ~90% vaccination rate, this is consistent with the vaccinated people making ~50% of all deaths. So, there is still a missing factor of ~125, when you compare the UK and Israeli data to the Fauci’s 0.8% claim for the US.
Thank you for the detailed response.
“Summary: I don’t quite believe that the death rates in other countries are now mostly among the vaccinated”
You don’t have to take my word on it. Here is a direct link to the official UK government data:
According to the UK report, 54% of all deaths from delta are in fully vaccinated patients. By contrast, according to Fauci’s claim only 0.8% of recent deaths in the US came from the fully vaccinated people (i.e., the odds ratio of 125:1 instead of approximately 1:1). There is a missing factor of ~125 here.
Yeah, you convinced me there’s something contradictory here.
All the other stories report very low death rates among the vaccinees, e.g. here.
Perhaps there’s some time skew, where the low death rates are pre-Delta? Or is somebody using a different denominator without actually describing what they’re doing? (My experience: more confusion comes from trying to “avoid math” and not write an equation. A bowl of word salad vaguely explaining it is an invitation to trouble.)
Hi Weekend Editor,
I found your blog through the ACX classified thread. I have a question about your post:
How can your statement (“Vaccinated people can (rarely!) get asymptomatic Delta”) and the claim that “Against severe disease requiring hospitalization, it was 97% efficacy” be reconciled with the data from UK and Israel where the majority of deaths come from fully vaccinated?
Given that the fraction of vaccinated among the elderly there is around 90% (less among the younger age groups), with the 97% efficacy the deaths among the vaccinated should be under 25%.
Welcome, and thanks for taking the time to read a post!
It’s nice to hear from somebody who is neither a Russian casino spammer nor a personal acquaintance/family. :-) This crummy little blog has a global readership of about 6, but 3 of them are my spouse, my cat, and myself. And the cat isn’t a particularly close reader.
Let me rephrase your question, so we can at least share a qualitative understanding of what we’re trying to wrap our heads around:
I’ll start by saying I haven’t read data saying that about the UK and Israel. This is not at all unexpected, since I haven’t sought it out. In fact, my (only very lightly informed) impression was the opposite: see here for evidence from a Qatari study and an Israeli study documenting near-100% efficacy vs severe disease after both doses of mRNA vaccines.
But for the sake of discussion I’ll take you at your word that you think this is the case, and you may even be right since I’m not especially informed here. The question then is: why do we think vaccination is so protective against death in the US, but in the UK and Israel it looks less so?
If I had access to actual data showing that, then maybe we could do some statistical hypothesis testing. But I don’t have that data, and given the chaotic patchwork of EMR systems in the US, it’s pretty much unobtainable. There’s a reason some of the best studies I’ve seen were the Israeli & Qatari ones: they have actual universal health care and universal EMR systems so the can get at reasonable datasets.
But with things as they are, I’ll just content myself with pointing out some of the problems of comparing across countries and medical systems:
Even inside the US, you have different subpopulations with radically different responses to public health guidance. New England masked up, socially distanced, and got vaccinated pretty fast. In the South, they’re still debating about whether the vaccine is from Satan, or just from Bill Gates & George Soros. Very different, and hard to compare!
Summary: I don’t quite believe that the death rates in other countries are now mostly among the vaccinated, and have some data that makes me think that. However, whatever the difference in vaccinated death rates are between countries, comparisons over different populations exposed to different variants with different medical care best practices and different compliance with public health guidance is… unlikely to be straightforward?