CDC, FDA Admitted to Using ‘Mostly Useless’ Tool to Detect COVID Vaccine Safety Signals

3 hours ago

Originally posted by: Children's Health Defense

Federal health officials knew that the statistical tool they relied on to look for COVID-19 vaccination safety signals in the Vaccine Adverse Event Reporting System (VAERS) was “mostly useless,” according to internal documents obtained by Sen. Ron Johnson (R-Wis.) and analyzed by scientists at Children’s Health Defense (CHD).

The documents show that officials at the Centers for Disease Control and Prevention (CDC) and U.S. Food and Drug Administration (FDA) internally acknowledged that the tool — empirical Bayesian (EB) data mining — had “blind spots” that rendered it “mostly useless” for picking up on safety signals of COVID-19 vaccines.

Yet, the agencies used the method in analyses and attempted to publish findings from those analyses — including studies that supported the safety of COVID-19 vaccines.

Karl Jablonowski, Ph.D., CHD senior research scientist who analyzed the documents, told The Defender:

“Imagine a night watchman has to find something on the ground. But instead of holding a flashlight, he is wearing sunglasses. In the morning, he says he didn’t find anything. That’s true, but it’s because he was using a tool that impeded his ability to see.”

The records obtained by Johnson’s office include emails between CDC and FDA researchers from 2021 to 2023, along with draft manuscripts and peer reviewer comments.

In one case, researchers sought to publish an analysis in The Lancet Infectious Diseases using EB data mining on early COVID-19 vaccine data. They dropped the plan only after a reviewer wrote that the likelihood of detecting a safety signal using the method was “likely close to zero.”

FDA official Dr. David Menschik, who initially was a co-author on the paper, wrote to the study’s lead author in December 2021 saying he knew the method was essentially useless.

“We acknowledged this in the limitations and understand that there is a considerable bias toward the null when using our data mining methods in this current, unprecedented situation,” he wrote.

Menschik agreed that the study authors should drop the data mining results and the authors who had contributed the results from the manuscript.

An investigation by The Defender also found that the study authors altered the manuscript to downplay deaths following COVID-19 vaccination.

In a September 2021 email, Menschik wrote that “data mining has blind spots and this is why it’s so good to have so many complimentary vaccine safety surveillance systems (e.g., VSD) that can cover different blind spots of other systems.”

The documents also show that CDC and FDA officials used EB data-mining results in a paper on tinnitus, or ringing in the ears, following COVID-19 vaccination — even though the authors knew the method failed to accurately detect tinnitus safety signals.

In August 2024, the authors published their findings in the American Journal of Otolaryngology. Rather than removing the faulty results from the study, the authors simply added the disclaimer that the findings “cannot definitively exclude the possibility” that COVID-19 vaccines may increase a person’s risk of tinnitus.

In a March 30 letter, Jablonowski called on the American Journal of Otolaryngology editor to retract the tinnitus paper. He said on today’s episode of “Good Morning CHD”:

“Doctors, please, save us the trouble and retract all papers that lean on the lack of a COVID vaccine EB data mining signal to support safety, such as the tinnitus paper, such as the safety in adolescents aged 12-17 years (published in MMWR), such as the pregnancy adverse events in the journal of Vaccine.”

Breaking: CHD Hits AAP with Rico Suit Over Fraudulent Vaccine Safety Claims

Tool is ‘mostly useless’ for COVID vaccines

Johnson obtained the new documents after he subpoenaed the U.S. Department of Health and Human Services (HHS) for COVID-19 vaccine safety records and communications about the COVID-19 pandemic.

On March 23, Johnson sent a letter to HHS Secretary Robert F. Kennedy Jr. about key findings from the documents, including revelations that Biden-era health officials knew EB data mining didn’t work for finding COVID-19 vaccine safety signals.

EB data mining is a type of disproportionality analysis that makes comparisons among data. FDA researchers use the tool to see whether a disproportionate number of VAERS reports have been filed for a particular vaccine compared to other vaccines.

However, the method can give inaccurate results if similar vaccines — such as the Pfizer and Moderna COVID-19 vaccines — are included in the comparison. This is sometimes called masking.

“For instance, EB data mining did a poor job of detecting myocarditis as a safety signal for the COVID-19 vaccines because both the Pfizer and Moderna shots had a high number of myocarditis reports,” Jablonowski said.

Johnson wrote:

“Another simplified way to hypothetically explain the effects of masking would be to test the adverse events of hemlock by comparing them to the adverse events of arsenic. Both may be equally dangerous when each is compared separately against a non-toxic substance such as saline.

“But, if the effects of hemlock were compared against a baseline that combines the data for the effects of arsenic and saline, the adverse events of hemlock may not result in a significant signal because they have been drowned out or masked by the baseline data which includes the toxic effects of arsenic.”

Judith Maro, Ph.D., an associate professor of population medicine at the Harvard Pilgrim Health Care Institute and co-author of the tinnitus paper, raised this concern in an email to Dr. John Su, a co-author who worked at the CDC’s Immunization Safety Office.

Maro wrote, “It will be mostly useless to try to make statements about the COVID vaccines because the database will have so many COVID reports that you can’t create a comparator.”

‘We were aware of this limitation before and during the pandemic’

Su responded by emailing his CDC and FDA co-authors, “I don’t know the methods well enough to address Judy’s comments. How do the methods FDA uses address these points?”

Dr. Narayan Nair, who worked in the FDA’s Office of Biostatistics and Pharmacovigilance, wrote that Maro “does bring up a good point.”

“We were aware of this limitation before and during the pandemic,” Nair said.

The paper authors had several options, he wrote. “We could report our data mining findings and just acknowledge this as a limitation (this is what we have done in other papers).”

The team could omit the data-mining results or develop a more accurate tool. “I am not sure how you would do this but you would need a statistician with DM experience. This would be beyond our capabilities at FDA,” Nair wrote.

The authors chose to publish their EB data-mining results and simply list them as a limitation of the study.

“It is unconscionable that FDA and CDC leaders knew throughout the COVID pandemic that the method was flawed but used it anyway,” Jablonowski said.

“What’s worse is that the CDC in 2022 called EB data mining a ‘gold standard’ technique and used it as an excuse to not perform a different kind of safety signal analysis called PRR, or proportional reporting ratios, of VAERS reports.”

This article was funded by critical thinkers like you.

The Defender is 100% reader-supported. No corporate sponsors. No paywalls. Our writers and editors rely on you to fund stories like this that mainstream media won’t write.

Please Donate Today

CDC praised EB data mining

According to Freedom of Information Act (FOIA) documents CHD obtained in 2022, reporters from several media outlets, including a Reuters fact-checker, asked the CDC in August 2022 if the agency had done any PRR analyses of VAERS reports.

The reporters’ queries came in response to the June 21, 2022, article in The Defender, “CDC Admits It Never Monitored VAERS for COVID Vaccine Safety Signals.” The article reported on the agency’s response to a CHD FOIA request about CDC’s analyses of VAERS reports following COVID-19 vaccination.

The CDC wrote in its response to CHD that “no PRRs were conducted by CDC. Furthermore, data mining is outside of the agency’s purview.”

The CDC public affairs office told the Reuters reporter in an email:

“A proportional reporting ratio (PRR) is a calculation. While PRRs can provide a rough means of examining disproportionate reporting, they are at best a crude measure prone to many false signals.

“Therefore, to detect disproportionate reporting, CDC and FDA chose to rely on Empirical Bayesian data mining. Given that it is a ‘gold standard’ mining technique, CDC will continue relying upon empirical Bayesian data mining at this time.”

Similarly, former CDC Director Rochelle Walensky told Johnson in a September 2022 letter, “CDC and the Food and Drug Administration (FDA) chose to rely on Empirical Bayesian (EB) data mining — a more robust technique used to analyze disproportionate reporting — rather than PRR calculations to mitigate potential false signals.”

When The Defender asked the CDC what methods it currently uses to analyze COVID-19 vaccine VAERS reports, the agency declined to answer.

Instead, HHS Public Affairs told The Defender in an email:

“On May 8, 2025, CDC and FDA expanded public access to VAERS data in the WONDER database (wonder.cdc.gov) and in VAERS downloadable files (vaers.hhs.gov) to provide more information for all reported adverse events following vaccination received.

“This enhancement is part of a broader CDC and FDA effort to improve transparency and access to vaccine safety data, while continuing to protect patient privacy. VAERS data change as new reports are received, so your results may change if you repeat the same search at a later date.”

The FDA appears to still use EB data mining. The agency did not immediately respond to our request for comment.

Watch Jablonowski talk on CHD.TV here:

Related articles in The Defender