How to read the evidence

How a safety signal becomes a finding

Every number on this site sits somewhere on this ladder. A report is the bottom rung; an established adverse effect is the top. Each step removes a different way of being wrong. The grey boxes follow one real example, myocarditis after mRNA COVID-19 vaccines, up the ladder.

1

A report

Someone reports an event after a vaccine (VAERS) or drug (FAERS). It shows that the event happened afterwards — not that the product caused it. Anyone can report; reports are unverified and incomplete.

Spring 2021: reports of chest pain with raised troponin in young men a few days after their second mRNA dose.
2

A pattern: disproportionality

Is the event reported more often with this product than with others in the same database (PRR, ROR, EBGM)? It flags patterns but is biased — on OpenPV, negative controls average an ROR of about 2, so most raw signals are noise (method validation).

Myocarditis was reported disproportionately often for mRNA vaccines compared with other vaccines, concentrated in males aged 12–29.
3

More than expected: observed vs expected

How many cases would occur anyway in that many people, in that time window, at background incidence? Because reporting is incomplete, observed exceeding expected is meaningful; observed below expected is not reassurance (observed vs expected).

On this site's analysis, COVID-19 myocarditis/pericarditis reports within 7 days exceed the all-age background expectation (O/E 1.14, 95% CI 1.10–1.17) despite under-reporting; in young men the excess is far larger.
4

Real cases: a case definition

Do the reports describe the actual disease? The Brighton Collaboration sets levels of diagnostic certainty (Brighton analysis). Individual cases can then be assessed with the WHO AEFI algorithm.

In this site's screen of VAERS narratives, more than half of COVID-19 myocarditis reports (56%) document findings meeting Brighton Level 1–2 — raised troponin with cardiac MRI, echo or ECG findings — not just chest pain.
5

A comparison group: controlled studies

Compare vaccinated with unvaccinated people, or the same people in a risk window vs a control window (self-controlled case series). This is the step that estimates whether and how much risk is raised (vaccine safety studies).

Vaccine Safety Datalink, Israeli, Nordic and English studies measured the excess: highest after a second dose in males 16–29 (on the order of 1 in 6,600 second doses in Israeli males 16–19 in one study), much lower in other groups; the English study also found infection itself carried a larger excess in most groups — but not in men under 40 after a second Moderna dose.
6

How sure are we: certainty of evidence

GRADE rates the body of evidence as high, moderate, low or very low, downgrading for bias, inconsistency, indirectness, imprecision and publication bias, and upgrading for large effects or dose–response. Each vaccine page now carries GRADE-style ratings.

Consistent findings across several large controlled studies, a clear dose and age pattern and a plausible mechanism: high certainty that mRNA vaccines cause rare myocarditis, mostly mild and in young males.
7

Decision: benefit and risk

A confirmed risk is weighed against the disease prevented, in the same population and on the same scale (per million), which can change product choice, dose spacing or age recommendations.

Several countries preferred Pfizer for young men or lengthened the interval between doses.

The same ladder for a signal that did not hold up

Varenicline (Chantix) and suicidal ideation: strong FAERS disproportionality (step 2), reports surging after media coverage and a 2009 boxed warning (see the reporting-over-time panel), then a large randomized trial (EAGLES) found no significant increase in neuropsychiatric events (step 5) — the boxed warning was removed in 2016. A strong signal is a question, not an answer.

Reviewed 2026-10-09 · Open Source Medicine Foundation. Sources for the example are linked on the COVID-19 vaccine page. Not medical advice.