Simpson’s Paradox
In the context of cancer treatment, researchers have found that the cure rate for cancer with medication is 76% and only 66% with surgery.
However, if these researchers analyze the data at a lower level of granularity by taking into account the size of the tumors, the cure rates with surgery (63% and 90%) are higher than those observed with the use of drugs (49% and 82%). The conclusions of the analysis of these same data are paradoxical.
The size of the tumor, which has an impact on both the cause (treatment) and the consequence (cure), is a confounding factor. The presence of a confounding factor explains Simpson’s paradox described above.
Causal Inference to Validate an Observational Study of Vaccine Efficacy
To validate clinical trials of a vaccine, researchers can choose between a randomized controlled trial and an observational study.
- The randomized controlled trial is an experimental clinical research protocol comparing an experimental (“intervention”) group testing the new vaccine with a group (“control”) taking a placebo.
- The objective of the observational study is to assess the causal relationship between a specific exposure to the vaccine and a health event (immunity).
The randomized phase 3 trial of the BNT162b2 mRNA COVID-19 vaccine included 21,720 individuals who were randomly assigned to the vaccine group, which only allowed estimation of vaccine efficacy in a small number of subpopulations. In addition, patients with chronic diseases were included only if their condition was deemed stable by the investigators.
Epidemiologists, including Miguel Hernán, wanted to complement this randomized trial with an observational study. They used data repositories from Israel’s largest health care organization to assess vaccine efficacy in SARS-CoV-2 infection, symptomatic COVID-19, hospitalization, severe illness, and death.
This observational study of the efficacy of the BioNTech, Fosun Pharma, Pfizer vaccine should not be biased by confounding factors. To avoid Simpson’s syndrome, the researchers used causal inference.
Causal inference refers to the process by which a causal relationship can be established between an item and its effects. To do this, it is necessary to eliminate confounding factors that may distort the conclusions of an observational study.
Identify confounding factors
To remove confounding factors. Miguel Hernán begins by identifying potential confounders. For example: age and gender. Vaccination campaigns prioritize older people, and older people are more likely to develop severe disease.
After adjusting for age, how do we know if there is residual confoundings? The researchers know from the randomized trial of 21,720 people that the vaccine has no effect in the first few days. So adjusting the observational data for age and sex should reproduce the fact that the vaccine has no effect in the first few days.
However, after adjustment for age and sex, the infection curves start to diverge from the first day. Conclusion: the adjustment for age and sex is insufficient. There are other factors that impact the effect of vaccination.
Taking into account other confounding factors such as location, co-morbidities, and access to care: the fact that the vaccine has no effect in the first few days (the curves do not diverge immediately) is replicated and validates the results of this observational study.
The efficacy of the BioNTech, Fosun Pharma, Pfizer vaccine is demonstrated. The cumulative incidence in % among vaccinated patients is much lower than among non-vaccinated patients at around day 12.
To go further on the subjects evoked in this article
Simpson’s Paradox
Causal Diagrams MOOC on EDX: Draw Your Assumptions Before Your Conclusions par Miguel Hernán
