Missing Data
Missing data is the one data problem that announces itself politely: an empty cell, and no error. What it costs you depends on why the cell is empty, and this section measures that cost instead of asserting it. It starts from the missingness mechanisms, prices the simple fixes, builds multiple imputation with mice and miceforest, and ends by checking the imputation against the truth it was trying to recover.
Every page pairs R and Python in synced tabs, and every number and figure comes from a real run of the code shown, in one pinned container, in both languages. The basics of detecting and filling missing values live in Missing Values; this section is about what to do when filling is not enough.
The cohort with the answer key
Section titled “The cohort with the answer key”All five pages compute on one synthetic cohort of 400 patients: age, body mass index, activity, smoking, cholesterol and systolic blood pressure, with missingness introduced deliberately into the last four. The complete version of the cohort is kept beside the missing one. That is the honest way to teach imputation, and it is the only way to score it: the evaluation page reports per-cell errors and interval coverage against values the methods never saw, and on real data neither number exists.
The path
Section titled “The path”| Page | What it does |
|---|---|
| Missing Data Mechanisms | MCAR, MAR and MNAR, pattern tables, and the one mechanism question the data can answer |
| Simple Imputation and Its Cost | deletion, mean, median, mode and kNN, each fitted and measured against the benchmark |
| Multiple Imputation with mice and miceforest | m imputations and Rubin’s rules, with the between-imputation disagreement priced into the standard error |
| Choosing the Imputation Model | predictor screening, typed methods, and the measured difference between mean matching and a posterior draw |
| Evaluating Imputation Quality | convergence, distributions, coverage against the truth, and a priced alternative to MAR |
Two engines, one contract
Section titled “Two engines, one contract”The R tabs use mice, the reference implementation of multiple imputation by
chained equations, with VIM for the nearest-neighbour method. The Python tabs
use miceforest, whose mean-matched gradient boosting plays the same role.
Where the two compute the same quantity, they agree to the printed precision.
Where they differ, the page says so and measures the difference, because the
choice of imputation engine is part of the result.
The code for every page lives in the companion repository under
guides/missing-data/, one folder per page, with the container definition and
the fixture generator beside it.