Skip to content
Bench to Insights
Blog

Simple Imputation and Its Cost

Every simple method for missing data does the same thing: it fabricates exactly one value per hole and then prices that value as real data. Deletion fabricates nothing but throws away patients. The mean fabricates the average. kNN borrows from similar patients. Each one is defensible in some situation, and the only way to see what each costs is to run them all on data where the truth is known, which is what the section’s cohort exists for.

The analysis model for this page regresses systolic blood pressure on age, body mass index, cholesterol and smoking. The benchmark is the fit on the complete cohort, the answer you would have reported had nothing gone missing.

The benchmark

Section titled “The benchmark”
#' Fit the analysis model.
#'
#' @param frame A data frame with the model variables.
#' @return An `lm` model fit.
FitModel <- function(frame) {
  lm(sbp ~ age + bmi + chol + smoking, data = frame)
}


#' Print the coefficient table of a model fit.
#'
#' @param fit An `lm` model fit.
#' @return The function returns no value.
PrintCoefficientTable <- function(fit) {
  print(coef(summary(fit)))
}


#' Get the requested coefficient estimate and standard error.
#'
#' @param fit An `lm` model fit.
#' @param term The coefficient name.
#' @return A named numeric vector with estimate and standard error.
GetCoefficient <- function(fit, term) {
  coefficient_table <- coef(summary(fit))
  c(
    estimate = coefficient_table[term, "Estimate"],
    standard_error = coefficient_table[term, "Std. Error"]
  )
}


complete <- read.csv("/opt/data/cohort_complete.csv", na.strings = "")
complete_fit <- FitModel(complete)
cat(sprintf("Complete data, n = %d:\n", nrow(complete)))
PrintCoefficientTable(complete_fit)
Complete data, n = 400:
              Estimate Std. Error   t value     Pr(>|t|)
(Intercept) 61.3338564 4.47833408 13.695686 3.300582e-35
age          0.8968146 0.05534082 16.205299 1.202286e-45
bmi          0.7016504 0.16599812  4.226858 2.946496e-05
chol         3.0369699 0.82988164  3.659522 2.869879e-04
smoking      8.9983663 1.23561947  7.282474 1.790082e-12

The complete cohort puts the cholesterol coefficient at 3.0370 mmHg per mmol/L, with a standard error of 0.8299 on 400 patients. That pair of numbers is the target every method below is trying to reproduce. Both engines solve the same least squares on the same file, and their tables agree to the fourth decimal.

Doing nothing: complete cases

Section titled “Doing nothing: complete cases”

The default in both languages is deletion. R’s lm() drops incomplete rows without printing a word about it; pandas makes you call dropna() yourself, which is the one honest difference between the two tabs here.

missing <- read.csv("/opt/data/cohort_missing.csv", na.strings = "")
# lm() deletes incomplete rows silently, which is the teaching point.
complete_case_fit <- FitModel(missing)
cat(sprintf("Complete cases, n = %d:\n", nobs(complete_case_fit)))
PrintCoefficientTable(complete_case_fit)
Complete cases, n = 238:
              Estimate Std. Error   t value     Pr(>|t|)
(Intercept) 61.4389269 6.41566564  9.576392 1.570032e-18
age          0.9483898 0.07365414 12.876260 5.188704e-29
bmi          0.5947483 0.22530261  2.639776 8.856386e-03
chol         3.2596195 1.17393194  2.776668 5.938539e-03
smoking      7.2167135 1.65108726  4.370886 1.864772e-05

The estimate itself survives: 3.2596 against the benchmark 3.0370, well inside the noise you would expect from losing patients under the MAR pattern this cohort carries. The cost shows up in the standard error, which grows from 0.8299 to 1.1739 because 162 patients were discarded. Deletion did not lie about the effect; it paid for honesty with precision, and on a smaller or less balanced cohort the same choice moves the point estimate too.

The mean

Section titled “The mean”

Mean imputation is the first method everyone reaches for and the first one that misbehaves. Fill each gap with the available mean and the sample size returns to 400, which is exactly the problem: the model now believes it measured 400 cholesterol values.

mean_imputed <- missing
mean_imputed$chol[is.na(mean_imputed$chol)] <- mean(
  mean_imputed$chol,
  na.rm = TRUE
)
mean_imputed$sbp[is.na(mean_imputed$sbp)] <- mean(
  mean_imputed$sbp,
  na.rm = TRUE
)
mean_imputed$smoking[is.na(mean_imputed$smoking)] <- mean(
  mean_imputed$smoking,
  na.rm = TRUE
)
cat(sprintf(
  "Cholesterol SD before = %.4f, after = %.4f\n",
  sd(missing$chol, na.rm = TRUE),
  sd(mean_imputed$chol)
))
imputed_smoking <- mean_imputed$smoking[is.na(missing$smoking)]
impossible_smoking_share <- mean(
  imputed_smoking != 0 & imputed_smoking != 1
) * 100
cat(sprintf(
  "Imputed smoking values not 0 or 1 = %.1f%%\n",
  impossible_smoking_share
))
mean_fit <- FitModel(mean_imputed)
cat(sprintf("Mean imputation, n = %d:\n", nobs(mean_fit)))
PrintCoefficientTable(mean_fit)
Cholesterol SD before = 0.8597, after = 0.7900
Imputed smoking values not 0 or 1 = 100.0%
Mean imputation, n = 400:
              Estimate Std. Error   t value     Pr(>|t|)
(Intercept) 74.5631856 4.82175950 15.463896 1.624731e-42
age          0.8602981 0.05393135 15.951727 1.426579e-44
bmi          0.4931440 0.16776003  2.939580 3.479656e-03
chol         1.7942299 0.87274732  2.055841 4.045455e-02
smoking      7.5074543 1.34449266  5.583857 4.382617e-08

Three failures, all visible in one run. The cholesterol coefficient collapses from 3.0370 to 1.7942, because 62 patients now sit at the mean and contribute no variation at all. The cholesterol standard deviation shrinks from 0.8597 to 0.7900, the direct measurement of that lost variation. And every one of the 48 imputed smoking values is a number that is not 0 and not 1, a patient who smokes 29 percent of a cigarette. The standard error on the cholesterol term, 0.8727, pretends the information came back with the rows. Mean imputation does not just bias the answer; it makes the wrong answer look well measured.

Median and mode

Section titled “Median and mode”

The median is the same idea for skewed variables, and the mode is the same idea for categories. Neither fixes what is wrong with the mean.

median_imputed <- missing
median_imputed$chol[is.na(median_imputed$chol)] <- median(
  median_imputed$chol,
  na.rm = TRUE
)
median_fit <- FitModel(median_imputed)
median_chol <- GetCoefficient(median_fit, "chol")
cat(sprintf(
  "Median imputation: chol coefficient %.4f (SE %.4f)\n",
  median_chol["estimate"],
  median_chol["standard_error"]
))


#' Print category counts and percentages.
#'
#' @param frame A data frame that contains the category column.
#' @param column The category column name.
#' @return The function returns no value.
PrintDistribution <- function(frame, column) {
  counts <- table(frame[[column]], useNA = "no")
  distribution <- data.frame(
    count = as.integer(counts),
    percent = as.numeric(prop.table(counts) * 100),
    row.names = names(counts)
  )
  print(round(distribution, 4))
}


mode_imputed <- missing
observed_activity <- mode_imputed$activity[!is.na(mode_imputed$activity)]
activity_mode <- names(sort(table(observed_activity), decreasing = TRUE))[1]
cat("Activity distribution before mode imputation:\n")
PrintDistribution(mode_imputed, "activity")
mode_imputed$activity[is.na(mode_imputed$activity)] <- activity_mode
cat("Activity distribution after mode imputation:\n")
PrintDistribution(mode_imputed, "activity")
Median imputation: chol coefficient 2.9343 (SE 1.0881)
Activity distribution before mode imputation:
       count percent
high     118 33.5227
low       96 27.2727
medium   138 39.2045
Activity distribution after mode imputation:
       count percent
high     118    29.5
low       96    24.0
medium   186    46.5

The median row fills only cholesterol, so the fit still loses every patient missing blood pressure or smoking and runs on 300 rows; the coefficient lands at 2.9343, better than the mean and still short of the truth. The mode fill shows the categorical version of the disease. Medium was the modal category at 39.2 percent of observed patients, and after the fill it is 46.5 percent, because all 48 patients with unknown activity are now declared medium. One fabricated label per hole has measurably rewritten the distribution.

Borrowing from similar patients: kNN

Section titled “Borrowing from similar patients: kNN”

kNN imputation fills a gap with the average of the k most similar observed patients, so the fabricated value at least respects the data’s local structure. The engines do it differently: VIM picks donor patients and lets a factor variable be voted on, while scikit-learn averages the five nearest rows of a standardised matrix, which is why the Python tab standardises first (blood pressure on its scale would dominate every distance) and rounds the imputed smoking values back to 0 or 1 afterwards.

suppressPackageStartupMessages(library(VIM))
knn_frame <- missing[c("age", "bmi", "smoking", "chol", "sbp")]
knn_frame$smoking <- factor(knn_frame$smoking, levels = c(0, 1))
set.seed(20260820)
knn_imputed <- VIM::kNN(
  knn_frame,
  variable = c("chol", "sbp", "smoking"),
  k = 5,
  imp_var = FALSE
)
knn_fit <- FitModel(knn_imputed)
knn_chol <- GetCoefficient(knn_fit, "chol")
cat(sprintf(
  "kNN imputation (k = 5): chol coefficient %.4f (SE %.4f), n = %d\n",
  knn_chol["estimate"],
  knn_chol["standard_error"],
  nobs(knn_fit)
))
kNN imputation (k = 5): chol coefficient 3.6512 (SE 0.8140), n = 400

Both engines come back near the truth, 3.6512 in R and 3.0271 in Python against the benchmark 3.0370, and the two rows differ because the two nearest-neighbour methods are genuinely different algorithms. This is the best simple method on the page, and its standard error of about 0.81 is still too small: 400 rows were priced, but only 338 cholesterol values were ever measured.

The scoreboard

Section titled “The scoreboard”
scoreboard <- data.frame(
  method = c("truth", "complete cases", "mean", "median", "kNN"),
  chol_coefficient = c(
    GetCoefficient(complete_fit, "chol")["estimate"],
    GetCoefficient(complete_case_fit, "chol")["estimate"],
    GetCoefficient(mean_fit, "chol")["estimate"],
    GetCoefficient(median_fit, "chol")["estimate"],
    GetCoefficient(knn_fit, "chol")["estimate"]
  ),
  chol_standard_error = c(
    GetCoefficient(complete_fit, "chol")["standard_error"],
    GetCoefficient(complete_case_fit, "chol")["standard_error"],
    GetCoefficient(mean_fit, "chol")["standard_error"],
    GetCoefficient(median_fit, "chol")["standard_error"],
    GetCoefficient(knn_fit, "chol")["standard_error"]
  ),
  n_used = c(
    nobs(complete_fit),
    nobs(complete_case_fit),
    nobs(mean_fit),
    nobs(median_fit),
    nobs(knn_fit)
  )
)
cat("Cholesterol coefficient by method:\n")
print(transform(
  scoreboard,
  chol_coefficient = round(chol_coefficient, 4),
  chol_standard_error = round(chol_standard_error, 4)
), row.names = FALSE)
Cholesterol coefficient by method:
         method chol_coefficient chol_standard_error n_used
          truth           3.0370              0.8299    400
 complete cases           3.2596              1.1739    238
           mean           1.7942              0.8727    400
         median           2.9343              1.0881    300
            kNN           3.6512              0.8140    400

Read the last two columns together. Every method except deletion reports n = 400, yet the only honest n in the table is 238. The methods that restore the rows all restore them with fabricated values, and all of them charge less standard error than the information in the data can pay for. kNN gets the point estimate right and still underprices the uncertainty. That gap between a good estimate and an honest standard error is what multiple imputation exists to close, and it is the next page.