Preprocessing & Data Leakage

Preprocessing turns raw data into a form a model can use. The single most important rule is to avoid data leakage. This page shows what leakage is, why it matters, and how scikit-learn Pipelines prevent it, while preparing a realistic mixed-type dataset.

Every code block runs in a container in the companion repository, so the numbers are real.

What is data leakage

Data leakage happens when information from the test set influences training. The model looks good in evaluation but fails on truly unseen data. It is subtle: no error, no warning, great metrics, poor generalization.

The classic mistake is scaling before splitting:

# WRONG: the scaler sees all 1000 samples, including the future test set
X_scaled = StandardScaler().fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.2)

The scaler computed its mean and standard deviation from every sample, so the training data is now contaminated with information about the test distribution. The correct order:

# RIGHT: split first, then fit on training data only
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y)

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)  # fit on training only
X_test_scaled = scaler.transform(X_test)         # transform with training stats

The rule: split first, fit on training data only, then transform everything. It applies to scaling, imputation, categorical encoding, feature selection, and PCA.

Train/test split

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

Use stratify=y to preserve the class balance in both sets (essential for imbalanced data), and random_state for reproducibility.

Group awareness

If your data has groups (multiple samples per patient), use GroupShuffleSplit or GroupKFold. The same patient in both train and test leaks, because within-patient variation is smaller than between-patient variation.

Handling missing values

Most algorithms need missing values imputed first.

from sklearn.impute import SimpleImputer

num_imputer = SimpleImputer(strategy="median")          # numeric, robust to outliers
cat_imputer = SimpleImputer(strategy="most_frequent")   # categorical

Strategy	Use when
median	Numeric with outliers
mean	Numeric without strong outliers
most_frequent	Categorical
constant	A specific fill value (0 or “missing”)

IterativeImputer predicts missing values from other features: more accurate, slower.

Scaling numeric features

Distance-based (KNN, SVM) and gradient-based (logistic regression, neural networks) models are sensitive to scale. Tree-based models (random forest, XGBoost, LightGBM, CatBoost) split on values, not distances, so they do not need scaling.

Scaler	Formula	Use when
StandardScaler	(x - mean) / std	Default for most algorithms
MinMaxScaler	(x - min) / (max - min)	Bounded features, neural networks
RobustScaler	(x - median) / IQR	Data with many outliers

Encoding categorical variables

Algorithms need numbers, so categoricals must be converted.

• One-hot encoding creates a binary column per category. Use for nominal categories. OneHotEncoder(drop="first", handle_unknown="ignore") avoids multicollinearity and tolerates unseen categories at test time.
• Ordinal encoding assigns integers in a given order. Use for ordered categories (low/medium/high) with OrdinalEncoder(categories=[["low", "medium", "high"]]).
• Target encoding replaces a category with the mean target for that category. Powerful but leaks target information unless cross-fit. Use scikit-learn’s TargetEncoder, which cross-fits internally.

Building a preprocessing Pipeline

A ColumnTransformer inside a Pipeline chains preprocessing and modeling into one object, so fit only ever touches training data. Here is a realistic dataset with mixed types and injected missing values.

np.random.seed(42)
n = 500
df = pd.DataFrame({
    "age": np.random.normal(55, 15, n).astype(int),
    "lab_value_1": np.random.lognormal(2, 0.5, n),
    "lab_value_2": np.random.normal(100, 20, n),
    "sex": np.random.choice(["M", "F"], n),
    "stage": np.random.choice(["I", "II", "III", "IV"], n),
})
df["target"] = (0.3 * df["age"] + 0.5 * df["lab_value_1"]
                - 0.2 * df["lab_value_2"] + np.random.normal(0, 5, n) > 15).astype(int)
df.loc[np.random.choice(n, 25, replace=False), "age"] = np.nan
df.loc[np.random.choice(n, 15, replace=False), "lab_value_1"] = np.nan
df.loc[np.random.choice(n, 10, replace=False), "stage"] = np.nan
df.isnull().sum()

age            25
lab_value_1    15
lab_value_2     0
sex             0
stage          10
target          0

Now build and fit the pipeline:

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression

numeric_features = ["age", "lab_value_1", "lab_value_2"]
categorical_features = ["sex", "stage"]
X, y = df.drop(columns=["target"]), df["target"]
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y)

numeric = Pipeline([("imputer", SimpleImputer(strategy="median")),
                    ("scaler", StandardScaler())])
categorical = Pipeline([("imputer", SimpleImputer(strategy="most_frequent")),
                        ("encoder", OneHotEncoder(drop="first", handle_unknown="ignore"))])
preprocessor = ColumnTransformer([("num", numeric, numeric_features),
                                  ("cat", categorical, categorical_features)])
pipeline = Pipeline([("preprocessor", preprocessor),
                     ("classifier", LogisticRegression(max_iter=1000, random_state=42))])
pipeline.fit(X_train, y_train)

print(f"Training accuracy: {pipeline.score(X_train, y_train):.4f}")
print(f"Test accuracy: {pipeline.score(X_test, y_test):.4f}")

Training accuracy: 0.9775
Test accuracy: 0.9700

When you call fit(X_train, y_train), the imputers, scaler, and encoder learn their statistics from the training data only, then the model trains on the transformed result. When you call predict(X_test), those same fitted transformers are applied to the test data. No leakage, all automatic.

Common sources of leakage

Source	Problem	Fix
Scaling before splitting	Test stats leak into training	Split first, scale second
Feature selection on full data	Selected using test information	Select inside cross-validation
Target encoding without CV	Target values leak into features	TargetEncoder with internal CV
Duplicate samples across split	Same sample in train and test	Deduplicate before splitting
Random split of time data	Future leaks into past	Use time-based splits
Random split of grouped data	Same patient in both sets	Use GroupKFold

Summary

1. Split into train and test (stratified, group-aware if needed).
2. Build a ColumnTransformer with separate numeric and categorical pipelines.
3. Chain the preprocessor with a model in a Pipeline.
4. Call fit on training data, score/predict on test data.

You never call fit_transform on the full dataset yourself. The Pipeline handles it.

Next steps

A single split gives a noisy estimate. The next page covers Cross-Validation for robust evaluation and tuning.