The Raw Dataset
Before running kNN we must understand what we have. This dataset records student attributes and whether they passed a course. Three problems must be fixed before kNN can run: missing values, categorical text features, and features on very different scales.
Why does kNN need clean, numeric, scaled data?
kNN works by computing the Euclidean distance between data points in feature space. Distance is undefined for text (
kNN works by computing the Euclidean distance between data points in feature space. Distance is undefined for text (
major = "CS"), impossible when a value is missing, and misleading when features have wildly different ranges (a score of 85 vs. 0.85 study hours).
Dataset — 100 students, 5 features + 1 target
◆ Missing
◆ Categorical
◆ Target
Feature types
Problems to fix
Missing values: study_hours and attendance have missing entries — kNN cannot compute distances with NaN.
Categorical feature: major (CS / Math / Business) — must be converted to numbers.
Scale imbalance: prev_score ranges 40–100 while study_hours ranges 1–9 — large-scale features dominate the distance.
Train / Validation Split
Hold back a portion of the data before training. The model never sees validation points during fitting — they are used only to measure how well the chosen k generalises to unseen examples.
20 %
for validation
80 training points · 20 validation points
Step 1 — Imputation of Missing Values
Missing values must be filled before computing distances. Common strategies: replace with the column mean (for normally distributed numeric data), the median (robust to outliers), or the mode (most frequent value, suitable for skewed or discrete data).
Choose imputation strategy
Column statistics for imputed features
Data table — missing cells highlighted (green = filled)
Step 2 — Encoding Categorical Variables
kNN requires all features to be numeric. For the
major column (CS / Math / Business) we have three options: Label Encoding assigns integers (0, 1, 2) but implies an ordering that doesn't exist. One-Hot Encoding creates a binary column per category — no false ordering. Frequency Encoding replaces each category with its relative frequency in the dataset — compact and ordinal-free, but loses distinction when categories share a frequency.
Encoding method
Transformation: major → numeric
Data table after encoding (purple = encoded columns)
Step 3 — Feature Normalization
Even with all-numeric features, kNN is sensitive to scale. A feature with large values contributes more to Euclidean distance than a feature with small values, regardless of its actual importance. Normalization rescales all features to the same range.
Normalization method
Distance distortion — before vs after
Normalized feature ranges (blue = normalized)
The kNN Algorithm
All preprocessing is done. Click anywhere on the scatter plot to place a query point and watch kNN find the k nearest neighbors and vote on a prediction. Change k, swap features on the axes, and toggle whether distance uses the displayed features or all features simultaneously.
k
5
Neighbors
to vote
to vote
— train ·
— val
Click on the plot to place a query point and get a prediction
Validation Accuracy @ k = 5
—
go to kNN Algorithm step to compute
Prediction
Click the plot
k Nearest Neighbors
Place a query point to see neighbors
Distance formula
d(a,b) = √ Σᵢ (aᵢ – bᵢ)²
Computing Euclidean distance across all 6 normalized features.
Complete preprocessing pipeline — what kNN actually sees
Step 6 — Model Evaluation
After tuning k on the validation set, we measure how well the model generalises using four metrics.
All metrics are computed from kNN predictions on the held-out validation points — data the model never saw during training.
Confusion Matrix
Each cell counts validation predictions by actual vs predicted class.
Positive class = Pass.
| Predicted | ||
|---|---|---|
| Pass | Fail | |
| Actual Pass | — | — |
| Actual Fail | — | — |
TP = correctly predicted Pass
TN = correctly predicted Fail
FP / FN = errors
Accuracy
—
(TP + TN) / total
What fraction of all predictions were correct? Simple and intuitive, but misleading when classes are imbalanced — a model that always predicts the majority class can have high accuracy.
Precision
—
TP / (TP + FP)
Of all students the model predicted as Pass, how many actually passed? High precision means few false alarms — useful when the cost of a false positive is high.
Recall
—
TP / (TP + FN)
Of all students who actually passed, how many did the model catch? High recall means few misses — important when failing to identify a positive case is costly.
F1-Score
—
2 · Prec · Rec / (Prec + Rec)
Harmonic mean of precision and recall. Balances both concerns — a single number that penalises models that are good at one but not the other.
Precision vs Recall — the tradeoff
Precision and recall pull in opposite directions. Predicting Pass more liberally increases recall (fewer misses) but lowers precision (more false alarms). The right balance depends on the application: in medical screening you want high recall; in spam filtering you want high precision. F1 provides a single balanced summary when you care about both equally.
Tip: Use the k-slider in Step 5 to tune k on the validation set, then come back here to see how each metric changes. The goal is to find k where the model generalises well — not just memorises training points.