Regression Metrics: MSE, MAE, and R²

When a model predicts a number instead of a label, you measure the size of its errors. MSE, MAE, and R² each answer a different question about those errors.

Classification asks “which category”. Regression asks “what number” — a price, a temperature, a delivery time. There is no right-or-wrong; there is only how far off you were. Regression metrics are all different ways of summarizing the distances between predictions and reality.

Each prediction produces a residual: error = actual − predicted. A model predicts a house sells for $420k and it sells for $450k; the residual is $30k. You have one residual per example, and every metric below turns that pile of residuals into a single number.

MAE: the honest average error

Mean absolute error is the average of the absolute residuals:

MAE = mean(|actual − predicted|)

If MAE is $18k, your predictions are off by $18k on average. That is the whole story — MAE is in the same units as the target and reads like plain English. A residual of +$30k and one of −$30k both contribute $30k.

MAE treats all errors proportionally: a $40k miss counts exactly twice a $20k miss. That makes it robust to outliers. One wildly mispriced mansion nudges MAE by its own size and no more, so MAE describes the typical case well.

MSE and RMSE: punish the big misses

Mean squared error averages the squared residuals:

MSE = mean((actual − predicted)²)

Squaring does two things. It removes the sign (so positives and negatives do not cancel), and it makes large errors count disproportionately — a $40k miss contributes four times as much as a $20k miss, not twice. MSE cares intensely about big errors.

The catch is units: MSE for house prices is in “dollars squared”, which is meaningless to read. So people take the square root: RMSE = √MSE, back in dollars and directly comparable to MAE.

The gap between them is diagnostic. Because RMSE inflates large errors, RMSE is always ≥ MAE, and a large gap between them signals a few big outliers dragging the tail. RMSE ≈ MAE means errors are uniform in size.

Choose by what hurts. If being off by $100k once is far worse than being off by $10k ten times — often true for risk, inventory, capacity — use MSE/RMSE, which penalizes the rare disaster. If every dollar of error is equally bad, MAE matches your reality.

R²: are you beating the dumbest baseline

MAE and RMSE are absolute — whether $18k is good depends on the price range. R² (coefficient of determination) makes it relative by comparing your model to the laziest possible predictor: always guessing the mean.

R² = 1 − (model's squared error / mean-guesser's squared error)

• R² = 1: perfect predictions, zero error.
• R² = 0: your model is no better than always guessing the average.
• R² < 0: your model is worse than guessing the average — yes, this happens, usually on data the model never trained on.

R² of 0.85 means the model explains 85% of the variance in the target that the mean-guesser leaves on the table. Its virtue is being unit-free and interpretable across problems; a house-price R² and a temperature R² are on the same 0-to-1 scale.

The trap: R² never decreases when you add features, even useless random ones, because more knobs can only fit the training data tighter. That is why adjusted R², which penalizes extra features, is the honest choice when comparing models with different numbers of inputs — and why R² on the training set is nearly worthless. Report it on held-out test data.

Reporting them together

No single number suffices, for the same reason it never does in classification. A useful default: report RMSE or MAE for the error in real units, and R² for whether the model beats the trivial baseline. If RMSE sits well above MAE, say so — it means outliers, and outliers are usually where the real product risk lives. And always plot residuals against predictions: a metric summarizes, but the plot shows whether errors are random noise (good) or a pattern the model is missing (fixable).

What to remember

• Regression measures the size of residuals (actual − predicted), not right-versus-wrong.
• MAE is the average error in real units — robust to outliers, treats all errors proportionally.
• MSE/RMSE square the errors, so big misses dominate; use them when rare large errors are the real danger. RMSE ≥ MAE always, and a big gap signals outliers.
• R² is relative: 1 is perfect, 0 ties the mean-guesser, negative is worse than it. Use adjusted R² across models with different feature counts, and always on held-out data.
• Report an error metric plus R², and plot the residuals to catch patterns a scalar hides.

Next: Text Generation Metrics