Precision, Recall, F1, and Accuracy

Four numbers that describe a classifier, and the specific question each one answers. Pick the wrong one and you optimize for the wrong outcome.

A classifier says yes or no. Every prediction it makes lands in one of four buckets, and every classification metric is just a different ratio of those four counts. Learn the buckets and the metrics stop being formulas to memorize.

Say you built a model that flags fraudulent transactions. For each transaction it predicts fraud or not, and reality is fraud or not. That gives four outcomes:

• True positive (TP): flagged fraud, and it was fraud.
• False positive (FP): flagged fraud, but it was legitimate.
• False negative (FN): passed it as legitimate, but it was fraud.
• True negative (TN): passed it, and it was legitimate.

Accuracy: the one that lies

Accuracy is the fraction of predictions that were correct: (TP + TN) / everything.

It is the metric everyone reaches for first, and it is misleading exactly when the classes are imbalanced. If 1 transaction in 1000 is fraud, a model that flags nothing scores 99.9% accuracy while catching zero fraud. The number is nearly perfect and the model is worthless.

Accuracy answers “what fraction did I get right” — a fine question only when the classes are balanced and the two kinds of error cost the same. In fraud, medical screening, spam, and most real problems, neither condition holds. That failure is visible the moment you look at a confusion matrix.

Precision: how much of what you flagged was real

Precision is TP / (TP + FP) — of everything the model called fraud, what fraction actually was.

High precision means few false alarms. You care about precision when acting on a positive is expensive or annoying: freezing a customer’s card, sending a security team, auto-deleting an email. A precision of 0.7 means 30% of your alerts are wasting someone’s time.

Recall: how much of the real thing you caught

Recall is TP / (TP + FN) — of all the actual fraud, what fraction did the model catch.

High recall means few misses. You care about recall when a missed positive is the expensive outcome: undetected fraud, a tumor read as clear, a critical security alert dropped. A recall of 0.7 means 30% of real fraud sailed through.

The tradeoff is the whole point

Precision and recall pull against each other. Flag more aggressively and you catch more real fraud (recall up) but raise more false alarms (precision down). Flag conservatively and the reverse. You move along this curve by adjusting the decision threshold — the score above which you call something positive.

There is no universally correct balance. It depends on the relative cost of the two errors:

• A cancer screen chooses recall. A missed tumor is far worse than a false alarm that a follow-up test clears.
• A spam filter chooses precision. Deleting one real email is worse than letting some spam through.

Deciding this ratio is a product decision, not a modeling one. The metrics only make the tradeoff explicit.

F1: one number when you must have one

Sometimes you need a single score to rank models. F1 is the harmonic mean of precision and recall:

F1 = 2 * (precision * recall) / (precision + recall)

The harmonic mean, unlike a plain average, punishes imbalance. Precision 1.0 and recall 0.0 averages to 0.5 but gives an F1 of 0 — which is correct, because a model that catches nothing is useless no matter how clean its (empty) alerts are. F1 is high only when both are high.

F1 weights precision and recall equally. When they are not equally important, Fβ tilts the balance — F2 weights recall higher, F0.5 weights precision higher — but at that point you are usually better served by reporting precision and recall separately and letting the reader see the tradeoff directly.

Reading the four together

No single number is the answer. A useful habit: state precision and recall as a pair, and only collapse to F1 or accuracy when a scalar is structurally required (ranking runs, a dashboard tile). And always check the class balance first — it tells you immediately whether accuracy is safe to trust or a trap. For the visual that makes all of this concrete, and the derived rates like specificity and false-positive rate, go to the confusion matrix.

What to remember

• Every prediction is a TP, FP, FN, or TN; every metric is a ratio of those four.
• Accuracy misleads under class imbalance — a do-nothing model can score near-perfect.
• Precision = of what you flagged, how much was real; optimize it when false alarms are costly.
• Recall = of what was real, how much you caught; optimize it when misses are costly.
• Precision and recall trade off via the threshold; F1 (harmonic mean) collapses them to one number that stays low unless both are high.

Next: The Confusion Matrix