What Is Machine Learning?

Machine learning is programming by example instead of by rule. Here are the three ways a machine can learn from data, and when each one applies.

Traditional programming is a person writing rules: if the email contains “free money”, mark it spam. Machine learning inverts that. You show the machine thousands of emails already labeled spam or not, and it writes the rule itself.

That inversion is the whole idea. You stop specifying how to get the answer and start supplying examples of the answer. Everything else in this layer — neural networks, gradient descent, the machinery behind every LLM — is detail on how a machine turns examples into a rule.

Why learn instead of program

Some tasks are easy to state and hard to write rules for. Describe, in code, exactly what makes a photo contain a cat. You cannot. There is no clean rule — cats come in every pose, color, and lighting. But you can easily collect ten thousand photos labeled “cat” or “not cat”.

Machine learning fits the situations where examples are cheap and rules are impossible. Handwriting varies too much for rules. Language is too fluid for rules. Fraud patterns shift faster than anyone can rewrite rules. In each case, data carries the pattern that no human can fully articulate.

The flip side: if you can write a simple, reliable rule, do that. Machine learning is overkill for computing sales tax.

Supervised learning: learning from labeled examples

The most common setup. Each training example comes with the right answer attached — a label — and the machine learns to reproduce it.

• Photos labeled with the animal in them → predict the animal in a new photo.
• Houses with their sale prices → predict the price of a house not yet sold.
• Emails labeled spam or not → flag new spam.

Two shapes show up constantly. Classification picks from a fixed set of categories (spam / not spam, which of 10 digits). Regression predicts a number on a continuous scale (a price, a temperature). The distinction matters because it changes how you measure being wrong, which loss functions covers.

Supervised learning dominates practice because labels, when you can get them, are the strongest possible teaching signal. Its cost is exactly that: someone has to produce the labels.

Unsupervised learning: finding structure without answers

Here the data has no labels. The machine gets the examples and must find structure on its own.

Clustering groups similar examples: given customer purchase histories with no categories attached, discover that shoppers fall into a few natural groups. Nobody defined the groups in advance — the algorithm found them.

Another form learns compact representations. An embedding squeezes a high-dimensional input into a shorter vector that keeps what matters. The training signal there is often the data predicting itself: hide part of an input, make the model reconstruct it. That “predict the missing piece” trick is exactly how LLM pretraining works — the text supplies its own labels by asking the model to predict the next token. It looks unsupervised because no human labeled anything, which is why it scales to the entire internet.

Reinforcement learning: learning from consequences

The third mode has no fixed dataset at all. An agent takes actions in an environment and receives a reward signal — points for good outcomes, penalties for bad ones. It learns a strategy that maximizes reward over time.

A game-playing system that gets +1 for winning learns which moves lead to wins, even though no one labeled any individual move as correct. The catch is credit assignment: a reward at the end of a long game has to be traced back to the moves that earned it, which is genuinely hard.

You have met this idea already if you have read about aligning LLMs with human feedback. There, the reward encodes which responses people preferred, and the model adjusts toward producing more of those. Same framework, different environment.

The one rule that governs all three

Whatever the mode, the goal is never to memorize the training examples. It is to generalize — perform well on data the machine has never seen.

A student who memorizes last year’s exam answers learns nothing and fails a new exam. A model that memorizes its training set does the same thing; the failure even has a name, overfitting. Every serious idea in this field — held-out test sets, regularization, the way we measure success — exists to force generalization instead of memorization.

This is why “it got 100% on the training data” is not good news by itself. The only score that matters is the one on data the model was never trained on.

What to remember

• Machine learning writes rules from examples instead of having a person write them by hand. Use it when examples are cheap and rules are impossible.
• Supervised learns from labeled examples (classification and regression); unsupervised finds structure in unlabeled data; reinforcement learns from reward and penalty.
• Modern LLMs are trained by making text predict itself, which needs no human labels and so scales enormously.
• The goal is always to generalize to unseen data, never to memorize the training set.

Next: How Data Becomes Numbers — before a machine can learn anything, the input has to become math.