The Forward Pass

The forward pass is how a network turns an input into an output: layer by layer, each computing weighted sums and activations. This is inference in one word.

The forward pass is the act of running an input through a network to get an output. Data enters at the first layer, flows forward through each layer in turn, and a prediction comes out the end. It is the network doing its job — and it is exactly what happens every time you send a prompt to an LLM.

One layer at a time

Start with your feature vector. The forward pass processes it through the layers in strict order.

Each layer does two things you have already met. It computes a weighted sum for every neuron — each input times its weight, summed, plus a bias. Then it applies an activation function to each of those sums. The result is a new vector, which becomes the input to the next layer.

That is the entire loop:

vector = input
for each layer:
    sums   = weights · vector + biases   # the weighted sums
    vector = activation(sums)            # the nonlinearity
output = vector

There is no branching, no backtracking, no cleverness. Information moves in one direction — forward — which is where the name comes from. A “deep” network just means this loop runs many times before the output appears.

A concrete trace

Say a tiny network judges whether an email is spam from two features: number of links and number of ALL-CAPS words, given as [4, 7].

The first hidden layer has, say, three neurons. Each computes its own weighted sum of [4, 7] with its own weights and bias, then squashes it. Maybe the layer emits [0.9, 0.1, 0.8] — three learned intermediate signals. You do not get to name what they mean; perhaps the first neuron responds to “lots of links,” but the network chose that, not you.

That [0.9, 0.1, 0.8] feeds the next layer, which combines the three signals into a final neuron whose output is 0.94. Interpret that as “94% spam-like.” One number, produced by a fixed chain of multiplications and additions running left to right. Change any weight and the output changes; the weights are the whole story, the forward pass just executes them.

The output layer speaks the task’s language

The final layer is shaped to the question. A regression predicting a price ends in a single neuron with no squashing, so it can output any number. A binary classifier ends in one neuron squashed to the 0-to-1 range, read as a probability. A multi-class classifier ends in one neuron per class, and their scores get turned into a probability distribution that sums to 1 — the mechanism there is softmax, and it is exactly how an LLM turns its final layer into a probability for every possible next token.

So the shape of the forward pass’s ending is not incidental. It is what makes the output mean something you can use.

Forward pass is inference

When a trained model is deployed and answering requests, all it is doing is forward passes. This is why running a model is called inference — the network infers an output from an input, weights held fixed.

For an LLM, generating text is a forward pass per token: run the whole sequence forward, get a probability distribution over the vocabulary, sample one token, append it, and run forward again. That repetition is autoregressive generation. Each individual step is nothing more than the loop above, which is why the cost of a forward pass — how many weights it must read — drives the cost of every generated token.

What the forward pass cannot do alone

The forward pass computes an output. It does not, by itself, know whether that output is any good, and it does not improve anything. A freshly initialized network with random weights runs a perfectly valid forward pass and produces perfectly useless output.

Turning useless into useful requires two more pieces: a loss function to measure how wrong the output is, and a way to push the weights in a better direction — gradient descent driving backpropagation, which runs backward through the same layers. Training alternates a forward pass to see what the network does with a backward pass to fix it. But at deployment, only the forward direction ever runs.

What to remember

• The forward pass runs an input through the layers in order, each computing weighted sums then activations, producing an output.
• It is strictly one-directional — no branching, no backtracking — and “deep” just means many iterations of the same step.
• The output layer’s shape encodes the task: one number for regression, a probability for binary, a distribution for multi-class.
• Running a trained model — inference — is nothing but forward passes; each generated LLM token is one such pass.
• The forward pass alone cannot learn; measuring and fixing errors needs a loss function and backpropagation.

Next: Activation Functions — the nonlinear step that makes deep networks more than the sum of their layers.