What Does "7B Parameters" Mean?
Parameters are the adjustable numbers that hold everything a model learned. Why the count matters, and why it stopped being the whole story.
On this page
Model names carry numbers: 7B, 70B, 405B. The B is billion, and the quantity being counted is parameters — individual adjustable numbers inside the model.
A 7B model contains seven billion of them. Every single one was set by training, and together they constitute everything the model knows and can do.
What a parameter actually is
A parameter is one number used in one arithmetic operation.
Inside the model, computation is mostly matrix multiplication: take a vector, multiply it by a large grid of numbers, get a new vector. Those grid entries are the parameters. Every embedding value, every attention weight matrix, every feedforward layer is made of them.
At the start of training they are random and the output is noise. Then billions of small adjustments accumulate, each nudging parameters toward better next-token prediction, and eventually the same numbers produce coherent language.
There is no separate database of facts. The parameters are the knowledge, stored as statistical structure rather than retrievable records. This is why a model cannot cite a source — there is no source, only the residue of patterns.
Why the count matters
More parameters means more capacity to represent patterns. Empirically, larger models handle multi-step reasoning better, hold more factual detail, follow instructions more reliably, and cope with unusual phrasing.
The effect is not gradual everywhere. Some capabilities were essentially absent in smaller models and present in larger ones with no visible ramp between — which is what drove the field to keep scaling.
Why the count is not the whole story
Parameter count alone stopped being a good quality predictor, for several reasons.
Data quality and quantity matter comparably. A model can be too large for its training data, carrying capacity it never fills usefully. Well-trained smaller models routinely outperform badly-trained larger ones.
Post-training does heavy lifting. Instruction tuning and preference training substantially change usefulness without touching parameter count.
Not all parameters are active. Mixture-of-experts architectures hold many parameters but route each token through only a fraction. A model advertising 400B total may activate 40B per token, which changes the cost picture entirely. Total and active counts answer different questions.
Architecture and training recipe vary. Two models at identical size can differ substantially in capability.
What the count does predict: cost
Parameter count maps directly to resource requirements, which is why it matters practically.
Memory. Every parameter must be held during inference. At 2 bytes per parameter, a 7B model needs roughly 14GB and a 70B model roughly 140GB — before the KV cache, which grows with context length.
Speed. Each generated token requires reading every active parameter. More parameters means slower generation and higher cost per token.
Feasibility. Whether a model fits on one consumer GPU, one server, or requires a cluster follows directly from this number.
Quantization stores parameters at lower precision, shrinking memory and speeding inference at some quality cost. This is what makes larger models runnable on modest hardware.
Reading model names
Llama-3.1-70B-Instruct decomposes as family, version, parameter count, and post-training stage. The Instruct suffix indicates instruction tuning; its absence usually indicates a base model.
Closed models frequently do not publish counts. The tier names a provider offers — small, medium, large — are the practical proxy, and Which Model Should You Use? covers choosing by task rather than size.
What to remember
- A parameter is one adjustable number; collectively they hold everything the model learned.
- No separate fact database exists — knowledge is diffused into the parameters.
- Count predicts cost reliably and quality only loosely; data, post-training, and architecture matter comparably.
- Mixture-of-experts models distinguish total from active parameters.