Vocabulary Size — AI Glossary

Reference details

Related terms

• Byte-Pair Encoding (BPE)
• Tiktoken
• Embedding Dimension

How it works

1. The vocabulary is the fixed set of tokens the tokeniser can emit, learned once before training and unchangeable afterwards.
2. A larger vocabulary covers more text with single tokens, so the same content costs fewer tokens — which matters most for scripts and languages the smaller vocabulary fragments.
3. It sets two parameter matrices directly: the input embedding table and the output projection, each of size vocabulary times hidden dimension.
4. Those matrices grow linearly with it, so vocabulary is a real contributor to parameter count independent of depth or width.
5. The output projection is also computed every decoding step, so vocabulary size appears in per-token inference cost as well as in memory.

Example: What the embedding table costs

At a hidden dimension of 4,096, a 50,000-token vocabulary gives an embedding table of 50,000 x 4,096 ≈ 205M parameters, and the output projection another 205M if untied — about 410M before any transformer layer exists. Raising the vocabulary to 128,000 makes that 128,000 x 4,096 ≈ 524M each, so roughly 1.05B. The gain is fewer tokens per unit of text, especially outside English; the cost is 640M extra parameters that do no reasoning. Tying input and output matrices halves it.

Commonly confused with

Confused with context length, which is unrelated: vocabulary is how many distinct tokens exist, context is how many can be processed at once. A large vocabulary makes each token cover more text, so it indirectly stretches how much content fits a fixed context — but the two are separate parameters. It is also not model size, though it contributes: a model can grow its vocabulary without any additional reasoning capacity, which is why comparing parameter counts across families with different vocabularies slightly favours the one with the smaller vocabulary.

When to use it

Reach for it when:

• Explaining why the same text costs different token counts across model families.
• Reasoning about multilingual efficiency, where fragmentation is the cost.
• Accounting for parameters that sit in embeddings rather than in transformer layers.

Reach for something else when:

• As a measure of capability — a larger vocabulary adds coverage, not reasoning.
• Interchangeably with context length, which is a different limit.
• Comparing parameter counts across families without noting the vocabulary difference.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Vocabulary Size — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path