Prompt Tuning — AI Glossary

Reference details

Related terms

• Prefix Tuning
• Adapter (PEFT)
• LoRA (Low-Rank Adaptation)

How it works

1. A small number of trainable vectors is prepended to the input embedding sequence. They occupy positions the model treats as tokens, but they are free parameters rather than entries from the vocabulary.
2. Every model weight stays frozen. Gradients flow back through the network to the prepended vectors only, so training touches a vanishingly small fraction of the parameters.
3. Because they are not vocabulary entries, the vectors can take values no real token has, which is what lets a handful of them encode more task information than any natural-language instruction of the same length.
4. The trained artefact is just those vectors, so a task is a file of a few hundred thousand values rather than a copy of the model.
5. One frozen model can therefore serve many tasks by swapping the prefix per request, which is the deployment property that distinguishes this from weight-modifying methods.
6. Effectiveness rises sharply with model scale: large models reach full fine-tuning quality with a short prefix, while smaller ones need more vectors and still fall short.

Example: The parameter and storage arithmetic that makes this cheap

Take 100 trainable vectors in a model of width 4,096: that is 100 x 4,096 = 409,600 parameters. Against a 7B model, 409,600 / 7e9 = 0.0000585, or 0.0059% of the weights — about one parameter in 17,000. Storage follows: at 4 bytes per value the artefact is 409,600 x 4 = 1,638,400 bytes, roughly 1.64MB, against the 7e9 x 2 = 14e9 bytes, 14GB, that a full fine-tune of the same model produces. That is 14e9 / 1,638,400 = 8,545, so one fine-tune costs the storage of about 8,500 prompt-tuned tasks. The deployment consequence is the real one. A hundred full fine-tunes is 100 x 14 = 1,400GB and a hundred models to load, version and serve; a hundred prefixes is 100 x 1.64 = 164MB against one frozen model, with the prefix selected per request. The trade is quality: at this scale a short prefix is competitive, and on a small model the same 409,600 parameters are a much larger fraction of a much smaller network and still carry less task capacity.

Commonly confused with

Prompt tuning and prefix tuning are both soft-prompt methods and differ in where the learned parameters act. Prompt tuning prepends vectors to the input embeddings only, so the learned signal has to propagate through the whole frozen network from position zero. Prefix tuning inserts learned key and value vectors at every layer, giving it far more capacity and far more parameters. The separate and more common confusion is with prompt engineering: that searches over real tokens a human can read, while this optimises continuous vectors that correspond to no words at all and cannot be inspected or edited as text.

When to use it

Reach for it when:

• Serving many tasks from one frozen model, where swapping a small prefix per request avoids holding many model copies.
• Large models, where a short prefix reaches competitive quality and the method's scale dependence works in your favour.
• Where catastrophic forgetting is a concern, since no weight changes means the base model's general capability is untouched by construction.
• Storage- or memory-constrained multi-tenant deployment, where the artefact is megabytes rather than gigabytes.

Reach for something else when:

• Small models, where the method's effectiveness falls off and more vectors do not close the gap.
• Tasks needing substantial new capability rather than task framing, which a frozen network cannot acquire from a prefix.
• Where the adaptation must be inspectable, since the learned vectors correspond to no readable text.
• When low-rank weight adaptation is available and quality matters more than per-task storage, which is the usual production trade.

Primary source

• Lester et al., The Power of Scale for Parameter-Efficient Prompt Tuning (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Prompt Tuning — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path