State Space Model (SSM) — AI Glossary

Reference details

Related terms

• Mamba
• Transformer
• Attention Mechanism

How it works

1. A hidden state is maintained and updated once per input position by a linear recurrence, so the per-token cost is constant regardless of position.
2. The formulation is continuous in origin and discretised for sequence data, which is where the specific parameterisations come from and why the models have a principled account of what the state retains over time.
3. With fixed parameters the recurrence is also a convolution, so training can be done in parallel across the sequence by computing that convolution rather than stepping through positions.
4. Making the parameters depend on the input gives the model selectivity and forfeits the convolutional equivalence, so parallel training needs an associative scan instead.
5. At inference the state is the model's entire memory of the past — a fixed number of values, independent of how many tokens have been seen.
6. That constant-size memory is the whole trade: cost and memory stop growing with context, and exact recall of individual earlier tokens stops being available.

Example: Constant memory against linear memory, and what it costs

An attention-based model's key-value cache grows linearly with context. For 32 layers of width 4,096 at 16 bits, the cache is 2 x 2 x 32 x 4,096 bytes per token, which is 524,288 bytes — half a megabyte per token. At 1,000 tokens that is 524,288,000 bytes, about 0.52GB; at 100,000 tokens, 52,428,800,000 bytes, about 52.4GB; at a million, roughly 524GB, which no single device holds. A recurrent state-space model's state is the same size at 1,000 tokens as at a million. That is why these architectures are interesting at all. The cost is an information argument rather than an engineering one: the attention cache is a lossless record of every token, while a constant-size state must compress an arbitrarily long history into a fixed number of values, so as context grows the state necessarily retains less about any particular token. This is the structural reason needle-style exact retrieval is the characteristic weakness, and why hybrid designs reintroduce a few attention layers specifically to restore it.

Commonly confused with

A state-space model is a recurrent architecture, but it is not a classical recurrent network and the difference is what makes it trainable. Earlier recurrent networks were trained by stepping through the sequence, which is sequential and suffers vanishing gradients over long ranges. These models use a linear recurrence with a structured parameterisation, which can be computed in parallel across the sequence — as a convolution when parameters are fixed, as an associative scan when they are input-dependent — and is constructed so that long-range information is retained. Calling them a return to recurrent networks describes the inference shape and misses the reason they work.

When to use it

Reach for it when:

• Long or unbounded sequences, where constant per-token cost and constant memory are the properties that matter.
• Streaming inference, where a fixed state matches the problem and a growing cache does not.
• Audio, time series and genomic data, where sequences are long and exact token-level lookup is rarely the task.
• Hybridised with attention when both long-range efficiency and exact recall are required.

Reach for something else when:

• Short-sequence tasks, where there is no growing cost to avoid.
• Workloads dominated by exact retrieval from context, which a compressed state handles worse than attention.
• Where ecosystem maturity is a constraint, since tooling, quantisation support and serving infrastructure lag.
• Assuming equivalence to a transformer of the same size, since the retrieval behaviour differs and only evaluation reveals by how much.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to State Space Model (SSM) — AI Glossary.

• Browse models
• Review providers
• Inspect benchmarks
• Explore workflows
• Learn AI concepts
• Read decision guides
• Review trust and methodology
• Start a decision path