ModelRefs / Product Quantization (PQ) — AI Glossary

Product Quantization (PQ) — AI Glossary

A vector compression technique splitting high-dimensional vectors into sub-vectors quantized independently, reducing memory 16–64×.

Overview

PQ (Jégou et al. 2011) divides a d-dimensional vector into M sub-vectors of d/M dimensions, quantizes each to a codebook of 256 centroids (1 byte per sub-vector). An IVF-PQ index can store 1B 768-dim vectors in ~6 GB vs 3 TB for float32. Asymmetric distance computation enables accurate search despite compression.

Reference details

Topic	rag
Last reviewed	2026-06-24

How it works

Each vector is split into equal contiguous sub-vectors, and the split count is the parameter that sets both the compression ratio and the accuracy cost.
A separate codebook is learned per sub-vector position by clustering that slice across the corpus, conventionally to 256 centroids so that a sub-vector's identity fits in one byte.
A vector is then stored as the sequence of centroid indices for its sub-vectors — one byte each — replacing the original floating-point values entirely.
Search uses asymmetric distance computation: the query stays in full precision and its distance to every centroid is precomputed into a small lookup table, so scoring a stored vector becomes a handful of table lookups and additions instead of arithmetic over the original dimensions.
Keeping the query uncompressed is what preserves accuracy — quantising both sides would compound the error, while quantising only the stored side keeps the query's information intact.
It is normally combined with a coarse partitioning step so that only a fraction of the corpus is scanned, with the quantisation reducing the memory each scanned vector occupies.

Example: Where the compression comes from, and what it costs

A 768-dimensional vector in 32-bit floats occupies 768 x 4 = 3,072 bytes. Split it into 96 sub-vectors of 768 / 96 = 8 dimensions each, quantise each to one byte, and the stored form is 96 bytes: a compression factor of 3,072 / 96 = 32. At a billion vectors that is the difference between 1e9 x 3,072 = 3.072e12 bytes, about 3.07TB, and 1e9 x 96 = 9.6e10 bytes, about 96GB — from a figure no single machine holds in memory to one that a large server does, which is the entire reason the technique exists. The cost is information: 8 original dimensions are replaced by one of 256 possible centroids, so each sub-vector retains at most 8 bits where it previously carried 8 x 32 = 256. Coarser splits compress harder and lose more — 48 sub-vectors of 16 dimensions gives 3,072 / 48 = 64x compression with twice as many dimensions collapsed into each byte. That trade is the only parameter that matters, and it is measured by recall against an exact search on a sample, which is cheap to run and the only way to know where the acceptable point is for a given corpus.

Commonly confused with

Product quantisation compresses stored vectors; it does not reduce the dimensionality of the embedding space, and the two are frequently conflated. A dimensionality reduction produces shorter vectors that are still real-valued and comparable by ordinary arithmetic. Product quantisation keeps the original dimensionality and replaces the values with codebook indices, so distances are approximated through lookup tables. It is also distinct from scalar quantisation, which maps each dimension independently to fewer bits — simpler, less compressive, and without codebooks to train or a partitioning scheme to tune.

When to use it

Reach for it when:

Very large vector sets where memory is the binding constraint and an exact index does not fit.
With a coarse partitioning step, which is the standard pairing and what keeps the scan fraction small.
Where some recall loss is acceptable and measurable against an exact search on a sample.
When the alternative is going to disk, since an approximate in-memory scan generally beats an exact one on storage.

Reach for something else when:

Corpora that fit in memory uncompressed, where exact search is simpler and more accurate.
Workloads requiring exact nearest neighbours, which the approximation does not provide.
Without measuring recall against exact search, since the acceptable split count is corpus-specific and not a default.
On small collections, where the codebook training and partitioning machinery costs more than the memory it saves.

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Product Quantization (PQ) — AI Glossary.

Frequently asked questions

What is Product Quantization (PQ)?