ModelRefs / Federated Learning — AI Glossary

Federated Learning — AI Glossary

A distributed training paradigm where each client trains on local data and only sends model updates to a central server, preserving data privacy.

Overview

Federated learning (McMahan et al. 2017, Google) enables training on sensitive data (medical records, financial transactions) without centralizing data. FedAvg aggregates client model gradients. Challenges: non-IID client data distributions, communication overhead, and Byzantine-robust aggregation. Used in Gboard keyboard next-word prediction and clinical NLP.

Reference details

Topic	safety
Last reviewed	2026-06-24

How it works

A central server distributes the current model to participating clients, and the raw data never leaves those clients.
Each client trains locally on its own data and returns only a weight update, not the examples that produced it.
The server aggregates those updates — typically a weighted average by client data volume — into a new global model.
The cycle repeats for many rounds, so the cost is communication rather than a single large transfer.
Updates can still leak information about the data that produced them, which is why differential privacy noise or secure aggregation is normally layered on rather than assumed unnecessary.

Example: Why communication, not compute, is the constraint

A 100M-parameter model at 4 bytes per parameter is 400MB per update. With 1,000 clients and 100 rounds, if a tenth of clients participate per round that is 100 x 100 x 400MB ≈ 4TB of transfer, in both directions. Centralising the data might have meant moving a fraction of that once. The arithmetic is why practical deployments compress updates, sample clients aggressively, and run far fewer rounds than a centralised training loop would use epochs.

Commonly confused with

Read as a privacy guarantee, which it is not on its own. It removes the need to centralise raw data, and the weight updates it does transmit can be inverted to reveal properties of that data — so the privacy claim comes from differential privacy or secure aggregation layered on top, not from the architecture. It is also distinct from distributed training, which splits one dataset across machines you control for speed; here the data is partitioned by ownership and cannot be moved, which is a constraint rather than an optimisation.

When to use it

Reach for it when:

Training data cannot leave its origin for legal or contractual reasons.
Many parties hold complementary data and none will pool it.
A privacy mechanism is layered on, since the updates alone are not private.

Reach for something else when:

The data can be centralised, where ordinary training is far cheaper and simpler.
Communication bandwidth is the binding constraint, which many rounds will exhaust.
As a privacy guarantee by itself, which it does not provide.

Primary source

McMahan et al., Communication-Efficient Learning of Deep Networks from Decentralized Data (arXiv)

Continue your research

Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Federated Learning — AI Glossary.

Frequently asked questions

What is Federated Learning?