Federated Learning: Train Models on Decentralized Data

WHY IT EXISTS Traditional machine learning requires a large, centralized dataset. But what if that data is private, sensitive, or too large to move? Think of medical records at different hospitals or user data on millions of phones. Centralizing it is a privacy, legal, and logistical nightmare. Federated learning was created to train a single, powerful model without ever moving or centralizing the raw data.

THE MENTAL MODEL Instead of bringing all the data to the model for training, you bring the model to the data. Imagine a team of apprentice chefs. Instead of shipping all the secret family recipes to a central kitchen, the head chef sends each apprentice a basic cooking technique. The apprentices practice on their local family recipes, then report back on what they learned. The head chef combines this feedback to improve the master technique for everyone. The recipes never leave their home kitchens.

HOW IT WORKS The process is a cycle. First, a central server initializes a global model and sends a copy to a selection of clients, like mobile devices. Second, each client trains this model on its own local data, creating a personalized model update. Third, clients send only these updates—the changes to the model, not their private data—back to the server. Finally, the server aggregates these updates, often by averaging them, to create a new, improved global model. This cycle repeats, progressively improving the model for all participants.

WHEN TO USE IT Use it when your training data is naturally decentralized and cannot or should not be brought to a central location. This is common in scenarios involving user privacy (training keyboard prediction on phones), data sovereignty (training models across different countries with strict data laws), or collaboration between competitive entities (training a fraud detection model across multiple banks).

WHEN NOT TO USE IT Don't use it if you can easily and legally centralize your data. Centralized training is faster, simpler, and avoids the core challenge of federated learning: statistical heterogeneity. If the data on different clients is not independently and identically distributed (non-IID), the model can struggle to converge or become biased. For example, if one group of users only types in emojis, their updates could harm the model's performance for users who type full sentences. The communication overhead can also be a bottleneck.

ONE CANONICAL EXAMPLE Google's Gboard keyboard uses federated learning to improve its next-word prediction. The base model is sent to your phone and learns from your typing patterns locally. The learned improvements (the model updates) are then sent back to Google's servers, aggregated with updates from millions of other users, and used to create a better global prediction model for everyone. Your actual conversations never leave your phone.