Federated-Learning Localism
From the AI Research Compass wiki
Federated-Learning Localism: Data should stay where it is generated; models travel to devices and institutions, learn locally, and only share updates.
Beliefs
Federated-Learning Localism starts from respect for where data lives. Phones, hospitals, banks and factories hold valuable data that cannot or should not be centralized, for reasons of privacy, regulation, bandwidth or plain ownership. So instead of bringing data to the model, bring the model to the data: each client trains on what it has and sends back only model updates, which a server aggregates into a shared model. Add secure aggregation and differential privacy, and the center learns from everyone without seeing anyone.
It sits well toward World because its home turf is data generated by devices and sensors in the physical world (keyboards, cameras, medical instruments), and just below the middle, on the Harness side, because the intelligence emerges from a distributed protocol across many clients rather than one centrally trained model. Its paradigm level is ultra-revolutionary: though well established in industry and research, it rejects the mainstream habit of pooling data into one central model.
Arguments
- Federated averaging showed that deep networks can be trained on decentralized, non-identically distributed client data with far fewer communication rounds than naive approaches.
- Production deployments such as on-device keyboard prediction demonstrated federated learning at real-world scale.
- Keeping raw data local reduces breach risk and eases compliance in regulated domains such as healthcare.
Critiques
- Model updates can still leak information about training data, so privacy requires additional defenses that cost accuracy.
- Heterogeneous clients and skewed data distributions make convergence slower and models less accurate than centralized training.
- Scale advocates note that federated settings struggle to train today’s very large models, limiting the approach to smaller ones.
Representative works
- Communication-Efficient Learning of Deep Networks from Decentralized Data2016
- Advances and Open Problems in Federated Learning2019
Relationships
Friends
Frenemies
How to get it
Land on this cell in the full or pro test: x ≈ +0.73, y ≈ −0.42, paradigm ≈ +0.69.