FedDO: Dynamic Client Optimization for Adaptive Federated Learning

Federated Learning (FL) allows multiple devices to collaboratively train machine learning models while keeping their raw data private. However, in real-world scenarios, FL often struggles with unstable optimization and slow convergence. This is largely caused by heterogeneous client data, where both the data distributions and dataset sizes vary significantly across devices. To address these challenges, we propose FedDO, an adaptive federated optimization framework guided by reinforcement learning. Instead of using predefined rules or binary selection, FedDO formulates client coordination as a continuous control problem. Specifically, we employ a sample-efficient Distributional Soft Actor-Critic with Three Refinements (DSAC-T) agent to dynamically allocate a continuous data-usage ratio for each client per round. This fine-grained adjustment effectively mitigates gradient drift. Furthermore, we introduce a low-rank parameterization technique to compress the agent’s action space, ensuring scalability to thousands of clients. Extensive experiments on standard image classification benchmarks show that FedDO consistently achieves higher accuracy and much faster convergence than state-of-the-art FL baselines, especially under highly unbalanced and heterogeneous data settings. Code is available at https: //github.com/leafuan/FedDO_code.