AnaPFL: When Closed-Form Solutions Meet Generalizationand Personalization in Personalized Federated Learning

Personalized Federated Learning (PFL) has emerged as aprevalent paradigm to deliver personalized models to individual clientsthrough collaborative training. Existing PFL methods often suffer fromthe issue of Non-IID data, due to their reliance on gradient-based up-dates. Recently, Analytic Learning (AL) has exhibited great potential toaddress this issue via analytical (i.e., closed-form) solutions in a gradient-free manner. However, there remains a significant gap in introducingAL into PFL, owing to the encountered generalization-personalizationdilemma. In this paper, to bridge this gap and address the associatedchallenges, we propose an Analytic Personalized Federated Learningapproach, named AnaPFL, for addressing the Non-IID issue in PFL byintroducing and advancing AL. In AnaPFL, we develop dual-stream an-alytic models with closed-form solutions, including (1) a shared primarystream for global generalization across all clients, and (2) a dedicated re-finement stream for local personalization of each client. Experimentally,we give comprehensive results to show AnaPFL’s superior performancewith over 99% efficiency advantages against gradient-based methods.