Identifiable Gated Residual Personalization for Federated Parameter-Efficient Fine-Tuning

Federated parameter-efficient fine-tuning (PEFT) enables scalable adaptation of large pre-trained backbones under communication constraints. In heterogeneous federated settings, personalization is commonly realized through gated residual mixing between shared and clientspecific branches. However, this parameterization is inherently scale nonidentifiable: only the product of residual magnitude and mixing weight determines the functional contribution. When private residuals are optimized locally, their scales may drift across clients and communication rounds, rendering mixing coefficients an unreliable measure of personalization strength. We address this limitation with FedSDG, a structuredecoupled gating framework for federated PEFT. FedSDG maintains separate shared and private LoRA branches and introduces projection-level scalar gates for depth-aware personalization. Crucially, we propose Dynamic Alignment, a backbone-anchored calibration mechanism that stabilizes private residual scale prior to gated mixing, thereby improving gate identifiability without additional communication. Extensive experiments demonstrate consistent personalization improvements and stable depth-wise adaptation across diverse non-IID regimes.