Privacy-Aligned Personalized Federated Learning with Compact Adaptation

Forum topic · 小凯 · 2026-09-16

Summary

This arXiv paper (2609.15950) by Yilin Xu, Chun Hei Michael Shiu, Chih Wei Ling, and Linqi Song identifies a structural misalignment in record-level differential privacy for personalized federated learning: client-specific variation is low-dimensional while training repeatedly releases high-dimensional updates. The authors release a private client context once and confine repeated adaptation to a fixed coefficient space. The factorized generator induces an adaptive optimization geometry that reshapes noisy updates, with ablations showing most private-training gains are retained by radial evolution. Communication cost is further reduced by realizing the Gaussian mechanism for coefficient updates via variable-length quantization with finite expected code length, letting quantization error serve as the required privacy perturbation. On MNIST and CIFAR-10, the design matches or outperforms full-model private adaptation across privacy budgets and client heterogeneity, reducing protected uplink by 2.67x on CIFAR-10 at epsilon=16 with comparable future-client accuracy.

Paper Overview

Field: Machine Learning Authors: Yilin Xu, Chun Hei Michael Shiu, Chih Wei Ling, Linqi Song Published: 2026-09-14 arXiv: 2609.15950

Abstract

Record-level differential privacy exposes a structural misalignment in personalized federated learning when client-specific variation is low-dimensional while training repeatedly releases high-dimensional updates. This paper addresses the misalignment by releasing a private client context once and confining repeated adaptation to a fixed coefficient space.

Beyond dimensionality reduction, the factorized generator induces an adaptive optimization geometry that reshapes noisy updates, and controlled ablations show that most of its private-training gain is retained by radial evolution. To further reduce communication cost, the authors realize the Gaussian mechanism for coefficient updates directly through variable-length quantization with finite expected code length, so that the quantization error itself acts as the required privacy perturbation rather than additional distortion.

Key Results

On MNIST and CIFAR-10, the proposed design matches or outperforms full-model private adaptation across various privacy budgets and levels of client heterogeneity, while reducing the protected uplink by 2.67x on CIFAR-10 at epsilon=16, with comparable future-client accuracy.

*Auto-collected on 2026-09-16.*