Don’t Teach Instability, Teach Robustness: Selective Sensitivity Gating for Adversarial Robust Distillation

Adversarial Robustness Distillation (ARD) is the most viableway for transferring defensive capabilities from over-parameterized teach-ers to lightweight student models. However, existing ARD techniques relyon the assumption that the teacher’s decision boundary is a perfect geo-metric oracle. In this paper, we identify a critical gap from the stabilityperspective that such exact matching forces blind inheritance of teacher’ssensitivity noise, punishing the student’s stability even if they are morerobust. To address this challenge, we propose the AEGIS (AdversarialError Gating and Instability Suppression) framework that replaces sim-ple logit matching with gated supervision. It leverages a pair of logit andsensitivity gates to rectify potential errors/instability on the categoricaland gradient-level, respectively. Extensive evaluations on CIFAR-10/100,and Tiny-ImageNet show that AEGIS significantly outperforms state-of-the-art methods, achieving a 3.12% gain under the strong AutoAttackand preserving nearly 50% accuracy under large perturbation intensities(ϵ = 16/255) where traditional baselines collapse. Code is available athttps://github.com/JingqiJi03/AEGIS.