OrthoEraser: Coupled-Neuron Orthogonal Projection for Concept Erasure

Text-to-image (T2I) models face significant safety risks fromadversarial induction, yet current concept erasure methods often causecollateral damage to benign attributes when suppressing selected neu-rons entirely. This occurs because sensitive and benign semantics ex-hibit non-orthogonal superposition, sharing activation subspaces wheretheir respective vectors are inherently entangled. To address this issue,we propose OrthoEraser, which leverages sparse autoencoders (SAE) toachieve high-resolution feature disentanglement and subsequently rede-fines erasure as an analytical orthogonalization projection that preservesthe benign manifold’s invariance. OrthoEraser first employs SAE to de-compose dense activations and segregate sensitive neurons. It then usescoupled neuron detection to identify non-sensitive features vulnerableto intervention. The key novelty lies in an analytical gradient orthog-onalization strategy that projects erasure vectors onto the null spaceof the coupled neurons. This orthogonally decouples the sensitive con-cepts from the identified critical benign subspace, effectively preservingnon-sensitive semantics. Experimental results on safety demonstrate thatOrthoEraser achieves high erasure precision, effectively removing harm-ful content while preserving the integrity of the generative manifold, andsignificantly outperforming SOTA baselines. WARNING: This papercontains results of unsafe models.