Weight Feedback Computes the Exact Jacobian Transpose in Modern Deep Networks

Predictive Coding (PC) offers a biologically motivated al-ternative to backpropagation via local weight updates, yet routing er-ror between layers still relies on an autograd Jacobian-transpose (J⊤ )product—the last non-local operation in PC. We show that this depen-dency is largely avoidable. For any layer f (x)=Act(Norm(L(x))) withfrozen normalization statistics, the exact J⊤ factors into three locallyavailable terms, J⊤ v = L⊤ (s ⊙ σ ′ (pre-act) ⊙ v), where σ ′ is the activationderivative and s=γ/σrun is the normalization gain. Prior weight-feedbackmethods omitted both corrections; restoring them closes the transportgap for this layer class. Note that locality here holds up to three as-sumptions, which we state upfront—weight symmetry (L⊤ mirrors theforward operator, as assumed by all PC), a soft spectral-norm controlthat is not synapse-local, and a nearest-neighbour approximation forMaxPool. Substituting the identity into PC yields WF-Act-PC, whichremoves the autograd backward pass from error transport. On CIFAR-10/100 (50 epochs, 5 seeds), WF-Act-PC is the only PC method whoseaccuracy improves with depth, surpassing iPC—the strongest classicalPC baseline—by 2.7–22.3 pp on CIFAR-10. With both methods tunedper architecture, it matches or exceeds a comparably-tuned backpropa-gation baseline on the deeper CIFAR-10 architectures (VGG-9: 93.57%vs. 92.43%; ResNet-18: 92.76% vs. 91.54%) and on the harder Tiny-ImageNet benchmark, while trailing tuned BP on the deeper CIFAR-100VGG cells. Our WF-Act-PC implementation is publicly available athttps://github.com/jlshen025/pcax.