WARP: Wide Attention with Rich Projections for Image Super-Resolution

State-of-the-art super-resolution (SR) transformers stack 16–30+ blocks of small shifted-window attention. Information between dis-tant positions must therefore traverse multiple attention layers, eachbounded by the local window—an indirect path that grows with net-work depth. We show that direct wide attention over large position-fixedwindows (64×64 at training, up to 128×128 at inference) can matchor surpass these deep narrow-window designs with only 12 blocks. Akey enabler is 2D Rotary Position Embeddings (RoPE), which encoderelative positions through query-key rotations rather than additive biasmatrices, enabling memory-efficient attention over thousands of tokens.To fully exploit this large spatial context, we replace the standard linearQKV projection with a rich nonlinear projection module that producesmore expressive features for each attention operation; sharing a singlemodule across all blocks keeps the model at 20M parameters while pro-viding the representational capacity that would otherwise require 115M.The shared module jointly produces spatial and channel QKV througha unified output. RoPE further enables resolution-adaptive inference viaYaRN-style scaling [22] with an inverted temperature correction tailoredfor SR. With only 20M parameters, our model, WARP, achieves PSNRcompetitive with or superior to state-of-the-art methods on standardbenchmarks.