Affine steerers for structured keypoint description

We propose a way to train deep learning based keypoint descriptors to make them approximately equivariant under transformations of the image plane that are locally affine. The main idea is to use the representation theory of GL(2) to generalize the recently introduced concept of steerers from rotations to affine transformations. Affine steerers give high control over how keypoint descriptions transform under image transformations. We demonstrate the potential of using this high control for image matching. Finally, we propose a way to finetune keypoint descriptors with a set of steerers on upright images and obtain state-of-the-art results on several standard benchmarks.