Implicit Neural Representation Facilitates Unified Universal Vision Encoding

Models for image representation learning are typically de-signed for either recognition or generation. Various forms of contrastivelearning help models learn to convert images to embeddings that areuseful for classification, detection, and segmentation. On the other hand,models can be trained to reconstruct images with pixel-wise, perceptual,and adversarial losses in order to learn a latent space that is compati-ble with image generation. We seek to unify these two directions with afirst-of-its-kind model that learns representations which are simultane-ously well-suited for recognition and generation. We train our model asa hyper-network for implicit neural representation, which learns to mapimages to model weights for fast, accurate reconstruction. We furtherintegrate our INR hyper-network with knowledge distillation to improveits generalization and performance. Beyond the novel training design, themodel also learns an unprecedented compressed embedding space withoutstanding performance for various visual tasks. The complete modelcompetes with state-of-the-art results for image representation learning,enables downstream generative capabilities, and produces high-qualitytiny embeddings.