EvoTok: A Unified Image Tokenizer via Residual Latent Evolution for Visual Understanding and Generation

The development of unified multimodal large language mod-els (MLLMs) is fundamentally challenged by the granularity gap be-tween visual understanding and generation: understanding requires high-level semantic abstractions, while image generation demands fine-grainedpixel-level representations. Existing approaches usually enforce the twosupervision on the same set of representation or decouple these two su-pervision on separate feature spaces, leading to interference and incon-sistency, respectively. In this work, we propose EvoTok, a unified imagetokenizer that reconciles these requirements through a residual evolutionprocess within a shared latent space. Instead of maintaining separatetoken spaces for pixels and semantics, EvoTok encodes an image intoa cascaded sequence of residual tokens via residual vector quantization.This residual sequence forms an evolution trajectory where earlier stagescapture low-level details and deeper stages progressively transition to-ward high-level semantic representations. Despite being trained on a rel-atively modest dataset of 13M images, far smaller than the billion-scaledatasets used by many previous unified tokenizers, EvoTok achieves astrong reconstruction quality of 0.43 / 0.25 rFID at 256×256 / 384×384resolutions on ImageNet-1K, respectively. When integrated with a largelanguage model, EvoTok shows promising performance across 8 out of 9visual understanding benchmarks, and remarkable results on image gen-eration benchmarks such as GenEval, GenAI-Bench, and DPG. Theseresults demonstrate that modeling visual representations as an evolvingtrajectory provides an effective and principled solution for unifying visualunderstanding and generation.