HER-Count: Learning Hyper-Exemplar Representation for Generalized Zero-Shot Object Counting

Zero-shot object counting aims to count objects in images from givenclass names. Existing methods typically select representative image patches orextract text embeddings as exemplars to match image regions for counting. How-ever, a limited number of selected patches is insufficient to cover intra-class di-versity, and the accumulated errors in exemplar detection and selection substan-tially degrade counting accuracy. While the text embeddings remain at a genericsemantic level, blind to the specific instance diversity of target objects in animage. To handle these issues, this paper proposes the Hyper-Exemplar Repre-sentation (HER) counting framework (HER-Count) to represent the objects ofa specific category in a given image. Rather than image patches or text embed-dings, HERs are synthesized from holistic images and textual class names viaan multi-modal large-language model. By jointly integrating all target objects ineach image, HERs avoid exemplar generation errors and are more customizedrepresentative, thus yielding more accurate counting results. To effectively ex-ploit HERs in counting, HER-Count introduces a hierarchical fusion strategy thatinjects HERs into multiple intermediate layers of the vision encoder when pre-dicting density maps. This enables the model to progressively refine countingcues across layers, significantly enhancing the performance. A global discrimi-nation enhancement constraint at both image sample and semantic class levels isdesigned to further improve the capacities for both representation and discrimi-nation of HERs. Extensive experiments on widely used benchmarks demonstrateboth the state-of-the-art performance and generalization ability of the proposedHER-Count. Benefiting from the compact end-to-end design, HER-Count alsoachieves promising inference speed.