Skip to yearly menu bar Skip to main content


Poster

Agglomerative Token Clustering

Joakim Bruslund Haurum · Sergio Escalera · Graham W. Taylor · Thomas B. Moeslund

# 48
Strong blind review: This paper was not made available on public preprint services during the review process Strong Double Blind
[ ] [ Project Page ] [ Paper PDF ]
[ Poster]  [ Supplemental] 
Tue 1 Oct 1:30 a.m. PDT — 3:30 a.m. PDT

Abstract:

We present Agglomerative Token Clustering (ATC), a novel token merging method that consistently outperforms previous token merging and pruning methods across image classification, image synthesis, and object detection & segmentation tasks. ATC merges clusters through bottom-up hierarchical clustering, without the introduction of extra learnable parameters. We find that ATC achieves state-of-the-art performance across all tasks, and can even perform on par as the prior state-of-the-art when applied off-the-shelf, i.e. without fine-tuning. ATC is particularly effective when applied with low keep rates, where only a small fraction of tokens are kept and retaining task performance is especially difficult.

Chat is not available.