MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

Personalized object detection aims to adapt a general-purposedetector to recognize user-specific instances from only a few examples.Lightweight models often struggle in this setting due to their weak se-mantic priors, while large vision-language models (VLMs) offer strongobject-level understanding but are too computationally demanding forreal-time or on-device applications. We introduce MOCHA (Multi-modalObjects-aware Cross-arcHitecture Alignment), a distillation frameworkthat transfers multimodal region-level knowledge from a frozen VLMteacher into a lightweight vision-only detector. MOCHA extracts fusedvisual and textual teacher’s embeddings and uses them to guide stu-dent training through a dual-objective loss that enforces accurate localalignment and global relational consistency across regions. This processenables efficient transfer of semantics without the need for teacher modi-fications or textual input at inference. MOCHA consistently outperformsprior baselines across four personalized detection benchmarks under strictfew-shot regimes, yielding a +10.1 average improvement, with minimalinference cost.