Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs

Generative vision-language models (VLMs) such as Qwen-VLand LLaVA achieve strong zero-shot performance on tasks overlappingwith their pretraining distribution, yet fail on specialized domains wherethe required discriminative features were never learned, a regime weterm distant out-of-distribution (OOD). Standard adaptation methodscannot overcome this representational absence because they operatewithin the encoder’s existing feature space. However, VLMs retain arobust descriptive capacity even when discrimination collapses: a modelthat cannot classify a medical scan can still articulate its visual patterns.Exploiting this asymmetry, we introduce Inductive Visual Logic (IVL), atraining-free framework that constructs classification knowledge from themodel’s surviving descriptive ability. IVL extracts visual traits from few-shot support images through dual-mode prompting, combining semanticdescriptions with primitive visual observations, and organizes them intoper-class trait dictionaries. At inference, hierarchical filtering identifiesspatially grounded trait evidence for classification. Across multiple distant-OOD benchmarks, IVL achieves the highest aggregate accuracy under twoVLM backbones while producing interpretable, trait-traceable predictions.