Multi-label Instance-level Generalised Visual Grounding in Agriculture

Understanding field imagery such as detecting plants anddistinguishing individual crop and weed instances, is a central challengein precision agriculture. Despite progress in vision–language tasks likecaptioning and visual question answering, Visual Grounding (VG), local-ising language-referred objects, remains unexplored in agriculture. A keyreason is the lack of suitable benchmark datasets for evaluating groundingmodels in field conditions, where many plants look highly similar, appearat multiple scales, and the referred target may be absent from the image.To address these limitations, we introduce gRef-CW, the first dataset de-signed for generalised visual grounding in agriculture, including negativeexpressions. Benchmarking current state-of-the-art grounding models ongRef-CW reveals a substantial domain gap, highlighting their inabilityto ground instances of crops and weeds. Motivated by these findings, weintroduce Weed-VG, a modular framework that incorporates multi-labelhierarchical relevance scoring and interpolation-driven regression. Weed-VG advances instance-level visual grounding and provides a clear baselinefor developing VG methods in precision agriculture. Code and data areavailable at https://github.com/MHaghighat98/WeedVG-gRefCW.