TransVLM: A Vision-Language Framework and Benchmark for Detecting Any Shot Transitions

Traditional Shot Boundary Detection (SBD) inherently strug-gles with complex transitions by formulating the task around isolated cutpoints, frequently yielding corrupted video shots. We address this fun-damental limitation by formalizing the Shot Transition Detection (STD)task. Rather than searching for ambiguous points, STD explicitly detectsthe continuous temporal segments of transitions. To tackle this, we pro-pose TransVLM, a Vision-Language Model (VLM) framework for STD.Unlike regular VLMs that predominantly rely on spatial semantics andstruggle with fine-grained inter-shot dynamics, our method explicitly in-jects optical flow as a critical motion prior at the input stage. Through asimple yet effective feature-fusion strategy, TransVLM directly processesconcatenated color and motion representations, significantly enhancingits temporal awareness without incurring any additional visual tokenoverhead on the language backbone. To overcome the severe class im-balance in public data, we design a scalable data engine to synthesizediverse transition videos for robust training, alongside a comprehensivebenchmark for STD. Extensive experiments demonstrate that TransVLMachieves superior overall performance, outperforming traditional heuris-tic methods, specialized spatiotemporal networks, and top-tier VLMs.