Human-Level Accuracy, Non-Human Strategies: Revealing Model-Human Divergence in Video Physical Reasoning

Video foundation models now reach human-level accuracy onphysical-reasoning benchmarks, yet such tasks require predicting unob-served physical outcomes. Do these models perform human-like forwardsimulation, or do they exploit statistical regularities in visible scenes?Accuracy alone cannot distinguish these strategies. We introduce a distri-butional evaluation framework that treats model seeds and human ratersas populations, enabling comparison of consensus, uncertainty, and strat-egy. On the Physion benchmark, we evaluate three ViT-L architectures(V-JEPA2, VideoMAEv2, DINOv2). V-JEPA2 narrows the accuracy gapto ∼1 percentage point (73.2% vs. 74.2%), yet model–human disagree-ment reaches 26.4%, far exceeding human–human disagreement (4.8%),with substantially lower agreement (κ ≈ 0.48 vs. 0.91). The divergenceis scenario-structured: models outperform humans on geometric reason-ing (linking, +11.8 pp), while human advantages appear on causal-chainand other future-dynamics scenarios in the released split (e.g., dominoes,−10.5 pp). Strategy fingerprinting confirms all three architectures sharenon-human strategies while none aligns with humans. Attribution analy-sis suggests that unobservable outcome features, rather than visible sceneproperties, predict this divergence, consistent with models relying moreon scene-level statistical regularities than on explicit forward simulation,a systematic divergence that accuracy alone cannot reveal. Code is avail-able at https://github.com/fanhong-li/model-human-divergence.