LatSearch: Latent Reward-Guided Search for Faster Inference-Time Scaling in Video Diffusion

The recent success of inference-time scaling in large languagemodels has inspired similar explorations in video diffusion. In particular,motivated by the existence of “golden noise” that enhances video quality,prior work has attempted to improve inference by optimising or searchingfor better initial noise. However, these approaches have notable limita-tions: they either rely on priors imposed at the beginning of noise sam-pling or on rewards evaluated only on the denoised and decoded videos.This leads to error accumulation, delayed and sparse reward signals, andprohibitive computational cost, which prevents the use of stronger searchalgorithms. Crucially, stronger search algorithms are precisely what couldunlock substantial gains in controllability, sample efficiency and genera-tion quality for video diffusion, provided their computational cost can bereduced. To fill in this gap, we enable efficient inference-time scaling forvideo diffusion through latent reward guidance, which provides interme-diate, informative and efficient feedback along the denoising trajectory.We introduce a latent reward model that scores partially denoised la-tents at arbitrary timesteps with respect to visual quality, motion qual-ity, and text alignment. Building on this model, we propose LatSearch,a novel inference-time search mechanism that performs Reward-GuidedResampling and Pruning (RGRP). In the resampling stage, candidatesare sampled according to reward-normalised probabilities to reduce over-reliance on the reward model. In the pruning stage, applied at the finalscheduled step, only the candidate with the highest cumulative reward isretained, improving both quality and efficiency. We evaluate LatSearchon the VBench-2.0 benchmark and demonstrate that it consistently im-proves video generation across multiple evaluation dimensions comparedto the baseline Wan2.1 model. Compared with the state-of-the-art, ourapproach achieves comparable or better quality while reducing runtimeby up to 79%.