Novel View Synthesis as Video Completion

We tackle the problem of sparse novel view synthesis (NVS)using video diffusion models; given K (≈ 5) posed images of a scene,we predict the view from a target camera pose. Many prior approachesleverage generative image priors encoded via diffusion models. However,models trained on single images lack multi-view knowledge, and we in-stead argue that video models contain implicit multi-view knowledge andare therefore easier to adapt for NVS. Our key insight is to formulatesparse NVS as a low-frame-rate video completion task. However, onechallenge is that sparse NVS is defined over an unordered set of inputimages, often too sparse to admit a meaningful order, so the modelsshould be invariant to permutations of that input image set. To thisend, we present FrameCrafter, which adapts video models (naturallytrained with coherent frame orderings) to permutation-invariant NVSthrough simple but effective architectural modifications, including per-frame latent encoding, query-centered camera conditioning and removalof temporal positional embeddings. Our results suggest that video modelscan be easily modified to “forget” about time with minimal supervision,producing state-of-the-art performance on sparse-view NVS benchmarks.