LumiTokens: 3D Relighting via Token-Space Lighting Transformation

Existing 3D relighting methods operate through either ex-plicit material decomposition, diffusion-based view-space generation, ora combination of both, requiring full recomputation for each new light-ing condition. We observe that recent latent scene representations, whichencode multi-view images into a set of compact tokens with no fixed phys-ical semantics, open up a novel design space for relighting. We presentLumiTokens, a framework that formulates 3D relighting as a directtransformation on latent scene tokens, without explicit 3D representa-tions, rendering equations, or physics-based decomposition. Our modelintroduces a Scene Token Editor that processes scene tokens jointly withlight-ray tokens through self-attention, producing updated tokens thatcan be decoded into multi-view-consistent relit images. To support di-verse lighting types through a unified interface, all lighting signals, in-cluding environment maps, point lights, and area lights, are parameter-ized as Plücker ray tokens, enabling native 3D user interaction with arepresentation that carries no explicit spatial structure. Crucially, thisdesign supports progressive relighting: because the editor’s output re-mains in the same latent space as its input, a user can incrementallybuild up illumination one light source at a time, with each edit compos-ing in token space. Experiments demonstrate that LumiTokens achievescomparable or superior relighting quality to other methods and supportsprogressive, composable lighting edits.