RoboStream: Weaving Spatio-Temporal Reasoning with Memory in Vision-Language Models for Robotics

Enabling reliable long-horizon robotic manipulation is a cru-cial step toward open-world embodied intelligence. However, VLM-basedplanners treat each step as an isolated observation-to-action mapping,forcing them to reinfer scene geometry from raw pixels at every decisionstep while remaining unaware of how prior actions have reshaped the en-vironment. Despite strong short-horizon performance, these systems lackthe spatio-temporal reasoning required for persistent geometric anchoringand memory of action-triggered state transitions. Without persistentstate tracking, perceptual errors accumulate across the execution horizon,temporarily occluded objects are catastrophically forgotten, and com-pounding failures lead to precondition violations that cascade throughsubsequent steps. In contrast, humans maintain a persistent mental modelthat continuously tracks spatial relations and action consequences acrossinteractions rather than reconstructing them at each instant. Inspiredby this human capacity for causal spatio-temporal reasoning with persis-tent memory, we propose RoboStream, a training-free framework thatachieves geometric anchoring through Spatio-Temporal Fusion Tokens(STF-Tokens), which bind visual evidence to 3D geometric attributesfor persistent object grounding, and maintains causal continuity via aCausal Spatio-Temporal Graph (CSTG) that records action-triggeredstate transitions across steps. This design enables the planner to tracecausal chains and preserve object permanence under occlusion withoutadditional training or fine-tuning. RoboStream achieves a 90.5% successrate on long-horizon RLBench tasks and a 44.4% success rate on challeng-ing real-world block-building tasks, where both SoFar and VoxPoser score11.1%, demonstrating that spatio-temporal reasoning and causal memoryare critical missing components for reliable long-horizon manipulation.