论文研究
arxiv.org ·
使用生成式真实到仿真的机器人操作测试时空间推理
论文提出 Simify，一个免训练、测试时框架：从单张 RGB-D 图像出发，利用 3D 生成模型和视觉语言模型重建可仿真资产，再针对奖励函数指定的任务（如搭建最高塔）在仿真中并行执行数千次 rollout，并通过进化搜索优化物体摆放，通常在数秒内收敛。作者在真实机器人硬件上开展定量实验，称该框架能端到端完成对未见物体的复杂重排任务，并优于此前的空间推理基础模型，同时指出完整准确的几何对 sim-to-real 迁移的重要性。
AI推荐理由 arXiv 机器人学论文，提出免训练、测试时框架 Simify：从单张 RGB-D 图像借助 3D 生成模型与 VLM 重建可仿真资产，再用大规模并行物理仿真与进化搜索优化物体布局，声称在真实机器人上完成未见物体的复杂重排任务并优于既有空间推理基础模型。新信息在于把生成式 real-to-sim 与推理期并行仿真结合用于空间推理，并强调几何完整性对 sim-to-real 的影响；局限是仅有摘要，缺少量化指标、基线细节与复现信息，效果需以论文正文为准。
78
正文
Computer Science > Robotics
arXiv:2609.33982 (cs)
[Submitted on 27 Sep 2026]
Title:Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim
Authors:Ivan Kapelyukh, Yafei Hu, Ran Gong, Brandon May, Tushar Kusnur, Laura Herlant, Karl Schmeckpeper, Edward Johns, Xiaohan Zhang
View a PDF of the paper titled Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim, by Ivan Kapelyukh and 8 other authors
View PDF
HTML (experimental)
Abstract:Spatial reasoning is fundamental to general robot intelligence, as it enables robots to complete long-horizon tasks involving multi-object interaction. We introduce Simify, a training-free, test-time framework that performs explicit spatial reasoning via massively parallel physics simulation. From a single RGB-D image of a scene, Simify reconstructs simulation-ready assets leveraging 3D generative models and vision-language models. Then given a task specified by a reward function (e.g., build the tallest tower), Simify launches thousands of parallel rollouts in simulation and performs an evolutionary search to optimize object arrangements, typically converging within seconds. We conduct quantitative experiments on real-robot hardware to demonstrate the ability of our framework to execute complex object rearrangement tasks end-to-end with previously unseen objects. Results show that our framework outperforms prior work on foundation models for spatial reasoning by effectively exploiting large-scale parallel simulation during inference, and also highlight the importance of complete and accurate geometry for successful sim-to-real transfer.
Comments:
Subjects:
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
Cite as:
arXiv:2609.33982 [cs.RO]
(or
arXiv:2609.33982v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.33982
Focus to learn more
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Ivan Kapelyukh [view email]
[v1]
Sun, 27 Sep 2026 22:28:10 UTC (3,474 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Test-Time Spatial Reasoning for Robot Manipulation Using Generative Real-to-Sim, by Ivan Kapelyukh and 8 other authors
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.RO
new
|
recent
| 2026-09
Change to browse by:
cs
cs.CV
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
export BibTeX citation
Loading...
BibTeX formatted citation
×
loading...
Data provided by:
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? |
Disable MathJax (What is MathJax?)