Embodied Reasoner

Recent advances in deep thinking models have demonstrated remarkable reasoning capabilities on mathematical and coding tasks. However, their effectiveness in embodied domains which require continuous interaction with environments through image action interleaved trajectories remains largely -unexplored. We present Embodied Reasoner, a model that extends o1 style reasoning to interactive embodied search tasks. Unlike mathematical reasoning that relies primarily on logical deduction, embodied scenarios demand spatial understanding, temporal reasoning, and ongoing self reflection based on interaction history. To address these challenges, we synthesize 9.3k coherent Observation Thought Action trajectories containing 64k interactive images and 90k diverse thinking processes (analysis, spatial reasoning, reflection, planning, and verification). We develop a three stage training pipeline that progressively enhances the model's capabilities through imitation learning, self exploration via rejection sampling, and self correction through reflection tuning. The evaluation shows that our model significantly outperforms those advanced visual reasoning models, e.g., it exceeds OpenAI o1, o3-mini, and Claude-3.7 by +9%, +24%, and +13%. Analysis reveals our model exhibits fewer repeated searches and logical inconsistencies, with particular advantages in complex long horizon tasks. Real-world environments also show our superiority while exhibiting fewer repeated searches and logical inconsistency cases.

In this paper, we present Embodied-Reasoner, a novel approach that extends deep-thinking capabilities to embodied interactive tasks. Our key insight is that effective embodied reasoning requires not just the ability to process multimodal inputs, but also to generate diverse thinking processes (analysis, planning, reflection) that adapt to different stages of an interaction.

• Data Engine: To develop this capability, we develop a data engine that automatically synthesizes coherent Observation-Thought-Action trajectories enriched with diverse, embodied-specific thinking processes. e.g., situational analysis, spatial reasoning, self-reflection, task planning, and verification.
• These coherent, image-text interleaved trajectories guide the model to learn how to plan and reason based on its interaction history and spatial layout, thereby boosting its spatial and temporal reasoning capabilities.
• Iterative Training Pipeline: We further introduce a three-stage iterative training pipeline for embodied model that combines imitation, self-exploration, and self-correction.
  • Begins with imitation learning on synthesized trajectories to develop basic interaction skills
  • Followed by rejection sampling tuning to enhance exploration abilities
  • Concludes with reflection tuning to foster self-correction

We evaluate our approach on four high-level embodied tasks in the AI2-THOR simulator: Search, Manipulation, Transportation, and Composite Tasks. These tasks require agents to locate hidden objects in unfamiliar environments through reasoning and planning, then manipulate or transport them to designated areas. Our data engine synthesizes 9.3k task instructions paired with interactive trajectories, containing 64k images and 8M thought tokens, spanning 107 diverse indoor scenes, 2,100 objects, and 2,600 containers. These trajectories are used for our three stage model training.

We cultivate 809 test cases across 12 novel scenarios, which are different from training scenes. We manually design instructions and annotate corresponding key actions and final states: <Instruction, Key Action, Final state >. Notably, our test-set contains 25 carefully designed ultra long-horizon tasks, each involving four sub-tasks and 14-27 key actions.

To evaluate the generalization of our reasoning model, we design a real-world experiment about object searching, covering 30 tasks across three scenes: 6 kitchen tasks, 12 bathroom tasks, and 12 bedroom tasks. During testing, a human operator holds a camera to capture real-time visual input. The model analyzes each image and generates an action command, which the operator executes the actions.