Why MolmoAct2 Is a Major Step for Embodied AI in 2026

Most robot AI lives in a lab. It works on a fixed table, with fixed objects, under fixed lighting. Move it to a kitchen, a warehouse, or a wet lab, and it falls apart. The model was never built for that world.

MolmoAct2, released by the Allen Institute for AI and the University of Washington, is built for exactly that world. It is a fully open Vision-Language-Action (VLA) model that deploys out of the box on real robots, reasons about 3D space before acting, and runs fast enough for closed-loop control. Every weight, every line of training code, and every training dataset is public.

This is not just another research paper. It is a blueprint for robot AI that works outside the lab.

The Problem With Today's Robot AI

Four problems stop current VLA models from reaching real deployment.

First, the best-performing models are closed. Their training data, recipes, and weights are proprietary. Researchers cannot reproduce them. Practitioners cannot adapt them.

Second, reasoning costs too much time. Models that think through chain-of-thought, predicted goal images, or full world-model rollouts must generate hundreds of tokens before emitting a single action. That latency breaks closed-loop control.

Third, open-weight models that actually run are tied to expensive hardware. Most academic labs and independent researchers cannot access the robots they require.

Fourth, success rates after fine-tuning remain too low for dependable use.

MolmoAct2 addresses all four.

The Backbone: Molmo2-ER

  benchmark comparison

Every VLA model is only as good as its visual-language backbone. Standard vision-language models do not understand metric distances, free space, or how objects look from different camera angles. They were not trained for that.

Molmo2-ER fixes this. It starts from the Molmo2 checkpoint and specializes it on a 3.3 million-sample corpus across six capability pillars: single-image embodied QA, image pointing, image detection, video embodied QA, multi-image and ego-to-exo correspondence, and abstract embodied reasoning.

Training uses a two-stage specialize-then-rehearse recipe. Stage 1 runs for 20,000 steps on the embodied corpus, rapidly moving the model onto spatial and embodied data. Stage 2 runs for 1,500 additional steps on a mixture of the new embodied data and Molmo2's original training data, with a 50/50 split found to give the best trade-off between embodied and general benchmarks.

The result is a backbone that scores 63.8% average across 13 embodied-reasoning benchmarks, beating GPT-5, Gemini Robotics ER-1.5, and Gemini 2.5 Pro. It improves over the base Molmo2 model by 17 percentage points.

The Training Data

  Training data

A strong model needs strong data. MolmoAct2 is trained on three new datasets alongside academic benchmarks and multimodal web data.

MolmoAct2-BimanualYAM is the largest open bimanual robot dataset to date. It contains 34,500 demonstrations across 28 real-world tasks, totaling 720 hours of teleoperated robot data. Tasks range from folding clothes and untangling cables to scanning groceries and packing medication. The entire collection setup costs under $6,000.

MolmoAct2-SO100/101 is curated from 1,222 public community-contributed datasets with 377 contributors, covering 38,059 episodes and 184 hours of interaction. A four-stage filtering pipeline removes structurally invalid data, eval-style datasets, license violations, and low-quality demonstrations using a TOPReward quality gate.

MolmoAct2-DROID filters the large-scale DROID dataset down to 74,604 valid Franka episodes with at least one language instruction and no idle segments. All episodes were re-annotated using a Qwen3.5-27B VLM to double the number of unique language labels from 22% to 46% of the dataset.

Architecture: How MolmoAct2 Works

  architecture

MolmoAct2 has a three-stage training pipeline: pre-training, post-training, and deployment fine-tuning.

Pre-training adapts the Molmo2-ER backbone into a discrete autoregressive robot policy using the OpenFAST Tokenizer. This tokenizer maps one second of robot action into a compact sequence of discrete tokens from a 2,048-token vocabulary. It applies a frequency-domain transform to the trajectory, quantizes the result, and uses byte-pair encoding to produce tokens. It is trained on one million action sequences across five embodiments, covering both absolute joint control and delta end-effector control.

During pre-training, images are encoded by a SigLIP2 ViT, pooled, and projected into the language model's embedding space. Robot examples extend the sequence with state tokens and discrete action tokens. The model trains on a mix that is 90% robot data and 10% multimodal data, for 200,000 steps across 64 H100 GPUs.

Post-training grafts a continuous action expert onto the pre-trained backbone. The expert uses flow matching to denoise a Gaussian noise vector into a continuous action trajectory. The key architectural innovation is the per-layer KV connection.

Instead of conditioning the action expert only on the VLM's final hidden state, MolmoAct2 connects them at every layer. For each layer ℓ, the VLM's key and value tensors from self-attention are projected by learned adapter matrices into the action expert's cross-attention space. Every block of the action expert attends to the corresponding VLM layer's representation. The expert has the same depth as the VLM backbone, with L = 36 layers each.

This gives the action expert dense access to the full hierarchy of visual and language features, not just a single bottleneck representation. Ablations confirm this matters: per-layer KV conditioning reaches 95.9% on LIBERO versus 94.0% for hidden-state conditioning.

The post-training objective combines next-token prediction on discrete actions with the flow-matching loss on continuous actions. The VLM is shielded from flow-matching gradients during post-training via knowledge insulation, so only the action expert and its adapter projections update from the continuous loss.

Deployment fine-tuning adapts the post-trained checkpoint to specific embodiments. Each robot-specific run increases the number of flow samples per action chunk from 4 to 8 for denser supervision. Fine-tuning removes knowledge insulation, allowing flow-matching gradients to flow through the full model.

MolmoAct2-Think: Reasoning Without the Latency Tax

  MolmoAct2-Think adaptive depth

Knowing where objects are in 3D space helps a robot act correctly. But generating spatial tokens at every step is expensive. MolmoAct2-Think solves this with adaptive depth reasoning.

Before generating an action, the model predicts a compact depth representation of the scene. Each depth map is quantized into a 10×10 grid of 100 spatial positions, each holding one of 128 learned depth-code values. These codes are ordinary autoregressive tokens, compatible with the same next-token interface used throughout the model.

The key insight is temporal redundancy. In a robot trajectory, most of the scene does not change between control steps. MolmoAct2-Think compares the current camera image to the previous one using cosine similarity across 10×10 image patches. A cell is updated only when its patch similarity drops below 0.996. Unchanged cells replay cached depth codes. Only changed cells are re-decoded.

This means depth reasoning cost scales with how much the scene changes, not with the total number of depth tokens. For static scenes, the savings are large. Geometric grounding is preserved at a fraction of prior latency.

At inference, CUDA Graphs capture the fixed-shape flow-matching loop, reducing kernel-launch overhead. Cross-attention states that depend only on the VLM context are cached and reused across flow steps.

Benchmark Results

The evaluation spans 7 simulation and real-world benchmarks, the most extensive study of any open VLA to date.

Embodied Reasoning (Molmo2-ER): Molmo2-ER scores 63.8% across 13 benchmarks, topping the leaderboard on 9 of 13 tasks. It beats Gemini Robotics ER-1.5 Thinking by 2.5 points overall and surpasses GPT-5 on the same suite.

LIBERO simulation: MolmoAct2 achieves 97.2% average across Spatial, Object, Goal, and Long suites, with 100% on Object. MolmoAct2-Think adds another 0.9 points to reach 98.1%, with the largest gain of 2.2% on the hardest Long suite.

MolmoSpaces (DROID-style simulation): MolmoAct2-DROID achieves 37.7% average, a 3.2% gain over the runner-up π0.5-DROID at 34.5%.

MolmoBot (harder simulation): MolmoAct2-DROID reaches 87.1% on real-world DROID tasks, a 38.7% absolute gain over MolmoBot, the next best model.

SO-100/101 real-world: MolmoAct2-SO100/101 scores 56.7%, an 11.4% gain over the π0 baseline fine-tuned on the same data.

Bimanual YAM real-world (8 tasks): MolmoAct2 achieves 50.1% average success, 15% above the runner-up OpenVLA-OFT, winning 7 of 8 tasks.

Inference speed: With caching and CUDA Graphs enabled, MolmoAct2 runs at 55.79 Hz on LIBERO with a single H100, a 2.42× speedup over the baseline. MolmoAct2-Think reaches 12.71 Hz.

Why This Matters

Most robotics research stays in the lab. It runs on one robot, with one camera setup, trained on data from that exact robot. Deploying anywhere else requires starting over.

MolmoAct2 is different in three ways that matter for real deployment. It is fully open, including weights, code, and data. It runs out of the box on three different robot platforms at low to medium cost. And it reasons about geometry without paying the latency price that makes reasoning models impractical for real-time control.

The BimanualYAM dataset alone, with 720 hours of teleoperated bimanual demonstrations, is now the largest open bimanual manipulation dataset in existence. That data will train future models long after this paper is forgotten.

Robot AI has spent years getting smarter in simulated kitchens. MolmoAct2 is a step toward robots that can work in real ones.

FAQs