Physical AI evaluation is no longer a downstream validation step—it is a central mechanism for guiding data collection, model design, and learning itself. Foundation models have outgrown isolated demos and narrow benchmarks. Rigorous, scalable evaluation infrastructure has become the critical bottleneck.

NVIDIA Isaac Lab - Arenais an open-source framework for large-scale policy evaluation, featuring GPU-accelerated parallel evaluation that could transform policy evaluation workflows. Today, we are sharing comprehensive benchmark results that validate those claims: Isaac Lab-Arena achieves up to 13.5× faster policy evaluation compared to sequential execution, reducing evaluation time from over 10 hours to under 1 hour for complex manipulation tasks.

This performance gain is not theoretical, and it directly powers RoboFinals, our industry-grade evaluation platform already adopted by leading foundation model teams. More broadly, it demonstrates how GPU-accelerated parallelism can unlock evaluation at the scale and speed required for modern physical AI development.

As generalist robot policies emerge, evaluation complexity grows rapidly. A single policy must be tested across thousands of combinations of tasks, objects, scenes, robots, and physical parameters.

Running these evaluations sequentially doesn’t scale. Waiting hours or days for results slows iteration and makes thorough testing impractical.

Isaac Lab-Arena addresses this with GPU-accelerated, parallel evaluation, enabling developers to benchmark policies at scale and iterate faster with confidence.

We conducted comprehensive performance benchmarks comparing Lightwheel-RoboCasa-Tasks built on Isaac Lab - Arena against the widely-used RoboCasa Benchmark built on Robosuite and MuJoCo simulation stack. The test was designed to isolate performance differences primarily in parallel simulation and runtime infrastructure, not the evaluation method or policy.

*For detailed benchmark methodology and raw data, see ourtechnical documentation*.

We executed 10 complex and long-horizon manipulation tasks from the RoboCasa benchmark (PrepareCoffee, QuickThaw, SteamInMicrowave, etc.) using the NVIDIA GR00T N1.5 policy on a Panda-Omron robot. Each task ran 200-step rollouts across 1,024 / 2,048 / 4,096 parallel environments on 8× NVIDIA RTX 6000D GPUs.

We tested three configurations to cleanly isolate performance factors:

All three configurations ran the same tasks, same number of episodes, same policy inference—ensuring throughput differences came purely from the simulator and runtime stack, not from task complexity or policy behavior.

• 1,024 envs: 10.7× faster than MuJoCo
• 2,048 envs: 12.4× faster than MuJoCo
• 4,096 envs: 13.5× faster than MuJoCo

A critical insight emerges when comparing Isaac Lab - Arena's sequential and parallel modes. When running sequentially, Isaac Lab - Arena actually takes 34.9 hours versus MuJoCo's 10.2 hours for the same workload.
Why? Because the Isaac Lab - Arena version of the RoboCasa Tasks uses higher-fidelity assets with refined collision geometry, improved contact surfaces, and enhanced materials/textures—all validated through teleoperation to ensure realistic interactions. This fidelity costs compute when running sequentially.
However, comparing Isaac Lab - Arena sequential (34.9h) to Isaac Lab - Arena parallel (0.76h) shows a 46× speedup from parallelization alone.
This cleanly demonstrates that the throughput gain comes entirely from GPU-accelerated parallelism, not from cutting corners on simulation quality. In fact, you're getting higher-fidelity simulation at 13× faster throughput—the best of both worlds.