Multi-Stage, Long-Horizon
The task runs through a sequence of dependent sub-goals over minutes, not seconds; getting stage four right depends on how stage two was done.
Today we're introducing RoboFinals RL-Env, a reinforcement learning environment built from real industrial tasks and calibrated physics. It is the training side of RoboFinals: the same tasks, scenes and physics RoboFinals uses to evaluate policies are now available as an environment to train them in.
Why the Environment Matters More than the Algorithm
Reinforcement learning in simulation works when the simulation is right. Two things usually aren't. The task is designed by a researcher rather than taken from the real job, so a policy that solves it has learned something nobody needs. And the physics is hand-tuned until it looks plausible, so a policy that solves it has learned a world that doesn't exist.
We built the RoboFinals RL environment to close both gaps. Every task comes from a human doing the real work. Every physical parameter comes from a measurement.
EgoSuite Human Data
A person does the real task. EgoSuite records it first-person.
Sim Env
The real task, rebuilt in simulation
Scene
Parts, tools and hand reconstructed from the EgoSuite capture as SimReady assets
Calibration
Physics measured against the real parts, not tuned by hand
RoboFinals RL-Env
Train policies here. Evaluate them on the RoboFinals benchmark, on the same task.
Real World·Simulation·Training and Evaluation
Agent-driven, human expert on the loop. Agents build each step; experts review each step.Sim Env from the Real World
Tasks start as EgoSuite captures: egocentric recordings of a person performing an industrial task with their own hands and tools. From that capture, an agent extracts the task structure, the sub-goals, the objects involved and the order they're handled in, and reconstructs the scene as SimReady OpenUSD assets.
EgoSuite Capture → Sim SceneSplit screen: the human's first-person view on the left, the reconstructed scene on the right
What comes out is harder than a typical benchmark task in three ways that matter for Physical AI.
Dexterous
The embodiment is a dexterous hand, not a parallel gripper, because the human was using one.
Deformable
Hoses, cables and gaskets behave like hoses, cables and gaskets.
Calibration from Measured Physics
The scene is then calibrated. Material parameters are measured, not guessed, and deformable response is validated against industrial-grade CAE. Contact between rigid, deformable and granular bodies is resolved by coupled solvers in a single timestep, running Newton-native.
This is the part that makes the environment trustworthy rather than realistic-looking: a policy trained here has learned contact dynamics that were checked against the physical world.
- Material Parameters
Measured
Friction, stiffness and damping taken from physical measurement of the parts and materials in the task.
- Deformable Response
Validated against Industrial-Grade CAE
Hose, cable and gasket behaviour checked against engineering-grade simulation before it enters the environment.
- Solvers
Coupled Rigid, Deformable and Granular
Contact forces exchanged across solver types inside one timestep, so mixed-material interactions resolve consistently.
- Backend
Newton-Native
Built on the Newton physics engine with SimReady OpenUSD as the scene format.
Agent-Driven, Human Expert on the Loop
The pipeline from capture to environment is driven by agents, with Lightwheel's simulation engineers reviewing at each stage. Agents do the construction; experts decide what passes. This is what lets one task become many without rebuilding the pipeline each time.
| Stage | Agent | Expert |
|---|---|---|
| Human DemoEgoSuite | Parses the capture into sub-goals, objects and ordering | Confirms the task structure matches the job |
| SceneSimReady OpenUSD | Builds the scene: parts, tools, dexterous-hand embodiment | Reviews the reconstruction |
| CalibrationMeasured Physics + CAE | Assigns measured parameters; runs deformable validation | Signs off on fidelity |
| RL-Env + EvalNewton · RoboFinals | Assembles the environment and runs evaluation | Releases the task |
From Training to Evaluation
RoboFinals is Lightwheel's policy evaluation benchmark. RoboFinals RL-Env uses the same task definitions, so a policy trained here is evaluated on the terms it was trained on. The full RoboFinals evaluation benchmark, with more tasks and a public leaderboard, opens later this year.
Scaling with EgoSuite
One task is a demonstration. A benchmark needs a distribution. The automotive repair task is the first environment we are releasing; the question that matters is how the library grows, and what it grows toward.
Our answer is to let EgoSuite set the distribution. Every EgoSuite capture is annotated along one taxonomy: the scene it happens in (domestic, industrial, retail, hospitality, healthcare and so on), the type and difficulty of the operation (from pick-and-place through alignment, tool use and force regulation, up to precision work under sustained closed-loop control), the kinds of assets involved (rigid, articulated, deformable, fluid), and the horizon of the task. We apply the same taxonomy to every environment we build. The environment library is therefore not a list of tasks someone chose; it is a sample of the real-world distribution EgoSuite observes.
That makes scaling mechanical rather than editorial. As EgoSuite captures grow, agents draw from the new captures along each axis of the taxonomy, build and calibrate the environments, and experts review them. The RoboFinals RL-Env library tracks the human-data distribution as it scales, including its skew: where real work is heavy on deformables or long horizons, the environments are too.
EgoSuite Captures
Human data, growing continuously
One Taxonomy
- Scene
- Operation and Difficulty
- Asset Type
- Task Horizon
Applied to captures and environments alike
Agents Build, Experts Review
Scene, calibration, environment, evaluation
RoboFinals RL-Env Library
Same distribution as the human data, at the same pace
Cooking Rice
Domestic · fluids and granular · multi-stage
Assembling an Engine
Industrial · rigid and articulated · precision, tool use
Organizing the Operating Room
Healthcare · rigid and deformable · long-horizon
Buffer Installation
Industrial · alignment · assembly
Join Us
Early Access
For teams training manipulation policies. Train in the RoboFinals RL environment ahead of the full benchmark release, and be on the leaderboard when it opens.
Become a RoboFinals Partner
For the people who own real tasks: factories, hospitals, retailers, logistics operators and robot makers. Bring the task; we capture it with EgoSuite, build the environment and release it as a RoboFinals benchmark task.