EgoSuite-Open100K The Largest Fully-Annotated Open Egocentric Human Dataset

Physical AI is bottlenecked by human data that combines scale, diversity, and annotation quality. Few datasets deliver all three.

Today, together with Hugging Face, we are releasing EgoSuite-Open100K: 100,000 hours of egocentric human data covering 15,000+ tasks across 15,000+ distinct collection scenes. It includes hand pose and body pose annotations, wrist-camera coverage for part of the dataset, and a complimentary event-level semantic annotation add-on for selected subsets. The first batch is now live on the Hugging Face Hub, with the remainder rolling out in stages.

Why we are releasing this

High-quality data determines what models can learn, and over the past year that principle has become measurable. Work from NVIDIA, Generalist AI, Sunday Robotics, and Dyna Robotics has converged on the same conclusion. NVIDIA EgoScale identified a log-linear scaling law across 20,854 hours of egocentric human video. Dyna extended the evidence from a thousand hours to a million, with the trend holding even for robots the model had never seen. Physical AI now has a scaling law of its own, powered by human data.

Across different methods, the finding is consistent: the scale, diversity, and annotation quality of human data set the ceiling on what a robot can learn. Yet datasets combining all three remain scarce, and those approaching the required scale are largely private. The evidence is public. The data is not.

By releasing the first batch of EgoSuite-Open100K, we are putting data on the table. Capture conventions, annotation schemas, and formats remain fragmented across the field; we hope this release helps the community converge on shared standards.

Open100K is not the endpoint. It is the first public layer of the data infrastructure we are building for Physical AI. What works, what is missing, and how the data performs in practice will shape what we open next.

Built at scale

At 100,000 hours, EgoSuite-Open100K is among the largest open egocentric human datasets released to date. It was built through a globally distributed workforce of tens of thousands and a continuous, standardized collection process that keeps capture conventions consistent across the dataset.

Diverse by design

Scale alone is of limited value. What a model can learn is bounded by how much of the world its training data reaches.

Our dataset includes over 15,000 tasks recorded across 15,000+ distinct collection scenes, with coverage spanning:

Diversity has to be designed into collection, and cannot be filtered in retrospectively. We built a systematic operation to that end: the recruitment and geographic distribution of collectors, the establishment and maintenance of the scene library, the definition and allocation of tasks. Each stage is controlled against coverage targets.

Rich, high-quality annotation

The dataset comes in two lines: EgoStandard, which carries most of the volume, and EgoPro, which adds a wrist-camera view for close-range contact and grasp. Three annotation modalities are available: hand pose and body pose are the primary offerings, while event-level semantic annotation is provided as a complimentary add-on. Coverage varies by subset; see the dataset card for details.

Reliable hand-pose recovery is especially difficult in egocentric video because hands are small, fast, and often occluded at contact. Our in-house algorithms are built for these conditions, while our work with the EgoVerse consortium aligns EgoSuite with emerging standards for data capture, annotation, and sharing.

Get started

The first batch of EgoSuite-Open100K is live on the Hugging Face Hub, open for academic research and commercial training, with the remainder rolling out in stages. It ships in LeRobot v3, training-ready and streamable directly from the Hub, alongside MCAP for teams running their own pipelines.

We expect this data to be put to use, and we want to understand how it performs in practice. If you train or benchmark on it, or identify where it falls short, open a discussion on the Hub or contact us directly. Your feedback will help determine the direction of what we release next.