HuggingFace Papers
7/10 signal
Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories
modelsresearch
Summary
This paper introduces Xiaomi-Robotics-1, a foundational vision-language-action (VLA) model trained on over 100,000 hours of real-world manipulation trajectories. The model uses a two-stage training recipe: pre-training on a massive dataset of UMI gripper trajectories with auto-labeled state-transition descriptions, followed by post-training to align with robot embodiments and imperative instructions. Experiments show that performance scales with data and model size, and this scaling translates to improved out-of-the-box real-robot performance. The model establishes new state-of-the-art results on multiple simulation benchmarks, including a 57.6% success rate on RoboCasa365, surpassing the previous best of 46.6%.
Problem
Scaling vision-language-action (VLA) models for robotics is hindered by a data bottleneck. The dominant paradigm of real-robot teleoperation is slow, costly, and produces data with limited diversity and high redundancy, making it difficult to collect the massive and varied datasets that have driven progress in large language and vision models. This work addresses the challenge of scalable data collection and annotation to enable the training of generalist robot foundation models.
Method
The model, Xiaomi-Robotics-1, uses a Mixture-of-Transformers (MoT) architecture combining a pre-trained VLM (Qwen3-VL) and a Diffusion Transformer (DiT). Training follows a two-stage recipe. First, it is pre-trained on over 100k hours of real-world UMI gripper trajectories, which are auto-labeled with state-transition descriptions by a scalable pipeline using Qwen3.5-27B. Second, it is post-trained on approximately 10k hours of cross-embodiment data (including in-house robots and open-source datasets like Bridge V2, RT-1, and DROID) to align with robot embodiments and imperative instructions. The model's scaling properties and performance were evaluated on real-robot tasks and four simulation benchmarks: RoboCasa, RoboCasa365, VLABench, and RoboDojo.
Details
Scaling Properties (Pre-training):
- Data Scaling: Using the 5B model variant on a 20k-hour subset of data, validation action MSE consistently decreased as the data fraction increased from 12.5% to 100%. Models trained on 12.5% and 25% of the data showed signs of overfitting.
- Model Scaling: On the same 20k-hour dataset, validation action MSE improved as model size increased from 2B to 5B to 10B parameters. The paper notes that performance gains from data scaling were more pronounced than from model scaling.
Out-of-the-Box Real-Robot Performance (Post-training):
- Models were evaluated on 4 tasks (shoe storage, bag packing, table organization, sofa tidying) in unseen environments.
- Effect of Pre-training Data: The 5B model's overall success rate increased from 26% (no pre-training) to 53% (12.5% of pre-training data) to 75% (100% of 20k pre-training data).
- Effect of Model Size: The overall success rate increased from 61% (2B model) to 75% (5B model) to 79% (10B model). The gain was most significant on the 'shoe tidying' task, rising from 58% (2B) to 92% (10B).
Downstream Fine-tuning on New Tasks:
- The model was fine-tuned on four new tasks: phone packing, laundry loading, printer refilling, and box packing.
- In a low-data setting (avg. <10 hours/task, 36 hours total), Xiaomi-Robotics-1 achieved a 75% average success rate, significantly outperforming π_0.5 (40%).
- On the 'printer refilling' task in the low-data setting, it improved the success rate from the best baseline's 20% to 70%.
Simulation Benchmark Results:
| Benchmark | Xiaomi-Robotics-1 | Previous SOTA |
|---|---|---|
| RoboCasa (Success Rate) | 74.5% | 72.6% (World2Act) |
| RoboCasa365 (Success Rate) | 57.4% | 46.6% (ABot-M0.6) |
| VLABench (Avg. Success Rate) | 59.1% | 53.2% (ERVLA) |
| RoboDojo (Avg. Score) | 20.07 | 13.07 (Hy-Embodied-0.5-VLA) |
- On RoboCasa365, the model achieved its largest improvement on the 'Composite-Unseen' split, with a 32.1% success rate.
- On VLABench, it achieved the highest average success rate (59.1%) and progress score (70.3%), outperforming the strongest baseline by 6.0 and 15.2 percentage points on the cross-category and texture shift tracks, respectively.
- On RoboDojo, it achieved an average score of 20.07 and success rate of 13.93%, compared to the prior SOTA's 13.07 score and 8.80% success rate.
What's new
This work presents a systematic study of scaling properties for vision-language-action models in robotics, enabled by a novel two-stage training recipe. The key contribution is the use of a massive dataset of over 100,000 hours of real-world manipulation trajectories, collected with UMI devices and annotated via a scalable auto-labeling pipeline that generates state-transition descriptions. The paper demonstrates that scaling pre-training data and model size directly translates to improved out-of-the-box performance on real robots.
Limitations
The authors note that for the RoboDojo benchmark evaluation, they did not incorporate history observation. As a result, the model scored lower on the 'memory' dimension compared to Hy-Embodied-0.5-VLA, a model which explicitly models memory.
Conclusion
The paper introduces Xiaomi-Robotics-1, a foundational VLA model that can perform a wide range of mobile manipulation tasks out-of-the-box and adapt efficiently to new tasks. Extensive experiments show that the model's performance consistently improves with increased data and model size during pre-training, and these scaling benefits transfer directly to real-robot performance after post-training. By achieving state-of-the-art results on four challenging simulation benchmarks, the work demonstrates the effectiveness of its large-scale training approach and provides a foundation for future exploration of scalable robot policies.
Don't read this site daily. Get it in your inbox.
The daily brief and Sunday deep dive — distilled, scored, and opinionated. For builders only.