Open Source Humanoid AI trained collaboratively by a community of builders.
First Milestone:
Ultimate Goal and Massive Transformative Purpose:
The zk0.bot SmolVLA Federated Learning model is now available on Hugging Face Hub!
You can also read this post with more detailes about the SmolVLA FL milestone.
- Model: ivelin/zk0-smolvla-fl
- Training: 30 rounds of federated learning with FedProx (ΞΌ=0.01)
- Final Policy Loss: 0.544
- Clients: 4 clients on diverse SO-100 robotics tasks
- Framework: Flower + SmolVLA + SO-100 datasets
from transformers import AutoModel, AutoConfig
import torch
# Load the federated model
model = AutoModel.from_pretrained("ivelin/zk0-smolvla-fl")
config = AutoConfig.from_pretrained("ivelin/zk0-smolvla-fl")
# Ready for robotics manipulation tasks!AI technology has advanced enough to speculate that within a decade most people will have their own humanoid buddy. By some estimates humanoids will become $100 Trillion market (5B humanoids * $20,000 per unit).
Today's leading closed source humanoid is trained on 100,000+ GPU farm with real world data collected from millions of cars labeled by able human drivers and a growing number of humanoid robot prototypes used in real world manufacturing environment. This is an enormous scale of compute and data that is hard to compete with as a centralized entity. However it would be interesting to see if a decentralized approach might produce useful results over time. On the chance that proprietary humanoids ever go rogue, it would be nice to have open source alternatives.
- Register now for the zk0 event at the upcoming DevConnect conference in Buenos Aires, Argentina on November 18, 2025.
- Watch a recorded presentation of the project at the Flower Monthly Webcast.
Join our Discord server to connect with other contributors, ask questions, and stay updated on the latest developments:
For more detailed contribution guidelines, see CONTRIBUTING.md and docs/DEVELOPMENT.md.
zk0 is composed of several major building blocks:
- Physical Embodiment:
- Open Source 3D printed robot parts
- Base 3D model so100 series from HuggingFace LeRobot
- Generative AI:
- End-to-end Vision Language Action models.
- Base SmolVLA model from HuggingFace LeRobot
- Federated Learning:
- Distributed network of nodes contributing local data and training compute to a shared model.
- FL framework: Flower
- Zero Knowledge Proofs that allow quick verification and data privacy:
- Onchain contributor coordination
- Immutable contribution history
- Programmable network participation rules, incentives and project governance
- Hosting blockchain: TBD
For detailed setup, see docs/INSTALLATION.md.
- Python 3.10+, Conda, Git.
- NVIDIA GPU recommended.
git clone <repository-url> .
cd zk0
# Create conda env
conda create -n zk0 python=3.10 -y
conda activate zk0
conda install ffmpeg=7.1.1 -c conda-forge
# Install deps
pip install -r requirements.txt
pip install -e .
# Env vars
cp .env.example .env # Edit as needed (e.g., HF_TOKEN)See docs/RUNNING.md for full instructions.
# Quick test (1 round, serialized GPU)
./train.sh
# Full run (5 rounds)
conda run -n zk0 flwr run . local-simulation-serialized-gpu --run-config "num-server-rounds=5"
# Docker alternative
./train.sh --docker- Defaults: 500 rounds, 4 clients, SO-100/SO-101 datasets.
- Outputs:
outputs/<timestamp>/with logs, metrics, charts (eval_policy_loss_chart.png), checkpoints (.safetensors), videos.
Tested: Completes 500 rounds in ~10-15 minutes; policy loss tracks convergence with early stopping.
- main: Stable releases. Use this for production setups and quick starts.
- staging: Final polish before merging with main. No new features. Only bug fixes and docs polish.
- dev: Active feature development. Pull requests should target dev. Clone or switch with
git checkout devfor latest features (may be unstable).
Advanced development with core FL for SmolVLA on SO-100/SO-101. Recent updates: Enhanced security, consolidated metrics, early stopping, dynamic learning rate.
- β Core Infrastructure: Flower 1.20.0 + Ray 2.31.0 + LeRobot 0.3.0.
- β Client Implementation: SmolVLA training, dataset partitioning.
- β Testing: 80%+ coverage, unit/integration suites.
- β CI/CD: GitHub Actions, auto-testing.
- β Config/Tooling: YAML datasets, env management.
- β Enhanced Security: Bidirectional SHA256 parameter validation.
- β Consolidated Metrics: Server-side evaluation files now include both aggregated and individual client metrics with dataset identification (v0.1.19).
- β Early Stopping: Configurable server-side early stopping with patience-based termination.
- Multi-task learning, advanced strategies (FedProx+), hyperparam tuning.
- ZK proofs, onchain coordination.
Full status: docs/ARCHITECTURE.md. Baselines: docs/TECHNICAL-OVERVIEW.md.
Config: 4 clients (LEGO bin, direction test, plush toy, stuffed animal); 500 rounds; policy loss metric; early stopping (patience=10); FedProx (ΞΌ=0.01).
- Installation: Full setup.
- Running: Execution, outputs, troubleshooting.
- Architecture: FL design, components.
- Development: Testing, logging, guidelines.
- Technical Overview: Comparisons, reproducibility, videos.
Internal: .kilocode/rules/memory-bank/ (e.g., brief.md).
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>It's time for a complete open-source stack for autonomy/robotics plus distributed learning. The first step is here: @LeRobotHF + @flwrlabs LFG π@comma_ai @wayve_ai @Figure_robot @Tesla https://t.co/8O8cSD3SbO https://t.co/oVUOLTvwzm
β nic lane (@niclane7) January 15, 2025
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>Open-source robots just got a boost. Frameworks like Flower FL enable faster learning, efficient scaling, and continuous knowledge sharing using real-world data. https://t.co/j8VSGiWF0W
β ππͺπΎπ‘π‘πΎ (@gm8xx8) January 15, 2025
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>We are not so far from a future where robots will be constantly learning by interacting with humans and their environments.
β Remi Cadene (@RemiCadene) January 15, 2025
Frameworks like @flwrlabs will enable these robots to learn much faster by continuously sharing their learnings.
We really live in a sci-fi movie π https://t.co/kAz3xZ2qvB
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>Federated Learning Meets Robotics: π€ LeRobot + πΌ Flower
β Flower (@flwrlabs) January 15, 2025
This demo demonstrates how robots in remote environments can collaboratively train an AI model using their local data, which is then aggregated into a shared model.
In this quickstart, you will train a Diffusion policy⦠pic.twitter.com/i32MkbxoPW
We welcome contributions from the community! At this Beta stage, we're particularly interested in:
If you have access to a LeRobot SO100 arm (or the newer SO101 version) and a local machine with an RTX 3090 GPU or better compatible with the LeRobot library, we'd love for you to join as a node operator. Your unique training data and compute resources will help improve the federated learning system.
We're also looking for developers to help with:
- Bug fixes and improvements
- Documentation enhancements
- New feature development
- Testing and quality assurance
- Fork the repository
- Create a feature branch
- Make your changes
- Submit a pull request
License: LICENSE
Repository: GitHub
Memory Bank: Internal docs in .kilocode/rules/memory-bank/ for contributors.