.. Maple documentation master file ====================================== Maple - Model and Policy Learning Evaluation ====================================== .. image:: assets/MAPLE.svg :alt: Maple Logo :align: center :width: 300px .. raw:: html

A unified CLI daemon for evaluating robotics policies across diverse simulation environments

GitHub PyPI Documentation License: MIT

---- Why Maple? ========== **The Problem** Evaluating robotics policies—whether Vision-Language-Action (VLA) models, foundation models, imitation learning policies, or reinforcement learning agents—across different simulation environments is painful: - **Environment chaos**: Every simulator has its own observation format, action space, and API quirks - **Dependency hell**: MuJoCo, PyBullet, Isaac Gym, LIBERO—each with conflicting dependencies - **Integration tax**: Each policy-environment combination requires custom glue code - **No standardization**: Comparing policies across environments means rewriting evaluation scripts **The Solution** Maple provides a **daemon-based architecture** that: 1. **Containerizes everything** — Policies and environments run in isolated Docker containers 2. **Standardizes the interface** — One CLI to rule them all 3. **Handles the translation** — Adapters automatically convert between policy outputs and environment inputs 4. **Scales evaluation** — Batch evaluation across tasks, seeds, and configurations .. code-block:: bash # Start the daemon maple serve # Pull and serve a policy maple pull policy openvla:7b maple serve policy openvla:7b # Pull and serve an environment maple pull env libero maple serve env libero # Run evaluation maple eval openvla-7b-xxx libero-yyy --tasks libero_10 --seeds 0,1,2 **That's it.** No dependency conflicts. No custom scripts. Just results. ---- Demo ==== .. raw:: html

Demo video coming soon

---- Key Features ============ 🐳 **Docker-First Architecture** Every policy and environment runs in its own container. No more dependency conflicts. 🔌 **Adapter System** Automatic translation between policy outputs and environment inputs. Write once, evaluate everywhere. 📊 **Batch Evaluation** Run evaluations across multiple tasks, seeds, and configurations with a single command. Get aggregated metrics and per-task breakdowns. ⚙️ **Flexible Configuration** YAML config files, environment variables, or CLI flags—use what works for you. 🏥 **Health Monitoring** Background health checks keep your containers running. Auto-restart on failure. 💾 **Persistent State** SQLite-backed state storage. Resume evaluations, track history, query results. ---- Architecture ============ .. code-block:: text ┌─────────────────────────────────────────────────────────────┐ │ maple CLI │ └─────────────────────────┬───────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Maple Daemon │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │ │ │ Policy │ │ Environment │ │ Adapter │ │ │ │ Backends │ │ Backends │ │ Registry │ │ │ │ │ │ │ │ │ │ │ │ • OpenVLA │ │ • LIBERO │ │ • openvla:libero │ │ │ │ • SmolVLA │ │ • SimplerEnv│ │ • smolvla:libero │ │ │ │ • ... │ │ • ... │ │ • ... │ │ │ └──────┬──────┘ └──────┬──────┘ └─────────────────────┘ │ └─────────┼────────────────┼──────────────────────────────────┘ │ │ ▼ ▼ ┌─────────────────-┐ ┌─────────────────┐ │ Policy Container │ │ Env Container │ │ (Docker + GPU) │ │ (Docker + X11) │ │ │ │ │ │ HTTP: /act │ │ HTTP: /step │ │ /load │ │ /reset │ │ /health │ │ /setup │ └────────────────-─┘ └─────────────────┘ ---- Supported Policies and Envs =========================== All the list of supported environments and policies can be Found - :doc:`guides/policy_envs`. Feel free to suggest any specific policy or environment you would like to be added in the repository. ---- Quick Links =========== .. toctree:: :maxdepth: 2 :caption: Getting Started guides/installation guides/quickstart guides/configuration guides/policy_envs .. toctree:: :maxdepth: 2 :caption: CLI Reference commands/serve commands/pull commands/run commands/eval commands/policy commands/env commands/list commands/remove commands/sync commands/config .. toctree:: :maxdepth: 2 :caption: API Reference api ----