RoboSmith

RoboSmith

Agentic Robot Policy Toolchain

Describe the behavior, inspect the robotics artifacts,
design rewards, train policies, and package the result.

LangGraph pipelineinspection toolsreward searchPython API + CLI

The Problem

Robot learning has two recurring kinds of friction.

First, training from scratch is still a pile of hand-written glue. A task starts as plain English, but the stack needs a structured task spec, a matching simulator, a reward function, a trainer, an evaluation plan, and an artifact bundle that someone can actually inspect later.

Second, much of robotics work does not start from scratch. Teams inherit a pre-trained policy, a LeRobot dataset, a rollout file, or a simulator. Before anything useful happens, they have to understand camera names, image sizes, action dimensions, state keys, normalization, and which adapter code is missing.

The failure mode is time lost between the idea and the experiment.

Workflow	Typical failure	RoboSmith response
Train a new behavior	The task is underspecified, reward design is brittle, and training failures do not feed back into the next attempt.	A checkpointed LangGraph pipeline parses the task, searches context, matches an environment, designs rewards, trains, evaluates, and retries.
Reuse existing work	Policy, dataset, and environment schemas are close but incompatible.	Inspectors expose schemas, compatibility checks explain mismatches, and generators produce adapter code.
Debug a run	A single reward number hides why the policy failed.	Diagnostics summarize trajectories, action statistics, failure clusters, and run-to-run deltas.
Manage experiments	Run directories become opaque piles of files.	robosmith runs lists, inspects, compares, cleans, and resumes pipeline runs.

The Thesis

RoboSmith treats robot policy development as an explicit, inspectable flow. Every major step is a typed node. Every output is persisted. Every integration tool uses the same premise: make the hidden interface visible before trying to fix it.

1. UnderstandTurn natural language, policies, datasets, environments, and robots into structured descriptions.
2. ChooseMatch a task to one of 30 registered environments and select a suitable training backend.
3. DesignGenerate, evaluate, and evolve reward candidates using environment introspection and literature context.
4. TrainRun SB3, CleanRL, rl_games, imitation learning, or offline RL through one trainer interface.
5. EvaluateMeasure behavior across seeds, decide whether to accept, refine, switch algorithms, or adjust the environment.
6. PackageWrite the reward, model checkpoint, report, video, task spec, eval report, and resumable state.

Read This In Order

The sidebar is organized as a launch path. Start with the problem and the first run, then move into workflow and reference pages when you need the details.

01Why RoboSmith?The motivation, two-workflow model, and current boundaries.02InstallationPython install, optional extras, API keys, and dependency checks.03Quick StartRun the training pipeline and inspect existing robotics artifacts.04Training PipelineIntake, scout, environment synthesis, reward design, training, evaluation, and delivery.05Integration ToolingInspect, diagnose, generate adapters, and run auto-integration.06Python APIPublic classes, functions, result models, registries, and extension points.

First Minute

Install the project in editable mode with the common simulation, training, and agent extras:

git clone https://github.com/Shaswat2001/robosmith.git
cd robosmith
pip install -e ".[sim,train,agent]"
robosmith deps

Run a dry pass before spending training time:

robosmith run \
  --task "Train a HalfCheetah to run as fast as possible" \
  --dry-run

Inspect existing work without any LLM key:

robosmith inspect policy lerobot/smolvla_base
robosmith inspect dataset lerobot/aloha_mobile_cabinet --schema --quality
robosmith inspect compat lerobot/smolvla_base lerobot/aloha_mobile_cabinet --fix

Use the Python API when you want RoboSmith inside another experiment runner:

from robosmith import TaskSpec, ForgeConfig
from robosmith.agent.graphs.run import run_pipeline

spec = TaskSpec(task_description="A Franka arm picks up a red cube")
config = ForgeConfig(max_iterations=2, scout_source="arxiv")

result = run_pipeline(spec, config)
print(result["run_id"])
print(result["artifacts_dir"])

Current Surface

Environment registry30Classic control, MuJoCo, Gymnasium-Robotics, and Isaac Lab entries.

Training backends5SB3, CleanRL, rl_games, imitation learning, and offline RL behind one interface.

Integration commands4inspect, diag, gen, and auto for working with existing robotics artifacts.

What Exists Today

RoboSmith currently ships a Python package and Typer CLI. The training pipeline uses LangGraph, persists checkpoints after every node, supports stage skipping, and can resume interrupted runs. The integration tooling can inspect datasets, environments, policies, robot descriptions, compatibility reports, trajectories, and generated adapter code.

Some pieces are intentionally still adapter-shaped. Isaac Lab, LIBERO, ManiSkill, imitation learning, and offline RL have interfaces and registry entries, but their practical availability depends on the external packages and installation steps for each framework. The docs call that out instead of treating all backends as equally installed by default.