Physical AI Notes
KO

Simulator: lessons

A virtual environment for training policies, generating demo data and evaluating before deployment. Also covers matching the simulator to reality (real2sim).

Around the robot

No lessons are published yet. Here are the planned topics.

Overview

The roles of a simulator (policy training, data generation, evaluation), where sim and reality diverge, how to choose a physics engine

  1. The role of simulators: policy training, data generation and evaluationComing soon

    How decoupling simulated time from wall-clock time changed the scale of robot learning.

  2. Where the simulator and reality divergeComing soon

    Five sources of the sim-to-real gap, the gap checklist, and how a large share of the gap traces back to settings people chose.

  3. Criteria for choosing a physics engine: comparing MuJoCo, PhysX, Genesis and NewtonComing soon

    Choosing among them by contact solver, parallelization, rendering and ecosystem.

Basics

URDF, MJCF and USD formats and conversion, a first MuJoCo experiment, setting up an RL environment in Isaac Lab, collecting demos in sim

  1. Robot model file formats: URDF, MJCF, USD and conversion between themComing soon

    Inertia, collision geometry and actuator definitions that get dropped or altered when converting between formats.

  2. A first MuJoCo experiment: the computation order in mj_step and the settings that eliminate sagComing soon

    The order in which forward dynamics, contact and integration are computed within a step, and how gravcomp, high-gain PD and disabling gravity each eliminate sag.

  3. Setting up a first RL environment in Isaac Lab: the manager-based workflow, decimation and parallel trainingComing soon

    Building a baseline locomotion policy using the manager-based workflow, decimation and parallel training, with MuJoCo Playground as the fallback when no GPU is available.

  4. Collecting demonstrations in a simulator: from keyboard teleoperation to the LeRobot formatComing soon

    Teleoperating and recording in simulation, and observing that action and state nearly coincide.

Advanced

How contact is computed, actuator modeling, domain randomization, system identification, automatic demo generation with MimicGen and SkillGen

  1. How contact is computed: penetration, slip and explosionsComing soon

    Soft contact and iterative solvers, and working backward from grasping failures and explosions to their causes.

  2. Modeling actuators in the simulator: from ideal torque sources to actuator netsComing soon

    Modeling armature, friction, torque-speed limits and latency.

  3. Domain randomization: what to randomize and by how muchComing soon

    The principle of identifying whatever can be known and randomizing only what remains unknown.

  4. System identification: a procedure for matching the simulator to realityComing soon

    Excitation trajectories, least squares and CMA-ES, and cross-validation, on a single SO-101 or on a simulator with hidden parameters standing in for the real robot.

  5. How automatic demonstration generation works: MimicGen and SkillGenComing soon

    Transforming demonstration segments relative to objects, and the survivorship bias that comes from keeping only successful demonstrations.

In practice

When simulator evaluation results can be trusted, reviewing simulator settings with the gap checklist, the benefits and limits of co-training with simulation data, using world models as simulators

  1. When simulator evaluation results can be trusted: rank correlation with real-world results, paired evaluation and sim2simComing soon

    Sim-to-real rank correlation, paired evaluation, and detecting loopholes in the simulator.

  2. Reviewing simulator settings with the gap checklist: settings that create the gap and settings that reduce itComing soon

    Finding the settings that remove or create the gap, and deciding which layer absorbs it.

  3. The benefits and limits of co-training: positive and negative transfer across simulation data mixing ratiosComing soon

    Positive and negative transfer as a function of the mixing ratio, checked on a simulator with altered settings standing in for the real robot.

  4. Can a world model serve as a simulator?Coming soon

    Compounding error over rollouts, and whether changing the input action changes the prediction accordingly (controllability).