Gymnastics → Deep RL Imitation Project - Working Notes

A project to reproduce my own gymnastics skills on a simulated character using deep RL motion imitation, and write it up as a post. Below is the plan and the reasoning we worked through.

The core idea

Take video of my own gymnastics, extract the motion, and train a physics-based simulated humanoid to reproduce it via RL imitation (DeepMimic-family methods). Show “me doing X” next to “an agent I trained doing X.” Along the way, learn reward shaping, rigid-body dynamics, and how compliant contact surfaces behave.

Two axes that decide how hard any move is

Every skill has to be judged on two independent axes, and they often disagree:

  1. Extraction - how cleanly can monocular video be turned into 3D reference motion?
  2. Simulation - how hard is the move to reproduce in a physics sim?

A move can be easy on one axis and brutal on the other. Keeping them separate is the key mental tool.

What breaks extraction (video → 3D pose)

  • Fast axial twisting (e.g. the Randy’s 2.5 twists) - estimators miscount/jitter through rotation.
  • Inverted & arched positions - front/back flips and left/right limb swaps are common failure modes.
  • Apparatus occlusion - bars, uprights, cables, and straps cross the body; the estimator has no model of the apparatus.
  • Motion blur, small-in-frame, foreground clutter - mundane but real.

What breaks simulation

  • Free flight is the EASY case - once airborne, angular momentum is conserved; it’s just a rigid body in the air.
  • Compliant / moving contact anchors are the HARD case - trampoline beds, springy bars, and especially rings cables store and return energy and exchange momentum with the body. Modeling this accurately is research-grade.
  • Strength holds (cross, planche, maltese) are trivial for a torque-rich humanoid and only meaningful if actuator strength is calibrated to human levels - a separate modeling mini-project.

Triage of my actual footage

Footage Extraction verdict Simulation verdict
Trampoline flips/twists Friendlier than expected - whole body in frame, clean wall behind, subject frozen (camera panned). Inverted & back-to-camera frames are the risky ones. Springy trampoline bed = compliant contact. Use for aerial imitation; not for takeoff physics.
Tkatchev (high bar) Worst case. Bar/uprights/cables slice through the body; release & re-grasp (the key frames) are the most occluded. Bad - intermittent grip contact + elastic bar. Frontier material, not a first result.
Rings (Yamawaki = double front swing-through, no release) Moderate–hard. Anchor is overhead so torso/legs swing in open space, but gripping hands lost in cable clutter; piked/inverted phases risk flips. Hardest version - fully contact-coupled start to finish, compliant moving anchor, no rigid-body segment to retreat to. Capstone move.

Key note on the Yamawaki: because it’s a swing-through (no release), there is no free-flight phase, so the “start the sim at the release point” shortcut does not apply. That shortcut only works for released dismounts.

Why robots can do this if simming is so hard

They don’t beat the hard simulation - they dodge it: - Choose achievable moves (backflips are fixed rigid-body ballistic moves; they avoid rings/high-bar-style compliant anchors by construction). - Sim-to-real with thousands of parallel sims on a single GPU (Isaac Gym). - Domain randomization - randomize friction, mass, motor delay, contact stiffness, noise across thousands of sims and train a policy robust to all of them, instead of modeling contact accurately. - Known rigid bodies - robot link lengths/masses/motor limits are specs, so they skip the whole video-extraction / human-capture pain that is half of my project.

My project is unusually hard precisely on the two axes robots most avoid: contact-rich apparatus and human motion capture from video. That’s also what would make the post distinctive.

Architecture (smaller than expected)

  • Networks are just MLPs (~2 hidden layers, ~1024 → 512). Millions of params, single GPU. The neural net is not the bottleneck - sim throughput is.
  • Action = PD targets, not torques. Policy outputs target joint angles; per-joint PD controllers compute torques. This is most of why DeepMimic is stable.
  • State = per-body position/orientation relative to root, linear & angular velocities, plus a phase variable φ ∈ [0,1]. Optionally the target pose at current phase.
  • Reward = weighted sum of exponential-of-negative-squared-error tracking terms:
    • pose (joint orientations) ≈ 0.65
    • end-effector (hand/foot positions) ≈ 0.15 (disproportionately important for landings)
    • velocity ≈ 0.1
    • center-of-mass ≈ 0.1
    • The reward is “match the footage,” not “do a flip” - the footage carries the flip.
  • Algorithm: PPO (default) or SAC. TRPO is historical; don’t bother.
  • Single-clip tracking → DeepMimic. Library of many clips → AMP (adversarial motion prior, adds a discriminator) → ASE for one agent doing many skills on command. Newer adversarial methods (e.g. ADD, 2025) improve exact-reference fidelity over AMP.

The three techniques that make a salto actually learnable

  • Reference State Initialization (RSI) - start episodes at a random phase, not frame 0, so the agent can practice the aerial phase in isolation. Near-mandatory for acrobatics.
  • Early termination - end the episode the moment torso/head drops below a height threshold or a non-foot body hits the ground. Stops wasted flailing samples.
  • Landing is last to click - expect ugly landings; the end-effector and CoM reward terms are the knobs.

Compute

  • Do NOT get a cluster. Single GPU is correct.
  • Isaac Lab/Gym runs thousands of parallel envs on one GPU - that’s the whole point, and what PHC/MaskedMimic assume.
  • A single RTX card (mine, or one cloud GPU e.g. AWS g5.xlarge / Lambda / Vast) beats any cluster here.
  • Colab disconnects will kill multi-hour runs - checkpoint obsessively or avoid.

“Tagging” the videos - what it actually means

No per-frame joint annotation by hand. The work is: 1. Segment footage into single-move clips. 2. Label each clip with the move name (metadata). 3. Curate for quality (cull blur/occlusion/out-of-frame). The automatic pose model produces the joint angles.

Relevant state of the art (2025)

  • VideoMimic and PhysHMR are essentially this pipeline (monocular video → humanoid RL). A gift to build on - and a signal to differentiate via my unusual, extreme gymnastics content.
  • Pose estimation tools: WHAM, GVHMR, TRAM (world-grounded, handle moving cameras). Extreme rotation/inversion is out-of-distribution for them → budget cleanup time.
  • PHC / MaskedMimic - robust SMPL-humanoid imitation controllers in Isaac Gym; using an SMPL-shaped humanoid makes retargeting nearly free.

The plan - one new hard thing per project

Project 1 (prove the pipeline): roundoff → back handspring → layout from tumble track. - Free flight, ground contact, no springy anchor to model. - Film on the track but retarget onto a RIGID floor in sim - don’t model the track’s bounce. - If the 3-skill pass fights back, isolate just the layout (salto) first, then prepend the roundoff + handspring. - Pipeline: WHAM/GVHMR → SMPL pose params (hand-check & fix inverted frames) → SMPL-shaped humanoid (free retarget) → fork an existing DeepMimic/AMP example (MuJoCo or Isaac) → train with RSI + early termination. - Expectation: getting it to run is a weekend; a crisp stuck landing is the follow-on grind (mostly reference cleanup + landing tuning).

Project 2 (add the twist): the Randy. Same machinery as Project 1; the ONLY new hard thing is twist extraction. Small delta, not a restart.

Project 3 (add the apparatus): rings. - Start with a dislocate (clean sagittal-plane extraction) to get a basic grip-contact model working. - Then the Yamawaki swing-through as the capstone - full-motion cable coupling, the deep end. - Optional counterpoint: a strength hold (cross/maltese) to illustrate strength-limited vs momentum-limited regimes on the same apparatus.

Capture tips for dedicated shoots

  • Tripod (no panning), side-on to the rotation/twist axis, close enough to fill the frame.
  • Second phone at ~90° - turns twist extraction from “fight” to “fine.”
  • For any release/apparatus move, position a camera to shoot along the bar/rings, not across, so the release comes out from behind the cables.

Post structure (writes itself)

Free flight → twist → compliant apparatus contact. Each stage introduces exactly one new hard thing - which is both good narrative and correct engineering (never debug two unknowns at once). Bonus angle: document the contact-rich, human-capture parts that polished robot demos quietly avoid.

Open next steps

  • Pick a specific starter repo to fork (DeepMimic/AMP/PHC).
  • Go deeper on the pose-cleanup step - the real bottleneck.