Beyond Kinematics: Benchmarking Simulation Fidelity for Muscle-Driven Imitation Learning
Ayah G. Ahmad, Claire E. Borden, Maegan Tucker
cs.LG · cs.RO
Quick verdict
Claims HyFyDy's physiological fidelity gives better EMG-aligned muscle activations than MuJoCo in MIRL gait; credible but confounded by differing rewards.
Summary
The paper benchmarks two motion-imitation reinforcement learning (MIRL) pipelines for musculoskeletal simulation: HyFyDy-IL (SCONE/HyFyDy, CPU, SAC) and MuscleMimic (MuJoCo/MyoSim, GPU/JAX, PPO), asking whether either reproduces not just kinematics but muscle activations. Both frameworks are trained on the same 140-second treadmill walking clip of subject AB08 from the Scherpereel et al. dataset and evaluated against synchronized motion capture and surface EMG for eight lower-limb muscles, using pooled RMSE and Pearson r over gait-cycle-normalized, speed-binned waveforms. In matched 2D models (H1090 vs MM240-2D), kinematic accuracy is comparable (pooled RMSE/r 11.88/0.89 vs 14.65/0.76), but HyFyDy activations track EMG substantially better (pooled 0.164/0.40 vs 0.344/0.11). Personalizing the HyFyDy model (H-AB08) and increasing dimensionality (H2190, MM240-3D) both degraded results; MM240-3D never walked stably despite 36 GPU-hours and is excluded, and three reward-matched MuscleMimic variants failed to take a step. The authors conclude HyFyDy's physiological realism currently makes it more suitable for activation prediction, while neither pipeline is yet reliable for assistive-device design. Key limitations: a single subject and 140-second clip, no seeds/variance or statistical tests, differing reward structures that confound the simulator comparison, a numeric inconsistency for MM240-2D pooled values between text (0.312/0.16) and Table II (0.344/0.11), and unspecified EMG preprocessing details.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- With matched 2D models and identical training data, both pipelines reproduce gait kinematics comparably, but HyFyDy-IL predicts muscle activations substantially more consistent with experimental EMG (pooled RMSE/r 0.164/0.40 vs 0.344/0.11, Table II).
- Increasing model dimensionality degraded both kinematic and activation prediction for both frameworks (H2190 vs H1090; MM240-3D never walked stably despite 36 A100-hours and was excluded, Sec. IV-C).
- Subject-specific scaling of the HyFyDy model (H-AB08) did not improve predictions over the generic H1090 model (Sec. IV-B).
- Neither framework is yet reliable for muscle-level outcome prediction in assistive-device design; HyFyDy's physiological realism currently favors activation prediction, but it lacks GPU parallelizability (Sec. V).
Strengths
First head-to-head of two state-of-the-art MIRL pipelines on identical data
HyFyDy-IL and MuscleMimic are trained on the same 140-second AB08 clip (Sec. III-B) with deliberate model alignment: MyoFullBody's 416 muscles are regrouped into MM240's 240 actuators to mirror H1090's 90-muscle set, arms removed, torso fused (Sec. III-D).
Evaluates muscle-level outcomes, not just kinematics
Against eight lower-limb EMG channels with pooled RMSE and Pearson r (Sec. IV, Table II) — the outcome measures assistive-device design actually optimizes (Sec. I), which prior kinematics-only validations left untested.
Honest negative results reported throughout
Systematic decomposition of confounding factors
Weaknesses
Simulator comparison confounded by reward structure
Internal numeric inconsistency in the headline result
No statistical rigor or run variance
EMG comparison pipeline underspecified
External validity and asymmetric coverage
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether state-of-the-art muscle-driven motion-imitation RL (MIRL) pipelines reproduce not just gait kinematics but the neuromuscular behavior behind them — a prerequisite for using musculoskeletal simulation in assistive-device design, where activation patterns and metabolic cost are optimization targets (Sec. I). It benchmarks HyFyDy-IL (SCONE/HyFyDy: CPU, SAC, ~7000 Hz error-controlled integrator, elastic tendons, variable pennation) against MuscleMimic (MuJoCo/MyoSim: GPU/JAX, custom PPO, 1000 Hz, inelastic tendons). Four contributions are claimed: an EMG-based activation benchmark, the first direct HyFyDy-vs-MuscleMimic comparison, systematic personalization and dimensionality studies, and released evaluation artifacts. Headline claim: with comparable kinematics, HyFyDy activations align better with experimental EMG (pooled RMSE/r 0.164/0.40 vs 0.344/0.11, Table II).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper compare?
It compares two musculoskeletal reinforcement-learning pipelines for imitating human walking: HyFyDy-IL (SCONE/HyFyDy simulator, CPU, SAC) and MuscleMimic (MuJoCo/MyoSim, GPU, PPO). Both were trained on the same 140-second treadmill walking clip of one subject (AB08) and evaluated on how well their joint angles and predicted muscle activations match motion capture and surface EMG measurements.
Which simulator predicted muscle activations better, HyFyDy or MuJoCo?
HyFyDy/SCONE: in the matched 2D-model comparison, its pooled activation RMSE was 0.164 with Pearson r 0.40, versus 0.344 and 0.11 for MuJoCo's MuscleMimic (Table II). Kinematic accuracy was similar for both, and neither pipeline matched the EMG closely overall.
Did 3D models improve the results?
No. The 3D HyFyDy model (H2190) scored worse than its 2D counterpart on both kinematics and activations, and MuscleMimic's 3D model (MM240-3D) never walked stably despite 36 hours of training on an NVIDIA A100 GPU, so it was excluded from the statistics. More model complexity hurt both frameworks.
Is the data or code from this study available?
The authors state they provide OpenSim-format reference data retargeted for MuscleMimic and a trained subject-specific HyFyDy model to enable future benchmarking, but no repository URL appears in the text. The underlying walking/EMG dataset (Scherpereel et al. 2023) and the SCONE simulator are publicly available.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1, 4 and 5 — the waveform evidence behind all qualitative agreement claims — cannot be verified from the extracted text; assessments rely on captions and tables.
- The appendix with plots of additional (non-primary) muscles is referenced but absent from the extraction.
- The Sec. IV-A vs Table II discrepancy (0.312/0.16 vs 0.344/0.11) could partly stem from text extraction, though both values appear coherent in their contexts.
- Contribution 4 promises data/model release but no repository URL appears in the provided text; availability and completeness are unverified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.