RLLBC-Lib: An Educational Code Library for Reinforcement Learning and Learning-Based Control
Bernd Frauenknecht, Emma Cramer, Artur Eisele, Paul Kruse, Lukas Kesper, Jonas Hertrampf, Ramil Sabirov, Jyotirmaya Patra, et al.
cs.LG · cs.AI
Quick verdict
An open-source teaching library spanning tabular to deep RL; well designed and documented, but pedagogical benefit claims rest on anecdote, not evaluation.
Summary
The paper presents RLLBC-Lib, an open-source educational code library for teaching reinforcement learning (RL) in the context of learning-based control (LBC), developed at RWTH Aachen and presented at EWRL 2026. The library consists of Jupyter notebooks in three tiers: (1) dynamic programming and tabular RL (policy/value iteration, first/every-visit Monte Carlo, Q-learning, SARSA, Dyna-Q) with custom implementations of Sutton & Barto examples such as the recycling robot; (2) deep RL notebooks (REINFORCE, A2C, TRPO, DQN, DDPG, TD3, SAC) largely adapted from CleanRL single-file implementations in PyTorch; and (3) LBC notebooks covering LQR, MPC, system identification, and Bayesian-optimization-based controller tuning. All notebooks share one structure (setup, hyperparameters, training loop, evaluation), use the gymnasium interface, a pixi-managed environment, and tensorboard/W&B logging with video recording and checkpointing. Three programming assignments with nbgrader-based automated grading - tabular, deep RL, and an open racing challenge with a leaderboard - support teaching at scale; classroom use with several hundred students and two edX MOOCs is reported. Appendix A.1 compares the library against roughly 15 existing teaching repositories. Main limitations: no quantitative evaluation of learning outcomes, limited algorithmic novelty beyond packaging of existing implementations, no benchmark validation of the implementations in the paper, and part of the grading infrastructure is unpublished.
Review complete
Assessment at a glance
3
risk flags identified
● high 0
● medium 1
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- RLLBC-Lib lowers the entry barrier to reinforcement learning by guiding learners from tabular foundations to state-of-the-art deep RL within a single consistent notebook design.
- A shared notebook structure between tabular and deep implementations (e.g., Q-learning vs. DQN) makes the algorithmic continuity between classical and modern RL visible to learners.
- Placing RL alongside other learning-based control methods (LQR, MPC, system identification, Bayesian optimization) builds intuition for when RL is the appropriate tool.
- The library is a suitable basis for programming assignments with automated grading (nbgrader), enabling hands-on RL teaching at scale, as used in courses with several hundred students and two edX MOOCs.
Strengths
Full pedagogical arc from tabular to deep RL
Sections 3-4 plus Appendix A.2 cover dynamic programming, Monte Carlo, TD/Dyna-Q through DQN, TRPO, SAC and more, all following a deliberately identical notebook skeleton (Section 2); the Table 1 survey shows no other repository ticks all five feature columns, supporting the stated scarcity of such libraries.
Anchored in standard research tooling
Gymnasium environments (Sections 2-3), PyTorch with CleanRL-style single-file implementations (Section 4), a pixi-managed environment, tensorboard/W&B logging, evaluation episodes, video recording and best-checkpoint saving (Figure 3) teach the workflows students will actually encounter in RL research.
Unique learning-based control context
Scalable assessment built into the library
Concrete didactic devices
Weaknesses
No empirical evaluation of pedagogical impact
Limited algorithmic novelty
No correctness or benchmark validation reported
Comparison table is admittedly subjective
Incomplete release of the teaching stack
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 3 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is RLLBC-Lib and who is it for?
RLLBC-Lib is a free, open-source collection of Jupyter notebooks from RWTH Aachen University for learning reinforcement learning (RL) and learning-based control. It targets students and self-learners, guiding them from tabular methods (dynamic programming, Q-learning, SARSA) to state-of-the-art deep RL (DQN, TRPO/PPO-family, SAC, TD3), and adds control methods such as LQR and MPC for contrast.
Where can I get RLLBC-Lib?
The library is available on GitHub under the Data-Science-in-Mechanical-Engineering/RLLBC repository, per the paper. It uses a single pixi-managed Python environment and standard gymnasium environments, so all notebooks run from one installation.
How is RLLBC-Lib different from CleanRL or Stable-Baselines3?
CleanRL and Stable-Baselines3 are research-oriented deep RL libraries. RLLBC-Lib is educational: its deep RL notebooks are largely based on CleanRL implementations, but are wrapped in extensive explanations and share a notebook structure with the tabular methods, plus it adds learning-based-control examples and auto-graded programming assignments, which the research libraries do not provide.
Does the paper show that students learn better with RLLBC-Lib?
No. The authors report using it in classroom and online courses with several hundred students and in two edX MOOCs, but the paper contains no controlled study or quantitative measurement of learning outcomes; the evidence is the authors' teaching experience.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 are present only as captions in the extracted text; the visual quality of the illustrations could not be assessed.
- The GitHub repository could not be inspected (no network access in this review); statements about the artifact rely solely on the paper text.
- The provided metadata lists 8 authors while the full text lists 12 (adding Berger, Brunzema, Solowjow, Trimpe); the review assumes the full-text author list.
- The notebook listing in Appendix A.2 may be partially truncated by text extraction; the count of provided notebooks is approximate.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.