Guoliang(Gavin) Zhu·2d agoQuantifying the garbage-collection overhead in the control plane of agentic RL trainingThis post looks at the layer that sits before rollout generation in RL training. Public reports agree that inference is the bottleneck of…
Guoliang(Gavin) Zhu·Oct 27How verl’s single controller worksRecently while chatting with claude, I came across verl and its single controller way of translating training.
Guoliang(Gavin) Zhu·Oct 21Understanding the dynamics of llm [Part 0]Back in the days when I was working on compiler, I always keep an motion in my brain of how instruction and data are feed to…
Guoliang(Gavin) Zhu·Oct 20Open problems on optimizing coding agent servingHere is some characteristics of coding agent and some open problems in my practice.A response icon1A response icon1
Guoliang(Gavin) Zhu·Mar 17Test time training to discover: ideas learnedReally enjoyed reading this paper, as a developer from system software, I always thought that the so called hardcore problem such as GPU…
Guoliang(Gavin) Zhu·Feb 26How memory is consumed during nemo RL GRPO trainingRecently I am working on tinkering post training. I have to figure out how model weight, optimizer state is managed.
Guoliang(Gavin) Zhu·Feb 24PPO Mock questions Q1You are training a model with PPO for RLHF. After 500 steps:
Guoliang(Gavin) Zhu·Feb 10Kaiming initialization in nemo RLKaiming initialization is a weight initialization strategy for ReLU-family activations.
Guoliang(Gavin) Zhu·Feb 2The learning rate that collapsed my trainingToday, during a reproduction run of RL, I encountered a training collapse.
Guoliang(Gavin) Zhu·Jan 25How to throw exception in pythonFirst. exception itself can be considered a deviation from normal control flow.
