AGM·Feb 20How Gradient Descent Really Trains GPT‑Like ModelsModern large language models like GPT are nothing more than gigantic collections of numbers: billions of weights that decide which word…
Priyanka Sadam·Feb 10Adam OptimizerBefore jumping into the Adam optimizer, we’ll discuss the drawbacks of Vanilla Gradient Descent.
Siroky Jan·Feb 6SGD, RMSProp, Adam, or AdamW? A High-Stakes Benchmark on Time-Series DataPutting the industry’s top 4 algorithms to the test against 200 years of noisy solar data.
Siroky Jan·Jan 20Searching for the Path to Minimum Loss: A Deep Dive into OptimizersTracing the mathematical evolution from Stochastic Gradient Descent to the adaptive scaling of AdamW.
Mayank Meshram·Oct 9, 2025The Unsung Hero of Optimizers: Understanding Adagrad, The First Step Toward Smarter Learning in…In the grand story of machine learning, not every hero gets the spotlight. We often celebrate Adam, the optimizer that powers most deep…
