Meet Vaddoriya·Oct 19Train, Infer, Serve: And How to Know When You Actually Need DistributionTL;DR — Modern AI models grew from “fits on your laptop” to “needs a warehouse of GPUs.” But not every job needs that warehouse. In this…
Meet Vaddoriya·Oct 6Why One GPU Isn’t Enough: The Memory Math of Giant AI ModelsBlog 1 of 3 — “Distributed AI Systems,” distilled for beginners.
Saurabh Arora·Aug 28Distributed Training: Data, Tensor and Pipeline ParallelismWhy one GPU is never enough, what all-reduce actually does, and how to plan TP, DDP and PP for the machines you really have.
Rishi Kumar·Aug 8How Are Large Language Models Trained at Scale?In the previous articles, we explored how Large Language Models are trained, how scaling laws work, and why model size, training data, and…
Samuel Adeyeye·Jul 31Inside a Live 35B MoE Fine-Tune: What Qwen3.5–35B-A3B Training Logs Really Tell YouA practical walkthrough of supervised fine-tuning a multimodal Mixture-of-Experts model on a four-node DGX Spark cluster — decoded from…
InOpen IntelligencebyDr. Fadi Shaar·Jul 27Training Models Bigger Than Your GPU: A Complete Guide to DeepSpeedHow Microsoft’s Optimization Library Turned Extreme Scale Deep Learning Into a Practical, Everyday Tool
Min Htet Myet (Mattral)·Jul 24MoE-Engine Part II: I Promised Real GPU Numbers.Here’s What Happened When I Finally Got Them!
Venkata Srinivas Kantamneni·Jul 16The Judgment Layer: What Separates Cloud AI Systems That Endure From Those That CollapseA field guide to architecting production AI across AWS, Azure, and Google Cloud — where the hard part was never the model
Mohamed Ahmed Mostafa·Jul 15WDLC: From a Web Deep Learning Compiler to a Collaborative Distributed Training PlatformAlhamdulillah, I have successfully defended my graduation project: WDLC (Web Deep Learning Compiler/Collaborative).
InDev GeniusbyMin Htet Myet (Mattral)·Jul 14A Silent Crash in Megatron-LM’s Gradient Clipping (and a Reviewer Who Made My Fix Better)A Reliability Bug in Megatron-LM’s Gradient Clipping and Why the Simplest Fix Isn't Always the First One You Write.A response icon1A response icon1
