Will Youzhi Zou·Oct 23Serving a capable OSS agentic model on accessible GPUs: Qwen3.5–27B on A100–40GB and H200System Dashboard
Will Youzhi Zou·Oct 11How Easy Is It to Start Serving LLMs?How we stood up Qwen3 on a single A10 or A100 from a plain Ubuntu box, benchmarked it, and opened up FlashInfer to see why it works. All…A response icon1A response icon1
Abinesh Siva Manikandan J R·Sep 25Serving Intelligence at Scale: A Deep Dive into vLLM, SGLang, and NVIDIA DynamoWhy the hardest part of AI isn’t building the model — it’s answering millions of questions a second without going broke
himanshuprajapati·Aug 27KV Cache: A Necessary Evil · Part 2: From One GPU to a Whole ClusterWhat happens when the cache outgrows the machine it lives on, and why companies now build entire clusters around it.
Furkan Zümrüt·Aug 17Serving GLM-5.2 with 160 GPUs on GKE: Prefix-Aware Routing with llm-d, vLLM, and LMCacheDistributing traffic across a large language model running on 20 GPU servers is a more complex problem than traditional round-robin or…
himanshuprajapati·Aug 11My laptop reads a gigabyte to write one wordTL;DR: I am at the very beginning of my AI inference journey. I ran a small language model on my 8 GB MacBook Air using llama.cpp, noticed…
Aniket Hingane·Aug 7Self-Hosting Inference Is a Control Decision Wrapped in a Cost DebateA decision audit for when owned GPUs beat hosted tokens, when they silently lose, and why most production teams end up routing both
InAI MindsetbyDr. Fadi Shaar·Aug 3The Hidden Bottleneck in AI at Scale: Why Serving a Trillion Parameter Model Is a Memory Problem…Inside the Systems Engineering Challenge Behind Large Scale LLM Serving, and Why PagedAttention Changed the Math
QuarkAndCode·Jul 20Serving LLMs with APIs and Local Runtimes: A Practical GuideA model checkpoint is not the same as an application. Think of it as an engine that lacks controls, safety features, or any practical way…
