InProfit4Cloud — AIbyIddo Kerkhof·2d agoPushing the DGX Spark to the Max with Qwen 3.6 and 3.8 Side by SideGitHub: https://github.com/Profit4Cloud/dgx-spark-vllm
Andolasoft·2d agoAir-Gapped AI Analytics: Apache Superset with Local LLMs (Ollama and vLLM)Somebody senior has seen a demo. A business user types “which branches had the worst collection efficiency last quarter” and a chart…
Allen Kuo (kwyshell)·3d agoAnother Route to vLLM-Style Serving, Part 2: CPU, Vulkan, and GGUF Across PlatformsHandwritten and Triton-CPU kernels, Vulkan tiling and data reuse, GGUF format costs, and the gaps that remain
Allen Kuo (kwyshell)·3d agovLLM 式 serving 的另一條路（下）：CPU、Vulkan 與 GGUF，走向多平台手寫與 Triton-CPU 的混合 kernel、Vulkan 的分塊與資料重用、GGUF 的格式代價，以及還沒追上的地方
Rishabh Nimje·5d agoStop Recomputing What Your AI Agent Already Knows: Prefix Caching with SGLang and vLLMImagine a coding agent at Turn 20.
InLevel Up Codingbyamankarki·5d agoMy GPU fix was 13% faster on one card and 81% slower on anotherA vLLM int8 GEMM change, a reviewer with a smaller Ada GPU, and the one idea that explained it: wavesA response icon1A response icon1
Prashanth Lande·6d agoServing an OCR Model in Production with vLLM: What It Does and What I LearnedI served an open-source OCR model with vLLM, benchmarked it on one GPU, and watched it in production. Here’s what each step taught me.
Varun Nuthalapati·Oct 27Netflix’s New Recommender Is an LLM That Never Writes a WordGenRec turns your viewing history into a prompt, scores the whole catalog in one forward pass, and never decodes a single token. Here’s how…
InTowards AIbyCameron Jagoe·Oct 27I Ran One Local Agent for 22 Hours on a Single DGX Spark. It Timed Out 1,330 Times.One DGX Spark, one Qwen3.8 27B, one agent, 21 hours 36 minutes of wall clock. The session export logs 2,874 model steps, 3.77M output…A response icon1A response icon1
Prashant Dandriyal·Oct 27vLLM on a 4070 Ti, Part 2: Eager Mode, 4-bit Weights and FP8 KV CacheLet’s make the benchmarking more realistic by
