Chhinsoportrakhou·3d agoLlama 3 Code vs. DeepSeek Coder Locally: The VRAM Trap That Freezes IDE IntegrationChoosing a local coding model isn’t just about code quality or parameter count — it’s about whether your GPU can keep the model, context…
Abdulrahman Maslmany·Oct 26Prompt Caching: How to Slash Local LLM TTFT & LatencyAutomatic Prefix Caching (APC) in vLLM & llama.cpp: How Radix trees eliminate prefill computation lag.A response icon1A response icon1
Erwin Hermanto·Oct 19Quantization Showdown: Running Llama 3 on Consumer HardwareGGUF vs. GPTQ vs. AWQ — quality, speed, and memory benchmarks on a MacBook M3
InAI Simplified in Plain EnglishbyFrank Morales Aguilera·Oct 13The Architecture of Permanence: Enforcing Deterministic Rigour in Llama-3.1–70BFrank Morales Aguilera, BEng, MEng, SMIEEE
Kartikey Hebbar·Oct 6Model-as-a-Scorer: How Jev Made Relevancy Scoring 86 Times FasterResearch Scout, an agentic system, was built as a CLI tool to gather relevant research papers and blogs, score them against the user…
Yuvraj·Sep 30Why your local model inferences are slow? (and how quantization fixes it)Running open-source models like Llama 3 or Mistral locally on workstation GPUs often hits memory limitations. Loading full 16-bit float…
surajit002·Sep 13Understanding LangChain IntuitivelyIn my previous article, I wrote up about adding RAG in fleet diagnosis flow. The next step was to understand the concept of LangChain and…
jaytank·Sep 10Demystifying RoPE: How Rotary Position Embeddings Power 128k+ Context WindowsWhy traditional absolute position embeddings fell apart on long documents — and how rotating token vector pairs like hands on a clock…
