Machine Learning

Artificial Intelligence

Large Language Models

Model Quantization

Quantization

Explore recommended reads, summarized by AI

Machine Learning Optimization

Techniques for reducing model size and improving inference speed through quantization.

LLM Quantization Methods

Quantization Implementation and Training

Popular this week

Enrico Papalini

Which Local LLM Should You Run on a 16 GB Mac mini?

Simrangrover

Quantized a Model to INT8. It Got Smaller, but Slower. Why?

Prabhudev Guntur

Ternary Bonsai 2 Packs a 27B Model Into 8GB and Claims 98.2%. I Gave It Five Real Jobs.

In

OpenVINO-toolkit

by

Passavee Losripat

GSoC 2026: Optimizing Quantized Model Inference Performance on ARM Devices with OpenVINO

In

Intel Tech

by

Intel

Scaling Intel® Chatbot Toolkit with Nutanix Enterprise AI on Intel® Xeon® 6

Selected by Medium

In

Towards AI

by

Satsawat Natakarnkitkul (Net)

Quantization Is Four Decisions, Not One

Latest stories

Mohamed Arbi Nsibi

Qdrant 1.20: BM25 Without Sparse Vectors, Turbo16 & Turbo8

Simrangrover

Quantized a Model to INT8. It Got Smaller, but Slower. Why?

Tanishq Shukla

Do you really need a 7B model for tool calling?

Hannah Igboke

How to Make a 14GB Model Lose Weight

Kevin Joy

Running Real-Time Segment Anything (MobileSAM) and YOLO in the Browser with ONNX Runtime & WebGPU

See more stories

Who to follow

Popular this week

Enrico Papalini

Which Local LLM Should You Run on a 16 GB Mac mini?

Simrangrover

Quantized a Model to INT8. It Got Smaller, but Slower. Why?

Prabhudev Guntur

Ternary Bonsai 2 Packs a 27B Model Into 8GB and Claims 98.2%. I Gave It Five Real Jobs.

In

OpenVINO-toolkit

by

Passavee Losripat

GSoC 2026: Optimizing Quantized Model Inference Performance on ARM Devices with OpenVINO

In

Intel Tech

by

Intel

Scaling Intel® Chatbot Toolkit with Nutanix Enterprise AI on Intel® Xeon® 6

Selected by Medium

In

Towards AI

by

Satsawat Natakarnkitkul (Net)

Quantization Is Four Decisions, Not One

Latest stories

Mohamed Arbi Nsibi

Qdrant 1.20: BM25 Without Sparse Vectors, Turbo16 & Turbo8

Simrangrover

Quantized a Model to INT8. It Got Smaller, but Slower. Why?

Tanishq Shukla

Do you really need a 7B model for tool calling?

Hannah Igboke

How to Make a 14GB Model Lose Weight

Kevin Joy

Running Real-Time Segment Anything (MobileSAM) and YOLO in the Browser with ONNX Runtime & WebGPU

See more stories

Who to follow