Quantization
Definition
A technique for shrinking an AI model by storing its numbers less precisely, which makes it smaller and faster, usually with only a small loss in quality. Think of it like compressing a photo: the file gets much smaller, and most people can't see the difference. Quantization is part of how large models get squeezed onto laptops and phones.
Why It Matters
It's a big part of why AI that once needed a data centre can now run on your own device. If you're interested in private, local AI, this is one of the words that makes it possible.
What does that look like in practice?
A big model gets quantized down so it runs on your phone, a bit like saving a photo at a smaller file size.
What people actually mean when they say this
When someone says a model was 'shrunk to run locally,' quantization is usually how: trimming precision to save space and speed, with little quality lost.
Last reviewed: June 2026
Related reading
In the Privacy Decoder
- Hermes AgentAn open-source personal AI agent that runs on your own computer, remembers your projects over time, and reaches you through messaging apps. A close cousin of OpenClaw.Low
- LM StudioA friendly desktop app for downloading and running AI models entirely on your own computer. It looks like a normal chat app, but everything runs offline.Low
- OpenClawAn open-source personal AI agent that runs on your own computer and connects to your apps. You message it through WhatsApp, Telegram, or Discord, and it carries out tasks for you.Low
- QwenAlibaba's capable AI model, strong in many languages and at coding. Available both as a hosted chatbot and as open models you can download.High
In the AI Lineup
- Hermes AgentAn open-source personal agent that lives on your computer, remembers your work, and reaches you through Telegram, Discord, or Slack.
- llama.cppThe lean open-source engine that quietly runs AI models on ordinary computers. Powerful, free, and not for the faint-hearted.
- oMLXA newer way to run AI models fast on Apple Silicon Macs, built to make the most of Mac hardware. Mac-only, and still early.
- MLXApple's own open-source framework for running AI on Mac hardware. The foundation, more for builders than for browsers.
See also
Knowing the word is just the start.
We help non-tech people go from looking things up to actually using AI confidently. Tutorials, prompts, plain English. All of it, for less than a cup of coffee a month.
Show Me How This Works →Start free. Cancel anytime. No judgment, ever.