🧭 AI Lineup
llama.cpp
by Georgi Gerganov (open source)
The lean open-source engine that quietly runs AI models on ordinary computers. Powerful, free, and not for the faint-hearted.
FreeRuns LocallyOpen Source
The details
- What it is
- A lightweight, open-source engine for running AI models on everyday computers — even without a high-end graphics card. It's the foundation many friendlier local-AI apps are built on.
- Cost
- Free — open source.
- Open Source?
- Yes — fully open
- What you need
- Mac, Windows, or Linux. Comfort with a command line helps — most people use it through a friendlier app instead.
Best for
- Running models efficiently on modest hardware
- Developers and tinkerers
- The engine behind other local tools
Kevin’s take
“llama.cpp is the engine under a lot of the local AI world — a stripped-down bit of software that runs models efficiently on normal hardware, even without a fancy graphics card. Most people never touch it directly; friendlier apps like LM Studio sit on top of it. But if you go deep into local AI, you'll hear its name constantly. Now you'll know what it is.”
— Kevin, formerly of IT Support
Related reading
From the AI Glossary
- Open SourceSoftware whose underlying code is made freely available for anyone to inspect, use, and modify. The idea predates AI by decades and powers much of the internet. In AI the label is used loosely, and often a bit generously: some models share everything, others share only parts and still claim it. Genuinely open AI lets people see how a tool works and run it themselves, rather than taking a company's word for it.
- Open WeightsA model whose trained internals, the weights, have been released publicly, so anyone can download and run it themselves. Meta's Llama models are a well-known example. It's not quite the same as fully open source, because the recipe and data used to build the model aren't always shared. But open weights let people run capable AI on their own machines, without sending anything to a company's servers.
- QuantizationA technique for shrinking an AI model by storing its numbers less precisely, which makes it smaller and faster, usually with only a small loss in quality. Think of it like compressing a photo: the file gets much smaller, and most people can't see the difference. Quantization is part of how large models get squeezed onto laptops and phones.
Knowing the tool is just the start.
We teach non-tech people how to actually use AI — what to ask, how to ask it, and what to do with the answers. Tutorials, prompts, plain English. All of it.
Show Me How This Works →Start free. Cancel anytime. No judgment, ever.