Inference
Definition
The moment an AI actually does its job: you give it a prompt, and it produces an answer. Training is when a model learns, which happens once and at great expense. Inference is every time it's used afterwards. When you type a question into ChatGPT and it responds, that response is inference. It's the difference between teaching someone and then asking them a question.
Why It Matters
It explains a lot about cost and speed. Inference is what you're paying for, and waiting on, every time you use an AI tool.
What does that look like in practice?
Every answer ChatGPT gives you is inference: the model doing its job, not learning anything new.
What people actually mean when they say this
When someone mentions 'inference costs,' they mean what it costs to run the model each time it's used, separate from the one-off cost of training it.
Not to be confused with
Last reviewed: June 2026
See also
Knowing the word is just the start.
We help non-tech people go from looking things up to actually using AI confidently. Tutorials, prompts, plain English. All of it, for less than a cup of coffee a month.
Show Me How This Works →Start free. Cancel anytime. No judgment, ever.