Training fits or adapts a machine-learning model using data. Inference uses a fitted model to make a prediction or produce another output, such as an answer.
That distinction explains why an AI answer can change without the model’s learned settings changing. A different input, an added document, or a saved preference can affect the result. Switching to a different model version can change which model responds.
Follow one small delivery-time model through training, inference, and a correction to see what changes at each step. For the broader workflow, see Machine Learning Explained.
Follow one model through three steps
Our deliberately simple model estimates delivery time:
Estimated minutes = 2 × distance in kilometers + offset
The offset is a fixed amount added to every estimate, whatever the distance. Training chooses that amount; it stays fixed while we use the fitted model. We keep the multiplier of 2 fixed throughout and learn only the offset, our model’s one learned numerical setting, or parameter.
These are fictional records chosen so you can check the arithmetic, not evidence of a working delivery forecast’s accuracy.
1. Training changes the saved offset
Start with an offset of 5 minutes. For a 1-kilometer trip, the model predicts 2 × 1 + 5 = 7 minutes. Our training record says that trip took 9 minutes.
Subtract 2 × distance from each recorded time to see the offset each example requires:
- Distance
- 1 km
- Recorded time
- 9 minutes
- Offset needed
- 9 − 2 × 1 = 7 minutes
- Distance
- 2 km
- Recorded time
- 11 minutes
- Offset needed
- 11 − 2 × 2 = 7 minutes
- Distance
- 3 km
- Recorded time
- 13 minutes
- Offset needed
- 13 − 2 × 3 = 7 minutes
All three examples point to 7. Fitting the offset changes the saved model from 2 × distance + 5 to 2 × distance + 7. A value used for future predictions has changed because of the training data.
2. Inference uses the fitted offset
Now keep that fitted model fixed and give it new distances:
- For 6 km: 2 × 6 + 7 = 19 minutes.
- For 8 km: 2 × 8 + 7 = 23 minutes.
The outputs differ because the inputs differ. The offset stays at 7 after both predictions. No fitting step occurred.
3. A correction can be saved without updating the model
Suppose someone reports that the 6-kilometer trip actually took 26 minutes. Saving that result in a database does not, by itself, change the offset from 7. Our original model still predicts 19 minutes for 6 km.
A later training process could include the new record, fit a revised model, evaluate it, and deploy the new version. Logging the correction is one step; fitting a model from it is another.
Ask what changed: the fitted model, the information supplied to it, or the information stored around it?
We can calculate predictions without knowing how long the new trips actually took. To judge their quality, we need suitable real outcomes and an evaluation plan. Also notice that 6 and 8 km lie beyond our training range of 1 to 3 km. This is extrapolation: extending a rule beyond the data used to fit it. The arithmetic alone does not establish that doing so is sensible.
What changes when you prompt an AI assistant
The same distinction helps explain a language model’s responses. For an ordinary request, your prompt and supplied context influence the answer without updating the model’s parameters, such as a neural network’s weights.
Asking for a shorter answer, adding example answers, or supplying a document changes what the model receives. Fine-tuning is additional training that can change the model or selected trainable components. Google’s guide to fine-tuning and prompting explains the difference.
Use these examples to identify what an AI application has changed:
What happened?
You supplied three example answers in the prompt.
- What changed?
- The context for the current request.
- Is that operation training?
- No, for ordinary few-shot prompting.
What happened?
The application retrieved a new policy document.
- What changed?
- The material available to the model.
- Is that operation training?
- Retrieval itself is not training.
What happened?
The application saved your preferred answer length.
- What changed?
- Application data that may be supplied later.
- Is that operation training?
- Saving a preference is not itself model training.
What happened?
A developer fine-tuned a model on reviewed examples.
- What changed?
- Trained parameters or components.
- Is that operation training?
- Yes.
What happened?
The provider switched to a different model version.
- What changed?
- The model used by the application.
- Is that operation training?
- The switch itself is not training; check how the replacement version was produced.
Retrieval and training can belong to the same system. The original RAG research used an external document index and also trained model components. Our RAG guide explains why retrieving information for a request differs from fitting a model.
Privacy matters separately. A request can leave the model’s weights unchanged and still be stored by the service. Retention, access, and possible later training use depend on the product and its policies. Check the service and account settings before submitting private material.
How training, inference, and testing fit together
Google’s machine-learning glossary defines inference as applying a trained model, including generating a language-model response. Here is how the jobs compare:
What is the job?
- Training
- Fit or adapt a model using data.
- Ordinary inference
- Use the fitted model to produce an output.
What changes in a parameter-based model?
- Training
- The parameters selected for training.
- Ordinary inference
- Inputs and working calculations; learned parameters stay fixed.
What comes out?
- Training
- A fitted or updated model, alongside training measurements.
- Ordinary inference
- A prediction, score, generated response, or other result.
A simple example
- Training
- Fit a delivery-time model from past trips.
- Ordinary inference
- Estimate the time for a new trip.
These jobs can recur: training may happen again after deployment, and predictions can be calculated during training and evaluation.
Training, validation, and test sets describe how data is used. For a supervised prediction task like our delivery example, evaluation runs the fixed model on inputs and compares its predictions with known outcomes. The prediction calculation resembles inference in use; its purpose is to measure performance.
Our three fitting records are training data. Their perfect fit is not a test score or evidence that the rule works on new trips. Do not update a model from held-out test answers and then present those same answers as an untouched final test. See Training vs Testing Data for validation, test data, and leakage prevention.
A closer look at fitting
With our multiplier fixed at 2, an offset of 7 minimizes the average squared prediction error on the three training examples. Real training usually involves noisier data and more parameters.
Neural-network training commonly calculates a prediction, measures its error with a loss function, computes gradients that guide changes, and uses an optimizer to update trainable parameters. PyTorch’s training tutorial shows these steps.
Not every learning method trains neural-network weights. A method such as K-nearest neighbors can retain prepared examples and compare a new input with them. The broader distinction is fitting or adapting a model versus using the fitted model. Scikit-learn’s nearest-neighbor documentation explains that approach.
Timing and cost do not tell you whether a model learned
On-demand predictions and model updates
Online inference produces a prediction when requested, even if the model has not changed for weeks. Batch inference groups predictions, such as estimates for tomorrow’s deliveries. Google’s inference guide distinguishes precomputed predictions from on-demand predictions.
Online learning or frequent retraining involves model updates over time. A system can combine prediction and updating, but making a prediction does not prove an update occurred. Google’s training guide covers these update patterns. More time or computation spent generating an answer does not, by itself, imply a weight update either.
Workload, cost, and quality
A neural-network training step generally includes the forward calculation plus work to calculate gradients and update parameters. Inference still needs computation and memory to process inputs and produce outputs.
Training is not always expensive and inference is not always cheap. Total cost depends on model size, request volume, input and output length, hardware, batching, and the surrounding application. A low cost per request can become a substantial ongoing bill at high volume.
Quality needs checks of both the model and the deployed system. Representative evaluation examples test the model; actual inputs, documents, tools, and constraints affect the service people use. Additional training cannot automatically fix a missing document, broken integration, or unsuitable task definition.
Check your understanding
Try these before opening the answers. Name what changed in each case.
- The delivery model still uses an offset of 7. What does it predict for a new 5-kilometer trip? Does producing that estimate train the model?
- The 26-minute result for the 6-kilometer trip is logged. No fitting or model replacement follows. What will the original model predict for 6 km?
- An assistant answers correctly after you paste an updated return policy. What changed, and is that evidence of retraining?
- A pipeline fits a new delivery offset from additional trip records. What changed?
- An app remembers that you prefer bullet points. Is that enough to conclude that the model’s weights changed?
Check your answers
- 17 minutes: 2 × 5 + 7. The input changed; the fitted offset stayed at 7. Producing this estimate is inference.
- 19 minutes: 2 × 6 + 7. The stored information changed, but the original fitted model did not.
- The supplied context changed. A better answer after adding the policy is not, by itself, evidence of retraining.
- The fitted model changed through training. The pipeline used data to fit a new parameter value.
- No. Application memory may have changed. Check how the preference is stored and reused before drawing a conclusion about model weights.
The distinction to remember
Training fits or adapts a model. Inference uses the fitted model. In our example, training changed the saved offset from 5 to 7. Inference used that same 7 to produce 19- and 23-minute estimates. Logging a correction changed the available data, not the original model.
When an AI answer changes, identify what changed in the system before naming the process. If the details are unavailable, describe what you observed: “The answer changed after I added a document.” A claim that the model learned the document into its parameters needs evidence of a training process.
Where to Learn Next
Next, learn how to check a model’s performance after fitting and give validation and test data their proper roles.
For model adaptation, read Transfer Learning Explained. For text generation, read Large Language Models Explained.