Model evaluation metrics tell you how well a machine-learning system performs—but no single score can tell you whether the system is ready to use. A useful evaluation connects the task, the data, the cost of mistakes, the decision threshold, and the conditions the model will face after deployment.
This guide gives you a practical workflow for choosing and interpreting metrics for classification, regression, and generative AI. It also explains why baselines, calibration, uncertainty, subgroup checks, and production monitoring belong in the evaluation process.
A Seven-Step Model Evaluation Workflow
- 1. Define the task
- 2. Split the data
- 3. Set a baseline
- 4. Choose metrics
- 5. Select a threshold
- 6. Stress-test
- 7. Monitor
Evaluation begins before calculating a metric. First decide what the model is predicting and how its output will be used. Then separate development decisions from final testing, establish a meaningful baseline, and select metrics that match the real consequences of mistakes.
Validation Metrics vs Test Metrics
Use validation data or cross-validation during development to compare models, tune hyperparameters, and choose thresholds. Use a final held-out test set after those decisions are fixed to obtain a more independent estimate of performance.
Repeatedly checking the test set and changing the model in response makes the test set part of development. Its score is then less credible as an estimate of performance on unseen data. See Training vs Testing Data for the full data-splitting foundation.
Choose Metrics by Task and Decision
Start with the prediction task, but do not stop there. Two teams using the same model type may need different metrics because false positives, false negatives, large numerical errors, or unreliable generated answers have different consequences.
On a phone: swipe horizontally to view every column.
| Task or need | Useful metric | What it asks | Important limitation |
|---|---|---|---|
| Balanced classification | Accuracy | What share of all predictions was correct? | Can hide failure on a rare class. |
| Costly false positives | Precision | How reliable are positive predictions? | Does not measure missed positives. |
| Costly false negatives | Recall | How many actual positives were found? | Can rise by creating many false alarms. |
| Precision and recall together | F1 | What is their harmonic mean? | Ignores true negatives and treats precision and recall symmetrically. |
| Ranking across thresholds | ROC-AUC or PR-AUC | How well are examples ranked across many cutoffs? | Does not evaluate one deployed threshold; PR-AUC changes with prevalence. |
| Typical numerical error | MAE | How large is the average absolute error? | Does not emphasize unusually large errors. |
| Large numerical errors matter more | RMSE | What is the square-root average of squared errors? | Can be dominated by large residuals. |
| Probability quality | Calibration | Do predicted probabilities match observed frequencies? | A calibrated model may still rank poorly. |
| Generated output | Rubric, human review, tests, outcomes | Is the output correct, useful, safe, and effective? | No single benchmark represents overall quality. |
Classification Metrics
Classification systems predict categories or scores that can be converted into decisions. For binary classification, the confusion matrix provides true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN).
Accuracy
Overall correctness across both classes.
Accuracy = (TP + TN) / (TP + TN + FP + FN)
Precision
Reliability of positive predictions.
Precision = TP / (TP + FP)
Recall
Coverage of actual positive cases.
Recall = TP / (TP + FN)
Specificity
Coverage of actual negative cases.
Specificity = TN / (TN + FP)
F1 score
A harmonic mean of precision and recall.
F1 = 2 × (precision × recall) / (precision + recall)
Balanced accuracy
The average of recall and specificity, useful when both classes matter.
Balanced accuracy = (recall + specificity) / 2
ROC-AUC and PR-AUC
ROC-AUC summarizes ranking discrimination across thresholds using the true-positive and false-positive rates. PR-AUC summarizes the precision–recall relationship across thresholds and is often more informative when positive cases are rare.
Neither tells you how the deployed system performs at its selected operating threshold. PR-AUC also depends on positive-class prevalence, so scores from datasets with different prevalence are not automatically comparable.
Worked Example: One Spam Filter, Several Metrics
Suppose a test set contains 100 emails: 50 spam and 50 legitimate. The model correctly flags 40 spam emails, misses 10 spam emails, incorrectly flags 5 legitimate emails, and correctly leaves 45 legitimate emails alone.
On a phone: swipe horizontally to view every column.
| Result | Count | Meaning |
|---|---|---|
| True positive | 40 | Spam correctly flagged |
| False negative | 10 | Spam missed |
| False positive | 5 | Legitimate email incorrectly flagged |
| True negative | 45 | Legitimate email correctly allowed |
- Accuracy: 85%
- Precision: 88.9%
- Recall: 80%
- F1: approximately 84.2%
If blocking legitimate email is especially harmful, the team may prioritize precision. If missing spam is more costly, it may accept more false positives to improve recall. The same model can look acceptable or unacceptable depending on the decision and its consequences. For a deeper comparison, read Accuracy vs Precision vs Recall.
Thresholds and Calibration
Many classifiers output a score or probability. A decision threshold converts that output into an action. Lowering the threshold often increases recall and false positives; raising it often increases precision while missing more positives. The direction is common, but not a guarantee at every possible cutoff.
Calibration asks whether probability estimates correspond to observed frequencies. If a calibrated model assigns about 0.8 probability to many comparable cases, roughly 80% should produce the event under similar conditions. Ranking quality and calibration are different: a model can rank cases correctly while producing unreliable probabilities.
Regression Metrics
Regression models predict numerical values. Inspect residuals—the differences between predictions and observed values—alongside summary metrics.
Mean absolute error
MAE reports the average absolute error in the target’s units.
MAE = average(|actual − predicted|)
Mean squared error
MSE gives larger errors more influence by squaring them.
MSE = average((actual − predicted)²)
Root mean squared error
RMSE preserves the larger-error penalty while returning to the target’s units.
RMSE = √MSE
R-squared
R² compares squared error with a mean-prediction baseline. It can be negative on evaluated data and is not expressed in the target’s units.
R² = 1 − (model squared error / baseline squared error)
Report more than one metric when the cost of a typical error and the risk of a very large error both matter. Residual plots and error distributions can reveal patterns that one average hides.
Evaluating Generative AI
Generated answers may have several acceptable forms, so evaluation usually combines evidence:
- Task tests: exact match, retrieval measures, code tests, or domain-specific checks.
- Rubric-based model evaluation: scalable, but the evaluator can introduce bias or inconsistency.
- Human review: useful for correctness, relevance, clarity, safety, and preference.
- Reliability testing: hallucinations, prompt injection, privacy, policy compliance, and tool-use failures.
- Application outcomes: whether the AI-enabled workflow actually improves the intended user or business result.
Baselines, Subgroups, and Uncertainty
A model should beat a reasonable alternative: a majority-class rule, a simple linear model, the current production system, or an established human workflow where appropriate. A complex model that barely improves a baseline may not justify its cost and maintenance burden.
Evaluate relevant subgroups when aggregate performance could hide uneven errors. Report uncertainty with confidence intervals, repeated folds, bootstrap intervals, or repeated runs when appropriate. A score calculated from a limited sample is an estimate, not an exact guarantee.
Distribution Shift and Production Monitoring
A model can perform well on historical test data and degrade after deployment. Monitor changes in inputs, class prevalence, user behavior, data pipelines, edge cases, calibration, error costs, and application outcomes. Offline evaluation and production monitoring are complementary.
When monitoring detects deterioration, investigate the cause before automatically retraining. A changed score may come from the model, the data pipeline, the population, the labeling process, or the decision policy.
Common Evaluation Mistakes
- Choosing a metric only because it is familiar.
- Tuning repeatedly on the final test set.
- Reporting accuracy without class-specific results.
- Using AUC without evaluating the deployed threshold.
- Ignoring calibration when probabilities drive decisions.
- Comparing scores produced from different populations without context.
- Reporting a point estimate without uncertainty.
- Assuming offline performance guarantees production value.
Sources and Further Reading
- Google Machine Learning Crash Course: Accuracy, precision, and recall
- scikit-learn User Guide: Metrics and scoring
- scikit-learn User Guide: Probability calibration
- NIST AI Risk Management Framework resources