Prompt Regression
How to detect and fix prompt regressions with eval baselines, monitoring, and rollback triggers.
Published:
Admin User
Updated:
published
Prompt Regression
Prompt regressions happen when a prompt change reduces quality, increases cost, or breaks safety expectations.
Enterprise handling: detect via eval, confirm via monitoring, rollback fast, and document evidence.
See also
Evaluation Harness Monitoring (Quality, Drift) Rollback Strategy AI Rollback RunbookFAQ
What is prompt regression?
A prompt change causes a drop in quality, safety, or cost performance compared to baseline.
How do we detect it?
Evaluation harness + monitoring of outcome metrics and drift signals.
What’s the first response?
Freeze changes, reproduce on test set, rollback if confirmed.
How do we prevent repeats?
Versioning, gates, and stronger rubrics/test sets.
What’s the first improvement?
Run a small evaluation set for every prompt change.