You tested your model on three examples, it performed well, and you shipped it. Three weeks later, a user submits an ambiguous, poorly worded prompt - the kind real users send every day - and the model hallucinates. You had no way to catch this because you never tested for it. Systematic evaluation exists to prevent exactly this.
Before you move a model to production, you need a systematic way to measure how it performs at scale. This lesson covers four evaluation methods and one purpose-built SAP tool.
- Human-in-the-Loop Evaluation
Experts or end users review model outputs and rate quality directly. Most accurate for nuanced decisions. Time-intensive. When to use it: High-stakes decisions where quality cannot be reduced to a metric- legal document summaries, sensitive customer communications, or any output that requires expert judgment.
- Automated Evaluation
- Runs scoring metrics like BLEU or ROUGE against reference answers across your full dataset. Fast and scalable. Does not catch nuanced quality issues, but gives you a reliable baseline. When to use it: Establishing a performance baseline quickly, or validating changes at scale.
- Adversarial Evaluation
Deliberately feeds the model tricky, ambiguous, or misleading inputs. The goal is to find failure modes before go-live. If it breaks in testing, it will break in production. When to use it: Any model that will handle real user input in production. Real users will send edge cases you did not design for. Find them in testing, not after launch.
- SAP AI Core Custom Evaluation
Runs systematic benchmarking at scale. You provide a labeled test dataset, a prompt template, target models, and scoring metrics. AI Core runs every model on every test case, applies your metrics, and returns aggregate results.
The key difference from manual testing: results are deterministic, versioned, and repeatable. Change the prompt, run the evaluation again, and see exactly how performance changed.
Five building blocks of Custom Evaluation
Test dataset — labeled inputs and expected outputs, uploaded to an object store
Prompt template — saved in the Prompt Registry; used for all models to ensure a fair comparison
Metrics — reference-based (exact match), LLM-as-judge, or custom Python scripts
Evaluation configuration — assembles dataset, template, models, and metrics into a reusable definition
Evaluation execution — a single run that produces per-model and per-instance results
Don't make this mistake
Relying on automated evaluation alone for a model that will handle live user input. Automated metrics tell you how the model performs on clean, labeled examples. They cannot catch the failures that adversarial testing finds. A model with a strong BLEU score can still break on the inputs your real users actually send. Use automated evaluation for speed and scale - but always combine it with adversarial testing before any production deployment.