Audience
Product, engineering, quality, risk and operating owners
Evaluation & Measurement · Updated August 8, 2026
By PlanckCyber
LLM Evaluation Template
Representative cases, acceptance criteria and regression evidence.
How to use this resource
A benchmark should represent the actual workflow, users, data, exceptions and harms — not only general model performance.
Important: General technical template; evaluation results do not establish absolute accuracy, security or compliance.
When to use it
An LLM-enabled feature needs pilot or release evidence.
Objective
Create a reproducible evaluation tied to the workflow.
Evaluation identity
- Feature / workflow
- Version
- Model and configuration
- Evaluation owner
- Acceptance owner
- Evaluation date
Case taxonomy
- Normal cases
- Edge cases
- Known failures
- Adversarial / abuse
- Policy / refusal
- Human escalation
Measures
- Task success
- Factual support / citation
- Policy compliance
- Exception / escalation
- Latency
- Cost
- User acceptance
Failure review
- Material failures
- Root cause patterns
- Control or design change
- Residual risk
- Release recommendation
Release decision
- Pass
- Conditional pass with documented controls
- Fail / do not release
- Further evidence required
Start with the problem
Have a problem AI might solve?
You do not need a specification. Tell us what you are trying to improve.