Evaluation
The systematic measurement of whether a system produces acceptable outputs against defined criteria, on a held-out or adversarial set, before and after deployment.
- StatusIndustry term
- AdoptionWidely used
- LayerIntelligence
Why it matters
Evaluation is what makes a governance claim falsifiable. Without it, 'the agent performs well' is an assertion. The hard part for agents is that the unit under test is a trajectory rather than an output: an agent can produce individually reasonable steps and an unreasonable outcome, and single-turn evaluation will not see it.
Where it comes from
Machine learning practice; extended to LLM and agent systems.
No published instrument defines this term yet; the note above is its origin.
What it is not
These are routinely confused with Evaluation. The distinctions have consequences for how a system is governed.
- Validation
Validation in model risk terms is an independent assessment of a model's fitness for purpose. Evaluation is measurement, and is one input to validation.
How it connects
MARQUE questions it addresses
- Standards & interoperabilityWhat does conformance actually mean here?
Related framework objectives
Connected through the MARQUE (a reference framework organized around eight questions) crosswalk, an editorial mapping. These objectives from the Financial Services AI Risk Management Framework are assigned to the same questions that this term addresses. The ones sharing most of this term's vocabulary are shown first, as a reading aid. The assignment is to the question, not to the term: the source does not map this term to any objective directly.
Question: Standards & interoperability · 62 control objectives assigned
- GV-1.3.4AI Use Risk Acceptance and Rejection FrameworkGovern
- GV-1.4.2AI Risk Documentation StandardsGovern
- MG-4.2.3Feedback Integration and Continuous ImprovementManage
Cite and reuse
Cite the identifier QIS-TERM-00042, not the URL: identifiers are stable, URLs may change. This entry is free to read, quote and index under the dual license. Corrections sent to editor@qisfund.com are published.
Technical details: identifier, graph relationships and record
| Predicate | Target identifier | Target | Predicate meaning |
|---|---|---|---|
relatedTo | concept:reproducibility | Reproducibility | An association too weak or too general for a stronger verb. |
addresses | marque:standards | Standards & interoperability | The subject speaks to the object as a question or concern. |
- Canonical identifier
QIS-TERM-00042- Tier
- Canonical industry term
- Domain · layer
- Stack categories · Intelligence
- Semantic aliases
- None recorded
- First published
- 2026-08-02
- Last reviewed
- 2026-08-02 · 180-day cycle