Interview objective: Evaluate the report as a collection of checkable claims, not as fluent prose.
Put to: Sarah Rudd, Arielle Dror and the supplier’s AI product lead.
Why it matters to buyers: NIST’s Generative AI Profile identifies confidently presented false content and fabricated supporting logic or citations as risks. Its guidance is cross-sectoral, not a football-specific accuracy certification. (nvlpubs.nist.gov)
A strong answer should cover:
- Which statements come from retrieved records, calculated metrics, scout observations or model inference.
- Source identifiers, match timestamps, dates and model versions sufficient for verification.
- Treatment of conflicting sources and deliberately unavailable information.
- Whether a citation actually supports the claim rather than merely mentioning the player.
- Human review responsibilities and correction propagation.
Sharp follow-up: Ask the system a plausible question that the licensed evidence cannot answer. Does it abstain, identify the gap or invent a conclusion?
Request this artefact: An AI Scouting Report Evidence Check with these proposed labels:
| Label | Review rule |
|---|---|
| Supported | The underlying evidence substantiates the claim |
| Inference | Reasoning is identified, bounded and distinguishable from observation |
| Unsupported | Evidence is absent or does not substantiate the statement |
| Contradicted | Available evidence conflicts with the statement |
| Unverifiable within access | The reviewer cannot inspect the claimed support |
Report factual support separately from football judgement. A verified event count does not, by itself, validate a broad claim about a player’s mentality or future performance.
Proposed expert interview; not yet conducted. No expert quotation is attributed.
Sources cited on this page
- nvlpubs.nist.govnvlpubs.nist.gov