An AI assistant should earn its place through supported answers and reduced total task time—not fluent prose. Earpiece documents football-specific models and conversational scouting functions. Separately, NIST’s Generative AI Profile identifies confabulation as a risk requiring attention. Neither source establishes the error rate of a particular scouting assistant in MLS or NWSL use. (twelve.football)
Our recommended trial separates three purchasing hypotheses:
| Hypothesis | What to compare | Evidence required |
|---|---|---|
| Better access to existing information | Assistant versus the current search/dashboard workflow | Identical source material, answer quality and total completion time |
| Additional modelling | Assistant’s modelled output versus the incumbent model or baseline | Input lineage, model scope, validation and uncertainty |
| Additional intelligence | Assistant’s findings versus everything already available to the club | A supported new fact or interpretation that changes the assessment |
Include deliberately difficult tasks: an ambiguous player name, an unsupported competition, missing minutes, contradictory records and a question whose answer is unavailable. The preferred behaviour may be a clear refusal to infer—not a confident completion.
For each substantive claim, require a source, observation period and explanation of whether it is a recorded fact, model output or judgement. Measure generation plus checking plus correction time. Do not count a faster draft as a saving if verification consumes the difference.
Sources cited on this page
- twelve.footballtwelve.football