Use an evidence ledger that separates new evidence, new interpretation, corroboration and faster delivery. Do not collapse them into one count of “insights”.
For each recruitment question, record the baseline answer before reviewing the challenger’s output.
| Field | What to record |
|---|---|
| Question | The specific uncertainty to resolve |
| Baseline result | Answer, source, remaining uncertainty and staff time |
| Supplier result | Finding, source, date, competition/season and limitations |
| Contribution type | New observation; new supported interpretation; independent corroboration; workflow acceleration; duplication |
| Practical consequence | Advance, reject, seek another match, commission live observation, or leave unresolved |
| Verification | Reviewer, supporting evidence and any disagreement |
Recommended acceptance tests:
- Traceability: can a reviewer reach the underlying match, event, report or documented observation?
- Novelty: could the baseline already answer the question under equivalent task conditions?
- Materiality: does the finding alter the next action or resolve a named uncertainty?
- Independence: is the second opinion genuinely independent, or does it reuse the same source and model?
- Reproducibility: can another evaluator follow the evidence and understand the conclusion?
Count a defensible rejection as a useful result. Do not reward agreement with the scouting lead automatically: a supported challenge may be more valuable.
Avoid double counting. Award additional-evidence value under criterion 1 and reduced task handling under criterion 5. Treat independent corroboration as corroboration, not a second novel observation.
Is a same-shortlist trial sufficient?
It tests assessment, but the recommended trial should also include a separate discovery task. Give every supplier the same role brief, markets, evidence requirements and constraints without supplying candidate names. Then check the returned candidates against the baseline and the brief.
Keep discovery and assessment results separate. A supplier could add useful assessment depth without improving discovery, or identify candidates that it cannot support with sufficient evidence.