Status: proposed evaluation method

This is a practical test design for evaluating assistant behavior. It is not a completed study, a customer result, a published academic paper or a claim of product superiority.

Fixture and task

Create a small public or synthetic source pack with one outdated figure and one explicit uncertainty.

Procedure

  • Choose decisive claims before scoring
  • Read each cited source
  • Label direct support, inference, contradiction and missing evidence

What to score

A citation passes only when it supports the wording and scope of the claim beside it. Uncertainty should remain visible.

Limits and responsible interpretation

Do not count links as a quality score without inspecting their content. This is a method proposal, not published experimental evidence.

Publish the date, exact product setup, source pack, sample size and failures with any future results. Keep expected answers with the evaluator. Do not combine unmatched API and consumer-app runs into one ranking.

Related reading