Skip to content

TEVV-001: From Measurement to Claim Resolution

Identity

  • Record ID: TEVV-001
  • Status: OPEN
  • Framework: NIST TEVV-Athlon
  • Focus: Measurement, evidence, and claim resolution

Source

  • Publisher: National Institute of Standards and Technology
  • Document: NIST AI 200-2, The TEVV-Athlon Framework for Evaluating AI Systems
  • Version: Initial Public Draft, 2026
  • Source: https://doi.org/10.6028/NIST.AI.200-2.ipd
  • Overview: https://www.nist.gov/artificial-intelligence/ai-research/tevv-athlon-framework-evaluating-ai-systems

Reviewed material:

  • NIST framework overview and abstract.
  • NIST request for public input.
  • Previously developed Structural Assurability crosswalk.

Pending:

  • Direct section-level review of the complete draft.
  • Verification of the exact locations relevant to this record.

Source Structure

The framework describes a method for constructing customized assessments of AI systems.

Its assessment structure includes:

  • Blocks representing measurement concepts of interest.
  • Events and Tools used to test AI systems.
  • Data produced by those assessment activities.

These are the source's concepts, not definitions introduced by Structural Assurability.

Evaluative Claim

Researcher-formulated question:

Does the evidence produced by a specified TEVV-Athlon assessment support the distinctions necessary to evaluate its intended claim?

No particular NIST assessment or claim has yet been selected for detailed evaluation.

Required Distinctions

The investigation will identify two or more claim-relevant alternatives.

It will then determine whether the selected assessment would produce distinguishable observations for those alternatives under specified conditions.

The alternatives must be derived from a concrete assessment question rather than assumed in advance.

Evidence

Source-described evidence mechanism:

  • Assessment data produced through Events and Tools associated with selected Blocks.

To be established for a concrete assessment:

  • The observations available to the evaluator.
  • How those observations relate to the intended claim.
  • Whether relevant differences could remain unobserved.
  • What evidence is required beyond the reported measurements.

Evaluator Conditions

The investigation must specify:

  • Evaluator role.
  • Information and instrumentation access.
  • Required cooperation.
  • Applicable resource constraints.
  • Assumptions about evidence provenance and integrity.

These conditions have not yet been established for a particular TEVV-Athlon assessment.

Assumptions

No universal claim-resolution guarantee is attributed to TEVV-Athlon.

Any model constructed for this record must state its own admissible-world and observation assumptions.

Structural Assurability Mapping

Relevant formal concepts include:

  • Assurance context.
  • Evaluator conditions and resource bounds.
  • Claim-material evidence.
  • Observation models.
  • Observational indistinguishability.
  • Claim-resolution bounds.

The theory establishes that a specified observation model cannot resolve a claim when two admissible worlds disagree on the claim but produce identical observations.

Applying that result to a TEVV-Athlon assessment requires independent justification of the modeled worlds, observations, and claims.

Research Question

For a specified TEVV-Athlon assessment, what additional information, if any, is needed to determine whether its measurements can distinguish the alternatives material to its evaluative claim?

Evidence and Analysis

Completed:

  • Initial conceptual crosswalk.
  • Identification of relevant formal concepts.

Pending:

  • Select one concrete assessment from the draft.
  • Identify its evaluative claim.
  • Record the relevant Events, Tools, and Blocks.
  • Specify evaluator conditions and assumptions.
  • Construct and test potential indistinguishable claim-disagreeing alternatives.
  • Review relevant adjacent material in the draft.

A constructed counterexample would establish a limitation only under its specified modeling assumptions.

Outcome

Status: OPEN

The outcome may be a supported finding, a clarification of applicable assumptions, an illustrative application of the theory, or no additional finding.