Structural Assurability Pilot
This research pilot investigates whether claim-relative Structural Assurability can contribute to the examination of concrete assurance claims and external evaluation frameworks.
Research Question
Can Structural Assurability identify useful distinctions among the observations available to an evaluator, the evidentiary capabilities those observations support, and the assumptions required to resolve a specified assurance claim?
Research Approach
Investigations begin with externally documented assurance questions.
For each question, the study identifies the relevant claim, source material, available evidence, evaluator conditions, and assumptions. Where applicable, these are compared with formal definitions, theorems, and modeling limitations in Structural Assurability.
The analysis distinguishes:
- What an external source actually states.
- How that statement is interpreted for the investigation.
- What the formal theory establishes.
- What any accompanying experiment demonstrates.
- What remains uncertain or requires independent justification.
Executable experiments may be developed when they help test a specific proposed distinction or expose a counterexample.
A finding must be supported by the examined sources and applicable formal or experimental evidence.
Initial Study: NIST TEVV-Athlon
The initial study examines NIST's TEVV-Athlon Framework for Evaluating AI Systems.
Previously developed mapping material provides the starting point. The investigation examines its source provenance and relationship to the current Structural Assurability theory.
Particular questions concern the relationship among evaluation claims, observations, evidentiary capabilities, evaluator conditions, and assumptions about evidence integrity and trust.
A public comment may be developed if the analysis supports a specific issue and a useful proposed clarification.
The investigation remains valuable even if it produces no public comment or proposed framework change.
See the TEVV-Athlon mapping records.
Research Boundaries
The pilot does not independently establish the safety, adequacy, or trustworthiness of an evaluated system.
Formal results belong to the Lean theory. Exploratory experiments establish results only for their specified models and assumptions.
A structural-property difference does not automatically establish a difference in evidentiary capabilities or claim resolution.