What is the most common gap auditors find on MP-2.3?

Benchmark accuracy reported with no argument that it measures the property claimed

`MP-2.3` Scientific integrity and TEVV considerations are identified and documented, including those related to…

Q: What evidence do auditors commonly ask for on MP-2.3 Scientific integrity and TEVV considerations are identified and documented, including those related to…?

Documented experimental design for evaluation of the system; Data collection and selection decisions, with availability, representativeness and suitability addressed; Construct validation showing the metric stands for the property claimed; The TEVV considerations identified and how each is handled.

Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection (e.g., availability, representativeness, suitability), system trustworthiness, and construct validation. The evaluation design is documented as a scientific claim: what was measured, on what data, and whether the measure validly stands for the property claimed.

Source: NIST AI RMF 1.0, MP-2.3. The statement above is our statement of the clause, read from the published text and cited so you can check it against the official text, not the instrument verbatim. Open NIST AI RMF 1.0 in the compliance library.

artefacts

held by a system

at each review

hard

to go live

Document repository

where the evidence lives

teal = a system already holds it · olive = produced at each review. To go live: easy when half or more of the artefacts are already held by a system, moderate from a quarter to a half, hard below a quarter.

These are the artefacts auditors commonly ask for on this requirement, our guidance rather than text from the standard. Your assessor decides what is sufficient in your context, and asks for samples across the period when a report covers one.

system holds itEvidence a system already holds

none for this control

periodic reviewEvidence produced at each review

Data collection and selection decisions, with availability, representativeness and suitability addressed · Document repository

governing documentDocuments that govern the control

Documented experimental design for evaluation of the system · Document repository
Construct validation showing the metric stands for the property claimed · Policy repository / GRC workspace
The TEVV considerations identified and how each is handled · Document repository

First move

No system holds this evidence for you: it is people and documents. Reminders and routing to the owner still save time. Put the document under version control with an owner and review date, and log each review as a record with reviewer and date.

Common gaps auditors find

Benchmark accuracy reported with no argument that it measures the property claimed
Data selection undocumented, so representativeness cannot be assessed
Evaluation designed by the same people optimising against it

Do this for your whole sheet

Paste the rows you run your controls from and get this mapping for every control at once, with the periodic-review ones flagged and a first move per row. No account for the first run.

Build my evidence sheet

MP-2.2 Information about the AI system's knowledge limits and how system output may be utilized and overseen by… · MP-3.1 Potential benefits of intended AI system functionality and performance are examined and documented