MP-2.3 Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection, system trustworthiness, and construct validation
Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection (e.g., availability, representativeness, suitability), system trus
4
artefacts
0
held by a system
0
at each review
hard
to go live
Document repository
where the evidence lives
teal = a system already holds it · olive = produced at each review
system holds itEvidence a system already holds
none for this control
periodic reviewEvidence produced at each review
none for this control
governing documentDocuments that govern the control
- Documented experimental design for evaluation of the system · Document repository
- Data collection and selection decisions, with availability, representativeness and suitability addressed · Document repository
- Construct validation showing the metric stands for the property claimed · Policy repository / GRC workspace
- The TEVV considerations identified and how each is handled · Document repository
First move
This control is evidenced by people and documents, not systems. Put the document under version control with an owner and review date, and log each review as a record with reviewer and date. Do not try to automate it.
Common gaps auditors find
- Benchmark accuracy reported with no argument that it measures the property claimed
- Data selection undocumented, so representativeness cannot be assessed
- Evaluation designed by the same people optimising against it
Do this for your whole sheet
Paste the rows you run your controls from and get this mapping for every control at once, with the periodic-review ones flagged and a first move per row. No account for the first run.
Build my evidence sheetMP-2.2 Information about the AI system's knowledge limits and how system output may be utilized and overseen by humans is documented · MP-3.1 Potential benefits of intended AI system functionality and performance are examined and documented