MS-2.5 The AI system to be deployed is demonstrated to be valid and reliable, and limitations of the generalizability beyond the conditions under which the technology was developed are documented
The AI system to be deployed is demonstrated to be valid and reliable. Limitations of the generalizability beyond the conditions under which the technology was developed are documented. Validity and reliability are demon
system holds itEvidence a system already holds
none for this control
periodic reviewEvidence produced at each review
- Evidence that validation failure would prevent deployment · Source control / CI pipeline
governing documentDocuments that govern the control
- Validation results demonstrating validity and reliability prior to deployment · Policy repository / GRC workspace
- Documented limitations on generalisability beyond the development conditions · Document repository
- The conditions under which the system was developed and tested · Document repository
First move
Common gaps auditors find
- Validity claimed from training performance rather than independent validation
- Generalisability limits known informally and not documented
- Validation performed after deployment as a formality
Do this for your whole sheet
Paste the rows you run your controls from and get this mapping for every control at once, with the periodic-review ones flagged and a first move per row. No account for the first run.
Build my evidence sheetMS-2.4 The functionality and behavior of the AI system and its components, as identified in the MAP function, are monitored when in production · MS-2.6 AI system is evaluated regularly for safety risks as identified in the MAP function, is demonstrated to be safe, its residual negative risk does not exceed the risk tolerance, and it can fail safely, particularly if made to operate beyond its knowledge limits