MS-2.4 The functionality and behavior of the AI system and its components, as identified in the MAP function, are monitored when in production
The functionality and behavior of the AI system and its components – as identified in the MAP function – are monitored when in production. Production monitoring covers the functionality and behaviour that mapping identif
system holds itEvidence a system already holds
- The production monitoring configuration and what it observes · SIEM / log platform
- Alerting and the named recipients of monitoring output · SIEM / log platform
periodic reviewEvidence produced at each review
none for this control
governing documentDocuments that govern the control
- Traceability from monitored signals to the behaviours mapping identified · Document repository
- Drift detection results and the thresholds applied · Document repository
First move
Common gaps auditors find
- Monitoring covers infrastructure availability rather than model behaviour
- Drift thresholds set but nothing acts when they are crossed
- Component behaviour unmonitored where the component is third-party
Do this for your whole sheet
Paste the rows you run your controls from and get this mapping for every control at once, with the periodic-review ones flagged and a first move per row. No account for the first run.
Build my evidence sheetMS-2.3 AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment settings, and measures are documented · MS-2.5 The AI system to be deployed is demonstrated to be valid and reliable, and limitations of the generalizability beyond the conditions under which the technology was developed are documented