NIST AI RMF 1.0: the evidence behind every control
72 controls. For each, the artefacts auditors ask for, which ones a system already holds, and the first move to stop evidencing it by periodic review.
GOVERN - NIST AI RMF 1.0
GV-1.1Legal and regulatory requirements involving AI are understood, managed, and documented hardGV-1.2The characteristics of trustworthy AI are integrated into organizational policies, processes, and procedures hardGV-1.3Processes and procedures are in place to determine the needed level of risk management activities based on the organization's risk tolerance hardGV-1.4The risk management process and its outcomes are established through transparent policies, procedures, and other controls based on organizational risk priorities hardGV-1.5Ongoing monitoring and periodic review of the risk management process and its outcomes are planned, organizational roles and responsibilities are clearly defined, including determining the frequency of periodic review hardGV-1.6Mechanisms are in place to inventory AI systems and are resourced according to organizational risk priorities hardGV-1.7Processes and procedures are in place for decommissioning and phasing out of AI systems safely and in a manner that does not increase risks or decrease the organization's trustworthiness hardGV-2.1Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization hardGV-2.2The organization's personnel and partners receive AI risk management training to enable them to perform their duties and responsibilities consistent with related policies, procedures, and agreements hardGV-2.3Executive leadership of the organization takes responsibility for decisions about risks associated with AI system development and deployment hardGV-3.1Decision-making related to mapping, measuring, and managing AI risks throughout the lifecycle is informed by a diverse team hardGV-3.2Policies and procedures are in place to define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems moderateGV-4.1Organizational policies and practices are in place to foster a critical thinking and safety-first mindset in the design, development, deployment, and uses of AI systems to minimize negative impacts hardGV-4.2Organizational teams document the risks and potential impacts of the AI technology they design, develop, deploy, evaluate and use, and communicate about the impacts more broadly hardGV-4.3Organizational practices are in place to enable AI testing, identification of incidents, and information sharing hardGV-5.1Organizational policies and practices are in place to collect, consider, prioritize, and integrate feedback from those external to the team that developed or deployed the AI system regarding the potential individual and societal impacts related to AI risks hardGV-5.2Mechanisms are established to enable AI actors to regularly incorporate adjudicated feedback from relevant AI actors into system design and implementation moderateGV-6.1Policies and procedures are in place that address AI risks associated with third-party entities, including risks of infringement of a third party's intellectual property or other rights hardGV-6.2Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk hardMANAGE - NIST AI RMF 1.0
MN-1.1A determination is made as to whether the AI system achieves its intended purpose and stated objectives and whether its development or deployment should proceed hardMN-1.2Treatment of documented AI risks is prioritized based on impact, likelihood, or available resources or methods hardMN-1.3Responses to the AI risks deemed high priority as identified by the MAP function are developed, planned, and documented, and risk response options can include mitigating, transferring, avoiding, or accepting hardMN-1.4Negative residual risks, defined as the sum of all unmitigated risks, to both downstream acquirers of AI systems and end users are documented hardMN-2.1Resources required to manage AI risks are taken into account, along with viable non-AI alternative systems, approaches, or methods, to reduce the magnitude or likelihood of potential impacts hardMN-2.2Mechanisms are in place and applied to sustain the value of deployed AI systems moderateMN-2.3Procedures are followed to respond to and recover from a previously unknown risk when it is identified hardMN-2.4Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use hardMN-3.1AI risks and benefits from third-party resources are regularly monitored, and risk controls are applied and documented hardMN-3.2Pre-trained models which are used for development are monitored as part of AI system regular monitoring and maintenance moderateMN-4.1Post-deployment AI system monitoring plans are implemented, including mechanisms for capturing and evaluating input from users and other relevant AI actors, appeal and override, decommissioning, incident response, recovery, and change management hardMN-4.2Measurable activities for continual improvements are integrated into AI system updates and include regular engagement with interested parties, including relevant AI actors hardMN-4.3Incidents and errors are communicated to relevant AI actors including affected communities, and processes for tracking, responding to, and recovering from incidents and errors are followed and documented hardMAP - NIST AI RMF 1.0
MP-1.1Intended purpose, potentially beneficial uses, context-specific laws, norms and expectations, and prospective settings in which the AI system will be deployed are understood and documented hardMP-1.2Inter-disciplinary AI actors, competencies, skills and capacities for establishing context reflect demographic diversity and broad domain and user experience expertise, and their participation is documented hardMP-1.3The organization's mission and relevant goals for the AI technology are understood and documented hardMP-1.4The business value or context of business use has been clearly defined or, in the case of assessing existing AI systems, re-evaluated hardMP-1.5Organizational risk tolerances are determined and documented hardMP-1.6System requirements are elicited from and understood by relevant AI actors, and design decisions take socio-technical implications into account to address AI risks moderateMP-2.1The specific task, and methods used to implement the task, that the AI system will support is defined hardMP-2.2Information about the AI system's knowledge limits and how system output may be utilized and overseen by humans is documented easyMP-2.3Scientific integrity and TEVV considerations are identified and documented, including those related to experimental design, data collection and selection, system trustworthiness, and construct validation hardMP-3.1Potential benefits of intended AI system functionality and performance are examined and documented hardMP-3.2Potential costs, including non-monetary costs, which result from expected or realized AI errors or system functionality and trustworthiness are examined and documented, as connected to organizational risk tolerance hardMP-3.3Targeted application scope is specified and documented based on the system's capability, established context, and AI system categorization hardMP-3.4Processes for operator and practitioner proficiency with AI system performance and trustworthiness, and relevant technical standards and certifications, are defined, assessed and documented hardMP-3.5Processes for human oversight are defined, assessed, and documented in accordance with organizational policies from the GOVERN function hardMP-4.1Approaches for mapping AI technology and legal risks of its components, including the use of third-party data or software, are in place, followed, and documented, as are risks of infringement of a third party's intellectual property or other rights hardMP-4.2Internal risk controls for components of the AI system including third-party AI technologies are identified and documented hardMP-5.1Likelihood and magnitude of each identified impact are identified and documented, based on expected use, past uses of AI systems in similar contexts, public incident reports, feedback from those external to the team, or other data hardMP-5.2Practices and personnel for supporting regular engagement with relevant AI actors and integrating feedback about positive, negative, and unanticipated impacts are in place and documented hardMEASURE - NIST AI RMF 1.0
MS-1.1Approaches and metrics for measurement of AI risks enumerated during the MAP function are selected for implementation starting with the most significant AI risks, and the risks or trustworthiness characteristics that will not or cannot be measured are properly documented moderateMS-1.2Appropriateness of AI metrics and effectiveness of existing controls is regularly assessed and updated, including reports of errors and impacts on affected communities moderateMS-1.3Internal experts who did not serve as front-line developers for the system and independent assessors are involved in regular assessments and updates, and domain experts, users, AI actors external to the team, and affected communities are consulted in support of assessments as necessary per organizational risk tolerance hardMS-2.1Test sets, metrics, and details about the tools used during test, evaluation, validation, and verification are documented moderateMS-2.2Evaluations involving human subjects meet applicable requirements including human subject protection and are representative of the relevant population hardMS-2.3AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for conditions similar to deployment settings, and measures are documented hardMS-2.4The functionality and behavior of the AI system and its components, as identified in the MAP function, are monitored when in production easyMS-2.5The AI system to be deployed is demonstrated to be valid and reliable, and limitations of the generalizability beyond the conditions under which the technology was developed are documented hardMS-2.6AI system is evaluated regularly for safety risks as identified in the MAP function, is demonstrated to be safe, its residual negative risk does not exceed the risk tolerance, and it can fail safely, particularly if made to operate beyond its knowledge limits moderateMS-2.7AI system security and resilience as identified in the MAP function are evaluated and documented hardMS-2.8Risks associated with transparency and accountability as identified in the MAP function are examined and documented hardMS-2.9The AI model is explained, validated, and documented, and AI system output is interpreted within its context as identified in the MAP function and to inform responsible use and governance moderateMS-2.10Privacy risk of the AI system as identified in the MAP function is examined and documented hardMS-2.11Fairness and bias as identified in the MAP function is evaluated and results are documented hardMS-2.12Environmental impact and sustainability of AI model training and management activities as identified in the MAP function are assessed and documented moderateMS-2.13Effectiveness of the employed TEVV metrics and processes in the MEASURE function are evaluated and documented easyMS-3.1Approaches, personnel, and documentation are in place to regularly identify and track existing, unanticipated, and emergent AI risks based on factors such as intended and actual performance in deployed contexts hardMS-3.2Risk tracking approaches are considered for settings where AI risks are difficult to assess using currently available measurement techniques or where metrics are not yet available hardMS-3.3Feedback processes for end users and impacted communities to report problems and appeal system outcomes are established and integrated into AI system evaluation metrics moderateMS-4.1Measurement approaches for identifying AI risks are connected to deployment contexts and informed through consultation with domain experts and other end users, and approaches are documented hardMS-4.2Measurement results regarding AI system trustworthiness in deployment contexts and across the AI lifecycle are informed by input from domain experts and other relevant AI actors to validate whether the system is performing consistently as intended, and results are documented hardMS-4.3Measurable performance improvements or declines based on consultations with relevant AI actors including affected communities, and field data about context-relevant risks and trustworthiness characteristics, are identified and documented hard