NIST's AI risk management framework - is this a way forward for AI ethics, and trustworthy AI?

(© gan chaonan - Shutterstock)

In December 2022, the National Institute of Standards and Technology (#NIST) released AI RMF 10, "Artificial Intelligence Risk Management Framework." I reviewed a draft copy a few months ago and Assessing the NIST AI Risk Management Framework.

At the time, it gave organizations a lens for managing AI risks - but mentions of AI ethics take a back seat. Is this a problem or a better approach? 

Every document about AI begins with a definition of AI. In this case, NIST chose to start with a broad but not very descriptive one adapted from OECD Recommendation on AI:2019; ISO/IEC 22989:2022:

The AI RMF refers to an AI system as an engineered or machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varying levels of autonomy.

This definition could apply to an estimated 80% of decision-making legacy systems still in production that are not based on AI concepts or algorithms. In the previous review, I used word counts to illustrate the emphasis on risk without devoting much ink to issues of ethical and responsible principles:

Risk tolerance - While the AI RMF can prioritize risk, it does not prescribe risk tolerance. Risk tolerance refers to the organization's or AI actor's (see Appendix A) readiness to bear the risk to achieve its objectives. Risk tolerance and the risk acceptable to organizations or society are highly contextual, application, and use-case specific. Risk tolerances can be influenced by policies and norms established by AI system owners, organizations, industries, communities, or policymakers. Risk tolerances will change over time as AI systems, policies, and standards evolve. 

Organizations may have varied risk tolerances due to their organizational priorities and resource considerations. Emerging knowledge and methods to better inform harm/cost-benefit tradeoffs will continue to be developed and debated by businesses, governments, academia, and civil society. To the extent that challenges for specifying AI risk tolerances still need to be solved, there may be contexts where a risk management framework still needs to be readily applicable for mitigating adverse AI risks. The NIST Framework is intended to be flexible and augment existing risk practices that align with applicable laws, regulations, and norms:

Organizations should follow existing regulations and guidelines for risk criteria, tolerance, and response established by organizational, domain, discipline, sector, or professional requirements. (“MAP 1.5 | AI RMF Playbook - NIST”) Some sectors or industries may have established definitions of harm or established documentation, reporting, and disclosure requirements.

Risk management may depend on existing guidelines for specific applications and use case settings within industries. Where established guidelines do not exist, organizations should define reasonable risk tolerance. Once tolerance is specified, this AI RMF can manage risks and document risk management processes.

A concise and helpful treatment of AI bias in just two paragraphs. 

The focus of the NIST document is risk, not ethics. Bias is presented as a risk factor, not an emotional, aspirational discourse on how philosophers are essential to getting AI responsible. NIST describes three significant categories of AI bias to be considered and managed: 

Systemic bias can be present in AI datasets, the organizational norms, practices, and processes across the AI lifecycle, and the broader society that uses AI systems. (From “How AI and Automated Systems Use Can Lead to Discrimination in Hiring”) 

Computational and statistical biases can be present in AI datasets and algorithmic processes, and often stem from systematic errors due to non-representative samples. 

Human-cognitive biases relate to how an individual or group perceives AI system information to decide or fill in missing information or how humans think about the purposes and functions of an AI system. Human-cognitive biases are omnipresent in decision-making processes across the AI lifecycle and system use, including AI design, implementation, operation, and maintenance. 

An excellent point is made about a widespread misunderstanding about bias: each category can occur without prejudice, partiality, or discriminatory intent. In other words, bias per se is a risk factor only in context. It can be argued that life insurance premiums are biased against men (they are), but women have a longer life expectancy. 

Bias exists in many forms and can become ingrained in the automated systems that help make decisions about our lives. While bias is not always a negative phenomenon, AI systems can increase the speed and scale of biases and perpetuate and amplify harm to individuals, groups, communities, organizations, and society. (“Artificial Intelligence Risk Management Framework Published By NIST”) Bias is tightly associated with the concepts of transparency and fairness in society.

Part two of the document: Core and Profiles, describes four specific functions to help organizations address the risks of AI systems in practice. In terms of these functions – GOVERN, MAP, MEASURE, and MANAGE – NIST heard numerous reviews that the pure emphasis on risk without acknowledging responsible and trustworthy AI were essential.

Compare the draft version: "The AI RMF Core provides outcomes and actions that enable dialogue, understanding, and activities to manage AI risks." (“What you need to know about NIST's AI Risk Management Framework ...”) 

To version 1.0: "The AI RMF Core provides outcomes and actions that enable dialogue, understanding, and activities to manage AI risks and responsibly develop trustworthy AI systems."

There are some refinements in version 1.0, such as the inclusion of the nature of AI risk, and the management of it through trustworthy AI includes guidance for addressing it. 

For more background on trustworthy AI and NIST, see NIST Releases Final AI Risk Management Framework - via GovWin.  As per GovWin, As per GovWIN, characteristics of trustworthy AI systems include:

Valid and reliable, safe, secure and resilient, accountable and transparent, explainable and interpretable, privacy-enhanced, and fair with harmful bias managed.

Creating trustworthy AI requires balancing these characteristics, based on the AI system's use context. While all aspects are socio-technical system attributes, accountability and transparency also relate to the processes and activities internal to an AI system and its external setting. Neglecting these characteristics can increase the probability and magnitude of adverse consequences. 

My take

This balanced treatise on AI risk management deals with AI's diverse and expanding portfolio. It recognizes that responsibility and transparency may be defined differently across contexts, but the need to have a framework is vital. I admire how authors could give the nod to the ethics issue but not burden the central premise of risk. 

But why has NIST not yet expanded this scope to all that make predictions, classify, recommend, and influence decisions?