Agencies don’t need to invent AI governance from scratch. The National Institute of Standards and Technology’s AI Risk Management Framework (AI RMF 1.0), released in January 2023, organizes AI risk management into four functions: Govern, Map, Measure, and Manage. Here is what each looks like in a human services program.
1. Govern: set the rules before the tool
- Name an accountable owner for each AI capability.
- Write down what the AI may and may not do. Make “no automated eligibility or service decisions” explicit.
- Define who reviews AI outputs, and how quickly.
- Include AI use in staff training and in client-facing privacy notices.
2. Map: know where AI touches people
- Inventory every AI feature: its purpose, inputs, outputs, and the people it affects.
- Flag high-sensitivity contexts: trafficking survivors, domestic violence, substance use records, minors.
- Identify the legal requirements that apply, such as HMIS standards, HIPAA, 42 CFR Part 2, and VAWA and VOCA confidentiality.
3. Measure: test it like it matters
- Check accuracy of AI-drafted notes against caseworker corrections.
- Review risk flags for disparate impact across race, gender, age, language, and disability.
- Track override rates; frequent overrides signal a tool that needs adjustment.
- Measure the outcome you adopted AI for, such as documentation time or time-to-intervention.
4. Manage: act on what you learn
- Set thresholds that trigger review, retraining, or retirement of a feature.
- Keep audit logs of AI outputs and human decisions.
- Have an incident process for AI errors, including client notification when appropriate.
- Revisit the inventory at least annually and whenever a feature changes.
Good AI governance is not a separate bureaucracy. It is the same discipline programs already apply to client records, extended to a new kind of tool.
Agencies that put these basics in place can adopt AI with confidence, and can show funders and oversight bodies exactly how risk is being managed.
Sources: NIST, AI Risk Management Framework