Abstract
The Usage Governance Advisor (UGA) is a human-in-the-loop system designed to automate the complex, often manual process of AI governance. By translating a user’s natural language intent into a structured risk profile, the system recommends the safest models, triggers automated evaluations, and generates audit-ready documentation to bridge the gap between AI development and regulatory compliance.
Introduction
Deploying Large Language Models (LLMs) in production is currently a "governance nightmare." Organizations face a fragmented landscape of stakeholders—business owners, risk officers, and ethics boards—who must manually curate information to assess risks. A critical challenge is that risk is context-dependent: a model used for internal code generation carries different risks than the same model used for a public-facing medical chatbot. Traditional governance relies on static, "one-size-fits-all" questionnaires that fail to capture these nuances. Furthermore, gathering data from heterogeneous sources (model cards, technical reports, and benchmarks) is incredibly time-consuming, often stalling AI projects in the prototype phase.
Methodology
The UGA system introduces a systemic, automated pipeline divided into four main stages, as illustrated in Figure 1.
- AI Systems Knowledge Graph (KG): The backbone of the system is an ontology-based KG (using the AIRO framework). It ingests metadata from model cards and technical reports using generative AI to perform entity recognition. It maintains an "evidence graph" that links every fact back to its source to ensure transparency.
- Auto-assist Questionnaire: To lower the barrier for users, the system uses LLMs with few-shot and Chain-of-Thought (CoT) prompting to help users fill out compliance forms. It suggests answers based on the user's stated intent, turning vague descriptions into semi-structured governance data.
- Risk Prioritization (LLM-as-a-Judge): The system maps the intended use case to specific risks. For instance, if a user specifies a "classification" task, the system automatically de-prioritizes risks related to "toxic language generation." It uses an LLM-as-a-judge approach to determine if specific use-case details amplify or mitigate known risks.
- Model Recommendation & Automated Evaluation: Based on the risk profile and the organization’s policy (risk tolerance), the system queries the KG to rank suitable models. It then automatically executes relevant benchmarks (e.g., testing for hallucinations or bias) and persists the results for auditing.
The system integrates several high-profile external sources and benchmarks, including the IBM AI Risk Atlas, OWASP Top 10 for LLMs, NIST AI Risk Management Framework, and evaluation benchmarks like TruthfulQA and SocialStigmaQA.
Key Insights
- Intent-First Governance: The core innovation is treating the intended use as a primary factor in the deployment pipeline. Governance is not just about the model; it is about the model-task pairing.
- The "Challenger" Model: Even if a developer has a specific model in mind, the system provides a "recommended" model from the KG to serve as a benchmark, forcing a justification if the developer chooses a riskier option.
- Automated Evidence Trails: By maintaining an evidence graph, the system solves the "black box" problem of automated governance, allowing risk officers to see exactly why a model was flagged or recommended.
Results and Discussion
As a functional end-to-end pipeline, the UGA demonstrates significant improvements in governance efficiency and auditability. It successfully unifies structured benchmark results with natural language model cards into a queryable Knowledge Graph. The system automates the mapping of use cases to risks—a process that previously required hours of cross-functional meetings. Finally, the tool produces a persistent record of the entire decision-making process, satisfying regulatory requirements for transparency and accountability.
Limitations and Future Work
Currently, the tool excels at identifying and evaluating risks but does not yet prescribe active mitigation strategies (e.g., automated guardrails). Future work will address the entire model lifecycle, including monitoring for "drift" or new risks that emerge post-deployment. Additionally, while the system automates much of the "grunt work," human-in-the-loop approvals remain a necessary but potential bottleneck in high-velocity environments.
Summary updated