IBM watsonx Governance Advisor cheat sheet

Positioning: watsonx.governance is an AI governance console inside the IBM watsonx platform; it governs models and generative AI assets across their lifecycle, not the models' training itself.

Core value pillars: (1) risk and compliance visibility, (2) lifecycle traceability and audit evidence, (3) automated monitoring of fairness, drift, and quality, (4) policy enforcement and approval workflows, (5) a single inventory spanning classical ML models and generative AI use cases.

Buyer personas: Chief AI Officer / CDO (enterprise inventory and value), Chief Risk Officer and compliance (regulatory evidence), model risk management (validation and documentation), MLOps and data science (monitoring and deployment gates).

Differentiator versus an MLOps tool: MLOps optimizes model performance and delivery speed; governance adds risk classification, approval gates, policy mapping, and defensible documentation artifacts.

Differentiator versus a pure GRC platform: GRC tracks controls and obligations manually; watsonx.governance attaches those controls to live model and AI-asset metadata, evaluation results, and lifecycle events.

Scope covers generative AI use cases (prompt assets, foundation models, RAG applications) alongside traditional predictive models, so one inventory serves both regulatory and engineering audiences.

Value narrative structure: regulatory driver (obligation) → risk exposure (what fails without governance) → capability (which watsonx feature addresses it) → measurable outcome (audit preparation time, approval cycle time, incident count).

Positioning caution: do not claim official certification or regulator endorsement; describe alignment with frameworks (NIST AI RMF, ISO/IEC 42001, EU AI Act) as mapping support, not compliance guarantee.

AI governance = the policies, roles, processes, and controls that direct how AI systems are built, approved, operated, monitored, and retired, with accountability assigned to named owners.

Three governance layers: policy (what the organization requires), process (how approval and review happen), and evidence (artifacts proving the process ran, such as evaluation reports, approvals, and monitoring logs).

Model lifecycle stages typically governed: intake/registration → risk classification → development and evaluation → independent review/validation → deployment approval → production monitoring → change/revalidation → retirement.

Key fairness metrics: demographic parity difference (selection-rate gap), equal opportunity difference (true-positive-rate gap), and disparate impact ratio (favored/unfavored rate; the 0.8 four-fifths threshold is a common screening heuristic, not a legal rule).

Drift types: data drift (input distribution shift), concept drift (input-to-output relationship shift), and prediction drift (output distribution shift); each maps to a different monitoring detector.

Explainability concepts: global versus local explanations, feature importance, SHAP-style attributions, and counterfactuals; governance records which explanation method was used and for which stakeholder.

Human oversight models: human-in-the-loop (approval before action), human-on-the-loop (monitoring with intervention), and human-out-of-the-loop (logging only); risk tier determines which is required.

Accountability artifacts: model card / AI factsheet, intended-use statement, training data provenance, evaluation results, approver sign-off, and change log; these are the primary audit deliverables.

Generative AI-specific concepts: prompt template versioning, grounding and retrieval sources, hallucination and toxicity evaluation, red-teaming, and output filtering as governed controls.

EU AI Act risk tiers: prohibited practices, high-risk systems (Annex III areas such as employment, credit, education, essential services), limited-risk transparency obligations (e.g., chatbot disclosure, synthetic content labeling), and minimal-risk.

EU AI Act obligations for high-risk systems include risk management, data governance, technical documentation, logging, transparency, human oversight, accuracy/robustness/cybersecurity, and post-market monitoring.

General-purpose AI (GPAI) model obligations include technical documentation, information to downstream providers, and copyright policy; systemic-risk models add evaluation, adversarial testing, and incident reporting.

NIST AI Risk Management Framework functions: Govern, Map, Measure, Manage; it is voluntary and process-oriented, so it maps well to policy and control design rather than to specific technical tests.

ISO/IEC 42001 defines an AI management system (AIMS) with Plan-Do-Check-Act structure, organizational context, leadership, risk treatment, and continual improvement; ISO/IEC 23894 gives AI risk management guidance.

Sectoral drivers: financial services model risk management expectations (model inventory, validation, ongoing monitoring, documentation), and fair-lending/disparate-treatment concerns for credit and marketing models.

Other regulatory threads: GDPR automated-decision and data-subject rights, US state and local AI laws, and emerging disclosure requirements for AI-generated content; track jurisdiction-specific applicability.

Risk scoring inputs commonly used: use-case criticality, affected population and vulnerability, degree of autonomy, data sensitivity, model type (foundation vs. classical), reversibility of harm, and regulatory classification.

Regulatory mapping practice: link each obligation to a control, then to an evidence artifact and an owner; unmapped obligations and controls without evidence are the two gaps auditors find first.

Capability map: AI use-case and model inventory, risk and compliance assessment, factsheet/model card generation, evaluation and fairness testing, production monitoring, approval workflows, and alerting/dashboards.

Architecture pattern: governance console connects to model-serving and development environments (watsonx.ai, watsonx.ai Runtime, external ML platforms, and generative AI providers) through integrations and APIs to pull metadata and metrics.

Inventory is the system of record: each governed asset carries owner, risk tier, lifecycle stage, linked policies, evaluation results, and approval history; everything else in the console references these records.

Evaluation capability includes quality metrics, fairness metrics across protected groups, drift detectors, and generative AI evaluations (faithfulness/grounding, answer relevance, toxicity, PII leakage).

Monitoring runs on scheduled evaluations against production payloads; thresholds trigger alerts and can feed approval or remediation workflows rather than acting only as dashboards.

Governance workflows route assessments and approvals to reviewers with role-based access; decisions and comments are retained as audit evidence.

Policy and regulatory mapping lets teams attach framework controls (NIST AI RMF, ISO/IEC 42001, internal policy) to use cases and track compliance status per control.

Integration surface: Python SDK and REST APIs for registering models, logging metrics and payloads, and running evaluations; supports CI/CD gates that block promotion when thresholds fail.

Deployment options and residency matter in architecture discussions: confirm where governance data and monitored payloads are stored, and whether the client needs SaaS, dedicated, or self-managed patterns.

Discovery goal: produce a quantified, prioritized AI use-case inventory plus a governance gap assessment, not a product demo recap.

Discovery inputs to collect: AI use-case list by business unit, model types and hosting locations, current approval process, existing policies and control frameworks, regulatory obligations in scope, and known incidents.

Stakeholder map: executive sponsor, AI governance lead, model risk/compliance, data science and MLOps leads, security/privacy, legal, and internal audit; each has a distinct success criterion.

Key discovery questions: Who approves a model today and how long does it take? Where is the inventory kept (spreadsheet, CMDB, none)? Which regulations apply by jurisdiction? What evidence exists for the last five models?

Current-state pain quantification: number of models and generative AI use cases, hours spent assembling audit evidence, approval cycle time, percentage of models monitored in production, and shadow AI count.

Prioritization criteria for first wave: regulatory exposure, use-case criticality, data readiness (are payloads and ground truth available?), stakeholder sponsorship, and integration feasibility.

Discovery outputs: use-case inventory with risk tiers, gap heat map against a chosen framework, prioritized roadmap, and a measurable value baseline for each proposed phase.

Discovery anti-patterns: scoping only classical models and missing generative AI, treating governance as a documentation exercise, and skipping internal audit, which later rejects the evidence model.

Phased rollout pattern: Phase 1 inventory and risk classification for a pilot business unit; Phase 2 evaluation and approval workflow; Phase 3 production monitoring and alerting; Phase 4 enterprise scale and framework mapping.

Pilot selection rule: choose a use case with real regulatory or reputational exposure, an engaged owner, and available evaluation data, so the pilot produces credible evidence rather than a synthetic demo.

Operating model decisions: who owns the inventory, who performs independent validation, who approves high-risk use cases, and how exceptions are granted and time-boxed.

Control design: set concrete thresholds per risk tier (e.g., fairness gap limits, drift thresholds, minimum evaluation frequency), define alert routing, and document remediation and escalation paths.

Integration plan: connect model-serving environments and CI/CD so registration, evaluation, and approval happen automatically; manual-only governance does not scale past the pilot.

Adoption levers: role-based training for data scientists and reviewers, governance embedded in existing stage gates rather than a parallel process, executive dashboards, and quick wins that reduce approval time.

Success metrics: percentage of AI assets inventoried, percentage with completed assessments, approval cycle time, monitoring coverage, mean time to remediate alerts, and audit findings count.

Change management risks: perceived slowdown of data science, duplicate inventory in spreadsheets, unclear ownership between risk and engineering, and policy set so strict that teams route around it.

Steady-state handover: documented runbooks, named owners per control, scheduled framework reviews, and a quarterly governance council that reviews risk posture and policy updates.