Certification Backlog in a Self-Service Marketplace
Summit Mutual, a property and casualty insurer, launched a self-service data marketplace with 900 datasets. Only 120 are certified; the rest carry no quality label or a 'use with caution' tag. Sixty percent of dashboards and models are built on uncertified data. Last month an actuary built a pricing model on an uncertified claims dataset that was missing three months of paid claims, producing a 6% rate indication error caught before filing. The stewardship team says certification takes nine months per backlog; business leaders say waiting for certification kills speed. The chief data officer must respond to the pricing incident and set a sustainable certification model. The marketplace product owner wants to remove all uncertified datasets, which would eliminate 87% of inventory.
1. Evaluate the product owner's proposal to remove uncertified datasets. What would you recommend instead, and why?
Read the worked answer
Reject wholesale removal. Removing 87% of inventory would push users to shadow spreadsheets, which is worse for governance-the data would still be used, just invisible. Instead, tier the marketplace: certified (fit for regulated and financial decisions), provisional (usable with documented limitations and a named steward), and restricted (visible but blocked from model or report publication without review). Apply certification by decision criticality, not dataset count. The actuary's error occurred because an uncertified dataset could feed a pricing model without a gate. Fix the gate, not the inventory. Set a certification SLA for datasets used in pricing, reserving, and regulatory reporting, and publish 'known limitations' for provisional sets. The product owner's proposal solves a labeling problem by destroying access; tiering preserves speed while making risk visible.
2. Define the steward's role in a self-service marketplace where business users can publish datasets. What should stewards own and what should they not own?
Read the worked answer
Stewards should own definition, quality rules, certification criteria, and fitness-for-use labels for domain data; they should not own every dataset published by business users, nor act as a central bottleneck. In a self-service marketplace, stewards certify domain-critical data, review datasets that feed regulated or financial decisions, and provide consultative review for provisional sets. Business publishers own documentation and remediation of their own datasets. Stewards should not be required to clean user-created data or approve every dashboard. Define thresholds: any dataset used in pricing, reserving, capital, or regulatory reporting requires steward certification; exploratory datasets can publish as provisional with automated quality checks. This keeps stewardship focused on risk and definitions, not on being a help desk.
3. The pricing incident needs a response. Outline the governance actions you would take in the next 30 days, distinguishing containment from root-cause remediation.
Read the worked answer
Next 30 days: Containment-freeze the pricing model, notify the pricing committee, re-run the rate indication on certified data, document the variance, and add a publication gate so uncertified claims data cannot feed a filed model. Root cause-determine why three months of paid claims were missing: pipeline failure, definition mismatch, or lag. Assign the data steward and pipeline owner, and add a completeness control with an alert. Governance-require a fitness-for-use attestation for any model input, log the exception, and brief the model risk committee. Communication-tell the business what changed without blaming the actuary; the failure was a missing gate, not individual error. Longer term, add certification SLAs for pricing-critical datasets and a steward review before model submission. Distinguish containment, which stops the bad output, from remediation, which fixes the data and the gate.