The Dashboard That Everyone Trusts and Nobody Checks
You lead a 12-person analytics team at a healthcare insurer. A weekly "provider performance" dashboard has been in production for 18 months, used by the network management team to decide contract renewals. Last week a hospital disputed its readmission rate, which the dashboard showed as 22%. Investigation found the dashboard counts a readmission if the patient returns within 30 days to any facility in the network, while the contract defines readmission as a return to the same facility. The error affects roughly 8% of providers, some favorably and some unfavorably. The dashboard has no documented metric definitions and no named owner. The network team has already sent renewal notices to three providers based on the numbers. Your manager says fixing the definition now would "undermine confidence in the data" and suggests quietly correcting it in the next release.
1. Evaluate your manager's suggestion to correct the definition quietly in the next release. What are the professional and ethical problems, and what would you do instead?
Read the worked answer
Quiet correction is the wrong call, and the reasoning is about accountability, not just optics.
Problems with the manager's approach:
- It conceals a known error from people who acted on it. Three providers received renewal notices derived from a metric now known to be wrong. Correcting silently leaves those decisions standing on bad data and prevents the affected parties from contesting them.
- It treats confidence as something preserved by hiding errors. Confidence in analytics is built by demonstrated correction, not by the appearance of never being wrong. A silent fix that is later discovered - and it will be, because the disputed hospital already knows - does far more damage.
- It leaves the root cause in place. No owner, no documented definitions. Fixing one metric quietly does nothing about the other metrics on the same dashboard that have the same problem.
- It may create contractual exposure. Renewal terms were communicated based on a figure that does not match the contract definition. That is a business risk the organization needs to see, not bury.
What I would do instead:
- Immediately flag the affected providers and the three renewal notices to network management and legal/contracts, with the size and direction of the error for each.
- Publish a correction notice with the dashboard release: what changed, why, which providers are affected, and what the corrected figures are.
- Assign a named owner for the dashboard and document every metric definition, starting with readmission.
- Add a reconciliation check against contract definitions before any metric is promoted to a decision-support dashboard.
The framing for my manager: the choice is not between confidence and transparency. It is between a controlled correction now and an uncontrolled one later.
2. The hospital that disputed the figure is a major contract worth significant revenue. How do you handle the relationship and the communication, and what do you refuse to do?
Read the worked answer
The relationship is a stakeholder-management problem layered on a data-quality problem, and the two must be handled separately.
What I would do:
- Acknowledge specifically, not vaguely. Tell the hospital what the dashboard got wrong (any-facility vs. same-facility readmission), that we have confirmed it, and that we are correcting it. Vague apologies read as evasion.
- Give them the corrected figure and the method. Show the definition, the data window, and how the number was produced. Let them check it. That is what converts a dispute into a shared fact base.
- Separate the correction from the negotiation. The renewal conversation should proceed on corrected data, not be traded against it. Mixing the two makes the data look like a bargaining chip.
- Check whether other providers are affected in the same direction and correct them proactively. Selective correction only for the loudest complainant is itself an integrity failure.
What I refuse to do:
- I will not adjust the metric to produce a number the hospital prefers. If the corrected same-facility figure is still high, that is a finding, not a problem to be smoothed.
- I will not withhold the correction from the other affected providers to limit the reputational hit.
- I will not let the commercial relationship determine the definition. The definition comes from the contract; the contract can be renegotiated, but the analysis reports what the definition says.
Escalation: if I am pressured to soften the corrected figure for a major account, I document the request and escalate to compliance or the data governance body. That is the line.