Paige Pilot with PII and Skeptical SREs
Corvus Financial has 140 services, 300 engineers, and 6 SREs. Only 20% of services have SLOs. Postmortems are completed for 40% of incidents, and incident review meetings are often skipped. Incident data lives in PagerDuty, logs in Splunk, runbooks in Confluence, and there is no reliable service catalog. The company is deploying PagerDuty's reliability platform and wants to use Paige, PagerDuty's SRE agent, to summarize incidents, draft postmortems, and recommend remediation. Legal requires that customer PII not be sent to any AI feature without review; security wants an audit trail for every AI-generated recommendation. The SRE team is skeptical after a previous AI tool produced confident but wrong root causes. Leadership wants a 60-day pilot with clear guardrails and evidence that Paige improves response without creating new risk.
1. Where should Paige be used first, and what should remain human-owned?
Read the worked answer
Start with lower-risk assistance: incident summarization, timeline consolidation, retrieval of similar past incidents, and drafting status updates or postmortem sections for human editing. These uses save responder time and are easy to verify against the incident record. Keep severity declaration, root-cause conclusions, customer communications, and any remediation action human-owned. Paige can recommend next steps, but an engineer must approve them, especially in a financial environment. Choose two pilot teams and a set of incidents that do not contain customer PII. Use the service catalog and dependency data to improve context; without a reliable catalog, Paige's recommendations will be weaker. The pilot should compare assisted versus unassisted incidents on time to summary, postmortem draft time, and responder-rated usefulness. The reasoning is that summarization has a clear source of truth, while root cause does not. Starting with summarization builds trust and creates evidence before anyone relies on remediation suggestions.
2. Design the data governance and validation model for the pilot.
Read the worked answer
Classify incident data before ingestion. Redact PII from incident notes and exclude any incident type flagged by legal until reviewed. Define retention and access rules, and require an audit log that shows which AI output was generated, when, for which incident, and who viewed or accepted it. Validate outputs by spot-checking summaries against the source timeline and requiring citations or links to the incidents and runbooks used. Track hallucination rate, omission rate, and the percentage of suggestions accepted, edited, or rejected. Give responders a one-click feedback control and review feedback weekly. Do not let Paige send external communications without approval. If the vendor cannot meet the audit and data-handling requirements, narrow the pilot to synthetic or fully redacted incidents. Governance is not a blocker; it is the condition for a 60-day pilot in a regulated company.
3. How would you improve postmortem and SLO adoption alongside Paige?
Read the worked answer
Treat Paige as a drafting aid, not the process owner. Create a standard postmortem template and require an incident review within five business days for major incidents. Use incident workflows to assign an owner and track action items to closure. Paige can draft the timeline and contributing factors, but the incident owner validates them and writes the final learning. For SLOs, start with the small set of customer-facing services that already have clear error budgets, then expand using the service catalog. Link SLO burn to incident priority so the relationship is visible. Measure postmortem completion, time from incident close to draft, review attendance, action-item closure rate, and SLO coverage. Do not measure the number of AI-generated documents, because that rewards volume over learning. The pilot should show whether Paige reduces the blank-page problem while the SRE team improves the review and follow-through that actually changes reliability.