Alert Noise at MediTrust
MediTrust Health runs IT for 14 hospitals and 2,000 endpoints. Its monitoring stack generates about 1,500 alerts a day; 92% are closed as noise, yet four major outages last quarter averaged three hours each. The configuration management database is 61% accurate and is updated mainly after incidents. The service desk uses five tools that do not share data: ITSM, monitoring, endpoint management, knowledge and identity. The knowledge base has 600 articles; 40% are never opened and 25% are more than two years old. Clinicians report that the same infusion-pump connectivity fault is diagnosed from scratch each time. A vendor has proposed an AIOps platform with anomaly detection and auto-remediation. The CIO has limited capital and wants a sequenced plan that improves detection and restoration before adding AI.
1. Prioritise the information and technology improvements. What would you do first, second and third, and why? Link each step to ITIL practices and value chain activities.
Read the worked answer
Sequence by risk reduction and dependency, not vendor excitement. First, fix the data and signal layer: reduce alert noise, define meaningful events, and establish accurate service and CI mapping for critical clinical services. This uses event management, service configuration management and measurement, and it supports deliver and support. Without this, AIOps would learn from unreliable data.
Second, integrate the five tools enough to create a closed loop: monitoring alert to event to incident to knowledge article to problem record. This involves the service desk, incident management and knowledge management, and improves restoration before adding intelligence. Third, automate low-risk, well-understood remediation, such as restarting a service or clearing a queue, with approval and audit. Fourth, pilot AIOps on a bounded service with human-in-the-loop review.
The 92% noise rate and 61% CMDB accuracy are preconditions. An AIOps purchase without them would increase false confidence. Clinical safety also demands explainability and manual override. Give each step an exit criterion before releasing the next tranche of spend.
2. Design the target event-to-incident workflow, including monitoring, event management, service desk, incident management and automation. Where would AIOps fit, and what human oversight is required?
Read the worked answer
Target workflow: instrumentation and monitoring produce events. Event management filters, deduplicates, correlates and enriches them with service and CI context. Significant events become incidents automatically with priority based on clinical impact. The service desk sees one queue with linked knowledge. Automation runs only for pre-approved low-risk actions. Unresolved incidents trigger problem management.
AIOps can contribute anomaly detection, correlation, noise reduction and suggested remediation. Keep human oversight for clinical services: a clinician or service owner confirms impact, and auto-remediation must have a kill switch, audit trail and rollback. Restrict the vendor's auto-remediation claim in the pilot to non-clinical, reversible actions.
Measure alert-to-incident ratio, percentage of alerts auto-closed as noise, mean time to detect, mean time to restore, anomaly-detection false positives, and percentage of incidents resolved with a knowledge article. Also track automation-induced incidents, which is a key risk. The goal is decision support, not autonomy, until evidence supports expansion.
3. How would you improve knowledge management and configuration data as enablers, and how would you measure their contribution?
Read the worked answer
For knowledge, stop measuring article count. Identify the top 20 repeat diagnoses, such as infusion-pump connectivity, assign owners, and convert validated fixes into short, searchable articles embedded in the incident workflow. Retire or archive outdated articles; 25% older than two years is a signal to review, not automatically delete. Use service desk and incident data to find gaps, and make article creation part of problem management and major-incident closure.
For the CMDB, prioritise CIs for clinical services and repeat incidents. Reconcile through discovery where possible, and make CI updates mandatory in change and incident closure. Measure accuracy by sampling, not by record count.
Contribution measures include first-contact resolution, repeat incident rate, time to restore, percentage of incidents with a linked article, sampled CMDB accuracy for critical CIs, and user-reported confidence. These are enablers of the value chain, so report them alongside restoration outcomes rather than as standalone IT vanity metrics.