AIOps Foundation cheat sheet

AIOps term coined by Gartner in 2016, defined as combining big data and machine learning to automate IT operations processes.

Precursors: 1990s expert systems for fault management; 2000s ITIL-driven monitoring; 2010s cloud-scale telemetry made manual correlation infeasible.

Gartner's 2017 Market Guide for AIOps Platforms distinguished AIOps from APM and ITOM by its cross-domain data aggregation and analytics.

Evolution stages: (1) siloed monitoring, (2) centralized log/event aggregation, (3) ML-based correlation and anomaly detection, (4) closed-loop automation.

Key enablers: cheap storage, distributed stream processing (Kafka, Flink), and supervised/unsupervised ML at scale.

AIOps differs from classic rule-based event correlation because models learn baselines from data rather than relying on static thresholds.

Industry adoption accelerated after 2018 as cloud-native architectures multiplied telemetry volume beyond human triage capacity.

Standards and communities: open-source projects (Prometheus, OpenTelemetry) and vendor-neutral observability initiatives shaped AIOps data models.

The three V's of big data: volume, velocity, variety; AIOps adds veracity (noise) and value (actionable insights).

Telemetry types: metrics (time-series numbers), logs (unstructured text), traces (causal request graphs), events (state changes).

Stream processing (Kafka, Flink) enables near-real-time ingestion; batch processing (Spark) supports model training and historical analysis.

Anomaly detection methods: statistical (z-score, EWMA), distance-based (k-NN), density-based (LOF), and reconstruction-based (autoencoders).

Supervised learning requires labeled incidents; unsupervised and semi-supervised methods dominate because incident labels are scarce.

Time-series forecasting (ARIMA, Prophet, LSTM) supports capacity planning and proactive alerting on predicted threshold breaches.

NLP techniques (tokenization, clustering, embeddings) group log messages and reduce alert noise via deduplication and correlation.

Feature engineering for AIOps: rolling statistics, seasonality indicators, service topology features, and deployment-change markers.

Model evaluation metrics: precision/recall for alerting, F1 for imbalanced incident classes, MTTR reduction for operational impact.

DevOps is a culture and practice for fast delivery; AIOps is an analytics layer that consumes DevOps-generated telemetry.

MLOps governs the ML lifecycle (training, versioning, deployment, monitoring); AIOps models should be managed under MLOps practices.

SRE defines SLIs, SLOs, and error budgets; AIOps can compute SLIs from telemetry and alert on error-budget burn rate.

AIOps supports SRE toil reduction but does not replace SLO ownership or incident command decisions.

CI/CD pipelines feed AIOps with change events, enabling correlation between deployments and incident onset.

Model drift monitoring is an MLOps concern; alert-quality drift monitoring is an AIOps concern; both require feedback loops.

AIOps platforms often integrate with ITSM (incident, change) and observability stacks (metrics, logs, traces) rather than replacing them.

Governance overlap: model approval, audit trails, and rollback plans must span MLOps, DevOps, and SRE teams.

Benefits: reduced alert noise, faster MTTR, proactive detection, capacity optimization, and toil reduction for operations staff.

Challenge: data quality - missing, inconsistent, or unlabeled telemetry degrades model accuracy and trust.

Challenge: tool sprawl and heterogeneous data schemas require normalization before correlation is possible.

Challenge: skills gap - teams need combined data science, operations, and domain expertise.

Change management: operators resist opaque models; explainability and human-in-the-loop review increase adoption.

Ethics: algorithmic bias in alerting can systematically ignore certain services or customer segments; audit model outcomes.

Ethics: privacy - logs may contain PII; data minimization and access controls are required before ML ingestion.

Ethics: accountability - automated remediation must have guardrails, approval thresholds, and clear human ownership of outcomes.

Cost: model training and telemetry retention consume budget; ROI should be measured against MTTR, incident volume, and staff hours saved.