Elastic Certified Engineer: Machine Learning

What to know

ML in Elasticsearch means anomaly detection jobs and data frame analytics. An anomaly detection job has an analysis_config containing a bucket_span, one or more detectors (a function such as count, mean, high_mean, rare, plus field_name and optional by_field_name, over_field_name, partition_field_name), and influencers. A datafeed streams documents from an index pattern and query into the job and requires a date field and _source. Results appear as bucket results, records, and influencer results, with rolling overall_buckets for a summary view. Jobs can be grouped with groups and can produce forecasts and model snapshots. Data frame analytics covers outlier_detection, regression and classification, writing results to a destination index and producing trained models consumable by an inference processor.

Traps

• A datafeed needs _source enabled and a genuine date-typed field; a string date or date_nanos mismatch will fail the job.
• Stop the datafeed before closing the job, and adding force (usually after deleting the datafeed) is required to delete a job that still owns results.
• Detector functions decide what is anomalous: count needs no field, mean/metric need a numeric field, and rare works on categorical values.
• ML work runs on nodes with the ml role, bounded by model_memory_limit; jobs fail rather than silently dropping models.
• Job states (opened, closed, failed) are distinct from datafeed states (started, stopped).

Rule of thumb

State the question as a detector plus a bucket before writing any JSON: "unusual request rate per 5 minutes per service" becomes count partitioned or split by service with bucket_span: 5m.