Elastic Certified Engineer cheat sheet

elasticsearch.yml lives in $ES_PATH_CONF (default $ES_HOME/config); the minimum keys to set are cluster.name, node.name, path.data, and path.logs.

Setting network.host to a non-loopback address triggers production bootstrap checks, including vm.max_map_count >= 262144 and a nofile ulimit of at least 65535.

Heap is set in config/jvm.options.d/*.options or via ES_JAVA_OPTS; keep Xms equal to Xmx, at or below 50% of RAM, and capped near 31 GB to preserve compressed oops.

Secure settings belong in the keystore, managed with bin/elasticsearch-keystore add|list|remove, not in elasticsearch.yml.

Setting precedence, highest first: transient cluster settings, persistent cluster settings, elasticsearch.yml, then node defaults.

Node roles are declared with node.roles; omitting the key grants the default set of roles, while node.roles: [] produces a coordinating-only node.

Bootstrap a new cluster with cluster.initial_master_nodes listing the initial master-eligible node names, then remove the setting once the cluster is formed.

cluster.initial_master_nodes must never be set on a node that is joining an existing cluster; it is only for the very first election.

xpack.security.enabled defaults to true in 8.x, so TLS and the elastic user password must be configured before a production node will serve traffic.

Cluster health: green means all shards are assigned, yellow means all primaries are assigned but at least one replica is not, red means at least one primary is unassigned.

cluster.routing.allocation.enable accepts all, primaries, new_primaries, or none; set it to none before a full-cluster restart or upgrade, then restore all.

POST _cluster/reroute takes commands move, cancel, allocate_replica, and allocate_stale_primary to resolve unassigned shards manually.

Snapshot repositories require path.repo in elasticsearch.yml pointing to a shared filesystem mounted on every node, then PUT _snapshot/ with type fs.

Disk watermarks default to low 85% (no new shards on the node), high 90% (shards relocate away), and flood_stage 95% (indices become read-only).

ILM phases run in the order hot, warm, cold, frozen, delete; rollover requires an alias that points at exactly one write index.

Remove a master-eligible node with POST _cluster/voting_config_exclusions?node_names= before shutting it down to avoid losing the master.

_cat endpoints (nodes, shards, allocation, indices, health) return plain text with ?v and machine-readable output with ?format=json.

POST _reindex with source and dest copies documents between indices; set wait_for_completion=false for long runs and poll _tasks for status.

The bulk body is newline-delimited JSON with Content-Type application/x-ndjson; no pretty-printing is allowed and the final line must end with a newline.

Each bulk action is an action line such as {"index":{"_index":"x","_id":"1"}} followed by a source line; delete actions carry only the action line.

A bulk request returns HTTP 200 even when items fail; check the top-level errors flag and inspect per-item status codes such as 429 or 400.

Use refresh=false (the default) or refresh=wait_for for bulk loads; refresh=true forces a shard refresh per request and sharply reduces throughput.

Ingest pipelines are ordered processor lists: set, remove, rename, convert, date, grok, dissect, geoip, script, foreach; create with PUT _ingest/pipeline/.

Grok patterns use named regex captures like %{IP:client.ip}; _ingest/pipeline//_simulate tests a pipeline without indexing documents.

index.default_pipeline applies a pipeline when the request specifies none; a request-level ?pipeline= overrides it, and index.final_pipeline always runs last.

POST _reindex accepts a pipeline parameter to transform documents in flight and conflicts=proceed to continue past version conflicts.

Processors execute in array order; on_failure at processor or pipeline level handles errors, otherwise the whole document is rejected.

text is analyzed and scored for full-text search; keyword stores the whole value as one term for exact match, sorting, and aggregation; multi-fields let one input be both.

dynamic accepts true (auto-map new fields), false (ignore new fields but keep them in _source), or strict (reject documents with unmapped fields).

index: false keeps a field in _source but makes it unsearchable; doc_values: false disables sorting and aggregation for that field.

enabled: false on an object stores the subtree in _source without parsing or indexing any of its subfields.

keyword ignore_above defaults to 256 characters; longer values remain in _source but are not indexed and cannot be searched or aggregated.

index.mapping.total_fields.limit defaults to 1000; exceeding it raises a limit of total fields error, so use dynamic_templates or the flattened type.

object fields flatten subfields into one document, losing per-object correlation in arrays; nested preserves correlation but each nested query costs like a join.

A field type cannot be changed in place; reindex into a new index with the corrected mapping, optionally using _update_by_query for other fixes.

Runtime fields are declared in the mapping or the search request with a Painless script and are computed at query time rather than stored in the index.

match analyzes the query text and builds a boolean query over the terms; term and terms match exact indexed values and therefore belong on keyword or numeric fields.

filter context (bool.filter or constant_score) answers yes or no without scoring and is cached; must and filter both require a match, should is optional.

In query context with only should clauses, at least one must match; in filter context should clauses are optional unless minimum_should_match is set.

from + size is capped by index.max_result_window (default 10000); deeper paging requires search_after with a point in time, or scroll.

search_after passes the sort values of the last hit and needs a deterministic sort with a tiebreaker such as _shard_doc; scroll is a snapshot for exports.

multi_match types: best_fields takes the maximum score, most_fields sums, cross_fields treats the fields as one, plus phrase and phrase_prefix.

fuzziness is an edit distance (AUTO gives 0 for 1-2 characters, 1 for 3-5, 2 above); prefix_length and max_expansions control the cost.

Default sort is _score descending; adding any sort field stops score computation unless track_scores is true.

range supports gte, gt, lte, lt with date math such as now-1d/d and an explicit format; exists matches documents where the field has an indexed value.

Aggregation families: bucket (grouping), metric (numeric result), pipeline (reads other aggregations), and matrix (multiple values per document).

terms aggregation size defaults to 10, and results are approximate because each shard returns its top shard_size terms before the coordinator trims.

date_histogram uses calendar_interval (day, month, year) for DST-aware buckets and fixed_interval (30m, 7d) for exact durations.

composite aggregation paginates buckets with an after key and supports multiple sources: terms, date_histogram, histogram, and geotile_grid.

cardinality relies on HyperLogLog++ and is approximate; precision_threshold, capped at 40000, trades memory for lower error.

min_doc_count: 0 fills empty histogram buckets, but only within the range given by extended_bounds.

Pipeline aggregations such as bucket_script, derivative, cumulative_sum, moving_fn, and bucket_sort take their input values from buckets_path.

The global aggregation escapes the query scope; use filter or filters for scoped sub-aggregations under a broader search.

text fields cannot be aggregated unless fielddata is enabled, which costs heap; aggregate on the keyword subfield instead.

An alias can point to many indices; with is_write_index true one index accepts writes, enabling an atomic reindex-and-swap of the live index.

_msearch sends alternating header and body NDJSON lines and returns one response per query, cutting round trips for multi-query pages.

The _analyze API returns the token stream for a field or an explicit analyzer, tokenizer, and filter chain, used to validate analyzers before indexing.

HTTP 429 means the request was rejected by a queue or bulk-size limit; clients should retry with exponential backoff and cap concurrency.

Composable index templates combine index_patterns, priority, template (settings, mappings, aliases), and composed_of referencing component templates.

Write-then-read consistency needs refresh visibility: use ?refresh=wait_for on the write or accept the default one-second refresh interval.

_mget fetches many documents by id in one call; _bulk combines index, create, update, and delete operations in a single request.

For consistent deep pagination open a PIT with GET //_pit?keep_alive=1m, pass its id to _search, and close it when the page walk ends.

Client libraries (elasticsearch-py, elasticsearch-js, Java API Client) wrap the same REST endpoints and must match the server's major version.

An anomaly detection job needs a detector (function plus field), a bucket_span, and a datafeed that queries the source index.

Detector functions include count, mean, min, max, sum, high_mean, low_mean, rare, freq_rare, time_of_day, and lat_long.

bucket_span sets analysis granularity; results are written to .ml-anomalies-* indices and ranked by record_score.

Start order is open the job, then start the datafeed; stop in reverse. Model state persists across closes via model snapshots.

Dedicated ML nodes use node.roles: [ml]; a job's model_memory_limit cannot exceed xpack.ml.max_model_memory_limit.

Influencers attribute anomalies to specific field values; a detector scope restricts analysis to chosen influencer values.

Forecasting extrapolates a job's model into future buckets and requires the job to be open with sufficient model history.

Data frame analytics jobs (regression, classification, outlier detection) write to a destination index and are separate from anomaly detection.

A trained model is deployed with POST _ml/trained_models//_deploy and used through the inference processor or the _inference API.