Mapping explosion, unaggregatable tags and cross-object false matches

The problem is not the number of documents, it is that every distinct key becomes a mapping entry and a field in cluster state on every node. That is why the master is struggling: cluster state publication is O(fields).

Options, in order of preference:

1. flattened - one field, depth_limit (default 20), arbitrary sub-keys, no mapping growth.

   "specs": {"type": "flattened", "depth_limit": 20}
   

   Lost: numeric semantics. Values are indexed as keywords, so specs.weight < 5 compares lexicographically ("10" < "5"). Fine for exact-match filters and term/prefix queries on things like specs.voltage: 220; wrong for range queries on measurements. Also no per-key field-level scoring, no highlighting, and the whole object shares one set of doc values, so per-key aggregations are coarse.
2. enabled: false - the object stays in _source but is not indexed at all. Cheapest; you keep the data for display and can re-map later. Lost: any query or aggregation on specs.
3. Schema-on-write with a controlled key set - promote the keys you actually query (weight, voltage) to real numeric fields via a dynamic template with path_match: specs.* and match_mapping_type: long/double, and map the rest as flattened or store them only in _source. This is the only option that preserves numeric range queries.

Also set "dynamic": "strict" (or a catch-all dynamic template) on products so the next supplier batch fails loudly at index time instead of quietly adding 3,000 fields. And check GET products/_mapping size before and after - a dramatic drop in cluster state is the proof the fix worked.

tags is text because it is tokenised by the standard analyzer; aggregation needs doc values, per-document ordinals, which text fields do not have. The error message is telling you exactly that.

The tempting shortcut, "tags": {"type": "text", "fielddata": true}, compiles the whole inverted index into memory per shard, per node, is unbounded, and will OOM the data nodes on an 8M-doc field with several million distinct values. It also changes nothing about tokenisation - you would still be aggregating on analysed terms.

Correct fix: a multi-field.

"tags": {
  "type": "text",
  "fields": {
    "keyword": {"type": "keyword", "ignore_above": 256}
  }
}

Mechanics that matter:

• PUT _mapping on an existing field is rejected - you cannot add a sub-field to a live text field? You can add a keyword sub-field to an existing text field, but documents already indexed will not have values for it; only new/updated documents will. So the aggregation would silently return partial results - worse than an error.
• Therefore: create products_v2 with the full mapping, reindex, then atomically swap the alias.

  POST _reindex {"source": {"index": "products"}, "dest": {"index": "products_v2"}}
  POST _aliases {"actions": [
    {"remove": {"index": "products",   "alias": "catalog"}},
    {"add":    {"index": "products_v2", "alias": "catalog"}}]}
  

• ignore_above: 256 prevents a pathological long tag from bloating the term dictionary; choose the value from actual max tag length, not by default.
• Keep tags (text) for relevance matching if needed and aggregate on tags.keyword.

Mechanism: an array of objects is flattened at index time into parallel multi-valued fields - variants.color: [red, black], variants.size: [S, XL], variants.price: [80, 20]. Lucene has no concept of "the same object"; a conjunction across those fields is satisfied by any combination of values. Hence red+S=80 and black+XL=20 satisfy the query as a whole.

Fix - nested:

"variants": {
  "type": "nested",
  "properties": {
    "color": {"type": "keyword"},
    "size":  {"type": "keyword"},
    "price": {"type": "scaled_float", "scaling_factor": 100}
  }
}

GET catalog/_search
{
  "query": {
    "nested": {
      "path": "variants",
      "query": {"bool": {"filter": [
        {"term": {"variants.color": "red"}},
        {"term": {"variants.size": "XL"}},
        {"range": {"variants.price": {"lt": 50}}}
      ]}},
      "inner_hits": {"size": 5, "name": "matched_variants"},
      "score_mode": "avg"
    }
  },
  "aggs": {
    "by_color": {
      "nested": {"path": "variants"},
      "aggs": {"colors": {"terms": {"field": "variants.color"},
                        "aggs": {"avg_price": {"avg": {"field": "variants.price"}}}}}
    }
  }
}

Costs to state out loud: each nested object is its own Lucene document, so 8M products × 6 variants is ~56M docs of postings and doc values; nested queries are join-like and do not benefit from many query optimisations. include_in_parent: true (or copy_to) is a legitimate optimisation when you also need cheap non-nested filtering, but it reintroduces cross-object matching for anything evaluated against the flattened parent fields - use it only for single-field filters.

Because this is a mapping change on existing data, it again means new index + reindex + alias swap, not a _mapping update.