Two Panels, Two Kinds of Invisible

Lookup matching is exact, whole-value, and case-sensitive. A CSV key of WEB07 does not match WEB07.corp.example.com or web07 or WEB07 - there is no implicit wildcard, and trailing whitespace is enough to break a match.

That accounts for both clusters precisely: the DMZ hosts fail on the FQDN/short-name mismatch, the second batch fails on whitespace.

Normalize both sides, not just the search:

index=itsm sourcetype=ticket
| eval host=lower(trim(host))
| eval host_short=mvindex(split(host,"."),0)
| lookup asset_owners host_short OUTPUT owner, criticality
| fillnull value="unassigned" owner

And rebuild the CSV so keys are already normalized:

| inputlookup asset_owners.csv
| eval host=lower(trim(host))
| eval host_short=mvindex(split(host,"."),0)
| fields - host | rename host_short AS host
| outputlookup asset_owners.csv

Notes:

• Normalize at ingest of the lookup, or every consumer must remember to do it. One forgotten lower() recreates this ticket.
• fillnull runs after the lookup, so "unassigned" becomes a visible, countable state instead of a silent null.
• CIDR-style match_type options exist for IP keys; they do not solve hostname fuzziness. Normalization is the only general fix.
• Confirm the miss rate is real, not a field-name mismatch (host vs Host), by checking | inputlookup asset_owners.csv | head 1 for exact column names.

tag=critical is a knowledge-object search, not a data search. It matches events whose field-value pair (or event type) has been tagged critical in the current app context, for a role that has permission to see it. An app update can remove any link in that chain while the underlying events are untouched.

Triage order - cheapest first:

1. Prove the data is still there: index=itsm sourcetype=ticket | stats count over the same window. If counts are normal, data is not the cause.
2. Test the tag definition directly: Settings → Tags, find critical, and note which field-value pairs or event types it decorates and which app owns it.
3. Test the equivalent without the tag: search index=itsm sourcetype=ticket severity=critical. If that returns rows, the tag or its scope is the problem; if it does not, the field-value the tag was built on no longer extracts - check the field in the sidebar on a known-good event.
4. Check app context and permissions: run the panel's search in the app that owns the tag, and confirm the role has read access. Tags created in one app are invisible elsewhere unless shared to All apps.

Common root causes: the tag was attached to an event type that the update disabled or renamed, the tag was created against a field extraction that the new props.conf no longer produces, or the tag was scoped to the app that just got a new version and lost its default context.

Fix at the knowledge-object layer (re-scope or re-tag), not by rewriting the panel to hardcode severity=critical - otherwise you have two definitions of "critical" that will drift.

Workflow actions are field-bound UI objects; they only render when their prerequisites are satisfied in the search results being viewed:

• The field must exist in that result set. A lookup-derived owner only appears if the lookup ran in that search. If some panels fillnull it to unassigned, decide whether unassigned should open the tool at all - configure the action to require a non-empty, non-sentinel value.
• App context and permissions matter, as with tags. The action must be shared to the app the user is searching in and be readable by their role. Actions defined in one app are invisible elsewhere by default.
• Field name must match exactly, including case, and the action is attached to a specific field - not to "whatever the table happens to show".
• Dynamic URLs must be built safely. The target URL is assembled from the field value; if values can contain spaces or special characters your lower()/trim() normalization must also be applied, or the link will 404 for the same DMZ hosts that broke the lookup.

Verification: implement, then test with a search that returns both a matched and an unmatched host, in the app, as the target role - not as admin. Admin-visible-therefore-works is the standard way these ship broken.