Multi-value instance variable returns no data
An operations team maintains a Grafana dashboard with two template variables. region is a custom variable with values eu, us, and apac, multi-value enabled, and an All option. instance is a query variable using label_values(node_cpu_seconds_total, instance), also multi-value with All. The main panel query is:
sum by (instance) (rate(node_cpu_seconds_total{instance="$instance", mode!="idle"}[5m]))
With one instance selected the panel draws lines. With two instances selected or All selected, the panel shows "No data". The dropdowns themselves list values correctly. The team also wants a share link that opens the dashboard on the last 6 hours with region=eu and instance=All. The dashboard is used by on-call engineers who often select All during incidents, and the team cannot add a recording rule this quarter.
1. Why does the panel return no data when multiple instances or All are selected?
Read the worked answer
The panel uses an exact-match label matcher, instance="$instance". When a Grafana variable has multiple values, Grafana interpolates them into a single string. With an exact matcher, Prometheus looks for one instance label equal to something like web-01,web-02 or to the All value, which does not exist. A single selection works because the interpolated string happens to equal one real label value.
The dropdown listing values correctly only proves the variable query works. It does not prove the panel query uses the right matcher. The fix is to use Prometheus's regex matcher, instance=~"$instance", so multiple values are joined into an alternation. For All, also confirm the variable's All value produces a pattern that matches every instance, such as .*, rather than a literal string that matches none.
2. Give the corrected panel query and the variable settings you would apply.
Read the worked answer
Use a regex matcher:
sum by (instance) (rate(node_cpu_seconds_total{instance=~"$instance", mode!="idle"}[5m]))
For the instance variable, keep multi-value enabled and set the All option. Set a custom All value of .* if the default does not do so, and test it explicitly. For region, decide whether it should filter the panel. If it should, add region=~"$region" to the selector; otherwise the region dropdown is decorative and will confuse on-call users. The instance query should also respect region if region is meant to narrow the instance list, for example by filtering the metric selector in label_values.
If instance names contain dots or other regex metacharacters, escape them or avoid a custom regex that assumes plain text. After the change, test one instance, two instances, and All, and verify the panel's legend and line count match the selection.
3. How would you build the requested share link, and what would you verify after opening it?
Read the worked answer
Use Grafana's variable URL syntax with explicit time bounds, for example:
?from=now-6h&to=now&var-region=eu&var-instance=All
Open the link in a private browser session or as a user who is not the dashboard author. Verify three things: the time picker shows Last 6 hours, the region dropdown shows eu, and the instance dropdown shows All with the panel rendering data. If the All option uses a custom value, the URL may need that custom value instead of the literal All; test the exact value rather than assuming.
Also confirm that any dashboard links or panel links preserve the variables, and that switching region after opening the link still updates the instance list if those variables are chained. Document the link format in the runbook so on-call users do not hand-edit the URL.