An Alert That Nobody Receives and a Project Nobody Can See
A Prism Central instance monitors 320 VMs across three clusters. The operations team reports that no email alerts have arrived for two days, although the Prism Central alert dashboard shows 27 open alerts, including six critical 'cluster disk usage' alerts. The alert policy 'Critical infrastructure' is enabled and scoped to the three clusters; its notification channel is set to 'email' with a distribution list. The SMTP server was migrated to a new relay with a new IP address last week. Separately, a new engineer in the 'Database' project can log in to Prism Central but cannot see the project's Calm applications or launch a blueprint. The engineer's AD group was added to a custom role that has project view but not blueprint launch; the role was cloned from a read-only role. A scheduled runbook that should restart a stuck service has also not run for three days; its schedule is enabled and the last run status is 'SKIPPED'.
1. The alert dashboard shows 27 open alerts, but no email has arrived for two days. Walk through the notification path and the most likely causes.
Read the worked answer
Alert generation is working because the dashboard shows open alerts; the break is in notification delivery. Trace the path: alert policy scope and severity -> notification channel enabled for that policy -> Prism Central SMTP configuration -> relay and allowlist -> distribution list -> recipient mailbox.
The SMTP relay migration is the leading suspect: the new relay IP may not be allowed through the network, the Prism Central SMTP settings may still point to the old relay, TLS or port or authentication may differ, or the sender domain may be rejected. Also verify that the 'Critical infrastructure' policy actually has email notification enabled, not just alerting, and that the distribution list accepts external senders. Confirm with a test notification from Prism Central, check SMTP logs on the relay, and inspect the policy's notification history if available. Do not close the incident when a test email reaches one admin; verify the distribution list and at least one critical alert end-to-end. Check spam or quarantine last, after the relay path.
2. The new engineer can log in but cannot see the Database project or launch its blueprint. What would you check, and why is adding Prism Central admin the wrong fix?
Read the worked answer
Project visibility is not the same as Prism Central login. Check: is the engineer's AD group a member of the Database project with an assigned project role? Has group synchronization completed? Does the role include project view and catalog view? Is the blueprint published to that project's catalog, and does the project have a valid environment and credential? The clue is that the role was cloned from a read-only role: it grants project view but not blueprint launch. Confirm the effective permissions on the project rather than the role name.
Adding Prism Central admin would bypass project isolation and grant far more than needed; it also makes audit and quota ownership ambiguous. Grant the minimum project-scoped role that includes blueprint launch, confirm the engineer can see the project and launch in a non-production environment, and verify the action is recorded under the engineer's identity. If the engineer only needs to view, do not add launch at all.
3. The scheduled runbook shows 'SKIPPED' for three days. What are the likely causes and how would you diagnose them without changing the schedule first?
Read the worked answer
'SKIPPED' usually means the schedule fired but the runbook did not execute, or the scheduler intentionally bypassed the run. Check the run history details first: was there a previous run still active, so the new run was skipped by an overlap or concurrency policy? Was the scheduled time outside an allowed window, or was the runbook's time zone different from the operator's expectation? Was the runbook disabled, unpublished, or missing a required project or credential at run time? Did the service account used by the schedule lose permission? Was Prism Central or the runbook service unavailable at the scheduled time, causing a missed fire?
Confirm the next scheduled time and the last successful run, then test a manual run with the same parameters. Do not change the schedule until you know whether the issue is overlap, authorization, or a missed fire; changing it can hide the cause and create duplicate runs. After the fix, run once manually and verify the audit record.