Choosing the Right Automation Layer for a Mixed Operations Backlog
A platform team manages 140 VMs across two Prism Elements registered to one Prism Central. They have three recurring needs: (1) when Prism Central raises a 'VM disk capacity high' alert, add 20 GB to the affected VM and notify the owner; (2) every Friday at 22:00, power off all VMs in a non-production category tag and start them Monday at 06:00; (3) provision a three-tier internal app with a fixed network profile, including approval before production. The team has one service account, no external orchestrator, and a strict audit requirement. An admin proposes a single Python script on a jump host that polls the v3 API every 60 seconds and performs all three actions. A second admin wants to use only Calm blueprints. The platform lead asks you to review the proposal. One alerting issue has already appeared: the disk alert has fired four times in 20 minutes for the same VM, and a manual disk expansion is still in progress.
1. Map each of the three needs to the most appropriate Prism Central automation construct and justify the choice. Explain why the single polling script and the Calm-only proposal are both poor fits.
Read the worked answer
Use a Playbook for need 1, a Runbook for need 2, and a Calm blueprint in a project for need 3.
- Need 1: The playbook's alert trigger receives the alert entity and context, so it can target the specific VM, record the run, and require an approval if needed. A polling script has to reconstruct alert state and may act on stale alerts.
- Need 2: A scheduled Runbook is the operational automation fit: time-based, repeatable, auditable, and not tied to an application deployment. A Calm blueprint is for provisioning, not weekly power state.
- Need 3: Calm gives app profiles, project quotas, catalog publishing, approvals, and Day-2 actions. A runbook alone does not model the application lifecycle.
The single script is not automatically bad for integration, but here it centralizes credentials on a jump host, adds 60-second detection latency, lacks native approval and audit, and is prone to duplicate actions on retries. Calm-only is worse: it does not natively react to the disk alert or perform the scheduled power operations as cleanly. Choose constructs by trigger type, lifecycle scope, and governance needs, not by convenience.
2. The disk alert has fired four times in 20 minutes while a manual expansion is in progress. What failure mode should you design against, and what guardrails belong in the playbook?
Read the worked answer
The failure mode is a race and duplicate remediation: the alert remains active because the threshold is still breached, and each firing can start another playbook run while the first expansion is still executing. A naive 'add 20 GB' action can add multiple disks or expand the same disk repeatedly. A fixed delay is not sufficient because it does not check actual state.
Guardrails:
- Deduplicate or suppress the alert, or configure the playbook to run once per alert incident.
- At the start, query the VM's current disk size and exit successfully if it is already at or above the target.
- Wait for any in-flight Prism task to finish before mutating.
- Use an approval or maintenance flag for production; the approval also prevents concurrent manual and automated changes.
- Make the action idempotent: target a desired size based on a baseline, not 'add 20 GB every run'.
- Record the task UUID and final size in the run output. If the playbook cannot query state, do not use it for this remediation.
3. What credential and audit practices would you require before this team automates all three needs?
Read the worked answer
Use separate least-privilege Prism Central service accounts per automation domain: one for alert remediation, one for scheduled operations, one for Calm provisioning. Do not share the interactive admin account or store passwords in the jump-host script. Put credentials in the supported secret or credential store and rotate them; scope roles to the projects, categories, and clusters each automation needs. For Calm, bind the project to an environment and credential that cannot be used outside that project.
Require every automation to emit an auditable record: who or what triggered it, target entity UUID, task UUIDs, parameters, approval decision, and final state. Use Prism Central audit logs as the system of record, not a local text file. For scheduled runs and playbooks, test in a non-production project, fail closed on authorization errors, and review run history after changes. This gives traceability and limits blast radius if one credential leaks.