Dell PowerMax Operate cheat sheet

PowerMax arrays are built from engine pairs (two V-bricks per engine) where each V-brick contains director nodes, an InfiniBand fabric, and attached media; a two-engine system has four V-bricks.

Directors run in active/active pairs; front-end (FA) directors host Fibre Channel, iSCSI, and NVMe/TCP ports, while back-end (BE) directors own disk I/O to the DAEs.

Each V-brick uses its own internal InfiniBand switch pair for node-to-node and node-to-media traffic; a failed IB switch degrades but does not stop the V-brick.

Media is NVMe flash in DAE trays (25-slot 2.5-inch and 120-slot dense trays); RAID protection is applied per DAE as RAID-5 (4+1 or 8+1) or RAID-6 (8+2 or 16+2).

A PowerMax T-shirt size (e.g., 2000, 8000, 2500) fixes the maximum usable capacity, engine count, and cache per engine; capacity cannot exceed the model ceiling without a model upgrade.

Cache is per-engine and shared by the two V-bricks in that engine; an engine is the failure domain for cache loss, so losing one engine leaves the other engine serving I/O.

SnapVX and SRDF use the same thin pools as host data unless separate pools are created; pool full conditions affect both host writes and snapshot allocation.

Physical capacity reported by Unisphere is raw, formatted (after RAID), and usable; usable capacity is what thin pools and SRDF devices can consume.

Unisphere for PowerMax (U4P) is the web UI and REST API server; it manages one or more arrays via Solutions Enabler or embedded management (Unisphere for PowerMax embedded on the array).

The REST API base path is /univmax/restapi/; authentication uses a Basic auth header or a session token from /univmax/restapi/100/system/session.

Solutions Enabler (symcli) commands use the form sym with -sid ; examples: symcfg list, symdev list, symsg list, symrdf query, symsnapvx list.

Unisphere roles include Admin, StorageAdmin, Auditor, and Monitor; Auditor is read-only including logs, Monitor is read-only without audit log access.

Performance data is collected by the Unisphere performance database; the default real-time view covers the last 4 hours and historical data retention is configurable per array.

Unisphere for PowerMax supports both local (embedded) and remote (external) deployment; remote U4P discovers arrays through Solutions Enabler gatekeepers or via the array's management interface.

Alert and threshold settings in U4P are per-array and can trigger email, SNMP trap, or syslog notifications; SNMP must be enabled on the array to send traps.

symcfg discover refreshes the local Solutions Enabler database of arrays and devices; running it is required after zoning changes before symdev or symsg commands see new devices.

Provisioning order: create thin pool (symconfigure or U4P), create devices (symdev create or U4P volume create), add to storage group, add initiators (masking view) to the storage group.

A storage group (SG) is the unit of masking and replication: SRDF, SnapVX, and mobility operate on storage groups, not individual volumes.

A masking view binds one storage group, one port group (FA director ports), and one initiator group (host HBAs); a volume can belong to only one storage group.

Host I/O limits (IOPS and MB/s) can be applied to a storage group or a port group; limits are enforced by the front-end director and are not cumulative across nested limits.

Thin devices allocate from a thin pool on first write; the pool's subscribed capacity can exceed physical capacity, so monitor subscription ratio (subscribed/physical).

Volume creation requires emulation type (e.g., FBA), capacity, and pool; U4P creates a device and adds it to an SG in one workflow, but symdev create alone does not mask the device.

Initiator groups can be created from WWN, iSCSI IQN, or NVMe NQN; a masking view with no initiator group is invalid and will not present LUNs.

LUN addresses in a masking view are assigned automatically unless explicitly set; changing a LUN address requires removing and re-adding the device to the SG, which disrupts the host path.

Each FA director port is identified by director-slot-port (e.g., FA-1C:4); a port group is a named list of FA ports used in masking views.

Fibre Channel zoning should be single-initiator, single-target (one HBA WWN per zone with one or more array ports); host OS multipathing (PowerPath or native MPIO) provides path failover and load balancing.

PowerPath requires the array to be in the PowerPath registration or uses auto-registration; without a valid license, PowerPath falls back to basic failover without load balancing.

iSCSI connectivity uses array iSCSI ports with CHAP (one-way or mutual) configured per initiator; VLAN and MTU settings on the host and switch must match the array port.

NVMe/TCP uses NQN-based host registration and requires the host to support the NVMe/TCP driver; NVMe-oF over FC uses FC-NVMe zoning.

Front-end port flags (SCSI, iSCSI, NVMe) determine which protocol the port serves; a port cannot serve SCSI and NVMe simultaneously without reconfiguration.

Path count per volume equals (number of FA ports in the port group) x (number of host HBA ports zoned to those ports); a minimum of two independent paths is required for redundancy.

symaccess list view and symmaskdb list database are the CLI tools to verify masking and HBA-to-device mapping; U4P shows the same under Hosts > Masking Views.

SnapVX is a pointer-based snapshot; the first snapshot consumes metadata only, and subsequent writes to the source consume pool space for changed tracks.

A snapshot session can be linked (writable copy-on-write clone) or not linked; linked targets can be used for backup, test, or restore.

Snapshot policies automate creation (interval and retention count); a policy can be assigned to a storage group, and snapshots are named with the policy prefix plus a timestamp.

Restoring a snapshot overwrites the source with the snapshot image; the source must be unmounted from the host or the host must be quiesced first.

Snapshot space usage is reported per storage group and per pool; if the pool fills, snapshots become invalid and the source may be affected depending on pool configuration.

symsnapvx list -sid -sg shows snapshots, generation numbers, and link status; generation numbers increase with each new snapshot.

Terminating a snapshot frees its allocated tracks back to the pool; terminating a linked target also removes the target device.

A snapshot of an SRDF device is taken on the R1 (source) side; taking snapshots on R2 requires the device to be in a valid SRDF state and is supported for some configurations.

SRDF modes: synchronous (SRDF/S) acknowledges host write after R2 acknowledgement; asynchronous (SRDF/A) uses cycle-based delta sets; adaptive copy (AC) is for data migration and does not guarantee RPO.

SRDF pairs are defined by RDF group (RA group) and device pairs; symrdf createpair -sid -rdfg -file -type <S/A/AC> creates pairs.

SRDF states include Synchronized, Consistent, Suspended, Partitioned, Split, and Invalid; only Synchronized and Consistent indicate replication is healthy.

srdf query returns pair state, mode, and RDF group; symrdf -sid -rdfg query -file -detail shows per-pair status.

Failover (symrdf failover) makes R2 read/write for disaster recovery; failback requires restore (symrdf restore) before resuming replication.

SRDF/A uses delta sets and cycle time; a cycle time that is too small for the write rate causes cycle switches to fail and the pair to suspend.

SRDF/Metro presents the R2 as an active/active copy with witness or bias arbitration; without a witness, a link failure can cause both sides to become read/write (split brain).

SRDF groups are numbered per array pair and require RA director ports and inter-site links (FC or IP); IP replication uses SRDF over IP with compression and encryption options.

Key array metrics: IOPS, MB/s, response time (ms), queue depth, and percent busy; response time above the service-level target (often 5-10 ms for OLTP) indicates contention.

Bottleneck analysis: high FA port busy (>70%) points to front-end; high BE director busy points to back-end; high device response time with low array utilization points to host or path issues.

Unisphere performance charts can be filtered by storage group, device, FA port, BE director, or pool; the default interval is real-time (last 4 hours) with historical views for longer periods.

Workload planning uses the array's performance database and tools such as Unisphere for PowerMax workload planner or Workload Planner in CloudIQ to model adding hosts or volumes.

Thin pool capacity management: monitor subscribed capacity, pool used percentage, and snapshot space; a pool above ~80% used should trigger expansion or cleanup.

Capacity reporting distinguishes physical (raw), formatted (after RAID), usable, and subscribed; subscribed capacity can exceed physical for thin devices.

Performance impact of SnapVX: snapshots add metadata overhead and pool writes; many snapshots of a busy volume increase back-end write amplification.

SRDF/A adds RDF write traffic to the R1 and requires adequate RDF group bandwidth; monitor RDF group throughput and cycle time to avoid suspension.

Array health is shown in U4P Dashboard and via symcfg list -sid -v; component states include Normal, Degraded, Failed, and Not Responding.

Alert categories include hardware (director, DAE, disk, power supply, fan), SRDF link, pool capacity threshold, and snapshot policy failures.

Call-home (Connectivity Hub or Secure Remote Services) sends alerts to support; verify call-home connectivity with symcfg list -sid -srdf and the connectivity status in U4P.

Routine tasks: review alert logs, check pool capacity, verify SRDF pair states, confirm backup snapshot success, and apply code upgrades per the support matrix.

symevent list -sid shows the event log; events have severity (Info, Minor, Major, Critical) and should be triaged by severity and recurrence.

Disk and DAE replacement follows hot-swap procedures; a failed disk in a RAID group triggers a rebuild, and the array reports degraded redundancy until the rebuild completes.

Firmware and microcode upgrades should be done with the array in a healthy state; SRDF and snapshot operations should be paused during a non-disruptive upgrade window.

Log collection for support uses U4P's Collect Logs or symmaint; logs include SPCollect, event logs, and configuration data.

Host cannot see a LUN: check masking view (SG, port group, initiator group), zoning, host HBA driver, and multipathing; symaccess list view -sid confirms the mapping.

Path failures: check FA port status (symcfg list -sid -fa all), switch zoning, and cable; a single path failure should be transparent to the host with multipathing.

SRDF pair suspended: check RDF link status (symrdf query -sid -rdfg ), cycle time, and R2 availability; resume with symrdf resume after correcting the cause.

Pool full: identify the pool (symcfg list -sid -pool), check which SGs and snapshots consume space, and expand the pool or delete snapshots.

Snapshot invalid: caused by pool exhaustion or source device changes; restore from a valid snapshot or recreate the snapshot after freeing pool space.

Performance issue: isolate by host, port, device, and pool; check for hot spots, queue depth, and host-side multipathing configuration.

Array in degraded state: check symevent list and U4P health for the failed component; replace hardware per procedure and verify rebuild completion.

symcfg discover and symdev list -sid -noport are useful to confirm device visibility and mapping before escalating.