I85PREP
⌘ K
DCAT study materials
DCAT study resources

The Failing Drive in a RAID Set

DCAT · Put your knowledge to work in context.

All case studies

The Failing Drive in a RAID Set

A storage array holds 24 SAS drives in a RAID 6 set (two parity drives). It has been running for four years. Overnight, monitoring shows drive slot 7 flagged as predictive-failure, and a scrub job reports two uncorrectable read errors on slot 12. The array is 78% full. The vendor's replacement policy ships a same-model drive in 24 hours, but the spare pool is empty. A junior technician suggests pulling slot 7 immediately and swapping in a drive from a decommissioned array in another room that is the same model but has 5 years of runtime. The application owner wants to know whether the array can survive a second failure tonight.

1. Assess the risk to the RAID 6 set and explain why the technician's plan is dangerous.

Read the worked answer

Risk assessment:

  • RAID 6 tolerates two simultaneous drive failures. Slot 7 is predictive-failure (still online, may fail soon). Slot 12 has two uncorrectable read errors - meaning data on that drive cannot be read reliably. If slot 7 fails and slot 12 also becomes unreadable during rebuild, the array loses data.
  • The array is 78% full, which lengthens rebuild time and increases the window of vulnerability.
  • The spare pool is empty, so there is no hot spare to absorb a failure.

Why the technician's plan is dangerous:

  1. Pulling a live drive from a healthy array to swap in an unknown drive is a high-risk operation - if the new drive is incompatible or fails, you have consumed a fault-tolerance slot for nothing.
  2. The decommissioned drive has 5 years of runtime and unknown SMART history; it is more likely to fail than a new drive, potentially during the rebuild of slot 7.
  3. Rebuild on a 78%-full array is I/O-intensive and can trigger latent errors on other drives (the classic "rebuild-induced failure").
  4. If slot 12's uncorrectable errors are on the parity or data path, a rebuild may not complete.

Correct posture: Do not pull slot 7 yet. The predictive-failure flag means the drive is degraded but online; a controlled replacement with a new drive is safer than swapping in a used one.

2. What is the safest sequence of actions tonight, and what would you tell the application owner?

Read the worked answer

Safest sequence:

  1. Stop the scrub - a scrub on a degraded array with known uncorrectable errors adds I/O and may surface more errors. Pause it.
  2. Verify backups - confirm the most recent backup is restorable, not just completed. This is the real safety net.
  3. Monitor slot 7 and slot 12 closely (SMART, error counters). Do not pull either drive tonight unless slot 7 actually fails.
  4. Order a new same-model drive (not the decommissioned one) for 24-hour delivery; if the vendor has an expedite option, use it.
  5. Pre-stage the replacement so the swap is quick and controlled when the new drive arrives.
  6. Do not use the 5-year-old drive unless it is the only option and has passed a full SMART/long self-test.

Message to the application owner: "The array is currently within fault tolerance - we can survive one more drive failure, but not two. We have a degraded drive and a drive with read errors, so we are in a reduced-margin state. We are pausing the scrub, verifying backups, and expediting a new replacement drive. We do not recommend pulling any drive tonight; doing so would consume our safety margin. If a second failure occurs before the new drive arrives, we will restore from backup."

3. The new drive arrives and the rebuild begins. What monitoring signals tell you the rebuild is at risk, and what is your contingency?

Read the worked answer

Rebuild risk signals:

  1. Media errors on other drives during rebuild - any new uncorrectable error is a red flag; RAID 6 can still recover, but margin shrinks.
  2. Rebuild percentage stalling or slowing dramatically - indicates I/O contention or a failing source drive.
  3. SMART reallocated-sector counts rising on any drive.
  4. Array controller warnings about degraded or failed members.
  5. Temperature spikes - rebuilds run drives hot; verify cooling.

Contingency:

  • If a second drive fails or errors during rebuild, stop the rebuild and prioritize data recovery from backup rather than forcing the array.
  • If rebuild is slow but stable, throttle rebuild priority to reduce stress, accepting a longer window.
  • If slot 12 shows more errors, consider replacing slot 12 next after slot 7's rebuild completes, one drive at a time.
  • Keep the application owner informed at each milestone; document the timeline for the post-incident review.