'Successful' exercise, absent engineers, stale SOC 2

Sufficiency is quantity; appropriateness is relevance plus reliability. Run each item against the test objective - operable, meaning it would work under pressure.

• Tabletop report, graded 'Successful'. Tests discussion and plan navigation, not technology recovery. The grade has no stated criteria, so it cannot be re-performed or challenged. Relevance: partial. Reliability: limited by being management-prepared.
• Attendance: 9 of 14, zero of four application support engineers. The people who would execute the technical recovery were absent from the exercise that supposedly validated it. That is not a sampling detail; it undercuts the conclusion.
• Pricing engine steps walked through verbally. The critical steps were never executed, so nothing in the file demonstrates they work or that they fit the RTO.
• SOC 2 Type II, period ending 16 months ago, no bridge letter. A Type II covers a period, so the evidence is 16 to 28 months old. Worse, its relevance is unknown until I read the scope and the complementary user entity controls - DR is frequently excluded or only partly covered.
• Vendor email. An undated, unscoped assertion by an interested party. Lowest reliability; cannot stand alone.
• Plan v7, 19 months old, three of five contacts departed. The plan fails a basic currency test on its face; a call tree would break at the first step.

Conclusion: the evidence supports 'a plan exists and a discussion-based exercise was held'. It does not support 'operable'. I would report at that level of precision rather than adopting management's word.

Ordered by cost against decisiveness:

1. Reconcile plan contacts to the HR roster (cheap, immediate). Confirms or dismisses the currency problem and quantifies how many plans are likely affected.
2. Obtain the SOC 2 scope section and CUEC matrix, then map each complementary user entity control to evidence of implementation. A SOC 2 opinion gives no comfort where the user entity has not performed its own controls. Request a bridge letter or a current report; if none, document the gap.
3. Corroborate the vendor's DR claim - inspect their test evidence, exercise right-to-audit or request an independent practitioner's report on DR. An email is not evidence.
4. Interview the four application support engineers who missed the exercise. Ask what they would do, in what order, and how long it takes. Divergence from the runbook is itself a finding.
5. Observe or witness an executed recovery test of the pricing engine - expensive and needs scheduling, but decisive. This is the procedure that actually answers the question.
6. Re-perform a call-tree test on a sample to test the notification mechanism, not just the list.

What would change my conclusion: an executed failover with timings inside the RTO, plus current contacts and a mapped CUEC position, gives me enough to conclude on operability. If a live test cannot be scheduled before the report date, I do not manufacture assurance - I report that operability is not evidenced, name the specific gap, and escalate the scheduling failure. 'Not evidenced' and 'not operable' lead to different findings and different management responses, and conflating them is unfair to management and indefensible at the audit committee.

Reliance on a service organisation is a decision that must be made explicitly and documented, not assumed because a report exists.

Steps I would take:

1. Define the control objective first - e.g. 'the order platform can be recovered within the 4-hour RTO'. Then ask whether the SOC 2 covers controls that bear on that objective. Often it does not: availability sections typically address uptime and backups, not failover timing or restore validation.
2. Read the scope and the period. Check the report covers the systems actually used, the period relevant to my audit period, and whether a bridge letter covers the gap to today.
3. Map CUECs. The service organisation's controls assume the user entity does specific things - identity management, change authorisation, monitoring of the DR schedule, data retention. If those are not implemented, the SOC 2 opinion tells me nothing about our exposure. I test implementation, not existence in a policy.
4. Use alternative procedures where the report is silent - vendor test evidence, right-to-audit, practitioner reports, contractual assurance, or our own test where feasible.
5. Record the reliance conclusion in the workpapers: what was relied on, for which assertions, over which period, and the residual gap.

If the report does not cover the recovery assertion, the correct position is that the third-party element of the plan is unassured, and that should appear in the report as a scope limitation affecting the conclusion - not be quietly absorbed into a 'satisfactory' rating.