Portside Tycoon 4.2: Risk-Based Planning With Six Testers
Skyline Interactive runs Portside Tycoon, a free-to-play mobile builder with 1.8 million daily active users. Version 4.2 ships in 14 calendar days and contains: a replacement of payment SDK v2 with a third-party v3, a new battle-pass season with 60 reward tiers, and an economy rebalance that adds a coin sink to curb inflation. You have six testers, 20 devices in the lab, and 10 working days. Audience data: 22% of DAU are on Android devices with 2 GB RAM, 9% are still on Android 9, and 61% are on iOS. The previous release had 99.2% crash-free sessions; the studio target is 99.7%. The payment vendor's sandbox has been unstable, with two outages in the last week. The producer wants a test plan by tomorrow and has already promised the date to the marketing team.
1. Build a risk register for 4.2. Identify at least four risks, score likelihood and impact, and justify the ranking you would present.
Read the worked answer
I would score likelihood 1-5 and impact 1-5 and rank by product, but the justification matters more than the number.
- Payment SDK v3 (likelihood 4, impact 5 = 20). New third-party integration, changed API surface, sandbox instability. Impact is revenue, refunds, chargebacks and store-policy exposure. Top of the list.
- Economy rebalance (likelihood 3, impact 5 = 15). A coin sink can be exploited or can strangle new players; both are hard to detect in a short window and both affect retention and revenue. High because the failure mode is a design-level exploit, not a crash.
- Low-RAM Android (likelihood 4, impact 3 = 12). 22% of DAU, new UI assets for the battle pass increase memory pressure; expect ANRs and OOM kills. High likelihood, moderate blast radius per user but large user count.
- Battle-pass entitlement and reward claiming (likelihood 3, impact 4 = 12). 60 tiers, cross-device claiming, restore-purchase paths; failure is visible and support-heavy.
- Android 9 TLS/compat (likelihood 2, impact 4 = 8). Older stack, new SDK, but narrower population.
I would present this as a ranked table with the owner for each risk and the test approach per risk, so the producer sees effort mapped to exposure rather than a flat list of features.
2. Propose the execution sequence for the 10 working days, and state explicitly what you would defer and why.
Read the worked answer
Sequence by risk and by dependency, not by feature order.
Days 1-2: build verification and smoke on the payment path end to end (purchase, restore, refund webhook, receipt validation, cross-device entitlement) on the two highest-share platforms. Nothing else matters if purchase fails. In parallel, a memory/soak pass on the two weakest 2 GB Android devices running the new battle-pass UI.
Days 3-5: battle-pass claiming across all 60 tiers via seeded accounts, plus economy scenarios with scripted player archetypes (new payer, whale, free rider) to hunt exploits. Run the Android 9 compatibility matrix here.
Days 6-8: full regression on core loops, device-matrix pairwise coverage, network-condition tests (poor connectivity during purchase is the classic double-charge), and localization smoke.
Days 9-10: fix verification, release-candidate regression, store submission checks.
Deferred: cosmetic-only visual polish on low-share devices, low-value localization typography, and non-blocking audio mix issues. I would document each deferral with residual risk. If the payment sandbox is down, I would not idle: run contract tests against recorded responses and keep a daily re-test slot.
3. Define entry and exit criteria, and explain what you would do if the vendor sandbox is unavailable for 3 of the 10 days.
Read the worked answer
Entry: a build with a versioned changelog, a testable payment configuration (sandbox credentials and test cards), the economy spreadsheet, and a stable device lab booking. Without sandbox access the payment risk cannot enter execution, so it stays 'blocked', not 'in progress'.
Exit: crash-free sessions >= 99.7% on the device matrix weighted by audience share; all payment paths pass on iOS and Android; no open severity-1 or severity-2 defects on the in-scope risk items; economy scenarios show no positive-arbitrage loop above the agreed threshold; and every deferred risk has a named acceptor.
If the sandbox is down for 3 days, three actions: (1) escalate through the vendor account manager with the schedule impact in writing, so the slip is documented; (2) reduce dependence on live sandbox by using recorded/contract-level stubs for functional flows, while marking real end-to-end payment as unverified; (3) re-plan so the payment end-to-end window moves to the last available days, and prepare a staged-rollout option (5% cohort) as the contingency if it stays unavailable. Do not silently absorb the loss into regression time; that trades a known high risk for a lower one.