Offline field-service sync produces duplicates and silent overwrites
A field-service app used by 40 technicians lets them create work orders and attach photos offline, then synchronises when connectivity returns. Technicians work in rural areas with intermittent 2G/3G and frequently leave the app in the background for hours. Since the last release the support desk has received 22 reports of duplicate work orders, and the server shows three conflicting versions of one job record edited by two technicians on different devices. The sync layer uses a last-write-wins rule with client timestamps taken from device clocks; one technician's tablet was 40 minutes slow. The test environment has Wi-Fi only, two shared test accounts, and no way to run two devices against the same record simultaneously until a new staging environment arrives next sprint. Release is in three weeks, and the development team says the defect is 'probably the network'.
1. Design a test approach for offline creation and synchronisation. State the states you would test, the oracles you would use, and how you would create two devices editing the same record given the current environment constraints.
Read the worked answer
Model the app as a queue plus a server and test the transitions.
Offline creation states: never online this session; goes offline mid-edit; goes offline after save but before sync; app killed while offline; device rebooted; storage full during photo attach; device clock changed or skewed.
Sync states: queued, in-flight, acknowledged, failed, retrying, conflicted, partially uploaded (3 of 7 photos).
Oracles - write these down before testing:
- Every locally created item eventually exists exactly once on the server (idempotency key per item, not per request).
- The conflict rule is documented and deterministic, and the losing version is retained in history rather than silently destroyed.
- Sync resumes after process death without re-sending acknowledged items.
- The user can see unsynced or failed items and act on them.
Creating two editors in the current environment: the two shared accounts are a constraint, not a blocker. Use one account on two devices with distinct device IDs, or seed the server via API to simulate the second technician's edit. Drive the device clock manually, and via a mock-time flag if one exists, to reproduce the 40-minute skew and a device set in the future. Simulate 2G with a proxy or Network Link Conditioner: high latency, packet loss, and dropping the response after the server has committed - that last case is where duplicates are born.
Add scale cases: 200 queued work orders and 50 photos, sync interrupted mid-batch, and a second sync triggered before the first finishes.
2. The development team claims the duplicates are 'probably the network'. Describe the experiments you would run to localise the defect, and what evidence would change your mind.
Read the worked answer
Treat 'probably the network' as a hypothesis with a discriminating experiment.
- Drop the response, not the request. Use a proxy to let the POST reach the server, then kill the connection before the response returns. If a duplicate appears, the retry path is unsafe.
- Replay the identical request twice with the same idempotency key. Two records means server deduplication is missing or keyed wrongly. One record means the server side is sound.
- Inspect the client retry payload. Same key or a freshly generated one? A new key per retry shifts the fault to the client.
- Kill the app mid-upload and restart. Does the queue mark the item 'sent, unacknowledged', or re-send blindly?
- Force a Wi-Fi to cellular handover during upload, and add 2G latency and packet loss.
- Correlate with production reports. For each reported duplicate, pull the server access-log entries and compare timestamps and payloads.
Interpretation matrix:
- Same key, two records → server defect.
- Different keys for one logical work order → client defect.
- Same key, one record, but two shown in the app → display or merge defect.
- Duplicates only when the response is lost → retry design defect, usually shared.
Change one network variable at a time, otherwise the result is not attributable. Capture proxy, client and server logs together for each run - that triad is the evidence that ends the argument.
3. Last-write-wins with client timestamps: what is wrong with this design from a testing standpoint, and how would you test the resulting behaviour?
Read the worked answer
Client clocks are user-controlled and cannot be trusted as an ordering mechanism. A device 40 minutes slow, set to a future date, changed timezone or adjusted for DST wins or loses conflicts arbitrarily, and the user sees data silently disappear.
From a testing standpoint this is worse than a bug: the expected result is undefined. Last-write-wins with client timestamps has no reliable oracle, so a test can 'pass' by accident.
What to do:
- Raise it as a design defect with a concrete scenario: technician A edits at 10:00 with a clock 40 minutes slow; technician B edits at 10:15 with a correct clock; the server receives A at 10:45 and B at 10:20, so the older edit overwrites the newer.
- Demand a specified rule: server-assigned receive time, per-record version vectors, field-level merge, or a conflict surfaced to the user with both versions.
- Test the specified rule once it exists: skew cases (slow, fast, future, timezone change, DST boundary), simultaneous edits, edit-during-sync, three-way conflicts.
- Regardless of the rule, verify history: the overwritten version must remain retrievable, and the UI must show who changed what and when.
- Add production monitoring: count conflicts and overwrites so the business can see the data-loss rate.
Until the rule is specified, record the area as blocked with a documented risk rather than reporting a passing test.