Cellular Water Meters with Factory Defaults
Calder Water Authority operates 2,500 NB-IoT water meters across a rural service area. Firmware 1.2 sets each meter's admin password to the last six digits of its serial number and leaves secure boot disabled on 700 units installed before 2023. A 400 KB firmware image moves slowly over NB-IoT, and the utility has no staged rollout tooling. The vendor says a fleet-wide credential rotation will leave 3–5% of meters unable to reconnect, requiring truck rolls. Last week one meter's SIM consumed 38 times its normal data volume over six days, opened TLS sessions to an IP in another country, then went quiet. The utility has three field technicians, a five-year replacement cycle, and regulatory pressure to avoid service interruptions. The meter vendor offers a cloud device-management console but no local agent.
1. Is the anomalous meter evidence of compromise, and what immediate containment is defensible?
Read the worked answer
Treat it as a probable compromise until disproved. The combination of 38x data volume, foreign TLS destinations, and then silence fits recruitment into a botnet or data exfiltration, though a vendor update or SIM malfunction could mimic it. Do not send a remote factory reset or a fleet-wide command, because that destroys evidence and may brick the meter.
Immediate containment: suspend or restrict the SIM at the carrier/APN so it can reach only the approved meter broker; preserve SIM session logs, flow records, and any device logs; dispatch a technician to capture the meter locally and replace it if needed. Check the other 2,499 meters for the same destination ASN, connection pattern, and credential failures. If the APN is shared, add an egress allowlist for the broker and DNS. Notify the vendor and carrier under the incident process. The judgment call is to contain at the network edge rather than the device, because the device has no trustworthy remote administration path.
2. Sequence the fleet hardening work across credentials, secure boot, and OTA given three technicians.
Read the worked answer
Sequence by risk and reversibility. First, stop the bleeding: move all meters to an APN egress allowlist and enable per-device authentication at the broker where possible. Second, rotate credentials in canary batches, not fleet-wide: 1% first, observe reconnection for 72 hours, then 10% batches, with a rollback plan and pre-staged truck-roll capacity. Budget for the vendor's 3–5% failure estimate (75–125 meters) and schedule rotations by geographic cluster so technicians are efficient.
Third, enable secure boot only where it can be done without a truck roll or at natural replacement; for the 700 legacy units, treat physical access as the constraint and bundle secure boot with battery or meter replacement. Fourth, build OTA staging: smaller delta images, resumable downloads, and maintenance windows. The key is not to let a security project create a larger service outage than the threat.
3. What telemetry would you require to detect future recruitment into a botnet?
Read the worked answer
Require per-device behavioral baselines and carrier-side telemetry: bytes in/out per day, session count, destination ASN and country, TLS SNI where visible, DNS queries, failed authentication counts, and reconnection storms. Alert on deviations such as a 5x data volume, a new ASN not on the broker allowlist, repeated connections to a single external IP, or a device that goes silent after anomalous traffic. Correlate SIM identity with device serial and firmware version. Add broker-side metrics: authentication failures, unexpected publish topics, and command rate.
Because NB-IoT devices are low-power, a small absolute increase can be significant; set thresholds per firmware profile, not fleet-wide. Retain flow logs long enough to investigate, and test the alerts with a red-team meter. Monthly billing reports are not detection.