Softphone Rollout on Undersized Branch Circuits
Northline Distribution runs 14 branch warehouses feeding a single ERP instance at headquarters. Nine branches connect over 3 Mbps MPLS circuits; the other five, all acquired in the last eighteen months, still run 1.5 Mbps circuits because their contracts do not expire until next year. Headquarters has two 10 Mbps circuits with BGP failover. Traffic is hub-and-spoke and branches never talk to each other directly.
Last quarter Northline added softphone VoIP at every branch and complaint volume tripled. Measured round-trip latency from the five small sites averages 205 ms during 09:30-11:30 and 15:00-16:30, against 55 ms elsewhere; jitter at those sites reaches 40 ms and packet loss hits 3%. ERP month-end batch jobs and the voice streams share a single FIFO queue on every router. The MPLS contract carries no class-of-service guarantee, and the carrier's SLA credits only trigger above 1% loss measured at the provider edge. The IT manager wants to order more bandwidth everywhere; the CFO has approved nothing.
1. Before any bandwidth is purchased, what does the evidence identify as the most likely cause of the voice quality problem, and how would you test that hypothesis cheaply?
Read the worked answer
Start with arithmetic, not purchase orders.
- The gap is queueing, not distance. A 150 ms difference between the small sites and the rest cannot be propagation delay on domestic MPLS; propagation is single-digit milliseconds. It appears only in the two busy windows and arrives with 40 ms jitter and 3% loss, the signature of tail-drop on a saturated single-queue interface.
- Serialization matters at 1.5 Mbps. A 1,500-byte packet takes roughly 8 ms just to clock onto the link; a small voice frame sitting behind a burst of ERP writes waits for the entire backlog.
- The ERP is the aggressor. Month-end batch and interactive ERP share one FIFO queue with RTP, so a single large write can starve voice for hundreds of milliseconds.
Cheap test: at one branch, apply a trial configuration with a priority queue for RTP (DSCP EF), shape the circuit to 90% of committed rate, and enable link fragmentation and interleaving. Collect IP SLA voice scores for a week before and after. If MOS rises with no added bandwidth, the hypothesis is confirmed and the fix is configuration, not circuits. Pull interface error and duplex counters in parallel to rule out a physical fault before blaming design.
2. Draft the WAN and QoS plan for the next twelve months, stating what you configure, what you buy, and what you deliberately do not buy.
Read the worked answer
Spend on configuration first, circuits second.
Configure on all 14 routers:
- Low-latency queueing with RTP in the EF class, capped by call admission control at what the link can actually carry (roughly 4-6 concurrent calls on 1.5 Mbps with G.729 plus header overhead).
- AF31 for interactive ERP, AF11 for batch, and a small class-default so batch cannot be starved completely.
- A shaper at 85-90% of committed rate so the provider queue never builds behind the customer queue.
- LFI on the 1.5 Mbps links so voice frames are not delayed behind 1,500-byte data frames.
- Matching policy at headquarters, because the hub is where the bursts enter.
Buy: upgrade the five small sites only where, after QoS is in place, sustained busy-hour utilization remains above roughly 70% - the point at which the priority queue itself starts to starve data. Time the upgrades to contract expiry to avoid early-termination charges.
Do not buy: extra bandwidth at the nine healthy sites, or a second headquarters circuit. Neither addresses the measured fault, and both lock in recurring cost that QoS would have avoided.
3. The carrier offers SLA credits of 5% of monthly circuit charge when loss exceeds 1% at the provider edge. Why is that credit unlikely to help, and what should the renewal contract require instead?
Read the worked answer
The credit measures the wrong segment. Loss between provider routers never sees the customer access tail, the customer router's queue, or the ERP burst that caused the drop. If congestion sits on the customer side of the demarcation point - which the two-window pattern suggests - credits never trigger, and 5% of a monthly circuit charge is trivial next to a dropped call to a customer.
Require instead:
- A class-of-service schedule with per-class latency, jitter, and loss objectives for EF traffic, measured end-to-end to the customer edge rather than only inside the provider cloud.
- Guaranteed committed rates per class and a stated oversubscription ratio for the access segment.
- Demarcation-to-demarcation measurement with monthly reports the customer can audit, not a portal screenshot.
- Credits that scale with the duration and frequency of the breach rather than a flat percentage.
- A trial period with an exit clause if voice scores miss target at two named sites.
Contract language is cheaper than bandwidth, it is the only lever that survives the next application rollout, and it converts an unverifiable promise into a measurable obligation.