LAG Hashing and a Single Elephant Flow
A server connects to a switch pair via a 4-member LAG of 10 Gb/s links (40 Gb/s aggregate) using LACP. The server runs a backup job that opens one TCP connection to a remote host and pushes data at line rate. Users report the backup completes far slower than expected - roughly 9-10 Gb/s, not near 40 Gb/s. Interface counters show all four LAG members up, LACP negotiated, and only one member carrying heavy traffic while the others are near idle. The switch uses a default Layer 3/Layer 4 hash.
1. Explain why the single flow does not use the full LAG bandwidth and whether this is a fault.
Read the worked answer
This is expected behavior of link-aggregation hashing, not a fault. A LAG distributes flows, not packets, across members; a single flow is hashed to exactly one member to preserve ordering. With one TCP connection, the hash lands on one 10 Gb/s member, capping throughput near 10 Gb/s regardless of the 40 Gb/s aggregate.
Confirm: check per-member utilization (show interfaces ... | include rate or similar) and verify the hash result. All members up + LACP healthy + one member saturated = normal single-flow behavior. The "40 Gb/s" is aggregate capacity across many flows, not per-flow bandwidth.
2. What options exist to improve the backup throughput, and what are the trade-offs?
Read the worked answer
Options, in order of practicality:
- Parallelize the flow - run multiple concurrent TCP streams (e.g., multi-threaded backup tool, multiple jobs). Multiple flows hash across members, approaching aggregate. Lowest risk, no config change.
- Tune the hash - if the switch supports a hash that includes more entropy fields (e.g., L4 ports, or a configurable hash seed), it can spread more flows, but it cannot split a single flow. Marginal benefit for a single elephant flow.
- Use a higher-bandwidth single member - e.g., 2×25/40 Gb/s instead of 4×10 Gb/s, so a single flow gets more per-link bandwidth. Costs optics/ports.
Trade-off: hashing changes affect all traffic and can imbalance other flows; per-flow bandwidth is fundamentally bounded by the fastest single member. Recommend parallel streams first, evaluate member speed only if single-flow performance is a hard requirement.