One Tenant Sees DNS Timeouts in Shared Internet VDOM
A managed security provider runs a FortiGate 1800F with 12 VDOMs. Each customer VDOM connects to a shared inet VDOM through an inter-VDOM link; the inet VDOM owns two WAN links, SNAT, and the default routes. Tenant7 reports intermittent DNS timeouts to 8.8.8.8 and slow access to a SaaS application, while the other 11 tenants are normal. A week ago, an engineer added a policy route in tenant7 for a new SaaS subnet, pointing to the inet VDOM, and lowered tenant7's per-VDOM resource limits to 8% CPU and 10% memory. The tenant's DNS servers are internal, but the FortiGate forwards some queries to 8.8.8.8 through the inet VDOM. The provider needs to prove whether the fault is in tenant7, the inter-VDOM link, or the shared inet VDOM before changing anything.
1. Outline a diagnostic sequence that isolates tenant7 from the shared VDOM.
Read the worked answer
Start at the tenant boundary. In tenant7, run diagnose sys session filter vd tenant7 and inspect DNS sessions to 8.8.8.8; then use diagnose debug flow filter vd tenant7 with the client IP and port 53. Check diagnose firewall proute list in tenant7 to see whether the new SaaS policy route also catches DNS or sends it to the wrong next-hop. On the inter-VDOM link, use diagnose netlink interface list and interface counters to confirm the link is up and not dropping. In the inet VDOM, check SNAT, WAN health, and session counts for the tenant7 source range. Compare the same test from tenant8. If tenant8 DNS works through the same inet path, the shared VDOM and WAN are probably healthy and the fault is tenant7 policy or resource limits.
2. What specific debug flow and session table findings would distinguish a policy route problem from an upstream DNS problem?
Read the worked answer
Debug flow should show the packet entering tenant7, matching the expected policy, and being forwarded to the inter-VDOM link. If it shows a different policy route or a route lookup to an unexpected interface, the new policy route is the cause. If it shows allowed and SNAT, the packet left the FortiGate; then a session table entry for UDP 53 with a short expiry and no reply points upstream. Check diagnose sys session list for proto=17, the translated source, and the outgoing interface. If there is no session at all, look for a deny or a routing failure. If sessions exist but expire without a reply, test DNS from the inet VDOM and from another tenant to separate upstream DNS from tenant policy.
3. What remediation and guardrails would you recommend for per-tenant changes?
Read the worked answer
Treat per-tenant changes as production changes. Require a named policy route with a specific source and destination, not a broad catch-all, and place it below existing DNS and business routes. Review per-VDOM resource limits before lowering them; an 8% CPU limit can throttle inspection and logging, so test under load and use session and memory limits rather than aggressive CPU caps where possible. Give tenant admins a profile scoped to their VDOM, and keep shared inet changes with the provider. After remediation, re-run the debug flow and session checks, compare tenant7 with a healthy tenant, and monitor DNS latency and policy route hit counts for 24 hours before closing the incident.