Solution Architecture and Design
Framing the architecture decision
Solution architecture questions test whether you can translate a business requirement into a defensible technical shape. Know the building blocks: the request path (client → gateway → orchestration layer → model API), the context assembly layer that decides what the model sees, the tool/action layer, the retrieval layer, and the observability and guardrail planes that wrap all of them. A strong answer names which component owns each responsibility and where state lives.
Concepts to master
- Statelessness of the model API vs. stateful conversation stores; sessions, memory, and durable workflow state are your design problem, not the model's.
- Synchronous vs. asynchronous patterns: streaming for latency-sensitive UX, queues and webhooks for long-running agent work, idempotency keys for retried actions.
- Coupling and boundaries: keep prompts, model choice, and retrieval policy behind configuration so you can swap models without rewriting business logic.
- Failure domains: what happens when the model times out, returns malformed output, or a tool fails mid-plan.
Traps
Questions love to bait you with over-engineering (multi-agent for a single classification task) or under-engineering (no fallback when the primary model is unavailable). Watch for "always" and "never" options; architecture answers are usually conditional. Also beware conflating the orchestration layer with the model - orchestration owns retries, routing, and validation.
Rule of thumb
Start from the simplest topology that satisfies the requirement, then add a component only when you can name the failure it prevents. If an option improves resilience but adds a hop you cannot justify, it is usually the distractor.