Orchestration of Agents in Production Environments
Many agents, one repository
Orchestration is the scheduling, isolation, and sequencing layer around the agents. Each task gets its own branch and workspace so two agents never edit the same file concurrently; the pull request is the durable state, so a crashed worker can be resumed or handed to a human. Give each task an idempotency key - issue number plus task id - so retries do not open duplicate PRs or post duplicate comments.
GitHub Actions as the runtime
workflow_dispatch and issue/PR events trigger runs; concurrency groups prevent overlapping runs on the same branch; matrix jobs fan work out; environments gate deployment jobs behind approvals. The key discipline is keeping orchestrator state outside the model - in issues, labels, and commit statuses - so it survives a failed session.
Useful patterns
- Planner → executor → reviewer as distinct steps with distinct prompts and distinct permissions.
- A queue with bounded concurrency and a per-run cost ceiling.
- Retries with backoff only for transient failures; escalate deterministic failures to a human.
- A merge queue for serialised integration, with required checks making parallelism safe.
Traps
- Unbounded fan-out: opening two hundred PRs overnight that no reviewer can absorb.
- No dead-letter or escalation path, so failures vanish silently.
- Assuming one agent knows another agent's in-flight work.
- Reusing one write-scoped token across every stage of the pipeline.
Rule of thumb
Design for at-least-once execution: every step must be safe to run twice. If re-running the workflow creates a second PR, a duplicate comment, or a double deployment, the step is not idempotent yet.