· 13 min read
A Request Walks Into Your System: Composing the Ten Agent Patterns Into One Production Pipeline
The pattern library tells you what each piece does. This is the missing article: the order a real build encounters them, where each hands off to the next, what each costs, and how each one fails in production.
The one decision that determines everything downstream
Every agent system starts with a decision most teams make by accident: how much control you hand to the model. Anthropic's "Building Effective Agents" draws the line cleanly — a workflow is a system where your code decides the sequence of LLM calls; an agent is a system where the model decides its own sequence. Everything else in the pattern library sorts onto one side of that line or the other. Prompt Chaining, Routing, Parallelization, and Evaluator–Optimizer are workflow patterns: the graph is fixed, the model fills in nodes. ReAct, Reflexion, Orchestrator–Workers, and the Autonomous Agent Loop are agent patterns: the graph is discovered at runtime.
The decision is not aesthetic. It sets your debugging story, your cost ceiling, and your failure modes for the life of the system. A workflow fails legibly — step 3 produced garbage, look at step 3. An agent fails narratively — it took a plausible wrong turn eleven steps ago and everything after is confidently downstream of that mistake. The engineering rule that falls out: use the workflow patterns for every part of the task whose structure you can know in advance, and spend agency only where you genuinely cannot enumerate the steps. Most production "agents" that work are mostly workflow, with a small agentic core doing the one thing that actually needs it. Teams that invert this — an open-ended loop wrapping a task that was actually three fixed steps — buy all the variance of agency and none of the benefit.
The honest test: write down, on paper, the steps a competent human would take for five representative inputs. If the five lists are the same list, you are building a workflow, whatever your framework calls it.
Shape the task before any model reasons about it
The cheapest quality win in the whole pipeline happens before the expensive model runs, and it uses the two humblest patterns: Routing and Prompt Chaining.
Routing is triage, and it should be boring. A small, fast classifier call — or, wherever possible, no model at all: a regex, a length check, a lookup — sends each request down a branch built for it. The point is not sophistication; it is that a prompt trying to handle refunds, technical questions, and abuse reports simultaneously does all three worse than three narrow prompts do one each. The failure mode is equally boring and very common: the router's categories drift out of sync with reality. Requests that fit no branch get silently jammed into the nearest one, and because the router "succeeded," nothing alerts. The fix is structural — give the router an explicit unclear bucket, route that bucket to a human or a generalist path, and track the bucket's rate as a first-class metric. A rising unclear rate is your users telling you the taxonomy is stale, a signal you get for free if you bothered to make room for it.
Prompt chaining is decomposition with checkpoints. Instead of one call that reads a document, extracts claims, and drafts a response, you run three calls — and the win is not primarily quality, it is that between calls your code gets to look at intermediate output. That gap is where validation lives: schema checks, allowlists, sanity bounds, a cheap programmatic gate that rejects malformed output before it poisons the next step. A chain without checks between links is just one long prompt with extra latency. The failure mode is error compounding: a slightly wrong extraction in step 1 becomes a confidently wrong answer in step 3, and the final output reads fluently enough that nobody suspects the middle. Log every intermediate. When the chain misbehaves, the transcript of hops is your stack trace — and if you didn't keep it, you have no stack trace, only vibes.
Where the loop goes — and how it stays legible
Somewhere in the pipeline is the part you couldn't script, and that is where the loop lives. Its canonical shape is ReAct (Yao et al., 2022): the model interleaves an explicit reasoning trace with tool actions, observes each result, and reasons again. Two things about ReAct matter more in production than in the paper. First, the reasoning trace is not decoration — it is the only artifact that lets you reconstruct why the agent did what it did, which makes it your primary debugging surface; systems that strip it to save tokens have traded their observability for pennies. Second, the observation step is where reality enters. An agent that acts without reading results — or whose tools return uninformative errors like failed with no reason — degenerates into a model hallucinating a world model instead of querying one. Tool output quality is agent quality. A tool that returns a precise, actionable error message is worth more than a better system prompt, and it is the fix teams reach for last because it feels like plumbing.
When the looped task is decomposable, the loop becomes an orchestrator over workers: one model reads the specific input, derives the subtask list it couldn't have known in advance, dispatches narrow worker calls, and synthesizes. The guide's existing write-up already makes the key point — decomposition and synthesis quality are the orchestrator's whole job, and underperformance is almost always an orchestrator problem, not a worker problem. What that write-up doesn't say: the orchestrator's second job is knowing when to stop decomposing. An orchestrator that recursively spawns workers for subtasks a single call could handle is the multi-agent version of premature abstraction, and it multiplies cost and failure surface with no quality gain.
Where subtasks are independent, dispatch them with plain parallelization — either sectioning (different workers, different pieces) or voting (same task, multiple samples, aggregate). Parallelization is the one pattern with a near-mechanical payoff: latency drops to the slowest branch instead of the sum. Its failure mode is dependency denial — running steps concurrently that secretly depend on each other, then "merging" contradictory results in a synthesis prompt that papers over the contradiction instead of surfacing it. If two branches can disagree, the merge step needs an explicit policy for disagreement, not a hope that the synthesizer smooths it out.
Wrapping all of this is the autonomous agent loop: plan, act, observe, repeat until done. The load-bearing engineering is not the loop — any framework gives you the loop — it is the termination contract. Production loops need a maximum step count, a budget ceiling, a wall-clock limit, and a defined behavior when any of them trips that is more useful than dying mid-task: return partial work, say what's missing, say what it would try next. An agent that must choose between "finished" and "crashed" will lie and pick finished.
Quality is enforced, not hoped for
Nothing so far guarantees the output is any good. Two patterns exist to make quality a mechanism rather than an aspiration, and they operate at different timescales.
Evaluator–Optimizer works within a single request: one call produces, a second call criticizes against explicit criteria, the first revises, loop until pass or budget. It genuinely works — but only under a condition teams routinely skip checking: the evaluator must be able to detect the failures you care about. Evaluation is often easier than generation (checking a summary against a source is easier than writing it), and that asymmetry is the entire reason the pattern pays. When the asymmetry doesn't hold — when judging correctness is as hard as producing it — the loop converges on output that satisfies the evaluator, which is not the same as output that is correct. You get well-formatted, confident, evaluator-approved wrongness, which is strictly worse than obvious wrongness because it survives review. Before shipping the loop, red-team the evaluator itself: hand it known-bad outputs and measure the catch rate. An evaluator you haven't tested is a rubber stamp with latency. And make the criteria concrete — "cites a source for every numeric claim" is checkable; "is high quality" is a horoscope.
Reflexion (Shinn et al., 2023) works across attempts: after a failed episode, the agent writes a verbal self-critique — what went wrong, what to try instead — and that reflection is injected into the next attempt's context. It is learning without gradient updates, and its production value is concentrated in one requirement the paper is explicit about: there must be a real outcome signal. Reflexion's strong results are in domains with ground truth — code against test suites, tasks with checkable end states. Ask an agent to reflect on a failure it has no way to detect and it will manufacture a plausible-sounding lesson about a mistake it may not have made, then dutifully apply the wrong correction. Reflexion amplifies whatever signal you feed it, including noise. If your domain has no cheap verifier, build one before building the reflection loop — or skip the loop.
There is a cheaper discipline hiding under both patterns, and it may matter more than either: route failure signals back into the system at all. A surprising number of production agents fail, get manually corrected by whoever noticed, and learn nothing, because the correction lives in a Slack thread instead of a store the next run reads. A lesson that changes no mechanism is not a lesson; it is a recurring incident with better documentation.
What the system remembers, and what it must not
Every pattern above assumes context, and context is a budget, not a bucket. Memory architecture in production is three separate problems that teams conflate at their peril.
Working memory is the current context window — the transcript, tool results, retrieved documents. Long-running loops fill it, and then something must be evicted or compressed. Summarization-based compaction is the standard move and it is lossy in a directional way: summaries preserve conclusions and shed caveats, dead ends, and the reasoning that ruled things out. An agent working from its own summaries will re-explore paths it already rejected, because the rejection's reasons didn't survive compression. Compact conservatively, and preserve constraints and rejected options with priority over narrative.
Episodic and long-term memory — what persists across runs — is where the real design decisions live, and the operative question is not "what should we store?" but "what must this system be structurally unable to remember?" Cross-tenant leakage through a shared memory store is a breach, not a bug. User PII in a vector index has no deletion story that survives an audit. Draw the boundary in the schema, not in the prompt: a retrieval layer that is scoped to tenant by construction beats a system instruction asking the model to please not use other customers' data.
Two failure modes deserve naming because they are quiet. Memory poisoning: whatever the system writes to memory, later runs treat as trusted context — so if tool outputs or user text can influence what gets stored, an attacker (or just a wrong answer) writes itself into every future session. Treat memory writes as a privileged operation with validation, not a side effect. Staleness: a remembered fact has no expiry, and the world moves. A memory that was true when written and false when retrieved is worse than no memory, because the system cites it with the confidence of experience. Prefer storing pointers to re-checkable sources over conclusions, and timestamp everything, so "when did we learn this?" is always answerable.
Where the human stands is architecture, not apology
Human-in-the-loop gates get designed last and bolted on, which is exactly backwards — where a human must approve is a load-bearing architectural decision, because it determines what the agent can do autonomously, and therefore what the agent is for.
The practical framing: classify actions by reversibility and blast radius, not by how nervous they make you. Actions that are cheap to undo — drafting, reading, computing, staging a change — need no gate; gating them buries your humans in approval fatigue until they rubber-stamp everything, at which point the gate is theater that costs latency. Actions that are irreversible or externally visible — sending money, deleting data, messaging a customer, changing production — get a hard gate that no amount of model confidence bypasses. The gate must be mechanical: enforced in code or permissions the model cannot talk its way past, never a prompt instruction, because a prompt instruction is a request and requests can be reasoned around.
Two design details separate gates that work from gates that decay. First, a gate must never block the loop: the agent writes a pending-approval record and moves on to other work, rather than stalling an entire pipeline on a human's inbox. Second, the approval surface must show the human what will actually happen — the actual message, the actual SQL, the actual amount — not the agent's summary of it. A human approving a summary is approving the agent's self-report, and the divergence between self-report and action is precisely the failure the gate exists to catch. Log every gated decision with what was shown, who approved, and when; the audit trail is half the point.
The bill, stated plainly
Composition multiplies costs, and it is worth being unromantic about the arithmetic. A routed, chained request with an orchestrator, four workers, and two evaluator passes is on the order of ten model calls where a naive design made one. Tokens scale worse than linearly, because agentic steps carry accumulated context — each ReAct iteration re-reads the growing transcript, so a 20-step loop pays for its history 20 times. Latency in chains is additive; parallelization claws some back; evaluator loops add a full generation per revision cycle.
Failure probability compounds too. Ten sequential steps at 98% each is roughly 82% end-to-end — before correlated failures, which are the ones that actually get you: a provider having a bad hour degrades every step at once, and retries amplify the load precisely when the upstream is least able to serve it. Budget retries per-request, not per-step, or a single stuck request will quietly spend fifty calls.
The uncomfortable, durable conclusion — the same one Anthropic's guidance leads with — is that the composed system must justify itself against the single well-crafted prompt, per route, with measurements. Some routes deserve the full pipeline. Many don't, and the routing layer you built in section two is exactly the mechanism that lets simple requests take the cheap path while hard ones take the expensive one. If you cannot articulate which of your routes needs which machinery, you haven't finished designing; you've finished accreting.
Monday morning
Concretely, in order. First, take your five most common request types and write the human's step list for each; where the lists are stable, replace agency with a workflow — this usually deletes cost and variance in the same commit. Second, add validation between every pair of chained calls, and give your router an unclear bucket with an alert on its rate. Third, read a week of your agent's reasoning traces — actually read them; if you can't reconstruct why it acted, fix logging before fixing prompts, because you cannot improve what you cannot see. Fourth, red-team your evaluator with known-bad outputs and write down its catch rate; if you don't have an evaluator, that number is zero and now it's written down. Fifth, enumerate every irreversible action the system can take and verify each is gated mechanically, not by prompt. Sixth, find one failure from last week and check whether anything the system will read next week records it. If the answer is no, you have an agent with amnesia, and every incident you've had is still scheduled to happen again.
None of this is novel, and that is the point. The patterns are stable; the engineering is in the seams — the checks between chain links, the termination contracts, the tested evaluators, the mechanical gates, the memory boundaries. The teams whose agents work in production are not the ones with the cleverest loops. They are the ones who made every hop observable, every failure recordable, and every irreversible action stoppable.
What to do about it
- Spend agency only where you cannot enumerate the steps in advance; script everything else as a workflow.
- Put a programmatic check between every pair of chained LLM calls — a chain without checkpoints is one long prompt with extra latency.
- Treat tool error messages as part of the agent: a precise error string improves behavior more than most prompt edits.
- Red-team your evaluator with known-bad outputs before trusting its loop; an untested evaluator is a rubber stamp.
- Use Reflexion-style self-critique only where a real outcome signal exists — it amplifies noise as faithfully as signal.
- Gate irreversible actions mechanically, show approvers the actual action rather than the agent's summary, and never let a pending approval stall the loop.
Sources: Anthropic — Building Effective Agents · ReAct: Synergizing Reasoning and Acting in Language Models · Reflexion: Language Agents with Verbal Reinforcement Learning
agent-architecturellm-workflowsproduction-systemsevaluationhuman-in-the-loopmemory