Building AI Agents That Actually Ship: A Practical Architecture

The gap between an agent demo and an agent in production is enormous. Demos are linear, forgiving, and watched. Production agents face hostile inputs, flaky APIs, and users who will type anything. After building automation agents for several businesses, I've settled on an architecture that holds up outside the playground.

The core loop is boring on purpose

Every agent I ship has the same skeleton: a loop that calls a model, executes tools, feeds results back, and knows when to stop.

async function runAgent(task: string, maxSteps = 10) {
  const messages = [{ role: 'user', content: task }];

  for (let step = 0; step < maxSteps; step++) {
    const response = await llm.chat({
      messages,
      tools: toolSchemas,
    });

    if (!response.toolCalls?.length) return response.text;

    for (const call of response.toolCalls) {
      const result = await executeTool(call);
      messages.push(toolResult(call.id, result));
    }
  }

  throw new AgentBudgetExceededError(maxSteps);
}

Three details matter more than they look:

1. maxSteps is a hard budget. Runaway loops burn money and trust.
2. Tool results get truncated before entering context. A 50KB API response will poison every subsequent call.
3. Errors become observations, not exceptions. The model can often recover if you tell it a tool failed and why.

Tools are APIs, so design them like APIs

The model reads your tool schema the way a developer reads docs. Vague names and mushy descriptions produce vague calls. My rules:

• Name tools after verbs: search_orders, send_email, create_invoice.
• Describe when to use the tool, not just what it does.
• Keep parameters flat and required-optional clean — models handle flat schemas far better than deep nesting.

const searchOrders = tool({
  name: 'search_orders',
  description:
    'Find customer orders by email, order ID, or date range. Use when the user asks about order status.',
  parameters: z.object({
    email: z.string().email().optional(),
    orderId: z.string().optional(),
    limit: z.number().max(20).default(5),
  }),
});

Guardrails before intelligence

An agent with write access is a liability until proven otherwise. Before any cleverness:

• Allowlist actions. The agent may call draft_refund, never issue_refund. A human approves drafts.
• Validate every tool argument at the boundary with a schema library — treat model output like untrusted user input, because it is.
• Log every step with inputs, outputs, token counts, and latency. When something goes wrong at 2 AM, this log is the whole story.

Memory: short context beats clever embeddings

For most business agents, "memory" means three things:

I store working notes as plain key-value facts appended to the system prompt rather than reaching straight for a vector database. Retrieval adds failure modes; a 2KB summary of relevant facts usually outperforms top-k chunk lookup for narrow domains.

Evaluate continuously, not once

The difference between teams that trust their agents and teams that don't is evaluation. Before any change ships, a fixed suite of tasks must pass:

Task 12: "Where is my order?" + no order id in history
  expect: asks clarifying question, never calls search_orders
Task 13: refund request under $50
  expect: draft_refund created, approval flag set

It's crude. It's also the only reason I can change a prompt on Friday afternoon without spending the weekend reading error logs.

What I'd tell my past self

Ship the smallest agent that touches one workflow end-to-end, instrument everything, and expand only when the evals say you can. The impressive part isn't the model — anyone can rent those. The impressive part is the scaffolding that makes it dependable, and dependable is what customers actually pay for.