Guide · AI in production

Building AI Agents That Work in Production

AI agents fail in production for different reasons than in demos. Task decomposition, tool design and error handling for agents that work reliably.

Published Updated 4 min read
Building AI agents that work in production: start with a workflow, break tasks into checkable steps, wrap tools in adapters, gate risky actions behind a person, and fail loudly.

An AI agent that impresses in a demo and one that runs your operations unattended are different things. The demo shows the happy path. Production is everything else: the input in the wrong format, the API that times out, the task that can't be finished because something upstream changed.

Most of what makes an agent reliable isn't clever prompting. It's a handful of engineering decisions made before the first run. Here they are, starting with the one most teams skip: whether you need an agent at all.

Start with a workflow, not an agent

Anthropic draws a useful line. In a workflow, the model and its tools follow paths you've written in code. In an agent, the model decides its own steps and which tools to use. Their advice is to find the simplest solution that works and add complexity only when it's needed, because agents trade cost and speed for flexibility, and their errors can compound (Anthropic).

From simple to complex. A single model call: classify an email or summarise a document. A workflow: fixed steps in code, the model fills in each step, for example read invoice, extract fields, check against the purchase order, post or flag. An agent: the model chooses its own steps and tools, for example research a lead across several sources until it has enough. Move right only when the step to the left can't do the job.
Most business tasks we see are workflows. Use an agent when the steps genuinely can't be known in advance.

In practice, most business tasks are workflows. Processing an invoice follows the same steps every time, and the model's job is to read and judge within each step. Research that needs to follow leads across sources, where the next step depends on what the last one found, is where an agent earns its place.

Break the task into steps you can check

Give an agent one broad instruction and it makes dozens of small decisions about scope and format, differently on every run. Give it a sequence of defined steps, each with an expected input, an expected output and a clear failure condition, and you get consistent results.

It also makes problems fixable. "Which step went wrong?" has an answer. "Why did the agent do the wrong thing?" usually doesn't.

Design tools the model can't misuse

Anthropic recommends putting as much care into the interface between the model and its tools as you'd put into a screen for people: clear names, descriptions written like documentation for a new colleague, and arguments shaped so mistakes are hard to make.

The pattern we use is a thin adapter around every external system. The model never calls the raw API. It calls an adapter that checks the input, makes the call, turns the response into a small, predictable shape and returns errors as plain results the model can read, instead of crashing.

A tool adapter between the model and an external API. The model calls a small tool such as find_customer with an email. The adapter validates the input, calls the API with retries and a timeout, keeps only the fields the model needs, and returns either a short result or a structured error such as not found or rate limited. The raw API response never reaches the model.
The adapter absorbs the messy parts of the real system. The model sees a small, steady interface.

Decide where a person approves

Whether an agent needs approval for an action is a decision to make before launch, based on two questions: how much does it cost if this is wrong, and can it be undone?

When an agent needs a person's approval. Low cost and reversible: run automatically, for example draft an email or update a CRM note. Low cost but hard to undo: run with a log and a daily review, for example tag a record. High cost but reversible: run, then alert, for example place a fulfilment hold. High cost and hard to undo: a person approves first, for example send to a customer, refund, delete or pay.
Two questions decide it: what does a mistake cost, and can it be undone?

Drafting an email or updating a note can run on its own. Sending to a customer, issuing a refund, paying a bill or deleting a record should wait for a person. Holding an order sits in between: it's easy to undo, so it can run and tell someone. That's the pattern in our Fraud Hold demo build, where a rule holds a risky order and only a person releases it.

Put limits on every run

An agent that gets the right answer after forty tool calls when five would do is still a problem. Costs and waiting times add up. Set a maximum number of steps and a cost ceiling per run, and when the agent hits either, it should stop and say so rather than keep going.

Make failure obvious

The most dangerous agent is one that fills a gap with something plausible. When it doesn't have what it needs, it should return a clear "couldn't complete" with the reason, not a guess that looks finished.

Log every tool call with its inputs and result. Before launch, build a test set from real past cases, including the awkward ones, and run it after every change to the prompt or tools. An agent is ready when it passes that set reliably and fails loudly on the cases it can't handle.

Where to go next

Agents usually connect to other systems through a standard such as MCP, which our guide to MCP as an integration layer explains. For the code that holds an agent together, prototype to production covers what to check before it goes live.

See it working

Lead Research and CRM Sync

A reference build of an intake pattern: an enquiry arrives, the contact and company are researched automatically, and a complete record lands in the CRM with an owner assigned.

Lab build, fictional business, synthetic data

Open the demo

Axioprax builds this

Frequently asked questions

What is the difference between an AI workflow and an AI agent?

In a workflow, the steps are fixed in code and the model does the reading or judging inside each step. In an agent, the model decides which steps to take and which tools to use. Workflows are cheaper and more predictable, so they're the better default.

When should an AI agent ask a person for approval?

When an action is costly if wrong or hard to undo: sending to a customer, refunding, paying, deleting. Cheap, reversible actions like drafting or tagging can run on their own with a log.

How do I test an AI agent before launch?

Build a test set from real past cases, including the awkward ones, and run it after every change to the prompt or tools. It's ready when it passes reliably and fails clearly on the cases it can't handle.