Building AI agents: the patterns that hold up in production

Geschatte leestijd: 9 minuten

Inhoudsopgave

1. What are AI agents?
2. When to use an agent, and when not to
3. When and how to use frameworks
4. Building blocks: from a single call to a full agent
5. Combining and adapting these patterns
6. Summary: three principles
7. In practice: two applications
8. Prompt engineering for your tools
9. Frequently asked questions

We build AI agents for clients ourselves, and we keep running into the same pattern: the best results rarely come from an elaborate framework. They come from a handful of simple, reusable building blocks. Anthropic reached the same conclusion after a year of working with dozens of teams putting LLM agents into production.

This article lays out those findings, together with what we run into at And AI when we build agents for Dutch companies.

What are AI agents?

There is no single definition of "agent". Some teams call any system that takes several steps on its own an agent. Others reserve the term for systems that run fully independently, without a fixed script. Anthropic draws a clear line between two kinds of agentic systems:

• Workflows: systems where a language model and separate tools are directed through a predefined path.
• Agents: systems where the language model decides for itself which steps to take and which tools to use.

The difference is not just theoretical. A workflow is predictable and easy to test. An agent is more flexible, but harder to keep under control.

When to use an agent, and when not to

Start with the simplest solution that gets the job done, and only add complexity once you actually need it. Often that means not building an agent at all. Agentic systems cost more time and money per task than a direct model call, in exchange for better performance on harder tasks. That trade-off is not always worth making.

For tasks with a fixed structure, a workflow usually works better: predictable and consistent. An agent becomes the better option once flexibility and independent decision-making matter more than tight control. For a lot of applications, one well-designed model call with the right examples and context is already enough.

When and how to use frameworks

Plenty of frameworks make agentic systems easier to build, from SDKs to drag-and-drop workflow builders. They take standard work off your hands: calling a model, defining tools, chaining outputs together.

The downside: frameworks add a layer of abstraction that hides exactly what happens under the hood, which makes debugging harder. They also make it tempting to add complexity you don't actually need. Our advice: start by calling the model's API directly. Many patterns fit in a few lines of code. If you do use a framework, make sure you understand what it does underneath. Wrong assumptions about the internals are a common source of mistakes.

Building blocks: from a single call to a full agent

The augmented language model

Every agentic system starts with a language model enhanced with extra capabilities: it can formulate its own search queries, pick the right tool, and decide what information to keep. Tailor these capabilities to your own use case and give the model a clear, well-documented interface to work with. The Model Context Protocol is a useful standard here: it connects a model to a growing set of external tools through one simple client.

Workflow: prompt chaining

Prompt chaining breaks a task into a sequence of steps, where each call processes the output of the one before it. Between steps, you can add programmatic checks to confirm the process is still on track.

This pattern works well when a task splits cleanly into fixed sub-tasks and you're willing to trade a bit of latency for higher accuracy. Think of generating marketing copy and then translating it, or drafting an outline, checking it against a set of criteria, and only then writing the full document.

Workflow: routing

Routing classifies an incoming request and sends it to a specialized follow-up step. That keeps a prompt tuned for one type of question from quietly hurting performance on another type.

Think of splitting customer questions (general questions, refund requests, technical support) across separate processes and tools, or sending easy questions to a smaller, cheaper model and hard questions to a stronger one.

Workflow: parallelization

With parallelization, several calls work on a task at the same time, and the outputs get combined programmatically. It shows up in two forms: sectioning splits a task into independent sub-tasks that run in parallel, voting runs the same task multiple times to get a range of outputs.

Sectioning works well for guardrails, where one call answers a question while a separate call checks it for inappropriate content. Voting is useful for reviewing code for vulnerabilities with several independent passes, or judging content where you want multiple opinions weighed against each other.

Workflow: orchestrator-workers

In this pattern, a central call breaks the task down itself, hands the pieces to separate "worker" calls, and combines their results. The difference from parallelization: the sub-tasks aren't fixed in advance, the orchestrator decides them based on the specific input.

This pattern fits complex tasks where you can't predict the sub-tasks up front, such as a code change that touches several files, or a search task that needs to gather information from a range of sources.

Workflow: evaluator-optimizer

Here, one call generates a response while a second call reviews it and feeds back, in a loop. This works well once you have clear evaluation criteria and iterating demonstrably adds value: literary translation, where a first draft misses nuances a second pass catches, or a search task that needs several rounds before the picture is complete.

Agents

Agents show up in production as language models get better at understanding complex input, reasoning and planning, using tools reliably, and recovering from mistakes. An agent starts from an instruction or a conversation with the user, then plans and executes independently, coming back to the user for more information or a judgment call when needed.

During execution, the agent needs to ground itself in the environment at every step, for example through tool results or code execution, so it can judge its own progress. Build in checkpoints and stopping conditions, such as a maximum number of steps, to stay in control.

Use an agent for open-ended problems where you can't predict how many steps are needed and a fixed path won't work. In exchange, expect higher costs and the risk that small mistakes compound. Test extensively in a sandboxed environment and put the right guardrails in place before an agent runs loose on production data.

Combining and adapting these patterns

None of these building blocks are a blueprint. They are recognizable patterns you can combine and adapt to your own situation. The key to success, as with any use of a language model, is measuring performance and adjusting based on what you find. Only add complexity once it demonstrably pays off.

Summary: three principles

Success with language models isn't about the most sophisticated system. It's about the right system for your situation. Start with a simple prompt, optimize it based on solid evaluation, and only add an agentic system once simpler solutions fall short. Three principles help:

1. Keep your agent's design simple.
2. Make the agent's planning visible to the user.
3. Put as much care into the interface between agent and tools as you normally would into documentation and testing.

A framework helps you get started fast, but don't be afraid to strip away layers of abstraction as you move toward production. That's how you end up with agents that are not just capable, but reliable, maintainable, and trusted by the people who use them.

In practice: two applications

Two applications stand out because they meet the same conditions: a mix of conversation and action, clear success criteria, room for feedback, and meaningful human oversight.

Customer support

Customer support combines a familiar chat interface with extra capabilities through tools. That fits a more open-ended agent well: the conversation naturally follows a logical flow, tools can pull customer data, order history and knowledge base articles, actions such as refunds or ticket updates can be handled programmatically, and success is directly measurable through resolved questions.

Coding agents

Software development is a strong fit for agents: code solutions can be verified with automated tests, an agent can iterate based on test results, the problem space is well defined, and output quality is measurable objectively. Automated tests help confirm functionality, but human review is still needed to check that a solution also fits the broader requirements of the system.

Prompt engineering for your tools

Whatever agentic system you build, tools are usually a key part of it. Tool definitions deserve as much attention in your prompt as the rest of the instructions. The same action can often be specified in more than one format: you can describe a file edit as a diff, or by rewriting the whole file. Some formats are simply harder for a model to produce than others.

A few guidelines for choosing a tool format:

• Give the model enough room to think before it writes itself into a corner.
• Keep the format close to what the model has naturally seen a lot of in text.
• Avoid unnecessary overhead, such as tracking exact line numbers or escaping characters.

Think about the effort that goes into a good interface for people, and put the same effort into the interface between agent and computer. Put yourself in the model's shoes: is it obvious from the description and parameters how to use a tool? Include examples, edge cases and clear boundaries between tools. Adjust parameter names and descriptions until they're obvious, the way you'd write a docstring for a new team member. Test how the model actually uses your tools, and design mistakes out of the tool itself where you can.

Frequently asked questions

What's the difference between a workflow and an agent? +

A workflow directs a language model and tools through a predefined path. An agent decides for itself, step by step, which actions are needed and which tools to use. Workflows are more predictable, agents are more flexible.

When should you not build an agent? +

When a direct model call, possibly with some extra context, already solves the task well enough. An agent costs more time and money per task, so that investment needs to pay off in better results.

What risks come with using agents? +

An agent's autonomy brings higher costs and the risk that small mistakes compound into bigger problems. Test extensively in a sandboxed environment and build in stopping conditions and human checkpoints before an agent works on production data.

Do you need a framework to build an agent? +

Not necessarily. Many patterns fit in a few lines of code through a model's API directly. A framework can help you get started fast, as long as you understand what happens underneath.