# How do you test an AI agent before customers use it?

In 3 stages. Replay past cases your team already decided and compare answers. Run it in shadow mode on live work, where it suggests but a person acts. Then let it act alone on a small slice, with pass marks agreed in advance. If a pass mark breaks, it stops at once.

By Surinder Thakur, AI Product Designer and Builder. Published 17 Nov 2026. Topic: [Trusting AI with money](https://www.surinder.design/work?type=trusting-ai-with-money).

## Why start with a replay?

Because past cases have known answers. You see where the agent agrees with your team, and where it would have failed. It costs little.

In the Ordering agent design, it orders alone only after matching sales reps on 90% of past orders.

## What does shadow mode add?

Real, messy, current work. Cases nobody saved in the archive, and timing your replay cannot show.

The agent suggests and a person still acts. Every disagreement becomes a rule fix before launch.

## What are pass marks?

The numbers that decide whether the agent keeps its job, agreed before it starts. Speed, mistakes, and the share of work it handles.

The Ordering agent's pilot reads them after 300 agent orders. If wrong-part returns double against sales reps, the pilot stops at once.

## Why start with a small slice?

Because a mistake on 1 category costs little. A mistake on everything costs the project.

Each slice that passes earns the next one. The agent grows by evidence, not by a launch date.

## In 1 picture

3 stages test an agent before customers rely on it.

1. Replay: Past cases, known answers
2. Shadow mode: It suggests, a person acts
3. Acts alone: On 1 small slice first
4. Pass marks met: Agreed in advance

The agent grows by evidence, not by a launch date.

## Next questions

- [How do you know if an AI agent is actually working?](https://www.surinder.design/work/how-to-know-if-an-ai-agent-is-working): Read 4 numbers every week. Tasks it finished without a person.
- [What happens when an AI agent fails halfway through a task?](https://www.surinder.design/work/when-an-ai-agent-fails-halfway): A well-built agent stops, keeps what it finished, and recovers in 1 of 3 ways.
- [How do you get an AI agent to say "I do not know"?](https://www.surinder.design/work/how-to-get-an-ai-agent-to-say-i-do-not-know): Give it 3 things. A narrow set of sources it must answer from.
- [How long does it take to build a custom AI agent?](https://www.surinder.design/work/how-long-to-build-a-custom-ai-agent): The Blueprint takes 2 weeks. The build time depends on scope, so the proposal fixes it in writing before you commit.

## Questions that lead here

- [What data do I need before building an AI agent?](https://www.surinder.design/work/what-data-do-you-need-to-build-an-ai-agent): 3 kinds. The live records the agent reads, like stock, prices or open invoices.

Page: https://www.surinder.design/work/how-to-test-an-ai-agent-before-launch
