Skip to content

InsightHow do you know if an AI agent is actually working?

How do you know if an AI agent is actually working?

Surinder Thakur, AI Product Designer and Builder

Published · 2 min read

The short answer

Read 4 numbers every week. Tasks it finished without a person. Handovers it sent to a person. Mistakes caught after the fact. And cost per task. Tasks finished also shows use: an agent can score well and still sit unused. Set a pass mark for each number before launch, so the result is not argued later.

In 1 picture

Weekly agent report

  • Finished, checked
  • Handovers, checked
  • Mistakes, flagged
  • Cost, checked

Above the pass mark

The owner reviews this week

4 numbers, read every week.

A live agent is judged on tasks finished, handovers, mistakes and cost per task.

Why not measure accuracy alone?

Because accuracy hides 2 problems. An agent can be right and slow, or right and unused.

Business numbers catch both. Tasks finished shows use. Cost per task shows whether it pays.

What do handovers tell you?

Where the rules are thin. A rising handover rate means the agent meets cases nobody planned for.

Read the handed-over cases weekly. Most turn into 1 new rule, and the rate falls.

What does a pass mark look like?

A number with a direction, set before launch. In the Ordering agent design: a median of 5 minutes or less from first message to order.

And a stop rule. If wrong-part returns double against the sales reps, the pilot stops at once.

Who reads the numbers?

The owner of the work, every week, in 10 minutes. Not a data team once a quarter.

A weekly read catches drift early, while the fix is still small.

Surinder ThakurAI Product Designer and Builder between Dubai and India, for UAE, US and EU teams

Start with 30 minutes.

You talk to me, the one who leads your build. Tell me the tool, the users and what it should move.

Book a scope call1 slot open for Q4 2026
Or write to hey@surinder.design
years in UX and product design
19
YoY revenue at Puffy
+67%

Design manager for 2 years at Puffy, a $1B brand.

What happens after you book

  1. You fill a short intake formSo the call starts with your problem, not with questions.
  2. We talk for 30 minutesYour tool, your users and what it should move.
  3. You get your planIn writing, within 24 hours. No pitch.

3 months design to market

“He consistently delivers on time often ahead of schedule and his professionalism is unmatched.”
Carlos AriasFounder, Medellin.co
Surinder T.