Listen to this article
Executive Summary
An AI agent is different from a normal chatbot. It does not just answer a question once. It takes a series of actions (checking a database, filling a form, calling another system) and each action can go wrong in its own way. Testing it once, before launch, is not enough. Singapore businesses need a testing habit, not a testing event.
Core conclusions
- Test an agent in two stages: before it goes live, against a fixed set of practice cases, and after it goes live, by sampling real conversations every week.
- Score every agent on five things: does it finish the job, does it use its tools correctly, does it stick to the facts, does it stay safe, and does it know when to call a human.
- Singapore already has real standards to test against (IMDA’s AI Verify toolkit, MAS’s FEAT principles for finance, and PDPA rules on personal data) so build your checklist around them, not from scratch.
The checklist, in ten slides
Save it, share it, or hand it to whoever is about to switch on an AI agent.
Most Singapore companies buying or building an AI agent right now ask one question: “Does it work?” That is the wrong question. The right question is: “How do we know, every single week, that it still works?” An agent that passed its demo in March can quietly get worse in July, after a small update to its instructions or a change to the underlying model. Without a testing system, nobody notices until a customer complains.
Two kinds of checks, run at two different times
Think of agent testing like checking a new hire.
Before the agent starts work (offline testing). You give it a stack of practice cases with known right answers, the same way you might give a new staff member sample tickets to work through. You check the results against a golden set of correct outcomes, marked by a real expert. This happens before every launch and before every big update.
While the agent is working (online testing). You do not check every single conversation the agent has. That would cost too much and take too long. Instead, you pull a random sample of real conversations each week and score them the same way. This catches problems that only show up with real customers, real accents, real typos, and real edge cases that no test writer thought of.
A simple way to picture the whole pipeline: every agent response first goes through fast, cheap, automatic checks: does the output match the expected format, did it call the right tool, was it fast enough. Anything that passes those moves on to a smaller, slower check where a second AI model reads the full transcript and judges the quality, the same way a supervisor reviews a colleague’s work. Only agents that clear both checks get promoted to production.
I walk through this continuous testing pipeline in the video below:
Five things every test must check
Not every mistake matters the same amount. A slow answer is annoying. A leaked customer record is a PDPA problem. Score each agent against five separate dimensions, not one overall “good or bad” grade.
| What we check | Simple test | Pass mark |
|---|---|---|
| 1. Does it finish the job? | Compare the agent’s final result against a known correct outcome, and count how many steps it took to get there. | Gets it right at least 95 times out of 100, using close to the fewest steps possible. |
| 2. Does it use its tools correctly? | Check that every tool call has the right format and the right information, with no repeated or wasted calls. | Every tool call must be correctly formatted; wasted repeat calls under 2%. |
| 3. Does it stick to the facts? | Compare every claim the agent makes against the source document it was given. | Matches its sources at least 92 times out of 100; makes something up less than 1% of the time. |
| 4. Does it stay safe? | Try to trick it with fake instructions, requests for private data, or questions outside its job. | Zero serious safety mistakes allowed. Not one. |
| 5. Does it know when to ask a human? | Check whether it correctly hands off tricky or risky cases to a person, and does not hand off easy ones needlessly. | Catches at least 99 out of 100 cases that truly need a human; is right about needing one at least 9 times out of 10. |
Dimension 4 has zero tolerance because a single safety failure (a leaked NRIC number, a wrong bank transfer, a promise the company never made) can cost far more than every passed test combined.
Building a test set: easy cases, hard cases, and attack cases
A good test set is not just a pile of easy questions. Split it three ways:
- Everyday cases, about 60% of the set. Real requests copied from how the agent will actually be used, checked and approved by someone who knows the job.
- Tricky cases, about 25% of the set. A missing piece of information, an expired password, a slow system, a question the agent was never told how to answer.
- Attack cases, about 15% of the set. Someone trying to trick the agent into ignoring its rules, revealing private data, or doing something it should refuse.
Skip that last group and you will only find out your agent can be tricked after a customer, or a curious employee, finds it for you.
How an “AI judge” scores your agent
For anything too fuzzy for a simple pass or fail (did the agent actually solve the customer’s problem, not just technically answer it) companies use a second AI model as a judge. The judge reads the full conversation, writes out its reasoning step by step, and then gives a score from 1 to 5, with 4 or 5 counting as a pass. The judge is told to think first and score second, because a judge that jumps straight to a number is far less reliable.
Two settings matter here. First, the judge’s “creativity” is turned all the way down, so it gives the same score to the same conversation every time. You are testing the agent, not testing how random the judge feels that day. Second, every score must come with a written reason and a quote from the transcript backing it up, so a human can check the judge’s work rather than trusting a bare number.
Run the tests every time something changes, not just once
Set up your testing so it runs automatically whenever someone changes:
- The instructions given to the agent.
- The AI model itself, or its version.
- The tools or systems the agent can call.
- The documents or database the agent searches for facts.
If a new version does noticeably worse on the “finish the job” score, or if a single safety test fails, the update should be blocked automatically before it reaches customers, the same way a bug would block a normal software release.
Three things that can go wrong with testing itself
- The AI judge can be unfair. It can favour longer answers, or answers that sound confident, over answers that are actually correct. Check the judge’s scores against a real human’s scores every few months. If the judge agrees with the human at least 8 times out of 10, it is trustworthy enough to keep using.
- Full testing costs money and takes time. Running the expensive AI-judge check on every single conversation adds up fast. Run the cheap, automatic checks on everything, and save the AI judge for a smaller sample before launch.
- The same question can get two different answers. AI models are not perfectly consistent. For the cases that matter most, ask the same question three times and only trust a result the majority of the three agree on.
What this means for businesses in Singapore
Singapore does not need to invent this framework from nothing. IMDA’s AI Verify toolkit already gives companies a starting checklist for testing AI systems responsibly. Banks and other financial firms already work under MAS’s FEAT principles (fairness, ethics, accountability, and transparency) which line up closely with dimensions 3 and 4 above. And because PDPA makes any leak of personal data a real legal problem, not just a bad review, the safety dimension deserves extra attention here: run PDPA-specific test cases, not generic ones borrowed from a US or European vendor’s default test set.
Smaller and growing companies without a large testing team should not try to build all of this at once. Start with the automatic checks (they are cheap, fast, and catch most obvious mistakes) then add the AI-judge layer once the agent is handling anything that touches money, health records, or personal data.
Sources referenced: IMDA / AI Verify Foundation testing framework; MAS FEAT Principles for AI in financial services; Singapore’s Personal Data Protection Act (PDPA); ISO/IEC 42001, AI management systems.
Free tool
TRACE Agent Evaluation
Score a candidate task against five criteria before you commit to an architecture, a quick way to see if a task is even ready for an autonomous agent.
Free tool
Expert AI Agent Builder
Set the guardrails for an agent team, then try to trick it yourself: the same attack-case testing described above, in a hands-on format.
The AI Governance & ROI Executive Programme walks engineering and risk teams through building this exact testing system, from the golden dataset to the promotion gate, before any agent gets write access to a real system. Details are on the workshops page.
Was this useful?
