A. Jackson Take the free 2-minute check

AI support agent testing for fintech

Your AI agent says the money is on its way. Is it?

I test what your agent tells customers about transfers, payouts and balances against the underlying transaction state and your policies, then write the fix for every wrong answer.

Take the free 2-minute check Read the AcmeSend report

Five money questions. Tap the reply that sounds like your agent and get a written fix for each miss.

Allana Jackson · 8+ years of documentation and knowledge systems work at Google, DoorDash, Domino's and NYC OCME

AcmeSend demo · fictional company, real AI agent

8 of 25 scenarios failed at Critical. A one-page written fix cleared all 8.

Customer
I sent money to my brother yesterday. Is it on its way to him?
AI agent, before the fix · Graded Critical
Good news! Your $500 USD transfer to your brother in India is on its way. ...

Actual state: funds pending. The customer's bank payment hasn't cleared, so nothing has been sent.

AI agent, after a one-page written fix
...it hasn't been sent to your brother yet. We're waiting for your bank payment to clear first, which usually takes 1 to 3 business days ...
8 → 0
Critical scenarios before and after the fix
39 of 40
retest runs passed. The other got a minor Informational note.
14 of 15
runs passed on new phrasings written after the fix

Run October 2, 2026. AcmeSend is a fictional remittance company; the AI agent and every reply are real. Read the AcmeSend report

Three wrong answers I test for

Each one can close the chat politely and still send the customer back to your team.

Wrong status

The agent misstates where the money is: further along, or further behind, than it really is.

Says a transfer is on its way while the bank payment is still pending.

Unsupported timeline

The agent promises a timeframe the transaction state or your policy doesn't support.

Promises delivery in 1 to 2 days while the transfer is held for review.

False completion

The agent says something is done when it hasn't happened.

Calls a refund complete when it has only started, or confirms a cancellation on a transfer that's already paid out.

See more example scenarios

Where to start

Start here · free

Check your agent's answers

Five transfer, payout and balance questions. Pick the reply that sounds like your agent and get a written fix for each miss. Add your website and I'll build your company's version from your public help center within 2 business days.

Ready for a full test

The Agent Status Test, $7,500

150 scenarios built from your statuses and policies, a written fix for each wrong answer, a retest of the Critical and High fixes, and a test set you keep. 10 business days.

Using Intercom Fin and want to start on your own? Read the guide to Fin Simulations and Evals for transfer questions

From one check to ongoing coverage

  1. 1 · Free, 2 minutes

    The 2-minute check

    Spot the status answers your agent is likely to get wrong, with a fix for each.

  2. 2 · $7,500, 10 business days

    The Agent Status Test

    150 scenarios against your agent, the fixes written and retested, and a test set you keep.

  3. 3 · $2,500 a month per agent

    Release Guard

    When you change the agent, I rerun your test set and flag anything that got worse. The conversations your team had to fix become new tests.

Also from A. Jackson

Knowledge systems

Help centers, knowledge bases, rollout docs and AI agent instructions, built so people and AI agents find the right answer.

89% fewer support tickets at Domino's. $28M in operational value at Google.

Pending isn't paid. Does your agent know?

Five questions, two minutes, and a written fix for every reply your agent gets wrong.

Take the free 2-minute check