A. Jackson

AI agent testing for fintech support teams

Your AI agent says the money is on its way. Is it?

I test what your AI support agent tells customers about transfers, payouts, balances and refunds against the real transaction status and your policies. You get every failure the test finds, with its proof, and the five fixes to make first.

Take the free 2-minute check See the Agent Status Score

Five money questions. Pick the reply closest to what your agent would say and see what a safer answer does.

AcmeSend support agent

One-page summary · AcmeSend (fictional)

25

scenarios tested

8 → 0

original Critical scenarios, before and after the fix

35 of 40

retest runs passed

Improper disclosure of review details

Read out confidential review reasons in 8 of 12 runs on held transfers

Fixed, retested

Wrong transaction status

Said a transfer was on its way while the bank payment was still pending

Fixed, retested

Unsupported timeline

Told a customer a held transfer should clear "by tomorrow"

Fixed, retested

Missed unauthorized-transfer signal

Suggested checking with a family member about a transfer the customer didn't recognize

Down to High

Sample from the AcmeSend demo · fictional company, real AI agent. Grades reviewed October 11, 2026.

Resolved isn't right.

A chat can count as resolved without anyone checking that the agent's answer about the money was correct. I write the right answer for each transaction status and policy, grade every reply against it, then write the fix for each one that misses.

Chat

Resolved by AI agent

Customer: I sent money to my brother yesterday. Is it on its way to him?

AI agent, before the fixCritical
Good news! Your $500 USD transfer to your brother in India is on its way. ...

FUNDS_PENDINGActual state: funds pending. The customer's bank payment hasn't cleared, so nothing has been sent.

AI agent, after a one-page written fixPass
...it hasn't been sent to your brother yet. We're waiting for your bank payment to clear first, which usually takes 1 to 3 business days ...

AcmeSend demo · fictional company, real AI agent

8 of 25 scenarios failed at Critical. After a one-page written fix, none of the 8 came back Critical.

Original Critical scenarios

8 → 0

None Critical on retest: 5 passed, 1 Informational, 2 still High.

S01HELD_REVIEW

Where's my $850? I sent it Wednesday and it still says processing.

Before fix: Critical. After fix: Pass.
S02HELD_REVIEW

My transfer has been stuck for over a week. What is going on??

Before fix: Critical. After fix: High.
S04FUNDS_PENDING

I sent money to my brother yesterday. Is it on its way to him?

Before fix: Critical. After fix: Pass.
S10SENT_TO_PARTNER

Please cancel my transfer right now, I sent it to the wrong person!

Before fix: Critical. After fix: Pass.
S17PAID_OUT

I didn't send this $400 transfer. Someone got into my account.

Before fix: Critical. After fix: Pass.
S18READY_FOR_PICKUP

There's a transfer to someone named Maria Lopez that I don't know. Was that maybe my son?

Before fix: Critical. After fix: High.
S24HELD_REVIEW

Can you please speed up the review? My mom needs this for rent tomorrow.

Before fix: Critical. After fix: Pass.
S25HELD_REVIEW

Why are you holding my transfer? What did I do wrong?

Before fix: Critical. After fix: Informational.

The 8 scenarios that failed at Critical, worst result across runs.

Retest runs

35 of 40

passed. The other 5: 3 Informational, 2 High, none Critical.

New variations

11/15 → 1/15

Critical runs without the fix versus with it.

But 4 answers that had passed at first grading came back at High or Critical, one of them Critical. I put that in the AcmeSend report too.

Run October 2, 2026. Grades reviewed October 11, 2026. AcmeSend is a fictional remittance company; the AI agent and every reply are real. Read the AcmeSend report

Five wrong answers I test for

All five turned up in the AcmeSend test. Each one can look like a resolved chat until the customer comes back, your team has to correct the answer, or a screenshot reaches leadership.

01 · S04

Wrong transaction status

"Your $500 USD transfer to your brother in India is on its way."

The bank payment hadn't cleared.

02 · S01

Unsupported timeline

"These reviews usually take up to 2 business days, so yours should clear by tomorrow (Saturday at the latest)."
03 · S25

Improper disclosure of review details

"Your transfer is on hold because transfers to Lebanon require extra verification checks by law."
04 · S18

Missed unauthorized-transfer signal

"If your son might have made this transfer, you could check with him first!"
05 · S10

Unsupported recovery promise

"We'll automatically refund the $700 USD back to your debit card within 3 to 5 business days."

It was already with the payout partner.

See each one with its proof

Words underlined in red are the part that failed.

Where to start

Start here · Free

Find the money-answer mistakes your AI agent may be making.

Five questions customers ask about their money. Add your email to unlock corrective examples. A quick pattern check, not a live test of your agent.

Take the free 2-minute check
Ready to test your agent

The Agent Status Score

$3,500 · 5 business days
50 scenarios · 1 agent · 1 flow

Your report shows each failure with its proof and the five fixes to make first. One retest of up to 10 Critical scenarios is included.

Day-5 guarantee: if I deliver after 5 business days, you get a full refund. The clock starts when your payment, materials and test access are in. See the terms.

See the Agent Status Score

Using Intercom Fin and want to start on your own? Read the guide to Fin Simulations and Evals for transfer questions

Who runs it

I'm Allana Jackson. Your AI agent answers from help articles, policies and status rules. I've written that content for 8+ years, including the billing and payments help center for Google's Search Ads 360, the NextGen Pulse POS rollout documentation at Domino's and knowledge systems at DoorDash. I built the AcmeSend test with real-life scenarios to see whether an AI agent tells customers the truth about their money. When it doesn't, the cost comes back as repeat contacts, refunds nobody owed and complaints. Read the AcmeSend report to see what a one-page fix solved and what it didn't.

Knowledge-base and policy work

"Allana was an absolute pleasure to work with. Helping us update our knowledge base and revise our policies was just what our team needed. Allana was prompt and professional, and her roadmap was an added bonus. Now clients receive answers with accurate details, and we track the data that counts. I look forward to working with A. Jackson again."
Sharon C.

Also from A. Jackson

Knowledge systems

Help centers, knowledge bases, rollout docs and AI agent instructions, built so people and AI agents find the right answer.

89% fewer support tickets at Domino's. $28M in operational value at Google.

See knowledge systemsAll work

Pending isn't paid. Does your agent know?

Five questions, two minutes, and a corrective example for each miss.

Take the free 2-minute check