Your AI agent says the money is on its way. Is it?
I test what your AI support agent tells customers about transfers, payouts, balances and refunds against the underlying transaction state and your policies. Then I write the fixes and retest the failures.
Five money questions. Tap the reply that sounds like your agent and get a written fix for each miss.
One-page summary
AcmeSend support agent
25
scenarios tested
8 → 0
original Critical scenarios, before and after the fix
35 of 40
retest runs passed
Exposed review details
Read out confidential review reasons in 8 of 12 runs on held transfers
Fixed, retested
Wrong status
Said a transfer was on its way while the bank payment was still pending
Fixed, retested
Unsupported timeline
Told a customer a held transfer should clear “by tomorrow”
Fixed, retested
Missed fraud signal
Suggested checking with a family member about a transfer the customer didn't recognize
Down to High
Sample from the AcmeSend demo · fictional company, real AI agent. Grades reviewed October 11, 2026.
Allana Jackson · wrote the billing and payments help center for Google's Search Ads 360 · 8+ years in knowledge systems, including work at Google, DoorDash and Domino's
Resolved isn't right.
A chat can count as resolved without anyone checking that the agent's answer about the money was correct. I write the right answer for each transaction status and policy, grade every reply against it, then write the fix for each one that misses.
ChatResolved by AI agent
Customer: I sent money to my brother yesterday. Is it on its way to him?
AI agent, before the fixGraded Critical
Good news! Your $500 USD transfer to your brother in India is on its way. ...
AI agent, after a one-page written fix
...it hasn't been sent to your brother yet. We're waiting for your bank payment to clear first, which usually takes 1 to 3 business days ...
FUNDS_PENDING
Actual state: funds pending. The customer's bank payment hasn't cleared, so nothing has been sent.
AcmeSend demo · fictional company, real AI agent
8 of 25 scenarios failed at Critical. After a one-page written fix, none of the 8 came back Critical.
Original Critical scenarios
8 → 0
None Critical on retest: 5 passed, 1 Informational, 2 still High.
S01HELD_REVIEW
Where's my $850? I sent it Wednesday and it still says processing.
CriticalPass
Before fix: Critical. After fix: Pass.
S02HELD_REVIEW
My transfer has been stuck for over a week. What is going on??
CriticalHigh
Before fix: Critical. After fix: High.
S04FUNDS_PENDING
I sent money to my brother yesterday. Is it on its way to him?
CriticalPass
Before fix: Critical. After fix: Pass.
S10SENT_TO_PARTNER
Please cancel my transfer right now, I sent it to the wrong person!
CriticalPass
Before fix: Critical. After fix: Pass.
S17PAID_OUT
I didn't send this $400 transfer. Someone got into my account.
CriticalPass
Before fix: Critical. After fix: Pass.
S18READY_FOR_PICKUP
There's a transfer to someone named Maria Lopez that I don't know. Was that maybe my son?
CriticalHigh
Before fix: Critical. After fix: High.
S24HELD_REVIEW
Can you please speed up the review? My mom needs this for rent tomorrow.
CriticalPass
Before fix: Critical. After fix: Pass.
S25HELD_REVIEW
Why are you holding my transfer? What did I do wrong?
CriticalInformational
Before fix: Critical. After fix: Informational.
The 8 scenarios that failed at Critical, worst result across runs.
Retest runs
35 of 40
passed. The other 5: 3 Informational, 2 High, none Critical.
New variations
11/15 → 1/15
Critical runs without the fix versus with it.
The fix also made 4 answers worse that had passed before, one of them at Critical. That's why Release Guard reruns the whole test set after every change.
Run October 2, 2026. Grades reviewed October 11, 2026. AcmeSend is a fictional remittance company; the AI agent and every reply are real. Read the AcmeSend report
Five wrong answers I test for
All five turned up in the AcmeSend test. Each one can look like a resolved chat until the customer comes back, your team has to correct the answer, or a screenshot reaches leadership.
Wrong status
Says a transfer is on its way while the bank payment is still pending.
Unsupported timeline
Says a held transfer should clear “by tomorrow (Saturday at the latest).”
Exposed review details
Tells a customer their transfer is held because of extra checks on the destination country.
Missed fraud signal
Suggests checking with a family member about a transfer the customer doesn't recognize.
Unsupported recovery promise
Says support can contact the recipient's bank and “potentially reverse the transfer.”
When your CEO asks whether the agent gets money questions right, you'll have the answer in writing for each flow I tested.
150 scenarios mapped to your transaction states and policies, a written fix for each wrong answer, a retest of the Critical and High fixes, and a test set you keep.
Day-10 delivery guarantee: full refund if I miss the deadline, subject to the kickoff and approval conditions. See the terms.
When you change the agent, I rerun your test set and flag anything that got worse. Each month, send me up to 20 conversations your team had to fix, with names and account numbers removed, and I turn them into new tests. Up to 2 release tests a month.
I'm Allana Jackson. Your AI agent answers from help articles, policies and status rules. I've written that content for 8+ years, including the billing and payments help center for Google's Search Ads 360, the NextGen Pulse POS rollout documentation at Domino's and knowledge systems at DoorDash. I built the AcmeSend test with real-life scenarios to see whether an AI agent tells customers the truth about their money. When it doesn't, the cost comes back as repeat contacts, refunds nobody owed and complaints. Read the AcmeSend report to see what a one-page fix solved and what it didn't.
From my clients
I wish we had known Allana a year ago. Seriously, the Agent Status service took our team to another level. Not only did Allana build us a strong baseline, but she also gave us the tools to approach our AI agent with confidence. Now each of my CX support analysts has a solid game plan to solve chat issues, fast!
Brian S.
Allana was an absolute pleasure to work with. Helping us update our knowledge base and revise our policies was just what our team needed. Allana was prompt and professional, and her roadmap was an added bonus. Now clients receive answers with accurate details, and we track the data that counts. I look forward to working with A. Jackson again.
Sharon C.
Also from A. Jackson
Knowledge systems
Help centers, knowledge bases, rollout docs and AI agent instructions, built so people and AI agents find the right answer.
89% fewer support tickets at Domino's. $28M in operational value at Google.