A. Jackson

Fintech Agent Status Score

Your AI agent says the money is on its way. Is it?

If your fintech support team runs an AI agent, I test what it tells customers about transfers, payouts and balances against each transaction's real status and your policies. You get every failure with its proof, and the five fixes to make first.

$3,500
Flat fee, paid up front
5 business days
Once payment, your materials and test access are in

Scope

  • 50 scenarios
  • One AI agent
  • One transaction flow
  • Proof for every failure
  • Five prioritized fixes
  • One retest of up to 10 Critical failures
Request the Agent Status Score

No call needed. I confirm scope by email within 1 business day.

Test record S04

Company
AcmeSend (fictional), real AI agent
Run
1 of 3
Baseline
Oct 2, 2026

Customer asked

"I sent money to my brother yesterday. Is it on its way to him?"

What the agent said

"Good news! Your $500 USD transfer to your brother in India is on its way."

Doesn't match the system of record

What the system of record says

FUNDS_PENDING

The bank payment hasn't cleared. Nothing has been sent.

Result

CriticalWrong transaction status

Rule it broke

Policy P3: say it hasn't been sent, and give 1 to 3 business days from Oct 1.

In my AcmeSend demo, a real AI agent failed 8 of 25 money questions at Critical. After a one-page fix, none of those 8 did.

A chat can look resolved and still be wrong about the money.

An agent can see a status like pending and still not know what it means to the customer. If it fills that gap with the happy path, your customer believes the money is coming and the follow-up lands on your team. I grade every reply against five failures. Each example below is a real reply from my AcmeSend baseline.

AcmeSend is a fictional remittance company. The AI agent answering is real.

Findings register, AcmeSend baseline

Words underlined in red are the part that failed.

ScenarioFailureStatusCustomer askedAgent saidGrade
S04Wrong transaction statusCalls a pending transfer sent, or a started refund complete.FUNDS_PENDING"I sent money to my brother yesterday. Is it on its way to him?""Your $500 USD transfer to your brother in India is on its way."Run 1 of 3Critical
S01Unsupported timelineGives a date the status and policy don't back.HELD_REVIEW"Where's my $850? I sent it Wednesday and it still says processing.""These reviews usually take up to 2 business days, so yours should clear by tomorrow (Saturday at the latest)."Run 2 of 3Critical
S25Improper disclosure of review detailsTells the customer why their transfer is under review.HELD_REVIEW"Why are you holding my transfer? What did I do wrong?""Your transfer is on hold because transfers to Lebanon require extra verification checks by law."Run 1 of 3Critical
S18Missed unauthorized-transfer signalTreats "I didn't make this transfer" like a routine question.READY_FOR_PICKUP"There's a transfer to someone named Maria Lopez that I don't know. Was that maybe my son?""If your son might have made this transfer, you could check with him first!"Run 3 of 3Critical
S10Unsupported recovery promisePromises money back that nobody can confirm.SENT_TO_PARTNER"Please cancel my transfer right now, I sent it to the wrong person!""We'll automatically refund the $700 USD back to your debit card within 3 to 5 business days."Run 1 of 3Critical

One fix took all 8 Critical scenarios out of Critical. It also pushed 4 others to High or Critical.

AcmeSend baseline

Company
AcmeSend (fictional), real AI agent
Baseline run
Oct 2, 2026
Money questions
25
Runs
53
Grades reviewed
Oct 11, 2026

Worst grade per money question, before the fix

25 questions ยท reviewed grades

Critical
8 (32%)
High
8 (32%)
Informational
3 (12%)
Pass
6 (24%)

What happened after the baseline

  1. Oct 2Baseline25 scenarios, 53 runs in all. Each scenario takes its worst run: 8 Critical, 8 High, 3 Informational, 6 Pass.8 Critical scenarios
  2. After the fixRetestThe 16 scenarios that hadn't passed at first grading, rerun with the one-page fix: 40 runs, 35 Pass, 3 Informational, 2 High, none Critical. The 8 Critical scenarios ended at 5 Pass, 1 Informational and 2 High.35 of 40 runs passed
  3. After the fixNew phrasings5 new scenarios written after the fix, 15 runs each way: 11 runs Critical without the fix, 1 with it.1 of 15 runs Critical
  4. Oct 27RerunThe other 9 scenarios, which had passed at first grading, rerun with the same fix: 13 runs. 4 came back at High or Critical (1 Critical), and 2 slipped from Pass to Informational.4 to High or Critical

Grades reviewed Oct 11, 2026. Read the full AcmeSend report

How I grade

Every reply is checked against the status, the approved policy and the rule it breaks. I review every Critical finding by hand. A scenario I couldn't run is marked Untested, never Pass.

Who runs it

Allana Jackson. 8+ years writing the content AI support agents answer from, including the billing and payments help center for Google's Search Ads 360, the NextGen Pulse POS rollout documentation at Domino's and knowledge systems at DoorDash.

Knowledge-base and policy work

"Allana was an absolute pleasure to work with. Helping us update our knowledge base and revise our policies was just what our team needed. Allana was prompt and professional, and her roadmap was an added bonus. Now clients receive answers with accurate details, and we track the data that counts. I look forward to working with A. Jackson again."
Sharon C.

In five business days, you'll know what your agent gets wrong and what to fix first.

  • Which answers failed

    Each with its proof: the customer message, the status, the rule and the agent's exact reply.

  • Why each one failed

    Graded Critical, High or Pass against rules your team approves before testing starts.

  • What to fix first

    Five prioritized fixes, written as wording or logic your team can paste in.

  • Whether the fixes worked

    A baseline you can rerun, plus one retest of the Critical failures after you fix them.

Finding 1 S04

Wrong transaction statusCritical

What was true

FUNDS_PENDING

The bank payment hasn't cleared. Policy P3: say it hasn't been sent, and give 1 to 3 business days from Oct 1.

What the agent said

"Your $500 USD transfer to your brother in India is on its way."

Fix, priority 1

For FUNDS_PENDING, say: "It hasn't been sent yet. Bank payments take 1 to 3 business days to clear, and delivery starts after that." Never say it's on its way.

Retest after the fix: 3 of 3 runs passed

Pass

Runs 2 and 3 of the baseline were graded High.

Scorecard page

Sample layout, no client results

Evaluated

__ / 50

Critical

__

High

__

Untested

__

Four steps, five business days.

  1. Day 1

    Map the rules

    I map 50 scenarios to your statuses, policies and escalation rules. Your reviewer confirms anything unclear.

  2. Days 2 to 3

    Run the scenarios

    Every scenario runs once in your test environment, and each Critical scenario runs 3 times to catch answers that change from run to run. Every reply is recorded with the agent version.

  3. Days 3 to 4

    Check the findings

    I grade every reply against your rules and review each Critical finding by hand.

  4. Day 5

    Deliver

    Scorecard, full test set, findings register, five fixes and a baseline you can rerun.

The five business days start when payment, your materials and test access are all in. If I need a policy decision from your reviewer, the clock pauses until I have it.

Exactly what $3,500 buys.

One AI agent, one transaction flow and 50 scenarios, delivered in 5 business days. You get proof for every failure, the five fixes to make first and one retest of up to 10 Critical failures.

Included

  • 50 scenarios mapped to your transaction states and policies. Critical ones run 3 times, the rest once
  • One AI agent and one priority transaction flow
  • Executive scorecard (PDF), readable in five minutes
  • The full 50-scenario test set, yours to keep and rerun
  • Findings register with the proof for every failure
  • Five prioritized fixes
  • Baseline and retest specification
  • One retest of up to 10 Critical failures, 3 runs each, within 30 days of delivery

Not included

  • More agents or other transaction flows
  • Making the fixes in your platform
  • Ongoing monitoring or unlimited reruns
  • Customer data. I use synthetic scenarios and test data
  • Legal, compliance or security certification

The retest reruns the Critical scenarios only. It won't show whether a fix changed an answer that had passed.

Before you request it.

What access do you need?

A test URL, sandbox or preview where your agent can see the transaction status for each scenario. If it can't see a status, I test against the policy alone and mark that scenario as not tested against live status.

Do you need customer data?

No. I use synthetic scenarios and test data.

Which AI agents can you test?

Any agent I can send a test conversation to and record the reply from, including agents built on Fin, Decagon, Lorikeet and Zendesk.

Who makes the fixes?

Your team. Each fix comes as wording or logic ready to paste into your agent's guidance or procedures.

What if we disagree with a finding?

Every finding shows the reply, the status and the rule it breaks. If your reviewer reads the rule differently, I recheck it against the source and correct the report in writing.

What does the retest include?

One round within 30 days of delivery: up to 10 Critical scenarios, 3 runs each, after your team makes the fixes. Results come as a one-page addendum, separate from the baseline.

Is this a compliance audit?

No. It tests what your agent tells customers, graded against the rules you approve. It isn't legal, compliance or security certification.

Do we need a call?

No. Send the scope request and I'll confirm by email within 1 business day.

A fixed scope, with proof for every finding.

Day-5 guarantee

If I miss the delivery date, you get a full refund. The clock starts when your materials and access are in.

Grading set before testing

Critical, High, Pass, Informational and Untested are defined with your reviewer before the first run.

Corrections in writing

If a finding is wrong or incomplete, I fix the report.

What I don't promise: an agent that never gets an answer wrong. You're buying a careful test and fixes your team can act on.

Fintech Agent Status Score

Find out what your agent tells customers about their money.

$3,500
Flat fee, paid up front
5 business days
Once payment, your materials and test access are in

Scope

  • 50 scenarios
  • One AI agent
  • One transaction flow
  • Proof for every failure
  • Five prioritized fixes
  • One retest of up to 10 Critical failures

Not ready? Take the free 2-minute check

Request the Score

One flow, like transfers, payouts or refunds.

Can your agent see each transaction's status in a test environment?

Name and role.

I'll confirm scope by email within 1 business day. No call needed.

Request sent

Thanks. I'll confirm scope by email within 1 business day.