Research, · Allana Jackson
How to write Fin Simulations for transfer status questions
For a Fin Procedure that answers transfer questions, build a separate simulation for each combination of transfer status and what the customer wants (a status update, a cancellation, a refund). Set the status in the simulation, then add two Fin reply criteria: what Fin should say, and the specific wrong claim it must not make. Add an outcome criterion for whether Fin should finish or hand off, and run the risky ones more than once.
The rest of this page shows how, with five worked examples from a controlled test.
Scope: Intercom says Simulations are exclusive to Procedures (Simulations vs. Batch tests vs. Previews). If your transfer questions are answered from help articles without a Procedure, Fin's Batch tests are the tool that reaches them. The same method applies: know the status, write down the claim Fin must not make, and vary the phrasing.
Why the status matters more than the question
"Is my transfer on its way?" has a different right answer for every status. If the customer's bank payment hasn't cleared, the answer is no. If the transfer is with the payout partner, the answer is yes, and it may no longer be possible to cancel it in the app. If it's paid out, Fin can give the payout date and account details, but it shouldn't say the recipient has the money unless your system shows that.
An AI agent that answers from your help center gives the general answer. In our AcmeSend test (AcmeSend is a fictional remittance company built for testing; the AI agent is real), the agent was asked that exact question about a transfer whose bank payment hadn't cleared. On all three runs it said:
"Good news! Your $500 USD transfer to your brother in India is on its way."
Nothing had been sent. A test that only checks whether the reply is relevant and polite would pass it. To catch it, the simulation needs the real status in its customer data and the claim Fin must not make in its criteria.
What a Fin Simulation can check
Per Intercom's help article on running simulations for Fin Procedures, you set up a manual simulation by describing the simulated customer and adding success criteria. A run passes only if it meets every criterion.
The simulated customer
| Field | Use it for |
|---|---|
| Simulate as | A real user or brand from your workspace |
| Customer's opening message | The question, in the customer's own words |
| Additional details | What the customer will say if Fin asks |
| Channel | Messenger or Email. Fin behaves differently on each |
| Simulation time | A fixed date and time, for cutoff and weekend logic |
| Customer data available to Fin | The attributes and data connector values Fin sees during the test. Only fields the Procedure references appear here. This is where you set the transfer's status |
The success criteria
| Type | What it checks |
|---|---|
| Fin reply | What Fin should or should not say |
| Attributes | Whether an attribute was set, or equals a value |
| Data connector | Whether a connector was triggered, not triggered, or triggered a set number of times |
| Instruction outcome | Whether the conversation finished, was handed to a teammate, or switched Procedures |
For transfer questions, Fin reply criteria do most of the work, because the failures are in what Fin tells the customer.
Set the status yourself. Per Intercom's comparison of its testing tools, Simulations don't call live external APIs, and only Preview can reach live integrations. So Fin only sees the transfer status you put in the Customer data available to Fin section. Make sure your Procedure references the status field, then check that the values you enter describe the state you mean to test. If the field is empty, you're testing a lookup that returned nothing, which is a different scenario.
Five worked examples
These come from the AcmeSend test, a fictional remittance company with a real AI agent. Each one is a status where the agent failed before the fix. Swap in your own status names and policy.
1. Transfer status: bank payment not cleared
| Opening message | "I sent money to my brother yesterday. Is it on its way to him?" |
|---|---|
| Set the status | Funds pending: bank payment started yesterday, transfer not sent |
| Fin reply: should | Says the transfer hasn't been sent yet because the bank payment is still clearing |
| Fin reply: should not | Says the transfer is on its way, has been sent, or will arrive on a specific day |
| Outcome | Finished |
| AcmeSend result | Failed at Critical on 3 of 3 runs |
2. Delivered, not received
| Opening message | "My mom says she never got the money. Can you check?" |
|---|---|
| Set the status | Paid out yesterday at 4:42 PM to the account ending 4471 |
| Fin reply: should | Confirms it was paid out and when, to the account ending 4471, and that the recipient's bank can take up to 1 business day to post it |
| Fin reply: should not | Says it hasn't been sent, is still processing, or can be canceled, or names a day it will arrive |
| Outcome | Finished |
| AcmeSend result | High. The agent confirmed the payout, then added the help center's general delivery time and told the customer she "should see it by Monday or Tuesday" |
3. Cancellation: already with the payout partner
| Opening message | "Please cancel my transfer right now, I sent it to the wrong person!" |
|---|---|
| Set the status | Sent to the payout partner |
| Fin reply: should | Says it can't be canceled in the app because it's already with the payout partner, and offers a person who can ask for a recall |
| Fin reply: should not | Tells the customer to cancel it in the app, or promises the money back |
| Outcome | Handed to a teammate |
| AcmeSend result | Failed at Critical on 2 of 3 runs. One reply: "you can still cancel it! Open the AcmeSend app, go to Activity, tap transfer AS-21075, and select Cancel." |
4. Refund: completed but not showing
| Opening message | "You said my refund was done but I don't see it in my account." |
|---|---|
| Set the status | Refunded: settled on September 29 to the debit card ending 8830 |
| Fin reply: should | Confirms the refund was completed on September 29 to the card ending 8830, and that the bank can take 1 to 2 business days to show it |
| Fin reply: should not | Says the refund hasn't happened, or gives the timeline for a refund still in progress |
| Outcome | Finished |
| AcmeSend result | High. The agent gave the 3 to 5 business day timeline for card refunds in progress, for a refund that had already settled |
5. Fraud: a transfer the customer doesn't recognize
| Opening message | "There's a transfer to someone named Maria Lopez that I don't know. Was that maybe my son?" |
|---|---|
| Set the status | Ready for cash pickup, not yet collected |
| Fin reply: should | Treats it as possibly unauthorized and connects the customer to the fraud team in this conversation |
| Fin reply: should not | Suggests checking with family first, or says the transfer is probably legitimate |
| Outcome | Handed to a teammate |
| AcmeSend result | Failed at Critical on 2 of 3 runs. One reply: "If your son might have made this transfer, you could check with him first!" |
How to write the Fin reply criteria
Check the fact, not the wording. "Says the transfer hasn't been sent yet" works. "Says 'Your transfer hasn't been sent'" fails a correct answer phrased differently. Intercom's own FAQ warns that overly rigid criteria can fail a conversation that was handled correctly.
Write the should-not as the exact wrong claim. "Doesn't give wrong information" can't be judged. "Says the transfer is on its way" can. Use the mistake that would cost you a repeat contact or a refund.
Keep one fact per criterion. When a run fails, you see which fact broke instead of rereading the whole transcript.
Name the details the customer needs. If the status has a date, an amount or an account ending, put it in the should criterion. A reply that gets the status right but leaves out the date still sends the customer back to you.
How many runs
Run each Critical scenario at least 3 times. The same question can get a different answer on each run: in the AcmeSend test, the cancellation scenario passed once and failed twice.
Then add new phrasings of the same question. In the AcmeSend test, we wrote 5 new customer phrasings after the fix, to check it wasn't tuned to the original wording, and ran them both ways. Without the fix, 4 of the 5 failed at Critical. With it, 14 of 15 runs passed. A suite with one phrasing per status would have missed those failures.
Fin Simulations have a monthly run allowance per workspace, based on the previous month's conversation volume. The limits run from 250 to 12,500 runs a month; a workspace with 1,000 to 15,000 conversations gets 1,000. Thirty scenarios run 3 times each uses 90 runs.
When to rerun
Rerun the whole set after any change to a help article, a Procedure, a status name in your app, or a policy such as cancellation windows or refund times. Fin saves every simulation in the Procedure, and Run all reruns the set. A fix to one article can break an answer you corrected last month, and the saved set is what catches it.
What Simulations won't catch
Simulations test what Fin does with the status and content you give it. They won't tell you that two of your help articles describe the same policy differently, or that a status label in your app means something different from the code Fin reads. Read those side by side when you write the expected answers, since that's where many wrong answers start.
Questions
Can Simulations check what Fin says, or only what it does?
Both, within a Procedure. The Fin reply criterion checks what Fin should or shouldn't say. Attribute, data connector and outcome criteria check what it does.
How many simulations do we need for transfer questions?
Start with the statuses your own system exposes, then take each question type (status, delivered-not-received, cancellation, refund, fraud) through the statuses where its right answer changes. AcmeSend had 10 status codes, and its 25 scenarios covered the riskiest combinations, before adding new phrasings.
Do simulation runs cost extra?
Per Intercom's article, Simulations come with Procedures and aren't billed separately. Each workspace gets a monthly run allowance based on its conversation volume.