Your sales head is convinced the agent should ask about budget in the first ten seconds. Your best telecaller says that puts people off. You could change the script next Monday and see what happens. But next Monday's list came from a different ad, in a different week, and if the numbers move, you won't know whether the script did it.
A/B testing settles this by splitting one list between two agents. Each contact always gets the same side, retries included. It splits contacts and not calls, so nobody hears both versions.
Test one thing
Make a copy of your agent and change one thing. If the B agent has a new opening, a new voice and a new offer, and it wins, you still won't know why. Good first tests include:
- When to ask the qualifying question: early or after the pitch.
- How the call opens: a question, or a reason for calling.
- A different voice, or a different tier.
- An agent against a team, on plans that include teams.
- Asking for a visit straight away, or offering to send details first.
Setting it up
In a campaign, choose Test against: another agent or team, and its share of the contacts. It starts at half. Keep it at half unless you are nervous about B, in which case give it a smaller share and accept that the test takes longer.
Each side waits alone. A paused B, or one outside its calling hours, holds back only its own share, and A carries on. So if your two agents have different calling hours, you are also testing the hours, and that muddies the result. Keep them the same.
Both sides ring real customers from the moment the campaign runs. Before you press Start, make a test call to B and run its agent tests.
Decide what winning means before you start
The results show answered, converted and cost per qualified lead for each side. Pick one of these before you begin, and write it down, or you will end up choosing whichever number suits the agent you already preferred.
For most sales teams, cost per qualified lead is the one to pick. Converted counts the outcomes you ticked Counts as a success, on both sides. Make sure those mean what you think, since a loose definition can make both sides look fine. If the two sides use different voices, they cost different amounts per minute, and a side can win on conversions and lose on cost.
Waiting for enough calls
The result shows no verdict until each side has at least 30 answered calls. That is a floor and no more. With 30 calls, a rate can still swing on a few people's moods. More is better, particularly if your two sides are close.
Don't stop the moment your favourite is ahead. Decide your stopping point in advance, such as when every contact has been called, or after a set number of answered calls per side. Then read the result once.
Read the calls as well
Numbers tell you which side did better. Transcripts tell you why. Read ten calls from each. If B converted more because it promised something you can't deliver, that is a bad win, and your numbers would have hidden it. Call QA can check that neither side has gone off your policy.
Ending the test
Make B the only agent, and the campaign carries on with it. You don't rebuild anything. If A won, drop B. If nothing separates them, keep whichever is cheaper or simpler, and test something bigger next time. A small change often shows no difference at all, and that is useful to know as well.
When not to bother
- Your list is too small to give each side 30 answered calls. Judge it by listening.
- You already know from reading calls that A has a problem. Fix it. A test takes time, and every call to A in that time is a call you knew was weak.
- The change is a matter of accuracy. If the fee was wrong, correct it for everyone.
What it costs
A test has no price of its own, and it is on every plan. Both sides are billed as ordinary calls at their own voice rates, as the pricing page shows. You also need a second agent within your plan's agent limit. Once you have a winner, a second round to the people who did not answer is a sensible place to put it to work, and scheduled reports can keep the team informed while the test runs.