Product review
Refund Pilot test plan
Read the idea, try both versions, and check where the signup and paid offer should appear.
1. Big idea
Use a support chat to help people with a specific subscription cancellation or refund. Once we understand their request, introduce MoneyPilot as a way to find other savings and manage more subscriptions.
Refund Pilot is clearly identified as a MoneyPilot service. The chat handles the conversation until the user reaches signup.
2. Customer context
Branded refund terms have cheaper clicks on Google Ads, but our conversion rate is low. People clicking these ads expect help with a refund from a specific company. A general savings pitch does not directly answer that request.
We want to start with the problem they came to solve, then explain the broader service.
Campaign context supplied by the team. CPC, conversion rate and baseline results still need to be added.
3. Hypothesis
If we first respond to the customer’s specific refund or cancellation request through chat, more ad visitors will continue into MoneyPilot and become paid customers.
We will compare two ways to do this: provide free help before signup, or ask for a trial signup before starting the first case.
4. Success metric
Main metric: paid customers per Google Ads visitor. Also compare the cost to acquire a paid customer.
We want a higher paid conversion rate and lower acquisition cost than the current funnel, without worse customer retention or more complaints. Agree on the minimum improvement and measurement window before launch.
- Supporting measures: chat completion, signup completion, account connection and useful first case progress.
- Quality checks: confirmed cancellation outcomes, customer complaints, support cost and retention after the first renewal.
- Comparison: send comparable traffic to both versions. Use the same brands, intent and offers when comparing results.
A completed signup or sample scan alone does not decide the winner. This demo has no real conversion data.
5. MVP variations
Both versions ask about intent, company, billing route, situation and reason. The difference is when help, signup and the full service appear.
Variation A: Free help before signup
Use the chat to understand the request. Give initial guidance, then offer a broader savings check and ask the user to sign up.
Chat → Free help → Savings offer → Signup → Connect → Findings → Paid offer
What to check: Is the free help useful? Does the savings offer follow naturally? Is there a clear reason to sign up?
Variation B: Trial signup before help
Use the same chat to understand the request. Ask the user to sign up for a trial, start the first case, then introduce the full service near the email or bank connection step.
Chat → Trial signup → First case → Full service + connection → Findings → Upgrade
What to check: Is the trial worth signing up for? Is the full service clearly different? Are connecting and upgrading separate choices?
How to review
- Open Variation A and use a sample company. Go through the chat, signup and sample scan.
- Open Variation B with the same sample request. Compare the point where signup is introduced.
- Use the mobile preview links to check typing, reading previous messages and reviewing the request.
- Check the questions listed under each variation. Note what feels unclear or unnecessary.
In each chat, use Review chat to read the conversation, Edit the details to correct the case summary, and Restart to try again.
What is a demo?
Signup, cancellation actions, connections and findings are simulated. The trial is shown as seven days free, with no card and no automatic billing. These are example terms. The actual trial, price and supported brands are still to be decided.