Experiments
Show new devices one of two prices for a plan, or one of two headlines on its paywall, and compare what each sold.
An experiment shows the devices that meet a tool's paywall one of two versions of it: A, as it is, or B. B changes one thing: the price of one plan, or the paywall's headline and the line under it. Toolaby counts what each side bought and tells you whether the difference is likely to be real.
This page covers starting a test, how devices are split, reading the results, and ending one. Experiments are part of Analytics and on Pro.
Before you begin
- The tool has at least one plan under Pricing.
- Your extension runs the client of 16 October 2026 or later:
npx -y toolaby@latest upgrade <tool-id>, then publish the build. An older copy always sees A. - Stripe is connected, for a price test.
- The tool is counted: Configure → Settings → Analytics. While it is not, no test starts, and a test that runs shows every device A.
1. Start a test
Open the tool, then Analytics → Experiments, and fill in Start a test:
| Field | |
|---|---|
| What B changes | The price of one plan, or The headline of the paywall. |
| Plan | The plan the test is about. A price test sells it at B's price to the devices on B. |
| B's price | In the plan's currency. Toolaby makes it as a price on your Stripe account, with the plan's own metadata, so every sale at it is a sale of the plan. |
| B's headline, B's line | Shown in place of the paywall's own, as you write them, in every language. |
| Shown B | The share of devices that see B, from 5% to 95%. Half and half answers soonest. |
A tool runs one test at a time. From the next minute, every device that opens the paywall is shown A or B.
2. How devices are split
A device's side is worked out from its id and the test's id, the same way every time, on every server: a device sees the same side on the paywall, on the checkout page and in the results, with nothing stored about it. Only a device that has checked in for the tool is shown B; any other sees A. A device that met the paywall before the test began sees its side too, but is not counted: the results compare only devices that met the paywall for the first time during the test.
A buyer who pays B's price keeps it: a licence at B's price is a licence of the plan, and a subscription at B's price renews at it.
B's price can be reached by someone who looks hard enough at how sides are chosen. Test only prices you are willing to sell at.
3. Read the results
| Column | |
|---|---|
| Met the paywall | Devices on that side that met the paywall for the first time while the test ran. |
| Bought | Of them, devices whose purchase came through the test. |
| Converts | Bought, as a share of met the paywall. |
| Revenue | What those purchases paid. |
| Per device | Revenue, shared out over the devices that met the paywall. |
Under the table, Toolaby says which side wins and whether a difference that size is unlikely to be chance, at 95%. Until it is, it says about how many more devices each side needs.
- A headline test is judged by how often each side sells: a two-sided test of two proportions.
- A price test is judged by Per device, what each side brings for each device that met the paywall. A lower price usually sells more often and can still earn less.
4. End the test
Press End: A is better, End: B is better or End, no verdict. Every device sees A again from the next minute. B's price is archived on your Stripe account: nobody new can buy at it, and those who did keep it.
Ending changes nothing else. To keep B's price, change the plan's price under Pricing. The paywall's own words are Toolaby's, in every language: a headline test tells you which words sell, to use on your listing and your own pages.
A test that runs when the workspace leaves Pro stays visible and can still be ended.