Tools

Sample size calculator
Find how many visitors each variant of an A/B test needs to detect the lift you care about, with a frequentist or a Bayesian calculation.
Power calculator
Find the chance an A/B test detects the lift you care about from the visitors in each variant, with a frequentist or a Bayesian calculation.
Hypothesis grader
Paste an A/B test hypothesis and get a score out of 100 for whether a result could prove it wrong, whether it names a metric and a direction, whether it explains why it should work, and whether evidence backs it.
- Predict an outcome that a result could prove wrong
- Name the metric you expect to move and the direction
- Say why the change should alter what users do
24steps from hypothesis to rollback
Experiment setup checklist
Check off each step of setting up an A/B test, from the hypothesis and metrics through randomization, sample size, duration, the decision rule, and the rollback plan, with your progress saved in this browser.
1. 51%
2. 54%
3. 56%
4. 59%
5. 61%
6. 63%
7. 65%
8. 67%
9. 69%
10. 71%
11. 73%
12. 74%
13. 76%
14. 77%
15. 79%
16. 80%
17. 81%
18. 83%
19. 84%
20. 85%
21. 86%
30days of traffic to reach 80% power
Planned end date calculator
Find the date an A/B test should end from its baseline rate, minimum detectable effect, significance level, power, traffic split, and daily visitors, with a calendar showing the power reached on each day and the end date rounded up to whole weeks.
What type is the primary metric?
- Yes or no per user
- A count per user
Recommended testTwo-proportion z-test
Which statistical test should I use?
Answer a few questions about your metric, how you randomized, and your sample, and get the statistical test to use for your A/B test, with a sentence to paste into your spec.