Bayesian sample size calculator
Find how many visitors each variant needs before the probability that it beats control can reach your threshold, with the power you choose.
Visitors needed per variant
- Total across both variants
- 23,236
- Variant conversion rate
- 11%
How the sample size is calculated
With a flat prior, the probability that the variant beats control is the normal CDF of the observed difference divided by its standard error. Reaching the probability you set means the difference has to reach zP standard errors, and the power is how often a variant with the lift you entered gets there, which gives the n below for each variant.
n = (zP + zβ)² · (p₁(1 − p₁) + p₂(1 − p₂)) ÷ (p₂ − p₁)²
The Bayesian figure comes out smaller than the frequentist one at the same confidence because it asks for one-sided evidence that the variant is better, where the frequentist test also guards against the variant being worse.