Is the winner real — and can you tell the business in one sentence?
Tip: run at least one full week, even if the maths says less, so weekday and weekend behaviour are both in the sample.
Two-sided two-proportion z-test with pooled standard error. The confidence interval on the difference uses the unpooled (Wald) standard error; the relative-uplift interval divides that by the control rate, so treat it as approximate.
Sample ratio mismatch: chi-square test (1 df) of observed visitors against the planned split, flagged when p < 0.01. A mismatch usually means a tracking, redirect or bucketing bug — fix that before reading the result.
Sample size: standard normal-approximation formula for two proportions at the chosen confidence and power. Fixed-horizon test: decide the sample size first and don't stop early when it "looks significant".
Built by Keith Chiu with AI-assisted development. Runs entirely in your browser.