Chi-Square Calculator
Free tool

Chi-Square Calculator: do your groups really behave differently?

Compare conversion rates across channels, ad variants, countries or customer segments, and find out whether the differences are real or just luck, and which groups stand out. Plain English, no statistics needed.

Free for everyone to use, no sign-up. Nothing you type leaves your browser.

  1. Pick your groupsFor example your four acquisition channels, or three ad variants.
  2. Enter the numbersFor each group: how many people, and how many converted.
  3. Read the verdictReal difference or luck? And which groups stand out?

"Sure" here means: if all groups really behaved the same, you would wrongly see a difference in only 5 tests out of 100 (with 95%). Not sure? Keep the standard.

Advanced settings (most people can skip this)

Use this to compare groups. Enter one row per group (a channel, an ad, a country, a customer segment). Comparing just two? Delete rows until two remain. For planning an A/B test or measuring the size of an improvement, use the A/B Test Calculator.

Up to 8 groups.

A worked example

Marta runs marketing for an online course. Last month she got 10,000 visitors from each of four channels and wants to know whether the channels really convert differently, or whether the gaps are just noise.

The numbers (already filled in above): Email 420 sign-ups (4.2%), Paid search 510 (5.1%), Social 300 (3.0%), Organic 480 (4.8%).

The verdict: the groups really do behave differently: a gap this big would happen by luck far fewer than 1 time in 1,000. Paid search and Organic convert more than average, Social converts less, and Email is in line with the average. The gap is big in practice: Paid search converts 1.7 times as often as Social. Marta shifts some budget from Social to Paid search and Organic, and keeps testing.

Good habits

  1. Use counts, not percentages. "5%" alone hides whether it is 5 of 100 or 5,000 of 100,000, and that is exactly what the test needs to know.
  2. Compare like with like. Same period, same definition of "converted" in every group.
  3. Count each person once. The same customer should not appear in two groups or twice in one.
  4. Do not go fishing. The more groups you compare, the more likely one stands out by luck alone. Decide what to compare before you look.

Key ideas explained

The chi-square test
A test that asks one question: are the differences between my groups bigger than luck alone would produce? "Chi-square" (χ²) is just the statistician's name for the number it calculates.
What happened vs what luck predicts
The test first works out what you would see if every group behaved exactly like the average of all groups. Then it measures how far the real numbers are from that. The further away, the harder it is to blame luck. Roll a die 60 times: you expect about 10 sixes, but 8 or 12 is normal. 25 sixes would make you suspect the die.
"Just luck" and a real difference
Even identical groups will show slightly different rates because people are different. A difference is called statistically significant when it would be very unlikely to appear by luck alone. This calculator says it as "by luck about N times in 100".
How sure you want to be
Your risk appetite. At 95%, you accept being fooled by luck in about 5 tests out of 100. At 99%, only 1 in 100, but small differences become harder to prove.
Size of the difference
A real difference is not necessarily a big one. With a huge audience even a tiny gap is "real". So the calculator also tells you how big the gap is: how many times more often the best group converts than the weakest, and by how many percentage points. (For tables with several outcomes it uses a statistician's measure called Cramér's V, translated into very small / small / moderate / strong.)
Groups that stand out
When the groups differ, this shows which ones are further from the average than luck would explain, and in which direction. It is a guide, not proof: the more groups you compare, the more likely one stands out by chance.
Too few people
Small groups make the maths unreliable. When a group is too small, the calculator warns you to gather more data or merge small groups.
Real difference does not mean cause
Different channels attract different people. A channel with a higher rate is not automatically a better channel.

Methodology (for analysts)

Chi-square test of independence on an r × c table of counts:

E(i,j) = (row total i × column total j) / grand total χ² = Σ (O − E)² / E degrees of freedom = (r − 1)(c − 1) p-value = P(χ²(df) > observed χ²) Cramér's V = √( χ² / (N · min(r − 1, c − 1)) ) adjusted residual = (O − E) / √( E · (1 − row total/N) · (1 − column total/N) )

"Groups that stand out" flags cells whose adjusted residual exceeds ±1.96 (about 95% two-sided), without correction for multiple comparisons. With the Yates correction (2×2 only), each |O − E| is reduced by 0.5 before squaring. In the groups tab the size of the difference is the ratio and the gap between the best and weakest conversion rates (Cramér's V is shown in the technical details, but it is always small when most people do not convert). In the advanced tab, Cramér's V is translated with Cohen's benchmarks scaled by the table size (for a 2×2 table: below 0.1 very small, 0.1 small, 0.3 moderate, 0.5 strong).

For a 2×2 table without correction, χ² equals the square of the pooled two-proportion z statistic, so it agrees with the A/B Test Calculator.

Assumptions: each observation is counted once and belongs to exactly one cell, observations are independent, and expected counts are large enough (the usual rule is at least 5 in every cell). The calculator warns when this fails. It does not correct for multiple comparisons or repeated looks at the data.

Frequently asked questions

What is a chi-square test, in plain English?

It checks whether the differences between your groups are bigger than luck alone would produce. It compares what really happened with what you would expect to see if every group behaved in exactly the same way.

When should I use this instead of the A/B test calculator?

Use this one to compare two or more groups such as channels, countries or customer segments and see whether they differ at all. Use the A/B Test Calculator when you want to plan a two-version test or measure how big the improvement is.

The result says the groups differ. Which one is best?

The verdict only says that not all groups behave the same. Check "Groups that stand out" and the conversion rate chart to see where the difference comes from.

Does a real difference mean a big difference?

No. With a lot of traffic even a tiny gap can be real. Look at the size of the gap, for example how many times more often the best group converts than the weakest, to judge whether it matters in practice.

Can I enter percentages instead of counts?

No, enter the number of people and the number who converted. The test needs real counts because the size of each group tells it how much to trust each percentage.

Does this prove that one channel causes better results?

No. It shows that groups differ, not why. Different channels attract different audiences, so a higher conversion rate does not automatically mean the channel itself is better.

Related tools