Sample Ratio Mismatch Check

easy
Checkpoint challenge
Solve this to show you know the material and jump straight to Unit 8. Everything before it counts as cleared.

If an experiment was meant to split 50/50 but the arms got 10,000 vs 10,400 users, something may be broken (bots, redirects, logging). A sample ratio mismatch check runs a chi-square test on the counts.

Implement srm_check(counts, expected_ratios, alpha=0.001) returning (chi2, p_value, mismatch). For two arms (1 degree of freedom) the p-value is math.erfc(math.sqrt(chi2 / 2)); for more arms, compute the upper tail with the series for the regularised incomplete gamma function (as in the chi-square goodness-of-fit problem) with k - 1 degrees of freedom. mismatch is p_value < alpha.

#ab-testing#data-quality#hypothesis-testing2 visible + 1 hidden tests
solution.py · Python 3.13
Loading editor…
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
Sample Ratio Mismatch Check · TechGradify