A/B Testing Ratio Metrics with the Delta Method

medium
Unit 16 · A/B Testing0/2 to milestone

Click-through rate is total clicks / total views, but users (the randomisation unit) each contribute many views, and their views are correlated. Treating every view as an independent coin flip underestimates the variance and produces false wins.

Pro problem

Unlock this and every premium problem across all tracks — plus hidden-test grading, advanced AI engineering modules and new content every week.

Start 7-day free trial
solution.py · Python 3.13
Run your code to see results. ⌘↵ to run · ⌘⇧↵ to submit
A/B Testing Ratio Metrics with the Delta Method · TechGradify