Data Science Dojo Meetup
Understanding A/B testing, requires some very basic understanding (and intuition) of Statistics at 101 level. To keep the tutorial self-contained, I will first give an overview of stats fundamentals needed to understand A/B testing. Next, I will explain how A/B testing is done in an online business. In the end, I will mention some of the pitfalls that are common when running online experiments.
This tutorial will roughly be divided into these parts:
• Introducing A/B testing.
• The motivation for running an A/B test. Some examples of how A/B test results are often counter-intuitive. Obama for America campaign and other examples. Multivariate testing
• Statistics fundamentals for hypothesis testing: Null and alternate hypothesis, Factors and levels, confidence intervals, p-values, t-test, sample size, power, Type I and Type II error.
• Art of interpreting metrics: Short-term, medium-term and long-term metrics.
• Common fallacies and pitfalls
Canada, North America, Toronto
Arham Akheel, Raja Iqbal, Rebecca Merrett