Statistics every analyst needs

You don't need a statistics degree, but you must not fool yourself or others.

Describing data

• Mean (sensitive to outliers), median (middle value, robust), mode (most frequent).
• Spread: range, standard deviation, interquartile range (IQR = Q3 − Q1).
• Distribution shape: right-skewed data (salaries, order values) has mean > median.
• Percentiles: "p90 delivery time is 3 days" means 90% of orders arrive within 3 days.

Correlation is not causation

Ice-cream sales and drowning both rise in summer: correlated, but one doesn't cause the other (temperature drives both). Before claiming "X caused Y", check for confounders, reverse causation and coincidence. The cleanest way to establish causation is a controlled experiment.

Correlation coefficient r ranges from −1 to 1 and only measures linear relationships.

Sampling and bias

• Selection bias: surveying only active users makes satisfaction look great.
• Survivorship bias: studying only successful startups hides what failed ones also did.
• Simpson's paradox: a trend reverses when groups are combined. Hire rates can be higher for women in every department but lower overall if more women apply to the most competitive departments. Always check segments.

A/B testing basics

Split users randomly into control (A) and variant (B), change one thing, compare a pre-chosen metric.

• Hypotheses: H₀ "no difference" vs H₁ "B is different".
• p-value: the probability of seeing a difference at least this large if H₀ were true. Below 0.05 is the usual (arbitrary) threshold for "statistically significant".
• Sample size: decide it in advance from the minimum effect you care about; don't stop the test the moment it looks good (peeking inflates false positives).
• Practical significance: a statistically significant 0.1% lift may not be worth shipping.

from scipy import stats
# conversion counts: control 480/10,000, variant 540/10,000
table = [[480, 10000 - 480], [540, 10000 - 540]]
chi2, p, _, _ = stats.chi2_contingency(table)

Check yourself

• When would you report the median instead of the mean?
• Explain Simpson's paradox with an example.
• What does a p-value of 0.03 mean, and what does it not mean?