Statistically testing your performance

"I'm winning" is a claim. This module gives you the tools to test it.

Testing ROI

Null hypothesis: your true ROI is zero (or equal to −margin). With $n$ flat-staked bets, an average ROI $\bar{r}$ and a per-bet standard deviation $s$:

If $|t| > 1.96$, the result is significant at the 5% level.

The multiple testing trap

If you test 20 strategies (leagues, markets, filters), on average one of them will look significant at 5% purely by chance. This is p-hacking.

• Apply a correction (for example Bonferroni: divide the threshold by the number of tests).
• Validate any strategy you find on fresh data before putting money on it.

Risk of ruin

The probability of losing a given share of your bankroll depends on your edge, your variance and your stake size. For a Kelly strategy (fraction $c$ of Kelly), a classic approximation is:

(These figures assume your estimated probability is exact. With estimation errors, reality is worse.)

The distribution of maximum drawdown

Rather than a formula, the most reliable method is Monte Carlo simulation:

1. Take your actual betting history (odds, estimated probabilities, stakes).
2. Replay it thousands of times, drawing the results at random according to your probabilities (or resample your history: bootstrap).
3. Look at the distribution of the maximum drawdown: the median, the 95th percentile.

If the 95th percentile of the drawdown exceeds what you can handle, reduce your stakes.