Practical guide · verified against the real thing
Survivorship bias: why testing on today's list of names flatters every result
In one line: If your dataset only contains things that still exist, you have silently deleted every failure. Here is how that bias sneaks in and how to defend against it.
Survivorship bias is the error of studying only the survivors and concluding the odds were always good. In quantitative work it shows up whenever a dataset contains the things that exist now rather than the things that existed then — and it makes almost every backtest look better than it is.
How it sneaks in
The classic case is a universe of companies or assets taken from a current list. That list excludes everything that was delisted, acquired, or went to zero along the way — precisely the outcomes a strategy most needs to be tested against. Buying "the survivors of the last decade" and measuring their returns is not a strategy; it is reading the answer key. The same trap appears in any domain: studying only the apps still in the store, the funds still open, the websites still online.
Point-in-time data is the defence
The fix is to test against the universe as it actually was at each moment — a point-in-time snapshot that includes the names that later died. This is harder to obtain and easy to get wrong, which is exactly why it matters: if your data source quietly backfilled a current membership list into the past, every result built on it is inflated. Ask of any dataset: "does this include the failures, or only the things that made it?"
It is a family of look-ahead errors
Survivorship bias is really a cousin of look-ahead bias — both let information from the future leak into a test of the past. The habit that catches both is the same: be paranoid about what your code "knows" at each simulated moment. The overfitting detection guide treats this as one of the standard things to check before believing any result.
Next