Module 6 of 6 in Technical analysis
Backtesting and the limits of technical analysis
How to backtest fairly, the biases that make backtests look better than reality, overfitting, sample size, out-of-sample testing and the evidence on technical methods.
4 lessons in this module. Progress is saved in your browser as you go.
A backtest asks how a set of rules would have performed on past data. Done carelessly, it produces impressive results that never repeat. This module explains how to test fairly and the specific ways backtests mislead, then looks honestly at what technical analysis can and cannot do.
By the end of this module you should be able to
- Describe how a backtest works and the costs and frictions it must include.
- Identify survivorship bias, look-ahead bias and unrealistic execution assumptions in a backtest.
- Explain overfitting, sample size and out-of-sample testing and judge whether a result is robust.
- Explain the limits of technical analysis and the evidence on chart-based methods.
Lessons in order
Module test
10 questions, chosen at random from a bank of 30 questions and balanced across this module's learning goals and difficulty levels. Every question comes with an explanation. You need 80% to pass, and you can retake it as often as you like with a fresh set of questions.
This module builds on
- Expectancy, losing streaks and risk of ruin: Backtests are judged by expectancy, drawdowns and losing streaks, explained there.
- Adjusted and unadjusted prices: Backtests must use prices adjusted for splits and bonuses.
- Trading costs and taxes on shares: Costs and taxes must be included, using the charges explained there.
- Correlation and causation: Fitted patterns are correlations in past data that may have no lasting cause.