Avoiding Overfitting
What overfitting is
Overfitting happens when you add rules to a backtest until the results look perfect — but what you've actually done is memorised the specific price history, not found a real edge. Every additional rule you add to 'fix' a losing period is another link in a chain that will break on new data.
Defence 1: keep the rule set small and logical
Every rule must have a logical, structural reason behind it — not just 'it worked in the backtest'. If you cannot explain why a rule should work going forward (e.g., 'fresh zones have more unfilled orders'), it is probably overfit.
Defence 2: out-of-sample testing
Divide your historical data: tune rules on the first 60%, then test on the remaining 40% without touching the rules. If performance drops significantly on the out-of-sample period, the rules are overfit to the tuning period.
Defence 3: minimum trade count
A rule tested on fewer than 30–50 historical examples proves almost nothing statistically. 10 trades could easily win by luck. Build in a minimum sample size requirement before trusting any result.
The red flag: too-perfect equity curves
An equity curve that rises smoothly without drawdowns almost certainly means the rules were tuned to the history. Real edges have rough, drawdown-filled equity curves — they are still profitable on average, but lumpy.
Practice checklist
- Every rule must have a logical reason, not just 'it worked'
- Reserve 40% of historical data for out-of-sample validation
- Require minimum 30–50 historical examples before trusting a result
- Be suspicious of equity curves with no significant drawdown periods
Mistakes to avoid
- Adding parameters until the backtest curve looks perfect — classic overfitting
- Trusting a backtest with only 8–10 historical trades
- Not setting aside out-of-sample data before tuning