How to Optimise a Trading Strategy Without Fooling Yourself

Here is the one line to hold onto: optimisation is not the process of finding the best parameters. It is the process of finding out whether your strategy has an edge that survives parameters you did not choose. Almost everything below follows from that.

This is a companion to Anatomy of a Trading Strategy, which read the Adamcator v7 script line by line. Here we ask the harder question: how do you tune it without lying to yourself?

The thing that goes wrong

The v7 script has about eight tunable numbers. Across sensible ranges, that is on the order of 100 billion combinations. You will not test them all - but the problem is there no matter how many you test:

If you run enough combinations against one dataset, some of them will look excellent purely by chance.

This is not a metaphor, it is arithmetic. Suppose the strategy has zero real edge and each parameter set produces a random result centred on break-even. Test a thousand sets and the best one will look great - not because it is good, but because you took the maximum of a thousand random draws. You are reading the noise, not the signal.

This is overfitting, and it is by a wide margin the reason most backtested strategies fail live. Not bad code. This.

The corollary people find hardest to accept: every parameter you optimise makes your backtest less trustworthy, even when you optimise it correctly. A strategy with 3 parameters and a profit factor of 1.3 is more believable than one with 12 parameters and a profit factor of 2.1.

Fix the measurement before you tune anything

Optimising against a broken backtest is worse than not optimising - it produces confident wrong answers.

• ▸Costs are real. The script charges 0.1% commission and 2 ticks of slippage. Double both and re-run. If the edge dies, it was living inside the spread - extremely common with EMA crossovers on short timeframes.
• ▸Sample size is not optional. Under 30 trades you have learned nothing; 100-300 lets you compare cautiously; 300+ lets you do real statistics. And you need roughly 10x more trades per parameter you optimise. Tuning 6 parameters on 150 trades is numerology.
• ▸One data source. Different exchanges give different histories. Pick one and stay on it, or your "improvement" may just be a data change.

Choose the right thing to maximise

Net profit is the wrong objective - it rewards the parameter set that caught the single biggest move, which is the definition of a fluke. Better is expectancy in R, the average result per trade in units of risk:

A strategy with expectancy of +0.08R over 400 trades is a real business. +0.4R over 40 trades is a coin that came up heads.

The single most important technique: plateaus, not peaks

If you take one thing from this post, take this. When you sweep a parameter, plot the whole range instead of reading off the maximum.

   Plateau (good)                 Peak (bad)

   PF                             PF    |
   1.4 |    ______                2.0  |      /\
   1.3 |   /      \___            1.5  |     /  \
   1.2 |  /           \           1.0  |____/    \____
       +----------------               +----------------
        1.0  2.0  3.0  4.0              1.0  2.0  3.0  4.0
             atrMultSL                       atrMultSL

A plateau means the edge is structural - values on either side of your choice also work, so when live conditions shift slightly you slide along it and stay profitable. A peak means you found a coincidence: 2.3 works and 2.2 and 2.4 do not. There is no mechanism in markets that makes 2.3 special.

Always choose the middle of the widest plateau, never the peak - especially when the peak shows a better number.

In-sample and out-of-sample

The discipline is simple and almost nobody follows it:

1. 1.Split your history: roughly 70% in-sample, 30% out-of-sample. The out-of-sample part must be the most recent data, and you must not look at it.
2. 2.Do all optimisation on the in-sample data only.
3. 3.Choose your parameters. Lock them.
4. 4.Run once on out-of-sample.
5. 5.If it fails, the strategy failed. You do not go back and re-tune. The moment you use that result to change parameters, the data is spent.

Step 5 is the hard part. If you re-optimise after peeking three times, you have just done in-sample optimisation over the whole dataset with extra steps.

Cheap tests that reveal a lot

• ▸Across instruments. A real trend-following edge should work, weakly, on many trending markets. If it only works on one ticker, you fitted that ticker.
• ▸Parameter jitter. Perturb every value by plus or minus 10%, twenty times. If a 10% nudge flips the sign, you have a peak, not a plateau.
• ▸Monte Carlo shuffling. Shuffle your trade order a thousand times and rebuild the equity curve. Your single historical drawdown is one draw from a distribution - usually not the worst you will see.

About that breakeven stop

In part one we flagged the breakeven feature as suspicious. Here is why. Mechanically it does two things: it converts trades that *would have* reached target into scratch trades when price dips through entry first, and it does nothing for your worst losses - gaps blow through entry before the stop ever arms. So it tends to remove winners and keep losers, buying a prettier win rate and a worse expectancy.

Test it with an A/B, everything else fixed, and compare expectancy in R, not win rate - the win rate will improve and it will be lying to you.

Warning signs your result is overfitted

• ▸Profit factor above ~2.5 on anything but a tiny, specific sample.
• ▸The equity curve is unusually smooth. Real edges are lumpy.
• ▸Removing the single best trade collapses the result.
• ▸Optimal parameters are oddly precise (13.7, 2.83) or sit at the edge of the tested range.
• ▸It works on exactly one instrument and one timeframe.

Honest framing

A real edge is usually small - expectancy of +0.05R to +0.15R per trade is a genuinely good systematic strategy. If your backtest shows +0.5R, look for the bug first. The lasting value of this exercise is the *process*: learning to build honest backtests transfers to every strategy you will ever test. A specific EMA configuration does not.

You can practice the judgement side of all this here on tradegame.org for free - trade a plan, keep the sample honest, and notice how often a run that felt brilliant was just a lucky streak.

Further reading: Robert Pardo, *The Evaluation and Optimization of Trading Strategies*; Bailey and Lopez de Prado, "The Probability of Backtest Overfitting"; Ernie Chan, *Quantitative Trading*.

*Nothing here is financial advice, and I am not qualified to give any. This is methodology for evaluating systems, not a claim that any system will work.*