Three ways to search
- Grid search tries every combination in a defined range. It is exhaustive and easy to reason about, but the number of runs grows fast as you add parameters.
- Bayesian search spends a limited trial budget where earlier results suggest the surface is promising. It reaches good regions with far fewer runs, at the cost of being less exhaustive.
- Walk-forward fits on a rolling window and judges on the window after it, so the answer is always from data the fit did not see.
Pick the ranking metric on purpose
The leaderboard obeys whichever metric you choose, so the choice is part of the strategy. Ranking by total profit favours settings that took the most risk. Ranking by Sharpe or Sortino favours smoother returns. Ranking by drawdown or the Ulcer index favours survival. A composite that blends several measures reduces the chance that one lucky metric decides the winner.
Decide the metric before looking at results. Changing it afterwards until a favourite wins is just another way of fitting the sample.
Look at the neighbourhood, not just the winner
A good setting sits in a plateau: nearby values perform similarly. A single sharp spike surrounded by poor results is usually noise. If moving a stop by a small amount flips the result from excellent to poor, the optimiser found a coincidence.
Five clones are not five ideas
The top of a leaderboard is often a cluster of nearly identical settings that win and lose on the same days. Choosing the top five of those gives you one position five times. Check the correlation across the leaderboard, and when building a combined portfolio prefer picks that are not correlated with each other.
Make the verdict out-of-sample
Keep the in-sample and out-of-sample windows strictly apart, and read walk-forward efficiency: the out-of-sample result as a share of the in-sample one. A large gap is the overfit showing itself. Treat an overfit-risk flag as a reason to stop, not a detail.
Keep the search small
Every extra parameter and every extra value is another chance to fit noise. Fewer parameters with a reason behind each one will generalise better than a large space searched until something scores well.
Common questions
What is overfitting in strategy optimisation?
Overfitting is when settings are tuned so closely to a particular stretch of history that they capture its random noise as well as any real pattern. The result looks excellent on that history and does not repeat on new data.
Grid search or Bayesian optimisation?
Grid search is exhaustive and simple but costly with many parameters. Bayesian search covers promising regions with fewer runs. Whichever you use, judge the winner on data it was not tuned on.
Which metric should I rank by?
Choose it before you see the results, and match it to what matters to you: Sharpe or Sortino for smoothness, drawdown or the Ulcer index for survival. A composite of several metrics is less sensitive to one lucky number.
Try it on the desk
Build the strategy without code, run it over history, and read the result before any money is involved.