Speaker: Adrian Reid
Topic: Beyond the Backtest - Better Optimization and Robustness Testing
In this presentation, Adrian Reid (Founder of Enlightened Stock Trading) delves into the realms of trade optimization and robustness testing, emphasizing the limitations and potential pitfalls of relying solely on backtesting when developing trading systems. The talk is structured to tackle several key points critical for enhancing trading performance and confidence in trading systems.
Introduction
Adrian begins by highlighting the ease of backtesting and the misleading sense of security it can provide. He asserts that backtesting alone often leads to systems that do not perform well in real-time trading. The presentation aims to offer advanced techniques for assessing the robustness and future profitability of trading systems.
Common Problems in Systematic Trading
Adrian addresses common issues including emotional impacts on trading, insufficient data for system optimization, and overfitting systems to past data. These elements often lead to a lack of trust in the system’s generated signals, causing traders to make emotional decisions that can be detrimental.
Harnessing Emotions for Better Trading
Instead of suppressing emotions, traders are advised to harness them. Techniques to convert emotional reactions into actionable rules are shared. Traders are encouraged to consciously evaluate each trade signal, list sources of uncertainty, and generate rules to isolate these uncertainties.
Evaluating System Robustness
A systematic way to evaluate trading systems is proposed. By backtesting with slight variations in parameters (plus/minus 20%), traders can determine the robustness of the system. The results provide insights into whether the chosen parameters are robust or prone to overfitting.
Better Use of Historical Data through Random Miss Technique
For portfolio systems, traditional optimization methods often miss important signals. The speaker introduces a 'random miss' technique, whereby 30% of trades are randomly skipped during optimization. This method helps incorporate more information into the analysis without overfitting, thereby enhancing the reliability of the system.
Compounding vs. All Trades Backtest
The merits and demerits of two backtesting methods—compounding and all trades—are discussed. Whereas compounding tests provide realistic growth and drawdown metrics but miss rejected signals, all trades tests capture all potential signals but lack realistic compounding insights.
Conclusion
By employing these advanced techniques for optimization and robustness testing, systematic traders can avoid the pitfalls of emotional trading and overfitting, ultimately leading to more reliable and profitable trading systems. The presentation wraps up with an offer for free resources aimed at further assisting traders in enhancing their systems.
240722_Quant SIG - Reid, Adrian