Replicating results
As I discuss in each of my books and speeches, I strongly believe that readers should be able to replicate the results of the author.
I have received several emails from readers in response to the analysis of the Mean Reversion System based on RSI with questions about replication.
There was an inconsistency in my original post. When I posted the original article the Initial Equity in the AmiBroker code had been set to $200,000. That was a leftover from an earlier test run. It should have been $100,000 to agree with the analysis that followed and has been corrected.
Many people wrote to say that they had successfully replicated my results. Several continued on to say that they were surprised by the relatively high probability of a long period of losing trades and underwater equity, even though the system appears to be reasonable. That is the point I hoped to make — that the decision whether or not to trade a system should be based on more information than simple metrics such as the mean return per trade. It is important to evaluate the distribution of profit potential and the distribution of risk.
Some people were concerned that they were not able to replicate my results exactly. It is unreasonable to expect your results to be an Exact replication of those published.
If your results are widely different, consider:
- The results I published were based on trading only SPY, not a portfolio or watchlist.
- I assumed that the trades were made at the close of the bar that generated the signal and at the closing price. Delaying the execution will cause different results.
- Each trade is a basic unit of $10,000. No compounding or leverage is used.
If your results are close, but not exact, the difference is probably due to use of a different data stream or a small change in settings.
- I set the commission to $5 per trade.
- I used Norgate Premium Data.
Recent Comments