Forex Automaton > Forex Research > Forex trading system: are we there yet?
Forex trading system: are we there yet?

A series of articles on the automated forex trading system development.



Minimizing the "benefit of hindsight" in trading system testing
Written by Mikhail Kopytine   
Monday, 27 October 2008 16:07

Once you know something, such as the past history, pretending that you do not is very difficult. The ways a trader can delude him-/herself while backtesting a trading system vary in degree of subtlety. A crude one is to develop a set of trading rules by observing the past history of the market, and apply the algorithm on paper to the same past history. A more subtle one has to do with the choice of adjustable parameters. It is common to abstract the algorithm, and split it into the "artificial intelligence" (AI) core, capable of "learning", and adjustable parameters which may control the learning process, the application of its effects, or both. A developer would then prevent the AI from learning the future, and run a trading simulation program on historical data, having the AI algorithm fixed, for a range of parameter values and then pick the value that gives the best result. By not letting the AI learn the future, the developer reduces the level of self-delusion somewhat. The best simulated performance would still incorporate "the benefit of hindsight" albeit in a refined way -- through the choice of the adjustable parameters. While the AI did not learn from the future directly, its way of learning would incorporate the benefit of hindsight, and the results might contain a survivorship bias of sorts. The result? In case of a poor AI, the result will be a trading strategy with a hidden "peso problem".

Below I demonstrate the quality of the Forex Automaton™ AI directly, by removing the selection step and thus the survivorship bias it brings in. I look simultaneously at the entire range of possible "ways of learning", refusing to reject any, even if there are a priori reasons to do so, no matter how convincing they might be. Nor do I reject any of the possible money management styles, for the same reasons.

Return vs risk for trading AUD/JPY algorithmically with real and simulated data

Fig.1:Return vs risk for paper trading in real AUD/JPY and simulated reference random walk markets. Vertical axis: annualized return, calculated as an average of independent monthly statistics, 1=100% per annum. Horizontal axis: standard deviation (RMS) of such annualized return from the same monthly statistics, same unit. Each point represents an instance of a trading system, a simulated trader; points differ by trading strategy which is subject to optimization. Black points: trading 4 fantasy markets with volatility of AUD/JPY, random by construction (no predictability or patterns). Red points: real AUD/JPY. Not all outcomes are included: in some cases, the trading did not last even a month therefore no RMS could be calculated -- there would be no other month of trading performance to compare with. In such cases the system is programmed to record a zero RMS, and such outcomes did not make it into the figure. The only other requirement is that of excluding the mean annualized returns above 5 (500%) and RMS above 20 -- happy as I am to include higher returns, they would make the picture difficult to analyze visually.

Last Updated ( Monday, 24 November 2008 17:57 )
Read more...
 
Back-testing a forex trading system
Written by Mikhail Kopytine   
Sunday, 25 May 2008 07:50

While it is true that past performance does not indicate the future, the only reliable information we have is about the past. A few important things make a difference between unbiased trading-system testing and self-delusion. Here I summarize my current understanding.

Last Updated ( Saturday, 25 October 2008 18:17 )
Read more...