|
|
Temporal (in)stability of trading system optimization curves |
Written by Forex Automaton | |
Friday, 21 May 2010 09:52 | |
For the first time, I address the question of how stable the optimization results are in time. While predictabilities of daily high and low show a highly stable pattern of dependence on the parameter subject to optimization, the positive results for close are mainly due to the high impact of a single period, which happens to cover the financial panic of the last quarter of 2008. This study relies on the version of the forecasting algorithm not yet deployed on the production host. The simulation approach is the usual one, with the same 14 currency pairs tracked by Danica and daily low-high-close data set covering the interval from August 20, 2002 through May 17, 2010. To study the time stability of optimization, the interval has been split into five equal pieces. The simulated trading period begins after the initial training period is over in middle of April 2006, and this period is split into 5 sub-periods of equal length. Sub-period 4 for example begins in October 2008 and lasts for somewhat less than ten months. The figure of merit used, Pearson correlation coefficient between predicted and actual logarithmic returns of daily low, high and close seems to need no further introduction -- there is hardly anything new to be said about them in this article. ![]() ![]() Fig.1. Different sets of symbols denote different sub-intervals, numbered from 1 to 5, of the total trading time interval from April 15, 2006 to May 17, 2010. Panel 1.1 shows dependence of the Pearson correlation coefficient between predicted and actual logarithmic returns in daily high on the optimization parameter nicknamed Fred, responsible for the forecasting, while panel 1.2 shows the same for daily close. Arrows indicate the choice of Fred implemented in the current version of Danica and affecting the forecasts we publish daily. ![]() ![]() Fig.2. Different sets of symbols denote different sub-intervals, numbered from 1 to 5, of the total trading time interval from April 15, 2006 to May 17, 2010. Just like in Fig.1, top panel shows dependence of the Pearson correlation coefficient between predicted and actual logarithmic returns in daily high on the optimization parameter nicknamed Fred, responsible for the forecasting, while the bottom panel shows the same for daily close. The difference with Fig.1 is that now L0 trigger word 3 is required in order for the "event" (day) to enter the plots. Selected data represent about 80% of all data. Sub-period 4 is the period that includes the financial panic of 2008; it is this period that looks most lucrative in simulations. There is no guarantee that the values of Fred that maximized returns during the panic will continue to work. In particular, the subsequent "recovery" period (sub-period 5 in figures) is seen to have its maximum Pearson at a different (lower) Fred value. These data show no single safe Fred zone for close: any choice of Fred would lead to positive results for two of the five periods and zero or even somewhat negative results for three other ones. This disturbing lack of a single optimization curve shape for daily close is in sharp contrast with the highly stable character of the similar curves for daily extremes (only daily high is shown in the figures). Not only are those curves stable, they also show much higher degree of correlation between predicted and actual logarithmic returns. A daily range strategy may be based on placing a market order with a stop-loss set at the previous day's extreme predicted to not be reached during the day and a profit-target set at the previous day's extreme predicted to be hit. Nominally this avoids the issue of the forecast quality for close. However, a comparison of Fig.1.2 and Fig.2.2 (and similar results from an earlier study) indicates that predictability of extremes is improved by requiring the forecast for close to point in the same direction as high and low. If that path is taken, then the question becomes -- how is it better than betting on close? The Day Range strategy with no selection and re-examined this week, was seen to result in a large proportion of wins and if one calculates Kelly allocation for it, it comes out in excess of 0.3 for some currency pairs when trading in the direction of "risk appetite" (long commodity currencies and interest differential), and in the ballpark of 0.15-0.25 when trading in the direction of "risk aversion". (This peculiar dependence of Kelly coefficient on the trading theme of the day has been already touched upon and awaits deeper investigations.) The problem of the Kelly Criterion in this context however is that the trades with two limits apparently break the cornerstone assumption of Kelly approach, namely the one of the parimutuel distribution of profits and losses (see Eq.1 of the article on Kelly Criterion). As a result, the high Kelly coefficients correspond to trading strategies with net loss: broker's spreads and the infrequent losses end up doing so much damage that it is not fully offset by the much more frequent profits. Once the profit target is dropped, Kelly begins to make more sense as the profit distribution acquires a tail which, even though does not completely turn it into Kelly's "fair odds" (minus first power) distribution, at least seems to put it within the same broad class of animals. In conclusion, I list some ways of making future progress.
|
|
Last Updated ( Saturday, 21 January 2012 16:55 ) |