Trading system optimization: Step One analysis update.

User Rating: / 1
PoorBest 
Written by Forex Automaton   
Tuesday, 15 September 2009 13:56

The historical data for the six major forex markets have been re-processed with the upgraded back-testing code in the Step One mode (fully isolated market analysis and fully isolated portfolio management). This Step One report is useful for a side-by-side comparison with Step Two results representing the next step in the algorithmic sophistication, combining multiple markets in the same portfolio. Both reports will serve for future reference, the current goals being 1) to ensure we do make progress in the trading system development from Step One to Step Two, and 2) understand the changes in the trading style that are necessary to realize the extra benefits of Step Two.

Step One, back-tested previously with a more limited time line of historical data, is the most basic method of the trading system operation. Not only is each market being predicted independently of the rest, with no attempt to discover and learn the inter-market patterns, but also the money-management component of the algorithm deals with one market. Essentially, each of the six most popular forex pairs (EUR/USD, USD/JPY, GBP/USD, USD/CAD, USD/CHF and AUD/USD) has a dedicated portfolio for itself and the returns refer to that portfolio. This is an operation of a trader who only knows and trades a single market. The problem of weighing the concurrent opportunities arising from the different markets does not exist.

 

Money management for the individual markets

separate

joint

Pattern analysis for the individual markets

separate

You are here: Step One. See also an older summary.

Step Two

joint

test-only

Step Three

Table 1. Various modes of the trading system operation. The present report deals with the configuration denoted as Step One, the simplest of the three.

The basic framework remains the same: a run of the program included simulations of trading histories of about 5 thousand independent "virtual traders" (forex robots), each of them being an incarnation of the same algorithm, differing by the setting of the adjustable knobs. The algorithm learns continuously, but at any point in time, it may use only data from the past (and not the future) of the time series. For the forecasting parameter knob (so called Fred), the range of the settings is narrowed down to the best values on the basis of back-tested performance seen in the first set of Step One simulations, that range being 71 through 80. (Note that the Fred range was not re-optimized following an extention of the time line into August 2009, just like the disciplined approach does not pre-suppose such revisions of key parameters in the course of real-life trading. Once the Fred range is fixed, it becomes part of the "trading philosophy" while the actual content of the robot's "memory" changes with time. The distinction here is the one between what the automaton knows and how she uses that knowledge, with how being fixed, and what being subject to updates.)

As in all other studies posted here so far, the trading is performed on one-decision-a-day time scale, with 1:100 leverage, risking no more than 10% of the trading capital at any point in time. This report uses the EUR/USD, USD/JPY, GBP/USD, USD/CAD, USD/CHF and AUD/USD day scale data covering the time interval from August 20, 2002 to August 21, 2009, with the actual trading starting in April 2006 (when the initial "training" of the system was completed). The trading system control parameters remain as previously defined.

The trading performance data are updated continuously and the mean annualized logarithmic return and its RMS are calculated on the monthly basis. These form the basis for a risk vs return approach to the trading system optimization. The rationale for using these measures of risk and return has been given before. Return vs risk analysis is done in the logarithmic variables, while the linear annualized return, an accountant's quantity you want to know, is used in parallel.

The simulations for this report have been performed with the most recent set of upgrades, documented separately and used in the most recent Step Two results.

Mean position duration vs stop-loss. Back-testing results shown as profile histograms. 1.1 Annualized return vs stop-loss. Back-testing results shown as a profile histogram. 1.2 Annualized logarithmic return mean minus RMS Back-testing results shown as a profile histogram. 1.3

Fig.1. Stop-loss optimization. 1.1: Mean position duration (days) vs stop-loss parameter; 1.2: annualized return (1=100%) vs stop-loss parameter; 1.3: same for the figure of merit, logarithmic return minus logarithmic risk as mean minus RMS of annualized monthly return series. This corresponds to Fig.1 of the recent Step Two report.

As always in these optimizations, needlessly tight stop-loss leads to overtrading and losses. Naturally, a more generous stop-loss leads to longer position life-time, and the more so as the more conservative the trade acceptance policy becomes with the increase of the entry threshold (Fig.1.1). The mean position duration at the same stop-loss parameter value is significantly longer than seen in Step Two. This is because of the continuous rebalancing among the different markets. Understandably, the effect (Step One vs Step Two difference) is the most significant for the trading styles with the least conservative trade entry policy. In Step One, the entry policy makes very little effect on the trading position life time.

Inspecting the return dependence on the stop-loss (Fig.1.2) we see the usual shifting of the maximum (towards lower stop-loss) with the increase in the entry parameter. Apparently those more conservative values increase the relative weight of the situations where tighter stop-loss makes sense -- a welcome development, consistent with presence of intelligence in the forecast signal to which the entry threshold is applied. Comparing with the corresponding figure in the Step Two report, you can notice that, first, the absolute difference in returns among the different entry cases is much larger in Step Two. Second, the maximum returns reached are higher in Step Two -- the extra complexity of Step Two does buy extra returns. Understandably, presence of multiple markets to choose from (Step Two) increases risk in the tight stop-loss sector of trading strategies, so that with the combination of a tight stop loss and conservative entry policy, the difference between the Step One and Step Two modes is the most dramatic: it becomes easier to find a trading opportunity there. Step Two makes the tight stop-loss magenta (very conservative) traders, who typically lose in Step One, profitable. The price for those traders is, as Fig.1.3 reveals, a deterioration in risk vs return standing. (There is nothing paradoxical about this if you recall that you can have a very good -- hard to beat, in fact -- difference of risk and return if you almost never trade and both risk and return approach zero).

Mean position duration vs entry. Back-testing results shown as profile histograms. 2.1 Annualized return vs trade entry parameter. Back-testing results shown as profile histograms. 2.2 RMS of the annualized logarithmic return vs trade entry parameter. Back-testing results shown as profile histograms. 2.3

Fig.2. Trade entry optimization. 2.1: new trade frequency (inverse days) vs trade-entry threshold parameter; 2.2: dependence of the annualized return on the trade-entry parameter; 2.3: same for the figure of merit, logarithmic return minus logarithmic risk. This corresponds to Fig.2 of the recent Step Two report.

The effects of trade-entry threshold are illustrated in Fig.2. The frequency with which new trade ideas are accepted for execution, decreases with the threshold. A high entry threshold makes it more difficult for a trade idea to even get considered, let alone become accepted.

As to the shape of the return dependence on the entry parameter in Fig.2.2, this is where the transition from Step One to Step Two gets interesting: the re-analysis confirms the shift of the maximum return position from entry=0.01 in Step One to 0.016 in Step Two. I stay with the interpretation that in the multi-market money management mode (which is the only difference between Step One and Step Two), there is competition between trade ideas coming from the different markets, and as a result, conservatism brings an additional benefit: being patient pays off as it increases a chance that another market will provide a better return if one is still in the waiting state when the opportunity arises. For this mechanism to work, the algorithm has to be intelligent enough to recognise such an opportunity and value it correctly. What we see is consistent with saying that the algorithm is intelligent enough. Thus, patience secures opportunity.

Logarithmic return minus risk figure of merit vs new trade frequency -- scatter plot

Fig.3. Logarithmic return minus risk (mean minus RMS of monthly annualized logarithmic return) vs new trade frequency, inverse days. Scatter plot view of the data. This corresponds to Fig.3 of the recent Step Two report.

As the new trade frequency drops with the trading style getting more conservative (points of different colors in the plot), the points approach the extreme of no trading: zero trading frequency, zero risk and zero return (it's zero logarithmically as well since logarithm of 1 is 0). This corresponds to the point (0,0) in the plot where the points seem to be heading. Sharpe ratio can be quickly estimated as an exponent of the quantity plotted along the vertical axis. There are visible differences as compared to the analogous figure in the Step Two report. First of all, the high trade frequency values (those above 0.35) are absent here. Those high trade frequency values in Step Two correspond to uncritical trading in the presence of multiple markets to choose from, and their reduction in Step One is understandable. From Fig.3 one can get the impression that Step One (this report) fares well on the "risk-adjusted return" basis, as the magenta points get all the way up to the 0 along the vertical axis of the mean-minus-RMS logarithmic returns. This impression is misleading as those magenta points represent steady losing traders here, while in Step Two there appear winners among them.

The interpretation of Sharpe ratio, as everything in the classical finance, rests on the assumption of normal returns. For a non-normal and asymmetric distribution, such as the distribution of trader's returns, the combination of the first and second moments (the basis for the Sharpe ratio) does not tell the whole story, does not allow one to draw conclusions about probabilities. The best would be to investigate the distribution of trader's returns on a month-to-month basis. In the absence of such a distribution, skewness could be used as a measure of asymmetry. Unfortunately, skewness data is has not been recorded either. In the previous analysis, an attempt was made to measure the asymmetry as the maximum - abs(minimum) of return. Needless to say, this is a ridiculous measure since both maximum and minimum are random quantities which do not estimate any meaningful statistic, they do not converge anywhere as the statistics grows. Neverthess this seems to be as good as any among the solutions available.

Logarithmic return distribution asymmetry vs new trade frequency -- profile plot 4.1 Annualized return vs new trade frequency -- profile plot 4.2

Fig.4.1: Logarithmic return distribution asymmetry (maximum-abs(minimum)) vs new trade frequency, inverse days. 4.2: Mean of the annualized return distribution (1=100%) vs new trade frequency. Profile histogram view of the data. This corresponds to Fig.4 of the recent Step Two report.

As Fig.4 indicates, the distribution of monthly returns is highly asymmetric and skewed in the direction of positive returns. This observation is true for the broadest possible classes of virtual traders. Therefore the risk of negative returns is less than the RMS would imply based on a normal distrubution hypothesis.

Unlike Step Two, the asymmetry keeps growing with trade frequency here. Step Two results look healthier as one would like to see a maximum of positive asymmetry at a comfortably low trade frequency. This preference is driven by risk-adjusted return consideration, since frequent trading strategies are seen in Fig.3 to generate a lot of risk with low return.

For the range of stop-loss parameters I like (green and red), the maximum distribution asymmetry occurs at new trade frequency around 0.05. Given that 6 markets are being traded, this naively translates into a multi-market trade frequency of 0.05×6=0.3 (trades/day). However, Fig.4 of the recent Step Two report shows the most profitable frequency to be 0.2 trades/day. Thus there is a drop in the "most profitable" trade frequency -- which complements an increase in the "most profitable" entry threshold from 0.010 to 0.016. I still believe this is related to the value of patience associated with the multi-market mode of operation.

To conclude, the next report will focus more closely on the return vs risk benefits of Step Two and what trading regimes are the best to realize them.  Higher returns and better quality of Step Two are already seen. A new to-do item has emerged: to obtain distributions of monthly returns for the virtual traders (not to be confused with distributions of traders by return, frequently shown in these reports) and make it part of the performance (risk and return) evaluation process.

Bookmark with:

Deli.cio.us    Digg    reddit    Facebook    StumbleUpon    Newsvine
Last Updated ( Monday, 04 January 2010 12:28 )