Heidi performance review, ten months since the parameter change

User Rating: / 2
PoorBest 
Written by Forex Automaton   
Saturday, 01 December 2012 14:43

Heidi is the hour time-frame trading system generating predictions of hourly high, low and close for the 14 popular currency pairs. This post reviews the summary performance metric of the system, the correlation coefficient between the predicted and actual hourly logarithmic returns. Results are presented graphically.

Data accumulation for the present incarnation of Heidi began on December 1, 2010 and the first predictions (in back-testing mode) were generated for March 16, 2011. The system was announced and began live operation on May 23. The 24-hour trading day is split into 8 seasonal windows and the system parameters can be adjusted separately for each window. There has been only one such adjustment so far, made on January 21, 2012. This affected just one of the seasonality windows, window 6.

As has become customary in this project, I use Pearson correlation coefficient between predicted and real changes in FX quote under study as the main figure of merit on the stage of forecasting optimization.

Correlation coefficients between predicted and real values of the logarithmic difference of hourly close with respect to the previous hour (logarithmic return), vs the intraday time window

Fig.1. Heidi's figure of merit: degree of correlation (Pearson correlation coefficient) between the predicted and real logarithmic price differences at the close of the hour, for the eight intra-day time windows. The correlation coefficients plotted are averaged over the 14 currency pairs tracked by the system. The vertical bars associated with points are measures of uncertainty inferred by comparing results for the 14 currency pairs.

Correlation coefficients between predicted and real values of the logarithmic difference of hourly close with respect to the previous hour (logarithmic return), denoted as C(forecast,reality|close), are plotted in Fig.1 for the 8 time windows, numbered from 0 to 7. The time windows are defined in the output manual of the system.

Central Europe 123456789101112 13141516171819202122230
Eastern US 19202122230123456 789101112131415161718
intra-day time window 0 111 222 333 444 555 666 777 00

Table 1. Definition of intra-day seasons and time zone conversion table. Seasonal time shifts, such as daylight saving time, may complicate the picture if the nations choose to enact them on different days, and are ignored.

Time intervals in Fig.1 are selected in an inclusive way. The positive effect of the January 21 upgrade of window 6 is seen quite clearly. Some degradation of performance is seen in window 2. This could not be due the upgrade as the upgrade only concerned window 6. The rest of windows shows stable performance.

Correlation coefficients between predicted and real values of the logarithmic difference of hourly close with respect to the previous hour (logarithmic return), vs the intraday time window. Data for 14 forex pairs: AUD/USD,EUR/USD, GBP/USD, USD/CAD, USD/CHF, USD/JPY, AUD/JPY, CHF/JPY, EUR/AUD, EUR/CHF, EUR/GBP, EUR/JPY, GBP/CHF, GBP/JPY are shown separately

Fig.2. Same as Fig.1, but the data are shown separately for the 14 currency pairs used by the system.

Fig. 2 is here to give more detail to a user of the Heidi trading system interested in comparing performance of the system for the various currency paris. As you see, time window 7 is currently the best, with positive results for all forex time series except USD/CHF and GBP/USD.

To conclude, I consider "victory condition" for Heidi met as long as the negative performance (or problematic performance, requiring parameter changes) is localized within the known time windows, and there remain time windows with statistically significant (three standard deviations of more) positive correlation between predicted and real hourly returns.

Bookmark with:

Deli.cio.us    Digg    reddit    Facebook    StumbleUpon    Newsvine
Last Updated ( Wednesday, 10 April 2013 17:21 )