

ForexAutomaton 2013. The Fifth Annual Summary of Researh Progress 
Written by Forex Automaton  
Sunday, 14 April 2013 16:53  
Success of algorithmic trading has as much to do with the market as with the nature of the algorithm. While the algorithms may vary and may have terms such as artificial intelligence, adaptivity, machine learning, neural networks and the like associated with them, the phenomenology of what it is that the machines are "learning" all comes down to correlation measurements of varying order  pairs, triplets, quadruplets...  over a variety of time lags and time scales, performed on the market time series. ForexAutomaton began its life on April 02, 2008, five years ago. Year V was exceptionally productive for the project. Here are the main highlights of this annual review:
FX Market Diagnostics: In The Wake Of The Year 2007 Regime SwitchCERCSI or the Currency Exchange Rate Correlation Strength Index is one of the quantities monitored monthly in the Forex Correlations section of the site. CERCSI measures the absolute magnitude (deviation from zero in either positive or negative direction) of correlation strength on the hourly scale among the 14 leading spot FX rates. The quantity is computed monthly and includes a month of hourly data. By construction, CERCSI is a quantity bounded between 0 and 1.
Fig. 1 shows that a change in the correlation regime with a dramatic increase in the absolute strength of intermarket correlations occured in Summer of 2007 and apparently was triggered by the subprime mortgage crisis in the US. CERCSI signaled an onset of the new regime months before the volatility spike of 2008. From the CERCSI point of view, we are still more or less in the same regime, despite the fact that the volatility has returned to normal. As a reminder, CERCSI (just like CERPI and volatility) is insensitive to the absolute level of prices and works only with logarithmic differentials thereof. A high absolute magnitude of correlations indicates a degeneration in the variety of market instruments, or rather, of their perception by the market participants. Everything degenerates into "safe haven" and "risk assets", and effectiveness of portfolio diversification is thus reduced. And when diversification stops working, portfolio managers realize that they can not maintain prior risk profiles of their portfolios without selling risk assets or shorting them. Which is why high CERCSI may signal trouble for investors as it did in 2007. The fact that algorithmic trading does or does not work may have to do with the market as well as with the nature of the algorithm. While the algorithms may vary and may have terms such as artificial intelligence, adaptivity, machine learning, neural networks and the like associated with them, the phenomenology of what it is that the machines are "learning", and whether there is anything to learn, all comes down to correlation measurements of varying order (pairs, triplets, quadruplets and so on), over a variety of time lags, and over a variety of time scales. The zero benchmark of the hypothetical efficient market corresponds to the absence of nontrivial correlation effects at nonzero time lags. Our monthly Forex Correlation reviews are currently limited to second order, or pairwise correlation effects, and include autocorrelations and intermarket correlations with nonzero time lags. The simplest benchmark of market predictability is the one based on such correlations, and it is not difficult to construct. CERPI is just such a quantity, and its chart is shown in Fig.2. CERPI, computed on a given time scale, is not sensitive to all types of predictability effects on all time scales, but nevertheless it provides a useful metric. Fig.2 shows that after the Regime Switch in Summer 2007, the FX markets do occasionally deliver months with high values of CERPI, although perhaps in a less reliable manner than before: the positive extremes reach higher but they are also less frequent. After this general quantitative characterization of the conditions we are in, we continue with the brief reviews of all three trading systems currently deployed: Heidi, Demi and Danica. We will conclude with the Twisted Pair idea so recently implemented for Heidi and Danica. HeidiHeidi is the predictiononly system operating on the hourly time scale publicly since August 11, 2010. In May of 2011, the system was upgraded to version 2, but the upgrade did not affect the forecasting concepts or parameters. A parameter upgrade was made on January 21, 2012. This affected just one of the seasonality windows, window 6. After that, the performance in window 6 was seen to have improved.
Overall, version 2 of Heidi looks like a realistically successful system, leaving aside the question of whether these positive Pearson correlations of forecast with reality can allow the user to beat the bidask spread given the time frame of one hour on which the system operates. But behind this success lies a rather crude implementation of the meanreversion model: take the "raw" forecast and flip the sign. The Twisted Pair mechanism, to be described, provides a much more sophisticated alternative. Version 3 of Heidi, with Twisted Pair, was launched on April 12, 2013. DemiDemi's origin and design has been discussed in great detail in one of the earlier Annual Reports. In brief, the idea behind this system is this: knowing that our prediction for a change in daily high is correlated with the actual change, and knowing that the prediction for a change in daily low is correlated with the actual change, we bet on a coincidence of the successful predictions for these two changes. Note that the existence of such a coincidence is, strictly speaking, a separate matter from the individual successes of both types of predictions. What makes this better than gambling is positiveness of the fourthorder cumulant among the forecasts and real changes in both high and low. We bet on the coincidence by opening a trade with a stoploss and a profit target. The stop loss and the profit target are the previous day's high and low, and the direction of the trade is long if today's low and high are expected to be above yesterday's; short if they are expected to be below yesterday's. Note the requirement of simultaneity: in a situation when high and low are expected to evolve in the opposite directions, a trade is not submitted. The four instances of Demi, differing by the hour of the decision making (24hour day definition) systems went live on August 25, 2010. In the following, the systems will be distinguished according to the hour of decision making during the day (the end and beginning of the day): 3, 5, 9 and 11 (all in the AM). System performance data presented below describe the situation as of March 30, 2013. This should be compared with the the previous Annual Report. Trading costs are simulated as bidask spread, with the spread being equal to a fixed fraction of price (see link for details about the simulation of the spread).
Table 3 gives a birdseye picture of Demi's progress for the year since the last report. This is a sum total of capital under management and accumulated P&L in quote units over the four systms. Both statistics reflect trading costs, but the P&L in quote units is a much simpler statistic, not reflecting capital allocation decisions (leverage) which differ trade by trade. Demi has suffered a 5.7% loss this year. Danica. The AntiSystem.The Danica model has been making daily forecasts on this site for three years. I said it before and will say it again... When launching the model, I expected one of two outcomes: either the model would prove successful by generating a statistically significant, even if "small", positive correlation with reality, in which case it would be declared a success  or it would look statistically uncorrelated with reality, in which case it would be no better than a random number generator, with no merit as a forecasting tool. The reality proved to be quite different from both of these scenarios. In Fig.4, the correlation coefficient is the quantity given in the Danica monthly reports, and is, by our adapted standard, the Pearson correlation coefficient over the last 24 business days of the model's operation, as reported in the last forecast of the month. The important thing is that the values of such coefficient for the different months (the different bars in Fig.3) are statistically independent. The actual outcome of over three years of live operation of the system is neither of the two naively expected scenarios: the model is neither systematically right nor worthless. The model forecasts have been systematically opposite to what happens later in reality. The data are not consistent with the academic notion of an hypothetic efficient market, understood as a guaranteed source of mathematically random, symmetric gains and losses, irrespective of the trading strategy. In the last year's report, I wrote: "The FX market, from Danica's point of view, seems to be a willful entity with metastable patterns of behaviour which are replaced by other metastable patterns. The ability of the system to convert input (FX quote) data with no visible twopoint correlation on the day scale (speaking about the direction of closing price), into a stream of output data (trading gain or loss) with visible positive autocorrelation (regadless of the sign of gain or loss itself) is an important achievement, as the problem becomes that of coming up with a transformation to change the broadly autocorrelated, even though stochastic and bipolar, quantity, into a positive one. One could think of some afterburner to Danica which would decide whether to treat Danica as a "system" or as an "antisystem" at this point in time." The "afterburner" was indeed the kind of solution adapted for Heidi 2, and its relative success has been already discussed in the context of that model. For Heidi 3 and Danica 2, a different, much deeper and more elegant solution, nicknamed Twisted Pair, has been implemented. Twisted PairTwisted Pair is a particular transformation applied to the input data. Naturally, a corresponding transformation is applied to the output data of the algorithm. The proof of principle for the Twisted Pair has been obtained with simulated data and is illustrated in Fig.5.
By mean reversion I will mean the tendency of the price time series to form quickly alternating rises and falls, more pronounced than in a fully unpredictable time series of the same volatility. In an intuitive sense, such a time series looks as if a rise during this hour is caused by the previous hour's fall, or this hour's fall is caused by the previous hour's rise, since rises and falls tend to cluster in an alternating fashion. Mathematically, using correlations, a manifestation of this behaviour would be an autocorrelation of (logarithmic) returns looking like a dumped oscillation. Meanreverting markets can be traded algorithmically or intuitively once the characteristic time scale (period of reversion) has been reconstructed or otherwise recognized. On an intuitive level, the trading involves the psychological attitude known as "being a contrarian". Pair trading involves artificial market instruments (pairs) whose meanreversion properties are enhanced, compared to those of the ingredients. Trending markets are the opposite extreme and present a situation psychologically comfortable to many people, where you can bet on the continuation of a trend, once the trend has been recognized. Mathematically, trends look like broad positive correlations of (logarithmic) returns. The simulated time series and the method used for synthesizing them have been discussed in the ForexAutomaton report of October 28, 2012. For the ForexAutomaton systems, the challenge has been to recognize the meanreversion pattern "natively". Twisted Pair was the first algorithm that solved this problem, as seen in Fig.5. The system has no prior knowledge of what kind of time series would be fed into it. Yet, in both cases, positive correlation of forecasts with reality is obtained. Curiously, the optimization curves in Fig.5 look like mirror reflections of one another. Possibly, this is a reflection of the deep inner opposition and thus complimentarity of the two basic patterns, that of trend and that of meanreversion. How adequate is this solution? How to take full advantage of its potential? The new incarnations of Danica and Heidi will attempt to provide material for answering these questions. Web Site AccessAt present, Danica is in public access and accessing Demi and Heidi requires a free registration. I intend to keep providing these services free for another year. Should that change, preregistered users will be granted a free year of access from the moment the free access is terminated. 

Last Updated ( Friday, 12 June 2015 15:51 ) 