EUR/USD 2002-2008: the Fairest Game in Town!

We begin with EUR/USD — the currency pair that keeps making head-lines. We skip the usual charts which the reader can easily find elsewhere and cut to the chase, that is to the specialty of this site — the ability to tell a martingale or a “casino” (or fair game) from the market in which you can profit in a sustainable way.

In this report we focus on the period from 00:00 2002-08-20 to 00:00 2008-02-01 (New York time).

Trend predictability

EUR/USD autocorrelation

Fig.1:Autocorrelation of logarithmic hourly returns in EUR/USD. The time lag is the lag is in “business time” (holidays are excluded).

In this figure we look for obvious arbitrage opportunities on the time scale of up to two days — and conclude that these are rather difficult to reliably locate in this particular currency pair. The hatched red band shows the range of statistical noise (namely its expectation plus minus its RMS deviation). Statistical noise was obtained by simulating 20 independent time series of the length corresponding to that of the EUR/USD series under study, each one constructed to reproduce the measured distribution of returns in EUR/USD for the time period under study (including the fat tails!), but completely devoid of correlations. From these, the expectation and RMS or the autocorrelation amplitude in each time lag bin were calculated. Market’s long-term bias in one direction (in favor of EUR over USD) is seen in the overall positiveness of the Monte Carlo autocorrelation. The level of noise (width of the red band) is dangerously high making it hard to draw reliable conclusions. However, if one is to trust that there is a negative signal for the range of lags from -30 to -14 hours, and a positive one from -14 and up to -6, then the strategy would be to bet on a reversal of the preceding trend which took place during time t-30 through t-14, if supported by dynamics of the past 14 hours. Or in other words if the market went against you 30 to 14 hours ago and changed direction in your favor about 14 hours ago, you are better off betting on such a trend to continue — that’s the dangerously noisy message of this particular autocorrelation.

Even if you can not use this pattern in your favour, not having it against you is part of a smart survival strategy.

24-hour cycle

EUR/USD bullish and bearish autocorrelation

Fig.2: Yellow: correlating only positive hourly returns. Blue: correlating only negative hourly returns.

The striking feature of this plot is the 24 hour cycle of bullish and bearish action, clearly seen as the maxima of the correlation are cleanly located at multiples of the 24 hour lag: 24, 48, 72, 96, 120 hours and so on. Therefore, smart trend following means something more than following a trend that existed in near past. It means following a trend that existed this time of the day yesterday, the day before yesterday, and so on — that gives you better than average chance of winning! Conversely, buying because the currency went up 12 hours ago (or selling because it went down 12 hours ago), all the rest being equal, is the least recommended strategy. (See why the sub-sample correlation feature is not in itself a prediction strategy.)

Note that whether this pattern is equally strong in all time zones is a question that requires a separate study.

Market memory

EUR/USD bullish and bearish autocorrelation long range

Fig.3: Axes and color codes as in the previous figure. Range expanded compared to the previous figure.

The memory effect seen in the previous figure has a certain life time. This figure shows this life time. The effect definitely persists for as long as 1000 hours or well over a month of trading time. It seems that the bullish memories (more pleasant in case of EURUSD) last somewhat longer as one can see more yellow boxes sticking out on the tail at large lag.

Summary

We conclude that EUR/USD for the period 2002-2008 is fairly “efficient” (in the EMH sense) with limited potential in the short term (days). Intelligent trend-following may be a possibility. Long term prospects of this currency pair are the subject of fundamental analysis and are outside the scope of this article. Cross-correlations with other markets are to be discussed in up-coming articles.

RMS

RMS (or RMSD) — Root-Mean-Square Deviation. In accordance with ROOT terminology (a de facto standard in certain circles, for better of worse) RMS denotes Standard Deviation, and it would be better to call it RMSD. Often, RMS does not involve mean subtraction, but in our usage it does.

RMS is a measure of the width of the distribution, corresponding to Sigma in the case of Gaussian distribution.

Cross-correlation

In our usage cross-correlation is a generalization of autocorrelation to a pair of time series:

C(T)=E[x(t),y(t+T)]|over all available t

where T is the time lag. We will also use the more context-specific term intermarket correlation for certain cross-correlations. Just like significant autocorrelation at non-zero time lags lets one predict (in the statistical sense) the process on the basis of its own past history (“auto-” means “self-“), significant cross-correlation at non-zero time lags lets one predict X on the basis of Y (in the same sense with the same caveates) or Y on the basis of X, depending on the sign of T where the signal occurs. For example if you notice that XXX/YYY lags begind UUU/VVV, and you know that UUU/VVV just went up, you can go long on XXX/YYY and have a better than average chance of ending up with a winning trade.

Feel free to browse our collection of cross-correlation data for the forex markets — some of them are highly non-trivial.

Martingale

Fig.1:A martingale market synthesized from hourly returns in AUD/JPY, over time. Time axis is labeled in MM-YY format. By construction, there is no predictable trend other than the long-term trend created by the tiny positive deviation from zero of the average hourly return. Are you a chartist? Do you believe in moving averages or Elliott waves? Do you feel you could day-trade this market? This is a particular example of what we refer to as “fair game” and for all practical purposes take to represent the embodiment of the efficient market hypothesis.

To synthesize such a chart, you first obtain a distribution of returns from the real time series. Then you histogram the returns. Then you start with an arbitrary number (1 was used in this case) and generate a random number according to the distribution of returns you got (a software package such as ROOT lets you do that). Having a starting price and a return, you obtain the next price in the series. You can continue this random walk process as long as you want. It may be counterintuitive to some that a random walk looks like this (Fig.1). Indeed you see a chart where you might be tempted to identify trend lines, points where the trend changes, and possibly even lines of support and resistance. From this standpoint you can understand the source of our Olympian attitude towards all kinds of current news: the pseudo-random market in Fig.1 could generate a very rich stream of news reports and “current analysis” — all totally content-free by construction. Chartists and reporters must admit: tools and concepts that let one distinguish between predictability and randomness are peripheral to their method of operation. However these tools and concepts are central to the Forex Automaton™ approach. Martingale is one of such concepts.

Martingale is a stochastic process (a time series like a forex exchange rate) where an expectation of any element does not depend on the prehistory (although properties of its distribution other than the expectation may depend on the prehistory).

By implication, the expectation for a change in an exchange rate to be recorded just now does not depend on a similar change recorded time T before now. The same would apply to returns or logarithmic returns.

The efficient market hypothesis can be restated to read that time series elements x(t) and x(t+T) are statistically independent variables for any value of T (other than, of course, 0) and therefore the autocorrelation of the time series is zero (except for, of course, at the zero time lag). Therefore a martingale market is the “efficient market”. Thus the efficient market hypothesis becomes falsifiable via observation of correlations.

In “Forecast of future prices, unbiased markets and martingale models” (Journal of Business, V.39, January 1966, 242-255) Benoit Mandelbrot defines martingale in a weaker and a stronger sense: “…to define a martingale, one may begin by postulating that it is possible to speak of a single value for E[Z(t+T)|Z(t)], without having to specify by which past values this expectation is conditioned. In a later stage, one will add the postulate that E[Z(t+T)|Z(t)]=Z(t).” I am bringing this up because the martingales constructed by random sampling from observed distributions of returns will never fall into the “later stage” category since the mean of the observed returns is never exactly zero. But they are certainly constructed to have Z(t+T) unconditioned by any specific past and therefore are martingales in the sense of the initial definition.

AUD/JPY autocorrelation compared with martingale autocorrelation

Fig.2:Autocorrelations of the martingale market from the Fig.1 (red) and of the actual AUD/JPY for the same time period (green).

You might be confused to believe that Fig.1 was a predictable market. But it is the Fig.2, autocorrelation, that tells the difference between actual (predictable to some extent) and pseudo-random behavior.

A spike in AUD/JPY hourly data of the kind that causes the autocorrelation feature specific to this market.

Fig.3:A spike in hourly AUD/JPY is the kind of pattern that causes the autocorrelation feature seen in Fig.2.

The remarkable feature of Fig.2 that distinguishes real data and is not present in the simulation is the downward spikes at one hour lag. Fig.3 illustrates what kind of real-market feature that corresponds to: a market jumping up and down (or down and up) on the hour-by-hour time scale. This happens when markets jump the gun or have a knee-jerk reaction to something, which they later come to regret, figurally speaking. The autocorrelation in Fig.2 tells you that this happens in real-life AUD/JPY a lot more often, or with a lot higher magnitude, than it does in the memory-free simulation (when you do not have memory, you do not regret!). Why does this make the market “predictable”? Because statistically, you can bet on market to regret the knee-jerk and be right more often than not. A forex trading system, such as the one under development here, can be built to continuously analyze the situation, learn the patterns like the one shown, and beat the market with confidence.

Autocorrelation

We use autocorrelation to quantify market inefficiency. The autocorrelation function is an old, common knowledge method which anybody with access to the data can in principle apply. Therefore it is perfect for demonstration in that we do not reveal any proprietary know-how by using it, yet it is quite convincing as it relates directly to the concept of a martingale.

The autocorrelation is defined as the expectation value of the product of the elements of the time series separated by time lag, and is a function of this time lag:

A(T)=E[x(t),x(t+T)]|over all available t

where T denotes the time lag and E is the expectation (averaging operator). Most often we will be dealing with autocorrelation of logarithmic returns. Unless stated otherwise, the time lag we show is the lag in “business time” or in other words, week-end and holiday periods (periods with no data) are excluded.

By construction, an autocorrelation is symmetric around 0. Therefore, plotting only one side (either positive or negative lags) is sufficient. Because in most contexts we talk about prediction, it is more intuitive to plot the negative lag side — that way one can interpret the axis of lags as a time axis, keeping in mind that what is about to happen (and is being predicted) is located at the 0 bin. Although in reality — and this needs to be said for the more rigorous reader — this axis is a diagonal direction of fixed time sums in the two-dimensional space of pairs of time points.

The time processes we know from experience to be predictable, such as the beating of our heart, variation of atmospheric temperature with season, ocean surf and tides, and the like, have informative autocorrelation values at non-zero time lags. Predictability does not imply causality, nor is causality always needed — even though winter does not cause spring, once you know you are in the middle of winter, you can predict that the temperature will be much higher in just a couple of months with good degree of confidence. It is often possible to transform the time series representing the process so that “informative” means non-zero autocorrelation, and non-zero means “informative”. Sometimes this can be done by replacing the original time series by that of increments or ratios (such as logarithmic returns), sometimes by subtracting an autocorrelation of a suitably constructed reference process — a synthetic, usually computer-simulated model of reality which incorporates the features we know about and consider trivial, but not the ones we want to learn about.

Forex Automaton™ has accumulated a collection of non-trivial correlation data for the forex markets. We regard their existence as a sufficient, but not a necessary condition of predictability. In other words, the market can be still predictable with completely trivial two-point autocorrelations (but with e.g. non-trivial genuine three-point correlations — although this does sound like one of those artificial math concoctions). If two-point autocorrelations happen to be non-trivial, that’s a sure sign of predictability — but you can’t count on that. Therefore, we do not suggest building a trading system bottom-up on the basis of autocorrelations (otherwise we would not make them public), or at least this is not how our own trading system was designed and built. But once the autocorrelations are found, ignoring them would be foolish.

Forex (FX)

This document has been written to provide a short introduction for people interested in the forex market. We will try to go through the basic terms and processes and provide some explanations.

Forex, sometimes abbreviated as FX, stands for the foreign exchange market. Money has been around us since ancient times but the forex market started to assume its present form around 1973, when the Bretton-Woods system was abandoned. In 1973, currencies of major industrialized countries became floating and since then, the exchange rates are mainly controlled by the supply and demand.

According to the Triennial Central Bank Survey, released by the Bank for International Settlements in 2007, average daily currency trading volume in 2007 exceeded 3.2 trillion US dollars making forex the biggest market in the world. For comparison, combined volume of all stock markets in the world is 10-15 times smaller.

The biggest difference between forex and other financial markets is that each currency trade consists of simultaneous purchase and sale operations which is why the market is called “foreign exchange market”. The next important definitions are: long and short positions. For example, taking a long position in EUR/USD means buying EUR and selling USD, thus exchanging USD for EUR on the speculation that EUR would appeciate against USD. Here is where the exchange is taking place, the counter (or quote) currency USD is being exchanged into the base currency EUR. Short position means exactly the opposite to the long. Because of this symmetry between long and short, there is always a bull market somewhere in forex no matter what happens with the financial markets at large, and you can benefit from it.

The Forex market has an internal hierarchy, whose tiers differ by the level of access determined by the amount of money traded.

  • The biggest component is the interbank market with a 43% share. The interbank market mainly consists of big investment banking firms. The big companies ensure a big flow of transaction for the significant amount and therefore enjoy a relatively small spread between bid and ask price. As we go down the hierarchy the spread has a tendency to increase as the volume traded decreases. Commercial traders (term used to denote non-speculative participants) and multinational companies are responsible for a big part of the market; they need currency transactions mainly to pay for commodities, goods and services.

  • Central banks control prices, money supply and interest rates and can sometimes intervene in the forex market by selling or buying currencies when the exchange rates rates are too high or too low which in general has a short term effect.

  • The next level is investment management firms, which mainly trade on behalf of the clients.

  • Finally, retail forex broker is at the bottom level of the market with the smallest market share.

There are three global forex sessions in the world: Asian-Pacific, European and North-American. Due to the time-zone layout of the financial centers the Forex market is literally open 24 hours during the business week. It’s open from the early Sunday hours (Asian-Pacific session) and closed on Friday in North-American session. Main trading currencies are EUR (Euro), GBP (British Pound), CHF (Swiss franc), JPY (Japanese Yen), AUD (Australian Dollar), NZD (New-Zealand Dollar) CAD (Canadian Dollar). They form exchange rate pairs against the USD (US Dollar) and cross-rates among themselves. The biggest moves on the market are tied to the data and news releases and often happen in the morning of the particular session. One of the best market characteristics is liquidity of the market, liquidity can be understood as trading volume. Liquidity varies from session to session and within the trading session. Liquidity usually peaks during in the overlap hours of the European and North-American sessions.

Currencies and other financial markets are highly correlated, among the most influential markets are: gold, oil, stocks and bonds. For example, gold is highly anti-correlated to the USD, oil price is often considered indicative of the inflation and growth expectations, and so on. The bottom line is that other financial markets influence currencies and the big picture has to be closely followed.

Opening a position through a retail broker requires maintaining a certain margin balance which is often defined by the broker. The ratio of margin balance to open positions has to be maintained all the time to avoid automatic positions liquidation which will lock your losses in and result in your margin balance decline. Most brokers provide real-time estimations of margin balance which is referred to as mark-to-market. That estimation shows trader’s unrealized P&L (profit and loss) and reflects would-be the output of the trade if the position were closed at this particular point in time. Realized P&L enters the margin balance once the position(s) has been closed. If a trader does not have an open position in the market such a state is called square or flat and if a trader wants to close an open position it’s often referred to as squaring up.

Before starting to operate in the forex market one has to have a trading plan. A trading plan is an organized approach to execute trading strategy, that has been developed based on the market analysis and outlook. An important part of the market analysis framework can be a which not only helps traders to make decisions and increase their profit but also allows them to lower the psychological pressure. Usually the system generates trading signals which can be easily understood and used to make decisions by the trader. Many traders use self-developed systems. Also there is a huge market of commercially developed trading systems with the price tag in the range from a couple of hundred dollars up to several hundred thousand.

Book: The Predictors. By T.A.Bass.

The Predictors : How a Band of Maverick Physicists Used Chaos Theory to Trade Their Way to a Fortune on Wall Street. T.A.Bass

This is a non-fiction book. The physicists are chaos theorists Doyne Farmer and Norman Packard — but this is not a chaos text. In fact, there is little technical information in the book. It’s more about the business side of starting a financial prediction company. To quote: “They begin an animated discussion about some of the basic ideas that will later be used at Prediction Company. The technique involves fitting functions to time series. Imagine a stream of data, a record of daily sunspot activity, for example. The sunspot activity on two successive days is two numbers, which can be represented as points in a plane. The sunspot activity on the next day can be thought of as a point suspended above this plane. The picture of sunspot activity for a year will be a cloud of points in three dimensional space. To make a forecasting model, one fits a surface, like a rumpled sheet, through this cloud of data. If the sheet fits in close proximity to all the data points, it will be a good model. Forecasts are made by stretching the sheet into regions where additional data points will be recorded when the future rolls into the present”. The idea is in the air… This description is recognisable to someone who has independently arrived at something similar — but probably not informative to the rest. What you want to do is data compression of some sort, and there are more conservative compression tools than a fit. One may  object to the business model of striking an exclusive deal with a single large client, as opposed to marketing the service to thousands of individual traders. Overall, valuable historic reading to someone with similar interests.

Efficient Market Hypothesis

“You can’t beat the market”.

Textbooks present three forms of this statement. To quote McCauley (who gives credit for these to “finance theorists”): “Weak form: it’s impossible to develop trading rules to beat market averages based on empirical price statistics. Semi-strong form: it’s impossible to obtain abnormal returns based on the use of any publicly available information. Strong form: it’s impossible to beat the market consistently by using any information, including insider information.”

The strong form makes a person with a natural science education background wonder whether this is a “law of nature” or a “law” enforced by “law enforcement”. The weak form seems to be formulated in a falsifiable way. It is this form that can be falsified in spectacular ways by using world’s most liquid market — the forex market.

Book: The Crowd. G. le Bon

The Crowd. G. le Bon

A book whose relevance seems to only grow with time since its first publication in 1898. Le Bon’s “crowd” is in most examples a group of people at the same physical location, seeing and hearing each other immediately, and responding to the stimuli they immediately observe. These days there emerges a global crowd interconnected by modern means of communication unheard of in le Bon’s age. As both the crowd and its scope of attention are thus expanded, the basic dynamics likely remains the same, as does the human nature. The relevance of the book grows with the crowd phenomenon it describes — that is, in proportion to the growth of the mass media and means of communication.

Here at ForexAutomaton.com where our business is to turn mood swings of the financial market crowd into profit opportunities, we cannot but add this book to our bookshelf (if not keep it on our desktop, as did Lenin according to some sources). What le Bon does using the methodology of the social siences, we do using that of the precise sciences. He describes, we measure. If you think markets become more fficient as their liquidity grows, think again or read le Bon: a larger crowd is still a crowd. And a crowd is driven by its own dynamics, and is less rational — and possibly is easier to predict and control — than any of its individul members, no matter who they are. This applies to a crowd of nineteenth century’s leading scientists in one of le Bon’s anecdotes as it may apply to a crowd of Wall Street CEOs in what created the subprime crisis episode of 2007-200?.

Logarithmic Returns

For a price time series p(t), discrete with a time increment dt, a logarithmic return variable is

x(t|dt) = log(p(t)/p(t-dt))

where dt is the time increment separating adjacent points in the time series.

This variable has several advantages. It is additive: the return of the entire series is the sum of the returns comprising the series:

x(tn|tn-t1) = x(t2|dt) + … + x(tn|dt),

dt = (tn-t1)/(n-1)

Non-negativity of the price is “built in” — especially useful when simulating artificial time series.

When used in the correlation analysis, logarithmic returns (as do ordinary returns p(t)/p(t-dt)) eliminate one trivial source of non-stationarity of the correlation functions which is the possible long time-scale trend in the time dependence of the price. Long-term absolute level of the price is almost irrelevant to a forex trader, what matters is relative movements.

Finally, the moments of the logarithmic returns may converge better than they would for the ordinary returns — although, notably, Mandelbrot postulated that the variance of this variable would be infinite.