Cross-correlation

In our usage cross-correlation is a generalization of autocorrelation to a pair of time series:

C(T)=E[x(t),y(t+T)]|over all available t

where T is the time lag. We will also use the more context-specific term intermarket correlation for certain cross-correlations. Just like significant autocorrelation at non-zero time lags lets one predict (in the statistical sense) the process on the basis of its own past history (“auto-” means “self-“), significant cross-correlation at non-zero time lags lets one predict X on the basis of Y (in the same sense with the same caveates) or Y on the basis of X, depending on the sign of T where the signal occurs. For example if you notice that XXX/YYY lags begind UUU/VVV, and you know that UUU/VVV just went up, you can go long on XXX/YYY and have a better than average chance of ending up with a winning trade.

Feel free to browse our collection of cross-correlation data for the forex markets — some of them are highly non-trivial.

Martingale

Fig.1:A martingale market synthesized from hourly returns in AUD/JPY, over time. Time axis is labeled in MM-YY format. By construction, there is no predictable trend other than the long-term trend created by the tiny positive deviation from zero of the average hourly return. Are you a chartist? Do you believe in moving averages or Elliott waves? Do you feel you could day-trade this market? This is a particular example of what we refer to as “fair game” and for all practical purposes take to represent the embodiment of the efficient market hypothesis.

To synthesize such a chart, you first obtain a distribution of returns from the real time series. Then you histogram the returns. Then you start with an arbitrary number (1 was used in this case) and generate a random number according to the distribution of returns you got (a software package such as ROOT lets you do that). Having a starting price and a return, you obtain the next price in the series. You can continue this random walk process as long as you want. It may be counterintuitive to some that a random walk looks like this (Fig.1). Indeed you see a chart where you might be tempted to identify trend lines, points where the trend changes, and possibly even lines of support and resistance. From this standpoint you can understand the source of our Olympian attitude towards all kinds of current news: the pseudo-random market in Fig.1 could generate a very rich stream of news reports and “current analysis” — all totally content-free by construction. Chartists and reporters must admit: tools and concepts that let one distinguish between predictability and randomness are peripheral to their method of operation. However these tools and concepts are central to the Forex Automaton™ approach. Martingale is one of such concepts.

Martingale is a stochastic process (a time series like a forex exchange rate) where an expectation of any element does not depend on the prehistory (although properties of its distribution other than the expectation may depend on the prehistory).

By implication, the expectation for a change in an exchange rate to be recorded just now does not depend on a similar change recorded time T before now. The same would apply to returns or logarithmic returns.

The efficient market hypothesis can be restated to read that time series elements x(t) and x(t+T) are statistically independent variables for any value of T (other than, of course, 0) and therefore the autocorrelation of the time series is zero (except for, of course, at the zero time lag). Therefore a martingale market is the “efficient market”. Thus the efficient market hypothesis becomes falsifiable via observation of correlations.

In “Forecast of future prices, unbiased markets and martingale models” (Journal of Business, V.39, January 1966, 242-255) Benoit Mandelbrot defines martingale in a weaker and a stronger sense: “…to define a martingale, one may begin by postulating that it is possible to speak of a single value for E[Z(t+T)|Z(t)], without having to specify by which past values this expectation is conditioned. In a later stage, one will add the postulate that E[Z(t+T)|Z(t)]=Z(t).” I am bringing this up because the martingales constructed by random sampling from observed distributions of returns will never fall into the “later stage” category since the mean of the observed returns is never exactly zero. But they are certainly constructed to have Z(t+T) unconditioned by any specific past and therefore are martingales in the sense of the initial definition.

AUD/JPY autocorrelation compared with martingale autocorrelation

Fig.2:Autocorrelations of the martingale market from the Fig.1 (red) and of the actual AUD/JPY for the same time period (green).

You might be confused to believe that Fig.1 was a predictable market. But it is the Fig.2, autocorrelation, that tells the difference between actual (predictable to some extent) and pseudo-random behavior.

A spike in AUD/JPY hourly data of the kind that causes the autocorrelation feature specific to this market.

Fig.3:A spike in hourly AUD/JPY is the kind of pattern that causes the autocorrelation feature seen in Fig.2.

The remarkable feature of Fig.2 that distinguishes real data and is not present in the simulation is the downward spikes at one hour lag. Fig.3 illustrates what kind of real-market feature that corresponds to: a market jumping up and down (or down and up) on the hour-by-hour time scale. This happens when markets jump the gun or have a knee-jerk reaction to something, which they later come to regret, figurally speaking. The autocorrelation in Fig.2 tells you that this happens in real-life AUD/JPY a lot more often, or with a lot higher magnitude, than it does in the memory-free simulation (when you do not have memory, you do not regret!). Why does this make the market “predictable”? Because statistically, you can bet on market to regret the knee-jerk and be right more often than not. A forex trading system, such as the one under development here, can be built to continuously analyze the situation, learn the patterns like the one shown, and beat the market with confidence.

Autocorrelation

We use autocorrelation to quantify market inefficiency. The autocorrelation function is an old, common knowledge method which anybody with access to the data can in principle apply. Therefore it is perfect for demonstration in that we do not reveal any proprietary know-how by using it, yet it is quite convincing as it relates directly to the concept of a martingale.

The autocorrelation is defined as the expectation value of the product of the elements of the time series separated by time lag, and is a function of this time lag:

A(T)=E[x(t),x(t+T)]|over all available t

where T denotes the time lag and E is the expectation (averaging operator). Most often we will be dealing with autocorrelation of logarithmic returns. Unless stated otherwise, the time lag we show is the lag in “business time” or in other words, week-end and holiday periods (periods with no data) are excluded.

By construction, an autocorrelation is symmetric around 0. Therefore, plotting only one side (either positive or negative lags) is sufficient. Because in most contexts we talk about prediction, it is more intuitive to plot the negative lag side — that way one can interpret the axis of lags as a time axis, keeping in mind that what is about to happen (and is being predicted) is located at the 0 bin. Although in reality — and this needs to be said for the more rigorous reader — this axis is a diagonal direction of fixed time sums in the two-dimensional space of pairs of time points.

The time processes we know from experience to be predictable, such as the beating of our heart, variation of atmospheric temperature with season, ocean surf and tides, and the like, have informative autocorrelation values at non-zero time lags. Predictability does not imply causality, nor is causality always needed — even though winter does not cause spring, once you know you are in the middle of winter, you can predict that the temperature will be much higher in just a couple of months with good degree of confidence. It is often possible to transform the time series representing the process so that “informative” means non-zero autocorrelation, and non-zero means “informative”. Sometimes this can be done by replacing the original time series by that of increments or ratios (such as logarithmic returns), sometimes by subtracting an autocorrelation of a suitably constructed reference process — a synthetic, usually computer-simulated model of reality which incorporates the features we know about and consider trivial, but not the ones we want to learn about.

Forex Automaton™ has accumulated a collection of non-trivial correlation data for the forex markets. We regard their existence as a sufficient, but not a necessary condition of predictability. In other words, the market can be still predictable with completely trivial two-point autocorrelations (but with e.g. non-trivial genuine three-point correlations — although this does sound like one of those artificial math concoctions). If two-point autocorrelations happen to be non-trivial, that’s a sure sign of predictability — but you can’t count on that. Therefore, we do not suggest building a trading system bottom-up on the basis of autocorrelations (otherwise we would not make them public), or at least this is not how our own trading system was designed and built. But once the autocorrelations are found, ignoring them would be foolish.

Forex (FX)

This document has been written to provide a short introduction for people interested in the forex market. We will try to go through the basic terms and processes and provide some explanations.

Forex, sometimes abbreviated as FX, stands for the foreign exchange market. Money has been around us since ancient times but the forex market started to assume its present form around 1973, when the Bretton-Woods system was abandoned. In 1973, currencies of major industrialized countries became floating and since then, the exchange rates are mainly controlled by the supply and demand.

According to the Triennial Central Bank Survey, released by the Bank for International Settlements in 2007, average daily currency trading volume in 2007 exceeded 3.2 trillion US dollars making forex the biggest market in the world. For comparison, combined volume of all stock markets in the world is 10-15 times smaller.

The biggest difference between forex and other financial markets is that each currency trade consists of simultaneous purchase and sale operations which is why the market is called “foreign exchange market”. The next important definitions are: long and short positions. For example, taking a long position in EUR/USD means buying EUR and selling USD, thus exchanging USD for EUR on the speculation that EUR would appeciate against USD. Here is where the exchange is taking place, the counter (or quote) currency USD is being exchanged into the base currency EUR. Short position means exactly the opposite to the long. Because of this symmetry between long and short, there is always a bull market somewhere in forex no matter what happens with the financial markets at large, and you can benefit from it.

The Forex market has an internal hierarchy, whose tiers differ by the level of access determined by the amount of money traded.

  • The biggest component is the interbank market with a 43% share. The interbank market mainly consists of big investment banking firms. The big companies ensure a big flow of transaction for the significant amount and therefore enjoy a relatively small spread between bid and ask price. As we go down the hierarchy the spread has a tendency to increase as the volume traded decreases. Commercial traders (term used to denote non-speculative participants) and multinational companies are responsible for a big part of the market; they need currency transactions mainly to pay for commodities, goods and services.

  • Central banks control prices, money supply and interest rates and can sometimes intervene in the forex market by selling or buying currencies when the exchange rates rates are too high or too low which in general has a short term effect.

  • The next level is investment management firms, which mainly trade on behalf of the clients.

  • Finally, retail forex broker is at the bottom level of the market with the smallest market share.

There are three global forex sessions in the world: Asian-Pacific, European and North-American. Due to the time-zone layout of the financial centers the Forex market is literally open 24 hours during the business week. It’s open from the early Sunday hours (Asian-Pacific session) and closed on Friday in North-American session. Main trading currencies are EUR (Euro), GBP (British Pound), CHF (Swiss franc), JPY (Japanese Yen), AUD (Australian Dollar), NZD (New-Zealand Dollar) CAD (Canadian Dollar). They form exchange rate pairs against the USD (US Dollar) and cross-rates among themselves. The biggest moves on the market are tied to the data and news releases and often happen in the morning of the particular session. One of the best market characteristics is liquidity of the market, liquidity can be understood as trading volume. Liquidity varies from session to session and within the trading session. Liquidity usually peaks during in the overlap hours of the European and North-American sessions.

Currencies and other financial markets are highly correlated, among the most influential markets are: gold, oil, stocks and bonds. For example, gold is highly anti-correlated to the USD, oil price is often considered indicative of the inflation and growth expectations, and so on. The bottom line is that other financial markets influence currencies and the big picture has to be closely followed.

Opening a position through a retail broker requires maintaining a certain margin balance which is often defined by the broker. The ratio of margin balance to open positions has to be maintained all the time to avoid automatic positions liquidation which will lock your losses in and result in your margin balance decline. Most brokers provide real-time estimations of margin balance which is referred to as mark-to-market. That estimation shows trader’s unrealized P&L (profit and loss) and reflects would-be the output of the trade if the position were closed at this particular point in time. Realized P&L enters the margin balance once the position(s) has been closed. If a trader does not have an open position in the market such a state is called square or flat and if a trader wants to close an open position it’s often referred to as squaring up.

Before starting to operate in the forex market one has to have a trading plan. A trading plan is an organized approach to execute trading strategy, that has been developed based on the market analysis and outlook. An important part of the market analysis framework can be a which not only helps traders to make decisions and increase their profit but also allows them to lower the psychological pressure. Usually the system generates trading signals which can be easily understood and used to make decisions by the trader. Many traders use self-developed systems. Also there is a huge market of commercially developed trading systems with the price tag in the range from a couple of hundred dollars up to several hundred thousand.

Efficient Market Hypothesis

“You can’t beat the market”.

Textbooks present three forms of this statement. To quote McCauley (who gives credit for these to “finance theorists”): “Weak form: it’s impossible to develop trading rules to beat market averages based on empirical price statistics. Semi-strong form: it’s impossible to obtain abnormal returns based on the use of any publicly available information. Strong form: it’s impossible to beat the market consistently by using any information, including insider information.”

The strong form makes a person with a natural science education background wonder whether this is a “law of nature” or a “law” enforced by “law enforcement”. The weak form seems to be formulated in a falsifiable way. It is this form that can be falsified in spectacular ways by using world’s most liquid market — the forex market.

Logarithmic Returns

For a price time series p(t), discrete with a time increment dt, a logarithmic return variable is

x(t|dt) = log(p(t)/p(t-dt))

where dt is the time increment separating adjacent points in the time series.

This variable has several advantages. It is additive: the return of the entire series is the sum of the returns comprising the series:

x(tn|tn-t1) = x(t2|dt) + … + x(tn|dt),

dt = (tn-t1)/(n-1)

Non-negativity of the price is “built in” — especially useful when simulating artificial time series.

When used in the correlation analysis, logarithmic returns (as do ordinary returns p(t)/p(t-dt)) eliminate one trivial source of non-stationarity of the correlation functions which is the possible long time-scale trend in the time dependence of the price. Long-term absolute level of the price is almost irrelevant to a forex trader, what matters is relative movements.

Finally, the moments of the logarithmic returns may converge better than they would for the ordinary returns — although, notably, Mandelbrot postulated that the variance of this variable would be infinite.