The 2008 commodity price boom: did speculation play a role?

Miner Econ (2012) 25:17–28 DOI 10.1007/s13563-011-0006-5 ORIGINAL PAPER The 2008 commodity price boom: did speculation play a role? Olle Östensson Received: 23 March 2011 / Accepted: 14 June 2011 / Published online: 8 July 2011 # Springer-Verlag 2011 Abstract In the wake of the commodity boom and bust, a number of articles and reports have made the case that the surge in commodity prices and the subsequent fall were caused by speculation, particularly by index funds. This claim is reviewed and an overview of the arguments is presented. By way of background, particular features of commodity markets in general and of futures markets in particular are reviewed. It is argued that some of the support for the speculation hypothesis derives from a failure to appreciate that price spikes have always formed a feature of commodity markets and that commodity markets have a characteristic that does not exist to the same extent in other markets, namely the possibility to sell short. Moreover, proponents of the speculation hypothesis do not offer any credible explanation of how activities by index funds on futures markets led to rising prices in markets subject to backwardation. Some simple statistical and anecdotal evidence is presented that contradicts the speculation hypothesis. Finally, published studies are briefly reviewed, and it is concluded that most authoritative econometric studies reject the speculation hypothesis. Those that do not, suffer from shortcomings with respect to methodology or are inconclusive. It therefore appears that there is very limited basis for the arguments that the nature of commodity markets has changed or that speculators, particularly index funds, were responsible for the commodity price increases in 2008. Keywords Trade . Commodity . Price spike . Speculation . Index funds O. Östensson (*) Caromb Consulting, Caromb, France e-mail: olleostensson@gmail.com Introduction From 2002 to 2008, world commodity markets experienced a dramatic increase in demand and spectacular price rises. The price rise was interrupted for some commodities when the United States and Europe entered recession in the last quarter of 2007, but prices of others continued rising a little longer until mid-2008. As seen from Fig. 1, the steep downturn in the world economy in the autumn of 2008 exacerbated the already initiated fall in commodity demand and resulted in severe declines in prices. In the wake of the commodity boom and bust, a number of articles and reports have appeared that attempt to make the case that the surge in commodity prices and the subsequent fall were caused by speculation. Specifically, they argue that the nature of commodity markets has changed as a result of financial investors redirecting funds into these markets in order to diversify portfolios and offset declining yields of stock and bond markets. This “financialization” of commodity markets led, it is argued, to steeper price rises than would otherwise have been the case, and to a steeper fall once prices started falling. In the following, this claim is reviewed. The aim here is not to provide a detailed statistical or econometric analysis of events, but to present an overview of the arguments in a form that is hopefully accessible to non-specialists. One of the reasons for thinking this may be useful is that the arguments of the proponents of the speculation hypothesis appear to have prevailed among non-experts, as well as in commodity trading circles, in spite of it having been rejected in most of the authoritative statistical and econometric studies carried out. This is understandable in view of the intuitively convincing nature of some of the arguments used by supporters of the speculation hypothesis. By way of contrast, academic economists and financial market 18 Fig. 1 Commodity price indices, by group, 2002-May 2009. Source: UNCTAD Commodity Price Bulletin regulators who have concluded that the hypothesis should be rejected have not made their views—admittedly based on less accessible reasoning—widely known. While the scope of the article is applicable to all commodities and uses examples for a broad range of products, the market for crude oil is often referred to. There are three reasons for this. First, the rise and subsequent fall in crude oil prices attracted a great deal of attention and for many people, this was probably the only aspect of the commodity price boom that they noticed. Second, the crude oil market is broadly representative of commodity markets both with respect to price formation and interaction with the rest of the economy. Third, the quality and availability of data for crude oil are better than those for most commodities. It should be noted that the discussion in the following is concerned with events during the period leading up to, and including, the peaks in commodity prices reached during the 2007–2008 period. Interesting developments have taken place in commodity markets since then, but they are outside the scope of this article, mainly because it would otherwise grow to impractical dimensions. The section, “The financial speculation argument”, sets out the arguments used by the proponents of the speculation explanation. The section “The nature of commodity markets” provides some basic descriptions of commodity markets. It is concluded from the description that the speculation argument is to some extent based on false analogies with other markets. The section “Some simple statistical evidence“reviews some simple statistical evidence, and the section “Brief review of published studies” provides a brief review of published studies, concluding the basic examination of the arguments. Finally, the “Conclusions” section sets out some conclusions. The financial speculation argument As already mentioned, this article focuses on the argument that financial speculation, particularly so called index funds, and the “financialization” of commodity markets O. Östensson caused the dramatic price increases in 2008, both since it has occupied the centre of the debate and because it is claimed to represent a new feature of commodity markets, appearing along with the 2002–2008 commodities boom market. An additional reason is that adherents of the argument have gone on to propose actions to deal with the perceived problem.1 These actions would obviously not be necessary or advisable if the importance of the problem is overstated or if it does not exist. While the debate was almost exclusively occupied with the effect of index funds, there were certainly instances of speculation by informed speculators2 and hoarding during the steep upturn in early 2008 by market operators who followed more or less elaborate strategies. Their role in the commodity price upturn ending in 2008 has not attracted a great deal of attention and appears to have been considered as less important than that of “new” types of financial investors, specifically, the index funds. Accordingly, the discussion in the following focuses on these funds and their activities. A few words of explanation are necessary for those who are not acquainted with commodity index funds. Index funds were created to meet the needs of investors who had noticed that commodity prices were increasing and were interested in profiting from the opportunity. However, they did not have the market knowledge to trade individual commodities and they would like to spread their risks. Accordingly, they bought “commodity indices”. Such indices, which are sold by “swap dealers”, mainly banks and other financial institutions, are financial products intended to reflect changes in commodity prices. Their composition roughly reflects the economic or trading importance of different commodities (with a consequent high weighting for oil, for instance). Since the sellers of these indices do not want to lose on the deal, they hedge their exposure by buying futures contracts for the commodities in the same proportions as the index. They do so by buying a futures contract for delivery in, for instance, 3 months, which they sell 2 months 1 Thus, UNCTAD argues that “the international economic system would gain coherence if new efforts were made at the multilateral level to contain price fluctuations on international commodity markets while allowing for smooth price adjustments that reflect market fundamentals and structural changes, for example in connection with climate change.” (UNCTAD 2008, p. 46). 2 Modern finance theory distinguishes between informed and uninformed speculation. According to this view, informed speculation is the channel through which private information becomes impounded in publicly quoted prices. Uninformed speculation should either not have such effects, or in less liquid markets, should not have persistent effects. If uninformed trades do move a market price away from its fundamental value, informed traders, who know the fundamental value of the asset, will take advantage of the profitable trading opportunity with the result that the price will return to its fundamental value. (Gilbert 2010b) The 2008 commodity price boom: did speculation play a role? later, thus “rolling over” their holding and keeping it constant so that they are covered at all times. The investors in index funds earn three different types of return: 1. Spot return represents the change in value of the different futures contracts held in an index portfolio 2. Roll yield is the return on rolling over positions whose expiry date have reached the month prior to the delivery month into positions with a longer maturity of the same futures contract. When prices of near term futures are higher than those of futures with a longer maturity, the roll yield is positive. 3. Collateral yield is the return—usually a short-term interest rate—on the collateral that the investors have to put up against their futures positions. During the 2007–2008 commodities price boom, the roll yield and spot return were both positive and represented most of the profits of investors in index funds. The financialization argument first received widespread attention in the spring of 2008 (although the term had been used before then) when various committees of the United States Senate organized hearings on commodity prices. One of the more widely quoted testimonies was the one by Michael Masters, manager of a hedge fund, who juxtaposed the increase in investment by index funds in commodity markets with the increase in commodity prices. His testimony relies on examples purporting to show that the two phenomena are related. The following passage is representative: “According to the DOE, annual Chinese demand for petroleum has increased over the last 5 years from 1.88 billion barrels to 2.8 billion barrels, an increase of 920 million barrels. Over the same 5-year period, Index Speculators’ demand for petroleum futures has increased by 848 million barrels. The increase in demand from Index Speculators is almost equal to the increase in demand from China!” (Masters 2008) Mr. Masters then asks “Doesn’t it seem likely that an increase in demand of this magnitude in the commodities future markets could go a long way in explaining the extraordinary commodities price increases in the beginning of 2008?” (Masters 2008). Many of the subsequent articles and policy statements quote Masters and base their policy conclusions entirely on the argument that the perceived increase in liquidity flowing to commodity futures markets must have influenced prices.3 The argument is usually accompanied by a graph, similar to the one shown in Fig. 2. A refinement of this argument is 3 See, for instance, Khan 2009; Institute for Agriculture and Trade Policy 2008; 19 based on the “weight of money” hypothesis,4 according to which large inflows of funds may temporarily overwhelm the arbitrage mechanisms that normally ensure that “uninformed” speculation does not affect prices.5 While the argument recognizes that speculation in commodity futures do not result in changes in the spot physical price if these markets operate normally, it is claimed that sometimes normal market mechanisms do not operate. The nature of commodity markets Much of the force of the speculation argument relies on misplaced analogies with other markets. In particular, the proponents of the speculation theory argue that the price variations are disproportionate to the underlying changes in supply and demand6 and that therefore the “unexplained” variation must be due to speculation. First of all, it deserves to be noted that commodity price instability is not a recent phenomenon. Commodity prices fluctuated long before there were any commodity exchanges and long before anybody had thought of the“financialization” of commodity markets. Table 1 only shows the history of the past 30 years, but it should be sufficient to demonstrate that commodity prices have varied in the past and that the variations over the past decade, while large, were not exceptional. It appears from the table that the volatility of commodity prices has increased somewhat during the last decade. However, the apparent increase is mainly due to developments in the markets for minerals and metals, where, after a period of low demand and depressed prices lasting for almost 20 years, demand improved due to the rapid Chinese industrialization and prices rose more or less continuously during the period under study. Thus, these markets went through a period of rapid change which is likely to have raised price volatility as measured. The proponents of speculation as the main factor behind the recent commodity boom are convinced that price spikes cannot be the result of changes in supply and demand from non-financial and non-speculative actors, that is, supply and demand “fundamentals”. For those more used to analysing markets for manufactured goods, it is not readily understandable how commodity markets differ from other markets. Accordingly, price changes of more than 100% appear inexplicable and give rise to suspicions that something strange must be going on. This view fails to give adequate weight to the fact that 4 See, Gilbert 2010a, for a presentation of this argument. See the section “The nature of commodity markets” for an explanation of these mechanisms. 6 For instance, in UNCTAD 2008 (p.31), it is argued that “recent price hikes cannot be explained solely by underlying consumption and production trends” 5 20 O. Östensson Fig. 2 Commodity index investment according to Michael Masters. Source, Masters, 2008 commodity supply and demand can be extremely inelastic in the short term. The inelastic supply and demand lead to situations where the price rises to what would otherwise be considered to be unrealistic levels—simply because suppliers cannot deliver more at the moment and for buyers, it is better to pay this price than to go without. The ease with which buyers accept the high price is of course partly the result of a belief that the price rise may indeed be permanent or at least long lasting, in which case there is no point in refusing to pay the market price and postpone the purchase. An industrial buyer who does so runs the risk of losing business to competitors who are not afraid to secure their supplies at the going price. Accordingly, precautionary purchasing is perceived as a good strategy, particularly when inventories are low, as they were for almost all commodities in 2007/2008. It deserves to be noted that many observers of commodity markets during the 2007–2008 period believed that the high commodity prices were part of a “super cycle” that would push prices to unprecedented levels for possibly decades. While prices were rising, a large minority, at least, was perfectly convinced that those very high prices reflected long term supply and demand fundamentals. The same phenomenon occurred in previous broadly based commodity booms, in 1953, 1973/74 and 1979/80, when commodity prices rose by magnitudes similar to those experienced in the latest boom. Similarly, price spikes have occurred for individual commodities when the balance between supply and demand was temporarily upset, as for nickel and cobalt in the late 1980s7 and in the 1970s respectively. Once it is recognized that there are good reasons for buyers to (temporarily) accept very high prices, it is easier to understand that there is not necessary to have recourse to speculation as 7 Between February 1988 and February 1989 nickel prices (monthly averages of cash quotations at the London Metal Exchange) rose by 114%. They then fell by 62% to February 1990 (UNCTAD Commodity Price Bulletin). an explanatory variable. A temporary deviation of prices from their long term trends does not mean that they do not reflect fundamental factors, except if fundamentals are defined to simply mean the long term trend, which is not useful for analytical purposes. Some proponents of the speculation theory fail to see that price spikes can and do result from ordinary market forces because of misunderstandings concerning the nature of commodity futures markets. Some definitions and explanations may therefore be in order. A futures contract, which is bought and sold on a commodity exchange, is an agreement between a seller and the buyer that calls for the seller to deliver to the buyer a specified quantity and grade of an identified commodity, at a fixed time in the future, and at a price agreed to when the contract is first entered into. A commodity exchange or market is a financial market where different groups of participants trade commodity contracts with the objective of transferring exposure to commodity price risks. The purpose of a futures contract is to provide a hedge against price changes. The buyer of a futures contract transfers the risk of price change to the seller, who may be a speculator or a commercial entity such as a processor who needs to assure a supply of the commodity at a given price at the same time in the future. Since the terms of a futures contract are standardized, the contract may be resold many times. Standardization of the contract minimizes transactions costs and increases market efficiency. By contrast, over the counter markets are used for non-standard transactions taking place outside organized exchanges, for instance, to manage price risks for commodities that do not meet exchange specifications or for longer maturities than are traded at futures markets. Figure 3 illustrates two market situations. Contango is the normal market condition. The reason for low spot prices The 2008 commodity price boom: did speculation play a role? 21 Table 1 Price instability indices for selected product groups 1978– 1987 All commodities All food Food and tropical beverages Vegetable oils and oilseeds Agricultural raw materials Minerals, ores and metals 1988– 1997 1998– 2007 10.4 11.6 12.5 6.8 6.8 6.8 13.3 10.9 10.3 16.6 9.1 10.8 10.1 6.7 10.5 19 8.8 20.8 Source: UNCTAD Handbook of Statistics 2008, Table 6.2. Instability is measured as the percentage deviation of the variables concerned from their exponential trend levels for a given period. relative to forward prices is a positive “interest rate”, also called “cost of carry”. Because of storage costs and interest, a higher price is charged for futures than for spot delivery. Backwardation, where the spot price is higher than the forward price, is typical of a market in physical shortage and is the less common condition. Two types of main actors are active on futures markets: hedgers and speculators. The hedgers are normally producers or processors of commodities who want to protect themselves against adverse price changes. Speculators or investors are placing bets on the price moving in a particular direction. They provide liquidity to the market and make hedging operations possible. A third category, arbitrageurs, is often also identified. The arbitrageur attempts to profit from price differences between markets, for instance, between the London Metal Exchange and the Shanghai Futures Exchange. Although the vast majority of futures contracts never involve the delivery of a physical commodity, the possibility of physical delivery is essential to the credibility of commodity exchange quotations, since in the absence of this “physical anchor”, little would prevent prices from diverging from supply and demand fundamentals. It is important to understand that (a) futures markets trade futures contracts, not commodities, and (b) futures markets for commodities are anchored by a link to physical markets for the same commodities. With respect to the first point, an assumption sometimes made by the supporters of the speculation theory is that an increase in demand for futures contracts should be seen in relation to the physical supply of the commodity in question, as, for instance, in the statement by Michael Masters just quoted.8 They compare 8 Masters also makes the mistake of confusing stock and flow variables by comparing the increase in annual Chinese oil consumption—a flow variable—with the cumulative change in holdings of futures over the same period—a stock variable. His comparison should appear less impressive once it is understood that the average annual demand for petroleum futures was less than a fifth of the average annual increase in Chinese oil demand. Fig. 3 Contango and backwardation the total volume of contracts traded and the amount of physical commodities this volume would represent if all contracts were liquidated with a physical deal to the size of the total physical market. They then argue that such a large contract volume “must” have an effect on prices. However, if one argues that the sheer weight of the demand for futures contracts can directly influence the market, then one must also be prepared to explain how this happens. In fact, changes in the demand for futures contracts do not say anything per se about changes in the demand for the physical commodity in question. The reason is quite simple. The demand for futures contracts, including for speculative purposes, should be seen in relation to the supply of futures contracts and not to the supply of the physical commodity. However, the supply of futures contracts is determined by the perceived direction and magnitude of price changes, not directly by the supply of the commodity in question, and the sellers of the contracts are hedgers, arbitrageurs and other speculators. While there are limits to the number of contracts that hedgers are willing to sell, determined by their hedging needs, there are no such a priori limits to the number of contracts potentially supplied by arbitrageurs and speculators. As long as the amount of money invested in commodity markets is only a small fraction of that invested in all financial markets, there will be no shortage of sellers of contracts.910 Thus, the relevant supply of futures contracts is determined by the willingness of the financial community to make bets on the direction of commodity prices rather than by the physical supply of the commodity itself. This is not to say that futures prices carry no information. On the contrary, they indicate market sentiment about the direction of commodity prices and of expected future demand and supply of the physical 9 In fact, if the notional value of over the counter derivatives is taken as an indicator of total speculator interest, commodities accounted for less than 2% of contracts at the height of the boom, at the end of June 2008 (Bank of International Settlements, quoted in IMF 2009, Table 4). 10 Hollands (2009) argues that the price needed to draw in speculative short sellers has increased as a result of the need to match the increase in speculative long positions held by, for instance, index funds. He does not explain, however, why the short sellers should have become more demanding. 22 commodity. But the volume of futures contracts does not necessarily bear any relation to the amount of physical trade taking place. At this point in the argument, the speculation proponents will point to the possibility that if the speculators all believe that the price will go up, then it will be driven up by their trades because they establish a market consensus. However, this is where the link between futures markets and the price of the physical commodity comes in. All commodity futures markets accommodate the possibility, although it may be used extremely rarely, of liquidating a contract through actual physical delivery. Moreover, in order for futures markets to be useful to the physical trade, there must be a very close correlation between the physical spot price and the near term price at the futures market. Otherwise, the physical trade would stop using the futures market for its hedging. There have been several examples of futures markets that became irrelevant to the physical trade and closed down because the hedging interest disappeared. But this also means that the market will not allow the futures price to become misaligned with the physical spot price. If, in a situation where general market sentiment does not anticipate a price rise, the futures price were to be bid up by speculators, be they of the traditional or the index fund variety, then there would be no shortage of arbitrageurs or speculators who would buy or retain the physical commodity and sell the futures contract, making a tidy profit on this arbitrage deal. The misalignment in price would then quickly disappear. In this context, it should be noted that commodity futures markets have one characteristic that make them much less vulnerable to the formation of speculative bubbles.11 Commodity futures contracts are simple to short, while restricted ability to sell an asset short is a common feature of markets that have undergone periods of excessive speculation. For example, comparisons have been drawn between rising commodity prices and the speculative bubble that formed in technology stocks in the late 1990s (see Phillips and Yu 2010). A common characteristic of the technology stocks that were influenced by “excessive speculation”, however, was a limited float of stock that created a difficulty in selling the stock short. When it is difficult to sell an asset short, those speculators who think the price is too high are unable to express that view in the market and have difficulty influencing prices. Unlike the new technology stocks, however, commodity futures are 11 It should be noted that an asset price bubble is not necessarily the result of speculation. Phillips et al. (2011) note, in the context of an analysis of the Nasdaq market that “the approach is compatible with several different explanations of this period of market activity, including the rational bubble literature, herd behavior, and exuberant and rational responses to economic fundamentals. All these propagating mechanisms can lead to explosive characteristics in the data.” O. Östensson derivative assets, with longs and shorts not restricted by the number of shares outstanding. Therefore, if market participants truly believed that the forward supply and demand fundamentals do not support commodity prices, they could easily sell commodity futures. It is of course perfectly possible to imagine a situation where the futures price is bid up by speculators, who manage through their actions to convince other market actors that they are right about the direction of prices, and where spot prices get bid up as a result, along with futures prices. The same effect may come about as the result of a large number of uninformed speculators or “noise traders”12 overwhelming the market with a very large demand for contracts. This could then lead to more purchases of spot commodities and so on, in a rising spiral of prices. Such a development could occur, for instance, if speculators exploit index funds’ purchases of futures, which drives up the futures price, by buying spot material and selling futures, thus liquidating the position. However, while such a process is certainly possible, it would have two results: (a) stocks would be built up as a result of the purchases of spot commodities and (b) the market would stay in contango, that is, far off delivery dates would be more expensive than nearby ones—since otherwise it would not be possible to make a profit by buying spot and selling futures and holders of the physical commodity would prefer to cash in their profits by selling spot. However, during the first half of 2008 there was little sign of rising stocks for any of the commodities that experienced rising prices and the markets were consistently in backwardation, that is, near term prices were higher than far out dates. This situation is in fact typical of a physical shortage, and incidentally, it is also one of the factors that made investment in index funds attractive. If a contango had developed as a result of speculation, then investor interest in index funds would probably have cooled relatively fast.13 It has been argued by some commentators that stock data for commodities are too unreliable and incomplete, particularly in developing countries and non-market economies, to permit the conclusion that there was no build-up in stocks during the 2008 boom (see, for instance, Khan 2009, p. 5). However, this misses the point. The argument is not 12 For a discussion of noise traders and their effect on the market, see Sanders et al. 1996. 13 It is interesting to note that stocks of crude oil in OECD countries increased from 2005 to early 2007 when there was a contango on the futures market. At least one commentator has attributed this phenomenon to index fund activity, noting however that investors did not realize the hoped for roll yield, but only the spot return for part of the period (see Verlegen 2007). During the latter part of the period, oil prices fell and did not resume their rise until the contango turned into a backwardation, which indicates that any effects on prices from speculative activity were more than offset by movements in physical demand. The 2008 commodity price boom: did speculation play a role? 23 about whether there was a net increase in global inventories or not, although such an increase would have provided supportive evidence of the speculation hypothesis. Rather, the argument refers to the expected behaviour of rational economic actors and the extent to which they react to a price rise by building up stocks. Data on oil inventories for OECD countries are considered to be very reliable and while they did show a small increase during the period concerned, the increase was well within usual seasonal variations.14 It is not clear why actors in these countries would not be expected to behave rationally. It is significant that very few of the defenders of the speculation hypothesis have made any attempt to explain how purchases by index funds led to higher prices when commodity markets were in backwardation. In particular, they present no convincing explanation of how prices were bid up by index fund purchases of futures when spot and near term futures prices (corresponding to index funds selling) were higher than those of the futures bought by the funds. As will be seen in the section “Brief review of published studies”, some of the studies published find significant correlations between index fund purchasing and prices for at least some commodities some of the time. However, in the absence of a credible explanation of the link between index fund activities and prices, the results constitute a reminder that correlation is not causality. The above argument needs to be tempered by reference to market imperfections. Price adjustments do not really take place instantly, although we assume that they do for convenience of analysis. The market adjusts to new signals in a process which takes time, even if the time necessary is very short. Moreover, the search for equilibrium is a never ending one. The market continuously chases the equilibrium without ever attaining it. Therefore, the arbitrage that acts as a correction to speculation does not take place at once and inconsistencies may emerge in the“yield curve” (the curve showing the development of futures prices and which is generally assumed to be a smooth curve with even upward or downward slope, see Fig. 3 for an example). For instance, when index funds were first introduced, the funds traded only from 1 month to another, that is, they bought 2 month futures and sold 1 month futures. The result was that the price for 2 month futures was bid up at the time when the funds made their purchases (always at the same date of the month) and the price of 1 month contracts was pushed down, although the overall price level was not affected. Accordingly, in order to improve trading profits (what is called rollover yield in the jargon of the trade), the funds started buying more far off futures, distributing their purchases over several months so that no month was overloaded. This shows that temporary distortions can appear. However, the trend in prices was not affected, only the shape of the yield curve, which disturbed hedging activities but did not have any other economic impact.15 14 15 While oil stored in tankers may not be included in inventory figures, such storage is unlikely to have been significant. Freight rates were high and rising in 2007–2008 and ship owners would have made more money carrying oil. Even if it is possible that in spite of this a small portion of total tanker tonnage was used for storage (unnoticed by shipping industry observers), say, 40 ships or 1% of the total number of crude carriers, the volume stored would have been equivalent to about 40 million barrels or less than half of 1 day’s world crude consumption, hardly a significant stock build-up compared to total OECD inventories of about 2,600 million barrels in mid-2008. Some simple statistical evidence If the financialization argument is valid, commodity prices would have been expected to behave in certain ways and it should be possible to confirm that they have done so: 1. Prices of commodities that are not traded on futures markets would be expected not to be affected by financial speculation. Thus, their prices would be expected to have risen by less during the 2002–2008 period. 2. The evolution of prices for commodities that are substitutes would be expected to differ if some of these commodities are traded on commodity exchanges and some are not. 3. On the other hand, prices of commodities that are exchange traded and that are included in index funds would be expected to be quite well correlated 4. Changes in net futures markets positions of index funds would be expected to be correlated with prices. 5. Finally, changes in positions of the index funds would be expected to precede price changes. Figure 4 illustrates the first point. It shows the change in prices of commodities that are traded on commodity exchanges and those that are not. The latter are represented by the top six horizontal bars in the figure.16 It is difficult to discern any significant differences between the two groups, both of which include commodities that saw only modest price increases, such as tea and sugar, and commodities that experienced explosive price increases, such as manganese Over longer periods, the relationship between the prices of contracts with different maturities may be quite loose. Büyükşahin et al. (2008) show that as recently as 2001, near- and long-dated futures for crude oil were priced as though traded in segmented markets. In 2002, however, the prices of 1-year futures started to move more in sync with the price of the nearby contract. Since mid-2004, the prices of both the 1-year-out and the 2-year-out futures have been cointegrated with the nearby price. 16 In fact, a contract for rice exists on the Chicago Board of Trade, but it is not actively traded and rice is not included among the commodities traded by commodity index funds. 24 Fig. 4 Commodities traded (bottom 15) and not traded (top six) on commodities exchanges: percentage change in prices 2002–June 2008. Source: UNCTAD Commodity Price Bulletin O. Östensson Manganese Cobalt Iron ore Rice Bananas Tea Crude oil Copper Rubber Nickel Lead Palm Oil Soybeans Coffee Maize Wheat Zinc Cocoa Aluminium Cotton Sugar 0 100 200 300 400 500 600 700 ore and crude oil. Some of the commentators who argue that speculation was a determining force discuss only one or two commodities that are traded on exchanges and thus could maybe not be expected to feel that it was incumbent on them to explain why there is no difference between the two groups of commodities. Those who discuss commodities in general, however, would be expected to at least attempt to explain the absence of a difference. Figure 5, which shows the evolution of prices for wheat and maize (that are traded on commodity exchanges) and rice (that is not), illustrates that, at least in this case, point 2 is not valid. Prices for exchange traded commodities and substitutes that are not traded on exchanges were fairly well correlated throughout the period. On the other hand, according to point 3, prices of commodities that are exchange traded and that are included in index funds would be expected to quite well correlated if the speculation argument is valid,17 since the commodities in question are included in the indices with fixed weights. In particular, prices of these commodities would be expected to peak roughly simultaneously, at the height of index fund investment. If the correlation is weak, then other factors would appear to have dominated the formation of prices. Figure 6 shows the evolution of prices for five nonferrous metals, all traded on the London Metals Exchange and all included in index funds. The reason for using this group of commodities is that their prices would be expected to be relatively closely correlated even in the absence of speculation, given that their demand tends to evolve in parallel and that they are more or less close substitutes for one another. As seen from the figure, prices for the metals peaked at very different dates and the degree of correlation does not appear to be very high.18 With respect to the correlation between index fund positions and prices, Fig. 2 in the foregoing, taken from Michael Masters’ testimony, appears to show that there is indeed such a correlation. It shows an impressive increase in index fund investment, evolving in parallel with commodity prices. However, five things about the diagram should be noted. First, prices are expressed in current US dollars and the diagram stretches over 38 years, meaning that the price increase is exaggerated by inflation. Second, the vertical axis of the diagram is truncated and no values below 100 are shown, which again has the effect of visually exaggerating the rise in prices. Third, prices are annual averages, which mean that shorter-term variations are not captured. Fourth, the diagram shows only gross investment, 17 18 Tang and Xiong (2011) found a higher increase in return correlation between commodities that were included in indices than for those that were not included. However, as noted in Section 5 in the following, their sample of non-index commodities included commodities that are very little traded. The holdings of futures underlying the index funds are regularly recomposed in order to maintain constant weights. Thus, if the price of a particular commodity rises, then index funds sell off some of their holdings in order to maintain the weight of the commodity in the index. The 2008 commodity price boom: did speculation play a role? 25 saw an opportunity to make a profit by reducing their normal hedging. The picture is similar for other commodities. Thus, the support for point 4 and the connection between the activities of index funds and price increases appears tenuous at best. Finally, a further piece of evidence concerns the timing of changes in net positions and in prices. The CFTC has analyzed this aspect. Its conclusion with respect to trade in crude oil deserves to be quoted: Fig. 5 Cereals prices, January 2004–May 2009, US dollars/tonne. Source: UNCTAD Commodity Price Bulletin meaning that purchases and sales are not netted out. Thus sales and purchases are added, in spite of, according to the argument, their having opposite effects on prices (one would tend to drive prices up, while the other would drive them down). Fifth, the diagram shows the value of investment rather than the number of contracts. As prices rise, so does of course the value of investment. Figure 7 represents an attempt to present a clearer picture. It shows the net crude oil positions of two categories of traders as reported to the United States Commodities Futures Trading Commission (CFTC). The first category is the “commercial” or “producer/merchant/ processor/user”. This is an “entity that predominantly engages in the production, processing, packing or handling of a physical commodity and uses the futures markets to manage or hedge risks associated with those activities” (CFTC, Disaggregated Explanatory Notes, http://www.cftc. gov/MarketReports/CommitmentsofTraders/index.htm). The “swap dealer” is an “entity that deals primarily in swaps for a commodity and uses the futures markets to manage or hedge the risk associated with those swaps transactions. The swap dealer’s counterparties may be speculative traders, like hedge funds, or traditional commercial clients that are managing risk arising from their dealings in the physical commodity.” (CFTC, op. cit.) This category includes mainly institutions that hedge their exposure on index funds. Accordingly, if the argument about the role of financial speculation is correct, then the development of their net positions would be expected to be closely correlated with the price. As seen from the figure, however, the net position of the swap dealers appears to have little correlation with the price and, if anything, the correlation appears to be negative. Particularly interesting is that this group of traders increased their net positions when oil prices fell. The net positions of the producers/processors/merchants, on the other hand, peaked at the same time as the oil price, probably illustrating that this category of market participant If a group of market participants has systematically driven prices, detailed daily position data should show that that group’s position changes preceded price changes. The Task Force’s preliminary analysis, based on the evidence available to date, suggests that changes in futures market participation by speculators have not systematically preceded price changes. On the contrary, most speculative traders typically alter their positions following price changes, suggesting that they are responding to new information—just as one would expect in an efficiently operating market. (CFTC, 2008, p. 3). The CFTC report used data for non-commercial traders or “traditional speculators”, which only included some, maybe very few, swap dealers. Later analyses based on disaggregated data have however not changed this conclusion. Another systematic attempt that has been made using the same type of analysis found a significant relationship for the soybean market, but failed to find any effects on soybean oil, wheat or corn futures (Gilbert 2010a). The author concludes: None of this implies that either speculation or commodity investment have been a major factor in the commodity boom of the first decade of the century. On the other hand, it is too simple to rule out the possibility that these activities may have affected prices in particular markets at particular periods of time. Fig. 6 Price indices for nonferrous metals, January 2007–May 2009 (January 2007=100). Source: UNCTAD Commodity Price Bulletin, based on London Metal Exchange data 26 O. Östensson 250000 200000 150000 140 120 100000 100 50000 80 -50000 60 -100000 40 -150000 -200000 -250000 Commercial net Swap dealers net 0 Oil price 20 0 Fig. 7 Net positions of different categories of market participants in the futures market for crude oil, number of contracts, and oil price (index, June 2006=100), June 2006–May 2009. Source: United States Commodity Futures Trading commission (CFTC), Commitments of Traders, http://www.cftc.gov/MarketReports/CommitmentsofTraders/ index.htm This prudent assessment would not seem to imply that there is cause for major concern or justify calls for radical market reforms. was about currency and commodity markets that drew the interest of these speculators and why these markets should move in such close parallel. The argument overlooks the basic axioms that cotemporaneous events are not necessarily related and that correlation does not prove causality. The latter point is particularly important since the authors do not make any attempt to explain how the relationship between selected exchange rates and commodity prices works and present no explanation of how speculation is supposed to have affected commodity prices. A paper by M.S. Khan (2009) argues that if futures prices of crude oil reflected fundamentals, then the price rise in 2008 would have been reflected in the share price of oil producing companies, which is not the case. Instead, it is argued, the share prices implied that a considerable part of the price increase is not accounted for by fundamentals. Since market valuations of oil companies take little account of price increases that are seen to be temporary, this argument appears to imply that short-term factors are somehow not “fundamental”. Again, the study presents no explanation of how speculation affected oil prices. A number of studies have used econometric techniques to test the speculation hypothesis. In Gilbert (2010a), use of the Granger non-causality test (a statistical test for causality in the sense that it tests the correlation between a hypothesized cause and an effect in a later period) that appeared to show that changes in soybean prices were somewhat affected by changes in index fund net positions. For corn, soybean oil and wheat, results were negative. In a later study by the same author (Gilbert 2010b), a quantum index of total net index-related futures positions on the United States agricultural markets from the start of 2006 to mid-2009 is related to price changes of crude oil, aluminium, copper, nickel, corn, soybeans and wheat, using Granger causality tests. It is concluded that “According to these estimates, it would be incorrect to argue that high oil, metals and grains prices were driven by index-based investment but index investors do appear to have amplified fundamentally driven price movements.” The agricultural Brief review of published studies As already mentioned, a number of studies and reports on the effects of speculation, particularly investment by index funds, have been published over the past couple of years. In the following, the results of these studies are briefly summarized. Some studies support the argument that index funds contributed significantly to the commodity price boom in 2007/2008. Several of these do not use econometric analysis but rely on “common sense” arguments. The most frequently cited by these refers to the correlation between exchange rates for currencies subject to “carry trade”19 and commodity prices. It is claimed that “This strong correlation between commodities and other asset classes during the second half of 2008 suggests that financial investors may have strongly influenced commodity price developments” (UNCTAD 2009, p. 67). Thus, the proponents of this view argue that a correlation between the exchange rate of currencies that are considered to be the subject of speculative investment and a commodity price index demonstrates that speculation affected commodity prices. Quite apart from the fact that the data appear to have been very carefully selected and that by the second half of 2008, most commodity prices had already peaked, the argument implies that the same group of speculators—or alternatively speculators that behaved in parallel—were active in both currency and commodities markets. It is not clear what it 19 Carry trade refers to the practice of borrowing at low rates of interest in certain currencies and investing the proceeds of the loan in a currency yielding a higher interest rate. The phenomenon was important during the economic upturn in 2007/2008, when large differences in interest rates between currencies were prevalent, and has again assumed importance in 2011. The 2008 commodity price boom: did speculation play a role? index is assumed to be representative of total index positions since index composition changes slightly or not at all. However, the result is perplexing since use of the agricultural product index (composed of net positions for corn (maize), soybeans, soybean oil, soft wheat, hard wheat, cocoa, coffee, cotton, sugar; feeder cattle, lean hogs and live cattle) produces significant correlations for crude oil and metals, but not for the agricultural products themselves. A possible explanation could be that indexes are reweighted at regular intervals in order to take in to account price changes and to avoid, for instance, that a commodity that has seen a steep increase in prices assumes too much weight. It is conceivable that this recomposition could have produced spurious correlations. Other studies have used the same technique as Gilbert but have obtained different results. Thus, as already mentioned, the United States Commodity Futures Trading Commission (CFTC) found with respect to the crude oil market that “changes in futures market participation by speculators have not systematically preceded price changes” (CFTC, 2008). Similarly, the IMF, when studying data for oil, copper, wheat, corn, soybeans, and rice with the same technique, found: “Overall, there is correlation between prices and positions in some commodities markets. However, we are unable to detect causality from financial positions to prices for major commodities used in the study.” (IMF 2008, annex 1.2). The most comprehensive study using Granger causality and non-causality tests is the one carried out by Irwin and Sanders for OECD (Irwin and Sanders 2010) which analyses price movements for all commodities included in indexes. The study finds that index funds did not cause a bubble in commodity futures prices. There is no statistically significant relationship indicating that changes in index and swap fund positions have increased market volatility. Another study (Tang and Xiong 2011) tests the difference between commodities that are included in index funds and some that are not with respect to the return on a hypothetical investment position in the first-month futures contract of a commodity which is held until the seventh calendar day of its maturity month before rolling into the next contract. It is found that the correlation between the returns on non-oil commodities that are included in indexes and returns on oil contracts increased by more during the period of strong price increases (2004–2009) than the correlation for commodities that are not included in indexes and oil. The correlation increased for both groups compared to an earlier period (1999–2004), which is not too surprising, given that prices of all commodities increased during the later period, which would be expected to result in a higher degree of correlation between returns. While the difference between the two groups is small (for the index commodities, the correlation increased by 0.3, while for the non-index commodities, the increase was 0.2), it is significant. However, the method has an 27 inherent weakness. Index funds consist of those commodities for which contracts are much traded and therefore more liquid, while contracts for the non-index commodities are less liquid, with prices therefore being more subject to erratic movements. This is important since the non-index commodities studied (rough rice, soybean meal, oat, Minnesota wheat, orange juice, lumber, platinum, palladium and pork bellies) include some that are hardly traded at all. It is therefore doubtful how much weight should be given to the result. Phillips and Yu (2010) found evidence of bubbles in prices of crude oil and platinum in 2008. The underlying idea in their analysis is that if trend-following behaviour is important, an upward movement in prices will tend to be extrapolated. The view that there is a strong (positive or negative) trend in an asset price will itself generate the momentum which validates this belief. However, while their study shows evidence of explosive price behavior, that is, of prices increasing at an accelerating rate and of autocorrelation in prices, such price behavior can result from several different causes (as pointed out by some of the same authors in the previously quoted Phillips et al. (2011)). In particular, as explained in the section “The nature of commodity markets”, explosive price behavior is consistent with the functioning of commodity markets at times of physical shortage, even in the absence of speculation. Gilbert (2010b), who relates similar results, notes: These bubbles are consistent with explanations in terms of extrapolative behaviour, perhaps on the past of CTAs and other trend-following speculators. However, there are also other possible interpretations. First, the “market fundamental” may have explosive over these periods. Crude oil and nonferrous metals are all industrial commodities and market opinion ascribes recent high prices to rapid growth in Asia, particularly Chinese demand. Second, even if demand growth was not explosive, it is possible that market perceptions of these possibilities grew in an extrapolative manner. Third, rapid demand growth brings markets close to stockout. This results in a nonlinear relationship between price and the fundamental which might appear as explosive behavior. Perhaps, more than one of these factors was operative with a resulting compounding effect. Finally, a number of government authorities have published reports that consist mainly of inventories of the results of other studies, although often with a considerable amount of reasoning added. Thus, the US Senate Subcommittee on Investigations of the Committee on Homeland Security and Governmental Affairs concluded in 2009: “there is significant and persuasive evidence that one of the major reasons for the recent market problems is the unusually high level of speculation in the Chicago wheat futures market due to purchases of futures contracts by index traders offsetting sales 28 of commodity index instruments” (United States Senate 2009). Although this study is often quoted in support of the argument that index funds affect the prices that are actually paid for the physical commodity it deserves to be noted that the “market problems” referred to consisted of spot prices not being correlated to futures prices and that farmers did not benefit from high prices on futures markets.20 A working group on the oil market appointed by the Ministry of Economy, Industry and Employment in France concluded that available statistical data did not allow causal links to be clearly established between open positions of financial investors on futures markets and observed spot prices. However, such a link could not be excluded either, and it was therefore reasonable to conclude that the activities of certain financial actors could have amplified upward and downward movements of oil prices, increasing their natural volatility (Chevalier 2010). Finally, a Task force of the International Organization of Securities Commissions, which was formed following the concerns expressed by the G8 Finance Ministers regarding the price rises and volatility in agricultural and energy commodities in 2008, concluded after a thorough review of the available studies that “Reports by international organizations, central banks and regulators in response to the above concerns that were reviewed by the Task Force suggest that economic fundamentals, rather than speculative activity, are a plausible explanation for recent price changes in commodities” (International Organization of Securities Commissions 2009). Conclusions As has been shown, some of the arguments in favour of the speculation hypothesis are based on false analogies with other markets, which ignore the particular mechanisms underlying price spikes in commodity markets. Moreover, proponents of the speculation hypothesis do not offer any credible explanation of how activities by index funds on futures markets led to rising prices in markets subject to backwardation. A number of simple statistical arguments also point to the importance of index fund speculation having at least been exaggerated. Finally, most authoritative econometric studies reject the speculation hypothesis. Those that do not, suffer from shortcomings with respect to methodology (use of indices instead of individual commodity data, skewed selection of commodities) or are inconclusive. It therefore appears that there is very limited basis for the arguments that the nature of commodity markets has changed or that speculators, particularly index funds, were responsible for the commodity price increases in 20 The reason for the “decoupling” of spot and futures prices appears to have been logistics problems which made it difficult for farmers to supply grain at delivery points. O. Östensson 2008. The same conclusion has been drawn by regulatory agencies, who have made very modest changes to the rules governing commodity markets and trade. It is to be hoped that these agencies can continue to resist calls for regulation that are not backed by factually based analysis. Acknowledgement I am grateful to two anonymous reviewers for several good suggestions which improved the draft considerably. References Büyükşahin B, Haigh MS, Harris JH et al (2008) Fundamentals, Trader Activity and Derivative Pricing, http://papers.ssrn.com/ sol3/papers.cfm?abstract_id=966692, accessed on 20 May 2011. Chevalier JM (2010) Rapport du groupe de travail sur la volatilité des prix du pétrole, Ministère de l’économie, de l’industrie et de l’emploi Gilbert C (2010a) Commodity speculation and commodity investment. Journal of Commodity Markets and Risk Management Gilbert C (2010b) Speculative influences on commodity futures prices 2006–2008, Discussion Paper 472. UNCTAD/OSG/DP/2010/1, Geneva Hollands P (2009) Metal Price Formation: what if everything we thought we knew was wrong? LME Week Supplement IMF (2008) Global Financial Stability Report, October IMF (2009) Global Financial Stability Report: Responding to the Financial Crisis and Measuring Systemic Risks, April Institute for Agriculture and Trade Policy (2008) Commodities Market Speculation: The Risk to Food Security and Agriculture, November 2008, Minneapolis International Organization of Securities Commissions (2009) Technical Committee, Report of the Task Force on Commodity Futures Markets, July. Irwin SH, DR Sanders (2010) “The Impact of Index and Swap Funds on Commodity Futures Markets: Preliminary Results”, OECD Food, Agriculture and Fisheries Working Papers, No. 27, OECD Publishing. doi:10.1787/5kmd40wl1t5f-en Khan MS (2009) The 2008 Oil Price “Bubble”, Peterson Institute for International Economics, Policy Brief number PB09-19, August 2009 Masters M (2008) Testimony before the Committee on Homeland Security and Governmental Affairs, United States Senate, May 20, 2008 Phillips PCB, Yu J (2010) Dating the Timeline of Financial Bubbles During the Subprime Crisis Phillips PCB, Wu Y, Yu J (2011) Explosive behaviour in the 1990s NASDAQ: when did exuberance escalate asset values? International Economic review 52:201–226 Sanders DR, Irwin SH, Leuthold RM (1996) Noise Trader Demand in Futures Markets, OFOR Paper Number 96–02 Tang K, Xiong W (2011) Index Investment and Financialization of Commodities, http://www.princeton.edu/~wxiong/papers/ commodity.pdf, accessed on 20 May 2011 UNCTAD (2008) Trade and Development Report UNCTAD (2009) The Global Economic Crisis, Systemic Failures and Multilateral Remedies, New York and Geneva United States Commodity Futures Trading Commission (CFTC) (2008) Interim Report on Crude Oil, Interagency Task Force on Commodity Markets, Washington, D.C., July United States Senate (2009) Excessive Speculation in the Wheat Market, Permanent Subcommittee on Investigations, Committee on Homeland Security and Governmental Affairs, Majority and Minority Staff Report. Verlegen PK (2007) Comments on Energy Markets, Volume I, Number 1, May 16