A few posts ago I criticized economists who either explicitly or implicitly say that the futures price for a particular date is "the market's best guess" of what the spot price will be on that date. I wrote:
You can see the difference pretty easily: We can always check whether we are in backwardation, since it is a snapshot thing between spot prices right now and "the futures price" right now. But in order to see whether oil prices decrease over time, you would actually have to wait a few months and watch.
Although the above is true, it may have misled some people as to how deep my criticism was. From reading my first post, one might have thought that I was merely saying, "The futures price right now is a different thing from what the spot price in the future will end up being."
Yes that's of course true, but that's not the extent of my point. To drive this home, I should relate that back when I was trying to get a job as a "quant" on Wall Street, I read up in the relevant literature and hung out at sites like Wilmott. At that time I thought that the futures price was the market's best guess of what the spot price would be.
When I asked if this were indeed true at Wilmott in the newbie section, it actually took a few posts for me to explain what I was asking. I.e. not only is this typical economist belief false, but experts in this stuff at first didn't even understand the claim because it was so ludicrous to them.
One guy in particular gave a really simple counterexample that blew the notion to smithereens, but I can't quite remember it. In any event, try the following if I still haven't convinced you:
What exactly do we mean by saying that a person or a group of people think the "best guess" of spot oil is (say) $145 on October 1, 2008? Would that person bet his left arm that West Texas crude would literally sell for $145 and zero cents on that day?
Of course not. We don't need to get too specific about the precise nature of it, but obviously people think there is a range of plausible oil prices on that date, with some intervals being more likely than others. For the sake of argument, let's stipulate that the person views the world as Bryan Caplan does, and so he has an actual probability distribution defined over possible spot oil prices ranging from $0 to infinity.
OK so let's suppose the expectation of this distribution is $145. Therefore, in a quite defensible sense, you could say this person would give a "best guess" of $145 as the oil price on that date.
Now does it follow that a person would be willing to buy a futures contract right now for oil with a futures price of $144? Right now, the guy's expectation of the value of this contract is $1 delivered on October 1. (Actually this is for a forward contract; let's not worry about the fact that futures are marked to market periodically.)
Well sweet, the guy can buy a futures contract at no cost right now (i.e. you don't hand over money when you go long or short on a futures contract) and have an expected gain of $1 on October 1. From a certain point of view, that's an infinite rate of return! So surely in equilibrium the futures price has to be exactly equal to the market's "best guess" of the future spot price, else arbitrage. Right?
No, because we haven't accounted for the risk. If oil drops to $44, then that futures contract imposes a loss of $100. Is it so clear cut that the guy should buy the futures contract, even when his expectation of the spot price is higher than the futures price?
So now we start to see that there is a lot more to it than typical economist stories assume. Maybe if every investor were risk neutral, then arbitrage would ensure that the futures price always equaled people's expectation of the spot price at that date. (Even here, what happens if people disagree? Then it's hard to define what "the market's best guess" even is, let alone come up with a way to ensure that it matches the futures price.)
OK I hope I've made my point. People who know more on this, please correct any dumb mistakes I've made in the above.