Efficient Markets¶
Forecasting, Random Walks, and Mean Reversion¶
So today I want to talk about the efficient markets hypothesis, the history of the hypothesis, reasons to think that markets are efficient, and reasons to doubt it. Let me start with the dinner experiment on Friday. So we had a nice dinner at Berkeley College, and as an experiment, those of you who were there will know my experiment, I passed out slips of paper with this chart.
So the chart shows the Standard & Poor 500 from 1950 until a couple days ago where it was, there it is, that's the stock market. And then I left room on the right out to the year 2050, and I asked each of you independently, I asked you not to look at each other when you did this, to pencil in a forecast for the stock market from 2016 to 2050. So I just gave you to 50.
you a couple examples that you filled in. This is, this person is forecasting, what does that look like? Another 20% drop in the market, then a correction, then a drop, and then another boom, and then another crash, then another boom. So I picked this one out as just typical of what was seen. I should have shown you more. I only have one other here. This is another one.
So you are a very pessimistic bunch. Now, what would the efficient markets theory say should be the forecast? There are one or two of you actually did that, or maybe it was three, so more or less. What should, if markets are completely unforecastable, what should it be? Well, one interpretation of efficient market, your forecast should have been zoom straight across.
It's not going to leave where, I mean, the, central tendency is tomorrow's price is the same as today's price, because I can't forecast changes. Up or down, who knows? That would be efficient markets. But there's another version of efficient markets, and some of you actually did this, but not many of you, your forecast would be not straight across, but perfectly growing exponentially along an exponential growth path.
That would be, the efficient markets hypothesis, taking account of the fact that there's a growth. It's not a random walk, but it's a random walk with growth. But almost none of you did that. Maybe my instructions weren't clear enough. But what you're doing is showing plausible paths for the stock market, rather than the expected path. That's how people seem to interpret when I say, please forecast it.
You're trying to show what's plausible. So you're making, if you look at the recent history, this person made it look like the recent history, right? But this is not, unless you can tell me, I don't know who did this forecast, but unless you can tell me, why did you have a turning point here in, what year is that? 2020? Why did you have this turning point? where you put it?
They have no reason. They just, this look plausible to this. to them, so they call that a forecast. I know you might have done better if you had more time to think about it. But I think this gets back to, it's related to a concept in behavioral economics that Connman and Tversky talked about. Those are two psychologists called the representativeness heuristic.
People don't behave like forecasters. They think that something they saw in the past is representative of what will happen in the future. Now, I'm not defining their theory. exactly, but I think this is how most people take it when they have to give a forecast. The random walk theory is a theory, the term was coined by statistician Carl Pearson in the scientific journal Nature in 1905.
And he said, a random walk is a process that changes in such a way that each change is independent of previous changes and totally unforecastable. So it has become popular lore to think of a random walk as they walk of a drunk at a lamp post. And this toy right here is a mock-up that someone once made and sold. If you want that, I found it on the web, you can buy It's amazing what you can buy today.
But there's your drunk. Now, if the drunk is so totally drunk that every step is completely random, so your job, according to Pearson, is to forecast the position of the drunk in 10 minutes, in 20 minutes, in 30 minutes. So what is your forecast? What do you think? What would be chariots? Carl Pearson's forecast. Well, the forecast is he's going to be right at the lamp post.
You'd never, and the reason you make that as your forecast is you have no idea whether it's to the right or to the left or whatever. The direction is random. So he's probably not going to be at the lamp post, but since you can't predict which way, you might as well predict at the lamp post. That's why I was saying on the previous slide that your forecast should have, well, go back.
I'll go back here. Your forecast should have been zoom straight across if stocks are pure random walk. But that's not what you did. In 1973, Burton Malkiel, who was a professor at Princeton, and I guess still is, he came to Yale for some years as dean of the Yale School of Management. But he wrote a best-selling book in 1973 called A Random Walk Down Wall Street, arguing.
that, well, it wasn't original in the book, this was a popularization, but it's a bestseller, and it's sold millions of copies. The idea was coming out then that stockmark prices are really random walks. It's all an illusion that you think you can forecast it. So you shouldn't try to forecast it. You should just hold a diversified portfolio. And that became conventional wisdom, starting around that time.
Burton Malkiel was a very well, his book was very well, timed and a very appropriate title. The funny thing about Malkiel's book, though, is that he didn't really believe the efficient markets hypothesis that he made the title of his book. Because if you look in at the last chapter of the book, he had investing advice, and he told you to do various things that were, I thought, inconsistent with random walk hypothesis.
And I finally met him at a cocktail party, and I asked him. I said, you know, some of you're investing. of your investment advice doesn't quite fit with random walk. And then he said, well, I know, I can't quote him exactly. He said, I can't get these other kinds of thinking completely out of my mind. Something like that. So it's like psychologically, we're not attuned to understand a random walk.
Now another alternative to a random walk, has the property x sub t if x is the random walk at time t, is equal to its previous value plus noise, and the noise is totally unforecastable. An alternative to a random walk model is a first-order auto-regressive model, an AR1 model. In this case, 100 represents the lamp post. It's the starting point. And then x-t, the value of the position at time t is equal to the starting point plus row, where row is between minus 1, and plus one, usually positive, times x sub t minus 1 minus 100.
So what this is is a model which is modifying the drunk at the lamp post a little bit. He is now has a piece of elastic wrapped around his ankle and the elastic is tied to the lamp coast. So if he starts walking away from the lamp post in any direction, he gets tugged a little bit back to it. And the further away he goes, the more he starts to the more he's tugged back.
So in this case, this is the x-t minus 1 minus 100, is how far he is from the lamp post. And if row is some high value, then that means there isn't much of a tug. It's a weak piece of elastic. So he can go wandering off, but eventually he's going to come back. So this is mean, reverting. It's reverting back to the lamp post. Now the question is, how do we, there were a lot of studies suggesting that the stock market is a random walk.
But how do we know whether it's really a random walk or it's an AR1? Maybe it, when it gets really, when the stock market gets really high, it's going to go back, but not right away, but you know, eventually, there's something tugging it back to a normal level. And when it gets really low, it's going to go back up. The problem is that, it's hard to tell a distinction if row is close to one.
If row equals one, you can see these hundreds drop out, and it's just a random walk. But if row is lower than one, but not a whole lot lower than one, then you really can't tell whether it's a random walk or not. So this is an example of realization of a random walk, the blue line, and an AR1 with row equals.9. And I defy you to tell me which one of those is Well, you can tell, in comparison, this one started not at 100, but at 1.
So you can see that there is some, the AR1 has come closer, that's the red line, has come closer back to the starting point. The random walk has no tendency to come back to the starting point. So if stock prices are an AR1, then it means you should stay out of the market when the price is too high and go back in when the price is too low, which is different advice than random walk theory.
Many would argue that when you're forecasting the future, whether economists or people in finance who are working in Wall Street, for instance, or forecasting what the future of a market is going to look like, to look at what's happened in recent history and try to project it forward. And using irrational. logic model to try to project into the future. But, of course, you talk a lot about sort of the irrational nature of the market and how we're behavioral or how we're irrational beings.
But what I'm curious is, how do you build in risk management to factor into sort of the irrational nature of the markets and to capture sort of some of that irrational exuberance that you talked about that might arise in such things as housing markets? Well, deep question, how do we handle? Why do we have irrational exuberance at some times and not other times?
And what can we do about that? Our principal tool that's talked about is central bank policy. And central banks, when they think that the market is become becoming overpriced, can tighten credit, and alternatively when they think it's underpriced, they can loosen credit. That is a tool that is being used, although central banks don't seem too enthusiastic about trying to stabilize financial markets.
It does seem to influence their decision-making. But that doesn't get at the basic psychology very much. So the problem is, people are looking at price of a speculative asset as something completely separate from the business, largely separate from the underlying business that is paying the bills, paying the dividends. They seem to often be looking at the psychology of the market.
They're looking at trends of technical analysis, and they become unfocused on the real fundamental, which you might think ought to determine the long-run value of an investment. They're focused too much on the short term, predicting what it will do in the next month or so between the time I buy and sell. So there have been proposals that might limit, reduce that short-term speculative component.
So one idea is to put a transactions tax on securities trade. that would force people to be, or encourage them to be longer term in their investment. We have a distinct, so that we have, but limited. We also have a capital gains tax that distinguishes between short-term and long-term capital gains. And a reason to do that is to discourage people from being short-term holders.
They'll end up paying a capital gain tax on the profit. There are other longer-term proposals. Michael Brennan at UCLA proposed that we should create separate markets for corporate dividends at various horizons so that you are focused on a dividend at some date in the future in making an investment. He thought that would focus people's thinking on the fundamental.
Well, now we do have markets for claims on dividends. I don't know. It hasn't had any major impact. The problem is that it is. The problem is here that human psychology is not easily managed. Talk to a psychiatrist, and you'll hear that. We have antidepressant drugs, but you know, they don't, and we have antipsychotic drugs. They help, but they don't cure in a routine way.
The same thing is true for anything you might think of that might deal with irrational exuberance in the markets. I wish we had lithium. which is what they use for irrational exuberance in bipolar disorder, but it's not functioning in the stock market.
Information Competition and the History of Efficient-Market Thinking¶
So let me just talk about what I'm going to give you a history of thought sort of about market efficiency and what people have thought over the time. Starting in the 19th century, information technology began to develop. There was a man named Reuter who just before the invention of the Telegraph decided that stock markets need up-to-date information. And so he thought, how can I get information?
I'll sell information. So what he did is these pigeons fly faster than any human transport, you know, carrier pigeons. So what he did is he set up offices in major European cities and bought pigeons. And as soon as some market event happened in London, he would tie a piece of paper to the leg of the pigeon in Paris, in London, and send the news to Paris. And how long was it take a pigeon to fly from London to Paris?
I don't know. It takes four to six hours for a pigeon to fly from London to Paris. So it gave a trading advantage to people. So he was a big success with his pigeon service. And later, when they invented the telegraph, that was here in New Haven, by the way. Samuel F.B. Morris invented the telegraph. What year was that? Wasn't it around 1840? I don't remember exactly.
The Reuters went to the telegraph to speed it up even more. By the late 19th century, everyone was getting information with the speed of electricity. And they got beepers in the 20th century installed that would beep them when there was some important news. They would carry that around. Even before you had a cell phone, you get a beeper. So the idea is that the information, there's so many smart people trying to get information that it must be hard to beat the market.
And that means that you really can't predict it. So the random walk became plausible because people thought, yeah, all these people are trying so hard to get the information as quickly as they can, and they trade on it as soon as they get it. So doesn't that mean that if anyone has some idea that the market, so if the market goes down in London, it will probably go down in Paris too.
So if the London market falls, you would like to sell in Paris. The problem is you can't get the information in Paris fast enough. But now that they get it really fast, the market must adopt almost, adjust almost immediately. So this is what it came to. The reason efficient market sounds plausible is you have to educate people out of naive ideas that they can predict the market.
So some people will go to their broker and say, you know, the other day I read in the Wall Street Journal that this company has a new drug that's about to come out. So is that a reason for me to buy? Then the broker will tell you, you read it when? Last week? He said, well, when that, your broker might say. When that news came out, I remember my beeper went off immediately, and I rushed to my colleagues and said, what's this news mean?
And someone said, sell, sell right now. And so I placed an order in 15 seconds. He had to say, you've got to beat the others. And then I said, I just sold, but I hadn't even had 15 seconds to talk about it. They talk more about it, and they think, well, maybe I shouldn't have sold. So 30 seconds later, you buy. And then within two minutes, They've gotten expert advice on what it means.
And the market has gone wild, and now it settles down at the new optimal level. So maybe the market wasn't perfectly efficient for 15 seconds. But you would better be educated that if you read something in the newspaper last week, it's already incorporated into market prices. Even in 1889, they didn't have beepers then. Here's a 190. four articles. It's a really nice article by Charles Conant in a magazine. And it's about the beauty of markets. And he was referring to the public, he referred to an ignorant public view that markets are a sort of gambling casino. But he said a moment's reflection should convince such persons that a function which occupies so important a place in the mechanism
of modern business must be a useful and necessary part of the economy. Yeah, what he's getting at is that those prices in markets are the result of people trying to figure out values. And if we didn't have the markets, we wouldn't know what anything was worth. Any business plan involves prices. As a business, you're going to have to buy commodities, or you're going to have to buy commodities, or you're going to have to borrow at interest rate.
All these prices are relevant to an intelligent decision. So not only are markets efficient, according to Kana, they're better knowledge than any individual because it involves so many people all putting their money on the line and trading on it. And then it produces values, which he implicitly assumes in 1904, are the best estimate of fundamental value. And then it drives businesses.
then make decisions for whether to build a new factory or to hire new people based on these prices. So it's a beautiful system that he talks about. Until the financial crisis, textbooks were glowing in defense of the financial market. Even your textbook, Fabozi, in an earlier edition, in 2002, said publicly available relevant information will lead to correct pricing of freely traded securities in properly functioning markets.
Here's another textbook of finance by Richard Breely and Stuart Myers. And in an earlier addition, efficient markets really took hold the theory in people's thinking, starting after Burton Malkale's book. It really was an intellectual revolution. But according to Breely Myers and Allen, security prices accurately reflect available information and respond rapidly to new information as soon as it becomes available.
That's a pretty strong statement. And then they did qualify it. Don't misunderstand the efficient markets idea. It doesn't say that there are no taxes or costs. It doesn't say that there aren't some clever people and some stupid ones. It merely implies that competition in capital markets is very tough. There are no money machine and security prices reflect the true underlying value of assets.
So he's not really qualified, they're not really qualifying it very much. They're telling you to believe prices you see in the markets as if they were truth. They have subsequently, however, in their 2008 edition, this is after the beginning of the financial crisis, they have qualified it. I don't mean to laugh. They're great people. They're reflecting, when you write a textbook, the ideas of the time.
But I'm just using this as a sign of how much thinking has changed. They now say much more research is needed before we have a full understanding of why asset prices sometimes get so out of line with what appears to be their discounted future payoffs. So the efficient markets hypothesis has taken a hit after the financial crisis, and you can see it in the textbooks.
So I think efficient markets is like that. It was a scientific revolution that came out really in the 60s and then the 70s. And it led to the extreme bull market that we had around the world in the 1990s. People then finally decided that whatever the stock market does is right. Whatever the housing market does is right because its markets are efficient. And that unfortunately wasn't quite right.
Harry Roberts is the coiner of the term efficient markets. Although I've traced the history back almost 100 years before him, they didn't call it efficient markets hypothesis. He said there are three forms for market efficiency. Weak form is that information in past prices can't help you to forecast. The semi-strong form of efficient markets is that all publicly information is already incorporated in the market prices.
And the strong form is that all information, including insights, information held by the companies is already incorporated into stock crisis because it leaks out. Companies can't keep secrets. Actually, I think the semi-strong form is the one that we focus on. I don't think that companies can't keep secrets. They sometimes leak out, but they don't always leak out.
So my first question is, so the efficient market's hypothesis was stated 50 years ago. But why is it still a hypothesis? Shouldn't there be enough research and data to either accept or reject it? And even more puzzling, how was the Nobel Prize awarded to someone who clearly advocates this theory, Eugene Phama, and someone who challenges this theory at the same time?
There's a lot of things I have to say about this topic. One of them, it's even more than 50 years old. They didn't call it the efficient markets hypothesis. But going back centuries, there were people who thought, markets are wonderful and perfect. They may not have articulated it as well. The term efficient markets was popularized by Eugene Fama, who won the Nobel Prize with me.
I actually am a big admirer of Eugene Fama, and his papers on this were very interesting. I would say that the efficient markets hypothesis is a half-truth. It's sort of true. It's good. I try to teach it to you. I hope I taught it. Maybe I didn't go far enough in extolling it. But it's useful for you to think of that if you want to go into investing because you probably have exaggerated expectations for what you can do.
So, I said that. Eugene Fama would be pleased to hear this. But then I have also a sense that it's not that a fit. It's not that perfect. and that we shouldn't trust it. So someone listening to efficient markets theory might say that the head of a central bank should never comment on the stock market because the stock market itself is smarter than any individual.
And now I think, you know, maybe they shouldn't usually comment on it. But there are times when the market looks crazy. And I'm sorry, I think that the Fed or the central bank head is smarter than the market, often at least, not always, and ought to comment, ought to make an opinion on that. And that also that certain anomalies have appeared. For example, buying stocks with low price earnings ratios has paid off historically for a long time.
I think that that's probably best explained by psychological theories. That some stocks become ignored and they get underpriced because nobody remembers them. Others get overpriced because they're hot. That's something to keep in mind to understand market fluctuation. So it is a hypothesis. I would say we know, I'll say maybe it's wrong to call it a hypothesis.
The better name for it would be the efficient markets half truth.
Testing Whether Prices Are Right with Present Value and P/E Ratios¶
Now the idea is the price of a stock should be the present discounted value of expected dividends. We'll come back to present discounted values less, but what we have here is that, oops, the price is equal to, well, I'm assuming earnings equal dividends. Price equals earnings divided by a discount factor, which is the interest rate, the discount rate minus the growth rate of earnings.
So price equals earnings over some number. This is called the Gordon model. And these are not necessarily exactly today's earnings, but we'll use that as an approximation. So what it implies then is that unless the risk factor is different for some stock than another, making for a different discount rate, The price earnings ratio should be the same in every stock.
That's the Gordon model. So efficient markets theory has to explain why some companies are priced higher relative to their earnings than others. And it would have to be, if you believe the Gordon model, it has to be something about risk or growth opportunities. So if a company is priced high relative to earnings, it would either have to be because it's low risk, as measured by beta, so we're willing to pay more for it because it's low risk.
Or it would have to be that people have reason to think that the earnings path, the growth rate G, is high, so that you're dividing by a smaller number. So the efficient market theory would attribute, it would explain differences in prices relative to earnings in terms of either the discount rate or the growth rate of earnings. So that markets are all priced right, even though in some markets, the price is high relative to earnings.
So if this then gets at, some stocks are much more expensive than others. Some sell at a price earnings ratio as much as, well, sometimes, even more, but let's say 100 times earnings. And you might ask, why would anyone pay 100 times earnings for a stock? Well, efficient markets theory would say, well, in its infinite wisdom, the market has decided either that this stock is a very good in terms of risk, or, that's R, or that the growth rate of the earnings of this company is going to be phenomenal.
So that a high price earnings ratio should predict high growth rates of earnings. So these are ideas. Why do people think markets ought to be efficient? There's the marginal investor story that the smart guys are the ones who trade regularly. They trade fast and immediately, and they dominate trading. There's also a survival of the fittest argument that some people go to Wall Street who look smart, but you know, they're really not.
smart. I see. On the PG ratios, though, do you think then that different industries shouldn't have different PG ratio averages? Good question. There are historic long-run differences across industries and across countries in price earnings ratios. A good example is Japan, which if you go back to the 1980s, had the overall Japanese market had a price earnings ratio almost as up to 100.
People back then were saying in everywhere but Japan, they were saying, this is crazy. What's with those Japanese? That's too high. But if you go to Japan, it seemed different. I actually did a questionnaire survey comparing U.S. and Japan attitudes toward the Japanese market. And I know this is true. The U.S. people were highly skeptical, but the Japanese were very enthusiastic.
So the price earnings ratios did come way down in Japan after that. I think it's because the Japanese, they're human like all, I'm not criticizing the Japanese, but they got kind of carried away with their success, and they bid the price up. And that can be a psychological cause for differences. And they can be prolonged differences in price earnings ratios.
There could also be other. of course differences like differences in accounting standards.
Smart Money, Market Crashes, and Half-Truths¶
here at Yale with Glenn Ellison did a study of smart money. And they judged smartness by the average score you got on your entrance exam, the SAT exam. Not you, they couldn't get it for the individual. They got it for the college they went to. And they found that people who had, by inference, the higher scores, did outperform the others. And I think that's very significant.
I hate to say it, but not everyone is equally in terms. is equally intelligent. Saturday's newspaper. Someone is doing it again. Mr. David Levine. At this point in the market, when three-quarters of you are predicting a crash as of last Friday, he says, how much of your nest egg to put into stocks? All of it. I was kind of shocked to see it. Right now, when the market looks so tumultuous, he's telling people.
He mentions, he says Warren Buffett has stayed that 90% of your portfolio should be in stocks. I'm wondering, is that recent? When did Buffett say that? But anyway, apparently at some point Buffett said that. He said, Mr. Buffett is too conservative. So I'll stop with that. What I'm trying to convey is efficient markets theory is a half-truth. There are a lot of smart people investing.
But a lot of that smartness is devoted to marketing. and manipulation of your psychology. And this guy, I don't know, he might be right. In fact, it was kind of a thoughtful article, but the sales pitch I thought was kind of a non sequitur. If you read the whole article, why is he so confidently telling you to put everything in the stock market? It's just a gut feeling.
Ultimately, these gut, and maybe he's well-meaning of the whole thing. I'm not trying to disparage people's ethics. I'm saying it may be well-meaning, but the kind of things that get heard are things like this, and they tend to drive market. Back to the efficient markets hypothesis. Do you think that large stock market crashes like that in 1987 or in 2008 disprove or prove the efficient markets hypothesis?
Is this like prices catching up with information or information catching up with prices? Yeah. Well, one problem, if you look at 1987 or 2007 or 2007, there's only a handful of examples. And to disprove a hypothesis, you need a lot of data. But on the other hand, people who lived through these crises have often reported that their faith in the markets was diminished by the experience.
And I think that maybe what they're referring to is how it was reading the news every day about these events and talking to people. about them. I think the experience of living through a crash makes it obvious that human emotions play a role and that people were buying and selling who didn't know what was happening. And so I think that it's often a human judgment thing.
Having experience an event like this makes it seem, you know, it's kind of crazy to think these markets are perfect. You know, I haven't met any way. You know, I haven't met anyone yet who is behaving in a completely rational way. How could the combined effect of all their buying and selling be completely rational? But actually, going back to Eugene Fama, when he was asked about the 2008 prices, he said that the market behaved really well because prices started to fall in advance of when people accepted there was a recession.
So the prices were already incorporated in the recession. So how do you respond to that? So the stock market has been documented. This is known for almost a hundred years. As a leading indicator, the stock market has a tendency to fall before a recession. So what do we take of that? Well, there's two possible explanations. You could say that the market is smarter than everyone else, and it's like a fortune teller, and it sees the future.
But there's another explanation. You could say that, what do you think? Does it cause? It causes the recession. So the market goes down and people say, what's, something's wrong, you know? It's something wrong. like if you take your temperature and you find it's low or high, and you suddenly think, maybe I'm sick. And you said, maybe I feel a little sick. And when you do that with masses of people, it can have that reverse causality.
But in 2009, you can also say that the stock market crashed due to like an underlying problem in the housing market. And so were the prices that the stock market was crashing actually a reflection of like new information the market provided? So is that maybe proving the efficient market hypothesis exactly? Well, this is something I've been involved. in debating for many years.
And there's different approaches to answering your question. But one is to say, how big, in any one of these crashes, how big was the loss to the economy because of the recession? And was the market acting appropriately to that? So for example, in 1929, we had the beginning of a stock market crash that bottomed out in 32, and the market lost over 80% of its real value.
So, wow, that's a big draw, 80%. But then people say, well, but we had the Great Depression after that. Maybe, we did, in fact, we did have the Great Depression. But you know what? The Great Depression wasn't that bad. It wasn't anything like an 80% drop in GDP. It was temporary, got over it. People like to tell stories, and they can tell stories. And they can tell us.
a story that justifies the market as forecasting amazingly. But to me, it's just a story.