The Finance Professionals’ Post educates readers in the finance and banking sectors on the forces that shape their business. The FPP is a publication of the New York Society of Security Analysts (NYSSA).

05/19/2010

Whither Efficient Markets? Efficient Market Theory and Behavioral Finance

The notion of efficient markets has been the subject of rigorous academic research and intense debate for more than a century. As early as 1889, George Rutledge Gibson wrote in The Stock Exchanges of London, Paris, and New York that when “shares become publicly known in an open market, the value which they acquire may be regarded as the judgment of the best intelligence concerning them.” A reference to the concept of efficient markets is also found in French mathematician Louis Bachelier’s 1900 dissertation Théorie de la Spéculation . But it wasn’t until the mid 1960s, through the independent work of MIT economist Paul A. Samuelson and Eugene Fama, then a PhD candidate at the University of Chicago, that the efficient markets hypothesis (EMH) gained widespread acceptance.

“Samuelson and Fama arrived at the concept of market efficiency from very different research paths,” explains Andrew Lo, director of the MIT Laboratory for Financial Engineering. Samuelson was studying temporal pricing models of storable commodities. In a 1965 article in the Industrial Management Review . he proposed that in an efficient market, properly anticipated price changes must be unforecastable (Lo 2007). That same year, Fama was independently focused on measuring the statistical properties of stock prices and in resolving the debate between technical and fundamental analysis. “He was seeking to determine if markets worked properly or if they required intervention, ” Lo explains. Fama ultimately determined they did not. He spent the next 10 years testing the hypothesis that stock prices reflect all known information and instantaneously adjust to reflect new information.

Fama defined an efficient market as a market: (1) with a large numbers of rational profit maximizers actively competing against each other to predict future market values of individual securities; and (2) in which important current information is almost freely available to all participants. “In an efficient market, on the average, competition will cause the full effects of new information on intrinsic values to be reflected instantaneously in actual prices” (Fama 1965).

The theory that stock prices instantaneously adjust to reflect new information leads to the view that stock prices are unpredictable and follow a random walk. “If the flow of information is unimpeded and immediately reflected in stock prices, then tomorrow’s price change will reflect only tomorrow’s news and will be independent of the price changes today. Since news is, by definition, unpredictable, the resulting price changes also must be unpredictable,” Burton G. Malkiel, professor of economics at Princeton University, explains in a 2003 paper .

In his groundbreaking paper, “Efficient Capital Markets: A Review of Theory and Empirical Work ,” Fama proposed three types of efficiency:

• The weak form asserts that current securities prices fully reflect all information contained in past price movements. Therefore, future prices cannot be predicted by analyzing past prices. According to Clarke et al (2001), “the weak form got its name for a reason—security prices are arguably the most public as well as the most easily available pieces of information. Thus, one should not be able to profit from using something that ‘everybody else knows.’”
• Semistrong-form efficiency holds that share prices fully reflect all the relevant publicly available information. This includes not only past price movements but also data contained in published financial reports and SEC filings, such as earnings and dividend announcements, rights issues, technological breakthroughs, resignations of directors, and announced mergers. The semistrong-form of efficiency implies that there is no advantage to be gained from analyzing publicly available information after it has been released, because the market has already absorbed it into the price.
• Strong-form efficiency asserts that all relevant information, including that which is privately held, is reflected in the share price. Here the focus is on insider trading, in which a few privileged individuals (for example directors) are able to trade in shares, as they know more than the normal investor in the market. In a strong-form efficient market even insiders are unable to make abnormal profits. The market is acknowledged to be inefficient at this level of definition.

Ray Ball, professor of accounting at the University of Chicago’s Booth School of Business, sums up significance of Fama’s efficient markets theory. By merging two crucial insights—the notion that competition enforces a correspondence between revenues and costs and that changes in asset prices are a function of the flow of information to the marketplace—the efficient markets theory tells us that competition among market participants will cause the return from using information to be commensurate with its cost. Therefore, one cannot expect to earn above normal returns from using publicly available information. “As simple as they may seem in hindsight, these predictions were revolutionary at the time,” Ball writes. They “irreversibly changed the thinking of not only many economists—but of a great many practitioners—about how securities markets behave” (Ball 2009).

In its purest form, the EMH obviates active portfolio management, calling in to question the very motivation for portfolio research. The theory’s crucial implication—that it’s impossible to beat the market—is the logic underlying the development of index funds. An index fund relies on a passively managed portfolio of securities to closely track an index, such as the Standard & Poor’s 500, while saving transactions costs and management fees.

Fama’s efficient markets hypothesis was subject to rigorous testing in the late 1960s and 1970s. By 1975 the preponderance of evidence supported the view that securities markets were efficient. “There is no other proposition in economics which has more solid empirical evidence supporting it than the efficient markets hypothesis,” Harvard economist Michael Jensen (1978) wrote.

But even as EMH gained dominance, academics and practitioners continued to debate its merits. By 1978 a significant body of research documented profitable selection rules based on publicly available information, such as the tendency for stocks with low price earnings ratios and high dividend yields to outperform the market and the incremental returns in excess of the amount needed to compensate for the additional risk of small capitalization stocks. In 1978 the Journal of Financial Economics devoted a special issue to articles documenting a dozen such pricing anomalies.

But the most persistent challenge to Fama’s efficient markets hypothesis has come in the last 30 years from the growing field of behavioral finance—the branch of finance and economics that applies research from the fields of psychology, sociology, and, more recently neuroscience to understanding investor behavior.

Behavioral finance takes issue with two crucial implications of the EMH: (1) that the majority of investors make rational decisions based on available information; and (2) that the market price is always right. Proponents of behavioral finance, or behaviorists. as they often are known, believe that numerous factors—irrational as well as rational—drive investor behavior. In sharp contrast to efficient markets theorists, behaviorists believe that investors frequently make irrational decisions and that the market price is not always a fair estimate of the underlying fundamental value.

Much of behavioral finance research is focused on determining the precise degree to which various market forces—including rational analysis of company-specific and macroeconomic fundamentals; human and social psychology; and cultural trends—influence investors’ expectations and determine their level of confidence or fear. Behaviorists believe that at times, the real determinants of stock market movements are the forces of human and cultural psychology, or animal spirits (a term coined by economist John Maynard Keynes in his 1936 book, The General Theory of Employment, Interest, and Money . to describe the naïve optimism and confidence that fuels periods of economic prosperity).

Refuting Fama’s crucial assertion that the market price is always right, behaviorists “believe investor psychology can drive market prices and fundamental value very far apart,” explains Hersh Shefrin, professor at the Leavey School of Business at Santa Clara University and author of Beyond Greed and Fear: Understanding Behavioral Finance and the Psychology of Investing .

Some proponents of behavioral finance believe it’s possible to capitalize on the pricing inefficiencies caused by investors’ behavioral biases and have designed investment strategies around the concept. To cite a few examples, legendary speculator George Soros has made a fortune by taking the other side of trades driven by investors’ fear and panic. David Dreman, chairman and chief investment officer of Jersey City–based Dreman Value Management, has long focused on exploiting equity analysts’ tendency to overreact to news—both good and bad. And a growing number of other investment firms, including Fuller and Thaler Asset Management, LSV Asset Management, and Martingale Asset Management, employ investment strategies designed to exploit investors’ tendency toward psychologically induced error. What’s more, “all mean-reversion strategies, including convertible bond arbitrage, risk arbitrage, and fixed-income arbitrage, are based on the investors’ tendency to overreact,” notes Lo.

Still, many proponents of behavioral finance agree with at least one implication of the efficient market theory—that it’s not possible to reliably earn abnormal returns. “One usually can’t capitalize on the pricing anomalies,” says Jay Ritter, professor of finance at the University of Florida.

“Investors tendency toward psychological error brings not just [new] opportunities for positive alpha, but also additional time-varying risk,” Shefrin explains. “The sentiment component of risk is very difficult to gauge. This is why, in practice, most investors should not expect to beat the market—despite the theoretical possibilities for doing so,” he says.

“I believe the market is irrational; but I am not a psychiatrist of the market,” adds Meir Statman, professor of finance at the Leavey School of Business at Santa Clara University and a pioneer in the field of behavioral finance. He makes no secret of the fact that he invests primarily in passively managed index funds.

The origins of today’s school of behavioral finance are generally traced back to the work of two psychologists, Daniel Kahneman and Amos Tversky, on how people make decisions involving risk. In the 1980s behavioral finance researchers, such as Werner De Bondt, Robert J. Shiller, Andrei Shleifer, and Richard Thaler, to name a few, began to focus on the study of the time series properties of prices, dividends, and earnings. The objective was to determine whether stocks exhibit volatility in excess of the amount predicted by the efficient market hypothesis.

“The [pricing] anomalies that had been discovered [in the 1970s] might be considered at worst small departures from the fundamental truth of market efficiency, but if most of the volatility in the stock market was unexplained, it would call into question the basic underpinnings of the entire efficient markets theory,” Shiller (2003) wrote. In 1981 Shiller published an article in the American Economic Review in which he documented evidence of price movements much greater than an efficient market would allow. Four years later Richard Thaler and Werner de Bondt (1985) published a study that concluded that the stock market tends to overreact to a long series of bad news.

Over the last 30 years, research on how investors make decisions has lead to a large body of literature documenting the systematic errors in human thinking that predispose investors toward irrational investment decisions. Chief among the human cognitive biases is the pervasive human tendency to be overconfident. “People think they know more than they do,” Shiller says. “Overconfidence appears to be a fundamental factor in promoting the high volume of trade we observe in speculative markets,” he says. Without overconfidence, Shiller reasons, there would be little trading in financial markets. If traders were completely rational, half would believe they are below average and would be unwilling to enter into speculative trades with the other half. Shiller also notes that magical thinking (such as the belief that if one buys a stock it will go up) also contributes to investor overconfidence.

Confidence, which implies trust, encourages investors to go beyond the rational. “The truly trusting person often discards or discounts certain information,” writes Shiller and Nobel Prize–winning coauthor George A. Akerlof, professor of economics at the University of California at Berkley, in their 2009 book, Animal Spirits . Confidence often prevents people from rationally processing available information. It can even prevent people from acting rationally on information they have processed, Shiller and Alkerlof explain. Overconfidence is among the critical factors that prevent investors from heeding the warning signs that often come months, or years, before a market crash.

As Shiller highlights in his recently updated book, Irrational Exuberance . investors’ confidence in bull markets (as well as their fear, or pessimism, in bear markets) is amplified through a feedback mechanism, often called a feedback loop. by which the market sends signals back to investors in the form of price increases (or decreases). These price signals confirm their investment decisions and encourage more investors to follow suit, creating a cycle that continuously repeats itself. Shiller believes that changes in earnings estimates, whether up or down, are themselves part of the same psychological feedback that amplifies the bubble or downturn—or, negative bubble, to use Shiller’s terminology.

There are many other human cognitive biases that cause investors to make systematic errors which fuel their tendency toward irrational exuberance and its polar opposite—fear and pessimism. Among these biases are:

• Representativeness: This heuristic explains the human tendency to make judgments by looking for familiar patterns and assuming that future patterns will resemble past ones—often without thinking about the reasons for the pattern in the place. “Many people fix their attention on plots of rising stock prices in newspapers every day, and they seemed to come away with an intuitive feeling that every decline is reversed, to be followed swiftly by new highs,” Shiller writes in Irrational Exuberance. The stock market’s repeated pattern of reversing declines has led investors to expect the pattern to continue.
• Confirmation Bias: This aspect of human behavior explains the tendency for people to actively seek information that supports their existing beliefs and, conversely, to underweight information that runs counter to their views.
• Quantitative Anchors: Psychologists have demonstrated that people tend to make decisions based on whatever information is available at hand, leading to the human tendency to overweight the recent past. In making judgments about stock prices, the most likely anchor is the recently remembered price. “The tendency of investors to use this anchor enforces the similarity of stock prices from one day to the next,” Shiller (2006) explains.
• Moral Anchors or Stories: Stories play a big role in driving the stock market. Stories provide investors with easy-to-grasp reasons to justify their investment decisions. As Shiller notes, those who sell stocks to the general public often tend to tell a story about the company and its product, omitting many facts—such as price, future earnings, and dividends—pertinent to a rational analysis of the stock’s investment merit. “These quantitative factors are not as congenial to the narrative-based decision making that comes naturally to people,” Shiller says.

Stories played a key role in fueling the dot-com bubble. “They caused investors to bid up of the prices of companies with no earnings at all,” notes David Dreman. He recalls: “Investors were so enamored with stories about the Internet; they were willing to pay 40 to 50 times earnings for stocks with very poor quality earnings.”

Herd Behavior and Information Cascade: Herd behavior is a key characteristic of every speculative bubble. It occurs when investors follow the majority view or believe authorities even when they plainly contradict matter-of-fact judgment. Herd behavior is a result of several biases, including authority bias, or the tendency to follow the actions and words of experts or those whom we believe know more than we do. “Going against the consensus activates the pain centers of the brain, even when the consensus clearly is wrong,” explains Richard Peterson, MD, a managing director at MarketPsy Capital Markets, a Santa Monica, California–based investment firm that uses a behavioral-finance-based strategy to arbitrage sentiment-based stock mispricing.

In Irrational Exuberance. Shiller refers to a similar phenomenon, information cascade, which occurs when investors observe the actions of others and then make the same choice, independent of their own private information signals. Because it is usually sensible to do what other people are doing, psychologists consider this behavior to be the result of rational choice. As the academic literature documents, information cascade is fragile and can be easily overturned by a small amount of public information. That is, even though a million people may have chosen one action, seemingly little information can induce the next million people to choose the opposite action (Bikhchandani et al 1992). “Ultimately, all information cascade theories are theories of the failure of information about true fundamental value to be disseminated and evaluated,” Shiller says.

Behaviorists assert that taken to the extreme, investors’ irrational tendencies, coupled with amplification and feedback mechanisms lead to destructive cycles of boom and bust and cite the 1998 to 2000 boom in Internet stocks and the more recent housing boom—and their sharp reversals—as evidence.

Not surprisingly, the debate between proponents of EMH and behavioral finance reached a fevered pitch in the 1990s, as an increasing number of market observers perceived the boom in Internet stocks as evidence of both investors’ irrationality and the fact that market prices stray far from fundamental value. “The markets [were] high because of the indifferent thinking of millions of people. motivated substantially by their own emotions, random attentions, and perceptions of conventional wisdom,” Shiller writes in Irrational Exuberance.

Stocks with no earnings whatsoever were selling at sky-high multiples. As an example, Shiller cites the case of eToys.com, a start-up Internet toy retailer which shortly after its 1999 initial public offering had a stock value of $8 billion; fiscal 1998 sales of $30 million; and profits equal to negative $28.6 million. In comparison, established competitor Toys “R” Us had a stock value of only $6 billion—despite fiscal 1998 sales of 11.6 billion, 400 times larger than that of eToys.com, and positive profits equal to $376 million.

“In conventional, classical economics there is no real place for bubbles or manias,” says Tim Lee, founder of Pi Economics in Stamord, Connecticut, and author of Why the Markets Went Crazy .

Of course, the housing boom, its reversal, and the resulting financial crisis have added much fuel to the fire. Behaviorists see investors’ failure to accurately account for low probability risks as further evidence of their tendency toward irrational exuberance. “The market as a whole made a conceptual mistake in underestimating the possibility of a big downturn in the housing market,” says Ritter. Ritter explains that the market was operating under the (erroneous) assumption that just because there had never been a significant decline in housing prices (unadjusted for inflation) in the past, it would not happen in the future. “Groupthink caused most investors to extrapolate from past prices without thinking about the downside risk associated with the run-up in real estate prices,” he said.

Many efficient market theorists and behaviorists alike, however, are quick to disagree that the housing boom was fueled by an irrational assessment of risk. Simon Gervais, associate professor of finance at the Fuqua School of Business at Duke University—a “cautious behaviorist,” in his own words, says that for behavioral biases to affect the economy as a whole, a large number of agents must simultaneously exhibit the exact same behavioral biases, while other investors are unable to identify the behavior and take the opposite position.” This wasn’t the case during the housing crisis, according to Gervais. He believes it’s possible that the majority of homeowners were affected by the similar behavioral biases when they chose to obtain the mortgage loans. But he says it’s unlikely the actions of bankers, brokers, rating agencies, institutional traders, and money mangers were driven by irrational thinking.

Gervais, as well many other economists—from both the behaviorist and EMH camps—believe the factors responsible for the recent financial crisis are more closely associated with traditional economic theory. “The housing bubble was fueled by people following rational incentives,” Gervais says.

“Smart investors know that financial models are only as good as the assumptions that are fed into them,” says John Cochrane, professor of finance at the Booth School of Business at the University of Chicago. He believes the executives at the firms that invested heavily in the high-yielding collateralized debt obligations (CDOs) that led to the financial crisis were aware of the risk exposures accumulating on the company’s books. They rationally chose to assume the risk because they were being paid to take it. “The firms that invested in these securities were making huge profits because they were, in effect, writing earthquake insurance,” Cochrane says.

Lee also believes that financial institutions loaded up on high-yielding CDOs for purely rational reasons. “They were operating with the belief that, in a worst case scenario, governments would step in to avert a crisis.”

In the aftermath of the financial crisis, some critics have blamed investors and regulators’ blind faith in efficient markets as the cause. “The incredibly inaccurate efficient market theory [caused] a lethally dangerous combination of asset bubbles, lax controls, pernicious incentives, and wickedly complicated instruments [that] led to our current plight,” market strategist Jeremy Grantham wrote in an October 2009 letter to shareholders.

Writing in the Washington Post . financial journalist Roger Lowenstein, was more direct: “The upside of the current Great Recession is that it could drive a stake though the heart of the academic nostrum known as the efficient-market hypothesis.”

Not so fast, say efficient markets theorists—and many other economists and financial market scholars. “The fact that yields on the [mortgage backed derivatives] were high despite their investment grade rating indicated that the market was rightly suspicious of the quality of the securities, and this should have served as a warning to prospective buyers,” Wharton School finance professor Jeremy Siegel wrote in the Wall Street Journal (2009).

“If you believe in market efficiency, you would not have made that mistake, says Jonathan Berk, professor of finance at Stanford University Business School. “You would know that there’s something fishy about a triple-A-rated security that is yielding more than other triple-A-rated securities. “The collapse of these securities is the best example of market efficiency that we have to date.”

One does not have to be an economist to make the prediction that the debate between behaviorists and efficient markets theorists will not end any time soon. Still, there are several points at which the two schools intersect. One is the recognition that the efficient markets hypothesis is among the most important contributions to modern finance. “It permeates and pervades everything we do in finance,” says Lo. Behaviorists also believe that efficient markets are the ideal to which we must strive. “If investors could learn to use behaviorists’ ideas to prevent themselves from succumbing to psychologically induced errors, we would wind up exactly where the efficient markets school predicts—with markets being efficient,” says Santa Clara University’s Shefrin.

“The efficient markets theory is not wrong; it’s just incomplete,” says MIT’s Lo. It tells only half the story—how markets behave under the right conditions. Expanding on a concept first expressed by James Surowiecki in his 2004 book, The Wisdom of Crowds . Lo explains that research has identified the one condition under which large groups are more likely to make a rational decision. That condition is a certain degree of independence among the individuals that comprise the group. “To have the wisdom of crowds, you need a broad and diverse a set of market participants without any single participated becoming too big or influential,” he adds. When the members of the group have different motivations; work independently; gather and analyze a lot information; and generate different ideas, the mistakes people make cancel out, and the aggregate estimate tends to be very accurate, Lo says.

Conversely, when the condition of independence is violated—that is, when all members of a group think exactly alike—the aggregate estimate tends to be inaccurate. “When everybody thinks exactly the same way and has the same information and motivations; we see, in lieu of the wisdom of crowds, the madness of mobs,” Lo says. Underlying the herd-like behavior is one of two emotions: greed or fear. “An extreme amount of greed or fear always leads to mob-like behavior,” Lo says. Greed can make the majority of investors want to buy the same group of stocks, like Internet stocks. On the other hand, fear of losing money causes everyone to pull their money out of the market at once.

Regardless of whether one subscribes to the efficient market theory or not, most observers agree that creating the conditions that promote the wisdom of crowds must become a top priority for investors, policymakers, regulators, governments, and central bankers. The precise role of each of these stakeholders, however, is the subject of an even larger academic debate.

Bikhchandani, Sushil, David Hirshleifer, and Ivo Welch. October 1992. “A Theory of Fads, Fashion, Custom, and Cultural Change as Informational Cascades .” Journal of Political Economy. vol. 100, no. 5. 992–1026.

Fama, Eugene F. September/October 1965. “Random Walks in Stock Market Prices .” Financial Analysts Journal. vol. 21, no. 5. 55–59.

Jensen, Michael C. June–September 1978. “Some Anomalous Evidence Regarding Market Efficiency .” Journal of Financial Economics .

Lo, Andrew. 2007. “Efficient Markets Hypothesis .” In The New Palgrave: A Dictionary of Economics. 2nd ed. New York, NY. Palgrave McMillan.

Malkiel, Burton G. April 2003. “The Efficient Market Hypothesis and Its Critics .” CEPS, working paper no. 91.

Siegel, Jeremy. October 27, 2009. “Efficient Markets Theory and the Crisis .” The Wall Street Journal .