by Ann Cullen

Editor’s Note Many tools for rating the performance of mutual funds and their managers rely heavily on past performance. But what about the future? Now comes a system devised by Randolph B. Cohen and Joshua D. Coval of Harvard Business School, and Lubos Pastor of the University of Chicago Graduate School of Business. At its core, the new system compares a fund manager’s investments with those of other successful fund managers, making past performance less important in the rating scheme. In short, funds are rated by the company they keep. Cohen discusses the system in this e-mail interview. Read an excerpt from the Working Paper.

Ann Cullen: How is your system different than traditional mutual fund rating systems?

Cohen: Our method takes advantage of a wealth of previously unexploited information about fund holdings. Standard metrics rate funds based on past performance. Some alternative approaches have tried to make use of knowledge about the stocks a given fund holds. What is unique in our work is that in rating each fund, we look at the overlap between that fund’s holdings and the holdings of every other fund. We expect a fund to do well in the future only if it holds stocks, which are primarily held by historically successful managers. So, even if your fund has a poor track record, if you hold good stocksi.e. stocks that are primarily owned by top performersyou will rate high by our future-expectations measure. We also apply a technique that looks at the overlap between the trades of funds, rather than the holdings. Using this method, a fund that holds similar stocks to a successful fund, but buys them one quarter later, would not get credit for being similar to the successful fund.

Q: What was the biggest surprise to you in your research?

We expect a fund to do well in the future only if it holds stocks that are primarily held by historically successful managers.

Randolph B. Cohen

A: The increase in precision that our method obtains compared to the standard approach was really striking to us. In many cases our approach is five to ten times more precise. In particular, our approach appears far better able to identify stock-picking ability (if it exists) among funds that started recently. This is exciting because there are good theoretical reasons to think these funds are the most likely to outperform.

Q: How should the average fund investor use your work?

A: Unfortunately the method is quite difficult to implementit took us many months to do it properly, and it required mutual fund holdings data that costs tens of thousands of dollars (which the HBS Division of Research was kind enough to pay for). It’s possible that one of the firms that rate mutual funds will adopt our approach. We’d love to work with such a service to help make this information available to investors at large.

Q: In your analysis you relied rather heavily on Spectrum and other computer-generated data tools. How has the twenty-year availability of computer generated data sets affected the way fund managers can be evaluated now and in the future? Can the data sets help fund managers make investment decisions?

A: In the last five years there has been an explosion of academic research using the holdings data, due to its becoming more accessible and affordable. I think this research has really improved our understanding of the performance of the investment management business. For example, although it is by now well known that net returns to fund investors have on average not exceeded index returns, only in recent years have holdings studies shown that the actual stock picks of fund managers have outperformed the market. It’s just that fees and trading costs have more than eaten up those stock-picking profits.

Our approach appears far better able to identify stock-picking ability (if it exists) among funds that started recently.

Randolph B. Cohen

As to the data helping managers, it depends on how they use it. The most obvious application is for managers to try to copy their more skilled colleagues. This could be an effective strategy, and lead to more efficiently priced stocks, if markets are slow to react and if managers successfully identify the right funds to follow. Since distinguishing ability from luck in investing is always a challenge, that last requirement may not be so easy to fulfill.

Q: Do you think the Web-based data input accounting standards such as XBRL would affect the fund investment strategies of fund managers in the future?

A: Historically careful analysis of corporate information on a large number of companies has required an investor (or at least the investor’s organization) to master a difficult and arcane set of data-handling tools. XBRL has the potential to make security analysis more accessible. In particular, testing and implementing quantitative strategies could become much easier. That could make it tougher for quant managers by introducing more competition, or it could help them by introducing less sophisticated competition. It’s too soon to say.

The Basketball Analogy

People are often judged by the company they keep. Various human characteristics that are difficult to observe directly are commonly inferred from the characteristics of others who behave in a similar manner. For example, a bored air traveler eager to chat about Madonna’s life story is more likely to turn to a fellow passenger reading the National Enquirer than to a passenger reading the Wall Street Journal. because the readership of the former periodical is generally known to be more interested in celebrity gossip. When people compete, their success is often predicted by their techniques, and the quality of a given technique is frequently evaluated based on the track records of the technique’s followers. As an example, suppose a group of basketball players, some of whom shoot with both hands and some with only one hand, have been taking 10 shots each at the basket. So far, the average score of the two-handers is 8/10, while the one-handers’ average is only 4/10. Two players, one one-hander and one two-hander, have completed only half of their shots so far, and both have scored 4/5. Suppose you are to bet on which of the two players is going to achieve a higher score out of 10.

Although the track records of both players are identical, it seems sensible to bet on the two-hander, because the track records of the other players show that two-handed shooters tend to score higher. The one-hander, who is employing what appears to be an inferior technique, is more likely to have been lucky in his first five shots.

Similar to basketball players, active mutual fund managers rely on a variety of techniques when trying to beat their benchmarks, and many of these techniques are common to groups of managers. For example, managers collect information from different sources and use different valuation methods, but there are clusters of managers who use similar sources and similar methods. Managers using similar techniques are likely to make similar investment decisions and also to have similar performance. This paper focuses on the aspect of performance that is due to commonality in the managers’ techniques.

In this paper, a fund manager’s ability to select outperforming stocks is judged by the extent to which his investment decisions resemble those of other successful managers.

One way to assess the similarity of the managers’ investment decisions is to compare the compositions of their portfolios. For example, consider two managers with equally impressive past returns, where one manager currently keeps a big chunk of his portfolio in the stock of Intel, while the other manager holds mostly Microsoft. Suppose also that Intel is currently held especially by managers with good past performance, whereas Microsoft is held mostly by managers with undistinguished records. It seems reasonable to rule that the first manager, whose decision to hold Intel is shared by a higher-caliber set of managers, has superior ability to select stocks, while the second manager, whose techniques coincide with those of sub par managers, has been merely fortunate.

Excerpted with permission from the introduction to the working paper, Judging Fund Managers by the Company They Keep, by Randolph B. Cohen and Joshua D. Coval, Harvard Business School, 2003.

About the author

Ann Cullen is a business information librarian at Baker Library, Harvard Business School, with a specialty in finance.