When the stock market crashed in 2000, it did not go unnoticed that one group of investors did just fine: those who had bet it would crashor, at least, had not bet that it would keep going up. Soon it seemed that the only smart investment strategy was to be market neutral to own good stocks and to short (bet against) bad ones. That way, you would make money regardless of what the market did. As rumors of spectacular bear-market hedge-fund results made the rounds, moreover, the excitement about this new strategy only grew. Hedge funds offered better returns and lower risk! They made you money even when the market dropped! Soon came the near-universal belief that everyone should own hedge funds.

Not unless you are really rich, really connected, and really know what you are doing.

Avoid hedge funds for the same reasons you should avoid most mutual funds—high costs and the difficulty of choosing a top fund in advance—as well as for reasons that are unique to hedge funds: higher risk, lower returns, and less diversification benefit than you probably think. The best hedge funds and funds of funds, moreover, come with an additional roadblock: You can’t invest in them. Many of the top funds are closed to new investors, and the ones that are open prefer to deal with massive institutions that can invest $10 million to $100 million apiece. And then there’s the overarching problem. To the extent there was a truly compelling opportunity in hedge funds, the secret is out, and thousands upon thousands of exuberant prospectors are already storming the Yukon in search of hedge-fund gold.

Back in ancient times (the mid-1990s), when there were relatively few hedge funds, the total amount of capital dedicated to each strategy was relatively small. For example, only a handful of funds specialized in going long and short technology stocks, and, because of this, there was a lot of low-hanging fruit, especially on the short side. Also, in the mid-1990s, rules about selective information dissemination were less stringent than they are today, so some funds routinely traded with what might today be described as material and nonpublic information. As a result, the top long/short tech funds made a killing.

The surest way to get rich in the investment business is to post a few years of extraordinary returns, because once investors see them, they will pitch camp outside your office and beg you to take their money. Unfortunately, the prosperity will be a mixed blessing, because, thanks to diseconomies of scale, the more cash you receive, the harder it will be to repeat the performance that attracted it. (It is more difficult to earn a 25 percent return when you are managing $10 billion than when you are managing $10 million, if only because you need to find $2.5 billion worth of opportunities instead of $2.5 million worth.)

As the 1990s progressed, and long/short tech funds posted strong returns, the amount of capital allocated to the strategy multiplied. By the end of the decade, there were hundreds of long/short tech funds, each managing tens or hundreds of millions of dollars (or more), and expertly serviced by Wall Street firms, each staffed by platoons of brilliant analysts jetting around the globe seeking the smallest information edge. As the number of funds increased, they began to compete with each other, in addition to competing with mutual funds, pension funds, and other traditional institutional investors. It became harder to maintain an advantage, and, thus, harder to post exciting returns—especially with technology stocks in the doldrums. In the last few years, the returns in the long/short tech strategy have dropped, disappointed investors have withdrawn money in disgust, and many funds have closed their doors (which probably means it’s a good time to invest).

Sooner or later, this pattern will repeat itself with all hedge-fund strategies: Capital will gush toward strategies with attractive returns until the returns are no longer attractive. At that point, only the best funds will earn returns that justify their risks, and, as ever, these funds will be hard to identify in advance. Judging from some recent academic work, in fact, it seems that this has already begun to happen. 

Everyone has heard stories about hedge funds making a killing, and some of the stories are even true: Some funds always hit the ball out of the park, regardless of what the market is doing. But what about the average hedge fund? What about even the above-average hedge fund? How do they do? Not as well as the excitement would make you think.

One problem with evaluating hedge funds is that the industry hasn’t been around long enough for researchers to have much confidence in the significance of its historical results. Researchers did not start gathering formal data on hedge funds until the mid-1990s, so today’s analysts can study only about a decade’s worth of returns (in comparison, data on mutual funds goes back at least to the 1960s). Furthermore, unlike the mutual-fund industry, which has strict reporting standards, hedge funds aren’t required to publish results. The major databases of hedge-fund performance, therefore, suffer from severe reporting biases. which most researchers conclude have significantly overstated the industry’s results.

For example, according to professors William Fung and David A. Hsieh, one of the major hedge-fund performance databases (TASS) reports that from 1994 to 2004, the average annual hedge fund return was an impressive 14.4 percent, net of fees. After adjusting for two major biases, however, Fung and Hsieh concluded that the average performance was only 10.5 percent, close to the return for the S&P 500. The average performance for the fund-of-funds industry over the period, meanwhile—funds of hedge funds run by experts who presumably possess an above-average ability to choose top hedge funds—was 9.4 percent, less than the S&P 500 return. And those numbers are pretax.

But don’t the world’s smartest investors work at hedge funds? Don’t they feast on mistakes made by long-only mutual-fund managers and individual investors? Some of them do, yes. But plenty of mutual-fund managers and individual investors are brilliant, too. And there are now thousands of hedge-fund superstars chasing the same opportunities. No matter how smart hedge-fund managers are, moreover, they have to overcome a performance hurdle that others do not, namely the cost of their own vast compensation.

Most hedge funds charge 1 percent to 2 percent of assets per year, plus 20 percent of gains (compared to most mutual funds, which don’t charge success fees). These fees wallop net returns, so a hedge fund has to generate greater gross returns than an index or mutual fund to post the same net. Even if a hedge-fund manager is skilled enough to beat the market by a few points before fees, in other words, investors won’t necessarily see any benefit. Funds of funds, meanwhile, often charge 1 percent of assets and 10 percent of gains on top of the underlying hedge-fund fees, so their gross returns must vastly exceed those of an index fund to generate the same net return.

For example, assume an S&P 500 index fund charges 0.2 percent of assets and generates a gross return of 10 percent a year, thus producing a net return of 9.8 percent. What gross return would a hedge fund charging the standard two and twenty (2 percent of assets and 20 percent of gains) have to generate to produce the same net return? Fifteen percent. The hedge fund’s fees take such a big bite that the fund manager has to produce a gross return 50 percent better than the index fund’s just to stay even. And a fund of funds? If a fund of funds takes another 1 percent and 10 percent on top of the hedge-fund fees, it would have to generate an 18.5 percent gross return to equal the index fund’s net.

Of course, all anyone really cares about are net returns, and if hedge-fund managers are skilled enough to deliver positive returns when the market tanks, then it doesn’t matter what they pay themselves. Before you wave off hedge-fund costs as irrelevant, however, don’t forget that most hedge funds also come with a healthy dollop of the biggest cost of all: taxes.

The goal of most hedge funds is to make money now. This means trading at a pace that makes even hyperactive mutual-fund managers seem patient. Hedge funds also avoid dividend-paying stocks, because what makes money now is price appreciation. As a result, successful hedge funds often rack up mountains of short-term gains that, for taxable investors, create a huge tax load.

How much can short-term taxes eat into hedge-fund performance? Assume a hedge fund generates annual pretax net gains (after expenses) of 10 percent a year and that 75 percent of the gains come from short-term trading. With a 50 percent total short-term gains liabilityindividual hedge fund investors are usually in the top tax bracketand a 20 percent tax on the balance of the return, this would produce an after-tax return of just under 6 percent. At this rate, a $100,000 investment would appreciate to $405,000 after 25 years. This doesn’t sound bad until you compare it to the after-tax performance of an index fund that generates the same 10 percent return through dividends and realized long-term gains. Assuming a 20 percent annual tax liability for the index fund, the same $100,000 investment would produce an 8 percent after-tax return and grow to $685,000 .

To generate the same after-tax wealth as a mutual fund, in other words, the hedge fund would have to generate an annual pretax return of 14 percent versus 10 percent for the long-term fund. This may be an extreme case—tax situations vary, most hedge funds generate some long-term gains, and most mutual funds have some short-term turnover—but suffice it to say the following: Pretax returns are not created equal, and tax inefficiency makes many hedge-fund returns look much better than they really are.

When you put all the costs together—fees and taxes—you get a sense of just how skilled a hedge-fund manager has to be to generate you, the taxable investor, an above-market, after-tax return. Using the same fee and tax assumptions as above, to equal the net after-tax return of an S&P 500 fund that generates a 10 percent gross return, a hedge fund would have to generate a gross return of no less than 20 percent. A fund of funds, meanwhile, would have to post a gross return of 24 percent. Few, if any, managers are skilled enough to produce such returns consistently.

As hedge-fund returns have declined, the industry has begun emphasizing their diversification benefits. Most hedge funds are not closely correlated with the broader equity and bond markets, so the theory is that a well-chosen basket of hedge funds (or funds of funds) will reduce the risk of a diversified portfolio. According to recent academic research, however, the diversification benefits of hedge funds are overstated.

As described, the most common measure of investment risk is standard deviation: The amount each annual (or monthly) return varies around the average return. When hedge funds are examined using standard deviation, the risk/return profile does indeed seem impressive: Hedge-funds appear to generate returns with less-than-average risk. The trouble is that, unlike most mutual-fund returns, most hedge-fund returns are not normally distributed, so standard deviation does not tell the whole story. Specifically, hedge funds tend to display what academics call negative skewness and positive kurtosis, which means that they have a greater-than-normal likelihood of generating extreme negative results.

The past performance of a hedge fund, in other words, can be really misleading. A mutual fund might slog along with quarterly returns and risks that are accurately described by mean and standard deviation—e.g. GOOD, OKAY, GOOD, GOOD, FAIR, GOOD. With a hedge fund, however, the quarterly pattern might look like this: GOOD, GREAT, GREAT, GOOD, HORRIFIC.

According to professor Harry M. Kat of Cass Business School in London, the risk of extreme negative performance cannot be addressed by buying a basket of hedge funds—a strategy easily implemented by buying funds of funds. Kat concludes that, when one combines several funds in a portfolio (as a fund of funds does), the standard deviation of the portfolio decreases (good), but the risk of an extreme negative outcome actually increases (bad). As the number of hedge funds in the portfolio increases, moreover, so does the basket’s correlation with the overall stock market. This, in turn, reduces the diversification value of hedge funds.

If this sounds like a whole lot of academic mumbo jumbo—If those profs really knew anything, they would be out there making billions instead of mewling from the ivory tower—then look at it this way. About 10 percent of hedge funds close their doors every year—a business strategy employed more often when things have gone badly than when they have gone well—and only one-third have existed for more than five years.

Of course, while the secret that hedge funds can make you money in bear markets was spreading among investors, another secret was spreading among professional money managers: Hedge funds can make you a fortune in any market. Before this revelation, it seemed that the mutual-fund business was the greatest game in town: Try to win but don’t worry if you lose, because your customers won’t know or care. After the compensation packages of a few hedge-fund managers became common knowledge, however, mutual-fund pay seemed like chicken feed. Why make a measly 1 percent a year when you can make 1 percent to 2 percent, plus 20 percent of gains? Answer: Why, indeed. And so the exodus began.

But aren’t those massive hedge-fund pay packages just compensation for higher risk? Isn’t working for a hedge fund riskier than working for a mutual fund? Don’t hedge-fund managers deserve to get paid for the value they generate?

Sort of. Hedge-fund managers, like most capitalists, deserve to get paid whatever the market will bear (in this case, an absolutely dizzying amount). Working for a hedge-fund start-up is indeed riskier than working at, say, Fidelity or Colgate-Palmolive, and, unlike many mutual fund managers, hedge-fund managers usually do tie their fees to performance. But the concepts of risk and performance-based compensation in the hedge-fund business are relative. If a hedge fund folds, its analysts and portfolio managers can usually find new jobs at other hedge funds (or they can just change the name on the door and start a new fund). In the meantime, even if the hedge fund never generates a profit, the managers can take home the same 1 percent to 2 percent of assets that support the average mutual fund in perpetuity. In other words, a hedge-fund manager’s risk is that he or she won’t make a killing. There are riskier pursuits in life.

And then, of course, there is the definition of profit. If the stock market tanks, few investors would resent paying a hedge-fund manager 20 percent of a gain. If the market soars, however, and a long/short equity hedge fund soars with it, has the hedge fund really made a profit? Shouldn’t the profit just be the amount over and above the underlying benchmark return? Arguably, yes. In most cases, however, the hedge fund gets paid for the whole absolute gain, regardless of what the market does.

The good news, for hedge-fund investors, is that academics have recently begun to isolate the passive market factors that drive most hedge-fund returns, the same way they have isolated the factors that produce most stock-market returns. As the conclusions become more accepted, sophisticated investors should be able to benchmark hedge-fund returns more effectively and pay managers only for manager-generated returns above the market average. Eventually, Wall Street will also probably develop passive funds based on the major hedge-fund strategies, which should allow small investors to take advantage of their diversification benefits without today’s costs and risks.

In the meantime, remember this: The risk of hedge funds is far greater for hedge-fund investors than hedge-fund managers, and this risk is bigger than you might think. As with the best mutual funds, the best hedge funds have delivered extraordinary risk-adjusted returns. As with mutual funds, however, to have identified these funds in advance would have required what one professor deems almost supernatural fund-picking talent. And the point of investing intelligently is to avoid needing occult help to succeed.