There are few types of investments that carry the cache of a hedge fund. Available to only a small share of Americans, the industry conjures up images of extraordinary wealth and privilege.

But a peak behind the veneer suggests that there may be less to the opportunity than one might assume. Thanks to a growing number of funds employing similar strategies, returns in the industry have plummeted over the past few decades. And today, most fail to even match the performance of the S&P 500 .

What are hedge funds?

A hedge fund is probably best described in relation to a mutual fund. Like the latter, a hedge fund pools its investors’ money and then invests the pool in an effort to maximize the return. The managers of the fund then collect fees stemming from both the size of the pool and how it performs relative to a predetermined benchmark.

Where hedge funds differ is that they have more regulatory leeway. Depending on the size of the fund and the number of investors, a hedge fund may not be obligated to register with the Securities & Exchange Commission or to file public reports. It follows that hedge funds are often free to employ investment strategies that mutual funds are not. Most notably, they can take on more leverage and short more aggressively than their mutual fund counterparts.

What is the history of the hedge fund industry?

The history of the industry traces its roots back to 1949, when a former Fortune magazine writer named A.W. Jones launched the world’s first hedge fund. His chief innovation was the idea that leverage and short-selling could be combined to decrease risk. In Jones’ language, he used speculative means for conservative ends.

Over the next two decades, the New York-based money manager racked up a seminal performance. According to Sebastian Mallaby’s More Money than God: Hedge Funds and the Making of a New Elite. Jones’ fund earned a cumulative return of nearly 5,000% by 1968. By some measures, Mallaby writes, Jones’ performance in those years rivaled that of Warren Buffett.

With this as a backdrop, it should come as no surprise that Jones soon spawned a cottage industry of innovators, many of whom are now household names. George Soros entered the business in 1969. James Simons of Renaissance Technologies paved the way for quantitative funds in the 1970s. Julian Robertson launched Tiger Management in 1980. And David Shaw, a computer scientist from Columbia University, founded D.E. Shaw in 1988.

But while all of these funds went on to become industry leaders, a much larger share of copycats failed to translate the newfound wisdom into market-beating returns. Over the last decade, for example, the average hedge fund has underperformed the S&P 500 by a factor of two. And the most notable abject failures — those of Long-Term Capital Management in 1998 and Amaranth Advisors in 2006 — even came close to triggering broader financial cataclysms.

How big is the hedge fund industry?

In the more than six decades since Jones inaugurated the modern hedge fund industry with $60,000 in investors’ money and $40,000 of his own, it has grown by leaps and bounds. By 1997, it had swollen to $100 billion. And by 2013, advisors of hedge funds accounted for over $4.6 trillion in cumulative regulatory assets.

Because much of the industry operates outside the purview of regulators, moreover, there’s no doubt that it’s even larger. For instance, the SEC estimates that 32% of hedge fund managers are exempt from registering with the commission. This group oversees an additional $819 billion in assets.

Where there once was a few hundred fund managers, says Bloomberg’s Barry Ritholtz, there are now about 10,000.

The scale of the industry is perhaps best illustrated by the size of its biggest participants. The largest, Bridgewater Associates, has a staggering $87 billion in assets under management. Expand that out to the top 10 and they collectively control over $375 billion.

Why invest in hedge funds?

The original reason to invest in the hedge fund industry was because it, at least theoretically, offered investors a better return than the broader market without also exposing them to higher degree of risk. Indeed, this was the very impetus for A.W. Jones’ original fund.

As the industry has expanded, however, this rationale has eroded. Since at least 2005, the average fund has dramatically trailed large-cap stocks. While the S&P 500 has returned roughly 60% over this time period, the hedge fund industry as a whole is up only 30%.

Now, to be fair, there continue to be a number of outliers. During the financial crisis, a handful of funds predicted the coming mortgage catastrophe, earning billions of dollars in profits for their investors. And more recently, the 10 best performing hedge funds in 2013 produced an average annual return of 43.7%, handily outpacing the equities market.

The answer to the question of why people invest in hedge funds, in turn, is no different than the reason anybody invests in anything. That is, people invest in hedge funds to make money. And just like any other type of investing, some are rewarded for the decision while many more are not.

The bottom line on hedge funds

Ultimately, only a small portion of Americans will ever have the opportunity to invest in a hedge fund. Is that a bad thing? If you look at their returns — outside the cream of the crop, that is — it certainly doesn’t seem like most of us are missing out on much.