MarkHulbert

It has always been difficult for investors to consistently beat index funds. It has been nearly impossible lately.

And there’s a double whammy: The small number of advisers who outperform the market rarely can keep doing so.

One big culprit, experts say: the rise of sophisticated computer trading programs.

Consider the 51 advisers out of more than 200 on the Hulbert Financial Digest’s list who beat the market in the decade-long period that ended April 30, 2012, as measured by the Wilshire 5000 Total Market index, including reinvested dividends.

Of that group, just 11 — or 22% — have outperformed the overall market since then. On average over the last year, they have lagged the Wilshire index by 6.2 percentage points.

That’s no better than the percentage that applies to all advisers, regardless of past performance. In other words, going with a recent market beater doesn’t increase your odds of future success.

“Before the era of computer-dominated trading, it was slightly easier to identify winning advisers in advance, because you could more easily understand and evaluate what they were doing,” says Lawrence G. Tint, chairman of Quantal, a risk-management firm for institutional investors, and former U.S. CEO of Barclays Global Investors.

Getty Images

Going with a recent market beater doesn’t increase your odds of future success.

One major reason why machines are winning is our inability to process lots of financial data, which is getting more complex and voluminous every year.

Terrance Odean, a finance professor at the University of California, Berkeley, has extensively studied the behavior and performance of individual traders. He points out that there used to be another human being on the other side of the trade when an individual bought or sold a stock. “Now it’s a supercomputer you’re competing with,” says Odean.

“Individuals are no longer playing against Grandmasters; they’re playing against Deep Blue,” he says, referring to the famous battle in the 1990s between chess’s Grandmasters and International Business Machines’ IBM, -2.34%  supercomputer Deep Blue. Individual investors “will almost certainly lose.”

Another reason traders are losing out to machines is their general inability to assess complex data. They look at the same set of facts on different occasions and reach different conclusions, and they unwittingly let their emotions dominate their intellect.

Daniel Kahneman, professor emeritus of psychology and public affairs at Princeton University and the 2002 Nobel laureate in economics, has widely studied this phenomenon. In his 2011 book “Thinking Fast and Slow,” he reviewed more than 200 academic studies over the past five decades that analyzed head-to-head contests between human beings and mechanical algorithms.

HEDGE FUNDS: SPECIAL REPORT

Kahneman reports that man consistently loses out to machine in a wide variety of pursuits, ranging from medicine to economics, business, psychology and even things like predicting the winners of U.S. football games and judging the quality of Bordeaux wine. In each of these domains, he reports, “the accuracy of experts was matched or exceeded by a simple algorithm.”

Betting on the pros

Some traders hold out the hope that they can beat the market by following the lead of an investment adviser. But it is close to impossible to identify these advisers in advance, according to Tint.

“The average reader of The Wall Street Journal simply won’t be able to identify these market-beating advisers,” he says. After all, “repeated studies have shown that even the best institutional investors have been unable to identify them in advance.”

Tint adds that there is an above-average chance that an awful adviser will continue to perform terribly. This creates the mathematical illusion that there also is persistence among high-ranking managers and that we can beat an index fund by following one of those top performers, he argues. But all it really tells us is that it’s a good idea to avoid a terrible adviser.

Regulators Urged to Beef Up Insider Trading Rules

Former SEC commissioner has urged the government to beef up rules that regulate corporate executives’ stock sales.

There’s another reason why it is so hard for top-performing advisers to beat the index over the long term, says Tint, even when their numbers were powered by genuine ability rather than sheer luck.