A further statistical analysis

We also performed a statistical test on the data using the analysis of variance (ANOVA). This allowed us to compare the means (alpha and excess return) between the analyst ratings and expense ratios groups.

There was only a 0.03 (3%) chance (the P-value) that the differences among the mean alpha values for analyst ratings could have occurred by chance. This would be regarded as significant at the 0.05 level. Since the average of the gold ratings outperformed the average of the silvers, which outperformed the other averages, this would seem to indicate that the ratings had significant predictive value. This effect would, however, be judged insignificant when measured on excess return instead of alpha since the P-value was 0.054.

However, the analyses of variance when performed on expense ratio alone (divided into categories in the same numbers as the analyst ratings) produced much more significant results. The P-value for alpha is 0.0004 or very highly significant. The P-value for excess return is 0.003, also highly significant.

In short, expense ratio alone is a highly reliable predictor of alpha and excess return, much higherin factthan the analyst rating. Since the analyst ratings correlate with expense ratio, the accuracy of the Morningstar predictions may be partly or wholly due to the expense ratio differences among the categories.

Conclusion

Our results are preliminary; they reflect a small portion of the overall mutual fund universe and they reflect three years of performance data. It may well be that if more funds were analyzed over a longer time frame, then the results would be more encouraging.

A half-century of academic research on mutual fund performance has failed to find any methodology to reliably predict risk-adjusted outperformance. When funds do outperform, research has shown, outperformance is rarely persistent, particularly over long time horizons.

Our results are consistent with this academic research. Advisors should be wary of any system that purports to identify funds that will deliver risk-adjusted outperformance. They should be equally wary of funds that promote a gold analyst rating as being predictive of future outperformance.

If youre looking for a single metric to select better funds, it is a low expense ratio.

1. We did not compute the probabilities for the negative/highest expense ratio funds because the sample size was too small.