Page 1

ssrn.com/abstract=1785319

Evaluating the Rating of Stiftung Warentest: How good are

Mutual Fund Ratings and can they be Improved?

Sebastian Müller and Martin Weber*

Juli 26, 2011

Abstract

We test the abilities of the Stiftung Warentest fund rating system to predict future fund performance among German

registered funds for six equity categories: Germany, Euro-Zone, Europe, North-America, Pacific, and World.

Stiftung Warentest is a consumer protection agency and a major provider of fund ratings in Germany. Our empirical

analysis documents predictive abilities of the rating system. The reason is that measures of past performance are

positively related to future performance in several of these markets, even after controlling for momentum. Measures

of fund activity are also helpful to predict performance, and in particular to identify likely future losers.

Keywords: mutual funds, performance evaluation, performance persistence, mutual fund ratings, active

management

JEL Classification Code: G11, G12, G1

*Sebastian Müller is from the Lehrstuhl für Bankbetriebslehre, Universität Mannheim, L 5, 2, 68131 Mannheim. E-

Mail: mueller@bank.BWL.uni-mannheim.de. Martin Weber is from the Lehrstuhl für Bankbetriebslehre,

Universität Mannheim, L 5, 2, 68131 Mannheim and CEPR, London. E-Mail: weber@bank.BWL.uni-

mannheim.de. The authors appreciate helpful comments and suggestions from an anonymous referee, Jieyan Fang,

Stephan Jank, Stefan Ruenzi, seminar participants at the University of Mannheim and the 17th annual meeting of the

German Finance Association (DGF) in Hamburg. We also thank Stiftung Warentest for providing data on the

ratings and fund classification, Morningstar for data on mutual fund portfolio holdings and total expense ratios, and

Nico Hemker for excellent research assistance. Special thanks goes to Andrew Patton for providing the code for the

monotonic relationship (MR) tests in Patton and Timmermann (2010) on his web-site. Financial support from the

Deutsche Forschungsgemeinschaft and Boerse Hamburg and Hannover is gratefully acknowledged.

Page 2

ssrn.com/abstract=1785319

1 Introduction

Focusing on the U.S. equity fund market a tremendous amount of academic research has examined

whether measures of past portfolio performance are informative about future performance. A

balanced reading of these papers suggests that there is little evidence of persistence in equity funds‟

risk-adjusted returns after controlling for survivorship bias and for momentum in stock returns (see

e.g. Busse et al. (2010), Carhart (1997), Malkiel (1995), Jensen (1968)). Further, recent studies by

Barras et al. (2010), Fama and French (2010), and Cuthbertson et al. (2010) highlight the problem

of identifying truly skilled fund managers if one has to rely on a limited sample of historical returns.

The results of these papers suggest that even most funds in the extreme right tail of the cross-

sectional estimated alpha distribution have been rather lucky than skilled. Consequently, it is not

surprising that studies analyzing the value of the Morningstar rating system, which is based on past

portfolio performance, find the rating to be a rather poor predictor of future mutual fund

performance for U.S. funds (see e.g. Blake and Morey (2000), Morey (2005) and Gerrans (2006)).

Moreover, as shown by Kräussl and Sandelowsky (2007), the forecasting abilities of the

Morningstar approach have declined over time. However, recent academic work by Amihud and

Goyenko (2009), Cremers and Petajisto (2009), Kacperczyk et al. (2005) and Wermers (2003)

shows that measures which quantify the degree of active portfolio management are associated with

higher risk-adjusted fund returns. These results suggest that investors should consider the extent to

which an open-end actively managed fund really pursues an active strategy in order to select well-

performing funds.

In this paper, we provide evidence on the value of a mutual fund rating system and other

measures of past performance as performance predictors in an international context. Specifically,

we test whether the mutual fund rating system of ”Stiftung Warentest” is able to differentiate

between outperforming and underperforming German registered funds that invest in one of the

following six equity categories: Germany, Euro-Zone, Europe, North America, Pacific, and World.

In addition, we analyze whether measures of fund activity ( ,

Page 3

and ) also predict future fund performance outside the U.S. fund market. Given the results of the

above mentioned studies, the degree of active management appears to be one candidate to better

differentiate between luck and skill in the mutual fund industry. If so, ratings which are based on

historical portfolio returns could potentially be improved by taking these additional measures into

account.

Analyzing the quality of the fund rating system of Stiftung Warentest is interesting for

several reasons. First, Stiftung Warentest is a major fund rating provider in Germany and it covers

the entire German fund market. This allows us to analyze the performance of funds that invest

outside the local German market and even funds that invest worldwide, whereas previous studies on

mutual fund performance and performance persistence outside the U.S. vastly focus on funds solely

investing in their domestic market (see e.g. Otten and Bams (2002), Griese and Kempf (2003),

Korkeamaki and Smythe (2004), Stotz (2007), and Cuthbertson et al. (2010)). Second, Stiftung

Warentest is a consumer protection agency, which aims at providing independent information on

products and services in a broad range of different fields in order to protect consumer interest. Since

Stiftung Warentest receives financial support from the German government and its constitution

prohibits any advertisements, its mutual fund recommendations should be free of any conflicts of

interest which have been documented for other financial advisers (see Reuter and Zitzewitz (2006)).

Third, information intermediaries like Stiftung Warentest exist in many countries all over the world.

Examples include ”Consumers Union” in the U.S. or ”Which?” in the United Kingdom. It is evident

that consumer protection agencies play an important role in generating and disseminating

information to the public. This is in particular true for Stiftung Warentest, which enjoys a very high

reputation among consumers in Germany. According to a recent survey, 96% of the population

above 18 years know the organization, 81% consider the test results as highly reliable and roughly

30% use the recommendations as an orientation guide when buying consumer products or services.1

1 The survey has been conducted by the commercial marketing research institute Forsa in 2007.

Page 4

Fourth, independent, reliable and easy to understand information might be of particular

relevance for private households which want to select a mutual fund. Alexander et al. (1998) show

that many mutual fund customers often lack the financial expertise to assess a product‟s quality.

Moreover, the mutual fund industry has seen a rapid growth over the last years in most developed

countries.2 This development has also led to an increased complexity in selecting a fund: As the

BVI (Bundesverband Investment und Asset Management e.V.) notes, more than 9200 competing

products were available for sale in Germany at the end of 2008.3

Our examination of the predictive abilities of Stiftung Warentest shows that the rating is

positively correlated with future fund performance. Funds in the lowest rating quintile

underperform funds in the highest rating quintile on average by roughly 10 to 20 basis points per

month over the next twelve months, depending on the performance measure. The performance

spreads between the highest and lowest rating quintiles are statistically and economically

significant, and — although they decrease — they do not erode after we control for momentum.

Furthermore, using the monotonic relationship test of Patton and Timmermann (2010), we are able

to empirically verify a strictly increasing relation between the rating and future fund performance

for many fund markets and performance measures under consideration.

The reason for the success of the Stiftung Warentest rating is that performance persists over

a short time horizon for several fund categories which we analyze. However, even high rated funds

do not deliver returns that are significantly above the returns of their benchmark. As a result, no

feasible trading strategy can be build upon solely the rating of Stiftung Warentest in order to

generate superior risk-adjusted returns. We test whether fund expenses might explain our results,

2According to the statistical releases of the Investment Company Institute (ICI), fund assets worldwide amounted to

US$ 19.0 trillion at the end of 2008 coming from a little more than US$ 7.6 trillion at the beginning of this century, see

Investment Company Institute (2009).

3See BVI (2008). The BVI is the central association of the German mutual fund industry and collects information about

the German fund market. It is comparable to the ICI in the U.S.

Page 5

but there are only minor differences in the total expense ratios of high and low rated funds. Finally,

we show that measures of the degree of active management are also related to future fund

performance. Hence, they may be used as additional predictors to select better performing funds

outside the U.S. fund market as well. Our results indicate that taking into account fund activity is

particularly useful to separate skill from luck among underperforming mutual funds.

The remainder of the study has the following structure. In section 2, we describe the

methodology of the Stiftung Warentest rating, the fund sample and our empirical evaluation

approach. Sections 3 and 4 present the major empirical results of this paper. We first analyze the

rating‟s predictive abilities for future fund performance, compare it to alternative measures of past

performance, and examine to which extent the different predictors are related to differences in

expenses. In Section 4, we then test the potential value of quantifying the degree of active portfolio

management in order to identify funds with superior future performance. Section 5 concludes.

2 Data and Methodology

2.1 Stiftung Warentest and Mutual Fund Ratings

The consumer protection agency Stiftung Warentest is a foundation under public-law which was

launched by the German government in 1964. The exclusive goal of the foundation is to evaluate

consumer products and services in an independent and objective manner and to disseminate

information about the quality of different products to the public. By doing so, it aims at enabling

consumers to make better purchasing decisions. Mutual fund ratings of Stiftung Warentest can be

found in its financial magazine Finanztest, which has a monthly print run of 300,000.4 Stiftung

Warentest receives financial support from the Federal Ministry of Food, Agriculture and Consumer

Protection in Germany and its magazines are free of advertisements.

4 To put this into perspective, ”Der Spiegel”, which achieves the highest circulation among magazines of general

interest in Germany, has a weekly circulation of slightly more than one million.