Several months ago, Standard and Poor’s issued a recommendation to use the Case-Shiller non-seasonally adjusted Home Price Index (HPI) instead of the seasonally adjusted version of the index. Their reasoning was that seasonality in the index was no longer constant across time but, instead, seemed to be increasing in amplitude in recent years.

At the time, we (more specifically, our team member, Yeng Bun ) reproduced S&P’s decomposition of HPI seasonality and compared it to that found in the Zillow Home Value Index (ZHVI). See the first figure below. Two aspects of this chart warrant attention. The first is the seasonality inherent in the Case-Shiller data (the pink line) which, of course, was the very reason behind S&P’s announcement in the first place. The second is the relative lack of seasonality in the ZHVI (the red line). This is one of the reasons that we frequently argue that the ZHVI gives you a better sense of what’s happening in the market than the Case-Shiller HPI.

Today, we decided to go a bit further though and actually explore the source of the seasonality present in the Case-Shiller HPI so that we can better understand the precise bias that is being introduced. We’ve said before that we suspect that a chief source of the seasonality is the changing proportion of foreclosure re-sales in the monthly mix of all sales transactions. Since the Case-Shiller HPI includes foreclosure re-sales (which are not included in the ZHVI), it’s measuring not just the change in values of conventional, full-value, non-distressed inventory but also the price drop associated when a home is sold as a foreclosure by a motivated seller (i.e. the bank) and the subsequent sharp price increase when that same home is sold again for a much higher price (because it is no longer being sold as an REO but, instead, as a conventional, non-distressed transaction).

The result of Case-Shiller’s inclusion of foreclosure re-sales in their home price index is that, in months when foreclosures make up a higher percentage of real estate transactions, the home price index is pushed down because more of the transactions included in the index for that month are cheap foreclosure re-sales (which, we’ve found. typically trade for almost 30% less than non-foreclosures). The converse is true in months when the percentage of foreclosure re-sales declines.

Winter months tend to have the highest number of foreclosure re-sales as a percentage of all sales because non-foreclosure real estate sales are at their lowest in these months. Conversely, spring and summer have the lowest foreclosure re-sale rates because the volume of real estate transactions is the greatest in these months. The result is shown in Figure 1 below which charts both the quarterly change in the Case-Shiller national HPI and the quarterly change in the foreclosure re-sale percentage. Spot any relationships? (Hint: one goes up when the other goes down.)

So, how much of the change in the Case-Shiller HPI is attributable to real changes in home values versus changes in the foreclosure re-sale rate? We’d like to know this, of course, because it’s the former that we care most about and that we look to a home price index to tell us. To answer this question, we can specify a statistical model that expresses the HPI as a function of both real home value changes and the foreclosure re-sale percentage. We’ve got the foreclosure re-sale rate already (it’s shown in Figure 1, after all). For the measure of real underlying home values, pardon our conceit, but we’ll use the national Zillow Home Value Index.

Before we do this, however, let’s start with a couple baseline models just so we can get our bearings and better see the contribution that each additional bit of information makes to the prediction of the HPI. Let’s first specify a model that tries to predict the current Case-Shiller national HPI value with only the HPI value from the prior quarter. Table 1 below presents the regression results from this model. It turns out that we can predict the next HPI value pretty well, a finding that is not terribly surprising since any single Case-Shiller data point is an average of that month’s value and that of the two preceding months. The R-squared value for the model indicates that we can explain 94% of the variation in the annualized Case-Shiller HPI just by knowing its prior value. Not bad.

Next, let’s add in our knowledge of how home values are performing in the current period by adding the ZHVI for the current period into the model (see Methodology below for the exact variable used). Here, we’ll attempt to predict the current Case-Shiller HPI value with two pieces of information: the prior value of Case-Shiller HPI and the current value of the ZHVI. The coefficients for this model are shown in Table 2 below. ZHVI is a highly significant predictor and the overall model fit measured by the R-squared coefficient increases to 98% (for those statistical wonks out there, an F-test for the improvement of this model over the previous baseline model is also statistically significant).

At this point, we’re ready to add in our knowledge of foreclosure re-sales and determine whether this information adds additional explanatory power to our prediction of the HPI (and, therefore, whether foreclosures are really biasing the HPI). Our final model adds the foreclosure re-sale percentage to the previous model. The coefficients for this model are shown in Table 3 below. As we suspected, the foreclosure re-sale variable is highly significant in the model and the overall model fit increases to 99.3% (and, again, an F-test for the improvement of the model over the prior model in Table 2 is statistically significant). The predictions from this model charted against the actual Case-Shiller HPI values are shown in Figure 2 below.

So, what does this all mean? If you’ve made it this far through the analysis, I’ll try to boil it down simply for you. The Case-Shiller Home Price Index is measuring a lot more than just the change in the values of homes sold through conventional, non-distressed transactions. It is also capturing the prevalence of foreclosures in the real estate market. If you like having a measure that tracks two elements of the market and asks the user to parse the two elements themselves, then stick with Case-Shiller. If, on the other hand, you’d really like a measure of just home value change itself, the Zillow Home Value Index is a better measure. The wild swings of the Case-Shiller HPI – down more than 7% on a quarterly basis in early 2009, then up more than 3% quarterly by mid-2009 and then down by almost 3% by early 2010 – are not just swings in home prices induced by the tax credits. These swings are not actually happening in underlying home prices. Because of the magnitude of the foreclosure crisis, many analysts are tracking an indicator (the HPI) that is becoming increasingly detached from underlying home values and is likely to remain detached for at least another couple of years while the market works through the backlog of foreclosures. Reliance on the Case-Shiller HPI makes the market look depressingly bad (worse than it should) in the winter months and gets our hopes up too much in the spring and summer. The reality is actually better in the winter and worse in the spring/summer than the Case-Shiller HPI would lead us to believe, and I believe it would help our characterization of the housing market if analysts understood and accounted for this fact.

The dependent variable in each model is the year-over-year change in the national Case-Shiller Home Price Index. measured on a quarterly basis. There are three independent variables: ZHVI. foreclosure re-sale percentage. and a lag of the dependent variable. The ZHVI is entered into the model as the difference from the prior quarter in the year-over-year change in the ZHVI. In other words, this variable is measuring the change in the annualized change in home values. The foreclosure re-sale percentage is expressed as the year-over-year change in this metric. The lagged dependent variable is expressed as the year-over-year change in the Case-Shiller HPI from the prior quarter.

Black background chart photo courtesy of Ponzi_unit on Flickr .