Moose near St. Anthony, Newfoundland and Labrador

Do you have a financial ratio that you want to create and test? Here are the next steps in my series on quant screening. The first post can be found here .

Earlier, I wrote about the first steps in quant screening and how one could find ideas or ratios for that strategy. The next step involves creating and testing those ideas. If you can’t test a ratio, then use (at your discretion) known or ‘unknown’ ratios/strategies and results, but keep in mind that you need to like the ratio that you’re using. If a strategy has worked in the past for someone else, or the back-test shows that it has worked in the past, don’t assume that it will work in the future, and don’t blame someone else if it doesn’t work for you. It is incredibly important that you develop your own “style” of quant investing. Over the course of these blogs, I’m going to share some of my “style”, but it doesn’t mean that it will work, or that it’ll be good for your style of investing. I want to give you an idea of how quant screening works, and a few tips for professionals and non-professionals alike. Hopefully (if you so choose to do it), these blogs can help you get started on your own quant screening strategy.

Create Custom Ratios

Programs that deal with quant strategies have basic ratios such as PE and price/book. For complex formulas, the professionals may use applications such as the ‘Formula Library Editor’ in FactSet. For those of us at home, we can try using AAII ‘s Stock Investor Pro’s, “Custom Field Editor“, to build the custom formulas. Note, the best programs will allow the creation and use of custom formulas, and the creation of nested formulas within one another. Some ratios aren’t readily available, and so you’ll need to create the formulas yourself. How can you do this? Here’s how:

1. Create each component of the formula.
2. Combine the formulas together.
3. If the data doesn’t exist, decide on how to interpret it.

Take a formula and break those into smaller components. Create formulas for each small component. In the Dechow f-score ‘fudging’ paper. an example is given for Enron and how the ‘f-score’ was calculated for it. The formula starts like this:

Predicted Value = -7.893 + 0.790 x (rsst_acc ) + 2.518 x (ch_rec) + ….

For each component of that formula, the component formulas are listed in Table 3 of the paper.

For example, in Table 3, the rsst_acc formula, starts like this:

(change in current assets – change in cash and short term investments – change in current liabilities – change in debt in current liabilities) …

I teach students about FactSet, and so in FactSet, one interpretation of the formula could look like this:

(FA_CUR_ASSETS-FA_CUR_ASSETS(-1)-FA_ST_INVEST+FA_ST_INVEST(-1)-FA_CUR_LIABS+FA_CUR_LIABS(-1)-FA_ST_DEBT+FA_ST_DEBT(-1)) …

At home, I use my own laptop and AAII’s Stock Investor Pro to pick my Marketocracy stocks. One interpretation of that formula could look like this:

([Current assets Y1]-[Current assets Y2]-[Cash Y1]+[Cash Y2]-[Current liabilities Y1]+[Current liabilities Y2]-[Short-term debt Y1]+[Short-term debt Y2]) …

Create each component of the formula and then combine the formulas together to create the master formula.

For example, after creating a formula called ‘PVALUE_FSCORE’ with FactSet, the final Dechow f-score could look like this:

EXP(PVALUE_FSCORE)/(1+EXP(PVALUE_FSCORE))/0.0037

And in AAII’s Stock Investor Pro, my final formula starts to look like this:

EXP(-7.893+0.79*[RSST Accrual]+2.518*[Change in Receivables]+1.191*[Change Inventory]+ …

Test Ratios

The professionals use statistical packages and programs such as FactSet, to back-test ratios. Many don’t get a chance to do this, but I’d like to describe the general approach. For those who don’t have access to such programs, then try to create strategies that are based on some historical results. In my last blog. I had highlighted some of those methods. In the case of FactSet, they have an application called “Alpha-Testing ” and it lets one test a ratio. Here are the basic steps for back-testing:

1. Define the ‘universe’ of stocks that you’d like to test. Be cautious that you don’t limit yourself to a small sample size. For example, if you test the PE ratio on companies in the ‘Marine’ sub-industry, you’ll have an incredibly small sample size and the results may not be statistically significant.
2. Create limitations on those factors that you are 100% sure that you do not want. For example, one may desire that the stock price be above $5, or that the market cap be above $50 million. Do not place limitations on things such as PE or other ratios! When one runs a back-test, it’s important to see how the low PE stocks compare to the high PE stocks. If you were to place a limit such as PE<5, then your results might not be significant and the results might not be meaningful.
3. Choose a time horizon for the test. Be mindful that a 10 year time horizon avoids the internet bubble. One may find that low PEs have worked well in the past 10 years, but expand the time horizon to 15 years (to include the internet bubble) and the results may not look as good.
4. Choose a time horizon for the creation of each portfolio and the results from that portfolio. Personally, I like students to create portfolios each month and look at returns for each year. I am aware that as people become less patient, and high-frequency trading becomes more popular, then I’ll presume that firms are calculating quant strategies down to partial milliseconds. This is not my preferred approach, as my style/discipline/strategy is not about day-trading; however, I am aware that firms hire those with Phd, math, and financial engineering degrees to calculate trades on a constant basis.
5. Run the test and hope for statistically significant results with spectacular returns. If the results aren’t what you expected, then ask yourself, “do I really want to use this ratio’?
6. Try not to ‘data-mine’. Have an idea for why you would like a ratio. That being said, it’s hard to imagine a world where people haven’t data-mined each and every ratio.

Some Quant Concerns

Survivorship Bias. Had I not chased this off the Labrador highway, this critter would have died.

Although back-testing can be rather ‘fun’ to quants, here are some concerns:

• Be aware of ‘survivorship biases ’. Although FactSet accounts for survivorship bias, some back-testing programs will test stocks that exist, but they neglect to test the stocks that stopped existing. The ‘survivorship bias’, is perhaps most reflected in the phrase: “history is written by the victors”.
• Data errors might exist. Software programs constantly work on improving this data, but sometimes academics will find errors in the data. Look at the paper, “Rewriting History ” by Alexander Ljungqvist (National Bureau of Economic Research. New York University. Centre for Economic Policy Research. European Corporate Governance Institute ), Christopher J. Malloy (Harvard Business School. National Bureau of Economic Research ) and Felicia C. Marston (University of Virginia. McIntire School of Commerce ). They found:

We document widespread ex post changes to the historical contents of the I/B/E/S analyst stock recommendations database. Across a sequence of seven downloads of the entire I/B/E/S recommendations database, obtained between 2000 and 2007, we find that between 6,594 (1.6%) and 97,579 (21.7%) of matched observations are different from one download to the next. The changes, which include alterations of recommendation levels, additions and deletions of records, and removal of analyst names, are non-random in nature: They cluster by analyst reputation, brokerage firm size and status, and recommendation boldness. The changes have a large and significant impact on the classification of trading signals and back-tests of three stylized facts: The profitability of trading signals, the profitability of changes in consensus recommendations, and persistence in individual analyst stock-picking ability.

• Results from a back-test may indicate a 30% return, but most of that return may not come from the ‘longs’, but from the ‘shorts’. It’s quite common to find back-test results whereby returns are delivered not by the stocks that you could buy, but by the ones that you could avoid and/or short. For this reason, ask the quant academic papers and the quant portfolio managers… ‘how much of the result comes from the shorts’? Also, since shorts can create most of the historical-tested-statistical returns, it can provide some evidence that shorting (and some hedgefunds) may not be as ‘evil’, or as ‘risky’ as some may believe.
• Some back-testing results may not take not take into account: risk, transaction costs, information costs, tax costs, bid-ask spread, shorting restrictions, etc. This is yet another reason as to why it is important to look at working papers. Sometimes the working papers still need to do ‘robustness’ checks with regard to these factors.
• In academic tests, due to differences in accounting/regulations/geography, academics will sometimes avoid financial, utility and non-USA stocks in their tests. Do you exclude them as well? It’s up to you.
• When testing strategies, check out the results from 2007 and 2008. In August of 2007, quant trading strategies experienced a ‘meltdown’, whereby many quant strategies failed to work. Although one should look at a long time horizon for quant strategies; the events surrounding that time can provide some interesting results. One day, I was sitting with a group of people around a lunch table, and a person from a really big investment firm joined us. After a bit of food, they told us that they had 12 quant trading strategies, but during the quant meltdown, 9 of the strategies failed. What they found interesting though, was not the ones that failed, but the ones that worked during that time. The strategies that worked during that time provided an indication to them, that they were aware of ratios that others weren’t aware of. So, when testing, check out 2007 and 2008 – you might find something interesting.
• With regard to the quant meltdown, be aware that ratios/strategies can go in and out of favor. You must decide if a strategy should still be followed, or not. For example, if the strategy of buying low PE stops working for a few years, do you remove PE from your quant screen? That might not be an easy decision to make. What if you remove PE, but it works again the next year?

Create ratios, test the ratios (if possible), and be aware of some quant concerns. The next quant blog will deal with ranking and sorting of ratios. Best of luck!

Moose near St. Anthony, Newfoundland and Labrador

Kai Petainen’s views on the market and stocks are his alone, and do not reflect the views of the Ross School of Business or the University of Michigan. Being a quant, Kai lives in Ann Arbor, Michigan, one of America’s 20 Geekiest Cities. Kai uses Stock Investor Pro by AAII at home, but he teaches (and is an avid fan of) FactSet at the Ross School of Business. Kai teaches a class on quant screening, F334 — Applied Quant/Value Portfolio Management, at the Ross School of Business. Kai is a MFolio master at Marketocracy. and is featured in Matthew Schifrin’s book, The Warren Buffetts Next Door. As for the moose photos – why not?