From Wikipedia

Backtesting is the process of evaluating a strategy, theory, or model by applying it to historical data. It can be used in situations like studying how a trading method would have performed in past stock markets, or how a model of climate and weather patterns would have matched past measurements. A key element of backtesting that differentiates it from other forms of historical testing is that backtesting calculates how a strategy would have performed if it had actually been applied in the past. This requires the backtest to replicate the conditions of the time in question in order to get an accurate result. Backtesting is a common and methodologically accepted approach to research, however a high or successful correlation between a backtested strategy and historical results can never prove a theory correct, since past results do not necessarily indicate future results. In other words, things are always changing, but in a world where yesterday bears some resemblance to today, backtesting can be a useful tool of analysis and prediction.

Contents

Backtesting in Finance and Economics

In the application of backtesting techniques to capital markets, backtesting is a specific type of historical testing that determines the performance of the strategy if it had actually been employed during past periods and market conditions. Since backtesting uses real-world data, it has advantages over testing with synthesized data sets. While backtesting does not allow one to predict how a strategy will perform under future conditions, its primary benefit lies in understanding the vulnerabilities of a strategy as it encountered real-world conditions of the past. This enables the designer of a strategy to learn from their mistakes without actually having to make them with actual money.

A key element of backtesting that differentiates it from other forms of historical testing is that backtesting calculates how a strategy would have performed if it had actually been applied in the past. This requires the backtest to replicate the market conditions of the time in question in order to get an accurate result. Examples of these market conditions include screening/buying/selling stocks that no longer exist, or using market index compositions as they were in the past, rather than current compositions. Due to the expense of obtaining these data sets, backtesting has historically been performed by institutions and professional money managers. With the advent of electronic trading and more accessible online databases, however, basic backtrading has become an option for casual traders as well.

Various types of capital market strategies can be backtested, such as asset allocation strategies, stock screening strategies, and trading strategies. Other types of strategies are less amenable to backtesting, such as programmed trading strategies for buying or selling large quantities of a stock at the best prices by spreading the trade over a period of hours, days or weeks. This is because the act of selling large quantities of an individual issue affect the trading price for that issue, resulting in a feedback loop. Since the feedback loop is the effect being studied, backtesting is inappropriate for such strategies.

Backtesting in Climate Modeling

Backtesting plays a critical role in the evaluation of weather and climate models. For example, in composing a new theory of hurricane formation, a model could be backtested against actual conditions that preceded and accompanied real hurricanes. If the model accurately forecast the location, strength, trajectory and duration of a past event, it would gain credibility for future predictions. With the field of climate modeling, backtesting plays a particularly important role because changes to the global climate happen on a scale for which it is prohibitively slow to just ‘wait and see’. Doing so could take decades or centuries to even begin to evaluate new ideas. Backtesting, then, allows the hundreds and thousands of years of past climate data to be employed as benchmarks for new theories. Afterall, if a model could not accurately predict what you know to have already happened, it probably is not worth holding onto going forward.