As you hopefully know, the results given for screen backtests represent averages from a series of independent cycles through the data.  A model with a 5-day hold will have 5 independent series of trades and, likewise, a model with a 21 day hold will have 21 independent series that are averaged together to calculate a backtest’s final stats.  The advantage of this methodology is that it utilizes all the available data rather than a small fraction which many backtesting methodologies base their results on.

This article will take a more in-depth look at the numbers behind the averages.  For this I chose a published screen by ‘inconversable’ titled CDGR – Amer. C. F. Mk 5.  This screen holds 3 positions and rebalances every 5 days.  As you can see from this image, the CAGR results are shown as 27.9% versus -1.1% for the SPY.

So what is behind this number?  Actually, there are 5 independent backtests behind these results.  Those 5 backtests start on 5 consecutive days beginning on 5/21/2007, and all end on 5/18/2012.  Did they all produce the exact same statistics?  Of course not.  There is a large amount of random noise in security prices so it is not unusual for these independent cycles to result in wildly different results.  The table below illustrates the 5 different cycles for this model.

The top couple of lines of this table show the first and last rebalance dates for each cycle.  You can see these are on 5 consecutive dates but might note that cycle 1 appears out of order.  This means nothing for these results; but, these tables will become more commonplace on the site, and the cycles are defined so cycle 1 on a 5 day model is always the same cycle.  That way, if you are following a model you can pick a cycle number and have real results to compare against.

Let’s refocus on the numbers, particularly on the CAGR line.  You can see that these cycles ranged from a 16% CAGR all the way up to 33%.  If you had studied backtest results based on a single cycle, you could have seen any of these numbers and possibly made an incorrect decision about the screen. Backtests are rarely a perfect indicator of future results, but backtests that utilize as much data as possible statistically improve the odds.

Similar variances exist in the other statistics as well.  In fact, if you look at the SPY results you will see a small variation there caused by the different starting dates.  Depending on market dynamics, there have been a few times that SPY returns were significantly different for models of different holding periods measured over the same time period.  Although rare, it has caused a few of you to rightfully question the numbers.

Going a step further, we can look at any cycle for actual trade-by-trade results grouped by date.  The table to the left shows the last few trades for cycle 1 of the screen above.

We are in the final stages of testing  before releasing these pages for your usage.  We have several other enhancements on deck as well.  To support these features it took a good bit of investment in our backtesting server/software, and there is a lot more data to pass around.  We are confident you will find this increased documentation valuable and supportive of your decision making.