Now that the new screener is up and running, and people are exploring the performance of their favorite screens, I want to pass along a hint or two about developing screens that test well.  There will likely be a follow up post, or two, to this one so if anyone wants to comment or otherwise share some of their expertise please do so.
First and foremost, focus on the group of funds you start with. Most of us want to begin with all funds and let our filter rules and our sorting rules determine the best from the rest. That’s asking a lot from our simple rules, especially if we include a number of overlapping, highly correlated, funds.  So the first suggestion is to narrow your list down to the basics and let each fund represent it’s sector of the market solely. Â
If you look at the groups of funds in the Sector Strategy and the International Single Country Strategy, you will see funds with generally lower correlations.  Yes, the Russell funds have high correlations and a few of the European countries are a bit high, but most are in the .70 to .90 range (Corr. for Sector Strategy / Corr. for Int’l Strategy).
Let’s look at some simple examples by beginning with a basic model. We’ll filter for [Price] > [SMA-200] and order by [RSf]. We’ll hold 5 positions and rebalance every 21 market days,
All Traditional Funds
or about monthly.  First lets look at all traditional funds (non-inverse and non-leveraged).   You can see from the graph on your left (or here) that the performance has been unimpressive, having a CAGR of 2.5% and an Ulcer Index of 30.6. A taste of that 50% drawdown and you’ll have heartburn for sure.
Now lets take those same rules applied to a portfolio of funds created by combining both the Sector and International Strategies. Shown below is the result of this model.
Select Sector and Single Country Funds
The CAGR has improved to 12.1% and the Ulcer Index has dropped to 16.0%. The drawdown is still a little steep at 30% or so, but much better than 50% and we have no bond funds in this portfolio. Remember both of these backtests use the same screening rules.
So hint one is to start with a pre-selected group of funds. If you don’t want to create your own, Ameritrade did a pretty good job with their list of commission free funds.  The same backtest applied to that list is here, it returned 16.3% with an Ulcer index of 10.7 and a 20% max. drawdown. Of course, in all of this we have to remember that backtests aren’t perfect and the future will always be different from the past.
Hint 2 is a quick one.  Do not mix leveraged funds in a backtest with non-leveraged funds because they are different animals. Leveraged funds are much faster acting and therefore timing signals must be tuned accordingly.
Hint 3 is similar. Use short funds cautiously in a backtest involving long funds. Most every trader will tell you how bear markets behave differently from bull markets, and they are right. The market generally falls much faster than it goes up, so again a system tuned to bull markets will generally not perform so well in bear markets.
I’ll leave testing these final two hints to you, they’re easy enough.  To wrap up, you will see some new links showing up on the left panel of the screener pages that will show you your recent activity.  This should make it easier to go back to a screen you ran a few minutes earlier. If you see a way to improve it let us know.
If there are no major problems the next few days, this new screener will likely replace the old one sometime this weekend.
Great work.
Just a question: what are the screen definitions that one would need to replicate your sector strategy and international single country strategy?
I imported the symbols and tried to do it, but to no avail. Cheers.
One of the key points of these strategies cannot be tested using this system, and that is the Hold-Til-Drop rule. Each of those strategies buys the top 2 or 3 positions and holds until the fund drops out of the top 6 or 7 slots. This gives room for the fund to have normal up and down relative movement without kicking it out of the portfolio. This generally helps returns, but greatly reduces turnover. It was not feasible to add this into the current system and meet other design parameters (like keeping it free – advertising supported).
That said, take a look at this screen, http://www.etfscreen.com/screener.php?sbt=658, which is similar to the Single Country strategy. You won’t be able to edit this screen since it uses a portfolio that belongs to me, but you can view it.
Hugh,
Was looking at the recent/sample screens and this one jumped out!
http://www.etfscreen.com/screener.php?sbt=3d58
What’s going on under the hood here, are the results accurate?
April Fools?
Nice little falling knives screen. I’ve tried trading those before, with stocks, – gave me real heartburn. You know all those filters in that screen do nothing, and all that is really relevant is the sort rule, the elimination of leveraged&short funds, and the short holding period. However, hold 5 positions to diversify, and add a few targeted filters, and you just might have something. It is April Fools day!
when performing back testing, does the testing take into account reverse splits? Thank you.
Yes, prices take into account splits and distributions.
I ran a screen on the commission-free TDA funds using the 100 day sma and RSF. Its returns were better than for the 200 day. My question is how does the screen update the monthly picks? Do they need to fall below a certain percentile or number of etfs as in the sector and international strategies? Or are the picks for say the top 5 rotated if they fall out of the top 5? Thanks. I’m thinking of using the screen in my TDA account with the top 3 or 4 etfs.
Becca,
The screener has no “hold til drop” functionality like the two model strategies. Therefore, if you’ve requested the top 5 it rotates to whatever the top 5 are. Good question. – hmt
The user defined variable validator seems to have a bug in it. For example, it allows me to enter a variable as [Rtn-1mo]+[Rtn-3mo] but not as [Rtn-1mo]+[Rtn-2mo] or [Rtn-1mo]+[Rtn-2mo]+[Rtn-3mo].
The error is “Unknown column Rtn in field list”
Screener fields are limited to the ones in the drop-down menus. [Rtn-2mo] is not a valid field name. For a little explanation, field values have been previously calculated and stored for quick access. This means that custom durations (like 2mo) are not an option in this system.
If that’s the case, what are the user-defined variables for?
The screener says one can use the user defined variables in filters. The user-defined variable is where I defined the [Rtn-1mo]+[Rtn-3mo] field, for example.
Sorry if I wasn’t clear in my first answer. You can combine fields mathematically as User Defined Variables or in Filter Fields, but you cannot create new fields. As an example, you can create a User Defined Variable by combining [Rtn-1mo] and [Rtn-3mo] and, as a UV, you can name it. But you cannot create a field that does not already exist, like [Rtn-2mo]. If you look at the Return related fields in the drop downs you will see [Rn-1mo] and [Rtn-3mo] listed, but not [Rtn-2mo]. If it is not in the drop downs, it’s not a valid field and cannot be used. -hmt
Could you please explain if my assumptions on the back test are correct:
1) If holding 5 positions / rebalancing monthly is selected then the back test assumes the top 5 positions in that period.
2) When selecting a monthly rebalance the backtest runs backwards 1 month from the date of the backtest- For example a backtest run today, May 5 will run back to April 5 for it’s first rebalance and so on.
Also, question- Is there a way to filter for a condition such as a rising MACD?
Thanks
Ques 1 – Yes, the back test assumes holding the top 5 positions. If filters are such that less than 5 symbols pass, the remaining positions are in ‘cash’ with no interest.
Ques 2 – Actually no, these tests are run using a rolling start so more data is utilized and statistical noise is reduced. The holding period is actually 21 market days when one month is chosen.
Hi, folks. This has to be one of the best financial sites on the net. Great contribution to all of us “retail” investors and thanks!
Here’s my suggestion: How about adding a screen of screens? In other words, allow us to screen all past screens and sort based on CAGR, ulcer index, and/or drawdowns?
Anyway, thanks again for the site!
Hi
Is there a way to change the default seeing? Suppose I want to sort 8 week total return descending. How can I do that.
I am new to this site.I found it is very good site.
Thanls
The default screener setting is for demonstration purposes only and is not settable by the user. If you have a screen you want to view repeatedly I would suggest setting is as a ‘Favorite’ and then you can access it from any Favorite Screens list.
I really like the back testing, but, on my first attempt (well, many tries!) I was unable to get a back test to run……. I tried to do a simple performance return like UV1=[Rtn-3mo]+[Rtn-6mo], …… so this would look at the top perfomers combining 3 month and 6 month returns, I hope … not sure what I did wrong, everytime, I got “Check Screen Definitions for errors”.
Any help is appreciated
Thx – Joel
Joel,
Sorry to hear you are having difficulties. I think we need to work on our error messages to be more helpful. I assume you have looked at some of the published screens and other screens users have run to compare with what you are doing? If not, that is a good place to start.
I just ran a screen to make sure things were working, so let’s go through that one. A link to it is: http://www.etfscreen.com/screener.php?sbt=35285. I left the top two sections as default values, but added a filter that the name include “vanguard”, so my screen starts with the Vanguard family of funds.
I then removed the default filter by deleting “[Price]”. You can delete the whole line but that is not necessary.
Let’s now skip down to the User Defined Variables section. Here I clicked on the text box and selected from the drop down Rtn-3mo. I entered a + sign using the keyboard and then again clicked the box with the mouse to bring up the menu and selected Rtn-6mo. I could have just as easily typed those in, but avoid the possibility of mistakes by selecting from the menu. I left the UV name as UV1.
Now lets move back up to the Sort By field, click it to bring up the menu, and then select _UV1 from the User Defined Variables section. Note all User Defined Variables begin with the underscore(_).
I then skip down to the backtest section select 1 for the number of positions and click Run Backtest.
If this doesn’t clear things up please get back to me. One suggestion I might have is to begin with a screen that you know works and then edit it to make one change at a time and then try it. Changing an existing screen is often easier than creating one from scratch.
Good luck – Hugh
Hugh,
Thank you for your help. I was selecting “UV1” in the Field column, “=” int the Logic column, and [Rtn-1mo]+[Rtn-3mo] in the Value column, and they should have been down in the User Defined Variables.
Now as I have read through the blog, and hints for a successful backtest, I have a question….. How do you select two, or three, groupings of funds to select and filter simultaneously?
Pls allow me to play devils advocate on this point…… you recommend doing the above so as to avoid overlapping, highly correlated funds, and my initial playing with the backtesting seems to support this. However, I would say/guess that it is better to allow all funds to be filtered and then to carefully select from these (manually select your funds) so as to insure that none of your selections are too similar/correlate too closely…… then you have the strongest in the universe without the overlapping issue……. (Of course this is not feasable with the backtesting feature, but is possible in someone’s real account.) Your thoughts?
Thanks again – Joel