Backtesting

[Guest listolyman]

Hi Kim,

This is a great educational site but lots of complicated concepts for a rookie. I am still working on grasping the terminology and methodology. Can you elaborate on how you backtest? I opened a TOS account but unsure how to use it. Is there a good book or article on backtesting. Also, is the Tharp positional sizing game worth the price? I played the first 3 levels and did well( 3x the the goal level and the game stopped early) but not sure how much i learned.

I am drawn to the non-directional trading paradigm but realize it will take some time. I will begin a paper account to see if i can formulate a strategy for learning from my mistakes. The Tharp book Super Trader is very good.

Thanks,

Mitch

[Kim 7,512]

Hi Mitch,

Just take it slowly, I'm sure you will master those strategies if you have the desire and ready to spend some time on learning.

In TOS, just go to Analize/thinkBack, enter the stock symbol and the strategy, then select the date. Check the price at that date, then compare it to another date(s). That's it.

[RobertB 5]

Has anyone else noticed how broken TOS's backtesting feature is? See the attached screenshot jpg file.

This ThinkBack screenshot is for symbol lulu on 4/13/2012 for some of its calls. Notice how the the greeks and the implied volatility data are missing. I've noticed this with many, many different symbols while using TOS backtesting over the past few weeks and months--for options data going back over the past few years. I've written to TOS support, and they say they are fixing it. They are aware that there are tons of missing data in their database. Additionally, about a month ago I Googled this problem with TOS's ThinkBack feature and found a non-TOS trading forum containing a discussion from over a year ago concerning the exact same problem of missing data. Reading that old discussion from over a year ago doesn't give me any real confidence that TOS will actually fix this anytime soon.

Worse, in my opinion, is that TOS ThinkBack software does not warn the user when an entire day of data is missing (a separate,but related, problem). When there is missing entire day data, the TOS ThinkBack software grabs the most recent day it can find even if it's a few days old, displaying it as a proxy for the later missing day as if it were valid! For example, suppose you want to know the greeks for Jan 5th for a particular option. If that day's data is entirely missing, the TOS software will grab the Jan 4th data and use it for the missing Jan 5th data. This, of course, provides worse than meaningless results. I have mentioned this problem to TOS support numerous times in the email threads I've had with them on this whole missing data subject. I've told them how really bad (I think I used the word unconscionable) it is that that they do not even warn the user when TOS ThinkBack is displaying meaningless, literally fabricated data. They have never responded to those portions of my emails.

I'm a software engineer who has had years of experience with databases and handling huge amounts of data. Saving end-of-day options data into a database--although grantedly huge in amounts of data--as a database problem itself is rather trivial. I don't know why TOS has dropped the ball on this. The TOS software itself for the ThinkBack feature is solid enough (the lack of proper warnings to the user notwithstanding). It looks as if pretty good software engineering occurred during its original development. But that's of no comfort if its database is swiss cheese and provides meaningless results without warning.

I'm surprised no one else here has mentioned this. Most of the options in ThinkBack do have correct price data. But for many options, as I've already mentioned, any greek and implied volatility analysis one wishes to performed is stopped cold. And even if it's not stopped cold, the fact that the displayed data could have been derived from previous days because of missing days makes much of ThinkBack's results suspect.

Bottom line: TOS really needs to fix this. It would be great if more people would beat on them about this.

Robert

[Kim 7,512]

Yes, this is annoying. In fact, I have never seen a screen like the one you attached where ALL data was missing. Usually it is just couple of fields, and in most cases it warned that there is no data for that trading day, showing data for the one of the previous days.

I hope in 2-3 cycles we will have our own database here, so we will not need TOS anymore.. Right now I already have pretty good idea which stocks work and which don't.

[mks212 0]

On the note of backtesting, I have begun using the OnDemand feature in TOS. In order to see it, you must be in live trading, not paper. It is in the upper right in an orange box.

It seems to be pretty robust. It essentially replays any given trading day for the past 3-4 years. I find it helpful for testing intraday strategies. For example, in one of Jeff Augen's books, Day Trading Options, he discusses the weekly Petroleum Status Report that comes out on Wed mornings at 10:30. I wanted to see if a straddle bought at 9:45 on Wed AM would produce a profit (it doesn't). I was able to test using this feature.

However, one huge problem, is that it is SLOW. Really slow. It took me about 90 minutes to pull 24 data points, most of that time spent waiting while the software "buffered."

Is there another way to get this historical market data? Maybe in csv/excel format?

For what it is worth, Jeff Augen seems to have based all of his books and trading strategies off historical data such as this.

Here is the data I pulled. I didn't complete the whole table since it was such a slow process. I'm sorry the data is hard to read, is there an easy way to paste tables into the forum?

9:45 10:15 10:45 11:15
Delta Neutral Strategy Price IV Price IV Price IV Price IV
9/5/2012 40 Straddle $1.85 26% $1.75 27% $1.80 27% $1.75 26%
9/12/2012 42 Straddle $1.30 22% $1.20 22% $1.20 23% $1.15 22%
9/19/2012 42 Straddle $0.90 26% $0.75 26% $0.70 25% $0.70 26%
9/26/2012 40 Straddle $2.20 25% $2.00 25% $1.95 25% $2.00 25%
10/3/2012		
10/10/2012 40/41 Strangle $0.50 27%/32% $0.35 27%/32% $0.35 25%/31% $0.35 25%/30%
10/17/2012		
10/24/2012		
10/31/2012		
11/7/2012		
11/14/2012		
11/21/2012		
11/28/2012		
12/5/2012 39 Straddle $1.55 22% $1.40 22% $1.55 24% $1.55 24%
[/CODE]

Edited December 19, 2012 by mks212

[paquibena 0]

As a newbie, I have been looking for a place in the forum where I can read the whole backtesting process as a detailed checklist. I would like to help with the backtesting of new trade candidates, but right now I don't know how to do it and couldn't find a place where I could learn it.

 

Kim, can you please let me know where can I find this information about backtesting?

[Kim 7,512]

The process is pretty much straightforward.

 

In TOS, just go to Analize/thinkBack, enter the stock symbol and the strategy, then select the date. Check the price at that date, then compare it to another date(s). That's it.

I usually start about 7-10 days before earnings and go forward. You can get the earnings dates at optionslam.com.

[paquibena 0]

The process is pretty much straightforward.

 

In TOS, just go to Analize/thinkBack, enter the stock symbol and the strategy, then select the date. Check the price at that date, then compare it to another date(s). That's it.

I usually start about 7-10 days before earnings and go forward. You can get the earnings dates at optionslam.com.

 

Thanks, Kim. Do I have to write down the different data by hand for every stock (dates, IV, VIX value and option prices) to compare them or there is any available database in Steady Options where I can check the historical data just by using a webpage, for example?

[Kim 7,512]

While IV and VIX can have some academic value, what we really care about is the price of the straddle. If it makes money consistently, that means that IV increase was usually enough to offset the theta, or the stock consistently moved causing gamma gains. In both cases, it would be a good candidate.

[paquibena 0]

I thought we would have a complete application or website to do the backtesting. Doing it manually is a bit too difficult for me now, given my current knowledge level.

 

This month has not been kind to me trading options, and I am cancelling my account to study more and settle myself down so I can come back in the future more prepared. Thanks to all, especially to Kim, for your help and support. Goodbye and good luck to all.

Edited January 25, 2013 by paquibena